beffroi-tools/docs/flipper-link.md

206 lines
13 KiB
Markdown

# Flipper Link: feasibility study
Status: proposed tool; no implementation or hardware validation yet.
Research date: 2026-11-06. [Back to future ideas](../Ideas.md).
## Verdict
Yes: a computer connected to a Flipper Zero over USB can act as a bridge between
web commands and device actions. A Bitfocus Companion connection module can
then expose those actions to Stream Deck buttons. Saved infrared remote buttons
are a strong first use case because the official firmware implements an RPC
mode for loading a remote and triggering its signals.
This is feasible with official firmware for a defined set of actions. It is not
a universal API for every internal function: each application must expose the
required CLI command or RPC behavior. Arbitrary application internals may need
an external Flipper application or firmware changes. These conclusions come
from documentation and source inspection, not tests on the connected device.
See the [official CLI documentation](https://docs.flipper.net/zero/development/cli)
and [Infrared RPC implementation](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/main/infrared/scenes/infrared_scene_rpc.c).
## USB interfaces
The Flipper itself provides a USB serial CLI; no Wi-Fi development board is
needed. Typical ports are `/dev/ttyACM*` on Linux, `/dev/cu.usbmodemflip_*` on
macOS and a COM port on Windows. Prefer device identity or Linux's
`/dev/serial/by-id/` over a hard-coded port number. Close qFlipper and any other
serial client before taking ownership. Run `help` to discover commands on the
actual firmware. Flipper also documents browser access through Flipper Lab and
Web Serial. [Connection instructions](https://docs.flipper.net/zero/development/cli).
Two automation paths are available:
| Interface | Suitable use | Tradeoff |
| --- | --- | --- |
| Text CLI | Exploration and commands without an RPC equivalent | Must handle prompts, echo, text changes and commands that keep running. |
| Protobuf RPC over USB serial | Structured device, storage, GUI and supported application operations | Requires framed binary messages, response correlation and session management. |
`start_rpc_session` switches the CLI into protobuf mode. After that, the bridge
must exchange binary protocol messages, not append CLI commands to the stream.
The firmware uses length-delimited protobuf messages and command IDs. Prefer RPC
for the main adapter; isolate any CLI fallback behind explicit session transitions.
[CLI reference](https://docs.flipper.net/zero/development/cli),
[RPC implementation](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/services/rpc/rpc.c).
Flipper Devices publishes Python bindings and a `flipperCmd` utility with
examples for device information, file listing and starting Infrared. This is
useful prototype material, but its README describes it as alpha: validate and
pin it before relying on it, including its protocol compatibility and license.
[Official Python bindings](https://github.com/flipperdevices/flipperzero_protobuf_py).
## What could be exposed
Source availability below means there is an implementation path, not that all
firmware versions, files or applications have been tested.
| Capability | Evidence and boundary | Proposed scope |
| --- | --- | --- |
| Device identity, ping and power information | Defined in the [system RPC schema](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/system.proto). | Initial status API; report unavailable fields honestly. |
| File browsing and reading | Demonstrated by the [official Python client](https://github.com/flipperdevices/flipperzero_protobuf_py). | Browse configured remote directories; leave writes out of the first version. |
| Saved infrared buttons | [Infrared RPC scene](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/main/infrared/scenes/infrared_scene_rpc.c) handles loading, named/indexed presses and release. | First useful control feature, including bounded holds. |
| Application launch, file load and exit | Defined in the [application RPC schema](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/application.proto). | Only explicitly tested applications and arguments. Launch does not imply full remote control. |
| Screen streaming and navigation | Frames and input events are defined in the [GUI RPC schema](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/gui.proto). | Optional diagnostic view; menu macros are sensitive to current UI state. |
| Saved Sub-GHz transmission | [Sub-GHz RPC scene](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/main/subghz/scenes/subghz_scene_rpc.c) handles transmit/release and rejects restricted frequencies. | Later, per-file validation; preserve firmware restrictions and surface errors. |
| GPIO, NFC, RFID, iButton and other utilities | CLI commands exist, but their workflows and RPC coverage differ. | Investigate individually after the infrared prototype. |
| Arbitrary FAP/app internals | Generic launch and data-exchange messages do not make every app remotely controllable. | Requires app cooperation; do not promise universal support. |
For the last two rows, consult the [CLI command list](https://docs.flipper.net/zero/development/cli)
and [application protocol](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/application.proto).
USB modes that replace the serial interface also require separate investigation;
do not assume a persistent control connection during such applications.
## Proposed architecture
```mermaid
flowchart LR
S[Stream Deck] --> C[Bitfocus Companion]
C --> M[Generic HTTP or Flipper Link module]
M -->|HTTP requests| B[Local Flipper Link service]
W[Web client or script] -->|HTTP requests| B
B --> Q[Device session and action queue]
Q <-->|USB serial: CLI or protobuf RPC| F[Flipper Zero]
B -->|Status polling or events| M
```
The bridge runs on the computer physically attached to the Flipper. Companion
can run there or on another reachable computer. The host must remain awake and
the service running. A Wi-Fi board is unnecessary for this arrangement.
Use one service as the serial-port owner, with one serialized action queue per
device. Web clients and Companion share that queue. Return explicit busy,
disconnected and unsupported states rather than starting competing apps.
A direct serial Companion module is possible, but would couple USB drivers,
protocol state and device ownership to Companion. A separate service also serves
scripts and web clients and keeps the Companion module small. Browser-only Web
Serial is useful for experimentation; a persistent service is the proposed path
for unattended Companion control.
For a proof of concept, a small Python adapter using the existing bindings is
reasonable. Keep HTTP routing, action validation and the device transport
separate. If adopted as a maintained repository tool, evaluate a Rust workspace
crate for portable delivery, using generated protobuf types and a serial library.
That choice needs a packaging/dependency review; no new dependency is selected
by this study. The Companion module would remain TypeScript.
## Proposed web contract
These endpoints are a design sketch, not existing commands:
| Endpoint | Meaning |
| --- | --- |
| `GET /api/v1/status` | Bridge version, device identity, firmware/protocol version, connection and busy state. |
| `GET /api/v1/actions` | Configured, supported action IDs and labels. |
| `POST /api/v1/actions/{id}/run` | Run a configured action, such as one saved infrared button. |
| `GET /api/v1/jobs/{id}` | Read queued/running/completed/failed/unknown execution state. |
| `POST /api/v1/jobs/{id}/stop` | Request release/cancellation where the adapter supports it. |
For example, configure `projector-power` to reference a particular `.ir` file
and button. A POST to `/api/v1/actions/projector-power/run` returns HTTP 202 and
a job ID. Completion means the Flipper operation completed; it does not prove
that the projector received the signal or changed state.
Start with status polling. Add an event stream later if feedback latency needs
it. Keep mutating actions on POST. Validate parameters and configured file paths;
do not expose arbitrary shell or raw CLI execution as the public API. Bind to
loopback by default, authenticate requests, and make LAN listening an explicit
configuration with suitable transport protection. A browser UI should use a
restricted origin policy.
## Companion integration
Build a **Companion connection module**, which is the extension point for
controlled devices and software. Companion already supplies Stream Deck support;
this design does not require a separate Elgato application plugin.
[Companion module guide](https://companion.free/for-developers/module-development/home/),
[Companion project](https://github.com/bitfocus/companion).
1. Validate the API with Companion's existing Generic HTTP module. It supports
POST bodies and custom headers, so a button can run an action with an
authorization header. [Generic HTTP help](https://github.com/bitfocus/companion-module-generic-http/blob/master/companion/HELP.md).
2. Once the API is stable, use the official TypeScript template and
`@companion-module/base`. Select versions compatible with the installed stable
Companion release; follow the template's runtime requirements.
[TypeScript template](https://github.com/bitfocus/companion-module-template-ts),
[module framework](https://github.com/bitfocus/companion-module-base).
3. Proposed configuration: bridge URL, token, device selection and timeout.
Proposed actions: run saved action, stop current job and refresh status.
Proposed feedback: disconnected, busy, transmitting and failed.
Proposed variables: device name, firmware, current action and last error.
Include ready-made infrared button presets.
4. Test locally through Companion's developer modules folder. Public distribution
is a later step following Bitfocus's module release process; it is not required
for the initial local integration. Check for an existing suitable module before
creating a new public one. [Development workflow](https://companion.free/for-developers/module-development/home/).
## Reliability and compatibility
- Discover the selected device again after unplug/replug or reboot. Display
actionable port-permission and port-in-use errors; do not silently choose a
different Flipper if multiple devices are attached.
- Query firmware and protobuf versions on connection. Maintain a tested
capability table; reject unknown operations rather than assuming that a
message in the schema guarantees application support.
- Give operations deadlines, bounded queue length and explicit cancellation.
A failed stop or lost connection must report an unknown device state.
- Bound hold duration in the bridge and release on normal cancellation. Test
firmware behavior on USB loss: a host timer alone cannot guarantee stopping
an action after the cable or process disappears.
- Never automatically replay a transmission after an ambiguous timeout or
reconnect. Deduplicate repeated HTTP requests with a request ID; a power
toggle sent twice could undo the first action.
- Report acknowledged execution separately from inferred target state. Neither
an IR send acknowledgment nor a green button proves the appliance is on.
## Validation plan and delivery stages
1. **Device discovery:** record OS, firmware and protocol versions; close other
clients; confirm CLI help, RPC ping and device information. No firmware change
should be needed for this stage.
2. **Infrared proof:** list a known remote, load it and run a selected button
against a test appliance. Verify tap, hold/release, invalid file/button,
application busy state and repeated runs.
3. **Local HTTP bridge:** implement the minimal contract and single-device queue.
Test malformed input, authentication, parallel requests, timeouts, cancellation,
duplicate requests and unplug/replug. Add transport tests using recorded or
synthetic responses; verify framing across partial reads.
4. **Companion proof:** run the same action from Generic HTTP. Measure press-to-send
latency and verify visible errors while the device is disconnected. Set a
practical latency target from these measurements, not an assumed guarantee.
5. **Dedicated module:** implement configuration, action discovery, feedback,
variables and presets. Test module restart and lost bridge connections.
6. **Expansion and packaging:** validate each additional action family and host
OS independently. Document tested firmware versions, startup and USB permissions.
The go/no-go gate is reliable infrared triggering and recovery from disconnects
on the actual device. USB/RPC plumbing is supported upstream; application
coverage and failure recovery are the main development uncertainties. A broad
promise to expose all Flipper functionality should wait for individual adapters
and device tests.
Upstream `dev`/`main` links describe moving source branches. Before implementation,
pin the firmware, protobuf definitions and client library revisions used for
testing. This study did not open the USB device, transmit signals, install
software or measure latency.