From 245c17c2d0d903b88f75e3f5e71f6ed26721db25 Mon Sep 17 00:00:00 2001 From: Dandaedre Date: Fri, 6 Nov 2026 07:19:14 +0100 Subject: [PATCH] Document Flipper Link feasibility and future ideas --- Ideas.md | 21 +++++ README.md | 1 + docs/flipper-link.md | 205 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 227 insertions(+) create mode 100644 Ideas.md create mode 100644 docs/flipper-link.md diff --git a/Ideas.md b/Ideas.md new file mode 100644 index 0000000..2ea1166 --- /dev/null +++ b/Ideas.md @@ -0,0 +1,21 @@ +# Future ideas + +These are feasibility notes for possible tools, not implemented features or +committed delivery plans. + +## Flipper Link: USB control from the web and Stream Deck + +**Feasible for selected actions with official Flipper Zero firmware.** A small +service on the computer connected to the Flipper would translate HTTP commands +into USB serial CLI commands or structured protobuf RPC requests. Bitfocus +Companion could call that service from Stream Deck buttons. + +Start with device status and saved infrared remote buttons. Prove the workflow +using Companion's Generic HTTP module, then build a dedicated Companion module +with action selection, variables and feedback. Remote support varies by Flipper +application; exposing every internal action is not guaranteed. USB ownership, +reconnection and stopping held actions need particular care. + +See [Flipper Link feasibility and proposed design](docs/flipper-link.md) for +the supported interfaces, limitations, architecture, implementation stages and +primary sources. Hardware validation is still required. diff --git a/README.md b/README.md index 77d55e2..c63dc96 100644 --- a/README.md +++ b/README.md @@ -46,3 +46,4 @@ opening reports, and reading video timings. - [Rundown converter](docs/rundown-converter.md) · [Sample findings](docs/rundown-corpus.md) - [Development](docs/development.md) · [CI and coverage](docs/ci.md) - [Cross builds and portable releases](docs/releases.md) +- [Future ideas](Ideas.md) diff --git a/docs/flipper-link.md b/docs/flipper-link.md new file mode 100644 index 0000000..f50af9f --- /dev/null +++ b/docs/flipper-link.md @@ -0,0 +1,205 @@ +# Flipper Link: feasibility study + +Status: proposed tool; no implementation or hardware validation yet. +Research date: 2026-11-06. [Back to future ideas](../Ideas.md). + +## Verdict + +Yes: a computer connected to a Flipper Zero over USB can act as a bridge between +web commands and device actions. A Bitfocus Companion connection module can +then expose those actions to Stream Deck buttons. Saved infrared remote buttons +are a strong first use case because the official firmware implements an RPC +mode for loading a remote and triggering its signals. + +This is feasible with official firmware for a defined set of actions. It is not +a universal API for every internal function: each application must expose the +required CLI command or RPC behavior. Arbitrary application internals may need +an external Flipper application or firmware changes. These conclusions come +from documentation and source inspection, not tests on the connected device. +See the [official CLI documentation](https://docs.flipper.net/zero/development/cli) +and [Infrared RPC implementation](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/main/infrared/scenes/infrared_scene_rpc.c). + +## USB interfaces + +The Flipper itself provides a USB serial CLI; no Wi-Fi development board is +needed. Typical ports are `/dev/ttyACM*` on Linux, `/dev/cu.usbmodemflip_*` on +macOS and a COM port on Windows. Prefer device identity or Linux's +`/dev/serial/by-id/` over a hard-coded port number. Close qFlipper and any other +serial client before taking ownership. Run `help` to discover commands on the +actual firmware. Flipper also documents browser access through Flipper Lab and +Web Serial. [Connection instructions](https://docs.flipper.net/zero/development/cli). + +Two automation paths are available: + +| Interface | Suitable use | Tradeoff | +| --- | --- | --- | +| Text CLI | Exploration and commands without an RPC equivalent | Must handle prompts, echo, text changes and commands that keep running. | +| Protobuf RPC over USB serial | Structured device, storage, GUI and supported application operations | Requires framed binary messages, response correlation and session management. | + +`start_rpc_session` switches the CLI into protobuf mode. After that, the bridge +must exchange binary protocol messages, not append CLI commands to the stream. +The firmware uses length-delimited protobuf messages and command IDs. Prefer RPC +for the main adapter; isolate any CLI fallback behind explicit session transitions. +[CLI reference](https://docs.flipper.net/zero/development/cli), +[RPC implementation](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/services/rpc/rpc.c). + +Flipper Devices publishes Python bindings and a `flipperCmd` utility with +examples for device information, file listing and starting Infrared. This is +useful prototype material, but its README describes it as alpha: validate and +pin it before relying on it, including its protocol compatibility and license. +[Official Python bindings](https://github.com/flipperdevices/flipperzero_protobuf_py). + +## What could be exposed + +Source availability below means there is an implementation path, not that all +firmware versions, files or applications have been tested. + +| Capability | Evidence and boundary | Proposed scope | +| --- | --- | --- | +| Device identity, ping and power information | Defined in the [system RPC schema](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/system.proto). | Initial status API; report unavailable fields honestly. | +| File browsing and reading | Demonstrated by the [official Python client](https://github.com/flipperdevices/flipperzero_protobuf_py). | Browse configured remote directories; leave writes out of the first version. | +| Saved infrared buttons | [Infrared RPC scene](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/main/infrared/scenes/infrared_scene_rpc.c) handles loading, named/indexed presses and release. | First useful control feature, including bounded holds. | +| Application launch, file load and exit | Defined in the [application RPC schema](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/application.proto). | Only explicitly tested applications and arguments. Launch does not imply full remote control. | +| Screen streaming and navigation | Frames and input events are defined in the [GUI RPC schema](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/gui.proto). | Optional diagnostic view; menu macros are sensitive to current UI state. | +| Saved Sub-GHz transmission | [Sub-GHz RPC scene](https://github.com/flipperdevices/flipperzero-firmware/blob/dev/applications/main/subghz/scenes/subghz_scene_rpc.c) handles transmit/release and rejects restricted frequencies. | Later, per-file validation; preserve firmware restrictions and surface errors. | +| GPIO, NFC, RFID, iButton and other utilities | CLI commands exist, but their workflows and RPC coverage differ. | Investigate individually after the infrared prototype. | +| Arbitrary FAP/app internals | Generic launch and data-exchange messages do not make every app remotely controllable. | Requires app cooperation; do not promise universal support. | + +For the last two rows, consult the [CLI command list](https://docs.flipper.net/zero/development/cli) +and [application protocol](https://github.com/flipperdevices/flipperzero-protobuf/blob/dev/application.proto). +USB modes that replace the serial interface also require separate investigation; +do not assume a persistent control connection during such applications. + +## Proposed architecture + +```mermaid +flowchart LR + S[Stream Deck] --> C[Bitfocus Companion] + C --> M[Generic HTTP or Flipper Link module] + M -->|HTTP requests| B[Local Flipper Link service] + W[Web client or script] -->|HTTP requests| B + B --> Q[Device session and action queue] + Q <-->|USB serial: CLI or protobuf RPC| F[Flipper Zero] + B -->|Status polling or events| M +``` + +The bridge runs on the computer physically attached to the Flipper. Companion +can run there or on another reachable computer. The host must remain awake and +the service running. A Wi-Fi board is unnecessary for this arrangement. + +Use one service as the serial-port owner, with one serialized action queue per +device. Web clients and Companion share that queue. Return explicit busy, +disconnected and unsupported states rather than starting competing apps. + +A direct serial Companion module is possible, but would couple USB drivers, +protocol state and device ownership to Companion. A separate service also serves +scripts and web clients and keeps the Companion module small. Browser-only Web +Serial is useful for experimentation; a persistent service is the proposed path +for unattended Companion control. + +For a proof of concept, a small Python adapter using the existing bindings is +reasonable. Keep HTTP routing, action validation and the device transport +separate. If adopted as a maintained repository tool, evaluate a Rust workspace +crate for portable delivery, using generated protobuf types and a serial library. +That choice needs a packaging/dependency review; no new dependency is selected +by this study. The Companion module would remain TypeScript. + +## Proposed web contract + +These endpoints are a design sketch, not existing commands: + +| Endpoint | Meaning | +| --- | --- | +| `GET /api/v1/status` | Bridge version, device identity, firmware/protocol version, connection and busy state. | +| `GET /api/v1/actions` | Configured, supported action IDs and labels. | +| `POST /api/v1/actions/{id}/run` | Run a configured action, such as one saved infrared button. | +| `GET /api/v1/jobs/{id}` | Read queued/running/completed/failed/unknown execution state. | +| `POST /api/v1/jobs/{id}/stop` | Request release/cancellation where the adapter supports it. | + +For example, configure `projector-power` to reference a particular `.ir` file +and button. A POST to `/api/v1/actions/projector-power/run` returns HTTP 202 and +a job ID. Completion means the Flipper operation completed; it does not prove +that the projector received the signal or changed state. + +Start with status polling. Add an event stream later if feedback latency needs +it. Keep mutating actions on POST. Validate parameters and configured file paths; +do not expose arbitrary shell or raw CLI execution as the public API. Bind to +loopback by default, authenticate requests, and make LAN listening an explicit +configuration with suitable transport protection. A browser UI should use a +restricted origin policy. + +## Companion integration + +Build a **Companion connection module**, which is the extension point for +controlled devices and software. Companion already supplies Stream Deck support; +this design does not require a separate Elgato application plugin. +[Companion module guide](https://companion.free/for-developers/module-development/home/), +[Companion project](https://github.com/bitfocus/companion). + +1. Validate the API with Companion's existing Generic HTTP module. It supports + POST bodies and custom headers, so a button can run an action with an + authorization header. [Generic HTTP help](https://github.com/bitfocus/companion-module-generic-http/blob/master/companion/HELP.md). +2. Once the API is stable, use the official TypeScript template and + `@companion-module/base`. Select versions compatible with the installed stable + Companion release; follow the template's runtime requirements. + [TypeScript template](https://github.com/bitfocus/companion-module-template-ts), + [module framework](https://github.com/bitfocus/companion-module-base). +3. Proposed configuration: bridge URL, token, device selection and timeout. + Proposed actions: run saved action, stop current job and refresh status. + Proposed feedback: disconnected, busy, transmitting and failed. + Proposed variables: device name, firmware, current action and last error. + Include ready-made infrared button presets. +4. Test locally through Companion's developer modules folder. Public distribution + is a later step following Bitfocus's module release process; it is not required + for the initial local integration. Check for an existing suitable module before + creating a new public one. [Development workflow](https://companion.free/for-developers/module-development/home/). + +## Reliability and compatibility + +- Discover the selected device again after unplug/replug or reboot. Display + actionable port-permission and port-in-use errors; do not silently choose a + different Flipper if multiple devices are attached. +- Query firmware and protobuf versions on connection. Maintain a tested + capability table; reject unknown operations rather than assuming that a + message in the schema guarantees application support. +- Give operations deadlines, bounded queue length and explicit cancellation. + A failed stop or lost connection must report an unknown device state. +- Bound hold duration in the bridge and release on normal cancellation. Test + firmware behavior on USB loss: a host timer alone cannot guarantee stopping + an action after the cable or process disappears. +- Never automatically replay a transmission after an ambiguous timeout or + reconnect. Deduplicate repeated HTTP requests with a request ID; a power + toggle sent twice could undo the first action. +- Report acknowledged execution separately from inferred target state. Neither + an IR send acknowledgment nor a green button proves the appliance is on. + +## Validation plan and delivery stages + +1. **Device discovery:** record OS, firmware and protocol versions; close other + clients; confirm CLI help, RPC ping and device information. No firmware change + should be needed for this stage. +2. **Infrared proof:** list a known remote, load it and run a selected button + against a test appliance. Verify tap, hold/release, invalid file/button, + application busy state and repeated runs. +3. **Local HTTP bridge:** implement the minimal contract and single-device queue. + Test malformed input, authentication, parallel requests, timeouts, cancellation, + duplicate requests and unplug/replug. Add transport tests using recorded or + synthetic responses; verify framing across partial reads. +4. **Companion proof:** run the same action from Generic HTTP. Measure press-to-send + latency and verify visible errors while the device is disconnected. Set a + practical latency target from these measurements, not an assumed guarantee. +5. **Dedicated module:** implement configuration, action discovery, feedback, + variables and presets. Test module restart and lost bridge connections. +6. **Expansion and packaging:** validate each additional action family and host + OS independently. Document tested firmware versions, startup and USB permissions. + +The go/no-go gate is reliable infrared triggering and recovery from disconnects +on the actual device. USB/RPC plumbing is supported upstream; application +coverage and failure recovery are the main development uncertainties. A broad +promise to expose all Flipper functionality should wait for individual adapters +and device tests. + +Upstream `dev`/`main` links describe moving source branches. Before implementation, +pin the firmware, protobuf definitions and client library revisions used for +testing. This study did not open the USB device, transmit signals, install +software or measure latency.