Phantom Vision VMS / Ghost Protocol

Appearance

About this article ​

  • Audience: Integrators and operators new to the Ghost Protocol firmware repository
  • Goal: Understand what this monorepo contains and how edge firmware, proxy services, and documentation fit together
  • Type: Concept

Summary ​

The gp-firmware repository ships sovereign oakapp firmware for Ghost Protocol Shadow edge cameras, a central Phantom Vision proxy for fleet visibility, and a VitePress documentation hub. Edge devices run detection and recording locally; the proxy owns cloud escalation and operator workflows.

Prerequisites ​

  • Shadow 1 or compatible shadow hardware
  • Network access to a Phantom proxy at https://<phantom-proxy>:8788
  • Quickstart for first deployment

Repository layout ​

AreaPurpose
oak-vms-firmware/Production edge stack: encoder, RF-DETR precomp, justified recorder, MQTT client, plugins
cost-sentinel/vms-proxy/Central fleet proxy: MQTT ingest, playback, MCP tools, cloud video escalation
oak-vms-firmware/docs/Public documentation hub (getting started, architecture, reference, troubleshooting)
oak-examples/Edge AI example catalog reused for pipeline patterns and model authoring
plugins/Packaged runtime extensions (M8 controller, integrations)

Edge firmware capabilities ​

Each shadow runs a coordinated stack inside a Ghost Protocol oakapp container:

  • Primary H.265 stream — 4K evidence encoding with rolling justified 60-second chunks and sidecar metadata
  • RF-DETR detection — On-device person and object detection at roughly 10 Hz, published over MQTT
  • Hermes peer mesh — UDP discovery and HMAC-authenticated peer status between site roles
  • Plugin runtime — Dynamic feature deployment without full firmware rebuilds
  • ONVIF MVP — Standards-based camera integration path for VMS clients

No cloud API keys ship on edge images. The proxy centralizes authentication and escalation.

Central proxy role ​

The Phantom proxy at port 8788 is the fleet back end:

  • Subscribes to devices/{device_id}/… MQTT wildcards and caches latest telemetry
  • Serves playback-ready clips and recording manifests
  • Runs person-alert enrichment, vision verification, and optional Gemini video understanding
  • Exposes MCP tools for health probes, identity checks, and orchestrator automation

Site roles use labels such as site-role-a and site-role-b rather than physical location names.

Lab broker connectivity (desk / COTA host Wi‑Fi) ​

Not the same as M8 dual-uplink. Production Shadows prefer LAN PoE+; M8 Wi‑Fi is the device-side secondary path — see Shadow connectivity. This section is about the host MQTT broker reachable on a lab WLAN for desk testing.

COTA lab deployments often set WLAN-only broker discovery (fleet_wlan_only: true in cost-sentinel/vms-proxy/fleet_registry.json). Shadows reach the VMS MQTT broker without public DNS, Cloudflare tunnels, or Tailscale:

  1. Hostname — cota-host.local (canonical broker identity)
  2. mDNS — _cota-vms-proxy._tcp advertised by the VMS host on stack up
  3. Cached IP — vms_lan_ip in fleet_registry.json and shadow endpoint
  4. Subnet scan — /24 port 1883 probe when mDNS or /etc/hosts is stale (MQTT_MDNS_FALLBACK=1)

Start the stack and provision shadows:

bash
FLEET_WLAN_ONLY=1 ./cost-sentinel/scripts/phantom-vision-stack.sh up
curl -sS -X POST http://127.0.0.1:8788/fleet/provision-shadow \
  -H 'Content-Type: application/json' \
  -d '{"all":true,"restart_service":true,"force":true}'

Details: cost-sentinel/mqtt-broker/README.md. Off-LAN access remains available with FLEET_WLAN_ONLY=0 and mqtt.ghostprotocol.app tunnel hostnames.

Documentation hub ​

Browse the full operator guide locally:

bash
cd oak-vms-firmware/docs && npm install && npm run docs:dev

The hub covers introduction, getting started, operator guides (including video understanding), architecture, reference APIs, and contributing workflows.

What you can do today ​

Operational capabilities include verified person detection with rich alerts, justified clip playback, soak automation, Phantom orchestrator scenarios, and plugin-based hardware extensions. Capability snapshots and repro commands live in WHAT_WE_CAN_DO_TODAY.md; the docs hub carries the public-safe subset.

Next steps ​

Operator depth ​

Live fleet state, container IDs, soak percentages, and session chronology live in private operator handoff (not published).