About this article

• Audience: Operators and integrators learning the Phantom Vision VMS mental model
• Goal: Understand how edge shadows, central proxy, and cloud escalation fit together before deploying or extending workflows
• Type: Concept

Summary

Ghost Protocol runs sovereign firmware on Shadow edge cameras, routes fleet visibility through a central proxy, and escalates justified video clips to cloud analysis only when richer temporal reasoning is needed. On-device RF-DETR detections remain authoritative; the proxy owns every cloud API call.

Prerequisites

• Quickstart completed on at least one shadow
• Phantom proxy reachable at https://<phantom-proxy>:8788

Product videos

Streamed via Cloudflare Stream (no downloadable MP4 links). Full briefs under Products:

Product	Stream key	Spec page
Shadow 1 / Shadow 2	shadow	Shadow Edge Devices
Phantom Vision VMS	vms	Phantom Vision VMS

Shadow fleet (edge)

Loading stream…

Phantom Vision VMS (center)

Loading stream…

System tiers

The stack has four cooperating layers. Read top to bottom for the control path; video flows upward from edge to proxy, then optionally to cloud.

Edge shadows

Each Shadow runs production Ghost Protocol firmware on industrial SoC-class vision hardware:

• ghost_core — 4K H.265 primary stream plus an independent RF-DETR substream for on-device detection
• ghost_agent_v2 — justified 60-second chunk recorder with sidecar metadata and a rolling manifest
• entrypoint — FIFO setup, sidecar launch, and peer communication startup

Edge images stay sovereign: no cloud API keys ship in the container. Layer3 depth fusion adds spatial context at reduced frame rate alongside detections.

Hermes peer mesh

Shadows discover peers over UDP broadcast with HMAC authentication and exchange status over MQTT. A dedicated mTLS sidecar on each device supports direct peer queries. Cross-site visibility is aggregated at the central proxy via /fleet/registry.

Proxy central VMS

The Phantom proxy at https://<phantom-proxy>:8788 is the fleet back end and observable tap:

• Ingests MQTT telemetry and exposes /fleet/mqtt/latest
• Serves playback-ready clips via /fleet/playback-ready
• Forwards MCP tool calls (get_device_identity, health probes, mesh status)
• Pulls recordings, transcodes, and runs cloud escalation on demand

Site roles use labels such as site-role-a and site-role-b rather than physical location names.

Phantom vision and automation

Phantom agents, playbooks, and the shadow orchestrator drive soak batches, visual gates, and handoff updates. TAP traces combine proxy curls, MQTT subscriptions, and checkpoint artifacts for reproducible verification.

Cloud escalation

Escalation follows a fixed order:

1. On-device RF-DETR precomp metrics (always first)
2. Gemini video understanding for temporal summaries and timestamped Q&A
3. Google Cloud Video Intelligence for structured label and shot detection

The proxy pulls justified clips, uploads to object storage or the File API, and fuses results back into MQTT, debug events, and operator workflows.

Dataflow

Edge recorder (60s justified chunks)
  → sidecar + recent manifest
  → proxy playback-ready or pull-recording
  → (optional) cloud video APIs
  → fused NL + structured labels
  → TAP / MQTT / A2A enrichment

Key contracts

Contract	Public expectation
Playback	Wall-clock 60s chunks; frame timing matches real time
Health / MCP	Identity fingerprint present; mesh and peer status reported
Soak gate	Sustained chunk growth, thermal within threshold, justified metadata valid
Video prompts	One video per request; proxy owns upload and auth

Detailed ffprobe checks, gate scripts, and harness evidence are documented in sub-pages—not duplicated here.

Cloud platform direction

The Ghost Protocol team is migrating central services to managed Google Cloud Platform components: Cloud Run for proxy workloads, Cloud Storage for clip archives, Cloud SQL with pgvector for embeddings, BigQuery for fleet telemetry history, and Vertex AI for generative video reasoning.

Next steps

• Shadow Edge Devices — S1/S2 roles, requirements, product video
• COTA POC topology — bus overlay (interactive + mermaid)
• Phantom Vision VMS — VMS benefits product video
• Architecture diagrams — ports, protocols, and component relationships
• Justified recording contract — sidecar fields and playback verification
• Cloud video escalation — Gemini and Video Intelligence integration
• Edge AI and RF-DETR — pipeline and precomp authority
• API reference — proxy endpoint lookup

Operator depth

Live fleet state, container IDs, soak percentages, and session chronology live in private operator handoff (not published).