Phantom Vision VMS / Ghost Protocol

Appearance

Reference: API & Surfaces ​

Consolidated catalog of the unified observable surface. Proxy at nexus.ghostprotocol.us:8788 (phantom-proxy/proxy service) is the primary BFF + tap for the fleet. It forwards MCP (8765 inside CID), ingests MQTT (1883), serves cached state, provides SSH fallbacks, and augments responses. All surfaces are live-probed in handoff.md and collector artifacts.

Public-safe docs: Examples use placeholders. Live fleet IDs, credentials, telemetry snapshots, harness evidence, and session chronology are maintained in operator-controlled handoff repositories — not on the public web.

Evidence artifacts (from API/TAP DEMO subagent 2026-06-08 and Phase 2 close): [operator-artifacts], [operator-artifacts], [operator-artifacts], [operator-artifacts], [operator-artifacts], [operator-artifacts], tap logs, orchestrator TAP. (operator handoff (private), harness artifacts).

Proxy (8788) — Primary Unified Surface ​

Routes implemented in phantom-proxy/proxy service (see specific lines below; full catalog from grep + operator handoff (private)).

Fleet Registry + Bus ​

  • GET /fleet/registry → 200 with registry, vms_lan_ip, shadows list (ips, labels, roles, bus_id). "bus=12345 confirmed in curls + registry. (operator handoff (private), harness artifacts, private implementation (operator repo)).
    • Verbatim live (from Phase2/collector probes): shadows include roles/labels (e.g. "site-role-a", "site-role-b" patterns in registry feeds); cross visibility for hermes/a2a.
    • "proxy bus=12345 (confirmed curls + registry)" (operator handoff (private) Phase 2 close).

Example (verbatim patterns from handoff/collector):

bash
curl -s http://nexus.ghostprotocol.us:8788/fleet/registry
# -> { "registry": { ... bus_id or bus=12345, shadows: [ {vms_lan_ip, device_id: "shadow1", ... labels/roles}, ... ] }, "path": "..." }

MQTT Latest Telemetry / Dets / Track ( ~10 Hz ) ​

  • GET /fleet/mqtt/latest → 200, devices: {shadow1, shadow2} with status, payload.telemetry{die_temp:<celsius>/ <celsius>, fps:10}, dets (track_id in history), recent_chunks, anomaly_alerts, stale_sec:120. (operator handoff (private), 293; harness artifacts).
    • Live rates: "die_temp:<celsius>/ <celsius>, fps:10", "track_id:<id>" (or example id range), MQTT ~9.9-10 Hz from contract probes. (operator handoff (private), 455, 495; harness artifacts).
    • "devices/shadow*/(telemetry|dets|health|status|...)" (operator handoff (private)).

Verbatim curl + output note (handoff/collector live examples):

bash
curl -s http://nexus.ghostprotocol.us:8788/fleet/mqtt/latest
# 200: devices: {shadow2: { ... telemetry: {die_temp:&lt;celsius&gt;, fps:10}, dets: [..., {track_id:&lt;id&gt;, ...}] , ... }}
# Saved: [operator-artifacts] (129kB)

MQTT paho wildcards (DEFAULT in proxy + taps): devices/+/status, devices/+/telemetry, devices/+/telemetry/#, devices/+/dets, devices/+/health, devices/+/data_manifest, devices/+/responses, devices/+/events, devices/+/events/# (private implementation (operator repo); operator handoff (private) "devices/shadow*/(telemetry/health/dets/status/...) ~10Hz"; tap_shadow_session.py paho subs).

Playback-Ready + Justified Clips ​

  • GET /fleet/playback-ready → 200, recent_chunks, count, justified metadata + playUrl (mp4 from transcode/cache). (operator handoff (private), 298; harness artifacts; private implementation (operator repo)).
    • "justified 60s H.265 + sidecar/recent_manifest as perfect input (duration_s~60, keyframe_count, justified, wall-time==depicted)" (gemini plan + operator handoff patterns, diagrams mermaid 3).

Tools / MCP Forward + Aug ​

  • POST /tools/call?target=nexus.ghostprotocol.us (or shadow-canary) with {"tool":"get_recording_status", "arguments":{}} → 200, result: {total_chunks:&lt;n&gt;, inventory_count:&lt;n&gt;, storage_health:"CRITICAL", storage_free_gb:&lt;gb&gt;, storage_used_pct:&lt;pct&gt;, latest_chunk:..., is_writing_data:true, recommendation:"Recording appears stalled..."}. Also get_latest_dets: {person_count, track_id:&lt;id&gt;, temp_c:&lt;celsius&gt;}. (operator handoff (private), 294; harness artifacts).
    • Other tools: get_device_health, get_system_health, get_device_identity (fp), get_hermes_mesh_status, get_latest_dets.
    • Aug (FULL_KEYS + enrichment): In private implementation (operator repo) (health path): for get_system_health / get_device_health, parallel forwardToolsCall for get_device_identity + get_hermes_mesh_status; injects cert_fingerprint, hermes_mesh, cert_fingerprint_present, peer_comm_status, listen_ports, overlay_mount, _vms_enriched: true (and into inner result). "Best-effort injection of the exact FULL_KEYS the mcp_health 5min test asserts" (readiness harness). (private implementation (operator repo), 476-479 "FULL_KEYS = ["peer_comm_status", "hermes_mesh", "listen_ports", "overlay_mount", "cert_fingerprint_present"]"; operator handoff (private); harness artifacts).
    • Device MCP (8765 inside CID, proxied): full surface forwarded via /tools/call. (operator handoff (private)).

Verbatim curl (handoff/collector):

bash
curl -s -X POST 'http://nexus.ghostprotocol.us:8788/tools/call?target=nexus.ghostprotocol.us' \
  -H 'Content-Type: application/json' \
  -d '{"tool":"get_recording_status","arguments":{}}'
# -> {tool, target, status, result: {storage_health:"CRITICAL", ...}, ... }  (or WARNING post-prune)
# Also exercised: get_device_identity (fp non-null e.g. [device-fingerprint] sha256), get_hermes_mesh_status.
# Saved: [operator-artifacts] shadow2_post_durable..._proxy_*.json

Aug in envelope (verbatim proxy code excerpt, private implementation (operator repo)):

text
# Health responses may be enriched server-side with identity and mesh fields.
# See operator handoff for the private proxy augmentation contract.

Snapshot + Streams ​

  • GET /stream/snapshot?target=nexus.ghostprotocol.us&w=320&q=3 → 200 (jpeg) or 503/JSON graceful ("live_jpeg_unavailable_mcp_and_ssh_failed"; falls back to cached tail .h265 ffmpeg in proxy). (operator handoff (private); harness artifacts; private implementation (operator repo)).
  • Related: /stream/live/status, /stream/live/info, /fleet/snapshots, /camera/profile*. (private implementation (operator repo)+).

A2A Query (HMAC) ​

  • POST /a2a/query (and /a2a/sign) → HMAC signed; q=peer_hello etc returns [peer-ack]. Forwards to target (private implementation (operator repo), 1162; operator handoff (private), 285, 305).
  • "POST /a2a/query HMAC". (harness artifacts).

Debug + Events ​

  • GET /debug/events → 200 ring buffer (vmsDebug: live/warm_ready, snapshot_fail, ssh_jpeg_fail, shadow (no .h265_status/ghost_core.log), recorder_status). Ingest + ring from vms-debug-log.js. (operator handoff (private); harness artifacts; private implementation (operator repo)).
  • /debug/shadow/*/logs (SSH overlay tails), /debug/log/tail, /debug/export. (private implementation (operator repo)+).

Pull Recording + Others ​

  • GET /pull-recording (ssh2 SFTP + discoverChunksViaSsh fallback; supports recent/latest; transcode to mp4 in cache). (operator handoff (private); harness artifacts; proxy service).
  • /fleet/health, /fleet/grade, /fleet/readiness, /fleet/remediate, /fleet/provision-shadow, /fleet/validate, /fleet/recorder-inventory, /fleet/host-manifest, /health, /phantom-agent/* (resolve/execute/memory via phantom-agents/*.yaml), Flask /api/* proxy. (operator handoff (private); private implementation (operator repo)).

ONVIF MVP Camera Surface (new post-2026-06; "stick this [https://github.com/onvif] mostly please for now") ​

  • Device exposes standard ONVIF (WS-Discovery 3702 + Device/Media2/Events per Profiles S/T) for "see configurable/encoding/detections".
  • Discovery: UDP 3702 Probe/ProbeMatch (scopes Profile/T + NVT).
  • Device: GetDeviceInformation (id/model/firmware/fp from device_identity).
  • Media2: GetProfiles/GetVideoEncoder* (H265_* from mcp + ghost_core), SetVideoEncoderConfiguration (maps to /data/config H265_* + camera_profile.env write like mcp 1419-1426; effect in ghost_core.log + chunks), GetStreamUri (rtsp://lan:8554/preview via go2rtc + existing H.265 from ghost_core fifo + justified recorder 60s).
  • Events: PullPoint (person c=0 from RF-DETR .latest_dets.json / ghost_agent precomp/enrich / get_latest_dets; motion + ConfigurationChanged topics).
  • Reuse: entrypoint bg launch (exact peer 8777 pattern), go2rtc.yaml 8554 + mcp hint, ghost_agent dets flow (c=0 person primary), mcp get/set_camera_profile + get_latest_dets, recorder sidecar/justified, device_identity fp, proxy 8788 for central VMS.
  • Test: ONVIF DM/VLC (rtsp play) + Milestone/Genetec events + python-onvif + orch/phantom onvif module + rates.
  • See: docs/reference/onvif.md (quickstart + reuse anchors + fidelity "Live in handoff always." + sacred 1780795835_170 4/4 bar). Cross handoff (MVP camera pivot section), FRESH_START (post-deploy ONVIF verif), oakapp.toml (baked sidecar notes).
  • "Live in handoff always." + clean bake only (FRESH_START). Sub speed. Best decisions: light sidecar, 0 new encoder/RTSP, reuse everywhere.

Proxy impl summary (private implementation (operator repo) from handoff): "http server routes for snapshot (live-stream-fast.js + ssh ffmpeg fallback), fleet/mqtt/latest (serves mqttLatest cache + extras), tools/call (forward to target:8765/tools/call or SSH discoverChunksViaSsh + normalize), debug/events (from ring), /fleet/health (mcpHealthCache), MQTT client subs DEFAULT topics..., startMqtt + ingestMqttMessage to mqttLatest + vmsDebug ingest + automation_logs; also /pull-recording (ssh2 SFTP), Flask /api/* proxy, phantom-agent, transfer queue."

Device MCP (8765, Proxied) ​

  • Forwarded via proxy POST /tools/call?target=.... Tools: get_recording_status, get_latest_dets (dets+track_id), get_device_health, get_system_health, get_device_identity (fp + cert_fingerprint), get_hermes_mesh_status, others (conversational_safety_query etc.).
  • "Device MCP (8765 inside CID, proxied via /tools/call): get_recording_status, get_latest_dets (dets+track_id), get_device_health etc. Full surface forwarded." (operator handoff (private)).
  • Health enrichment lives in proxy aug (see above); edge mcp_server.py provides base (health, identity fp) + precomp RF-DETR grounding (mcp_server health enrichment + device_identity fp referenced in tests/unit/private test module, private test module "cert_fingerprint non-null", harness artifacts).

MQTT Bus (1883, Direct or via Proxy) ​

  • Topics: devices/shadow1|shadow2/(telemetry|dets|health|status|data_manifest|responses|events|...) ~10 Hz.

Refactor + plugin verification: class-based /check-work PASS; orthogonal to RF-DETR core (precomp/ghost/onvif Events); CLI vms_cli full + API dual-ns/proxy 8788 + RF-DETR precomp/sacred 1780 preserved post-refactor. Tested with ... + plugin tests + onvif + full gates. Crosses to handoff refactor section. Live in handoff always. API surfaces (dual-ns 8765/1883 + proxy 8788) unchanged.

  • "MQTT bus (1883, tap via sub or /fleet/mqtt/latest): devices/shadow2/(telemetry|dets|health|status|...) ~10Hz." (operator handoff (private)).
  • Paho in scripts/tap_shadow_session.py + proxy ingest (DEFAULT wildcards above).
  • "tap_shadow_session.py: paho sub, on_mqtt_message captures hz/die_temp/dets_count/track_ids/storage; + live dump + proxy debug replay + reverse manifest/fifo". (operator handoff (private)).

A2A / Hermes / Phantom Integration ​

  • POST /a2a/query HMAC (peer_hello [peer-ack]; cross via registry bus=12345). (operator handoff (private), 304).
  • Hermes: UDP 11555 HMAC (mesh), 8777 mTLS sidecar (standalone_peer_comm), device_identity fp non-null, registry bus=12345, MQTT hello. Phantom agents (5min modules: device_identity_fp, peer_sidecar, hermes_mesh, mcp_health with FULL_KEYS post-aug, cross_a2a_federation, depth_fusion + video modules). (operator handoff (private), 300-306; readiness harness DEFAULT_MODULES, 479 FULL_KEYS; harness artifacts).
  • /phantom-agent/resolve|execute|memory|... on proxy.

Scripts / CLI / Orchestrator as Executable API ​

  • tools/v3_shadow_orchestrator.py --tap --visual ... (emits TAP lines for every proxy/MQTT/script call; probes via /fleet/* + /tools/call).
  • scripts/tap_shadow_session.py --test-dump, automate_shadow_soak.py, verify_fleet_contract.py, batch_*, visual_soak_gate.py.
  • "Scripts/CLI/Python (executable API): ... tap_shadow_session.py ... tools/v3_shadow_orchestrator.py ...". (operator handoff (private)).
  • "orchestrator --tap for broad sub + FIFO/manifest". (operator handoff (private)).

New Video Understand Surface (or Aug) ​

  • Surface via existing /tools/call (tool=video_understand or aug) or planned GET /video/understand?target=...&clip=last2m... (per gemini plan integration points; "Augment existing; no new surface needed" decision).
  • Inputs: justified from /fleet/playback-ready + /pull-recording + recent_manifest groups. Outputs fuse to /debug/events, MQTT, handoff, TAP. (gemini plan:3. Concrete; harness artifacts "new /video endpoints or tool aug"; proxy aug pattern for health extends to video).
  • Phantom modules + orchestrator SCENARIO "video_understanding" expose via same proxy/MCP paths. (See architecture/video-cloud.md and operator-guides/video-understanding.md).

Cross-References ​

  • Full live justified clips + rates + Phase2 artifacts: handoff.md (Phase2 close, API surface catalog, justified + playback proof).
  • Existing inventory: docs/shadow_api_mqtt_inventory.md (legacy 439 routes context), docs/MQTT_ARCHITECTURE.md (namespace), docs/IMPLEMENTATION_PLAN_DEPTHAI_V3.md (component map).
  • Phantom integration + 5min readiness: tools/phantom_vms_readiness_agent.py.
  • Spatial fusion + device_identity tests: tests/unit/test_spatial_fusion.py, test_device_identity.py, test_mcp_server.py (FULL_KEYS assertions post-aug).
  • Operator guide: operator-guides/video-understanding.md.

ONVIF MVP Camera Surface (MVP + NIST Hooks) ​

  • ONVIF sidecar (standalone_onvif.py baked in entrypoint): WS-Discovery UDP 3702 (Probe/ProbeMatch, scopes Profile/T), Device GetDeviceInformation (fp from device_identity), Media2 GetProfiles/GetVideoEncoder*/SetVideoEncoderConfiguration (H265_* writes via mcp pattern -> ghost_core), GetStreamUri (rtsp://lan:8554/preview via go2rtc + justified H.265), Events PullMessages (person c=0 events b/p/conf/track_id from RF-DETR .latest_dets / precomp / get_latest_dets).
  • Proxy aug: /fleet/mqtt/latest + /debug/events + /tools/call get_latest_dets (person c=0) + get_device_identity (fp) surface the same.
  • NIST: RMF map + FATE/FRTE (nist_fr liveness/quality/blur) + video cyber (ZT, precomp sacred, prune, signed GCS) + model card hook: see appendices/nist-compliance.md + reference/model-card.md (PERSON_CLASS_ID=0, conf, precomp sacred first).
  • Verbatim cross (handoff approved plan): "Phase 2 complete (tracks 2,3,4,5,7,8 closed per approved plan) ... GChat person alerts on dets now live-verified".
  • Gates: vision-verify + soak (encode/streak/thermal/storage post-prune) + unit (test_mcp_server FULL_KEYS + fp) + e2e; conf in precomp 0.35; blur/audit in nist_fr + .h265_status justified.
  • Mermaid (ONVIF + API surfaces):

Live in handoff.md always. Verbatim crosses from handoff + onvif.md + depthai.md + FRESH_START + precomp_metrics (PERSON=0) + nist_fr + gates (full assertions in unit/e2e). See reference/onvif.md + nist-compliance + model-card. Fidelity: bak-first + anchors.

(End of API page extensions.)