Phantom Vision VMS / Ghost Protocol

Appearance

HIL / bus demos · Edge detects · Mac host reasons · Proof moments for executives

Doctrine

RF-DETR and justified recording run on Shadows. Gemma / VLM synthesis runs on the Mac Studio host through Phantom proxy—not on the camera.

Demo day shortlist ​

PriorityActivityWhy show it
101 Semantic searchNatural language find without scrubbing hours of video
202 A2A + MCPZero Trust agent access to the fleet
304 Threat responseEdge detect → physical deterrent path
412 / VMS playbackCentral forensic review story

Category index ​

Advanced AI & Agentic Integration ​

01 — Semantic Search ​

AGENTIC EDGE HUB & AXIS VAPIX SYNERGY

Deploy one Phantom Shadow 2 as an agentic edge hub (industrial SoC-class vision node, ~52 TOPS). On-device RF-DETR precomp stays authoritative; semantic search synthesis uses host Gemma on the Mac Studio. The unit ingests 3–5 external Axis streams via VAPIX/RTSP. Operators monitor DSP thermal bands (~75°C) and RAM headroom for stable loops.

Proof moment: Reduces search time from hours to seconds.

Takeaways

  • Reduces search time from hours to seconds.
  • Handles ambiguous queries (semantic vs. keyword).
  • Zero cloud dependency for core search retrieval.

Walkthrough

  1. The Incubation Event — Have the volunteer wearing the yellow jacket walk through the camera's field of view for 5 seconds, then exit the frame.
  2. Edge Processing (Silent Phase) — Phantom Shadow is passively generating vector embeddings of the scene locally on device storage (128 GB POC / ≥1 TB pilot eMMC) drive. Zero video is being uploaded to the cloud.
  3. The Query — In the VMS Dashboard search bar, type the natural language query exactly: 'Find a passenger wearing a yellow jacket.' Hit Execute.
  4. The Agentic Handshake — Observe the terminal logs. The Google Cloud VMS pushes a lightweight text payload via MQTT to the ghost/{device_id}/cmd/request topic.
  5. Local Vector Retrieval — The Mac host Gemma path (via proxy) parses the text, compares it against its local event index on-device (vectors/metadata), and identifies the exact timestamps of the match.
  6. Targeted Chunk Upload — Phantom Shadow automatically cuts the precise 5-second H.265 video chunk and pushes it to the VMS storage bucket.
  7. Verification — The VMS UI updates instantly, playing the high-res clip from the edge, bypassing hours of manual review.

02 — A2A + MCP ​

The Agentic Bridge (NIST SP 800-207)

Prove military-grade network isolation while demonstrating interoperable, secure communication between COTA's external AI agents and the Ghost Protocol edge hardware.

Proof moment: Eliminates implicit trust vulnerabilities.

Takeaways

  • Eliminates implicit trust vulnerabilities.
  • Standardizes AI access to physical hardware.
  • Audit-ready logs for every interaction.

Walkthrough

  1. The Unauthorized Attempt — From the SOC Terminal, attempt to query the camera without passing the token. Observe the immediate 403 Forbidden response.
  2. The Secure Handshake — Execute the connection script passing the valid Cloudflare token. Watch the Edge Logs register an A2A Auth Success event.
  3. The Interrogation — In the SOC Terminal, type the natural language prompt: 'Agent, what is your current hardware health and threat status?'
  4. MCP Tool Execution — The A2A protocol hands the request to Phantom Shadow 1's internal agent. The agent securely executes local hardware polling (temp < 70C).
  5. The Response — The SOC Terminal receives a perfectly formatted JSON response over the A2A bridge confirming hardware optimal status.

03 — ML Pipeline ​

On-Edge LoRA Fine-Tuning

Demonstrate rapid, targeted model adaptation without retraining the entire base model, utilizing strict .dlc formats to maintain thermal compliance.

> Proof moment: 99.8% smaller update size vs. full models.

Takeaways

  • 99.8% smaller update size vs. full models.
  • Rapid adaptation to environmental shifts.
  • Zero downtime for intelligence updates.

Walkthrough

  1. The Baseline Failure — Hold the novel pass up to the camera. It labels the object generically (e.g., 'unknown object').
  2. Edge Data Capture — From the VMS, click 'Initiate Target Capture.' The camera captures a burst of 50 high-res frames.
  3. The Vertex Pipeline — Show the VMS interface pushing those frames to GCP. Vertex AI generates a tiny 5MB LoRA file.
  4. OTA Deployment — Click 'Deploy OTA' in the VMS to push the 5MB .dlc adapter file to the fleet via A/B rollover.
  5. The Hot-Swap (A/B) — Observe the Edge Logs on Shadow 2. The camera received the 5MB file and injects it into neural memory.
  6. Successful Inference — Hold the pass up to the camera. The VMS immediately tags the object as 'COTA VIP Pass'.

Core Edge Intelligence & Response ​

04 — Threat Response ​

The Autonomous Guard

Prove true zero-latency threat deterrence using the dual-model AI synergy (RF-DETR triggering host Gemma (Mac)) to fire a physical relay.

Proof moment: Eliminates latency-induced response lag.

Takeaways

  • Eliminates latency-induced response lag.
  • Fires physical deterrents autonomously.
  • Hardened against connection failure.

Walkthrough

  1. The Baseline Calm — Show the VMS dashboard. The physical strobe light is off. System is in passive monitoring.
  2. The Threat Introduction — Have the executive step into Shadow 1's field of view and raise the prop weapon.
  3. Instant Edge Inference — RF-DETR processes the frame locally in <20ms on the 48 TOPS DSP, identifying the threat.
  4. The Physical Trigger — In under 1 second, Shadow 1 commands the M8 Controller Box. Relay 1 clicks, strobe flashes.
  5. Cloud Alert Generation — Shadow 1 escalates a justified clip to the Mac host for Gemma narrative, which generates a rich text insight and pushes alert to Google Cloud Platform.

05 — Audio Trig ​

Acoustic Forensics

Demonstrate audio anomaly detection triggering autonomous, pre-buffered video archival utilizing the 8GB RAM 10s Ring Buffer architecture.

> Proof moment: Captures context before the trigger event.

Takeaways

  • Captures context before the trigger event.
  • Privacy-compliant wake-word architecture.
  • High forensic value for off-camera incidents.

Walkthrough

  1. Ambient Monitoring — Show that the camera is not uploading video. It is passively monitoring audio levels matching predetermined signatures.
  2. The Acoustic Trigger — A volunteer shouts the trigger word. Shadow 2's audio model detects the anomaly in <50ms.
  3. The RAM Rescue — The logic forces the system to dump the 10-second pre-event buffer from RAM to eMMC, then streams to disk.
  4. AI Contextualization — host Gemma (Mac) parses the clip to explain the event (e.g., 'Passenger in distress'). Alert pushes to Shadow Chat.

06 — J1939 Alert ​

The Panic State Matrix

Showcase how a physical J1939 CANbus trigger from the vehicle can instantly escalate edge operational state, initiating real-time VLM narration.

> Proof moment: Instant bidirectional visual link.

Takeaways

  • Instant bidirectional visual link.
  • Fuses driver input with AI intelligence.
  • Reduces dispatcher decision fatigue.

Walkthrough

  1. CANbus Interrupt — Press the button. A high-priority PGN is broadcast across the vehicle bus. The M8 Adapter catches it in <5ms.
  2. State Escalation — Phantom Shadow 1 immediately switches to 'Emergency Mode': 4K 60fps local recording + real-time 5G streaming.
  3. Physical Escalation — Status LEDs instantly switch to RED. Phantom Shadow 1 publishes event to phantom/{device_id}/telemetry/can.
  4. VLM Live Narration — host Gemma (Mac) begins analyzing the video locally and streams a real-time text narration directly to the SOC.

Liability Defense & Telemetry ​

07 — CANbus Sync ​

The Invisible Mechanic

Prove that the M8 CAN adapter ingests high-speed vehicle telemetry and syncs it with the video timeline for undeniable mechanical forensics.

> Proof moment: Frame-accurate telemetry alignment.

Takeaways

  • Frame-accurate telemetry alignment.
  • Reduces troubleshooting time.
  • Enables cloud-based digital twins.

Walkthrough

  1. Baseline Telemetry — Point out the line graph in the VMS showing steady, normal RPMs correlated with the live feed.
  2. The Mechanical Event — Execute 'Erratic RPM' simulator script (PGN 61444). The M8 captures this proprietary PGN in real-time.
  3. The Frame Sync — The telemetry graph spikes in perfect synchronization with the exact millisecond of the video feed.
  4. The Liability ROI — Explain to COTA maintenance that they no longer have to guess vehicle states during accidents—the raw J1939 data is an immutable forensic record.

08 — Sensor Fusion ​

Automated Liability Defense

Illustrate how fusion of CANbus telemetry and Edge AI vision completely mitigates fraudulent claims by creating an immutable incident report.

Proof moment: Automatic incident packet generation.

Takeaways

  • Automatic incident packet generation.
  • Non-repudiation via crypto signing.
  • Factual oversight via AI models.

Walkthrough

  1. Establish Telemetry State — Execute simulation for 'Doors Open' and 'Speed: 0 mph'.
  2. The Staged Incident — Have the volunteer safely simulate a slip-and-fall event in front of Shadow 1.
  3. Sensor Fusion Activation — The AI detects a person falling and immediately grabs the CANbus state from the M8 adapter.
  4. VLM Synthesis — host Gemma (Mac) analyzes the event, correlating the visual anomaly with the mechanical data (Stationary & Doors Open).
  5. The Immutable Report — Shows VMS report with H.265 frame, 0 mph telemetry, and AI summary definitively disproving moving-bus claims.

09 — Compliance ​

Transit Operational Auditing

Verify that the system can autonomously audit ADA compliance using Shadow 2 stereo depth sensing to identify wheelchairs and loading sequences.

Proof moment: Automated loading sequence audit.

Takeaways

  • Automated loading sequence audit.
  • Reduces audit overhead by 90%.
  • Evidence-based driver coaching.

Walkthrough

  1. Vision Detection — RF-DETR and NeuralDepth correctly identify the mobility device based on spatial coordinates (RDF format).
  2. The Audit Sequence — In the simulator, fire the signals in correct ADA order: Parking Brake -> Turn Indicators -> Ramp.
  3. Successful Score — Show VMS dashboard issuing automated 'ADA Compliance: 100%' score.
  4. The Failure State — Reset script. Deploy ramp before engaging parking brake. VMS instantly flags the violation.

Advanced Swarm Logic & IT Extensibility ​

10 — A2A Swarm ​

Autonomous A2A Self-Healing Swarm

Demonstrate that multiple edge nodes can dynamically negotiate workloads over the local MQTT broker to cover physical hardware failures.

Proof moment: Dynamic blind-spot mitigation.

Takeaways

  • Dynamic blind-spot mitigation.
  • Reduces downtime from vandalism.
  • Autonomous node coordination.

Walkthrough

  1. Baseline Operation — Show both cameras running standard AI matrices with overlapping situational awareness.
  2. The Hardware Failure — Physically cover Shadow 1's lens. The node instantly detects obstruction via edge vision logic.
  3. A2A Negotiation — Watch the terminal logs. Shadow 1 detects blindness and broadcasts a CRITICAL_BLINDNESS payload over MQTT.
  4. The Dynamic Shift — Shadow 2 receives the payload and automatically shifts its optical focus and processing to cover Shadow 1's blind spot.
  5. SOC Notification — VMS generates an automated IT ticket, proving the fleet self-heals dynamically.

11 — API / CLI ​

Enterprise Control Plane: API & CLI VMS Access

Prove to COTA's IT department that they can programmatically manage the fleet without proprietary vendor lock-in.

Proof moment: Programmatic fleet control.

Takeaways

  • Programmatic fleet control.
  • Developer-first infrastructure.
  • Rapid prototyping for SOC tools.

Walkthrough

  1. Status Polling via CLI — Type 'vms-cli fleet status --json'. System returns JSON showing health, DSP temp, and uptime.
  2. Programmatic Video Fetch — Use a raw cURL command to request a specific 10-second H.265 chunk from Shadow 1.
  3. The Download — The chunk downloads directly to the machine over the secure connection. No direct IP access needed.
  4. Remote GPIO Trigger — Use Postman to fire an HTTP POST targeting Shadow 1's M8 Controller. Strobe flashes instantly.

12 — Failover ​

Network resilience (LAN preferred · M8 Wi‑Fi secondary)

Doctrine: production Shadows prefer LAN over PoE+. This activity proves seamless failover to the M8 Wi‑Fi adapter when the hardline data path is cut — while PoE+ power stays up. See Shadow connectivity.

Proof moment: Zero-interrupt mission data.

Takeaways

  • PoE+ LAN is the preferred primary path.
  • M8 Wi‑Fi adapter provides secondary uplink.
  • Resilient to infra sabotage on the data plane.

Walkthrough

  1. The Active State — Continuous telemetry on primary Ethernet/PoE+ hardline (phantom/{id}/health).
  2. The Sabotage — Disable the switch data port. Keep PoE+ power so the camera stays on.
  3. Instant Failover — Edge detects eth0 down and activates wlan0 via the M8 Wi‑Fi adapter.
  4. Telemetry Continuity — MQTT resumes; RSSI stats on telemetry/wifi.
  5. The Security Narrative — Cutting the primary router uplink does not stop the node when M8 WLAN is configured.

13 — Edge DVR ​

The Cellular Bandwidth Saver (Smart Edge DVR)

Prove that 48MP camera arrays will not exhaust transit cellular data plans by leveraging H.265 chunking and edge storage.

Proof moment: 99.5% reduction in cloud costs.

Takeaways

  • 99.5% reduction in cloud costs.
  • Local storage allows 4K retention.
  • Metadata-first discovery.

Walkthrough

  1. The Passive Baseline — Show that despite Shadow 1 actively recording 4K video, outbound bandwidth is nearly zero (lightweight MQTT only).
  2. Querying the Edge — Execute search. VMS requests manifest over MQTT, revealing available 5-minute video chunks.
  3. The Surgical Request — Request only chunk #3. Shadow 1 cuts the relevant H.265 chunk locally.
  4. The Transfer — Network monitor spikes temporarily as specific 5MB file transfers, dropping to zero after.
  5. The ROI Pitch — Ghost Protocol reduces data footprints by over 99% compared to traditional cloud NVRs.

14 — Network Health ​

Edge Observability (Network Health Monitoring)

Show that the edge AI nodes actively monitor and audit the transit authority's own network infrastructure health.

Proof moment: Real-time fleet signal heatmap.

Takeaways

  • Real-time fleet signal heatmap.
  • Identifies dead zones automatically.
  • Audits carrier performance.

Walkthrough

  1. Payload Inspection — Once upload completes, inspect incoming VMS API payload via Network tab.
  2. Schema Analysis — Point out the 'upload_metric' object adhering strictly to the MQTT architecture schema.
  3. Data Verification — Show sizing, latency, and speed metrics generated autonomously by the edge node.
  4. The IT Value Add — Phantom Shadow nodes report degraded upload speeds of Cisco routers automatically, allowing proactive IT resolution.

15 — P2P Stream ​

Zero-Latency Dispatcher (WebRTC Live Stream)

Prove that dispatchers can view crisp video with zero cloud-proxy delay during high-threat scenarios.

Proof moment: Sub-100ms visual feedback.

Takeaways

  • Sub-100ms visual feedback.
  • Bypasses server congestion.
  • Crisp 1080p on cellular backup.

Walkthrough

  1. Initiate Stream — Click 'Connect Live'. The go2rtc sidecar negotiates a direct P2P connection to your browser.
  2. The Latency Test — Volunteer claps on camera. Action appears on screen almost instantaneously.
  3. The Dispatcher Benefit — Zero-latency WebRTC feed gives police true real-time situational awareness vs 10s cloud delay.

16 — Legacy Hijack ​

The Legacy AI Multiplier (Axis + VAPIX Synergy)

Demonstrate that purchasing Ghost Protocol hardware acts as a 'Fleet-Wide Edge Compute Upgrade Module' for existing Axis cameras.

Proof moment: Upgrades legacy gear with AI.

Takeaways

  • Upgrades legacy gear with AI.
  • Prevents rip-and-replace costs.
  • Unified AI logic across mixed fleets.

Walkthrough

  1. The Hijack — Shadow 1 pulls RTSP from Axis without interrupting the Axis primary recording.
  2. The Brain Upgrade — Shadow 1 routes Axis video into its own 48 TOPS DSP, running detection models over old footage.
  3. The Physical Takeover — Shadow 1 detects threat, fires VAPIX POST command back to Axis to trigger its LED/local SD recording.
  4. The Cloud Bridge — Shadow 1 passes Axis frame to host Gemma VLM, generates text insight, and pushes to VMS.
  5. The Financial Flex — Buy one Shadow per bus and upgrade the whole network with Tier-1 AI without rip-and-replace.

17 — Compliance ​

Federal Compliance: NIST FRVT

Prove that the architecture adheres to strict federal biometric privacy laws and NIST standards for POI tracking.

Proof moment: NIST-vetted algorithms.

Takeaways

  • NIST-vetted algorithms.
  • FIPS 140-2 encryption in transit.
  • Compliant with federal privacy grants.

Walkthrough

  1. Edge Detection & Crop — Edge node detects face and crops high-res bounding box. No biometric templates stored locally.
  2. FIPS 140-2 Transmission — Crop is transmitted to Cloud VMS over FIPS-compliant mTLS tunnel.
  3. NIST FRVT Matching — Backend routes image through DeepFace vector pipeline vetted by NIST FRVT.
  4. The Match — VMS dashboard flashes positive match for POI.
  5. The Compliance Guarantee — FIPS encryption and NIST-vetted algorithms ensure 100% eligibility for federal transit security grants.