Phantom Vision VMS / Ghost Protocol

Appearance

About this article ​

  • Audience: Operators interpreting soak gate results or debugging video understanding runs
  • Goal: After reading, you can read PASS/INCOMPLETE/FAIL decisions and avoid common video QA pitfalls
  • Type: Troubleshooting

Summary ​

Soak gates combine thermal sampling, H.265 chunk growth, and storage health into PASS, INCOMPLETE, or FAIL. Video QA adds cloud escalation on justified clips—watch fps settings, cache reuse, and playback timing. Re-probe through PHANTOM_PROXY after any fix; gate decisions in orchestrator summary.json are authoritative for automation runs.

Prerequisites ​

  • Phantom proxy: export PHANTOM_PROXY=https://<phantom-proxy>:8788
  • Access to soak logs or test-results/v3-runs/<run>/summary.json
  • API quick reference for re-probe curls

Soak gate: PASS ​

Symptom: Orchestrator or soak_gate.sh reports PASS; a .pass marker file exists.

Cause: Encoder produced new H.265 chunks on most samples, die temp stayed at or below 78°C on most samples, recent_chunks was at least 1 in the last two minutes, and storage_health was not blocking.

Fix: No action required. Optional confirm via proxy:

bash
curl -s -X POST "$PHANTOM_PROXY/tools/call?target=<device-ip>" \
  -H "Content-Type: application/json" \
  -d '{"tool":"get_recording_status","arguments":{}}' | jq '.storage_health, .recent_chunks, .is_writing_data'

Soak gate: FAIL (encoder silent) ​

Symptom: FAIL with OK thermals; recent_chunks flat after mid-soak; chunk growth samples drop to zero.

Cause: Disk pressure or a stalled recorder stopped the encoder while the container stayed up. Historical patterns show encoder silence after roughly half the soak duration when storage was critical before prune.

Fix:

  1. Prune oldest chunks (KEEP_CHUNKS=30) — see Updates, prune & storage
  2. Restart the app through the proxy or --restart-only prune script
  3. Re-probe get_recording_status until recent_chunks advances
  4. Re-run soak with orchestrator --prune-first

Soak gate: INCOMPLETE ​

Symptom: Soak watcher reports finished but no SOAK_GATE marker or final summary line; orchestrator marks INCOMPLETE.

Cause: The soak process exited or was interrupted before writing the pass marker. Post-probe data may still show recovery.

Fix: Read probes.json and summary.json in the run directory. If chunks and storage look healthy, treat as a soft pass for manual review; otherwise re-soak. Use orchestrator --watcher for consistent completion detection.

Soak gate: FAIL (thermal streak) ​

Symptom: More than 25% of thermal samples exceed the gate threshold.

Cause: Sustained die temp above the soak limit—often workload or ambient, occasionally throttling.

Fix: Check /fleet/mqtt/latest telemetry trend. Reduce concurrent load, verify fan/airflow, and re-soak after temps stabilize below threshold.

Playback contract mismatch ​

Symptom: justified_ok false, or sidecar duration_s does not match wall-clock chunk length; ffprobe PTS deltas are not linear ~0.033s.

Cause: Legacy fast-forward slice overcount or stale sidecar after overlay lag.

Fix: Confirm .h265_status and recent_manifest.jsonl on device. Enable GATE_PLAYBACK_CALIBRATION=1 in orchestrator runs. Verify via /fleet/playback-ready and ffprobe on a pulled mp4. See Justified recording.

Video QA: fps and clipping ​

Symptom: Cloud model misses motion or returns sparse timestamps on bus footage.

Cause: Default 1 fps sampling is efficient for static 60s clips but under-samples motion.

Fix: Use videoMetadata with fps=5 for transit scenes, or clip the last two minutes from recent_manifest offsets. See Video understanding.

Video QA: cache and size limits ​

Symptom: Upload fails or duplicate analysis charges on repeated clips.

Cause: Clips exceed inline size limits or the same manifest group is re-sent without cache.

Fix: Use File API for clips over ~100 MB or for reuse across prompts. Proxy central owns pull, transcode, and upload—no keys on edge. One video part per prompt; place text after the video part.

Video QA: overlay and proxy lag ​

Symptom: Proxy storage_health still CRITICAL after prune; snapshot returns 503 or cached fallback.

Cause: Overlay merged view lags behind SSH df truth; recorder restart not yet reflected in proxy cache.

Fix: Trust SSH df and .h265_status on device. Wait one probe cycle or restart app, then re-fetch /fleet/mqtt/latest and /stream/snapshot. Full routes: API quick reference.

Visual repro checklist ​

bash
python3 tools/v3_shadow_orchestrator.py \
  --shadow <device-ip> \
  --scenarios normal_short,video_understanding \
  --visual --tap --update-handoff

cd [repo-root]/cost-sentinel
npx playwright test e2e/soak-gate-visual.spec.ts

Next steps ​

Operator depth ​

Live fleet state and harness evidence live in private operator handoff (not published) (private).