Phantom Vision VMS / Ghost Protocol

Appearance

About this article ​

  • Audience: Operators running soak gates or automation when recording storage is tight
  • Goal: After reading, you can prune H.265 archives, restart a stalled recorder, verify storage through the proxy, and run a soak gate before longer automation
  • Type: How-to

Summary ​

Low free space on the recordings volume is the most common soak blocker. Prune oldest chunk archives, confirm storage_health through PHANTOM_PROXY, then run a soak gate. Use --prune-first in orchestrator flows so disk relief happens before apply, verify, or long soaks.

Demo: Specter finding past-retention orphan video chunks (KEEP live streams) and purging on Shadow 1 / Shadow 2 — Video orphans purge. Full promo pack: Product promo videos.

Loading stream…

Prerequisites ​

  • Phantom proxy: export PHANTOM_PROXY=https://<phantom-proxy>:8788
  • SSH to shadows: root@<device-ip> with YOUR_DEVICE_PASSWORD
  • Scripts in [repo-root]: cost-sentinel/scripts/shadow-prune-recordings.sh, scripts/soak_gate.sh
  • One shadow targeted (start with shadow-canary / site-role-a)

1. Check storage health ​

Probe recording status before and after any prune:

bash
curl -s -X POST "$PHANTOM_PROXY/tools/call?target=<device-ip>" \
  -H "Content-Type: application/json" \
  -d '{"tool":"get_recording_status","arguments":{}}' | jq .

Also read fleet health for a cross-shadow view:

bash
curl -s "$PHANTOM_PROXY/fleet/health" | jq .

Verify: storage_health is not CRITICAL, is_writing_data is true, and recent_chunks is at least 1. Proxy responses can lag overlay mounts after a prune—confirm with SSH df on the device when numbers disagree. Full routes are in API quick reference.

CRITICAL storage

When storage_health is CRITICAL, encoder samples often fail and soak gates return FAIL or INCOMPLETE even if thermals look fine. Prune, restart the recorder if stalled, re-probe, then re-soak.

2. Prune recordings ​

From [repo-root], run the fleet retention script. It deletes oldest chunk_*.h265 files and matching sidecars while keeping the newest N per host, then restarts ghost_agent_v2 when recording appears stalled.

bash
# Plan only — no deletes or restarts
DRY_RUN=1 bash cost-sentinel/scripts/shadow-prune-recordings.sh

# Prune both shadows, keep newest 30 chunks each
KEEP_CHUNKS=30 bash cost-sentinel/scripts/shadow-prune-recordings.sh

# Stall check + recorder restart only (no prune)
bash cost-sentinel/scripts/shadow-prune-recordings.sh --restart-only

# Prune without restart
bash cost-sentinel/scripts/shadow-prune-recordings.sh --no-restart

Environment overrides:

bash
KEEP_CHUNKS=20 \
SHADOW1_IP=<device-ip> \
SHADOW2_IP=<device-ip> \
SHADOW_SSH_PASS=YOUR_DEVICE_PASSWORD \
DRY_RUN=1 bash cost-sentinel/scripts/shadow-prune-recordings.sh

The script searches internal and external recording roots (including overlay views) and prefers a large external mount when one qualifies. Always start with DRY_RUN=1 on unfamiliar fleets.

3. Post-prune verification ​

Re-probe through the proxy, then confirm disk on the device:

bash
curl -s -X POST "$PHANTOM_PROXY/tools/call?target=<device-ip>" \
  -H "Content-Type: application/json" \
  -d '{"tool":"get_recording_status","arguments":{}}' | jq '.storage_health, .storage_free_gb, .is_writing_data'

ssh root@<device-ip> 'df -h /overlay /app-storage /data/ext-recordings 2>/dev/null; \
  cat /app-storage/recordings/.h265_status 2>/dev/null | head -20'

Verify: Free space increased, chunk inventory dropped to the keep count, and .h265_status shows a recent last_write.

4. Run a soak gate ​

After storage is acceptable, run the one-hour promote gate (adjust duration for a short check):

bash
SOAK_LOG=logs/soak_gate_run.log \
SOAK_PASS_MARKER=logs/soak_gate.pass \
MQTT_HOST=<phantom-proxy> \
CAMERA_ID=shadow-canary \
bash scripts/soak_gate.sh <device-ip> YOUR_DEVICE_PASSWORD 60

Pass criteria: Die temperature stays at or below 78°C on most samples, and recent H.265 chunks appear on most samples. More than 25% failures on either check marks FAIL. A .pass marker file is written on success.

For automated prune + soak + handoff updates, use the orchestrator with --prune-first (see soaks and automation).

5. External USB relief (optional) ​

Chronic internal pressure on shadow-secondary (site-role-b) may need a qualifying USB SSD mounted and redirected to the recordings subtree. Entrypoint detection and prune scoping already support external roots when mounted.

See USB extended storage for format, mount, and test steps.

Typical sequence ​

  1. Prune (KEEP_CHUNKS=30 or orchestrator --prune-first)
  2. Optional firmware or layer update (deploy)
  3. Short contract verify or sanity soak
  4. Full soak or batch matrix with --visual --update-handoff
  5. Phantom readiness and promotion checks

Troubleshooting ​

Proxy still shows CRITICAL after prune: Overlay accounting can lag. Trust SSH df and .h265_status, wait one probe cycle, or restart the app.

Soak FAIL with OK thermals: Usually encoder silence from disk pressure. Prune, --restart-only, re-probe recent_chunks, then re-soak.

Chunks on wrong disk: Confirm external mount and H265_RECORDINGS_DIR redirect before pruning—deletes target the active recording root.

Next steps ​

Operator depth ​

Live storage metrics, prune deltas, soak decisions, and harness evidence live in private operator handoff (not published) (private).