About this article

• Audience: Contributors running unit, harness, soak, and end-to-end checks before a promote or docs release
• Goal: After reading, you can run each verification layer in order and interpret PASS, INCOMPLETE, and FAIL outcomes
• Type: How-to

Summary

Ghost Protocol verification stacks in layers: fast unit tests, live MCP harness scenarios, hour-scale soak gates, Phantom readiness modules, and host-side Playwright visuals. Run from lightest to heaviest. Keep the Phantom stack up for anything that probes /fleet/* or MQTT through the proxy.

Prerequisites

• Repo at [repo-root] with Python 3 and Node 22 for e2e
• export PHANTOM_PROXY=https://<phantom-proxy>:8788
• Phantom stack up: cd cost-sentinel && npm run stack:up
• SSH to shadows with YOUR_DEVICE_PASSWORD
• Soaks and automation for orchestrator flags

1. Unit tests

Run targeted unit modules before long soaks. Core areas include device identity and bus contract, Hermes mesh helpers, MCP server keys, spatial fusion for layer3 depth, and fleet contract schema validation.

cd [repo-root]/oak-vms-firmware
python3 -m pytest tests/ -q --tb=short -k "identity or hermes or mcp or fusion or contract"

The orchestrator and readiness agent can wire selected unit gates before and after live probes in batch mode.

2. Live MCP harness

The device harness exercises mass-transit safety scenarios over live MCP with repeats. Use it after encoder and MQTT baselines look healthy.

# on-device or via harness container per your install
python3 test_harness.py --mode live-mcp --scenario all --repeats 3

Treat sub-threshold scenario pass rates as a release blocker for detection authority tiers. Precomputed RF-DETR output on device is the authoritative detection path during harness runs.

3. Soak gate (thermal + encoder + storage)

The soak gate samples once per minute for a configured duration. It watches H.265 chunk growth, die temperature, and encoder fail streaks.

cd [repo-root]/oak-vms-firmware
bash scripts/soak_gate.sh

For automated apply → verify → soak → decide cycles, use the orchestrator:

python3 tools/v3_shadow_orchestrator.py \
  --shadow <device-ip> \
  --layer layer2_tracker \
  --duration 60 \
  --prune-first --visual --tap --update-handoff

Add --hermes when validating mesh self-heal scenarios. Start with --dry-run on batch matrices to confirm scenario names before live hardware.

4. Phantom readiness modules

The readiness agent runs interleaved modules over roughly five minutes. It is the gate before long batch work when Hermes or federation changed.

python3 tools/phantom_vms_readiness_agent.py \
  --proxy-url "$PHANTOM_PROXY" \
  --hermes-validate \
  --modules device_identity_fp,peer_sidecar,hermes_mesh,mcp_health,cross_a2a_federation,depth_fusion \
  --tap \
  --report [operator-artifacts]/readiness-report.md

Transient FAILs right after a layer apply are common—wait for container stabilization, then re-run. Proxy equivalents: GET /fleet/readiness and POST /fleet/readiness/remediate.

5. Visual and Playwright e2e

Host-side visuals capture proxy snapshots, MQTT metrics overlays, and full-page UI states. Playwright specs require the stack up.

cd [repo-root]/cost-sentinel
npm run stack:up
npx playwright test

Annotated soak visuals can also run via visual_soak_gate.py inside an orchestrator --visual cycle. Store reports under [operator-artifacts]—not in public doc pages.

6. Production grade script

After soak and contract checks pass, run the production grading helper for a letter summary:

bash cost-sentinel/scripts/grade-production-soak.sh

Interpret failures against Common issues and Production grading overview. Formal rubric tables remain in operator-controlled grading sources.

Interpreting decisions

Decision	Meaning
PASS	Soak samples, encoder growth, and thermal checks met policy
INCOMPLETE	Run ended without final markers—review watcher logs and post-probe
FAIL	Encoder stall, thermal breach, or storage CRITICAL blocked promote

When storage is CRITICAL, prune recordings before retrying. When MQTT is stale, restart the stack and edge app before another harness or soak.

Next steps

• Batch scenario catalog: Soaks and automation
• Failure playbooks: Common issues
• Release gates summary: Production grading overview

Operator depth

Live fleet state and harness evidence live in private operator handoff (not published) (private).