Appearance
- Audience: Labs that add a GPU appliance beside the Mac Studio desk
- Goal: Run LLM API only on Spark; keep Phantom Vision UI/MQTT on Mac
- Type: Reference
TIP
Mac Studio remains the recommended primary host for Phantom Vision (:5173 / :8788 / :1883) and operator tooling. A DGX Spark (or similar aarch64 GPU box) is an optional second appliance that serves OpenAI-compatible inference only.
Gemma on host(s) only — never on Shadows.
Stack summary
| Need | Stack |
|---|---|
| OS | Ubuntu 24.04 aarch64 (DGX baseline) |
| GPU | CUDA / drivers as NVIDIA ships for the platform |
| Runtime | Docker + vLLM / NIM |
| Pairing C | Gemma 4 26B-class NVFP4 where available |
| Pairing D | Nemotron Super / Nano after quality eval |
| API | Often OpenAI-compat on :8000 (firewall carefully) |
Split stack
| Box | Role |
|---|---|
| Mac Studio | Phantom Vision UI, MQTT, vms-proxy, Cline, Hermes, SSH to fleet · ≥1 TB storage |
| Spark | LLM HTTP API only; no public bind without auth |
Ports to coordinate
- Spark listen: 8000/tcp (or site standard) for vLLM
- Mac egress to Spark: that port only
- Shadows still publish MQTT to Mac :1883, not Spark