SPECTER FLEET · LIVE REMOTE

Canonical report · updated every hour · dual SpecterS1Watch / SpecterS2Watch

mDNS + Specter viability — 60‑minute fleet soak

Run start 20260721T123147Z · interval 900s · planned samples 1 · proxy http://127.0.0.1:8788 · auto-refresh 120s · out sample-032-v6_5fps_rf1_720p

7
S1 good
8
S2 good
7
Fleet worst
Fleet average · sample 1/1
7.5/10
Worst lane: s1
S1 die 95.1°C · S2 die 85.2°C
Specter grade 80 · {'s1': 'S1 95° · 30% · CRITICAL', 's2': 'S2 85° · 39% · WARN'}
MQTT
connected
Pulse
healthy
Issues
['s1:thermal_critical', 's2:thermal_warn']
Latest hourly diff
Baseline sample (no prior hour).

Hourly history + diffs

Each row is one 60‑minute sample. Diffs compare grade dimensions, die temp (≥1°C), repair focus, and Specter issues vs the prior hour.

#UTCS1 grade / tempS2 grade / tempFleetDiff vs previous hour
12026-07-21T12:31:52Z7/10 good
95.1°C		8/10 good
85.2°C		7/10Baseline sample (no prior hour).

Latest grading checklist (1–10)

S1 · SpecterS1Watch · 7/10

DimensionScore 1–10Evidence
reachability10/10MQTT age 11.4s + SSH OK
thermal1/10die 95.1°C severe CRITICAL | CAUSE: Primary cause: Severe SoC heat (≥95°C) — die=95.1°C. Class=compute_encode_npu_heat_soak. Also: VIDEO_FIFO write timeouts (encode path pressure); Internal heat soak (high die, moderate skin); Recorder coar
recording7/10chunk present (writing flag thin): ngs/2026/07/21/chunk_1784637094_971.h265
rfdetr7/10RF-DETR path live precomp=None n_dets=0 persons=None
storage10/10used 26.0%
services8/10core+agent seen (no-oak-line)
host_mcp10/10health+recording host_mcp OK
specter5/10specter thermal critical flag ['s1:thermal_critical']

Thermal cause investigation

Primary: Severe SoC heat (≥95°C) · class=compute_encode_npu_heat_soak · die=95.1°C skin=65.0°C

Primary cause: Severe SoC heat (≥95°C) — die=95.1°C. Class=compute_encode_npu_heat_soak. Also: VIDEO_FIFO write timeouts (encode path pressure); Internal heat soak (high die, moderate skin); Recorder coarse_mode engaged (thermal protection path).

CauseWeightDetail
Severe SoC heat (≥95°C)1.0die=95.1°C
VIDEO_FIFO write timeouts (encode path pressure)0.9517 FIFO stall lines in recent ghost_core log — frame drops under thermal/load
Internal heat soak (high die, moderate skin)0.9die=95.1°C skin=65.0°C Δ=30.1°C — package retains heat under continuous NPU/encode
Recorder coarse_mode engaged (thermal protection path)0.7h265_status coarse_mode=1 — pipeline already throttling under heat policy
Repeated CRITICAL SoC log spam (sustained, not one-shot)0.73 CRITICAL temp lines in recent log — sustained overheat, not transient spike
Continuous justified H.265 encode (~30fps surveillance)0.65justified_ok=True est_fps=30.0 chunks_5m=5 — steady encode load
Elevated 1-min load average0.55load1=2.91 load5=2.38 — host/container CPU busy
RF-DETR / NPU inference path active alongside encode0.55rfdetr telemetry present n_dets/persons≈0 — DSP/NPU + encode stacks heat

Resolution / repair plans (score ≤ 6)

thermal (1/10)
die 95.1°C severe CRITICAL | CAUSE: Primary cause: Severe SoC heat (≥95°C) — die=95.1°C. Class=compute_encode_npu_heat_soak. Also: VIDEO_FIFO write timeouts (encode path pressure); Internal heat soak (high die, moderate skin); Recorder coarse_mode engaged (thermal protection path).
  1. Review ghost_core VIDEO_FIFO stalls; reduce concurrent host pulls; confirm justified encode rate.
  2. coarse_mode already on — expect reduced quality until die < ~85°C; do not force full-rate.
  3. NPU+encode heat stack: keep RF-DETR but avoid extra SSH/model jobs until cooler.
  4. Continuous 30fps H.265 is a baseline heat source on OAK-4 — plan cool windows for maintenance reboots.
  5. Heat soak: soft reboot (HITL) after brief idle cool; not PoE-first.
  6. Prune pre_event_thermal_*.h265 dumps to cut disk I/O heat contribution.
  7. Run log hygiene (shadow-log-hygiene.sh) — reduces log I/O heat slightly.
specter (5/10)
specter thermal critical flag ['s1:thermal_critical']
  1. Fix SSH targets so Specter probe is not probe_error.
  2. POST /fleet/specter/sync and re-read chips/issues.
  3. Confirm no false offline when SSH true (icmp_unreachable is soft).

S2 · SpecterS2Watch · 8/10

DimensionScore 1–10Evidence
reachability10/10MQTT age 20.5s + SSH OK
thermal4/10die 85.2°C high WARN | CAUSE: Primary cause: VIDEO_FIFO write timeouts (encode path pressure) — 17 FIFO stall lines in recent ghost_core log — frame drops under thermal/load. Class=encode_path_pressure. Also: Recorder coarse_mode engaged (t
recording7/10chunk present (writing flag thin): ngs/2026/07/21/chunk_1784637061_467.h265
rfdetr7/10RF-DETR path live precomp=True n_dets=0 persons=1
storage10/10used 39.0%
services9/10core+agent seen (no-oak-line)
host_mcp10/10health+recording host_mcp OK
specter6/10specter warn ['s2:thermal_warn']

Thermal cause investigation

Primary: VIDEO_FIFO write timeouts (encode path pressure) · class=encode_path_pressure · die=85.2°C skin=52.1°C

Primary cause: VIDEO_FIFO write timeouts (encode path pressure) — 17 FIFO stall lines in recent ghost_core log — frame drops under thermal/load. Class=encode_path_pressure. Also: Recorder coarse_mode engaged (thermal protection path); Repeated CRITICAL SoC log spam (sustained, not one-shot); Continuous justified H.265 encode (~30fps surveillance).

CauseWeightDetail
VIDEO_FIFO write timeouts (encode path pressure)0.9517 FIFO stall lines in recent ghost_core log — frame drops under thermal/load
Recorder coarse_mode engaged (thermal protection path)0.7h265_status coarse_mode=1 — pipeline already throttling under heat policy
Repeated CRITICAL SoC log spam (sustained, not one-shot)0.73 CRITICAL temp lines in recent log — sustained overheat, not transient spike
Continuous justified H.265 encode (~30fps surveillance)0.65justified_ok=True est_fps=30.0 chunks_5m=5 — steady encode load
SoC warn band (78–90°C)0.45die=85.2°C
Many pre_event_thermal dumps (disk I/O + prior heat events)0.3551 pre_event_thermal_*.h265 files counted — prune if storage/I/O pressure
RF-DETR / NPU inference path active alongside encode0.3rfdetr telemetry present n_dets/persons≈0 — DSP/NPU + encode stacks heat

Resolution / repair plans (score ≤ 6)

thermal (4/10)
die 85.2°C high WARN | CAUSE: Primary cause: VIDEO_FIFO write timeouts (encode path pressure) — 17 FIFO stall lines in recent ghost_core log — frame drops under thermal/load. Class=encode_path_pressure. Also: Recorder coarse_mode engaged (thermal protection path); Repeated CRITICAL SoC log spam (sustained, not one-shot); Continuous justified H.265 encode (~30fps surveillance).
  1. Review ghost_core VIDEO_FIFO stalls; reduce concurrent host pulls; confirm justified encode rate.
  2. coarse_mode already on — expect reduced quality until die < ~85°C; do not force full-rate.
  3. Continuous 30fps H.265 is a baseline heat source on OAK-4 — plan cool windows for maintenance reboots.
  4. Prune pre_event_thermal_*.h265 dumps to cut disk I/O heat contribution.
  5. Run log hygiene (shadow-log-hygiene.sh) — reduces log I/O heat slightly.
  6. Confirm coarse_mode / VIDEO_FIFO in ghost_core logs (expected under heat).
  7. Avoid stacked heavy pulls; schedule soft reboot in a cool window (HITL).
specter (6/10)
specter warn ['s2:thermal_warn']
  1. Fix SSH targets so Specter probe is not probe_error.
  2. POST /fleet/specter/sync and re-read chips/issues.
  3. Confirm no false offline when SSH true (icmp_unreachable is soft).

Hour #1 · 2026-07-21T12:31:52Z

Specter chips: {'s1': 'S1 95° · 30% · CRITICAL', 's2': 'S2 85° · 39% · WARN'} · issues: ['s1:thermal_critical', 's2:thermal_warn'] · mqtt=True pulse=healthy

Hourly diff: Baseline sample (no prior hour).

S1 · 7/10 good

DimensionScore 1–10Evidence
reachability10/10MQTT age 11.4s + SSH OK
thermal1/10die 95.1°C severe CRITICAL | CAUSE: Primary cause: Severe SoC heat (≥95°C) — die=95.1°C. Class=compute_encode_npu_heat_soak. Also: VIDEO_FIFO write timeouts (encode path pressure); Internal heat soak (high die, moderate skin); Recorder coar
recording7/10chunk present (writing flag thin): ngs/2026/07/21/chunk_1784637094_971.h265
rfdetr7/10RF-DETR path live precomp=None n_dets=0 persons=None
storage10/10used 26.0%
services8/10core+agent seen (no-oak-line)
host_mcp10/10health+recording host_mcp OK
specter5/10specter thermal critical flag ['s1:thermal_critical']

Thermal cause investigation

Primary: Severe SoC heat (≥95°C) · class=compute_encode_npu_heat_soak · die=95.1°C skin=65.0°C

Primary cause: Severe SoC heat (≥95°C) — die=95.1°C. Class=compute_encode_npu_heat_soak. Also: VIDEO_FIFO write timeouts (encode path pressure); Internal heat soak (high die, moderate skin); Recorder coarse_mode engaged (thermal protection path).

CauseWeightDetail
Severe SoC heat (≥95°C)1.0die=95.1°C
VIDEO_FIFO write timeouts (encode path pressure)0.9517 FIFO stall lines in recent ghost_core log — frame drops under thermal/load
Internal heat soak (high die, moderate skin)0.9die=95.1°C skin=65.0°C Δ=30.1°C — package retains heat under continuous NPU/encode
Recorder coarse_mode engaged (thermal protection path)0.7h265_status coarse_mode=1 — pipeline already throttling under heat policy
Repeated CRITICAL SoC log spam (sustained, not one-shot)0.73 CRITICAL temp lines in recent log — sustained overheat, not transient spike
Continuous justified H.265 encode (~30fps surveillance)0.65justified_ok=True est_fps=30.0 chunks_5m=5 — steady encode load
Elevated 1-min load average0.55load1=2.91 load5=2.38 — host/container CPU busy
RF-DETR / NPU inference path active alongside encode0.55rfdetr telemetry present n_dets/persons≈0 — DSP/NPU + encode stacks heat

Resolution / repair plans (score ≤ 6)

thermal (1/10)
die 95.1°C severe CRITICAL | CAUSE: Primary cause: Severe SoC heat (≥95°C) — die=95.1°C. Class=compute_encode_npu_heat_soak. Also: VIDEO_FIFO write timeouts (encode path pressure); Internal heat soak (high die, moderate skin); Recorder coarse_mode engaged (thermal protection path).
  1. Review ghost_core VIDEO_FIFO stalls; reduce concurrent host pulls; confirm justified encode rate.
  2. coarse_mode already on — expect reduced quality until die < ~85°C; do not force full-rate.
  3. NPU+encode heat stack: keep RF-DETR but avoid extra SSH/model jobs until cooler.
  4. Continuous 30fps H.265 is a baseline heat source on OAK-4 — plan cool windows for maintenance reboots.
  5. Heat soak: soft reboot (HITL) after brief idle cool; not PoE-first.
  6. Prune pre_event_thermal_*.h265 dumps to cut disk I/O heat contribution.
  7. Run log hygiene (shadow-log-hygiene.sh) — reduces log I/O heat slightly.
specter (5/10)
specter thermal critical flag ['s1:thermal_critical']
  1. Fix SSH targets so Specter probe is not probe_error.
  2. POST /fleet/specter/sync and re-read chips/issues.
  3. Confirm no false offline when SSH true (icmp_unreachable is soft).

S2 · 8/10 good

DimensionScore 1–10Evidence
reachability10/10MQTT age 20.5s + SSH OK
thermal4/10die 85.2°C high WARN | CAUSE: Primary cause: VIDEO_FIFO write timeouts (encode path pressure) — 17 FIFO stall lines in recent ghost_core log — frame drops under thermal/load. Class=encode_path_pressure. Also: Recorder coarse_mode engaged (t
recording7/10chunk present (writing flag thin): ngs/2026/07/21/chunk_1784637061_467.h265
rfdetr7/10RF-DETR path live precomp=True n_dets=0 persons=1
storage10/10used 39.0%
services9/10core+agent seen (no-oak-line)
host_mcp10/10health+recording host_mcp OK
specter6/10specter warn ['s2:thermal_warn']

Thermal cause investigation

Primary: VIDEO_FIFO write timeouts (encode path pressure) · class=encode_path_pressure · die=85.2°C skin=52.1°C

Primary cause: VIDEO_FIFO write timeouts (encode path pressure) — 17 FIFO stall lines in recent ghost_core log — frame drops under thermal/load. Class=encode_path_pressure. Also: Recorder coarse_mode engaged (thermal protection path); Repeated CRITICAL SoC log spam (sustained, not one-shot); Continuous justified H.265 encode (~30fps surveillance).

CauseWeightDetail
VIDEO_FIFO write timeouts (encode path pressure)0.9517 FIFO stall lines in recent ghost_core log — frame drops under thermal/load
Recorder coarse_mode engaged (thermal protection path)0.7h265_status coarse_mode=1 — pipeline already throttling under heat policy
Repeated CRITICAL SoC log spam (sustained, not one-shot)0.73 CRITICAL temp lines in recent log — sustained overheat, not transient spike
Continuous justified H.265 encode (~30fps surveillance)0.65justified_ok=True est_fps=30.0 chunks_5m=5 — steady encode load
SoC warn band (78–90°C)0.45die=85.2°C
Many pre_event_thermal dumps (disk I/O + prior heat events)0.3551 pre_event_thermal_*.h265 files counted — prune if storage/I/O pressure
RF-DETR / NPU inference path active alongside encode0.3rfdetr telemetry present n_dets/persons≈0 — DSP/NPU + encode stacks heat

Resolution / repair plans (score ≤ 6)

thermal (4/10)
die 85.2°C high WARN | CAUSE: Primary cause: VIDEO_FIFO write timeouts (encode path pressure) — 17 FIFO stall lines in recent ghost_core log — frame drops under thermal/load. Class=encode_path_pressure. Also: Recorder coarse_mode engaged (thermal protection path); Repeated CRITICAL SoC log spam (sustained, not one-shot); Continuous justified H.265 encode (~30fps surveillance).
  1. Review ghost_core VIDEO_FIFO stalls; reduce concurrent host pulls; confirm justified encode rate.
  2. coarse_mode already on — expect reduced quality until die < ~85°C; do not force full-rate.
  3. Continuous 30fps H.265 is a baseline heat source on OAK-4 — plan cool windows for maintenance reboots.
  4. Prune pre_event_thermal_*.h265 dumps to cut disk I/O heat contribution.
  5. Run log hygiene (shadow-log-hygiene.sh) — reduces log I/O heat slightly.
  6. Confirm coarse_mode / VIDEO_FIFO in ghost_core logs (expected under heat).
  7. Avoid stacked heavy pulls; schedule soft reboot in a cool window (HITL).
specter (6/10)
specter warn ['s2:thermal_warn']
  1. Fix SSH targets so Specter probe is not probe_error.
  2. POST /fleet/specter/sync and re-read chips/issues.
  3. Confirm no false offline when SSH true (icmp_unreachable is soft).

JSONL: /Users/deanbarrett/gp-firmware/cost-sentinel/data/thermal-campaign-8h-20260721T043521Z/sample-032-v6_5fps_rf1_720p/samples.jsonl · summary: /Users/deanbarrett/gp-firmware/cost-sentinel/data/thermal-campaign-8h-20260721T043521Z/sample-032-v6_5fps_rf1_720p/SUMMARY.md · viability JSON: /Users/deanbarrett/gp-firmware/cost-sentinel/data/mdns-specter-viability-20260718T163941Z.hourly.json · scale 1=critical 10=excellent · thermal investigation uses VIDEO_FIFO / coarse_mode / H.265 / RF-DETR / log spam weights