The first run went green but the best-effort tiers silently didn't exercise
anything:
- state.sh: the mpd `audio_output { ... }` block was on one line, which mpd
rejects ("Unknown tokens after '{'"), so mpd never started and the mpd
sub-tier (the #5183 empty-queue repro) skipped. Split the block across lines.
Also move teardown into a cleanup() with `|| true` so a dead-pid kill under
`set -e` no longer makes the step exit 1.
- hardware.sh: umockdev-run's LD_PRELOAD lands ahead of the linked-in ASan
runtime, so ASan aborted before main() ("ASan runtime does not come first")
and waybar never started -- the backlight/slider (#5179) path wasn't tested.
Set verify_asan_link_order=0 so the instrumented binary runs under the preload.
- leakcheck.sh: abort_on_error=1 turned LSan's exit report into a core dump;
force abort_on_error=0 for this report-only tier.
- lib.sh: assert_clean now also flags "ASan runtime does not come first" so a
future preload/link-order regression fails loudly instead of passing.
The smoke test only exercised steady-state rendering and killed waybar with
SIGTERM, so whole crash classes were invisible: exit-time use-after-free
(#5182), module state transitions (#5183) and hardware-backed modules (#5179).
Add tiers that hit those paths under ASan (+ _GLIBCXX_ASSERTIONS):
- lib.sh: assert_clean_exit (SIGINT teardown -> checks segfault/abort + ASan
report emitted during destruction), output hotplug helpers, signal/reload,
WAYBAR_WRAP hook, opt-in leak detection.
- lifecycle.sh (gating): clean exit, runtime output hotplug (Bar/module
destroy), toggle/reload churn, fast-interval teardown race (#5182 class).
- fuzz.sh (gating): pathological custom-backend output (empty, nonzero exit,
invalid JSON, huge, non-UTF8, empty format).
- coverage.sh (gating): every Factory module must be classified for smoke
coverage; a new unlisted module fails the job (#5179 slipped through).
- modules.sh: also render the whole matrix inside a group; SIGINT teardown per
module instead of SIGTERM.
- state.sh (best-effort): real mpd driven through an empty queue (#5183) +
stop/clear; pulseaudio + slider on a null sink.
- hardware.sh (best-effort): backlight, backlight/slider (#5179) and battery
via umockdev.
- leakcheck.sh (report-only): clean-exit run under LSan.
Turn the smoke test into a real runtime safety net. Waybar is now built with
AddressSanitizer and exercised end to end in a headless compositor:
- Tier 0: run everything under ASan; fail on ASan reports and Gtk/GLib
criticals in the log (lib.sh assert_clean)
- Tier 1: per-module render matrix (modules.sh) — each headless-safe module
rendered in isolation
- Tier 2: pointer interaction (interact.sh) — inject clicks via sway, assert
on-click side effect and format-alt toggle; AT-SPI assertions (a11y.sh/.py)
check module labels semantically instead of by pixels
- Tier 3: layout matrix (positions.sh) — top/bottom/left + HiDPI scale 2;
second compositor run under labwc
Shared helpers extracted to lib.sh. Adds a workflow_dispatch `bless` input to
regenerate the golden reference in one click. Trigger push only on master to
avoid duplicate PR runs. AT-SPI and labwc steps are continue-on-error.
Blessed from the CI artifact of the passing headless run (Waybar / CI smoke
test / 12:34 on a pinned DejaVu Sans bar). Regenerate from a new artifact when
an intended visual change trips the comparison — see test/smoke/README.md.
The static custom modules used `exec: echo` (empty output), so the modules
rendered nothing and the bar strip was blank. Have exec print the label text
and use the default `{}` format.
Launch the real Waybar binary inside a headless, software-rendered sway
compositor and verify it actually runs and renders — something the unit
tests and build jobs never do.
- test/smoke/run.sh: boots sway (WLR_BACKENDS=headless, pixman), starts
waybar, asserts it stays alive with no fatal log, optionally grabs a
screenshot with grim
- level 1: real modules (clock/cpu/memory/disk) load without crashing
- level 2: deterministic config is screenshotted and checked to be non-blank
- level 3: screenshot compared to test/smoke/reference.png (fuzz 8%,
800px tolerance); screenshot + diff uploaded as artifacts
- .github/workflows/smoke.yml runs it on push/PR
The reference image must be blessed from a CI artifact (see test/smoke/README.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Alex