Zander671/Waybar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ci(smoke): cover teardown, state transitions and fake hardware

Browse Source 
The smoke test only exercised steady-state rendering and killed waybar with
SIGTERM, so whole crash classes were invisible: exit-time use-after-free
(#5182), module state transitions (#5183) and hardware-backed modules (#5179).
Add tiers that hit those paths under ASan (+ _GLIBCXX_ASSERTIONS):

- lib.sh: assert_clean_exit (SIGINT teardown -> checks segfault/abort + ASan
  report emitted during destruction), output hotplug helpers, signal/reload,
  WAYBAR_WRAP hook, opt-in leak detection.
- lifecycle.sh (gating): clean exit, runtime output hotplug (Bar/module
  destroy), toggle/reload churn, fast-interval teardown race (#5182 class).
- fuzz.sh (gating): pathological custom-backend output (empty, nonzero exit,
  invalid JSON, huge, non-UTF8, empty format).
- coverage.sh (gating): every Factory module must be classified for smoke
  coverage; a new unlisted module fails the job (#5179 slipped through).
- modules.sh: also render the whole matrix inside a group; SIGINT teardown per
  module instead of SIGTERM.
- state.sh (best-effort): real mpd driven through an empty queue (#5183) +
  stop/clear; pulseaudio + slider on a null sink.
- hardware.sh (best-effort): backlight, backlight/slider (#5179) and battery
  via umockdev.
- leakcheck.sh (report-only): clean-exit run under LSan.
This commit is contained in:
Alex
2026-07-05 22:21:46 +02:00
parent 445e2aec1f
commit 9a10949938
10 changed files with 558 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/smoke/README.md
+26 -6
View File
@@ -10,18 +10,31 @@ AddressSanitizer** and runs the real binary inside a **headless** compositor
| Tier | Check | Script |
| --- | --- | --- |
| 0 | Launches, stays alive, and logs **no ASan / `Gtk-CRITICAL` / crash** — for every launch below | [`lib.sh`](lib.sh) `assert_clean` |
| — | **Coverage tripwire** — every `Factory` module must be classified (source-only, runs before the build) | [`coverage.sh`](coverage.sh) |
| 1 | **Renders** — deterministic config screenshotted with `grim`, bar strip not blank | [`run.sh`](run.sh) |
| 1 | **Per-module matrix** — each headless-safe module rendered in isolation | [`modules.sh`](modules.sh) |
| 1 | **Per-module matrix** — each headless-safe module rendered standalone **and inside a group**, each torn down via SIGINT | [`modules.sh`](modules.sh) |
| 2 | **Golden** — screenshot vs [`reference.png`](reference.png) (fuzz 8%, 800px) | workflow |
| 2 | **Interaction** — inject pointer clicks via sway; assert `on-click` side effect + `format-alt` toggle | [`interact.sh`](interact.sh) |
| 2 | **Accessibility** — assert module labels via the AT-SPI tree (robust to fonts) | [`a11y.sh`](a11y.sh) / [`a11y.py`](a11y.py) |
| 3 | **Layout matrix** — top / bottom / left (vertical) + HiDPI (scale 2) | [`positions.sh`](positions.sh) |
| 3 | **Second compositor** — same launch under `labwc` | workflow (`COMPOSITOR=labwc`) |
| 4 | **Lifecycle / teardown** — clean SIGINT exit, runtime **output hotplug**, toggle + reload churn, fast-interval teardown (the C++ destructor path a `kill` never runs, #5182) | [`lifecycle.sh`](lifecycle.sh) |
| 4 | **Custom-backend fuzz** — empty / non-zero-exit / invalid-JSON / huge / non-UTF8 / empty-format exec output | [`fuzz.sh`](fuzz.sh) |
| 5 | **State transitions** *(best-effort)* — real `mpd` driven through an **empty queue** (#5183) + stop/clear, `pulseaudio` + slider on a null sink | [`state.sh`](state.sh) |
| 6 | **Fake hardware** *(best-effort)* — `backlight`, `backlight/slider` (#5179) and `battery` via **umockdev** | [`hardware.sh`](hardware.sh) |
| — | **Leak check** *(report-only)* — clean-exit run under LSan | [`leakcheck.sh`](leakcheck.sh) |
AddressSanitizer runs under *all* of the above, so any memory bug in the real
render path aborts the run. The interaction step is best-effort
(`continue-on-error`): a headless seat exposes no pointer capability, so injected
clicks may be dropped — hardening it needs a virtual pointer (uinput/wlr-virtual-pointer).
AddressSanitizer (plus `-D_GLIBCXX_ASSERTIONS`) runs under *all* of the above, so
any memory bug in the real render/teardown path aborts the run. Tiers 5–6 and the
leak check are best-effort (`continue-on-error`): they depend on daemon setup /
mock devices that vary by runner. The interaction step is likewise best-effort — a
headless seat exposes no pointer capability, so injected clicks may be dropped
(hardening it needs a virtual pointer, uinput/wlr-virtual-pointer).
Because the teardown path (`~Bar` → window unmap → `toggleSuspend`) is where the
exit-time crashes live, tests assert on a **clean SIGINT exit** via
`smoke::assert_clean_exit` rather than killing with SIGTERM, and re-check the log
for sanitizer reports emitted *during* destruction.
## Updating the reference image
@@ -37,13 +50,20 @@ Do this whenever an **intended** visual change trips the golden comparison.
## Run locally
On a machine with `sway`, `grim`, `imagemagick` and Waybar built:
On a machine with `sway`, `grim`, `imagemagick`, `jq` and Waybar built:
```sh
export WAYBAR_BIN=$PWD/build/waybar
./test/smoke/coverage.sh # no compositor needed
./test/smoke/run.sh test/smoke/config.jsonc test/smoke/style.css /tmp/bar.png
./test/smoke/modules.sh /tmp/module-shots
./test/smoke/lifecycle.sh
./test/smoke/fuzz.sh
./test/smoke/interact.sh /tmp/interaction-shots
./test/smoke/positions.sh /tmp/layout-shots
dbus-run-session -- ./test/smoke/a11y.sh
# best-effort (need extra daemons / tools):
./test/smoke/state.sh # needs mpd + mpc, pulseaudio
./test/smoke/hardware.sh # needs umockdev
./test/smoke/leakcheck.sh
```