Alex
eb10a511f0
Turn the smoke test into a real runtime safety net. Waybar is now built with AddressSanitizer and exercised end to end in a headless compositor: - Tier 0: run everything under ASan; fail on ASan reports and Gtk/GLib criticals in the log (lib.sh assert_clean) - Tier 1: per-module render matrix (modules.sh) — each headless-safe module rendered in isolation - Tier 2: pointer interaction (interact.sh) — inject clicks via sway, assert on-click side effect and format-alt toggle; AT-SPI assertions (a11y.sh/.py) check module labels semantically instead of by pixels - Tier 3: layout matrix (positions.sh) — top/bottom/left + HiDPI scale 2; second compositor run under labwc Shared helpers extracted to lib.sh. Adds a workflow_dispatch `bless` input to regenerate the golden reference in one click. Trigger push only on master to avoid duplicate PR runs. AT-SPI and labwc steps are continue-on-error.
50 lines
2.4 KiB
Markdown
50 lines
2.4 KiB
Markdown
# Smoke / render / interaction tests
|
|
|
|
The [`smoke`](../../.github/workflows/smoke.yml) workflow builds Waybar **with
|
|
AddressSanitizer** and runs the real binary inside a **headless** compositor
|
|
(software-rendered, no GPU) to exercise it end to end. All helpers share
|
|
[`lib.sh`](lib.sh).
|
|
|
|
## What it checks
|
|
|
|
| Tier | Check | Script |
|
|
| --- | --- | --- |
|
|
| 0 | Launches, stays alive, and logs **no ASan / `Gtk-CRITICAL` / crash** — for every launch below | [`lib.sh`](lib.sh) `assert_clean` |
|
|
| 1 | **Renders** — deterministic config screenshotted with `grim`, bar strip not blank | [`run.sh`](run.sh) |
|
|
| 1 | **Per-module matrix** — each headless-safe module rendered in isolation | [`modules.sh`](modules.sh) |
|
|
| 2 | **Golden** — screenshot vs [`reference.png`](reference.png) (fuzz 8%, 800px) | workflow |
|
|
| 2 | **Interaction** — inject pointer clicks via sway; assert `on-click` side effect + `format-alt` toggle | [`interact.sh`](interact.sh) |
|
|
| 2 | **Accessibility** — assert module labels via the AT-SPI tree (robust to fonts) | [`a11y.sh`](a11y.sh) / [`a11y.py`](a11y.py) |
|
|
| 3 | **Layout matrix** — top / bottom / left (vertical) + HiDPI (scale 2) | [`positions.sh`](positions.sh) |
|
|
| 3 | **Second compositor** — same launch under `labwc` | workflow (`COMPOSITOR=labwc`) |
|
|
|
|
AddressSanitizer runs under *all* of the above, so any memory bug in the real
|
|
render path aborts the run. The interaction step is best-effort
|
|
(`continue-on-error`): a headless seat exposes no pointer capability, so injected
|
|
clicks may be dropped — hardening it needs a virtual pointer (uinput/wlr-virtual-pointer).
|
|
|
|
## Updating the reference image
|
|
|
|
`reference.png` must be generated by the CI environment (local rendering differs
|
|
from Ubuntu's). Two ways:
|
|
|
|
- **One click:** Actions → *smoke* → **Run workflow** with `bless = true`. The
|
|
job regenerates the screenshot and commits it to your branch.
|
|
- **Manual:** download the `waybar-smoke` artifact from a run and commit its
|
|
`bar.png` as `test/smoke/reference.png`.
|
|
|
|
Do this whenever an **intended** visual change trips the golden comparison.
|
|
|
|
## Run locally
|
|
|
|
On a machine with `sway`, `grim`, `imagemagick` and Waybar built:
|
|
|
|
```sh
|
|
export WAYBAR_BIN=$PWD/build/waybar
|
|
./test/smoke/run.sh test/smoke/config.jsonc test/smoke/style.css /tmp/bar.png
|
|
./test/smoke/modules.sh /tmp/module-shots
|
|
./test/smoke/interact.sh /tmp/interaction-shots
|
|
./test/smoke/positions.sh /tmp/layout-shots
|
|
dbus-run-session -- ./test/smoke/a11y.sh
|
|
```
|