The smoke test only exercised steady-state rendering and killed waybar with SIGTERM, so whole crash classes were invisible: exit-time use-after-free (#5182), module state transitions (#5183) and hardware-backed modules (#5179). Add tiers that hit those paths under ASan (+ _GLIBCXX_ASSERTIONS): - lib.sh: assert_clean_exit (SIGINT teardown -> checks segfault/abort + ASan report emitted during destruction), output hotplug helpers, signal/reload, WAYBAR_WRAP hook, opt-in leak detection. - lifecycle.sh (gating): clean exit, runtime output hotplug (Bar/module destroy), toggle/reload churn, fast-interval teardown race (#5182 class). - fuzz.sh (gating): pathological custom-backend output (empty, nonzero exit, invalid JSON, huge, non-UTF8, empty format). - coverage.sh (gating): every Factory module must be classified for smoke coverage; a new unlisted module fails the job (#5179 slipped through). - modules.sh: also render the whole matrix inside a group; SIGINT teardown per module instead of SIGTERM. - state.sh (best-effort): real mpd driven through an empty queue (#5183) + stop/clear; pulseaudio + slider on a null sink. - hardware.sh (best-effort): backlight, backlight/slider (#5179) and battery via umockdev. - leakcheck.sh (report-only): clean-exit run under LSan.
157 lines
6.1 KiB
YAML
157 lines
6.1 KiB
YAML
name: smoke
|
|
|
|
# Launch the real Waybar binary (built with AddressSanitizer) inside a headless
|
|
# compositor and exercise it: render, per-module matrix, pointer interaction,
|
|
# accessibility-tree assertions, and a layout matrix. Complements the unit
|
|
# tests and build jobs, which never actually run the bar.
|
|
|
|
on:
|
|
push:
|
|
branches: [master]
|
|
pull_request:
|
|
workflow_dispatch:
|
|
inputs:
|
|
bless:
|
|
description: "Regenerate and commit the golden reference screenshot"
|
|
type: boolean
|
|
default: false
|
|
|
|
permissions:
|
|
contents: write
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
WAYBAR_BIN: ${{ github.workspace }}/build/waybar
|
|
|
|
jobs:
|
|
smoke:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install --no-install-recommends -y \
|
|
sway labwc grim imagemagick fonts-dejavu-core jq \
|
|
at-spi2-core python3-pyatspi dbus-x11 \
|
|
mpd mpc pulseaudio pulseaudio-utils umockdev \
|
|
meson ninja-build g++ pkg-config scdoc wayland-protocols \
|
|
gobject-introspection libdbusmenu-gtk3-dev libevdev-dev libfmt-dev \
|
|
libgirepository1.0-dev libgtk-3-dev libgtkmm-3.0-dev libinput-dev \
|
|
libjsoncpp-dev libmpdclient-dev libnl-3-dev libnl-genl-3-dev \
|
|
libpulse-dev libsigc++-2.0-dev libspdlog-dev libwayland-dev \
|
|
upower libxkbregistry-dev libxkbcommon-dev
|
|
|
|
# Cheap static gate (source-only): every Factory module must be classified
|
|
# for smoke coverage. Runs before the build so it fails fast.
|
|
- name: Module coverage tripwire
|
|
run: ./test/smoke/coverage.sh
|
|
|
|
- name: Build Waybar with AddressSanitizer
|
|
run: |
|
|
# _GLIBCXX_ASSERTIONS turns libstdc++ container bounds/precondition
|
|
# violations (e.g. out-of-range vector access) into aborts under ASan.
|
|
meson setup build -Dman-pages=disabled -Db_sanitize=address \
|
|
-Dcpp_args=-D_GLIBCXX_ASSERTIONS
|
|
ninja -C build
|
|
|
|
# --- Tier 0 + 2: render + sanitizer/critical gate ---
|
|
- name: Render deterministic config
|
|
run: ./test/smoke/run.sh test/smoke/config.jsonc test/smoke/style.css bar.png
|
|
|
|
- name: Check the bar is not blank
|
|
run: |
|
|
sd=$(convert bar.png -crop 1920x30+0+0 +repage -colorspace Gray \
|
|
-format '%[fx:standard_deviation]' info:)
|
|
echo "bar strip stddev = $sd"
|
|
awk "BEGIN{ exit !($sd > 0.01) }" || { echo "::error::bar looks blank"; exit 1; }
|
|
|
|
# --- Manual re-bless of the reference image ---
|
|
- name: Bless reference screenshot
|
|
if: github.event_name == 'workflow_dispatch' && inputs.bless
|
|
run: |
|
|
cp bar.png test/smoke/reference.png
|
|
git config user.name "github-actions[bot]"
|
|
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
|
git add test/smoke/reference.png
|
|
git commit -m "ci(smoke): re-bless reference screenshot [skip ci]"
|
|
git push
|
|
|
|
- name: Golden comparison
|
|
run: |
|
|
if [ -f test/smoke/reference.png ]; then
|
|
ae=$(compare -metric AE -fuzz 8% test/smoke/reference.png bar.png diff.png 2>&1 || true)
|
|
ae=$(printf '%s' "$ae" | grep -oE '^[0-9]+' || echo 0)
|
|
echo "differing pixels (fuzz 8%): $ae"
|
|
[ "$ae" -gt 800 ] && { echo "::error::visual regression: $ae px (see diff.png artifact)"; exit 1; } || echo "✓ within tolerance"
|
|
else
|
|
echo "::warning::No reference image; run this workflow via 'Run workflow' with bless=true to create it."
|
|
fi
|
|
|
|
# --- Tier 1: per-module matrix (standalone + grouped) ---
|
|
- name: Per-module render matrix
|
|
run: ./test/smoke/modules.sh module-shots
|
|
|
|
# --- Tier 4: lifecycle / teardown (exit, output hotplug, reload) ---
|
|
# Runs the C++ destructor path a plain kill never touches (#5182).
|
|
- name: Lifecycle / teardown
|
|
run: ./test/smoke/lifecycle.sh
|
|
|
|
# --- Tier 4: custom-backend fuzz (pathological exec output) ---
|
|
- name: Custom-backend fuzz
|
|
run: ./test/smoke/fuzz.sh
|
|
|
|
# --- Tier 5: state transitions with real daemons (mpd/pulseaudio) ---
|
|
# Best-effort: daemon setup varies by runner; keep non-blocking for now.
|
|
- name: State transitions (mpd / pulseaudio)
|
|
continue-on-error: true
|
|
run: ./test/smoke/state.sh
|
|
|
|
# --- Tier 6: fake hardware via umockdev (backlight/slider, battery) ---
|
|
# Best-effort: the mock device descriptions may need per-distro tuning.
|
|
- name: Fake hardware (umockdev)
|
|
continue-on-error: true
|
|
run: ./test/smoke/hardware.sh
|
|
|
|
# --- Leak check (report-only) — we fixed several leaks recently ---
|
|
- name: Leak check (LSan, report-only)
|
|
continue-on-error: true
|
|
run: ./test/smoke/leakcheck.sh
|
|
|
|
# --- Tier 2: pointer interaction (best-effort) ---
|
|
# Headless seats may not expose a pointer capability, so injected clicks
|
|
# can be dropped; keep this non-blocking while it is hardened.
|
|
- name: Interaction (click + format-alt)
|
|
continue-on-error: true
|
|
run: ./test/smoke/interact.sh interaction-shots
|
|
|
|
# --- Tier 2: accessibility assertions ---
|
|
- name: Accessibility (AT-SPI) assertions
|
|
run: dbus-run-session -- ./test/smoke/a11y.sh
|
|
|
|
# --- Tier 3: layout matrix (positions + HiDPI) ---
|
|
- name: Layout matrix (top/bottom/left + scale 2)
|
|
run: ./test/smoke/positions.sh layout-shots
|
|
|
|
# --- Tier 3: second compositor ---
|
|
- name: Launch under labwc
|
|
run: COMPOSITOR=labwc ./test/smoke/run.sh test/smoke/config.jsonc test/smoke/style.css labwc.png
|
|
|
|
- name: Upload screenshots & diffs
|
|
uses: actions/upload-artifact@v4
|
|
if: always()
|
|
with:
|
|
name: waybar-smoke
|
|
path: |
|
|
bar.png
|
|
diff.png
|
|
labwc.png
|
|
module-shots/
|
|
interaction-shots/
|
|
layout-shots/
|
|
if-no-files-found: warn
|