The gitlab issue https://gitlab.freedesktop.org/gpiccoli/kdumpst/-/issues/19
was first reported by Saurabh, regarding GPU drivers getting included on
the kdump initrd. We've managed to improve the code a bit, but the solution
we proposed there was both wrong (for initcpio) *and* incomplete.
The user Ethanell reopened the issue, mentioning that even after the
patch that allegedly fixed the problem, they were getting amdgpu driver
included in the kdump initrd. That was the "wrong" portion of the fix:
in my naivety, I thought we could just remove the modules in the kdump
install hook. But...happens that initcpio works with a lazy approach: some
install hooks execute add_module(), and that signals to the later effective
inclusion of the modules by mkinitcpio; so in other words, the kdump
install hook was deleting no modules at all.
Special thanks to Foxboron and noclaf for the discussion about that on
IRC, which was very enlightening.
Now, the proper approach is simple: just to skip hooks that add the GPU
drivers, like plymouth and kms. Also took the opportunity to include the
microcode hook in the ban list, since we don't want kernel updating CPU
ucode in case of the microcode packages were updated (would be rare, but
why not save a bit of space dropping that?). Finally, I've removed the
GPU drivers delete code from the initcpio kdump install hook, since it
was a nop in the end.
That's it, right? No. Still there's the part two of my statement above:
the prior approach was wrong, but not only, it was also *incomplete*. That
is due to cases like Ethanell's: users that manually include GPU drivers
in their initramfs images, for things like crypto or splash screen. It's
also hard or better saying, *impossible*, to guess all hooks that add GPU
drivers to the initramfs.
So to cover manual inclusions or other hooks, we also add hereby extra
kdump command-line flags to avoid the DRM and some GFX drivers probe
during the kdump kernel boot. In order to not grow the drivers ban list
too much, my attempt was to add common in-tree GPU drivers, but also block
the DRM init call. Hopefully this time this is enough to disable GFX on kdump!
Fixes: 4767b8095f ("initramfs: Fix the removal of GPU drivers from the minimal initrd")
Reported-by: Ethanell (@flifloo)
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
In the description we have in the 00-default config file, both
Pstore memory configs are presented as if they're in MB. They
are in *bytes* indeed, so fix the braino.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
Currently kdumpst only allows a proper RAM buffer to be used as ramoops
memory area - this is the proper way of doing that, but there are cases
in which it's not trivial to have these buffers available:
(a) In case it's not there by default (due to kernel alignment / device-tree)
for example, there is no easy parameter to reserve such area right now
(using mem= is the way to go, but a bit complex to gather the exact size).
(b) The kernel parameter crashkernel= does something like this: reserve
a piece of memory not to be used by the kernel. It's meant for kdump, but
can be (ab)used for other purposes.
So, we hereby add a way for kdumpst to make use of crash reserved area
if the users wish so; the risks are exposed in the config file text.
Also, since we're changing this code, take the opportunity to improve the
sanitization of addresses from /proc/iomem and fix a corner case of empty
RANGE in the iomem parsing routine.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
Big but self-explanatory commit: rename the tool. The name choice was
kdumpst, since it's a tool to enable both kdump and pstore setting, also
it's a silly wordplay with the superlative of kdump, as in "kdumpest".
It's an invasive change (touches most of the files), but should
offer no functional change other than logging messages showing
kdumpst now, instead of kdump, and some filenames.
Notice it doesn't touch documentation, which will be done in
a subsequent commit.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
Currently a lot of our configuration settings and panic sysctls are
highly specific to SteamOS, so let's make them distro-agnostic, more
focused on generic HW / usage of a regular Arch Linux user. The affected
configs/sysctls are detailed below:
(a) Pstore memory settings: since on Steam Deck we have somewhat
pre-reserved RAM for pstore (~15M due to kernel memory alignment
rounding), makes sense to have a bit more of such memory effectively
available for pstore. In the general case though, likely users will
require to manually reserve it, so 4M of total memory with 1M buffer
seems more than enough to collect a dmesg, specially considering point
(e) below.
(b) The log storage folder was tuned for Deck, in which we have A/B
partitioning scheme and a persistent /home, but in general (following
standard kdump tools "on the market", like Debian's/Fedora's), /var
is used for that, so we follow the trend here.
(c) Grub file location was also special on SteamOS, so let's make
it follow the default /boot/grub/grub.cfg here.
(d) Kdump-specific tunings: the goal for people using kdump (not pstore!)
is usually to collect the vmcore of the panicked kernel to explore it,
using tools like crash/drgn. This is not the main goal on SteamOS, in
which we want to collect as much info we can get *on dmesg* and that's
it for most cases...
With that in mind, we needed "crash_kexec_post_notifiers" parameter
to dump more info on dmesg during a panic (a potentially problematic
parameter in some HWs BTW, but tested in depth on Deck) and we disabled
the vmcore saving by default as well. So, let's "revert" it here, having
vmcore capturing enabled by default and dropping the post_notifiers
parameter (see next point as well).
(e) About the sysctls, we are more aggressive on panicking on Deck
(like panic on soft lockups) and the goal is to collect the most info
we can on dmesg, so needed to enable panic_print to dump tasks and
whatnot on dmesg during a panic event. In the general case, people
that wish to have the most information as possible would go with
kdump, collecting vmcore, not with pstore collecting just the dmesg.
With that said, "reduce" panic_print here to only show memory info
and CPUs backtraces, and disable soft lockup panic.
(Also we cleared the file to drop mentioning the choices of *not*
panicking on hung tasks or RCU stalls).
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
Also document a bit better why some parameters are added and why
we remove huge pages parameters, for example.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
The kdump-load script is the only one that has more than a
single functionality, potentially being invoked in different ways.
With that in mind, just add a small usage/help for users'
information (thanks Emil for the suggestion).
While at it, changed the additional commands to be more friendly
and require exactly one command to be passed - changes summary:
s/initrd/create-initrd
s/clear/clear-initrd
s//load
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
This is somewhat a big refactor. The early design of kdump/pstore was
meant to support the A/B scheme of Steam Deck and a dracut-based initrd
only. In this scheme, we had a DEVNODE (like nvme partition or a devlink)
that represented the device node to be mounted early in boot when kdump
was in use. Also, we had a folder defined in the config file on top of
such dev node, and a temporary file used to "carry" the composition of
the full kdump path across boot time scripts. Yeah, kinda complex setup.
We are now moving to a more generic approach, and for that, the design
choice was a more convenient/simple one for the common cases, that
requires some operations to properly work on SteamOS dracut-based initrd.
Now we have only a single path on config file, which should be accessible
in a R/W filesystem by both scripts executed in the systemd service. No
devnode information or temp file is used anymore.
But with that, comes the need of discovering the proper devnode and base
folder for kdump'ing early in boot, from the initrd. Using the findmnt
tool we manage to derive all the necessary data during the initrd
preparation phase. Also, while at it we manage to fix an "inconsistency"
of our dracut initrd creation script: installkernel() should be responsible
to deal with DRM modules removal, not install().
On top of this (already big) change, now our dracut initrd excludes not
only amdgpu driver/FWs, but radeon, nvidia and i915 as well. And due to
our refactor of the mount point information (using findmnt to collect info
during dracut initrd creation), we also allow now arbitrary filesystem
drivers to be included, i.e., we don't hardcode/limit for ext4 only.
Again, mea culpa for not splitting this in multiple atomic/simple commits,
the burden to keep a pretty git log is starting to consume precious time.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
Tried here to give a brief reasoning on why we followed Debian;
or at least, have it explicitely mentioned in the comments, also
mentioning Ubuntu. These parameters make sense since Debian/Ubuntu
parameter approach is quite simple, hence mimic'ed here.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
As part of the upstreaming effort, we need to add some extra tunings
in the package. Especially related to GRUB autosetting and Pstore
memory settings:
(a) Currently the ramoops record size and memory amount are hardcoded
in the kdump-load script - we change it here, by having these settings
on the kdump config file;
(b) GRUB autosetting is pretty simple and everything is hardcoded.
We hereby add a bunch of configurable settings in the kdump conf file,
in order we can customize the GRUB handling, to make it work in both
Arch and SteamOS.
While at it, fixed some related comments and renamed some variables,
usually dropping KDUMP_ name when it applies to pstore as well.
Also, bumped the crashkernel memory from 192M to 256M - recent kernels
demand more memory, let's play safe.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
This is somewhat an intrusive change, but necessary if we want
to upstream the kdump tooling while allowing great extent of
customizations on SteamOS.
With this change, we have now a kdump.d folder on /usr/share,
that holds configuration files in the same way sysctl.d does.
In other words, we can easily override default settings by
just having more configuration files, which are sourced
following natural name sorting, i.e., we have now the concept
of config file precedence in kdump.
Our default config file is called 00-default, so we eventually
might have a 01-steamos e.g., with Deck's custom settings.
This is planned to other package though.
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
