kdump_load.sh, kdump.etc: Introduce GRUB auto-setting support

With this addition, kdump-steamos is now capable of editing grub.cfg
to automatically add the required kdump parameters, in case kdump
is used. If Pstore ends-up being used and the grub.cfg was mofified
by kdump-steamos automatically, we're also able to undo the change
and save users from memory waste.

Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>

PKGBUILD

@@ -25,12 +25,12 @@ source=('20-kdump-steamos.conf'
 
 sha256sums=('dbedff54addfb5dce51614c73df04c90fca9f27d0d3a690243259ccbbfcca07c'
             '2514f79a496f76af847e262eadd55a5c2f8d95375cc513efa8cadd4cd98fe1d2'
-            'fca89993e6ca76ec14f9a8e6761bec521903e2743ab9e05bfe2cd2b77e5e89f9'
+            'd33778b5ba356f96ae2b78998deaeb7059260b8e2aa4253be3f7da4e9877f94c'
-            'c1feeecf8fc3befbe241e7c78665af3b20258b2593393f91bf300c237623b2de'
+            '7956c6cf1ce5c5e9aaf573ceee8c6ac2a0cad7e0cfa8f5b21adaa20f9f3db929'
             '06b38bd9f09da5fb22a765b6f1945fc349cc5f9d13cd32c9218b9b60b40a9010'
             '12a9124b907f208471ba7aaac0f3261cbbd34a168cce3260fa9e7793994beebd'
             '26bc2b64af0d468f050c0e0dd9e2053176d56886edad9146bc495797bf2c5810'
-            '97d055f9e6e66cc4168767f5b51ff8e728c6935d270a997cf5b082ddd2d30634'
+            'd33778b5ba356f96ae2b78998deaeb7059260b8e2aa4253be3f7da4e9877f94c'
             'b1a4a0f77cae137d3f5155a0fcc4b9d1e1b7ffd0089adb06b6bb21e71c751e27'
             'cbb207ecc0f6bacefbeed41f0d4910daac6500ac2345366e1f95f09a7653c65a')
 

README.md

@@ -51,9 +51,9 @@
 #  can choose Kdump instead of Pstore (USE_PSTORE_RAM), and if using Kdump,
 #  collect the full vmcore (FULL_COREDUMP). The vmcore is not stored in the
 #  ZIP file, but it's saved in "/home/.steamos/offload/var/kdump/crash/".
-#  NOTICE that, if Kdump is used instead of Pstore, the following flags are
+#  NOTICE that, if Kdump is used instead of Pstore (either per user's choice
-#  needed in GRUB cmdline: "crashkernel=192M crash_kexec_post_notifiers" and
+#  or due to some failure in Pstore), a reboot is necessary before kdump is
-#  a regular reboot is necessary.
+#  usable, in order to effectively reserve crashkernel memory.
 #
 #  6. Error and succeeding messages are sent to systemd journal, so running
 #  'journalctl | grep kdump' would hopefully bring some information. Also,
@@ -64,14 +64,14 @@
 #  ##############################  DETAILS  ##################################
 #  CAVEATS / INSTRUCTIONS
 #  ###########################################################################
-#  (a) Currently, we don't automatically edit GRUB config; see TODO (1) below.
+#  (a) We automatically edit GRUB config in case Pstore fails or if the user's
-#  This is not required if Pstore is used (which is the default).
+#  choice is to use Kdump. But it requires one reboot in order the crashkernel
+#  memory is effectively reserved by kernel.
 #
-#  In case Kdump is used, boot-time reserved memory is required, check step (5)
+#  In case Kdump is used, the crashkernel necessary memory was empirically
-#  in the HOW-TO above. The memory amount was empirically determined - setting
+#  determined; setting 144M wasn't enough, 160M is unstable, so 192M seems
-#  144M wasn't enough and 160M is unstable, so 192M seems good enough. This
+#  good enough. This amount might change in future kernel versions, requiring
-#  amount might change in future kernel versions, requiring tests using the
+#  tests using the approach suggested in the step (4) above.
-#  approach suggested in the step (4) above.
 #
 #  (b) The kdump-steamos package requires a RW rootfs in case it's not currently
 #  embedded in your image. Users can make use of 'tune2fs' or 'steamos-readonly'
@@ -95,32 +95,29 @@
 #
 #  TODOs
 #  ###########################################################################
-#  (1) We'd like to be able to automatically edit GRUB and recreate its config
+#  (1) Would be interesting to have a clean-up mechanism, to keep up to N most
-#  file - implementation tests are ongoing.
-#
-#  (2) Would be interesting to have a clean-up mechanism, to keep up to N most
 #  recent ZIP log files, instead of keeping all of them forever.
 #
-#  (3) Hopefully we can fix/prevent the unnecessary re-creation of all initramfs
+#  (2) Hopefully we can fix/prevent the unnecessary re-creation of all initramfs
 #  images - it happens due to our package installing files on directory
 #  "/usr/lib/dracut/modules.d" which triggers the unfortunate initramfs rebuild.
 #
-#  (4) We have a "fragile" way of determining a mount point required for Kdump;
+#  (3) We have a "fragile" way of determining a mount point required for Kdump;
 #  this is something to improve maybe, in order to make the Kdump more reliable.
 #  Also in the list of fragile things, VDF parsing is...complicated. Something
 #  that would be nice to improve as well.
 #
-#  (5) Pstore ramoops back-end has some limitations that we're discussing with
+#  (4) Pstore ramoops back-end has some limitations that we're discussing with
 #  the kernel community - right now we can only collect ONE dmesg and its
 #  size is truncated on "record_size" bytes, not allowing a file split like
 #  efi-pstore; thankfully we still can collect 2MiB dmesg, but hopefully we can
 #  improve that upstream.
 #
-#  (6) Add a more reliable reboot mechanism - we had seen issues in the past
+#  (5) Add a more reliable reboot mechanism - we had seen issues in the past
 #  with "reboot -f", and relying in sysrq reboot as a quirk managed to be a safe
 #  option, so this is something to think about. Should be easy to implement.
 #
-#  (7) Maybe a good idea would be to allow creating the minimum image for any
+#  (6) Maybe a good idea would be to allow creating the minimum image for any
 #  specified kernel, not only for the running one (which is what we do now).
 #  Low-priority idea, easy to implement.
 #

kdump.etc

@@ -18,15 +18,19 @@ MOUNT_DEVNODE="nvme0n1p8"
 KDUMP_FOLDER="/.steamos/offload/var/kdump"
 KDUMP_MNT="/tmp/kdump.mnt"
 
-#  Dump controlling settings - for now we don't have network/iscsi dumps, only
+#  Kdump controlling settings - for now we don't have network/iscsi dumps, only
 #  local storage dumps. If FULL_COREDUMP is !=0, we collect a full compressed
 #  vmcore, which might require a lot of disk space. The *_CMD settings refer
-#  to tunnings on makedumpfile - we rely on zstd compression and maximum page
+#  to tunings on makedumpfile - we rely on zstd compression and maximum page
-#  exclusion for the full vmcore, mimic'ing Debian kdump.
+#  exclusion for the full vmcore, mimic'ing Debian kdump. Finally, GRUB_AUTOSET
+#  determines if kdump-steamos should try to automatically set grub.cfg in
+#  order to reserve memory for kdump (if needed, i.e., when pstore is not in
+#  use) - keep it as '1' for enabling the mechanism (this is the default).
 
 FULL_COREDUMP=0
 MAKEDUMPFILE_COREDUMP_CMD="-z -d 31"
 MAKEDUMPFILE_DMESG_CMD="--dump-dmesg"
+GRUB_AUTOSET=1
 
 #  Pstore-RAM setting - if enabled, Kdump won't be loaded, instead the Pstore
 #  RAM backend will be configured. In order to have success, this operation

kdump_load.sh

@@ -11,6 +11,41 @@
 #  avoid failing here to not risk a boot hang.
 #
 
+#  This function has 2 purposes: if 'kdump' is passed as argument and we don't
+#  have crashkernel memory reserved, we edit grub config file and recreate
+#  grub.cfg, so next boot has it reserved; in this case, we also  bail-out,
+#  since kdump can't be loaded anyway.
+#
+#  If 'pstore' is passsed as argument, we try to unset crashkernel iff it's
+#  already set AND the pattern in grub config is the one added by us - if the
+#  users set crashkernel themselves, we don't mess with that.
+grub_update() {
+	GRUBCFG="/etc/default/grub"
+	CRASHK="$(cat /sys/kernel/kexec_crash_size)"
+	SED_ADD="s/^GRUB_CMDLINE_LINUX_DEFAULT=\"/GRUB_CMDLINE_LINUX_DEFAULT=\"crashkernel=192M crash_kexec_post_notifiers /g"
+
+	if [ "${GRUB_AUTOSET}" -eq 1 ]; then
+		if [ "$1" = "kdump" ] && [ "${CRASHK}" -eq 0 ]; then
+			sed -i "${SED_ADD}" "${GRUBCFG}"
+			update-grub 1>/dev/null
+			sync "/boot/grub/grub.cfg" 2>/dev/null
+			sync "/efi/EFI/steamos/grub.cfg" 2>/dev/null
+
+			logger "kdump-steamos: kdump cannot work, no reserved memory in this boot..."
+			logger "kdump-steamos: but we automatically set crashkernel for next boot."
+			exit 0
+		fi
+
+		if [ "$1" = "pstore" ] && [ "${CRASHK}" -ne 0 ]; then
+			sed -i "s/\"crashkernel=192M crash_kexec_post_notifiers /\"/g" "${GRUBCFG}"
+			update-grub 1>/dev/null
+			sync "/boot/grub/grub.cfg" 2>/dev/null
+			sync "/efi/EFI/steamos/grub.cfg" 2>/dev/null
+			logger "kdump-steamos: clearing crashkernel memory previously set..."
+		fi
+	fi
+}
+
 if [ ! -f "/etc/default/kdump" ]; then
 	logger "kdump-steamos: /etc/default/kdump not present - aborting..."
 	exit 0
@@ -58,18 +93,21 @@ if [ "${USE_PSTORE_RAM}" -eq 1 ]; then
 
 	if [ ${MEM_SIZE} -ge ${MEM_REQUIRED} ]; then
 		if modprobe ramoops mem_address=0x${MEM_START} mem_size=${MEM_REQUIRED} record_size=${RECORD_SIZE}; then
+			#  If Pstore is set, update grub.cfg to avoid reserving crashkernel memory.
 			logger "kdump-steamos: pstore-RAM was loaded successfully"
+			grub_update pstore
 			exit 0
 		fi
 		logger "kdump-steamos: pstore-RAM load failed...will try kdump"
 	fi
-		#  Fallbacks to kdump load - if we fail when configuring pstore, better try kdump;
+		#  Fallback to kdump load - if we fail when configuring pstore, better
-		#  who knows and we may be lucky enough to have some crashkernel reserved memory...
+		#  trying kdump; in case we have crashkernel memory reserved, lucky us.
+		#  If not, we're going to set that automatically on grub_update().
+		#  Notice that if it's not set, we bail-out in grub_update() - there's
+		#  no point in continuing since kdump cannot work.
 fi
 
-#  TODO: insert code here to validate that crashkernel is configured and
+grub_update kdump
-#  memory is reserved; if not, set it on grub.cfg and recreate the EFI grub
-#  config file, warning users that in the current boot kdump is not set.
 
 #  Stolen from Debian kdump
 KDUMP_CMDLINE=$(sed -re 's/(^| )(crashkernel|hugepages|hugepagesz)=[^ ]*//g;s/"/\\\\"/' /proc/cmdline)