Kernel update from 4.18 to 4.19 renders encrypted system unbootable

I have an encrypted system using LVM on LUKS, what means that all my partitions are logical volumes in an encrypted volume group. Only exception, the boot partition.

After the update the boot process ends in a kernel panic instead of asking for the passphrase to decrypt the hard disk. My first idea was that some modules or hooks are missing in the initramfs even though I did not touch /etc/mkinitcpio before the kernel update. But lsinitcpio -a shows that the two initramfs are almost identical. As much as I can see in 4.19 only these modules are missing:

xxhash
zstd
zstd_compress
zstd_decompress

Could this be the reason why the disk is not decrypted?

BTW, after moving my system to an encrypted disk I updated the kernel twice successfully, from 4.15 to 4.16 and from 4.16 to 4.18.

perhaps xxhash but i suppose zstd* will be used from the btrfs filesystem.

a shot in to the dark: try systemd and sd-encrypt hook in the mkinitcpio.conf hook array
https://wiki.archlinux.org/index.php/Dm-crypt/System_configuration#Using_sd-encrypt_hook

I’ve another question concerning the generation of initramfs. The one generated in October for kernel 4.18 has more than 12 MB, the one generated recently for kernel 4.19 is only half as big. Playing around with Hooks in /etc/mkinitcpio.conf I also generated a new initramfs for kernel 4.18 which is significantly smaller than the one generated in October and ends up in the same kernel panic as the one for kernel 4.19. This also holds true when I go back to the version of systemd that was installed in October when I generated the last working initramfs. Thus my question: Has anything changed between October 2018 and January 2019 that changes the way the initramfs is generated?

Well, eventually I got the new kernel working. Thanks to @brikler for his support.

To solve the problem I added these 34 libraries to the BINARIES section of /etc/mkinitcpio.conf

usr/lib/libe2p.so.2 <
usr/lib/libcom_err.so.2 <
usr/lib/libext2fs.so.2 <
usr/lib/libtinfo.so.6 <
usr/lib/libncursesw.so.6 <
usr/lib/libreadline.so.7 <
usr/lib/libdevmapper-event.so.1.02 <
usr/lib/libdl.so.2 <
usr/lib/libpopt.so.0 <
usr/lib/libm.so.6 <
usr/lib/libudev.so.1 <
usr/lib/libgpg-error.so.0 <
usr/lib/libdevmapper.so.1.02 <
usr/lib/libattr.so.1 <
usr/lib/liblz4.so.1 <
usr/lib/libidn.so.11 <
usr/lib/libseccomp.so.2 <
usr/lib/libip4tc.so.0 <
usr/lib/libgcrypt.so.20 <
usr/lib/libcryptsetup.so.4 <
usr/lib/libcap.so.2 <
usr/lib/libacl.so.1 <
usr/lib/libkmod.so.2 <
usr/lib/systemd/libsystemd-shared-239.so <
usr/lib/librt.so.1 <
usr/lib/libmount.so.1 <
usr/lib/libuuid.so.1 <
usr/lib/libblkid.so.1 <
usr/lib/libpthread.so.0 <
usr/lib/libz.so.1 <
usr/lib/liblzma.so.5 <
usr/lib/ld-linux-x86-64.so.2 <
usr/lib/libc.so.6 <
usr/lib/libcrypt.so.1 <

They were in the old one but missing in the new.

Honestly, I do not know which of these was responsible for not being able to open my LUKS encrypted LVM. Anyhow, what is annoying is that important libraries are kicked out and thus systems are rendered unbootable.

BTW, this bug must have been introduced between October 24th - when I generated the last working initramfs for kerrnel 4.18 - and January 2019. Since the hooks did not change and all initramfs generated for kernel 4.18 after October 2019 did not work, it must have been introduced with another package update. No clue which one it might be, thus I can’t file a bug report.

1 Like

this is not the end from this drama and what do you do when systemd will be upgrade to 240 or higher? :wink:
mkinitcpio will fail and the next boot will fail with a not working initramfs.

I know, it’s a workaround an not a bugfix. But I do not know what the bug is and hence where to report it.

difficult to say, I don’t remember something in particular that could affect the size of the initramfs. Mine for example is around 19MB and should be the normal size.

Which initramfs are you talking about? In my case the working ones have roughly these sizes:

initramfs-linux.img   12 MB
initramfs-linux-fallback.img   31 MB

The ones where important libraries were missing:

initramfs-linux.img   8 MB
initramfs-linux-fallback.img   27 MB

And what do you mean by “normal size”? The size should differ for various system setups. For an encrypted LVM based setup a bunch of hooks had to be added and thus there should be more modules and libraries in the initramfs.

With normal size I mean for a normal chakra installation, and yes, I have more or less the same values. An image generated after an update that is half the size is not normal to me.
This my values:

initramfs-linux.img   19 MB
initramfs-linux-fallback.img   40 MB

Since there are different topics with kernel issues / kernel upgrade, I will check an review our packages.

Wow these are big initramfs. I added the hooks for LVM and encryption last year to the “normal” hooks and my working initramfs has only two third of your size. I know Chaka users for whom the initramfs with about 8 MB work. But in that case the root file system is not encrypted.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.