Menu

6 years of selfhosting, hit with RAM corruption

Reading 0:00
15s threshold

My Proxmox host kernel panicked and hung. I assumed it was a Intel GVT driver bug at first, I used LLMs to debug this issue, cause i've never faced something like this before. I want people to learn from my mistakes.

Journal had nothing. The panic record was in pstore ( /var/lib/systemd/pstore ): a general protection fault on pointer 0xffbf8… that should have been 0xffff8… . That's one flipped bit, bit 54. Digging back, I found 9 October oopses on a different kernel build with the same flipped bit. A BIOS update didn't help; it panicked again a day later, same bit.

The machine is a Lenovo tiny PC with non-ECC SO-DIMMs, so there were only crashes.

Files affected

In my Docker VM, a Firefox-based container segfaulted on every launch. dpkg --verify flagged a library. Read with dd iflag=direct , which bypasses the page cache, the file matched the package. The cached copy had 8 bytes flipped.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.