Merge incremental upgrades to allow endless upgrading

While investigating overlayfs (#8415) an idea was proposed to work around its filesystem stacking limitation of two that also would have the added benefit of allowing a Tails installation to be incrementally upgraded essentially endlessly, instead of just 4-5 times as currently restricted by the Tails partition size.

The basic idea is to stop stacking the IUKs’ squashfs:es but instead merge the new ones into the old one. To keep down the space requirements (which translates into RAM requirements in our setting) while doing the upgrade we’d want to modify the old squashfs on a file-by-file basis (the CPU (and hence time) requirements would be increased a lot though due to the extensive re-compression needed). We’d have to deal with deleted/whiteout:ed files some how too, of course.

With this approach we’d be able to upgrade Tails endlessly given that the Tails system partition is big enough (around 1 GiB after a “Clone & Install” should go a long way). A further optimization to consider would be to merge the IUK’s squashfs into the filesystem.squashfs to eliminate incremental upgrade disk overhead completely (then only a few 100 MiB should be enough for a long time, i.e. we could significantly decrease the current size of the Tails system partition).

A serious issue is that we may not be able to use squashfs any more since mksquashfs seemingly can only append new files to an existing squashfs archive, but neither remove files nor modify existing ones. In addition to squashfs the only filesystem live-boot supports that have compression is JFFS2, which perhaps is an alternative. Other filesystems with compression support could be considered although it’d require us to add support for them to live-boot.

It should be noted that verifying the integrity of a Tails installation after installing incremental upgrades will become a lot harder after this.

Parent Task: #11131 (closed)

Original created by @anonym on 8534 (Redmine)