2020–09–08:
PinePhone multi-boot image optimizations
Since I'll be using my kernel for all the distros anyway, I thought I might
spice it up a bit, and use something a bit more modern than ext4.
Instead of complicated partitioning scheme that wastes space and is not easy
to optimize, I decided to use a single btrfs filesystem for everything with
each distribution having its own subvolume, and all files being compressed by
zstd. This made it possible to create a 8GiB SD card image, with
9 distributions on it.
The image creation process is now much simpler. First I create a
distros/
directory with a subdirectory for each distribution. Then
I download and prepare rootfs.tar.zst
tarballs, for each
distribution I want to have in the image, as previously described in
this blog.
And then I create the image:
#!/bin/bash
if [ "$(whoami)" != "root" ] ; then
exec sudo bash "$0" "$@"
fi
set -e -x
. ./config
rm -f $IMG
truncate -s $IMGSIZE $IMG
sfdisk -W always $IMG <<EOF
label: gpt
first-lba: 16
table-length: 8
start=16, size=1072, type=D7B1F817-AA75-2F4F-830D-84818A145370, name="mb-aw", attrs=RequiredPartition
start=1088, size=31680, type=D7B1F817-AA75-2F4F-830D-84818A145370, name="mb-rk", attrs=RequiredPartition
start=16M, size=128M, type=d210f963-0fbe-43a0-8fb2-9d1b6bab6fe4, name="mb-pbootfs", attrs=RequiredPartition
size=128M, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, name="mb-boot", attrs="RequiredPartition"
size=+, name="mb-rootfs"
EOF
L=`losetup -P --show -f $IMG`
mkfs.btrfs ${L}p5
mkdir -p m
mount -o compress-force=zstd:12 ${L}p5 m
for ddir in distros/*
do
test -f $ddir/config || continue
name=${ddir#distros/}
btrfs subvolume create m/$name
specs="$specs m/$name:$ddir/rootfs.tar.zst"
#bsdtar -xp --numeric-owner -C m/$name -f $ddir/rootfs.tar.zst
done
./tools/extract1 $specs
./mkimage-apply-fixes.sh
umount m
losetup -d "$L"
#./mkimage-boot.sh
This does away with the need for having to deal with partition numbers.
Subvolumes have the name of the distro, and are easily referenced in the boot
arguments.
During boot I just mount the appropriate subvolume instead of the main
volume of the btrfs filesystem. All that's needed is to pass
rootfsopts=subvol=distro-name
.
Modifying and accessing files in all distributions at once also becomes
trivial. All I need to do is to mount the main volume, and the subvolumes for
each partition appear nicely available there as subdirectories in the main
volume. Previously I had to juggle with mounting 10 arbitrarily named
partitions.
It's also very easy to add/remove distributions without the need for
re-partitioning, and all distributions share the remaining free space on the
SD card.
I find this setup excellent! Btrfs partition with 9 simultaneously
installed distributions takes just 6.5GiB of space. Previously just mobian alone
took 3GiB of used space, and a 4 GiB partition.
Making the image bootable
To make the image bootable, I just build the boot.conf file for p-boot,
based on a per-distribution config
file, and use it to format the
boot partition.
Config file looks like:
version=2023-06-04
name="Ubuntu Touch"
bootargs="logo.nologo vt.global_cursor_default=0"
And the script to format make the image bootable:
#!/bin/sh
if [ "$(whoami)" != "root" ] ; then
exec sudo sh "$0" "$@"
fi
set -e
KVER=6.4
. ./config
TMP=`mktemp -d`
# loglevel=15
serial="console=ttyS0,115200 earlycon=ns16550a,mmio32,0x01c28000"
serial_pro="earlycon=uart8250,mmio32,0xff1a0000 console=ttyS2,1500000n8"
silent="quiet loglevel=0 systemd.show_status=false"
bootargs_base="$silent cma=256M console=tty1 consoleblank=0 panic=3 rw rootwait root=PARTLABEL=mb-rootfs rootfstype=btrfs rootflags=compress-force=zstd,nodatacow,subvol"
kbuilds=../builds
echo "device_id = Distro Demo Image 2023-06-11" > boot.conf
rm -rf boot
mkdir -p boot
cp "$kbuilds/ppp-$KVER/board.dtb" boot/
cp "$kbuilds/ppp-$KVER/Image" boot/
cp splash.bmp boot/
no=0
pno=1
for ddir in distros/*
do
test -f $ddir/config || continue
dist=${ddir#distros/}
unset version name bootargs variant splash
. ./$ddir/config
if test -z "$splash" ; then
splash=$dist
fi
if test "$variant" = "og" -o -z "$variant"
then
(
echo "no = $no"
echo " name = $name $version"
echo " atf = ../p-boot/dist/fw.bin"
echo " dtb = $kbuilds/ppd-$KVER/board-1.1.dtb"
echo " dtb2 = $kbuilds/ppd-$KVER/board-1.2.dtb"
echo " linux = $kbuilds/ppd-$KVER/Image"
echo " bootargs = $serial $bootargs_base=$dist $bootargs"
if test -f files/$dist.argb ; then
echo " splash = files/$splash.argb"
else
echo " splash = files/xnux.argb"
fi
) >> boot.conf
no=$(($no+1))
fi
if test "$variant" = "pro" -o -z "$variant"
then
echo -n " \"$name\"" >> "$TMP/pro-menu-items"
(
echo " elif test x\$ret = x$pno ; then"
echo " setenv bootargs \"$serial_pro $bootargs_base=$dist $bootargs\""
if [[ $bootargs =~ eg25-manager ]] ; then
echo " fdt rm /serial@ff1b0000/modem"
echo " fdt set /vcc-4g-5v regulator-always-on"
echo " fdt set /vcc-4g regulator-always-on"
fi
) >> "$TMP/pro-menu-handlers"
pno=$(($pno+1))
fi
done
# JumpDrive is special
if test -d distros/jumpdrive
then
(
echo "no = $no"
echo " name = Jumpdrive 0.8"
echo " atf = ../p-boot/dist/fw.bin"
echo " dtb = $kbuilds/ppd-$KVER/board-1.1.dtb"
echo " dtb2 = $kbuilds/ppd-$KVER/board-1.2.dtb"
echo " linux = $kbuilds/ppd-$KVER/Image"
echo " initramfs = distros/jumpdrive/initramfs"
echo " bootargs = loglevel=0 silent console=tty0 vt.global_cursor_default=0"
echo " splash = files/jumpdrive.argb"
) >> boot.conf
no=$(($no+1))
fi
cat <<EOF >boot/boot.cmd
# disable output to vidconsole
setenv stdout serial
setenv stderr serial
# display splash screen
load mmc 1:4 0x10000000 splash.bmp
bmp d 0x10000000
load mmc 1:4 \$kernel_addr_r Image
load mmc 1:4 \$fdt_addr_r board.dtb
fdt addr \$fdt_addr_r
if tmenu "Power off"$(cat "$TMP/pro-menu-items") ; then
bmp d 0x10000000
if test x\$ret = x0 ; then
poweroff
$(cat "$TMP/pro-menu-handlers")
fi
fi
booti \$kernel_addr_r - \$fdt_addr_r
EOF
mkimage -A arm -T script -C none -d boot/boot.cmd boot/boot.scr
#../p-boot/.build/p-boot-conf-native . mb-pbootfs.img
#L=`losetup -P --show -f $IMG`p
L=/dev/sdg
../p-boot/.build/p-boot-conf-native . ${L}3
dd if=p-boot.bin of=${L}1
# boot filesystem for PPP
mkfs.vfat ${L}4
mkdir $TMP/m
mount ${L}4 $TMP/m
cp -r boot/* $TMP/m
umount $TMP/m
# u-boot for PPP
dd if=../u-boot-ppp2/.build/uboot-rkbin-1088.bin of=${L}2
#losetup -d $L
sync
And that's really all there's to it, as far as making a bootable
multi-distro image is concerned.
Some annoying details are
still remaining
- p-boot doesn't support automatic selection of dtb based on detected
pinephone variant
- modem manager doesn't enable the modem automatically
- I have to automate per-distro modifications of
/etc/fstab
and
/etc/shadow
files
- patch my kernel to be able to deal with idiosyncracies of each
distribution
- delete custom modem initialization scripts from each distro
Support
If you'd like to support this effort, you can contribute at https://xnux.eu/contribute.html