megi's PinePhone Development Log RSS

Surgenons in Gaza Surgeons in Gaza
table of contents | email: x@xnux.eu | xnux.eu | my kernels | git | donate :)

2020–09–08: PinePhone multi-boot image optimizations

Since I'll be using my kernel for all the distros anyway, I thought I might spice it up a bit, and use something a bit more modern than ext4.

Instead of complicated partitioning scheme that wastes space and is not easy to optimize, I decided to use a single btrfs filesystem for everything with each distribution having its own subvolume, and all files being compressed by zstd. This made it possible to create a 8GiB SD card image, with 9 distributions on it.

The image creation process is now much simpler. First I create a distros/ directory with a subdirectory for each distribution. Then I download and prepare rootfs.tar.zst tarballs, for each distribution I want to have in the image, as previously described in this blog.

And then I create the image:

#!/bin/bash

if [ "$(whoami)" != "root" ] ; then
        exec sudo bash "$0" "$@"
fi

set -e -x

. ./config

rm -f $IMG
truncate -s $IMGSIZE $IMG

sfdisk -W always $IMG <<EOF
label: gpt
first-lba: 16
table-length: 8

start=16, size=1072, type=D7B1F817-AA75-2F4F-830D-84818A145370, name="mb-aw", attrs=RequiredPartition
start=1088, size=31680, type=D7B1F817-AA75-2F4F-830D-84818A145370, name="mb-rk", attrs=RequiredPartition
start=16M, size=128M, type=d210f963-0fbe-43a0-8fb2-9d1b6bab6fe4, name="mb-pbootfs", attrs=RequiredPartition
size=128M, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, name="mb-boot", attrs="RequiredPartition"
size=+, name="mb-rootfs"
EOF

L=`losetup -P --show -f $IMG`

mkfs.btrfs ${L}p5

mkdir -p m
mount -o compress-force=zstd:12 ${L}p5 m

for ddir in distros/*
do
        test -f $ddir/config || continue
        name=${ddir#distros/}
        btrfs subvolume create m/$name
        specs="$specs m/$name:$ddir/rootfs.tar.zst"

        #bsdtar -xp --numeric-owner -C m/$name -f $ddir/rootfs.tar.zst
done

./tools/extract1 $specs

./mkimage-apply-fixes.sh

umount m
losetup -d "$L"

#./mkimage-boot.sh

This does away with the need for having to deal with partition numbers. Subvolumes have the name of the distro, and are easily referenced in the boot arguments.

During boot I just mount the appropriate subvolume instead of the main volume of the btrfs filesystem. All that's needed is to pass rootfsopts=subvol=distro-name.

Modifying and accessing files in all distributions at once also becomes trivial. All I need to do is to mount the main volume, and the subvolumes for each partition appear nicely available there as subdirectories in the main volume. Previously I had to juggle with mounting 10 arbitrarily named partitions.

It's also very easy to add/remove distributions without the need for re-partitioning, and all distributions share the remaining free space on the SD card.

I find this setup excellent! Btrfs partition with 9 simultaneously installed distributions takes just 6.5GiB of space. Previously just mobian alone took 3GiB of used space, and a 4 GiB partition.

Making the image bootable

To make the image bootable, I just build the boot.conf file for p-boot, based on a per-distribution config file, and use it to format the boot partition.

Config file looks like:

version=2023-06-04
name="Ubuntu Touch"
bootargs="logo.nologo vt.global_cursor_default=0"

And the script to format make the image bootable:

#!/bin/sh

if [ "$(whoami)" != "root" ] ; then
        exec sudo sh "$0" "$@"
fi

set -e

KVER=6.4

. ./config

TMP=`mktemp -d`

# loglevel=15
serial="console=ttyS0,115200 earlycon=ns16550a,mmio32,0x01c28000"
serial_pro="earlycon=uart8250,mmio32,0xff1a0000 console=ttyS2,1500000n8"
silent="quiet loglevel=0 systemd.show_status=false"
bootargs_base="$silent cma=256M console=tty1 consoleblank=0 panic=3 rw rootwait root=PARTLABEL=mb-rootfs rootfstype=btrfs rootflags=compress-force=zstd,nodatacow,subvol"
kbuilds=../builds

echo "device_id = Distro Demo Image 2023-06-11" > boot.conf

rm -rf boot
mkdir -p boot

cp "$kbuilds/ppp-$KVER/board.dtb" boot/
cp "$kbuilds/ppp-$KVER/Image" boot/
cp splash.bmp boot/

no=0
pno=1
for ddir in distros/*
do
        test -f $ddir/config || continue
        dist=${ddir#distros/}
        unset version name bootargs variant splash
        . ./$ddir/config

        if test -z "$splash" ; then
                splash=$dist
        fi

        if test "$variant" = "og" -o -z "$variant"
        then
                (
                        echo "no = $no"
                        echo "  name = $name $version"
                        echo "  atf = ../p-boot/dist/fw.bin"
                        echo "  dtb = $kbuilds/ppd-$KVER/board-1.1.dtb"
                        echo "  dtb2 = $kbuilds/ppd-$KVER/board-1.2.dtb"
                        echo "  linux = $kbuilds/ppd-$KVER/Image"
                        echo "  bootargs = $serial $bootargs_base=$dist $bootargs"
                        if test -f files/$dist.argb ; then
                                echo "  splash = files/$splash.argb"
                        else
                                echo "  splash = files/xnux.argb"
                        fi
                ) >> boot.conf

                no=$(($no+1))
        fi

        if test "$variant" = "pro" -o -z "$variant"
        then
                echo -n " \"$name\"" >> "$TMP/pro-menu-items"
                
                (
                        echo "  elif test x\$ret = x$pno ; then"
                        echo "    setenv bootargs \"$serial_pro $bootargs_base=$dist $bootargs\""
                        if [[ $bootargs =~ eg25-manager ]] ; then
                                echo "    fdt rm /serial@ff1b0000/modem"
                                echo "    fdt set /vcc-4g-5v regulator-always-on"
                                echo "    fdt set /vcc-4g regulator-always-on"
                        fi
                ) >> "$TMP/pro-menu-handlers"

                pno=$(($pno+1))
        fi
done

# JumpDrive is special
if test -d distros/jumpdrive
then
        (
                echo "no = $no"
                echo "  name = Jumpdrive 0.8"
                echo "  atf = ../p-boot/dist/fw.bin"
                echo "  dtb = $kbuilds/ppd-$KVER/board-1.1.dtb"
                echo "  dtb2 = $kbuilds/ppd-$KVER/board-1.2.dtb"
                echo "  linux = $kbuilds/ppd-$KVER/Image"
                echo "  initramfs = distros/jumpdrive/initramfs"
                echo "  bootargs = loglevel=0 silent console=tty0 vt.global_cursor_default=0"
                echo "  splash = files/jumpdrive.argb"
        ) >> boot.conf

        no=$(($no+1))
fi

cat <<EOF >boot/boot.cmd
# disable output to vidconsole
setenv stdout serial
setenv stderr serial

# display splash screen
load mmc 1:4 0x10000000 splash.bmp
bmp d 0x10000000

load mmc 1:4 \$kernel_addr_r Image
load mmc 1:4 \$fdt_addr_r board.dtb
fdt addr \$fdt_addr_r

if tmenu "Power off"$(cat "$TMP/pro-menu-items") ; then
  bmp d 0x10000000

  if test x\$ret = x0 ; then
    poweroff
$(cat "$TMP/pro-menu-handlers")
  fi
fi

booti \$kernel_addr_r - \$fdt_addr_r
EOF

mkimage -A arm -T script -C none -d boot/boot.cmd boot/boot.scr

#../p-boot/.build/p-boot-conf-native . mb-pbootfs.img

#L=`losetup -P --show -f $IMG`p
L=/dev/sdg

../p-boot/.build/p-boot-conf-native . ${L}3
dd if=p-boot.bin of=${L}1

# boot filesystem for PPP
mkfs.vfat ${L}4
mkdir $TMP/m
mount ${L}4 $TMP/m
cp -r boot/* $TMP/m
umount $TMP/m

# u-boot for PPP
dd if=../u-boot-ppp2/.build/uboot-rkbin-1088.bin of=${L}2

#losetup -d $L

sync

And that's really all there's to it, as far as making a bootable multi-distro image is concerned.

Some annoying details are still remaining

Support

If you'd like to support this effort, you can contribute at https://xnux.eu/contribute.html