In Bullseye and later, the `/etc/machine-id` file needs to have the
value `uninitialized` for it to be recognized as first boot and
therefor trigger the generation of a new machine-id.
Fixes: #52
Checking the root filesystem before mounting it is always a good idea.
But after changing the root filesystem, it is especially important to
(re-)check it to ensure everything went fine. So add that check.
While fsck.ext4 and fsck.vfat are part of the initramfs, it turns out
that fsck itself isn't!
While it normally gets included in the initramfs, for some reason that
doesn't happen with our image creation, so it isn't available on first
boot, so explicitly copy 'fsck' so that it will be included.
For that to work, 'logsave' is needed too, so include that as well.
During boot up, initramfs wants to check the filesystems and it does so
via `fsck`, which then (presumably) checks the filesystem being used
and calls the appropriate fsck.* to do the actual checking.
But when `fsck` itself isn't available you get the following warning:
"Warning: fsck not present, so skipping root file system"
When it is available, you'll get a message like this:
"Begin: Will now check root file system ... fsck from util-linux 2.38.1"
Let's also follow our own advise and not *assume* the needed fsck
programs are present in initramfs, but add them explicitly.
Now we can start the resize operations while knowing the current
filesystem is in a proper/clean state.
The main trigger was a missing program in the initramfs, which probably
everyone assumed was there ... but wasn't. (See next commit ...)
So instead list every program that we need/call, so that we *know* that
that program is included in the initramfs.
Also document this new 'policy'.
Also group the programs by the package which has them and sort the list
alphabetically by the package names.
The 'ROOTFLAGS' parameter should NOT be quoted as it then introduces an
(empty) extra argument to 'mount', causing a boot failure.
Thanks to 'mjt' for the help and identifying the issue.
Lessons learned:
1) Don't blindly follow the *suggestions* that shellcheck makes
2) Test your changes (properly) before submitting/merging them
ad 1) I find shellcheck a very useful tool, but it doesn't (and likely
can't) understand the full context. The developer does (or should) and
should evaluate each suggestion whether it's applicable in this case.
ad 2) This should've been obvious and certainly for me.
I'm normally very dilligent and test all my changes, but I got sloppy
this time and did not. With logical consequences.
SC2154: var is referenced but not assigned.
The 'rootmnt' variable is set in /usr/share/initramfs-tools/init and if
it's not available, that would be bad.
According to SC2154 you can fix that issue by using '${var:?}' so it
would fail if unset or empty.
So apply that and reference that SC item in the comments.
Also remove it from the exclude list in the CI config file.
To build the yaml files, we only need python3 and make.
All the other/old packages that got installed are needed for building
actual images, so get rid of them.
This should also significantly speed up the job.
While checking the 'artifacts' I noticed that git operations didn't
produce the output as expected, both in top comment as at the end where
build information is written to /etc/raspi-image-id.
That was caused by git not working as it wasn't installed.
Check all the scripts, under 'rootfs/etc/initramfs-tools' (for now) with
'shellcheck'.
A couple of checks are excluded of which SC1090 and SC1091 will remain.
But SC2086 and especially SC2154 should be fixed! And when they are
fixed in the current code, those should be enabled again for checking.
Also rename 'check yamls' job to 'yamllint', so that's directly clear
which tool is being used.
Add a new stage 'check input' to our CI configuration to check whether
our input files contain errors.
The first check is for the YAML files whether they contain errors or
warnings according to 'yamllint'.
Add 'yamllint' config file to not consider too long lines as errors.
And fix the errors/warnings in 'salsa-ci.yml' itself.
Apparently job names can contain space, so rename 'build-yamls' to
'build yamls'.
All RPi models which have wireless, also have bluetooth, so add the
firmware package to those images so that the hardware is supported.
Also remove the 'Extra' from the wireless firmware comment.
Upgrade the existing packages before installing new ones.
It's better to have a fully up-to-date system to work on and it could
also fix/prevent an installation issue if a package depends on an
existing, but un-upgraded one. With this change we get
"and 0 not upgraded", which was not always the case before.
Also add a '/' after the 'build' directory to make it clear it is a dir.
The 'eatmydata' program is "designed to disable fsync and friends" and
can speed up tasks at the risk of potential data loss.
The speed up is welcome and the data loss not important as it would only
make the pipeline fail (most likely), but you can run it again.
It's unclear (to me) what they filter out. And why.
https://salsa.debian.org/help/ci/yaml/index#only-except says:
"NOTE: only and except are not being actively developed. rules is
the preferred keyword to control when to add jobs to pipelines."
So if filtering is desired, it should be added using the 'rules' keyword
and the reasoning behind it should be clarified in comments and/or
commit messages.
As we're just starting out using CI, it is especially important to be
aware of any failure, so that we can fix it.
Such a workaround can be added later in case it is deemed necessary.
Debian's Salsa team has their own registry of images they maintain, so
use them instead of ones maintained by gitlab.
At https://salsa.debian.org/salsa-ci-team/pipeline/container_registry
one can view the various images they maintain.
Switch to using 'unstable' instead of 'sid'. The idea is to later also
add steps based on 'stable' and then 'unstable' matches better.
It's common to define variables/constants in the top of a file, so do
that here too.
Move 'stage' field directly under step name as it's quite significant
and seen in other salsa-ci.yml as well.
Move 'image' line after 'stage' line as it's quite important as well.
This was the initial trigger for the tag-rename 'operation' as it caused
confusion (with me) as it was also a mislabeled tag, so remove the
ambiguity by renaming it to 'tag-firmware'.
Using path identifiers for tag names causes ambiguity as it's not
(immediately) clear whether a reference to it is a path or a tagname.
This causes hard to read/interpret log files and can lead to subtle and
hard to detect bugs.
The same tag can refer to different things based on the context
(partition/device/mountpoint/etc), so just use the 'tag-' prefix.
On the previous 'mount: /' step, I also added that it's to be mounted on
"dirname: '/'" to make it (more) explicit.
And removed it again as it seems you need to specify both 'dirname' AND
'mount-on' or neither. See https://bugs.debian.org/1023321
The firmware partition holds a copy of the initramfs and the kernel and
over the years we have seen a steady increase in its sizes.
Resizing the firmware partition later on is cumbersome as the root
partition follows directly, so it's better to make the firmware
partition not too small. A size of 508MB should be enough to accommodate
4-5 kernels+initramfs, which seems desirable.
Previously one had to calculate how large the /boot/firmware partition
would be, but expressing it directly in MiB units is much clearer.
This also has the benefit that the /boot/firmware partition's size would
not change if the total image size would be changed.
Such a change should be a deliberate decision and not some side-effect.
As that 'side-effect' did happen since first submitting this patch,
revert the /boot/firmware partition's size back to 300MB.
Signed-off-by: Diederik de Haas <didi.debian@cknow.org>
The wireless firmware is in the firmware-brcm80211 package, but that is
still in 'non-free', so add 'non-free' back to the sources until all
the needed firmware packages are in 'non-free-firmware'.
https://salsa.debian.org/kernel-team/firmware-nonfree/-/merge_requests/36
is where the move to non-free-firmware is proposed, but not yet merged.
Previous commit not only replaced 'non-free' with 'non-free-firmware',
it also removed the 'contrib' archive area from the sources.list.
It added a note about it in the code comments, which I think is the
wrong place. The 'why' of a change belongs in a git commit message,
where one can be as verbose as needed.
Code comments should be used to clarify the 'what' (it does) in case it
would not be immediately obvious.
The removal of 'contrib' totally makes sense though.
We did not use it and 'contrib' and 'non-free' are not part of
(official) Debian, whereas 'non-free-firmware' is now part of Debian
(official media) as a consequence of the change to the Debian Social
Contract following the GR vote.
With this change, we only use what Debian itself would only use.
Fixes: 1ffce8e6bb
When the image was build also determines which package versions got
installed in the generated image and could help explain why a user has
problems with the downloaded image.
Signed-off-by: Diederik de Haas <didi.debian@cknow.org>
Regression introduced in f89f71560d2ca1bd60d97dbb26b89782657d56ae:
the sed call modifies /etc/default/raspi-firmware, which used to be
/etc/default/raspi3-firmware; while not ideal, working on
/etc/default/raspi*-firmware shouldn't interfere on unrelated files.
Diederik de Haas