Bug#1036828: debian-cd: wrong firmware archives built and published for D-I releases?

[Cyril Brulebois]

Package: debian-cd
Severity: serious

Hi,

During a previous release, I spotted we had two firmware builds, but let
the topic go once I was reassured that was to be expected. For RC 4:

1/43: Starting firmware_bookworm build at 2023-05-27:09:03:53
[…]
9/43: Starting firmware_sid build at 2023-05-27:09:04:01
[…]
firmware_bookworm finished successfully (started at 2023-05-27:09:03:53, ended at 2023-05-27:09:06:31, took 0h02m38s)
[…]
firmware_sid finished successfully (started at 2023-05-27:09:04:01, ended at 2023-05-27:09:07:07, took 0h03m06s)

Now, waiting to see if someone would join the testing efforts, I diffed
firmware lists between rc3 and rc4, and spotted those differences:

-./firmware-sof-signed_2.2.4-1_all.deb
-./intel-microcode_3.20230214.1_amd64.deb
-./intel-microcode_3.20230214.1_i386.deb
+./firmware-sof-signed_2.2.5-1_all.deb
+./intel-microcode_3.20230512.1_amd64.deb
+./intel-microcode_3.20230512.1_i386.deb

The intel-microcode bits are OK:

intel-microcode | 3.20230512.1 | testing/non-free-firmware | source, amd64, i386
intel-microcode | 3.20230512.1 | unstable/non-free-firmware | source, amd64, i386

The firmware-sof-signed, not so much:

firmware-sof-signed | 2.2.4-1 | testing/non-free-firmware | all
firmware-sof-signed | 2.2.5-1 | unstable/non-free-firmware | all

It's a relatively new upload, and it's of course blocked at the moment:

[2023-05-15] Accepted firmware-sof 2.2.5-1 (all source) into unstable (Mark Pearson) (signed by: Vincent Bernat)

For the record, those archives end up being published in locations like
the following, and I definitely expected those to match the firmware
packages getting shipped into the images, not be some kind of snapshot of
what's in unstable at the time the release is built!

https://cdimage.debian.org/cdimage/firmware/bookworm/bookworm_di_rc3/

We should definitely clarify the situation, and get to the bottom of that
double firmware build.

From the log lines quoted above, if both bookworm and sid builds end up
shipping files in the same destination directory, the last build wins and
overrides the first one entirely?


See also the “rsync noise” that seemed somewhat OK to ignore. Not sure
whether that's directly related though… ISTR it was probably about some
timestamp discrepancy due to the underlying filesystem. For RC 4:

file has vanished: "/home/debian-cd/publish/.bookworm_di_rc4/firmware/firmware.zip"
rsync: stat "/dsa/cdimage/.incoming/.bookworm_di_rc4/firmware/.firmware.tar.gz.VQGfUC" failed: No such file or directory (2)
rsync: rename "/dsa/cdimage/.incoming/.bookworm_di_rc4/firmware/.firmware.tar.gz.VQGfUC" -> "firmware.tar.gz": No such file or directory (2)


Cheers,

[Debian Bug Tracking System]

tag -1 patch pending

Bug #1036828 [debian-cd] debian-cd: wrong firmware archives built and published for D-I releases?
Added tag(s) pending and patch.

[Cyril Brulebois]

Control: tag -1 patch pending

Post by Cyril Brulebois
For the record, those archives end up being published in locations like
the following, and I definitely expected those to match the firmware
packages getting shipped into the images, not be some kind of snapshot of
what's in unstable at the time the release is built!
https://cdimage.debian.org/cdimage/firmware/bookworm/bookworm_di_rc3/
We should definitely clarify the situation, and get to the bottom of that
double firmware build.
From the log lines quoted above, if both bookworm and sid builds end up
shipping files in the same destination directory, the last build wins and
overrides the first one entirely?

I'm considering the following change for the upcoming (pseudo) RC 5 release:
https://salsa.debian.org/images-team/setup/-/commit/9a77631

This means nothing changes for weekly builds, which are detected as being
built with DEBVERSION set to “testing” (please note that I didn't
investigate what happens to firmware directories in this case).

Meanwhile, actual releases get that sid job skipped (since the release
specific config file, e.g. CONF.sh.bookworm_di_rc4, sets DEBVERSION to
“bookworm-DI-rc4” instead of sticking to the default “testing” value).


Cheers,

[James Addison]

Followup-For: Bug #1036828
X-Debbugs-Cc: ***@debian.org

Hi Cyril,

Should this bug be closed? (the logic to skip the experimental/sid firmware
image build during non-testing builds is in place for both bookworm and trixie)

Regards,
James