Request: more images in the weekly build -> arm64

[Roland Clobus]

Hello Steve, list,

Since the Minidebconf in Hamburg earlier this year, the support for
arm64 in the Debian live images has seen a lot of progress.
Together with Emanuele Rocca, I was able to to confirm that live images
that are built natively on arm64 contain the same as the cross-built
live images from amd64. Additionally some of the bug reports that we
created have been resolved since then.

From my point of view, the weekly image building (organised in
live-setup) can be extended to include arm64 was well [1].

I could prepare a MR, but need a bit more info:
How are the non-amd64 images (e.g the current netinst-images) created?
Are they the result of a cross-build, or are they natively generated on
other hosts?
The latest gnome image took 3 hours according to the log [2].
The cross-build on my computer take 6 to 10 times as much computing
time, so that might be considered.
(I've already identified one cause, which I intend to remove later)

If they are cross-builds, perhaps it would be wiser to start with one
image at first (e.g. the gnome image) and then set up the openQA testing
chain, before progressing to other variants.

With kind regards,
Roland Clobus

[1] https://get.debian.org/images/weekly-live-builds/
[2]
https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-gnome.log
(from 03:12:32 to 06:04:48)

PS: Don't let this delay the focus from the upcoming point releases
planned for next week.

[Thomas Schmitt]

Hi,

Post by Roland Clobus
The latest gnome image took 3 hours according to the log [2].
[2] https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/debian-live-testing-amd64-gnome.log (from 03:12:32 to 06:04:48)

Interesting log. There are two time zones used in two kinds of time stamp.
The more frequently given stamps say "[2024-06-17 04:12:33]" to
"[2024-06-17 07:04:19]".

The first timestamp after ISO production is
[2024-06-17 04:58:02] lb binary_onie

It seems that most of the two hours after building the ISO are spent with
activities which have nothing to do with the ISO production.
More than half of the overall 3 hours get spent with "lb chroot_archives"
runs. (I assume that those don't happen in debian-cd.)

In particular there is a strange time span of more than 1 hour from
[2024-06-17 04:58:32] lb chroot_archives source install
P: Configuring file /etc/apt/sources.list
to
[2024-06-17 06:03:35] lb source_live
P: Begin copying live-build configuration...
with not much reported inbetween.
Less than a minute before this time the ISO was produced.

Further i see a potential slow network connection at the very end.
[2024-06-17 06:32:48] lb chroot_archives source remove
P: Deconfiguring file /etc/apt/sources.list
Get:1 http://deb.debian.org/debian trixie InRelease [169 kB]
...
Get:13 http://deb.debian.org/debian trixie/non-free-firmware Icons (64x64) [29 B]
Fetched 40.9 MB in 31min 26s (21.7 kB/s)
Reading package lists...
[2024-06-17 07:04:19] lb chroot_hostname remove


Have a nice day :)

Thomas

[Steve McIntyre]

Hey Roland!

Since the Minidebconf in Hamburg earlier this year, the support for arm64 in
the Debian live images has seen a lot of progress.
Together with Emanuele Rocca, I was able to to confirm that live images that
are built natively on arm64 contain the same as the cross-built live images
from amd64. Additionally some of the bug reports that we created have been
resolved since then.

Cool. :-)

From my point of view, the weekly image building (organised in live-setup)
can be extended to include arm64 was well [1].

That's awesome!

How are the non-amd64 images (e.g the current netinst-images) created? Are
they the result of a cross-build, or are they natively generated on other
hosts?

The installer images are all generated using debian-cd on casulana,
our big central amd64 with lots of fast storage. There's nothing that
needs native execution in debian-cd, so there's nothing that needs
cross-building for installer images. Live images will be different, of
course.

The latest gnome image took 3 hours according to the log [2].
The cross-build on my computer take 6 to 10 times as much computing time, so
that might be considered.

Right. We build the live images in parallel inside a single VM on
casulana, so build times that you're seeing are not necessarily
representative. Last time I checked, a single gnome image build took
~1h. Running in paralllel lets us build multiple images in that 3h
period, so it's definitely worth it overall to do that.

For an example of the kind of performance cost of cross-building, we
*do* still build some amd64 images and cross-build some equivalent
arm64 openstack images on casulana. The last builds took:

* amd64 0h09m55s
* arm64 2h52m25s

which is approximately a factor of 17.

(I've already identified one cause, which I intend to remove later)

OK.

If they are cross-builds, perhaps it would be wiser to start with one image
at first (e.g. the gnome image) and then set up the openQA testing chain,
before progressing to other variants.

Sure, that would make sense. Building a full set of arm64 images
probably isn't feasible unless we do something to make the builds much
faster.

[Steve McIntyre]

Hey Roland,

Post by Steve McIntyre
Hey Roland!

Since the Minidebconf in Hamburg earlier this year, the support for arm64 in
the Debian live images has seen a lot of progress.

...

Post by Steve McIntyre

I could prepare a MR

The MR is at
https://salsa.debian.org/images-team/live-setup/-/merge_requests/5
Let's start only with a GNOME image at first.
I'll update the matching openQA tests, as soon as the new image appears.

ACK. As you've referenced, I'll also need to tweak some other bits in
the image setup. Maybe this weekend once we've done the point
releases...

[Steve McIntyre]

Post by Steve McIntyre
Hey Roland,

Post by Steve McIntyre
Hey Roland!

Since the Minidebconf in Hamburg earlier this year, the support for arm64 in
the Debian live images has seen a lot of progress.

...

Post by Steve McIntyre

I could prepare a MR

The MR is at
https://salsa.debian.org/images-team/live-setup/-/merge_requests/5
Let's start only with a GNOME image at first.
I'll update the matching openQA tests, as soon as the new image appears.

ACK. As you've referenced, I'll also need to tweak some other bits in
the image setup. Maybe this weekend once we've done the point
releases...

I've merged everything and added that config and run a couple of times
to test. I'm seeing a (slow, as expected!) live image build OK, then
right at the end:

...
P: building the debian-installer
cp: cannot stat 'chroot/debian-installer/build/dest/cdrom/vmlinuz': No such file or directory
E: An unexpected failure occurred, exiting...
P: Begin unmounting filesystems...
P: Saving caches...
...

Did you test this to build an arm64 image yourself?