Upcoming D-I Bookworm RC 3 and RC 4

[Cyril Brulebois]

Hi all,

Current status
==============

Some of you might have read bits and pieces here and there, but I
thought I'd share my plans a little more formally with some teams.

As anticipated, it seems the last bits from the release team on dda@ got
some attention, and we've been receiving many more installation reports
than previously. Some of those reports are directly actionable, and
various fixes are flowing in a bunch of components; some of them trigger
some kind of chain reaction, and might end up leading to fixes via a
point release instead.

Installation reports have been mostly positive, and I'm not aware of any
critical bug that would jeopardize the proposed timeline. \o/

For those who want to follow at home, I have a list of things I'm
tracking and that might be fixed or at least considered before 12.0.0;
nothing is mandatory in there!
https://salsa.debian.org/installer-team/debian-installer/-/issues/1


Proposed plan
=============

I think it'd make sense to have at least 2 releases:
- 1 around mid-May;
- 1 around end of May.

The first one would bundle a bunch of the fixes or improvements being
worked on these days, making sure everything works as intended. The
second one would be an “everything is frozen, we upload d-i one last
time” release, which could include a few last-minute fixes or
improvements if required or desired.

This means we should be able to pick whatever linux kernel is in testing
for the first release, while the kernel team prepares the final linux
upload on their own accord, which we'll pick up with the second release.

Communication and dynamics between the installer and kernel teams are
well-established and give good results, see earlier discussion:
https://lists.debian.org/debian-kernel/2023/05/msg00031.html
https://lists.debian.org/debian-kernel/2023/05/msg00043.html

Salvatore had good questions about how to best handle possible critical
fixes (security or otherwise) during the last few days, and whether
bookworm-security would make sense if that happened. On the installer
side I'm happy to work with whatever ends up in testing (via unstable or
t-p-u), but we don't currently support building or running d-i against
security suites (the former is probably trivial, the latter might be
very tricky). I'm also not sure how going through security would factor
in when it's time to build final images (12.0.0). But maybe we can think
about it if and when we actually encounter such problems.

I'm happy to touch base with the release team again when we approach end
of May, to see what would be considered best for the (hopefully) final
d-i upload (and d-i-n-i, at least a dinstall afterward). It might make
sense not to wait too much before doing so, in case we end up having to
fix a package or two, and re-upload


Regarding the final release, I'm happy to perform a final d-i upload if
some packages needed an update since RC 4
 but hopefully the last build
can be reused without any changes.


Interactions with other teams
=============================

You might have noticed the `dak copy-installer` step that copies the
`installer-<arch>/` directories from unstable to testing is now
something that I can trigger on my own, which gives us some appreciated
flexibility: contrary to point releases that are planned and frozen in
advance, testing keeps evolving (less so with the freeze progressing)
and I couldn't really set fixed dates in advance so that various teams
would be on stand-by


The other major team involved is debian-cd, and Steve and I usually come
up with a set of days where both of us are available, and image building
starts whenever the right bits have reached testing (the d-i package via
britney, the `installer-<arch>/` directories via dak).

I've been in the debian-cd group for a long while, but I've only been
taught a crash-course recently on how to operate a d-i release and I
should be able to operate one or two on my own.

[ In the context of debian-cd, I'm calling a “d-i release” one of the
“D-I $CODENAME Alpha|RC $N” releases; as opposed to initial stable
releases (e.g. 12.0.0) or point releases (e.g. 11.7.0), which involve
more work, lots of testers, the press team, etc. I wouldn't call myself
ready for those just yet
 ]

This gives me a little more work but also some more flexibility as to
when RC 3 and RC 4 happen.

A side-effect, with regular d-i builds and live builds being started
together when building images for a d-i release, is that I'll have to
keep an eye on the live images as well. As mentioned to Jonathan earlier
this week, my focus has only ever been on d-i itself (which keeps me
busy already, and I'm not trying to wear yet another hat), and that
shouldn't change in the near future, but I'm more than happy to keep an
eye on debian-live needs, and wait for an extra package or an extra
commit in live-setup.git before starting building images for a d-i RC
(see things like #1035360 and #1035560).

I don't think I'll be actively polling debian-live for input though; I'm
more than happy to be told what is important (so that it can be tracked
via the Salsa issue mentioned above, or one of the “point release
variations”, see issues 2 and 3). I'm usually sharing progress towards a
d-i release on IRC on #debian-boot, before moving to #debian-cd. I know
at least Jonathan is present in both channels so we should be good; I'm
happy to consider alternatives if needed though.

Back to d-i, we have the release announcements going to dda@ and to the
website, and I'm autonomous on both counts as usual. For the final
release, I'll step back (even if d-i gets a final upload), and let you
folks organize the Bookworm announcement.


Conclusion
==========

I'm happy to answer any questions you might have, and to amend my plans
if you'd like some things to be done differently. Thanks for your time
and attention, that's quite a long mail
 With the release approaching,
I thought it'd make sense to be as explicit as possible to make sure
everyone is on the same page.


Cheers,

[Cyril Brulebois]

Hi all,

Post by Cyril Brulebois
- 1 around mid-May;
- 1 around end of May.
The first one would bundle a bunch of the fixes or improvements being
worked on these days, making sure everything works as intended.

This part has happened as planned.

Post by Cyril Brulebois
The second one would be an “everything is frozen, we upload d-i one
last time” release, which could include a few last-minute fixes or
improvements if required or desired.

And that part I'd like to plan a little more.

Post by Cyril Brulebois
I'm happy to touch base with the release team again when we approach
end of May, to see what would be considered best for the (hopefully)
final d-i upload (and d-i-n-i, at least a dinstall afterward). It
might make sense not to wait too much before doing so, in case we end
up having to fix a package or two, and re-upload


Dates that have been announced[1] so far:
- 2023-05-24: full freeze
- 2023-05-28: last moment to file unblock requests
- 2023-06-03: bookworm totally frozen
(per “last week prior to the release”)

1. https://lists.debian.org/debian-devel-announce/2023/04/msg00007.html

On the d-i side, we'll have a round of translation updates[2], along
with some last tweaks, before RC 4.

As far as I understand, at least at the moment, we aren't expecting a
new linux upload before the release. But if the need arises, we should
be able to deal with it.

2. https://lists.debian.org/debian-boot/2023/05/msg00250.html


I'm not sure what the best timeline would be for RC 4. Let's consider
two options:
- After the full freeze is in effect: we would have RC 3 and RC 4
roughly two weeks apart, which matches what I had in mind initially,
but we might have a few more changes coming in via late unblock
requests, that could impact the installer.
Example: May 25.
- After a green light from the release team, i.e. once all unblock
requests have been considered, and once it's expected there should
be no changes in the archive.
Window: May 28 - June 3.

In the first case, we would have a little more time to sort incoming
installation reports, and to react if needed. We might need a final
upload of debian-installer(-netboot-images).

In the second case, we would have a little less time, but the message
in the release announcement could be a clear “there are no more pending
changes for bookworm, this is the closest thing to the release, please
test extensively” (better wording welcome). We would probably not need a
final upload of debian-installer(-netboot-images), with debian-cd
picking up the exact same files that were used for RC 4, for the final
release.

Both options seem equally reasonable to me, please let me know whether
you have a preference. I don't need an answer right away, that can be
discussed during the upcoming release team meeting.

(If we go for “d-i/d-i-n-i are the last packages changing in bookworm”,
please keep in mind we need at least 1 britney run and 1 dinstall run
between the d-i upload and the d-i-n-i one.)

Post by Cyril Brulebois
Regarding the final release, I'm happy to perform a final d-i upload
if some packages needed an update since RC 4
 but hopefully the last
build can be reused without any changes.

No changes there, I'll be on stand-by for anything d-i related until the
(tentative) release date.

Post by Cyril Brulebois
Interactions with other teams
=============================
[ kibi does dak copy-installer ]

That was confirmed to be working fine while preparing RC 3.

Post by Cyril Brulebois
[ kibi does debian-cd ]

That was also confirmed to be working fine while preparing RC 3.


Cheers,

[Cyril Brulebois]

Hi all,

Post by Cyril Brulebois
- 2023-05-24: full freeze

[x] ← You are here!

Post by Cyril Brulebois
- 2023-05-28: last moment to file unblock requests
- 2023-06-03: bookworm totally frozen
(per “last week prior to the release”)
1. https://lists.debian.org/debian-devel-announce/2023/04/msg00007.html
On the d-i side, we'll have a round of translation updates, along
with some last tweaks, before RC 4.

We should be all done in a few hours.

I might pick a last minute tasksel change as well (for lxqt); I think it
would help, could break, but that would be trivially revertable (and
there would be room to do so, see below).

Post by Cyril Brulebois
As far as I understand, at least at the moment, we aren't expecting a
new linux upload before the release. But if the need arises, we should
be able to deal with it.

Checked with Salvatore: still no upload planned before the release.

Post by Cyril Brulebois
I'm not sure what the best timeline would be for RC 4. Let's consider
- [Soon] Example: May 25.
- [Later] Window: May 28 - June 3.

No preferences have been expressed by the release team during today's
meeting.

Post by Cyril Brulebois
In the first case, we would have a little more time to sort incoming
installation reports, and to react if needed. We might need a final
upload of debian-installer(-netboot-images).

I think I'll go for this one, aiming for May 25 or May 26 unless some
issues pop up.

We know we have at least the apt vs. adduser issue that's going to get
resolved, and I'd prefer not to wait for it, and also not to rush the
update into testing


Also, we might have other packages that directly (because they produce
udebs) or indirectly (because they're installed on every system, like
apt) affect the installer or the installation process
 migrate to
testing later on.

Let's consider a last debian-installer(-netboot-images) upload once
Bookworm is definitely frozen, maybe a few days before the release to
minimize the chances of having to consider a last-minute critical
bugfix. Once it's in testing, we could even build images like we would
for “D-I Bookworm RC 5”, just to be on the extra safe side. Those could
be fetched by testers, but wouldn't be signed or announced (keeping them
in the “usual dot directory”, deleting them a few days later). That
would give us some extra peace of mind for the actual 12.0.0 images
build that will happen on release day.

It looks to me this would combine the best of two worlds:
- Not wait too much before RC 4, leaving us some more days to deal with
incoming reports;
- Minimize risks of merging final changes in Bookworm, by having this
“canary” RC 5 release.


Notes:
- This would be different from what we did for Bullseye, where we had
an RC 3 built using debian-installer 20210731, which was then reused
as is for the 11.0.0 images build.
- This would be different from what we did for Buster, where we had
an RC 3 built using debian-installer 20190702, which was then reused
as is for the 10.0.0 images build.
- Given each D-I release is “lighter” than a full release build (be it
for an initial release or a point release), it seems to me going for
RC 4 plus pseudo-RC 5 is cheap enough to buy us peace of mind than
delaying RC 4 until we think (but still aren't sure) nothing is going
to change in Bookworm.
- Release early, release often! (even if late in the release cycle
)
- I'm doing most of the work anyway



Cheers,

[Cyril Brulebois]

Hi,

Here's a quick update about my current plans regarding the installer and
regarding debian-cd dry runs.

(Trimming recipients; also, I'm using 12.0.0 below as I'm referring to
the version identifier on the debian-cd side, but that's hopefully
synonymous with Debian 12.0, i.e. the initial Bookworm release.)

Post by Cyril Brulebois
I might pick a last minute tasksel change as well (for lxqt); I think
it would help, could break, but that would be trivially revertable
(and there would be room to do so, see below).

I did that, didn't hear bad things about it, be it from users or
maintainers.

Post by Cyril Brulebois
Checked with Salvatore: still no upload planned before the release.

I've asked Salvatore to update me on this one, but I haven't heard of
any changes in that area at the moment.

Post by Cyril Brulebois
I think I'll go for this one, aiming for May 25 or May 26 unless some
issues pop up.

Details disagreed so that ended up being May 27 instead, but close
enough.

Post by Cyril Brulebois
We know we have at least the apt vs. adduser issue that's going to get
resolved, and I'd prefer not to wait for it, and also not to rush the
update into testing


That migrated after the release, and will be part of pseudo-RC 5 and
more importantly 12.0.0.

Post by Cyril Brulebois
Also, we might have other packages that directly (because they produce
udebs) or indirectly (because they're installed on every system, like
apt) affect the installer or the installation process
 migrate to
testing later on.

Last packages that migrated (that I know of) are openssl (CVE fixes) and
libselinux (+b6 to minimize ESO). I'd be happy to be kept explicitly in
the loop for any further updates to udeb-producing packages (it appears
unblock-udeb does the job, including for binNMUs
), or things that the
release team would identify as being part of a standard installation, so
that we keep as many eyes open as possible


Post by Cyril Brulebois
Let's consider a last debian-installer(-netboot-images) upload once
Bookworm is definitely frozen, maybe a few days before the release to
minimize the chances of having to consider a last-minute critical
bugfix. Once it's in testing, we could even build images like we would
for “D-I Bookworm RC 5”, just to be on the extra safe side. Those could
be fetched by testers, but wouldn't be signed or announced (keeping them
in the “usual dot directory”, deleting them a few days later). That
would give us some extra peace of mind for the actual 12.0.0 images
build that will happen on release day.

I plan to work on that mid-week, sometime on Wednesday (ideally, unless
we know more packages need migrating
) or Thursday (as a fallback). I'll
check with other debian-cd members, but I might even try and see how a
full build would work, so that we have some kind of advance knowledge
regarding the following week-end.

At the moment I've spotted three uploads:
- apt-setup (1:0.183), for ports architectures.
- preseed (1.118), for Hurd, even if the changelog is not quite verbose
about that part

- vte2.91 (0.70.5-1): new upstream (bugfix) release with no particular
bugfixes identified.

Unless specifically requested, I don't plan on including the first two
before Bookworm because we don't need it for release architectures. I
/think/ hurd-i386 gets somewhat released from unstable, so that
shouldn't matter
 not sure about ports architectures. Those /could/ be
considered but truth be told, my default setting is: only change what is
absolutely required before Bookworm. vte2.19 seems quite vague, and
ignoring it entirely should be fine.

Since there was good progress on the arm64 console thing (#1036952), and
considering the current results of the investigation as to where vt102
comes from, and why arm64 isn't quite the deciding factor (a busybox
limitation instead), I'd be happy to consider getting an updated
rootskel for migration before pseudo-RC 5 and 12.0.0. Such an upload
would need to happen quickly though
 (ema bcc'd, as possible uploader,
no obligations though!).


Cheers,

[Holger Wansing]

Hi,

Post by Cyril Brulebois
- apt-setup (1:0.183), for ports architectures.
- preseed (1.118), for Hurd, even if the changelog is not quite verbose
about that part…
- vte2.91 (0.70.5-1): new upstream (bugfix) release with no particular
bugfixes identified.

I have just uploaded grub-installer 1.194, which completes Croatian and
Ukranian translation.
Feel free to include it in bookworm, if possible.


Holger

[Samuel Thibault]

Post by Cyril Brulebois
Unless specifically requested, I don't plan on including the first two
before Bookworm because we don't need it for release architectures. I
/think/ hurd-i386 gets somewhat released from unstable, so that
shouldn't matter…

Yes.

Post by Cyril Brulebois
not sure about ports architectures.

Ports don't have a testing either.

Samuel

[Emanuele Rocca]

Hi,

Post by Cyril Brulebois
Since there was good progress on the arm64 console thing (#1036952), and
considering the current results of the investigation as to where vt102
comes from, and why arm64 isn't quite the deciding factor (a busybox
limitation instead), I'd be happy to consider getting an updated
rootskel for migration before pseudo-RC 5 and 12.0.0. Such an upload
would need to happen quickly though… (ema bcc'd, as possible uploader,
no obligations though!).

I see that Samuel took care of the rootskel upload. Thanks!

ema