Nearly reproducible Bookworm 12.6 live images

[Holger Levsen]

Hi Roland,

I'm sooo close...

[...]

hehe, congrats & thanks for keeping us in the loop! In honor of your
work I've manually written the signature of this email...! :)
--
cheers,
Holger

⢀⣎⠟⠻⢶⣊⠀
⣟⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄

The devel is in the details.

[Philip Hands]

Hello lists,
I'm sooo close...
Now that that 12.6 live images have been generated [1], there is only
one embedded timestamp left (/boot/grub/live-theme/theme.txt and its
parent folder)
I've not seen this timestamp issue before, as I am using a slightly
different way of building the image than live-setup uses.
live-setup uses a local cache of the git repository of live-build, to
avoid being blocklisted by salsa for heavy traffic, caused by parallel
builds.
This local cache brings that this file could have an old timestamp,
whereas my local builds always used 'now' (caused by git clone), which
would be properly truncated to SOURCE_DATE_EPOCH, and therefore be
reproducible.

AFAIK it's the repeated full clones that get you blocklisted.

I was having the same issue with the openQA workers which I've fixed by
adding a --reference-if-able option pointing at a locally accessible
copy of the repo on the workers (in fact the "local" copy is an nfs
mount, but it's enough to ensure that the worker is not trying to do a
full clone from salsa, and so avoids the blocklist)

I guess the thing to do is a fetch against the cache repo first just to
make sure it stays updated, then a clone while pointing at that repo
via the --reference-if-able option, then it'll still give you the same
result as doing a clone, without overloading salsa.

Cheers, Phil.

[Steve McIntyre]

Hello lists,
I'm sooo close...
Now that that 12.6 live images have been generated [1], there is only one
embedded timestamp left (/boot/grub/live-theme/theme.txt and its parent
folder)
I've not seen this timestamp issue before, as I am using a slightly different
way of building the image than live-setup uses.
live-setup uses a local cache of the git repository of live-build, to avoid
being blocklisted by salsa for heavy traffic, caused by parallel builds.
This local cache brings that this file could have an old timestamp, whereas
my local builds always used 'now' (caused by git clone), which would be
properly truncated to SOURCE_DATE_EPOCH, and therefore be reproducible.
I'll prepare a patch, and hopefully the next set of live images (12.6.1 or
12.7.0) will be fully reproducible.

Awesome stuff! :-)

[James Addison]

Hi Roland!

[ ... snip ... ]
live-setup uses a local cache of the git repository of live-build, to
avoid being blocklisted by salsa for heavy traffic, caused by parallel
builds.
This local cache brings that this file could have an old timestamp,
whereas my local builds always used 'now' (caused by git clone), which
would be properly truncated to SOURCE_DATE_EPOCH, and therefore be
reproducible.

Limiting the timestamp to a maximum of SOURCE_DATE_EPOCH makes sense so that
the modification-time is consistent for equivalent rebuilds.

However: if the contents of the txt file / other content in the git repository
change, would that prevent an independent rebuilder from recreating the
identical CD image as output?

(note: given that the package archive is regularly updated, I think that there
are other reasons why an identical point-in-time rebuild may require sharing
of some build-time/archive metadata -- the reason I'm asking is that I'd like
to check whether this git repo could require similar co-ordination during
rebuilds. fixed commit ID / signed tag, or other mechanism for example)

Regards,
James

[Steve McIntyre]

Hello list, Steve,

I'll prepare a patch, and hopefully the next set of live images
(12.6.1 or 12.7.0) will be fully reproducible.

https://salsa.debian.org/images-team/live-setup/-/merge_requests/6

Have you found the time to look at this?

Just merged now, apologies for keeping you waiting.