sha512sum-error at debian-live-11.5.0-amd64-gnome.iso

[Gerd Mühlenbruch]

Dear maintainers

Today I downloaded the present debian-live-11.5.0-amd64-gnome.iso via my
prepared script
*********
Optionen="-c -nv --show-progress --limit-rate=800k -a Ziele.log"
Pfad="https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid"
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.iso
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.log
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.packages
rm SHA512SUMS
rm SHA512SUMS.sign
wget $Optionen $Pfad/SHA512SUMS
wget $Optionen $Pfad/SHA512SUMS.sign
*********

The final check
    sha512sum --ignore-missing -c SHA512SUMS
resulted into
    debian-live-11.5.0-amd64-gnome.iso: FEHLSCHLAG
    debian-live-11.5.0-amd64-gnome.log: OK
    debian-live-11.5.0-amd64-gnome.packages: OK
    sha512sum: WARNUNG: 1 berechnete PrÌfsumme passte NICHT
It was my first failed check.

Already since some years, the check
    gpg --keyserver keyring.debian.org --verify SHA512SUMS.sign SHA512SUMS
reports that the key doesn't have a trusted signature
    gpg: Signatur vom So 11 Sep 2022 01:00:38 CEST
    gpg:                mittels RSA-SchlÌssel
DF9B9C49EAA9298432589D76DA87E80D6294BE9B
    gpg: Korrekte Signatur von "Debian CD signing key
<debian-***@lists.debian.org>" [unbekannt]
    gpg: WARNUNG: Dieser SchlÌssel trÀgt keine vertrauenswÌrdige Signatur!
    gpg:          Es gibt keinen Hinweis, daß die Signatur wirklich dem
vorgeblichen Besitzer gehört.
    Haupt-Fingerabdruck  = DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D
6294 BE9B
This may be my own problem, because I haven't got a book to learn gpg.

Best regards

Gerd MÃŒhlenbruch
Hauptstraße 36
D-89522 Heidenheim

[Cyril Brulebois]

Hallo Gerd,

Please set something like LC_ALL=C, not everybody understands German. :)

Post by Gerd Mühlenbruch
Today I downloaded the present debian-live-11.5.0-amd64-gnome.iso via my
prepared script
*********
Optionen="-c -nv --show-progress --limit-rate=800k -a Ziele.log"
Pfad="https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid"
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.iso
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.log
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.packages
rm SHA512SUMS
rm SHA512SUMS.sign
wget $Optionen $Pfad/SHA512SUMS
wget $Optionen $Pfad/SHA512SUMS.sign
*********
The final check
    sha512sum --ignore-missing -c SHA512SUMS
resulted into
    debian-live-11.5.0-amd64-gnome.iso: FEHLSCHLAG
    debian-live-11.5.0-amd64-gnome.log: OK
    debian-live-11.5.0-amd64-gnome.packages: OK
    sha512sum: WARNUNG: 1 berechnete PrÌfsumme passte NICHT
It was my first failed check.

I downloaded the same ISO, and SHA{256,512}SUMS with Firefox, and both
validate.

For the record, that's what I'm seeing:

3efed2698da7fab0218a505eb504410d4cd9c7bc09478483bdbb3c593013b371 debian-live-11.5.0-amd64-gnome.iso
450d09f1f1556effce4bc072ad395e45652aa40ed035b8ab35616c54fbef0c6edf803acb483ac25737b12bbae858277c1aa261eee36c4edf505a85c915d73c67 debian-live-11.5.0-amd64-gnome.iso

Post by Gerd Mühlenbruch
Already since some years, the check
    gpg --keyserver keyring.debian.org --verify SHA512SUMS.sign SHA512SUMS
reports that the key doesn't have a trusted signature
    gpg: Signatur vom So 11 Sep 2022 01:00:38 CEST
    gpg:                mittels RSA-SchlÌssel
DF9B9C49EAA9298432589D76DA87E80D6294BE9B
    gpg: Korrekte Signatur von "Debian CD signing key
    gpg: WARNUNG: Dieser SchlÌssel trÀgt keine vertrauenswÌrdige Signatur!
    gpg:          Es gibt keinen Hinweis, daß die Signatur wirklich dem
vorgeblichen Besitzer gehört.
    Haupt-Fingerabdruck  = DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294
BE9B
This may be my own problem, because I haven't got a book to learn gpg.

The English version:

$ gpg --keyserver keyring.debian.org --verify SHA512SUMS.sign SHA512SUMS
gpg: Signature made Sun 11 Sep 2022 01:00:38 CEST
gpg: using RSA key DF9B9C49EAA9298432589D76DA87E80D6294BE9B
gpg: Good signature from "Debian CD signing key <debian-***@lists.debian.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: DF9B 9C49 EAA9 2984 3258 9D76 DA87 E80D 6294 BE9B

The important part is that the signature is correct. The warning is
about your not having set a level of trust for the signing key. That's
not a problem.


Cheers,

[Thomas Schmitt]

Hi,

Post by Gerd Mühlenbruch
Pfad="https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid"
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.iso
sha512sum --ignore-missing -c SHA512SUMS
    debian-live-11.5.0-amd64-gnome.iso: FEHLSCHLAG

I did

wget https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-11.5.0-amd64-gnome.iso

and got redirected to

https://laotzu.ftp.acc.umu.se/debian-cd/current-live/amd64/iso-hybrid/debian-live-11.5.0-amd64-gnome.iso
Connecting to laotzu.ftp.acc.umu.se (laotzu.ftp.acc.umu.se)|2001:6b0:19::166|:443... connected.

The SHA512SUMS file came directly from

https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/SHA512SUMS


The SHA512 of debian-live-11.5.0-amd64-gnome.iso is then

450d09f1f1556effce4bc072ad395e45652aa40ed035b8ab35616c54fbef0c6edf803acb483ac25737b12bbae858277c1aa261eee36c4edf505a85c915d73c67

which matches the line in the SHA512SUMS file

450d09f1f1556effce4bc072ad395e45652aa40ed035b8ab35616c54fbef0c6edf803acb483ac25737b12bbae858277c1aa261eee36c4edf505a85c915d73c67 debian-live-11.5.0-amd64-gnome.iso


So do you get a different SHA512 from your downloaded
debian-live-11.5.0-amd64-gnome.iso ?

Or does the downloaded SHA512SUMS file show a different SHA512 ?

---------------------------------------------------------------------------

Post by Gerd Mühlenbruch
    gpg: WARNUNG: Dieser Schlüssel trägt keine vertrauenswürdige Signatur!
    gpg:          Es gibt keinen Hinweis, daß die Signatur wirklich dem vorgeblichen Besitzer gehört.
    Haupt-Fingerabdruck  = DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294 BE9B

The true safety check is not whether the signature is supported by a
chain of trust, but whether it yields one of the fingerprints which are
listed on

https://www.debian.org/CD/verify

In your case it has the fingerprint of

pub rsa4096/DA87E80D6294BE9B 2011-01-05 [SC]
Key fingerprint = DF9B 9C49 EAA9 2984 3258 9D76 DA87 E80D 6294 BE9B
uid Debian CD signing key <debian-***@lists.debian.org>

(Beware of alternative ways to express space characters when comparing
the strings automatically.)


Have a nice day :)

Thomas

[Gerd Mühlenbruch]

Hi,

I think I found the reason.
 While copying the logfile I remembered that I used wget twice with
"-c" option.
 After downloading the first bytes, I remembered, that my batch
terminal will close automatically. So I stopped my batch and started it
again in a terminal. Now "weg -c" continued to download the file.

I downloaded the file again at once and now the test works fine.

Many thanks

Gerd Mühlenbruch

 Just for information:

"cmp" showed a difference in size:
    cmp debian-live-11.5.0-amd64-gnome.iso
Err-debian-live-11.5.0-amd64-gnome.iso
    cmp: EOF on debian-live-11.5.0-amd64-gnome.iso after byte
2910683136, in line 11367081
"ls" told the same:
-rw-r--r-- 1 ghdm ghdm 2911003302 Sep 10 13:49
Err-debian-live-11.5.0-amd64-gnome.iso
-rw-r--r-- 1 ghdm ghdm 2910683136 Sep 10 13:49
debian-live-11.5.0-amd64-gnome.iso


Nevertheless I added LANG=C and run the pure test on the first iso file
just for translation.
===================================================================
gpg: Signature made Sun Sep 11 01:00:38 2022 CEST
gpg:                using RSA key DF9B9C49EAA9298432589D76DA87E80D6294BE9B
gpg: Good signature from "Debian CD signing key
<debian-***@lists.debian.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the
owner.
Primary key fingerprint: DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294 BE9B
===================================================================

===
debian-live-11.5.0-amd64-gnome.iso: FAILED
debian-live-11.5.0-amd64-gnome.log: OK
debian-live-11.5.0-amd64-gnome.packages: OK
sha512sum: WARNING: 1 computed checksum did NOT match
===================================================================
debian-live-11.5.0-amd64-gnome.iso: FAILED
debian-live-11.5.0-amd64-gnome.log: OK
debian-live-11.5.0-amd64-gnome.packages: OK
sha256sum: WARNING: 1 computed checksum did NOT match
===================================================================

Post by Thomas Schmitt
Hi,

Post by Gerd Mühlenbruch
Pfad="https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid"
wget $Optionen $Pfad/debian-live-11.5.0-amd64-gnome.iso
sha512sum --ignore-missing -c SHA512SUMS
    debian-live-11.5.0-amd64-gnome.iso: FEHLSCHLAG

I did
wget https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-11.5.0-amd64-gnome.iso
and got redirected to
https://laotzu.ftp.acc.umu.se/debian-cd/current-live/amd64/iso-hybrid/debian-live-11.5.0-amd64-gnome.iso
Connecting to laotzu.ftp.acc.umu.se (laotzu.ftp.acc.umu.se)|2001:6b0:19::166|:443... connected.
The SHA512SUMS file came directly from
https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/SHA512SUMS
The SHA512 of debian-live-11.5.0-amd64-gnome.iso is then
450d09f1f1556effce4bc072ad395e45652aa40ed035b8ab35616c54fbef0c6edf803acb483ac25737b12bbae858277c1aa261eee36c4edf505a85c915d73c67
which matches the line in the SHA512SUMS file
450d09f1f1556effce4bc072ad395e45652aa40ed035b8ab35616c54fbef0c6edf803acb483ac25737b12bbae858277c1aa261eee36c4edf505a85c915d73c67 debian-live-11.5.0-amd64-gnome.iso
So do you get a different SHA512 from your downloaded
debian-live-11.5.0-amd64-gnome.iso ?
Or does the downloaded SHA512SUMS file show a different SHA512 ?
---------------------------------------------------------------------------

Post by Gerd Mühlenbruch
    gpg: WARNUNG: Dieser Schlüssel trägt keine vertrauenswürdige Signatur!
    gpg:          Es gibt keinen Hinweis, daß die Signatur wirklich dem vorgeblichen Besitzer gehört.
    Haupt-Fingerabdruck  = DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294 BE9B

The true safety check is not whether the signature is supported by a
chain of trust, but whether it yields one of the fingerprints which are
listed on
https://www.debian.org/CD/verify
In your case it has the fingerprint of
pub rsa4096/DA87E80D6294BE9B 2011-01-05 [SC]
Key fingerprint = DF9B 9C49 EAA9 2984 3258 9D76 DA87 E80D 6294 BE9B
(Beware of alternative ways to express space characters when comparing
the strings automatically.)
Have a nice day :)
Thomas

[Thomas Schmitt]

Hi,

While copying the logfile I remembered that I used wget twice with "-c"
option.
[...]
    cmp debian-live-11.5.0-amd64-gnome.iso
Err-debian-live-11.5.0-amd64-gnome.iso
    cmp: EOF on debian-live-11.5.0-amd64-gnome.iso after byte 2910683136,
[...]
-rw-r--r-- 1 ghdm ghdm 2911003302 Sep 10 13:49 Err-debian-live-11.5.0-amd64-gnome.iso
-rw-r--r-- 1 ghdm ghdm 2910683136 Sep 10 13:49 debian-live-11.5.0-amd64-gnome.iso

That's really strange. Looks like wget -c added 320166 bytes to the
2910683136 bytes of the original ISO.
This collides with the urge to use wget -c rather than a web browser:
https://www.debian.org/CD/http-ftp/

I fail to reproduce the problem with a netinst CD image (which has only
400 MB):

$ wget -c https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.5.0-amd64-netinst.iso
[...]
^C
$ ls -l debian-11.5.0-amd64-netinst.iso
[...] 6727900 Oct 3 21:53 debian-11.5.0-amd64-netinst.iso
$ wget -c https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.5.0-amd64-netinst.iso
[...]
HTTP request sent, awaiting response... 206 Partial Content
Length: 400556032 (382M), 393828132 (376M) remaining [application/x-iso9660-image]
[...]
$ ls -l debian-11.5.0-amd64-netinst.iso
[...] 400556032 Sep 10 14:40 debian-11.5.0-amd64-netinst.iso

In my experiment the correct size was downloaded and sh512sum confirms
that the content is as announced in .../amd64/iso-cd/SHA512SUMS .

It would be interesting to learn under which circumstances wget yields
the result which you observe.


Have a nice day :)

Thomas

[Thomas Schmitt]

Hi,

I started my batch by doubleclicking the file in nautilus.

Being a fvwm user i can only riddle whether this start by nautilus has
something to do with the problem.
But your post of 3 Oct 2022 18:28:08 +0000 says that you started the
original download by wget -c. So i assume that nautilus is not significant.

I expected differences in bytes anywhere in the beginning, but comp -l and
comp -b didn't showed such result.
==>>> cmp: EOF on debian-live-11.5.0-amd64-gnome.iso after byte 2910683136

That astonished me already with your original post. There seems to be
a surplus tail appended. What's in that tail ?

E.g. what do you get from

dd if=Err-debian-live-11.5.0-amd64-gnome.iso bs=1 skip=2910683136 | \
od -c | less

Is it all 0 ? Is some human readable text to see ?

(It would be more lightweight to inspect Err*-netinst.iso, if you still
have it
dd if=Err-debian-11.5.0-amd64-netinst.iso bs=1 skip=400556032 | \
od -c | less
)

Do you remember how large the ISOs were after their first interrupt ?
Are perhaps the differences between oversize and real size the same
numbers ?
(I compute from your numbers: 4184168 for live and 328163 for netinst.)


There are some bug reports about wget -c on
https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no&src=wget
but none seems to match what you experience.

I'm using debian bullseye with gnome 3.38 and wayland.

That's an ideal base for posting a bug report.

However, I can try your lines the next days.

It would be better for the bug report if the problem could be reproduced
with wget alone.

If you like, I could add the shell selecting in the first line "#...".

It would be really odd if wget -c would behave differently depending
on the shell which started wget.


Have a nice day :)

Thomas

[Thomas Schmitt]

Hi,

So, it seems to be a problem of nautilus on my computer.

wget -c still has a share in this.
But currently i have no ideas how to find out what makes an incomplete
nautilus download so confusing for wget.

dd if=debian-11.5.0-amd64-netinst.iso bs=1 skip=400556032 | od -c | less

Ausgabe.txt

(My proposal to pipe into the pager "less" was meant to avoid the need
for redirection.)

1177740  \0  \0  \0   E   F   I       P   A   R   T  \0  \0 001  \0   \
1177760  \0  \0  \0 302 354 241 223  \0  \0  \0  \0 377 357  \v  \0  \0

This is a GPT header block signature, but not aligned to a full block
of 512 bytes, as GPT prescribes.
The "E" of "E F I" is supposed to be at the start of the block.
24 bytes from there on we have a little-endian 8-byte number which tells
the intended block address:

(gdb) p 0377 + 0357 * 256 + '\v' * 256 * 256
782335

This is the address of the last 512-block of the original ISO:

782335 = 400556032 / 512 - 1

So we have a displaced copy of a GPT backup header with the proper
block address which it would have in the netinst ISO.
Checking this theory:

dd if=debian-11.5.0-amd64-netinst.iso bs=512 skip=782335 | od -c

yields

0000000 E F I P A R T \0 \0 001 \0 \ \0 \0 \0
0000020 302 354 241 223 \0 \0 \0 \0 377 357 \v \0 \0 \0 \0 \0
0000040 001 \0 \0 \0 \0 \0 \0 \0 @ \0 \0 \0 \0 \0 \0 \0
0000060 312 357 \v \0 \0 \0 \0 \0 f 027 253 N 247 r 5 B
0000100 215 256 z 9 353 M 314 313 357 \v \0 \0 \0 \0 \0
0000120 320 \0 \0 \0 200 \0 \0 \0 1 231 o 227 \0 \0 \0 \0
0000140 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
*
0001000

Both, the existence of the copy 327648 after the original end of the ISO
and its non-alignment by -29 (or 483) bytes are hard to explain.


Have a nice day :)

Thomas

[Thomas Schmitt]

Hi,

nautilus is starting a bash terminal.

Do you know which program it runs in there ?
(And with which arguments ?)

After the break it told me a size of 7.900 Byte.
ls -l is telling me 150431900 which is huge for the short download time.

I would guess that the download worked multi-threaded in order to get
different pieces of the ISO simultaneously.
But this does not match the fact that the resulting oversized ISO shows
no differences to the correct ISO in the legitimate size interval.

In the terminal start wget startet at ~250 MB which is disturbing me.

None of these numbers give me ideas in relation to the final oversize
of 480165 bytes.
The only hope for insight seems to be with determining what nautilus
started in that bash window.


Have a nice day :)

Thomas

[Thomas Schmitt]

Hi,

just for the records i report what Gerd Mühlenbruch and i found out
off list:

The reason for the bad dowload was that two wget processes were writing
to the same file.

Gerd had started the first wget by letting nautilus executing it. For
this it created a terminal window with a running wget, which Gerd soon
after closed by means of the desktop.
Then he wanted to continue downloading by starting wget in terminal
window.

Closing the nautilus-made terminal would normally shut down the program
that is running in it. Not so with wget, where the man page says:

Wget is non-interactive, meaning that it can work in the background,
while the user is not logged on. This allows you to start a retrieval
and disconnect from the system, letting Wget finish the work.

We did not explore what exactly causes the oversize of the download result.
The situation of two simultanously working wgets on the same file is ill
enough to serve as explanation.


Have a nice day :)

Thomas

[Steve McIntyre]

Thanks Thomas!

Post by Thomas Schmitt
Hi,
just for the records i report what Gerd Mühlenbruch and i found out
The reason for the bad dowload was that two wget processes were writing
to the same file.
Gerd had started the first wget by letting nautilus executing it. For
this it created a terminal window with a running wget, which Gerd soon
after closed by means of the desktop.
Then he wanted to continue downloading by starting wget in terminal
window.
Closing the nautilus-made terminal would normally shut down the program
Wget is non-interactive, meaning that it can work in the background,
while the user is not logged on. This allows you to start a retrieval
and disconnect from the system, letting Wget finish the work.
We did not explore what exactly causes the oversize of the download result.
The situation of two simultanously working wgets on the same file is ill
enough to serve as explanation.
Have a nice day :)
Thomas