Bug#1024346: cdrom: debian-11.5.0-amd64-netinst.iso boots to Grub CLI on Dell Optiplex 5090

[Rob Klingsten]

Package: cdrom
Severity: important
Tags: d-i

Dear Maintainer,

*** Reporter, please consider answering these questions, where appropriate ***

* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?

*** End of the template - remove these template lines ***

Downloaded debian-11.5.0-amd64-netinst.iso, verified SHA512 signature and flashed to USB stick (dd if=<debian.iso> of=/dev/sdb). The
Dell Optiplex 5090 is a UEFI-only system. In the BIOS, I previously disabled TPM, Secure Boot and Absolute (computer Lojack).

Booting from the netinst USB stick, the computer boots into the Grub CLI. 'ls' shows the following:

(proc) (hd0) (hd0,gpt4) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2)

There does not appear to be any usable partition detected on the USB stick that contains a kernel. The contents of (cd0,msdos2) are
just an 'efi' directory.

I have tried multiple USB sticks, downloaded the ISO several times all with a good SHA512, tried dd and also cp <iso> /dev/sdb, makes
no difference. I've tried the live Gnome image as well, same problem.

I expect the computer to boot properly into the Debian installer.

[Andrew M.A. Cater]

Post by Rob Klingsten
Package: cdrom
Severity: important
Tags: d-i
Dear Maintainer,
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
Downloaded debian-11.5.0-amd64-netinst.iso, verified SHA512 signature and flashed to USB stick (dd if=<debian.iso> of=/dev/sdb). The
Dell Optiplex 5090 is a UEFI-only system. In the BIOS, I previously disabled TPM, Secure Boot and Absolute (computer Lojack).
(proc) (hd0) (hd0,gpt4) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2)
There does not appear to be any usable partition detected on the USB stick that contains a kernel. The contents of (cd0,msdos2) are
just an 'efi' directory.
I have tried multiple USB sticks, downloaded the ISO several times all with a good SHA512, tried dd and also cp <iso> /dev/sdb, makes
no difference. I've tried the live Gnome image as well, same problem.
I expect the computer to boot properly into the Debian installer.

First of all:

It may be better to use a longer dd line and also to use the unofficial
firmware image available at https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/11.5.0+nonfree/amd64/iso-cd/firmware-11.5.0-amd64-netinst.iso

dd if=firmware-11.5.0-amd64-netinst.iso of=/dev/sdb bs=4M oflag=sync status=progress

That makes absolutely sure that the transfer is synced to ensure that it is
written to the stick and also gives you some idea of how well the transfer
is going.

Using the firmware .iso will potentially solve any problems with missing
firmwware.

The writing to a stick *should* work well.

All the very best, as ever,

Andy Cater

[Andrew M.A. Cater]

Rob - see below, you might want to subscribe to the bug too.

Suggestion is to use firmware .iso and a more verbose dd line to ensure
you've actually written the whole image correctly.

Also, I would suggest enabling TPM and secure boot unless you are *absolutely*
sure that you don't need them. Secure boot is well supported in Debian.

Hope this helps,

Andy Cater

Post by Andrew M.A. Cater

Post by Rob Klingsten
Package: cdrom
Severity: important
Tags: d-i
Dear Maintainer,
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
Downloaded debian-11.5.0-amd64-netinst.iso, verified SHA512 signature and flashed to USB stick (dd if=<debian.iso> of=/dev/sdb). The
Dell Optiplex 5090 is a UEFI-only system. In the BIOS, I previously disabled TPM, Secure Boot and Absolute (computer Lojack).
(proc) (hd0) (hd0,gpt4) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2)
There does not appear to be any usable partition detected on the USB stick that contains a kernel. The contents of (cd0,msdos2) are
just an 'efi' directory.
I have tried multiple USB sticks, downloaded the ISO several times all with a good SHA512, tried dd and also cp <iso> /dev/sdb, makes
no difference. I've tried the live Gnome image as well, same problem.
I expect the computer to boot properly into the Debian installer.

It may be better to use a longer dd line and also to use the unofficial
firmware image available at https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/11.5.0+nonfree/amd64/iso-cd/firmware-11.5.0-amd64-netinst.iso
dd if=firmware-11.5.0-amd64-netinst.iso of=/dev/sdb bs=4M oflag=sync status=progress
That makes absolutely sure that the transfer is synced to ensure that it is
written to the stick and also gives you some idea of how well the transfer
is going.
Using the firmware .iso will potentially solve any problems with missing
firmwware.
The writing to a stick *should* work well.
All the very best, as ever,
Andy Cater

I'm not sure but maybe Rob Klingsten is not on the list so I'm not sure that
he has read your reply please consider to re-send your answer to
kind regards
--
Franco Martelli

[r***@tekhax.io]

Thanks, after messing with it for quite awhile, I finally got it to work with the standard ISO.

I booted with the Arch live image and did:

wipefs -a /dev/nvme0n1
dd if=/dev/zero of=/dev/nvme0n1 bs=512 count=100000

then I used efibootmgr to delete all existing entries.

Once I did that, the netinst booted into the installer immediately. Not sure if it was the actual existence of valid partitions on the drive, or just the existence of EFI entries in the table.

I can further test to see which scenario it is. I would still consider this a bug?




------- Original Message -------

Post by Andrew M.A. Cater
Rob - see below, you might want to subscribe to the bug too.
Suggestion is to use firmware .iso and a more verbose dd line to ensure
you've actually written the whole image correctly.
Also, I would suggest enabling TPM and secure boot unless you are absolutely
sure that you don't need them. Secure boot is well supported in Debian.
Hope this helps,
Andy Cater

Post by Andrew M.A. Cater

Post by Rob Klingsten
Package: cdrom
Severity: important
Tags: d-i
Dear Maintainer,
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
Downloaded debian-11.5.0-amd64-netinst.iso, verified SHA512 signature and flashed to USB stick (dd if=<debian.iso> of=/dev/sdb). The
Dell Optiplex 5090 is a UEFI-only system. In the BIOS, I previously disabled TPM, Secure Boot and Absolute (computer Lojack).
(proc) (hd0) (hd0,gpt4) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2)
There does not appear to be any usable partition detected on the USB stick that contains a kernel. The contents of (cd0,msdos2) are
just an 'efi' directory.
I have tried multiple USB sticks, downloaded the ISO several times all with a good SHA512, tried dd and also cp <iso> /dev/sdb, makes
no difference. I've tried the live Gnome image as well, same problem.
I expect the computer to boot properly into the Debian installer.

It may be better to use a longer dd line and also to use the unofficial
firmware image available at https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/11.5.0+nonfree/amd64/iso-cd/firmware-11.5.0-amd64-netinst.iso
dd if=firmware-11.5.0-amd64-netinst.iso of=/dev/sdb bs=4M oflag=sync status=progress
That makes absolutely sure that the transfer is synced to ensure that it is
written to the stick and also gives you some idea of how well the transfer
is going.
Using the firmware .iso will potentially solve any problems with missing
firmwware.
The writing to a stick should work well.
All the very best, as ever,
Andy Cater

I'm not sure but maybe Rob Klingsten is not on the list so I'm not sure that
he has read your reply please consider to re-send your answer to
kind regards
--
Franco Martelli

[Steve McIntyre]

Hi Rob,

I see Andy has been helping you!

Post by r***@tekhax.io
Thanks, after messing with it for quite awhile, I finally got it to work with the standard ISO.
wipefs -a /dev/nvme0n1
dd if=/dev/zero of=/dev/nvme0n1 bs=512 count=100000
then I used efibootmgr to delete all existing entries.
Once I did that, the netinst booted into the installer
immediately. Not sure if it was the actual existence of valid
partitions on the drive, or just the existence of EFI entries in the
table.

If your system has (had) existing EFI boot entries, then the firmware
would normally attempt to boot those. AIUI you selected the USB stick
and that failed to boot?

The partitioning on Debian images is slightly complex, to make them
work as a so-called "isohybrid". (This means that you can use the same
image both when written to optical media and when written to a USB
stick.) But the partitions should still show up. For example, looking
at the netinst image file here:

$ fdisk -l debian-11.5.0-amd64-netinst.iso
Disk debian-11.5.0-amd64-netinst.iso: 382 MiB, 400556032 bytes, 782336 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5004a58b

Device Boot Start End Sectors Size Id Type
debian-11.5.0-amd64-netinst.iso1 * 0 782335 782336 382M 0 Empty
debian-11.5.0-amd64-netinst.iso2 4064 9247 5184 2.5M ef EFI (FAT-12/16/32)

The first partition covers the whole of the image; the second one is
*just* the EFI boot setup that you've seen already. If you're only
seeing the second partition then it appears there is some other
problem here.

Checking your original report here, you said you wrote to the USB
stick using dd if=<debian.iso> of=/dev/sdb. Did you run "sync" or
similar to make 100% sure that the image was all flushed to the USB
stick before removing it / booting it? Unless you tell it otherwise,
Linux will cache writes to USB drives and it can appear that writes
have completed well before the data is actually written to the
drive. This is a common cause of confusion for people in this
situation, I'm afraid.

Andy already mentioned a different way to force writing data, using
the "oflag=sync" option to dd. Using that with "bs=4M" should also
give good performance when writing out an image to a USB stick.

Could you possibly retry this and check if it works for you please?

[r***@tekhax.io]

Thanks for the detailed message. I started everything over to try to reproduce the problem.

I wiped out the NVMe drive again (wipefs -a and dd if=/dev/zero of=/dev/sdb bs=512 count=100000), and re-installed Pop!OS 22.04 non-NVidia version, which was successful. The computer was operating normally and booting into Pop!OS, no problem.

I then downloaded a new Debian 11.5.0 netinst ISO, checked the SHA512, wrote it to the USB drive with extra parameters mentioned earlier.

I powered off the computer, inserted the USB 3.0 stick into a USB 3.0 port and powered on. I hit F12 to reach the Dell one-time boot menu and chose the USB stick to boot from.

It immediately boots into the Grub CLI as I described before. So either the presence of the Pop!OS partition or the presence of the entry in the UEFI boot table seems to be confusing the Debian USB stick.






------- Original Message -------

Post by Steve McIntyre
Hi Rob,
I see Andy has been helping you!

Post by r***@tekhax.io
Thanks, after messing with it for quite awhile, I finally got it to work with the standard ISO.
wipefs -a /dev/nvme0n1
dd if=/dev/zero of=/dev/nvme0n1 bs=512 count=100000
then I used efibootmgr to delete all existing entries.
Once I did that, the netinst booted into the installer
immediately. Not sure if it was the actual existence of valid
partitions on the drive, or just the existence of EFI entries in the
table.

If your system has (had) existing EFI boot entries, then the firmware
would normally attempt to boot those. AIUI you selected the USB stick
and that failed to boot?
The partitioning on Debian images is slightly complex, to make them
work as a so-called "isohybrid". (This means that you can use the same
image both when written to optical media and when written to a USB
stick.) But the partitions should still show up. For example, looking
$ fdisk -l debian-11.5.0-amd64-netinst.iso
Disk debian-11.5.0-amd64-netinst.iso: 382 MiB, 400556032 bytes, 782336 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5004a58b
Device Boot Start End Sectors Size Id Type
debian-11.5.0-amd64-netinst.iso1 * 0 782335 782336 382M 0 Empty
debian-11.5.0-amd64-netinst.iso2 4064 9247 5184 2.5M ef EFI (FAT-12/16/32)
The first partition covers the whole of the image; the second one is
just the EFI boot setup that you've seen already. If you're only
seeing the second partition then it appears there is some other
problem here.
Checking your original report here, you said you wrote to the USB
stick using dd if=<debian.iso> of=/dev/sdb. Did you run "sync" or
similar to make 100% sure that the image was all flushed to the USB
stick before removing it / booting it? Unless you tell it otherwise,
Linux will cache writes to USB drives and it can appear that writes
have completed well before the data is actually written to the
drive. This is a common cause of confusion for people in this
situation, I'm afraid.
Andy already mentioned a different way to force writing data, using
the "oflag=sync" option to dd. Using that with "bs=4M" should also
give good performance when writing out an image to a USB stick.
Could you possibly retry this and check if it works for you please?
--
< liw> everything I know about UK hotels I learned from "Fawlty Towers"

[Steve McIntyre]

Post by r***@tekhax.io
Thanks for the detailed message. I started everything over to try to reproduce the problem.

Thanks for that!

Post by r***@tekhax.io
I wiped out the NVMe drive again (wipefs -a and dd if=/dev/zero
of=/dev/sdb bs=512 count=100000), and re-installed Pop!OS 22.04
non-NVidia version, which was successful. The computer was operating
normally and booting into Pop!OS, no problem.

OK, cool.

Post by r***@tekhax.io
I then downloaded a new Debian 11.5.0 netinst ISO, checked the
SHA512, wrote it to the USB drive with extra parameters mentioned
earlier.
I powered off the computer, inserted the USB 3.0 stick into a USB 3.0
port and powered on. I hit F12 to reach the Dell one-time boot menu
and chose the USB stick to boot from.
It immediately boots into the Grub CLI as I described before. So
either the presence of the Pop!OS partition or the presence of the
entry in the UEFI boot table seems to be confusing the Debian USB
stick.

Right.

By pure lucky coincidence, yesterday another Debian developer saw a
very similar issue on a Lenovo machine (see bug #1024720), and we
debugged his problem together. I'm thinking that you may have another
instance here. Could you please check for me: does the Pop!OS
installation include a file in the path "/.disk/info" on any of its
partitions?

If so, that will be the cause of the problem here. If so, if you
(temporarily) rename that file then you should find that the Debian
installer will work just fine.

[r***@tekhax.io]

------- Original Message -------

Post by Steve McIntyre

Post by r***@tekhax.io
Thanks for the detailed message. I started everything over to try to reproduce the problem.

Thanks for that!

Post by r***@tekhax.io
I wiped out the NVMe drive again (wipefs -a and dd if=/dev/zero
of=/dev/sdb bs=512 count=100000), and re-installed Pop!OS 22.04
non-NVidia version, which was successful. The computer was operating
normally and booting into Pop!OS, no problem.

OK, cool.

Post by r***@tekhax.io
I then downloaded a new Debian 11.5.0 netinst ISO, checked the
SHA512, wrote it to the USB drive with extra parameters mentioned
earlier.
I powered off the computer, inserted the USB 3.0 stick into a USB 3.0
port and powered on. I hit F12 to reach the Dell one-time boot menu
and chose the USB stick to boot from.
It immediately boots into the Grub CLI as I described before. So
either the presence of the Pop!OS partition or the presence of the
entry in the UEFI boot table seems to be confusing the Debian USB
stick.

Right.
By pure lucky coincidence, yesterday another Debian developer saw a
very similar issue on a Lenovo machine (see bug #1024720), and we
debugged his problem together. I'm thinking that you may have another
instance here. Could you please check for me: does the Pop!OS
installation include a file in the path "/.disk/info" on any of its
partitions?
If so, that will be the cause of the problem here. If so, if you
(temporarily) rename that file then you should find that the Debian
installer will work just fine.
--
"...In the UNIX world, people tend to interpret `non-technical user'
as meaning someone who's only ever written one device driver." -- Daniel Pead

That seems to have been the problem. Thanks!

[Steve McIntyre]

Post by r***@tekhax.io
That seems to have been the problem. Thanks!

Awesome, thanks for confirming. I've just pushed a fix to the
debian-cd build scripts which should fix this for future builds.

[Debian Bug Tracking System]

Your message dated Sun, 15 Jan 2023 12:35:05 +0000
with message-id <E1pH2E5-009iaW-***@fasolo.debian.org>
and subject line Bug#1024346: fixed in debian-cd 3.1.36
has caused the Debian Bug report #1024346,
regarding cdrom: debian-11.5.0-amd64-netinst.iso boots to Grub CLI on Dell Optiplex 5090
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ***@bugs.debian.org
immediately.)