HashMap::new() panics inside UEFI environments that don't support RNG protocol

[ifd3f]

When you construct a HashMap inside a UEFI environment that does not support the RNG protocol, there will be a panic.

Here is a repo containing a minimal working repro of this issue: https://github.com/ifd3f/uefi-hashmap-panic-repro

I modified the uefi-std-example from the uefi-rs repo to construct a hashmap:

#![feature(uefi_std)]

use std::os::uefi as uefi_std;
use std::collections::HashMap;
use uefi::runtime::ResetType;
use uefi::{boot, Handle, Status};

/// Performs the necessary setup code for the `uefi` crate.
fn setup_uefi_crate() {
    let st = uefi_std::env::system_table();
    let ih = uefi_std::env::image_handle();

    // Mandatory setup code for `uefi` crate.
    unsafe {
        uefi::table::set_system_table(st.as_ptr().cast());

        let ih = Handle::from_ptr(ih.as_ptr().cast()).unwrap();
        uefi::boot::set_image_handle(ih);
    }
}

fn main() {
    println!("Hello World from uefi_std");
    setup_uefi_crate();
    println!("UEFI-Version is {}", uefi::system::uefi_revision());

    // Attach panic hook so we can actually read the errors
    std::panic::set_hook(Box::new(|p| {
        let backtrace = std::backtrace::Backtrace::capture();
        println!("{p}");
        println!("{backtrace}");
        loop {
            boot::stall(1_000_000_000)
        }
    }));

    let map = HashMap::<String, String>::new();

    println!("Successfully constructed a hashmap! {map:?}");
    loop {
        boot::stall(1_000_000_000)
    }
}

Running this in QEMU with OVMF firmware, instead of constructing the HashMap and printing it, it produces the following output:

BdsDxe: loading Boot0001 "UEFI QEMU HARDDISK QM00001 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Primary,Master,0x0)
BdsDxe: starting Boot0001 "UEFI QEMU HARDDISK QM00001 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Primary,Master,0x0)
Hello World from uefi_std
                         UEFI-Version is 2.7
                                            panicked at library/std/src/sys/random/uefi.rs:7:53:
                                                                                                failed to generate random data: Os { code: 9223372036854775822, kind: NotFound, message: "The item was not found." }
                                                                                                       disabled backtrace

Based on the panic, my understanding is that this happens because:

1. HashMap uses RandomState to randomly seed itself with the environment's secure RNG
2. The RNG implementation delegates to the UEFI environment's RNG protocol
3. OVMF (or perhaps the way I set up QEMU) does not provide an RNG protocol The VM does not have an RNG, so the EFI environment does not advertise any RNG protocols. Adding -device virtio-rng-pci to the QEMU invocation does allow this code to work.

This is the set of commands used to set up ESP using a virtual VFAT partition, and to launch QEMU with that ESP:

cargo build --target x86_64-unknown-uefi --release
rm -rf esp
mkdir -p esp/efi/boot
cp -r target/x86_64-unknown-uefi/release/uefi-std-example.efi esp/efi/boot/bootx64.efi

qemu-system-x86_64 \
    -D ./qemu.log \
    -drive if=pflash,format=raw,readonly=on,file=./OVMF_CODE.fd \
    -drive if=pflash,format=raw,readonly=on,file=./OVMF_VARS.fd \
    -drive format=raw,file=fat:rw:esp

Meta

rustc --version --verbose:

rustc 1.87.0-nightly (f5a1ef712 2025-03-07)
binary: rustc
commit-hash: f5a1ef7121ad661b5a21a1d02941c8064d54ee0b
commit-date: 2025-03-07
host: x86_64-unknown-linux-gnu
release: 1.87.0-nightly
LLVM version: 20.1.0

Backtrace: I could not produce a backtrace. Setting RUST_BACKTRACE=1 doesn't exactly work the same way in UEFI after all :)

[ryanavella]

HashMap uses RandomState to randomly seed itself with the environment's secure RNG

More specifically, the issue is that HashMap::new returns a map with std::hasher::RandomState as the state. You can sidestep this by using HashMap::with_hasher and providing a hasher that does not rely on random seeding.

[ifd3f]

You can sidestep this by using HashMap::with_hasher and providing a hasher that does not rely on random seeding.

Unfortunately, you can't really do that nicely if a crate is doing the construction. In the specific case where this came up, I was attempting to parse a TOML file using the toml crate, which constructs its own hashmaps. While I theoretically could fork it and every dependency I use and replace all of their hashmap constructions, I don't think you should have to do that to sidestep this problem.

[Ayush1325]

You can sidestep this by using HashMap::with_hasher and providing a hasher that does not rely on random seeding.

Unfortunately, you can't really do that nicely if a crate is doing the construction. In the specific case where this came up, I was attempting to parse a TOML file using the toml crate, which constructs its own hashmaps. While I theoretically could fork it and every dependency I use and replace all of their hashmap constructions, I don't think you should have to do that to sidestep this problem.

Hi, so this happens because the random implementation only works when the protocol is present (here). I remember having a discussion regarding this in the initial implementation, but maybe there is a way to use some register to get random values when the protocol is missing.

On a sidenote, you can get rng to work in QEMU by passing the following argument: -device virtio-rng-pci.
Here is the script I use to run programs on QEMU for testing: https://github.com/Ayush1325/dotfiles/blob/master/.local/bin/uefi-run

I tested your example program with my run script, it does work:

UEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
Mapping table
      FS0: Alias(s):CD0c0:;BLK1:
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/CDROM(0x0)
      FS1: Alias(s):F0d:;BLK2:
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
Press ESC in 5 seconds to skip startup.nsh or any other key to continue.
Starting UEFI Application...
Hello World from uefi_std
UEFI-Version is 2.7
Successfully constructed a hashmap! {}

[the8472]

On x86 we could try rdrand as fallback and I think aarch64 has something similar?

[Ayush1325]

On x86 we could try rdrand as fallback and I think aarch64 has something similar?

So, I did some testing and well, it seems that when rdrand is available, rng protocol is also available. And when not passing -cpu max and/or -device virtio-rng-pci, both are not available. So not sure how useful it is to provide a fallback.

It might be that if the hardware can do random number generation, the protocol will be present, at least for all implementations based on edk2 (which is most of them).

[the8472]

Then it seems like it's just a bad qemu configuration. Is there any reason to support such a setup?

[Ayush1325]

Then it seems like it's just a bad qemu configuration. Is there any reason to support such a setup?

I don't think so. Unless there is real world hardware example, I don't see any use in providing a backup like this.

Fix for qemu: add -cpu max to the args. Or just -device virtio-rng-pci is fine as well.

I did find something interesting. Caching the handle using a static atomic pointer is slower than just calling locate handles every time. So I guess, I don't need to do any optimization there.

[ifd3f]

On a sidenote, you can get rng to work in QEMU by passing the following argument: -device virtio-rng-pci.

Yeah, I ended up figuring that out shortly afterwards. That's on me for not thinking hard enough about where that RNG would come from 😅

I guess if this is merely a result of bad QEMU configuration and there aren't any actual physical devices that have features like this, there isn't really a problem and we can close this.

[seijikun]

I guess if this is merely a result of bad QEMU configuration and there aren't any actual physical devices that have features like this, there isn't really a problem and we can close this.

@ifd3f Unfortunately, this is not limited to QEMU vms.
We have Super Micro Mainboards that don't seem to provide RNG (Super Micro X11SPL-F). They don't use EDK2 but American Megatrends Inc. v3.3 (UEFI Spec Version 2.7)
I'd be very thankful for a fallback!

[Scripter17]

Would a #[global_hasher] thing similar to #[global_allocator] be a viable option?

[the8472]

@seijikun can you check if is_x86_feature_detected!("rdrand") == true on your hardware?

[seijikun]

let isdrand = is_x86_feature_detected!("rdrand");
println!("rdrand available: {}", isdrand);

Yes, that's available. RNG is also not the only core protocol missing from this firmware, it seems.

[the8472]

If there are (fixable) problems with other protocols please file separate issues. This one is just about the random source used by hashmaps.

[Ayush1325]

let isdrand = is_x86_feature_detected!("rdrand");
println!("rdrand available: {}", isdrand);

Yes, that's available. RNG is also not the only core protocol missing from this firmware, it seems.

Hi, do you want to create a PR for this? I did implement this fallback in the past, but well, I guess I never uploaded to github, so it's lost. You basically need to add the fallback here:

rust/library/std/src/sys/random/uefi.rs

Line 26 in 7068c8b

panic!("failed to generate random data");

Alternatively, I can create a PR and ping you for testing by the weekend.

[seijikun]

@Ayush1325 I wrote a working rdrand fallback:

--- a/rust/library/std/src/sys/random/uefi.rs
+++ b/rust/library/std/src/sys/random/uefi.rs
@@ -19,5 +19,21 @@ pub fn fill_bytes(bytes: &mut [u8]) {
         }
     }
 
+    // on x86, try using rdrand directly as a fallback
+    if is_x86_feature_detected!("rdrand") {
+        for chunk in bytes.chunks_mut(core::mem::size_of::<u64>()) {
+            let mut rand_val: u64 = 0;
+            unsafe {
+                if core::arch::x86_64::_rdrand64_step(&mut rand_val) == 0 {
+                    panic!("failed to generate random data using rdrand");
+                }
+            }
+
+            let bytes = rand_val.to_le_bytes();
+            chunk.copy_from_slice(&bytes[..chunk.len()]);
+        }
+        return;
+    }
+
     panic!("failed to generate random data");
 }

I tested the block I added in isolation (outside of the std, as part of my UEFI application), against the offending machine.
But I would really prefer (and be thankful!) if you could create a PR for this.