Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pi5 failed to contact RP1 firmware #6642

Open
llllllllqq opened this issue Feb 1, 2025 · 22 comments
Open

Pi5 failed to contact RP1 firmware #6642

llllllllqq opened this issue Feb 1, 2025 · 22 comments

Comments

@llllllllqq
Copy link

llllllllqq commented Feb 1, 2025

Describe the bug

dmesg says:

...
rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware [ 3.957885] rp1-pio: probe of 1f00178000.pio failed with error -2
...

Although nothing strange occurred.
The OS still works fine without reboot.

Steps to reproduce the behaviour

It randomly occurs after sudo reboot --halt and then manually press the on-board power button to power on (with default eeprom config).

It also randomly occurs when use raspi-config to change power off behavior to "full power off" and then reboot/poweroff. (Sorry I cannot remember whether it is reboot or poweroff)

Update: It also occurs after a suddenly plug-out the usb-c power cable and then plug-in back.
It seems only poweroff-related behaviors will trigger this bug. I tested 30 times reboot, no bugs occurred.
Update2: reboot can indeed also trigger this bug, only less frequent.

Device (s)

Raspberry Pi 5

System

Raspberry Pi reference 2024-11-19 Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 891df1e21ed2b6099a2e6a13e26c91dea44b34d4, stage4

2025/01/22 00:16:51
Copyright (c) 2012 Broadcom version a7753063 (release) (embedded)

Linux rpi5 6.6.74+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.74-1+rpt1 (2025-01-27) aarch64 GNU/Linux

Logs

1.txt

Additional context

Can it be safely ignored?

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

That's interesting - it looks like a dependency/ordering issue, and not one I've seen before:

[    3.922665] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    3.922671] rp1-pio: probe of 1f00178000.pio failed with error -2
...
[    3.932284] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

I get:

[   13.445656] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[   13.550428] rp1-pio 1f00178000.pio: Created instance as pio0

The code for communicating with the firmware exists in the firmware module, so I thought that would enforce an ordering. Some investigation is required.

@llllllllqq
Copy link
Author

That's interesting - it looks like a dependency/ordering issue, and not one I've seen before:

[    3.922665] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    3.922671] rp1-pio: probe of 1f00178000.pio failed with error -2
...
[    3.932284] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

I get:

[   13.445656] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[   13.550428] rp1-pio 1f00178000.pio: Created instance as pio0

The code for communicating with the firmware exists in the firmware module, so I thought that would enforce an ordering. Some investigation is required.

Thanks for your reply.
Here is my dmesg when the rp1&pio works fine.
I hope it can help to investigate this issue.

normal.txt

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

Yes - in the working case the order is reversed:

[    3.955929] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[    3.961971] vc4-drm axi:gpu: bound 107c580000.hvs (ops vc4_hvs_ops [vc4])
[    3.967397] rp1-pio 1f00178000.pio: Created instance as pio0

@aBUGSworstnightmare-rpi
Copy link
Contributor

[ 5.557565] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware [ 5.557571] rp1-pio: probe of 1f00178000.pio failed with error -2 [ 5.558534] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

Seeing that on reboot as well /custom kernel on a Pi5 DUT, trying to get an overlay to work)

Linux Pi5uSDDUT 6.6.74-v8-16k+ #1 SMP PREEMPT Sun Feb 2 10:41:23 CET 2025 aarch64 GNU/Linux Revision : c04170 Serial : cb30eb1fd7e7e672 Model : Raspberry Pi 5 Model B Rev 1.0

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

It's really strange. I can't get it to fail in that way without hacks - even with a massive delay in the firmware driver, the pio driver waits. Anyway, I'm just testing a fix.

@aBUGSworstnightmare-rpi
Copy link
Contributor

seems quite random. Seen it frequently on reboots (running 'sudo reboot' on CLI)

pelwell added a commit to pelwell/linux that referenced this issue Feb 3, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit to pelwell/linux that referenced this issue Feb 3, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
@mayo228
Copy link

mayo228 commented Feb 3, 2025

is there a way to fix this? I keep getting this error and I can't boot the OS, should I be worried that I will loose data on my raspberry?

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

I don't understand why this affecting some people while I can't reproduce it without hacking the driver, but regardless, #6645 should fix the issue. After about 40 minutes you should be able to install a trial kernel with sudo rpi-update pulls/6645, after first backing up any important data.

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

And no, you shouldn't worry - it would only affect you if you wanted to use piolib to drive the GPIOs.

@aBUGSworstnightmare-rpi
Copy link
Contributor

aBUGSworstnightmare-rpi commented Feb 3, 2025

I have no idea if and how below is related but I just wanted to state that I'm not making use of piolib for accessing GPIO's.

[   13.678330] ieee80211 phy0: brcmf_p2p_send_action_frame: Unknown Frame: category 0xa, action 0x8
[   15.374753] Bluetooth: RFCOMM TTY layer initialized
[   15.374765] Bluetooth: RFCOMM socket layer initialized
[   15.374772] Bluetooth: RFCOMM ver 1.11
[   27.660651] brcmfmac: brcmf_set_channel: set chanspec 0xd022 fail, reason -52
[   27.768681] brcmfmac: brcmf_set_channel: set chanspec 0xd026 fail, reason -52
[   27.872656] brcmfmac: brcmf_set_channel: set chanspec 0xd02a fail, reason -52
[   27.976661] brcmfmac: brcmf_set_channel: set chanspec 0xd02e fail, reason -52
[   29.636663] brcmfmac: brcmf_set_channel: set chanspec 0xd090 fail, reason -52
[   29.636875] brcmfmac: brcmf_set_channel: set chanspec 0xd095 fail, reason -52
[   29.637196] brcmfmac: brcmf_set_channel: set chanspec 0xd099 fail, reason -52
[   29.637400] brcmfmac: brcmf_set_channel: set chanspec 0xd09d fail, reason -52
[   29.637594] brcmfmac: brcmf_set_channel: set chanspec 0xd0a1 fail, reason -52
[   29.637791] brcmfmac: brcmf_set_channel: set chanspec 0xd0a5 fail, reason -52

@pelwell
Copy link
Contributor

pelwell commented Feb 3, 2025

I have no idea if and how below is related

You should know that WiFi and Bluetooth information and warning messages are unrelated.

pelwell added a commit to pelwell/linux that referenced this issue Feb 3, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
@llllllllqq
Copy link
Author

It seems after applying sudo rpi-update pulls/6645, this issue disappeared. (I tested about 10 times reboot/halt/suddenly poweroff). Thanks for your work! If anything strange found, I'll post here.

BTW, can I safely stay at this firmware and wait for the next stable release offered by apt? Or do I need a sudo apt install --reinstall raspi-firmware to restore first before the next stable release?

@pelwell
Copy link
Contributor

pelwell commented Feb 4, 2025

You should be fine with this kernel until the next proper release - I'm not aware of any new issues.

pelwell added a commit to pelwell/linux that referenced this issue Feb 4, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit to pelwell/linux that referenced this issue Feb 4, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: raspberrypi#6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
pelwell added a commit that referenced this issue Feb 4, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
@randomMesh
Copy link

I can confirm the error happens on my PI 5 too:

[    0.692347] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    0.692370] rp1-pio: probe of 1f00178000.pio failed with error -2

I am running the latest firmware:

BOOTLOADER: up to date
   CURRENT: Wed 22 Jan 00:16:51 UTC 2025 (1737505011)
    LATEST: Wed 22 Jan 00:16:51 UTC 2025 (1737505011)
   RELEASE: latest (/usr/lib/firmware/raspberrypi/bootloader-2712/latest)
            Use raspi-config to change the release.

@helpful-emu57
Copy link

A week ago I also started receiving these errors during bootup:

rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware

At the same time I also started getting protocol errors for the wireless mouse and wireless keyboard
logitech-hidpp-device - hidpp_root_get_ protocol_version: received protocol error 0x08
(Although - both devices continued to work appropriately.)

Changing to wired devices there are no mouse or keyboard errors but still have the rp1-pio error

I have performed the "sudo rpi-update pulls/6645" and no change was observed.

@pelwell
Copy link
Contributor

pelwell commented Feb 9, 2025

What does rpi-eeprom-update show?

@nickongit
Copy link

How do you do "sudo rpi-update pulls/6645" if you can't boot into it?

My new pi-500 seems to be bricked with this error. I just tried to install moonlight-qt and performed the "sudo reboot" and can't get it to boot up.

@pelwell
Copy link
Contributor

pelwell commented Feb 10, 2025

My new pi-500 seems to be bricked with this error. I just tried to install moonlight-qt and performed the "sudo reboot" and can't get it to boot up.

From the detailed description of your problem I'm going to guess that:

  1. You had a working Pi 500.
  2. You installed some new software.
  3. It no longer boots.
  4. The only error message you can see on screen is the one about the RP1 firmware.

As I've explained above, the only effect of that error is to make PIO unavailable - it will not cause the boot to fail. I suggest you edit cmdline.txt in another computer, replacing quiet with ignore_loglevel, then try again. You should see much more output, and that may make it clearer what is going wrong.

popcornmix pushed a commit that referenced this issue Feb 10, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
popcornmix pushed a commit that referenced this issue Feb 10, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
popcornmix pushed a commit that referenced this issue Feb 10, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
popcornmix pushed a commit that referenced this issue Feb 10, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
popcornmix pushed a commit that referenced this issue Feb 10, 2025
If the RP1 firmware has reported an error then return that from the PIO
probe function, otherwise defer the probing.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
popcornmix pushed a commit that referenced this issue Feb 10, 2025
To avoid pointless retries, let the probe function succeed if the
firmware interface is configured correctly but the firmware is
incompatible. The value of the private drvdata field holds the outcome.

Link: #6642

Signed-off-by: Phil Elwell <[email protected]>
@helpful-emu57
Copy link

RE: follow-up 3 days ago - Boot RP1 Firmware Error

  Excerpt From Boot Log
  [    0.858841] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
  [    0.858866] rp1-pio: probe of 1f00178000.pio failed with error -2
  
  Is location 1f00178000 reserved for particular hardware? 
  What is an "rp1-pio error -2" error ?

RE: Wireless Device Errors
The protocol error is gone when using original wireless mouse, keyboard and dongle
(would similar make/model with different dongle cause error?)

@pelwell
Copy link
Contributor

pelwell commented Feb 12, 2025

And my response three days ago was:

What does rpi-eeprom-update show?

You will see that harmless error until you update the EEPROM firmware.

@nickongit
Copy link

Apologies, thank you for the response, i meant to reply yesterday when i had a chance to fix it.

The root cause of no boot was a cron entry to start moonlight-qt @reboot.

Strangely i accidentally found the solution to fixing it by tryinf another monitor, which resulted in the pi booting before i plugged in the monitor. Then i could plug in monitor and take action removing the cron entry.

As you suggested the error is always there and was not the cause of the boot problem, just the last entry visible on the boot screen.

@pelwell
Copy link
Contributor

pelwell commented Feb 12, 2025

I think it might be best if the error was demoted to a warning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants