ostree.fedoraproject.org/iot/ Returns Forbidden, causing 01_update_platforms_check.sh failing #68
Comments
|
When logging in to my rpi4 running fedora-iot I also noticed this issue: |
|
I am seeing this same failure. The script fails when the system is booting but runs successfully when the system is online. |
|
Maybe this runs before network is up? |
|
Also, what hardware do you run this on? Does it have a RTC? |
|
I am currently testing this on a Raspberry Pi 4 but I also see this same failure on a Supermicro system in my environment that has an Intel processor. Here is the relevant entry from the journal: I see NetworkManager entries above this failure in the journal which seems to indicate that networking is online when this script runs. |
|
I'm using it on J4125. It triggers when I ssh to the device and network is working properly. Also https://ostree.fedoraproject.org/iot/ returns 403 on any of my devices. I doubt this may be a regional issue caused by reverse proxy configuration. |
|
Found |
|
The URL seems to either be broken, or not correct. I'd expect it to be reachable though, but I guess rpm-ostree really only needs access to the mirrorlist to get access to the contents. @nullr0ute what say you? Should we just change the test to check availability of In the meantime, what are the outputs of
Note that the test isn't run when logging in, but when upgrading. It's only the MotD message previously generated by the test that is shown upon login. |
|
|
|
For comparison, in the Fedora CoreOS case, both |
Actually, it seems that
That sounds like a hack workaround rather than a fix. It's abusing the fact that the check treats a redirect response as successful rather than that it's actually returning valid data. |
|
Can confirm the issue still exists with a fresh install of Fedora IoT 37. The connection to the update servers are 100% fine (without any reverse proxy or other stuffs) but the check always fails. And, indeed, browsing to This bug effectively makes the check bogus, and since there's no config option to skip any checks, we're now stuck with a error message that appears every time the system boots. Modify the script to check for |
|
Just chiming in to note that this is still an issue on a F37 IoT installation I deployed yesterday. I'm greeted with the same error as @Iolaum in comment #2 when connecting trough ssh. Here's my installation as of now: There is something weird with the URL in the mirror repo though, https://ostree.fedoraproject.org/iot/ returns 403: while https://ostree.fedoraproject.org/iot/config returns 200 So, to me it seems like the recent commit based on #93 doesn't work @miabbott, unless this is unrelated to that. My bash-skills are too limited to debug this. |
|
@natterangell I'm not able to reproduce this: Is the problem repeatable in your environment? I agree that the infrastructure of the ostree repo is setup in such a way where you can get misleading HTTP error codes if you don't hit the right endpoint. I've thought about how to make the |
|
Well, this is my first and only client on Fedora IoT. It's reproducible in the sense that it happens consistently on every reboot and after a reinstall. |
|
|
I just did a fresh install of Fedora IoT 37 and I'm still not able to reproduce this issue: Even after upgrading to the latest commit, I'm still not able to reproduce this: Could you copy the script to a different system and see if the script works there? I wonder if there is something specific to your environment. |
|
I can confirm that this error message no longer appears as of the update I performed today. |
|
Unfortunately the update does not work for my case, at least partially. Just updated my Fedora IoT 37, and got the latest version of I don't really understand what's going on right now, and I can only assume that we have different formats for |
|
@EZForever I think what you are encountering is discussed in #98 |
|
Seen from the outside, this seems like a real non-issue. What's holding this up? Tried a freshly downloaded Fedora IOT ISO this weekend, and I still get this same 403 error, both before and after rpm-ostree upgrade. It makes the whole IOT spin look like alpha software, and gives the impression that this is something Fedora wont commit to if even such a (for the outsider) basic looking bug as this can stay unfixed for the most part of a year. I "fixed" it by editing the fedora-iot.conf file and amending the url line to end with /iot/config - if that is the correct fix, why isn't it part of greenboot already? |
I've isolated the reproduction case - it's specifically the trailing slash on the url that's causing this to fail. Examples follow: Success case - no trailing slash[remote "fedora-iot"]
url=https://ostree.fedoraproject.org/iot
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/
contenturl=mirrorlist=https://ostree.fedoraproject.org/iot/mirrorlist[root@k4 ~]# bash -x /usr/lib/greenboot/check/wanted.d/01_update_platforms_check.sh
+ set -e
+ REPOS_DIRECTORY=/etc/ostree/remotes.d
+ URLS_WITH_PROBLEMS=()
+ [[ ! -d /etc/ostree/remotes.d ]]
+ get_update_platform_urls
+ mapfile -t UPDATE_PLATFORM_URLS
++ grep -P -ho 'http[s]?.*' /etc/ostree/remotes.d/fedora-iot.conf
+ [[ 2 -eq 0 ]]
+ assert_update_platforms_are_responding
+ for UPDATE_PLATFORM_URL in "${UPDATE_PLATFORM_URLS[@]}"
++ curl -o /dev/null -Isw '%{http_code}\n' https://ostree.fedoraproject.org/iot
+ HTTP_STATUS=301
+ [[ 301 == 2* ]]
+ [[ 301 == 3* ]]
+ for UPDATE_PLATFORM_URL in "${UPDATE_PLATFORM_URLS[@]}"
++ curl -o /dev/null -Isw '%{http_code}\n' https://ostree.fedoraproject.org/iot/mirrorlist
+ HTTP_STATUS=200
+ [[ 200 == 2* ]]
+ [[ 0 -eq 0 ]]
+ echo 'We can connect to all update platforms'
We can connect to all update platforms
+ exit 0Failure case - trailing slash[remote "fedora-iot"]
url=https://ostree.fedoraproject.org/iot/
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/
contenturl=mirrorlist=https://ostree.fedoraproject.org/iot/mirrorlist[root@k4 ~]# bash -x /usr/lib/greenboot/check/wanted.d/01_update_platforms_check.sh
+ set -e
+ REPOS_DIRECTORY=/etc/ostree/remotes.d
+ URLS_WITH_PROBLEMS=()
+ [[ ! -d /etc/ostree/remotes.d ]]
+ get_update_platform_urls
+ mapfile -t UPDATE_PLATFORM_URLS
++ grep -P -ho 'http[s]?.*' /etc/ostree/remotes.d/fedora-iot.conf
+ [[ 2 -eq 0 ]]
+ assert_update_platforms_are_responding
+ for UPDATE_PLATFORM_URL in "${UPDATE_PLATFORM_URLS[@]}"
++ curl -o /dev/null -Isw '%{http_code}\n' https://ostree.fedoraproject.org/iot/
+ HTTP_STATUS=403
+ [[ 403 == 2* ]]
+ [[ 403 == 3* ]]
+ URLS_WITH_PROBLEMS+=("$UPDATE_PLATFORM_URL")
+ for UPDATE_PLATFORM_URL in "${UPDATE_PLATFORM_URLS[@]}"
++ curl -o /dev/null -Isw '%{http_code}\n' https://ostree.fedoraproject.org/iot/mirrorlist
+ HTTP_STATUS=200
+ [[ 200 == 2* ]]
+ [[ 1 -eq 0 ]]
+ echo 'There are problems connecting with the following URLs:'
There are problems connecting with the following URLs:
+ echo https://ostree.fedoraproject.org/iot/
https://ostree.fedoraproject.org/iot/
+ exit 1This is extra tricky because there's a redirect from If you're running into this, strip the trailing slash off the url in If there's any way to reach the folks running ostree.fedoraproject.org, the combination of the redirection and the 403 on |
|
I think this is still an issue. Using Fedora IOT and the latest package version: $ cat /etc/os-release
NAME="Fedora Linux"
VERSION="40.20241010.0 (IoT Edition)"
$ rpm -qa | grep greenboot
greenboot-0.15.6-1.fc40.x86_64
greenboot-default-health-checks-0.15.6-1.fc40.x86_64After a fresh install, the script initially failed: Oct 14 12:20:05 Fedora-IOT rpm-ostree[1518]: client(id:cli dbus:1.16 unit:greenboot-healthcheck.service uid:0) added; new total=1
Oct 14 12:20:05 Fedora-IOT rpm-ostree[1518]: client(id:cli dbus:1.16 unit:greenboot-healthcheck.service uid:0) vanished; remaining=0
Oct 14 12:20:05 Fedora-IOT rpm-ostree[1518]: In idle state; will auto-exit in 64 seconds
Oct 14 12:20:05 Fedora-IOT greenboot[1450]: Script '02_watchdog.sh' SUCCESS
Oct 14 12:20:05 Fedora-IOT greenboot[1450]: Running Wanted Health Check Scripts...
Oct 14 12:20:05 Fedora-IOT 00_wanted_scripts_start.sh[1895]: Running greenboot Wanted Health Check Scripts
Oct 14 12:20:05 Fedora-IOT greenboot[1450]: Script '00_wanted_scripts_start.sh' SUCCESS
Oct 14 12:20:05 Fedora-IOT agetty[1421]: failed to open credentials directory
Oct 14 12:20:06 Fedora-IOT 01_update_platforms_check.sh[1900]: There are problems connecting with the following URLs:
Oct 14 12:20:06 Fedora-IOT 01_update_platforms_check.sh[1900]: https://ostree.fedoraproject.org/iot/ https://ostree.fedoraproject.org/iot/mir>
Oct 14 12:20:06 Fedora-IOT greenboot[1450]: Script '01_update_platforms_check.sh' FAILURE (exit code '1'). Continuing...Running the script manually works (via SSH after boot): sudo /usr/lib/greenboot/check/required.d/01_repository_dns_check.sh
All domains have resolved correctly
sudo /usr/lib/greenboot/check/wanted.d/01_update_platforms_check.sh
We can connect to all update platformsI tried editing The same fresh install was previously working a few days ago. So I am not really sure what is going on. I am trying to further investigate and narrow-down this issue. Related Github issues: |
I'm not sure the issue should be created here or it's a reverse proxy configuration issue. The way it checks url valid is
And access https://ostree.fedoraproject.org/iot/ will return 403. Although ostree updating is still working correctly.
The text was updated successfully, but these errors were encountered: