Repo metadata GPG key isn't working for version 3.0

Hi Folx,
We are about to update to OOD 3.1, where this issue does not occur, but thought you might want to know about it for version 3.0. This is the error we’re getting:

Error: Failed to download metadata for repo 'ondemand-web': repomd.xml GPG signature verification error: Bad GPG signature

As soon as I set repo_gpgcheck=0 (gpgcheck is still set to 1), everything works fine.

Also, when I try to verify the repomd files, I get this:

gpg  --verify repomd.xml.asc repomd.xml
gpg: Signature made Mon 09 Oct 2023 04:22:28 PM EDT
gpg:                using RSA key FE143EA1CB378B569BBF7C544B72FE2B92D31755
gpg: Can't check signature: No public key

I’m not sure if it’s not finding the OOD public keys, or if it doesn’t think they’re the right keys. I do have them:

rpm -q gpg-pubkey |while read line; do echo -n "The packager for $line is"; rpm -qi $line | grep Packager | sed 's/Packager    //';done |grep OnDemand
The packager for gpg-pubkey-92d31755-5a81ef2e is: OnDemand Release Signing Key <packages@osc.edu>
The packager for gpg-pubkey-fd775498-62ab688f is: OnDemand Release Signing Key (SHA512) <packages@osc.edu>

Thanks,
Sarah

Hi Sarah,

Could you say more about how the issue was found? What commands/process led to the “Failed to download metadata for repo” error?

Yes! /usr/bin/dnf check-update leads to the error message.
There’s a lot more info on repo metadata keys here: including a description of what they do:

You can then use GPG to sign the repository metadata. Doing this guarantees to users of your repository that you generated the metadata.

Oh and we’re on Rocky 8. Not sure if that makes a difference for the metadata key.

Darn. Just found out we’re having the same problem with version 3.1. So this might be something on our part.

Aday is going to try installing locally and see if he gets the same error. But if this were a general error, I’m sure you would have heard about it before.

We’ll keep trying to figure it out.

Thanks for the info! I haven’t heard of others with the same issue, but i’ll look into it some and if I find anything i’ll let you know. Please update us if you get a solution and let us know if there are any other questions or issues.

@tdockendorf do you know what the issue could be?

I’d say for now don’t worry about it, since it seems to be something on our end. We’ll let you know when we figure it out.

We’re trying to validate the key with this command: gpg --verify repomd.xml.asc repomd.xml but we’re not sure if it’s working right. Do any of you know the right command?

Cheers,
Sarah

$ curl -O https://yum.osc.edu/ondemand/RPM-GPG-KEY-ondemand
$ gpg2 --yes -o RPM-GPG-KEY-ondemand.gpg --dearmor ./RPM-GPG-KEY-ondemand
$ gpg2 --no-default-keyring --keyring ./RPM-GPG-KEY-ondemand.gpg --verify /var/www/repos/public/ondemand/3.0/web/el8/x86_64/repodata/repomd.xml.asc
<SNIP>
Primary key fingerprint: FE14 3EA1 CB37 8B56 9BBF  7C54 4B72 FE2B 92D3 1755

The public key for EL8 is different from EL9. The above is what will work for EL8.

Thanks Trey, those commands are super useful. This is definitely some kind of issue with our OS, because when we run the verify on the metadata key on those systems, it gives a “Good Signature” message. So I’d say no need for you folx to keep looking at this, we’ll figure it out.

Cheers,
Sarah