# Errors with rhel-server-rhscl-7-rpms repo in the past week?

**URL:** <https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494>\
**Category:** Feature Requests and Roadmap Discussion\
**Tags:** question\
**Created:** [September 18, 2019, 3:53pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494 "2019-09-18T15:53:20Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![jpr](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jpr/32/35_2.png) [@jpr](https://discourse.openondemand.org/u/jpr)\
**Post date:** [September 18, 2019, 3:53pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/1 "2019-09-18T15:53:20Z")

</div>

We are having an issue that seems to have appeared since last week with building OOD on our centos platform and using the rhel-server-rhscl-7-rpms repo. In fact it appears to have emerged just over the past day. Yesterday I could work around it by forcing a specific baseurl for this repo. Today, even the base url I used yesterday is not working. 😕

The `yum install ondemand` results in a bunch of 404 errors looking for packages:

rh-git29-2.3-5.el7.x86\_64.rpm FAILED  
[http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86\_64/rh/rh-git29/rh-git29-2.3-5.el7.x86\_64.rpm:](http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86_64/rh/rh-git29/rh-git29-2.3-5.el7.x86_64.rpm:) [Errno 14] HTTP Error 404 - Not Found

---

<div class="post-metadata">

**Author:** ![efranz](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/efranz/32/27_2.png) [@efranz](https://discourse.openondemand.org/u/efranz)\
**Post date:** [September 18, 2019, 4:10pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/2 "2019-09-18T16:10:21Z")

</div>

Hmmm [http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86\_64/](http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86_64/) is accessible but [http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86\_64/rh/](http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86_64/rh/) took a long time to access. Then the only git available from gtlib.gatech.edu is now Git 2.18 ([https://www.softwarecollections.org/en/scls/rhscl/rh-git218/](https://www.softwarecollections.org/en/scls/rhscl/rh-git218/)) and Git 2.9 ([https://www.softwarecollections.org/en/scls/rhscl/rh-git29/](https://www.softwarecollections.org/en/scls/rhscl/rh-git29/)) is no longer available from that source. Investigating…

---

<div class="post-metadata">

**Author:** ![krishmoodbidri](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/krishmoodbidri/32/196_2.png) [@krishmoodbidri](https://discourse.openondemand.org/u/krishmoodbidri)\
**Post date:** [September 18, 2019, 5:20pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/3 "2019-09-18T17:20:02Z")

</div>

Hey @efranz  
We were debugging this issue last night, the techlinks mirror wasn’t working  
Today it seems like the gitlab repo is down as well  
[http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86\_64/rh/rh-git29/rh-git29-2.3-5.el7.x86\_64.rpm:](http://www.gtlib.gatech.edu/pub/centos/7/sclo/x86_64/rh/rh-git29/rh-git29-2.3-5.el7.x86_64.rpm:) [Errno 14] HTTP Error 404 - Not Found

---

<div class="post-metadata">

**Author:** ![jpr](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jpr/32/35_2.png) [@jpr](https://discourse.openondemand.org/u/jpr)\
**Post date:** [September 18, 2019, 5:23pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/4 "2019-09-18T17:23:59Z")

</div>

Here’s an example of the error we saw on teklinks (this frequently resolves as our closest mirror via mirrorlist). It’s the same error as for gtlib just a different package. In the full output it’s all the sclo packages.

[http://mirror.teklinks.com/centos/7.7.1908/sclo/x86\_64/rh/rh-nodejs6/rh-nodejs6-nodejs-realize-package-specifier-3.0.3-2.el7.noarch.rpm:](http://mirror.teklinks.com/centos/7.7.1908/sclo/x86_64/rh/rh-nodejs6/rh-nodejs6-nodejs-realize-package-specifier-3.0.3-2.el7.noarch.rpm:) [Errno 14] HTTP Error 404 - Not Found

Given this is what I saw yesterday afternoon and was then able to use gtlib successfully, I suspect whatever the change was it occurred with yesterday’s mirror updates and teklinks just updated sooner than gtlib.

---

<div class="post-metadata">

**Author:** ![tdockendorf](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/tdockendorf/32/202_2.png) [@tdockendorf](https://discourse.openondemand.org/u/tdockendorf)\
**Post date:** [September 18, 2019, 5:31pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/5 "2019-09-18T17:31:59Z")

</div>

Because of the release of CentOS 7.7 they EOL SCL packages are now not included in the CentOS SCL repos as they didn’t build those RPMs as they are EOL.

As a temporary work around you will have to install the CentOS SCLo Vault.

```
cat > /etc/yum.repos.d/centos-scl-vault.repo <<EOF
[C7.5.1804-SCLo]
name=CentOS-7.5.1804 - Updates
baseurl=http://archive.kernel.org/centos-vault/7.5.1804/sclo/x86_64/sclo/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo
enabled=1

[C7.5.1804-SCLo-rh]
name=CentOS-7.5.1804 - Updates
baseurl=http://archive.kernel.org/centos-vault/7.5.1804/sclo/x86_64/rh/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo
enabled=1
EOF

```

I was unable to reproduce this with centos:7 docker container using this in `/etc/yum.repos.d/CentOS-SCLo-scl-rh.repo`

```
[centos-sclo-rh]
name=CentOS-7 - SCLo rh
baseurl=http://mirror.centos.org/centos/7/sclo/$basearch/rh/
#mirrorlist=http://mirrorlist.centos.org?arch=$basearch&release=7&repo=sclo-rh
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo
```

---

<div class="post-metadata">

**Author:** ![tdockendorf](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/tdockendorf/32/202_2.png) [@tdockendorf](https://discourse.openondemand.org/u/tdockendorf)\
**Post date:** [September 18, 2019, 5:33pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/6 "2019-09-18T17:33:06Z")

</div>

If/when the CentOS 7.6 vault gets populated, you could update the `baseurl` in the work around. As far as I could tell the CentOS 7.6 vault repos are still empty.

---

<div class="post-metadata">

**Author:** ![jpr](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jpr/32/35_2.png) [@jpr](https://discourse.openondemand.org/u/jpr)\
**Post date:** [September 18, 2019, 5:36pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/7 "2019-09-18T17:36:31Z")

</div>

So if we update to CentOS7.7 does this issue magically go away? Is it just that we happen to be building on 7.6?

---

<div class="post-metadata">

**Author:** ![tdockendorf](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/tdockendorf/32/202_2.png) [@tdockendorf](https://discourse.openondemand.org/u/tdockendorf)\
**Post date:** [September 18, 2019, 5:40pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/8 "2019-09-18T17:40:02Z")

</div>

It depends on how your repo files are defined. If your repo files are pointed to SCL repos that are pinned to what was available when CentOS 7.6 was the latest OS , then you should be fine. If your repos are not pinned then this issue will manifest itself on a fresh install. If you installed OnDemand on CentOS 7.6 then do “yum update”, you’ll be fine but fresh installs won’t be able to access the EOL SCL packages if your just install “CentOS 7”.

If your doing a fresh install on CentOS 7.6 or 7.7 using stock CentOS repo files, the work around is necessary. If your upgrading existing CentOS 7.6 to 7.7 then this issue shouldn’t occur.

Our build environment was built using CentOS 7.6 a few months ago and all the packages were available then.

---

<div class="post-metadata">

**Author:** ![jpr](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jpr/32/35_2.png) [@jpr](https://discourse.openondemand.org/u/jpr)\
**Post date:** [September 18, 2019, 5:43pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/9 "2019-09-18T17:43:57Z")

</div>

We install fresh all the time in dev. 🙂 We are using packer to build OOD images for our dev deployments and using the CentOS7 cloud images as the base. Current is v1905 which appears to be CentOS7.6. I’m gonna try out 1907 which might be 7.7.

So just to be clear, you are saying that regardless of 7.6 or 7.7 a fresh install will suffer this error unless we use the work around above?

---

<div class="post-metadata">

**Author:** ![tdockendorf](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/tdockendorf/32/202_2.png) [@tdockendorf](https://discourse.openondemand.org/u/tdockendorf)\
**Post date:** [September 18, 2019, 6:00pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/10 "2019-09-18T18:00:11Z")

</div>

I believe the workaround is needed regardless of 7.6 or 7.7 if you are doing a fresh install. Normally CentOS repos will just install CentOS 7 (latest) which now doesn’t include several SCL packages needed by OnDemand. Usually the only way to install non-latest is to use the Vault repos or use local copies that are points-in-time. Normally if you install CentOS 7.6 with stock repos then do “yum update” you’ll be at CentOS 7.7. Also installing packages with stock repos from CentOS 7.6 will pull in packages built for CentOS 7.7 as CentOS is a rolling upgrade type OS. If your using 7.6 specific repos this may not be the case but only ones I know of are the vault repos.

---

<div class="post-metadata">

**Author:** ![tdockendorf](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/tdockendorf/32/202_2.png) [@tdockendorf](https://discourse.openondemand.org/u/tdockendorf)\
**Post date:** [September 18, 2019, 6:09pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/11 "2019-09-18T18:09:36Z")

</div>

For my work around it looks like CentOS main mirror has the packages:

Use these base URLs:

`http://mirror.centos.org/centos/7.6.1810/sclo/x86_64/rh/`

`http://mirror.centos.org/centos/7.6.1810/sclo/x86_64/sclo/`

---

<div class="post-metadata">

**Author:** ![tdockendorf](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/tdockendorf/32/202_2.png) [@tdockendorf](https://discourse.openondemand.org/u/tdockendorf)\
**Post date:** [September 18, 2019, 6:19pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/12 "2019-09-18T18:19:05Z")

</div>

Upon further investigation the SCL repos from CentOS are minor version specific. The mirror URL generates 7.7 URLs so replacing the `mirrorlist` with a `baseurl` should be sufficient:

```
cat > /etc/yum.repos.d/centos-scl.repo <<EOF
[C7.6.1810-SCLo]
name=CentOS-7.5.1804 - SCLo
baseurl=http://mirror.centos.org/centos/7.6.1810/sclo/x86_64/sclo/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo
enabled=1

[C7.6.1810-SCLo-rh]
name=CentOS-7.6.1810 - SCLo rh
baseurl=http://mirror.centos.org/centos/7.6.1810/sclo/x86_64/rh/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo
enabled=1
EOF
```

---

<div class="post-metadata">

**Author:** ![jpr](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jpr/32/35_2.png) [@jpr](https://discourse.openondemand.org/u/jpr)\
**Post date:** [September 18, 2019, 8:51pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/13 "2019-09-18T20:51:55Z")

</div>

Fixing just the SCLo rh repo seems to fix the problem for us. Thank you!

> [@tdockendorf](#):
>
> baseurl=[http://mirror.centos.org/centos/7.6.1810/sclo/x86\_64/rh/](http://mirror.centos.org/centos/7.6.1810/sclo/x86_64/rh/)

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [October 16, 2019, 2:19pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/14 "2019-10-16T14:19:19Z")

</div>

@jpr I just happen to recall this. Sounds like you have a really advanced CD pipeline, do you have any documentation on that? Blogs or anything? We’d really like to hear what you’re up to and what benefits you’ve gotten out of such a workflow.

---

<div class="post-metadata">

**Author:** ![westburg.2](https://avatars.discourse-cdn.com/v4/letter/w/0ea827/32.png) [@westburg.2](https://discourse.openondemand.org/u/westburg.2)\
**Post date:** [May 26, 2022, 4:11pm UTC](https://discourse.openondemand.org/t/errors-with-rhel-server-rhscl-7-rpms-repo-in-the-past-week/494/15 "2022-05-26T16:11:39Z")

</div>