I’m currently in the process of upgrading Open OnDemand from version 3.0 to 4.0, following the official guide (v4.0 Release Notes & Upgrade Instructions). However, I’ve run into an issue and would appreciate any advice or insights from the community.
Here’s what I’ve done so far:
I updated Ruby to version 3.3 as recommended.
When running sudo /opt/ood/ood-portal-generator/sbin/update_ood_portal, I encountered an error related to Ruby and the libruby.so.3.3 library. The system seems to expect libruby.so.3.0 instead.
I also tried following the guide without updating Ruby, but during the sudo yum update ondemand step, I received an error stating that the Ruby update is required.
I’ve attached a couple of screenshots to illustrate the issue.
Questions:
Has anyone else encountered this Ruby version conflict during the upgrade?
Are there any known workarounds or solutions to resolve this issue?
Any help or guidance would be greatly appreciated! Thanks in advance.
Hi Edwin.
Yes we are using a Rocky8 cluster for this test, we tried to update once again the ruby module after and before the ood package update, but we keep having the same errors. I will add here a screenshot with the case.
One thing is you may not have loaded the environment correctly. Did you source /opt/ood/enable before issuing the command? (I pulled that path from memory could be slightly different).
Also - it may just be moot, because you don’t have to run that command anymore. Just bounding httpd through the systemd unit file will regenerate the configurations (the httpd systemd unit file will issue this command).
Hi Jeff and Edwin
First of all thanks for your quick responses. I have tried both methods but we had already installed the psych gem. The environment had also loaded correctly. The error seems to be more of a conflict with different versions of the ruby (?¿) gems. We are working in parallel on having a fresh installation from scratch of version 4.0 to confirm that the bug is due to version upgrade.
Can you think what could be happening? (I attach a screenshot)
Yea it’s got to do with your Ruby environment. issuing gem env may provide some insight.
For whatver reason the psych gem installed by the system is still being loaded as the 3.1 gem. gem pristine psych as root_may_ fix it, but I’m not 100% sure, so you’d have to take a great deal of caution here with that. (there’s also the --all flag for all of them, likely even more risky).
Greetings once again!
Regarding what you were telling us Jeff, both “gem env” and “gem pristine psych” commands cannot be executed, this is because any “gem” command throws the same Ruby error (even a “gem -help”).
We have been doing a series of tests, we have performed an upgrade of the Ansible installer that we use in deployments with the version 4 RPMs, and tested on a test cluster that everything works correctly (it does, so the error had to be with upgrading an existing environment).
As this DEV cluster is a test environment in which I have Snapshots of the machines I have been curious about this problem at the time of the upgrade. The first thing I did was to remove all the contents of /usr/local/lib64/gems, /usr/share/gems, /usr/share/gems, /usr/local/lib64/gems /usr/share/gems, /usr/lib64/ruby and /usr/local/bin/ruby’ just before doing the ‘sudo yum update ondemand’ (although I know it is not a good idea in a real environment, here it is useful to confirm that the error is from these files). So we confirm it, doing this the update to version 4 is done correctly, the problem is that Ruby seems to lose things and the OOD does not work correctly.
Having done this and looking more in the error itself (in this line more specifically:
Yea it seems that you’d want to remove the ondemand-gems package from 3.0. Then if you dnf module enable ruby:3.3 It seems to me that DNF should remove/replace all the gems in /user/local/share, but I’ll have to look more into it.
Our ansible roles test for this use case of upgrading and they all pass. Though I may have to inspect them more closely.
I have tried what you said about removing the “ondemand-gems” package from version 3.0 before enabling the new one. Unfortunately the test has not been successful
As I said before I have snapshots of the machine and Ansible installer that installs version 4 of OOD (working correctly), my idea to test what may be happening is to remove OOD V3 from this node (using an uninstall script that I did at the time) and install again with the Ansible. After doing this I have seen that the error still persists so the conflict has to be with respect to something that my uninstall script does not contemplate (I attach it in this post).
I also did one more small test, for this I removed the content of “ /usr/local/lib64/gems/” just after the update, to avoid the errors thrown by the “gem” commands. Once this was done I used “gem pristine --all” and everything seemed to work fine, there were no errors when using the portal_generator or when starting the services. Still the OOD portal has problems from the dashboard to any of the applications so it seems to be ruby bugs.
I do not know if you have been able to dig a little more on this issue and I hope these tests can provide more information.
Best regards!
Hello everyone again.
It has been an exhausting few days of testing to try to identify the bug we have on this machine.
We have ruled out many things, we even thought it was a bug inherited from version 2 (since we upgraded from 2 to 3 and now to 4). Although we have not been able to fix it, we have done upgrades from 2 to 3 and to 4, as well as from 3 to 4 directly on other machines and have not found this bug.
We deduce that it is a VM issue and a possible conflict with some Ruby library or dependency.
We will work on other machines to continue our testing on the upgrade. Thank you all very much and best regards!
