Open OnDemand 4.0 seems slow

Dear developers,

We updated the OOD version from 3.1.10 to 4.0.0.
It’s just a feeling, but it feels like it’s become slower.

I looked at /var/log/ondemand-nginx/<my_account>/error.log.
Every time I click on an application icon, the following message is being output every second.

App 788877 output: [2025-01-20 14:33:47 +0900 ]  WARN "Announcement file not found: /etc/ood/config/announcement.md"
App 788877 output: [2025-01-20 14:33:47 +0900 ]  WARN "Announcement file not found: /etc/ood/config/announcement.yml"
App 788877 output: [2025-01-20 14:33:47 +0900 ]  INFO "method=GET path=/pun/sys/dashboard/apps/icon/VisIt/sys/sys format=html controller=AppsController action=icon status=200 allocations=580057 duration=803.33 view=0.00"
App 788877 output: [2025-01-20 14:33:48 +0900 ]  WARN "Announcement file not found: /etc/ood/config/announcement.md"
App 788877 output: [2025-01-20 14:33:48 +0900 ]  WARN "Announcement file not found: /etc/ood/config/announcement.yml"
App 788877 output: [2025-01-20 14:33:48 +0900 ]  INFO "method=GET path=/pun/sys/dashboard/apps/icon/XCrySDen/sys/sys format=html controller=AppsController action=icon status=200 allocations=579877 duration=811.34 view=0.00"
App 788877 output: [2025-01-20 14:33:49 +0900 ]  WARN "Announcement file not found: /etc/ood/config/announcement.md"
App 788877 output: [2025-01-20 14:33:49 +0900 ]  WARN "Announcement file not found: /etc/ood/config/announcement.yml"
App 788877 output: [2025-01-20 14:33:49 +0900 ]  INFO "method=GET path=/pun/sys/dashboard/apps/icon/WHEEL/sys/sys format=html controller=AppsController action=icon status=200 allocations=579975 duration=801.40 view=0.00"

Since we have about 50 applications installed in our Open OnDemand site, this message lasts for 50 seconds.

OOD v3.1.10 did not output such a message, so could this message be the reason for the slowness?

Best,

Hey sorry for the delay in the response. I don’t think those log messages should cause noticeable delay, but I’ll look into why they’re there at all.

1 Like

Dear Jeff,

I also notice a difference in performance after updating from OOD 3.1.11 to 4.0. The time required for the Dashboard to fully load in OOD 4.0 is approximately 3300ms, whereas in OOD 3.1.11, it was only 1200ms—so about twice as slow in the newer version.

With kind regards,
Dzung

@jeff.ohrstrom

I installed OOD v4.0.2, but the slowdown did not change. So I investigated the difference in behavior with v3.1.10. In our environment, there are about 50 applications and 50 png small image files for icon. OOD v4 takes a long time to load the icons.

As mentioned in the first post, the duration= value is about 800 in OOD v4, but it is about 350 in OOD v3. Furthermore, when opening a web form page for entering time etc., the 50 lines INFO "method=GET path=/ in error.log in the first post are displayed, but in OOD v3 only one line is displayed.

To enable caching, I added a line expires_in 1.day, public: true in the icon method in /var/www/ood/apps/sys/dashboard/app/controllers/apps_controller.rb in OOD v4, and only one line was displayed just like in OOD v3, and the speed improved. However, the duration value remains large.

I hope this is helpful.

Best,

Are you doing lots of embedded ruby in the various applications? It’s possible you are doing a ton under the hood when you load the apps that would be better done as initializers.

@mjbludwig

Are you doing lots of embedded ruby in the various applications?

Yes.

It’s possible you are doing a ton under the hood when you load the apps that would be better done as initializers.

I agree with you.
I believe Open OnDemand evaluates the form.yml.erb for all applications when opening a dashboard or opening an application page. But I understand the intent of that behavior; it would be nice if we could separate the initialization process from the rest in Ruby in the form.yml.erb, but I don’t know how to do that.

You want to use global initializers that only run when the dashboard is initially loaded and then use those variables down stream in your apps. For instance, if you want to figure out what groups a user is in, instead of having embedded ruby in each app do something like getent group which then runs a bunch of times for each app its in, you put that in an initializer which stores the result in a variable and then you just reference the variable in your various apps.

This is a pretty extensive example of initializers: OnDemand-info/config/apps/dashboard/initializers/ood.rb at master · chpc-uofu/OnDemand-info · GitHub

As that file mentions, you stick your .rb file in /etc/ood/config/apps/dashboard/initializers/ood.rb.

1 Like

Yea apologies for any issues in 4.1. We’re looking into caching those rendered erbs.

Along with an initializer, if you know them straight out, you can use global batch connect form items in 4.1 if you have a lot of things you reuse. These are evaluated once on load through an ondemand.d file and all the forms just reference them.

2 Likes

@mjbludwig @jeff.ohrstrom

Thank you for your advices. I will try to optimize it using the method you taught me.
I am also looking forward to the release of v4.1.

Thank you in advance.