Distinguishing between built-in and site-added apps; Sharing app path by webservers of OOD instance

Feature Requests and Roadmap Discussion
1

We have a few OOD instances, and some of them have multiple backend webservers. For apps that we add, we need the same apps to be shared by all webservers for an OOD instance (and similarly, for users’ sandbox apps). Restating: when using an OOD instance, the same apps (under sys and dev) must be seen no matter which webserver a session is on.

We achieve this by NFS mounts of an app space for each OOD instance to /var/www/ood/apps on the webservers:
storage-host:/ood_apps/one mounted to /var/www/ood/apps on all webservers for OOD one instance
storage-host:/ood_apps/two mounted to /var/www/ood/apps on all webservers for OOD two instance

This has the intended effect of sharing sys and dev apps for each OOD instance.

However, there is a complication due to “built-in” apps (dashboard, shell, etc.) being under the same path (/var/www/ood/apps/sys/) as apps which we add. We would like almost-no-outage updates to be possible by removing half the webservers from the backend, updating them, switching servers in the backend, updating the rest and returning them. We can do this process, but as soon as the first webserver (not serving any sessions) updates the dashboard or other built-in apps, the site breaks and is not available until updating the first batch of servers and putting them in the backend is done. (Here “backend” is HAProxy backend, although could also apply to other types of load balancing.)

In considering ways to address this, one possibility is to make separate storage spaces for each OOD version for each OOD instance, so instead of the two shown above, there would be something like:
storage-host:/ood_apps/one-v3 mounted to /var/www/ood/apps on all webservers for OOD one instance for v3
storage-host:/ood_apps/one-v4 mounted to /var/www/ood/apps on all webservers for OOD one instance for v4
storage-host:/ood_apps/two-v3 mounted to /var/www/ood/apps on all webservers for OOD two instance for v3
storage-host:/ood_apps/two-v4 mounted to /var/www/ood/apps on all webservers for OOD two instance for v4

This approach has the disadvantage of unnecessary duplication of apps (for sys and dev). It also doesn’t address the conceptual mismatch between the built-in apps (from rpm/deb package) where it may be preferable for each webserver to have its own copy, and the site-added apps where a single copy should be shared for an OOD instance.

I understand that this is a design issue and there may be complex implications/dependencies involved. I am interested in how these issues have been handled at other sites, and further questions/discussion/advice about this.

2

Not really sure what you could do here. I will say that dev routes/apps are served from a users’ $HOME directory, so I would not suggest this scheme with dev apps. This seems like it would limit your developers and/or cause conflicts between them. Dev apps are supposed to be yours and yours only, not impacting other users.

As to the issues with sys apps - I’d maybe suggest symlinks. You can mount the same mount point, but it seems to me that you’d need a symlink for every single app so you can have the mount somewhere else and all the sys app symlinks point to it (or at least a subdirectory of the mount point for each app).

3

@jeff.ohrstrom Thanks for your reply. To clarify, our dev setup is as documented (sandbox for user is enabled by creating gateway → …/user/ondemand/dev in /var/www/ood/apps/dev/user). The sharing is between webservers of the same OOD instance (which see the same home directories), so that the required link is created once rather separately on each webserver. Users cannot see other users home directories.

4

Maybe a stupid remark but why don’t you ship your own apps with either as deb or rpm package?

It’s not hard, clean and you can easily update?

5

Oh i see. I guess you don’t have the same $HOME directory on all the systems?

For me (at OSC) - I have 1 single HOME drive. When one instance of OOD has me enabled as a developer - it points to this HOME drive. When I login to another OOD instance, and it also has me enabled as a developer, that instance also points to my same HOME drive. So in my situation, it’s already taken care of that my dev apps are everywhere becaue the filesystem that they actually reside on is 1 single NFS drive (my $HOME).

Yes thank you @wpoely86! We also use RPMs and puppet to keep every instance at specific versions.

6

As a second to the symlink suggestion, I do something similar by moving the default apps dir and having relative links to ../apps-dist in the shared storage space I link to instead. The dev symlinks for each development enabled user are also already created in the same location.

mv /var/www/ood/apps /var/www/ood/apps-dist
ln -s /storage/prod/${NODE}/var/www/ood/apps /var/www/ood/apps

ln -s /storage/prod/${NODE}/etc/ood/config/apps /etc/ood/config/apps
7

@wpoely86 That’s an interesting idea, thanks! We’ll look into that.

@jeff.ohrstrom The $HOME directory is shared on all our systems. Currently, the sandbox for a user can be enabled on a particular OOD instance without being enabled on another.

8

Oh I see. That sounds strange, but I guess if it works, it works. We use puppet to automate everything at OSC - even creating these symlinks. So when a new instance is brought online (for example a new VM to get a newer OS) puppet takes care of who’s enabled where.