Sharing: HiGlass OOD Application

General Discussion
1

Hi everyone,

I’d like to share an Open OnDemand Batch Connect application we’ve been working on for running HiGlass interactively on an HPC cluster:

GitHub: GitHub - CPOS-HPC/ood_bc_higlass · GitHub

HiGlass is a powerful browser-based visualization tool for large-scale genomic data, but integrating it into an HPC environment through Open OnDemand turned out to involve a few interesting challenges, especially around reverse proxying, containers, persistent data, and URL handling.

This repository is our reference implementation for running HiGlass as an OOD Basic Batch Connect application. HiGlass runs inside an Apptainer container on a compute node and is exposed to the user through Open OnDemand’s authenticated /rnode/HOST/PORT reverse proxy, without requiring the HiGlass port to be exposed directly outside the cluster.

Our current reference environment is:

  • Open OnDemand 4.2

  • OpenPBS

  • Lmod

  • Apptainer

  • higlass/higlass-docker:0.10.4

Some of the integration details covered by the project include:

  • per-session OOD-assigned ports;

  • nginx in front of the HiGlass/uWSGI service;

  • support for HiGlass behind the OOD /rnode/HOST/PORT URL prefix;

  • rewriting root-relative HTML, JavaScript, CSS, API, and font URLs;

  • persistent user-selectable HiGlass data directories;

  • zero-copy dataset registration using read-only Apptainer binds;

  • automatic chromosome-size registration for formats such as cooler, bigWig, BAM, and chromsizes;

  • readiness checking before OOD enables the Connect button;

  • isolation of ports, sockets, PID files, configuration, and logs between concurrent sessions.

One particularly interesting part was making HiGlass behave correctly when it is no longer hosted at /, but instead behind an OOD URL such as:

/rnode/compute-node.example.org/12345/

There are several places where HiGlass/Django/nginx behavior needs to be handled carefully for static resources and API requests to remain behind the OOD proxy.

The repository also contains a porting checklist and documentation for adapting the app to other sites.

It is intentionally not a drop-in application. Scheduler directives, cluster IDs, queue/account policies, module initialization, filesystem layout, and container deployment are site-specific. The goal is instead to provide a reasonably complete reference showing the integration points and some of the failure modes we encountered.

I’d be especially interested to hear from anyone who:

  • is already running HiGlass through Open OnDemand;

  • wants to try adapting this to a Slurm-based cluster;

  • has suggestions for making the Batch Connect integration more portable;

  • has experience packaging similar containerized web applications for OOD.

Feedback, issues, and pull requests are very welcome.

Hope this is useful to other Open OnDemand sites working with genomic visualization tools!

2

This is great! Thanks for sharing. Can you please submit this to the Appverse The Appverse | Open OnDemand Community Hub ?

3

I’d love to, but Appverse webpage doesn’t response to my application. (It said “The application did not respond in time.”)

4

Thank you! I see your submission in the backend. A moderator will be in touch.

5

Hi Emik. This looks like a great app. Thanks for sharing it with the community!

The error message you saw revealed an issue with how the website responded to a missing manifest or appverse.yml that we addressed today. It looks like you since added an appverse.yml which the cron sync picked up and the app should be showing in your app management dashboard at https://openondemand.connectci.org/appverse/my-apps. There you can click the button to send it for review:

image
image1149×266 14.5 KB

Regards,

Andrew

6

Thank Andrew, but I still can’t submit it to the Appverse.