Proposal for an integrated HDF5 Viewer

The idea is to add an integrated HTML5/JS viewer (similar to how NoVNC works) along with an interactive app for starting a REST API for viewing HDF5 files.

  1. A web form to launch an HDF5 interactive app has a file picker so the user can choose the file, and perhaps the number of workers (we could attempt to calculate a default based on the size of the file as an enhancement)
  2. When the job starts, an HDF5 REST API that can spread the work of interacting with the file across multiple workers, taking advantage of multiple cores (so a single node job? or should it be multiple jobs? or maybe we want to use K8s)
  3. A static CSS/JS front end would be used to interact with the REST API, similar to how NoVNC works for VNC apps in OnDemand. The OnDemand session card for the job would have a “Connect” button and when you click it you are taken to the CSS/JS front end.

For the backend, HSDS looks most promising but there is also h5serv, though I think only HSDS supports multiple workers, but I do not have a good graps on when multiple workers are required and how many - at what size of an HDF5 file will a single process server be too slow? We would like to support viewing HDF5 files from 100s of MB to 100s of GB in size.

For the front end, the closest thing to the type of HTML5/JS viewer we might be looking at would be MAXIV hdf5-web-gui, though that designed to consume the MAXIV hdf5-simple-server REST API, which is a stripped down version of h5serv, likely not something that can handle an HDF5 file 100s of GB in size.

So it is possible we would have to fork and modify, or write from scratch our own HDF5 HTML5/JS viewer.

Does anyone have any experience with these tools? Is MAXIV hdf5-web-gui a good web GUI to model off of, or are there better options?

Any other thoughts or ideas?