File browser cloud storage

Hello,

At CSC we would be interested in being able to access cloud storage (mainly S3 and Swift in our case) through the OOD file browser, as we haven’t found any good enough alternative that would provide this functionality.
I noticed that there are two issues related to this feature on Github, #1187 and #1188. The changes suggested in the issues seem like a good way forward, with first refactoring to have a PosixFile model and then adding additional models that would use Rclone. Has there been any further work or planning of how this could be implemented?
This is also something that we at CSC can contribute to the development of.

Hi Robin.

Yes, there has definitely been thoughts on implementing integration with cloud storage services.
I will present this to the team today, and get it on the schedule as a feature request.
Right now, I can’t guarantee a timeline, but will keep the community informed.

Thanks,
-gerald

Robin:

I’ll add that when we’ve discussed this in recent years, our initial plan has generally focused around integration with Globus (which then facilitates connections to a variety of cloud providers). This has stalled out, but we’d be happy to facilitate some introductions with the Globus team if you’d like to talk to them about what CSC might be able to assist with.

One other thought as an alternative / short-term solution. Many sites do support the VSCode app, which has a full function file browser as part of it

VSCode has a variety of plugins that allow it to interface with cloud storage providers:

@gbyrket Thanks. Let us know if there is anything we can help implement to speed up the process, for example refactoring the current code.

@alanc Thanks for the interesting alternatives, though they don’t seem to be what we are looking for right now.

This feature isn’t really under development or scheduled for any release. Indeed - as you point out there’s already refactoring work that I’d like to get done before we add new features.

That said, pull requests are always welcome! We’re happy to evaluate any changes for this feature - but I’d only ask that they be small changes. It’s just hard to evaluate large changes. What a refactor would look like - I don’t know offhand but if you submit some small change that moves us in this direction without breaking current behavior, we’d love to see it.

This ticket just came in for the same.

I think relying on s3fs and providing a button that’s able to mount an s3 bucket to your local filesystem is something that’s achievable in the short term.

I noticed, s3fs was something I tried earlier, but at least currently our system isn’t configured for FUSE. That seemed like a decent option otherwise.

We’ll try to submit some pull requests that would move us closer to the goal.

1 Like