OOD app to display interface to a python app

zihaddad · December 14, 2023, 4:27am

.
I am looking for help creating an app that allows users to interact with a python chatbot that uses gradio (https://www.gradio.app/) to present a webpage for users to interact with the app.
I have modified the jupyter (and VS code server) app to start the app and present the webpage. For some unknown reason I can’t display the webpage. The link look correct (OOD_server/node/compute_node/port#) but the page display {“details” :”Not Found”}
if I create a desktop session on the same compute node and open firefox at 127.0.0.1:port the app display the chatbot.
Interested if someone already built such an app to connect to a python app that serves a webpage. I am open to switching gradio if anyone has a successful connection via OOD with a different interface. For example, any success using Uvicorn or FastAPI.
I also tried modifying the matlab app and starting firefox instead of matlab but what I found Is that firefox is not stable… randomly firefox crashes (channel error ) and vnc session closes.
This is my script.sh.erb (the conda env is created and the app start but fails to display the webpage {“details” :”Not Found”}

Set working directory to home directory

SCRIPT_DIR=“/hpc_share/project_mosaic/MosaicPhase1App”
cd $SCRIPT_DIR
echo “Script directory: $SCRIPT_DIR”
echo “port= $port”

Start Project Mosaic Server

module purge

CONDA_DIR=$(echo ~/anaconda3)
if test -d $(echo ~/anaconda3/envs); then
CONDA_DIR=$(echo ~/anaconda3)
fi
if test -d $(echo ~/.conda/envs); then
CONDA_DIR=$(echo ~/.conda)
module load cuda
module load anaconda
conda init
fi
CONDA_NAME=“Mosaic_Env”
CONDA_ENV_PATH=“${CONDA_DIR}/envs/${CONDA_NAME}”

if ! test -d “$CONDA_ENV_PATH”;
then
echo “Building Env”
conda create --name $CONDA_NAME python=3.9 -y
source $(which conda activate 2> /dev/null | tail -n 1) $CONDA_NAME
echo $(conda info)
conda install -c conda-forge git-lfs -y
export LLAMA_CUBLAS=1
export CMAKE_ARGS=“-DLLAMA_CUBLAS=on”
CMAKE_ARGS=“-DLLAMA_CUBLAS=on” pip install llama-cpp-python --force-reinstall --no-cache-dir
pip install -r requirements.txt --no-cache-dir

else
echo “Activating Env”
source $(which conda activate 2> /dev/null | tail -n 1) $CONDA_NAME
echo $(conda info)
fi

List loaded modules

module list

Benchmark info

echo “TIMING - Starting Project Mosaic at: $(date)”

Launch the Project Mosaic Server

set -x

python3 app.py $port

travert · December 14, 2023, 5:18pm

Is there a reason you are having it launch from within other apps? Have you considered just making this a Phusion passenger app instead? Or,is there some kind of context being shared between the app and the chatbot which necessitates this setup?

We have docs here for a simple phusion passenger app:
https://osc.github.io/ood-documentation/latest/tutorials/tutorials-passenger-apps.html

There’s a more extensive writeup on this on a current PR branch that has some technical cleanup needed before it can be merged, but this also may provide some more insight into setting this as a passenger app:
https://osc.github.io/ood-documentation-test/passenger-tutorial/tutorials/tutorials-passenger-apps/phusion-passenger.html

zihaddad · December 14, 2023, 6:01pm

The app needs GPU to run depending on what resources are available the backend may run on different compute node. you can’t submit a batch job with a passenger app can you.
tried to build an app that opens firefox but firefox crashes when using the modified matlab app. Does not crash when using a desktop session… is that because xfce panel is light window vs desktop session is a heavy window…
Jupter app works and from what I understand jupyter is a web server. .what is the magic clue that makes the jupyter app display connect to jupyter that i am missing with gradio …
The main question how to build an generic app to connect to any webserver (not just jupyter) running on a compute node serving a page

travert · December 14, 2023, 6:29pm

Got it.

OOD is a passenger app so technically, yes. But I think I’m understanding this better now and a passenger app may not be what is needed.

With a desktop, you are on the compute node and i’m guessing what you mean by Heavy vs Light is correct. The matlab app is more stripped down and simply presenting that app on the desktop:

github.com

OSC/bc_osc_matlab/blob/master/template/script.sh.erb

#!/usr/bin/env bash

<%- gpu = context.node_type.include?("vis") -%>

# Clean the environment
module purge

# Set working directory to home directory
cd "${HOME}"

#
# Launch Xfce Window Manager and Panel
#

(
  export SEND_256_COLORS_TO_REMOTE=1
  export XDG_CONFIG_HOME="<%= session.staged_root.join("config") %>"
  export XDG_DATA_HOME="<%= session.staged_root.join("share") %>"
  export XDG_CACHE_HOME="$(mktemp -d)"
  export $(dbus-launch)

This file has been truncated. show original

Jupyter comes with an HTTP server built in, so the way to connect up to it is not the same as for something like Matlab where we use vnc. For jupyter, it can send and receive HTTP traffic. That’s why if you look at the view you see some code around using a password, a host, and port that is sent back to connect up to the server:

github.com

OSC/bc_classroom_jupyter/blob/main/view.html.erb

<%-
  base_url  = "/node/#{host}/#{port}"
  login_url = "#{base_url}/login"
  next_url  = "#{base_url}/#{jupyter_api}"

  full_url="#{login_url}?next=#{CGI.escape(next_url)}"
  form_id = "juypyter_form#{login_url.gsub('/', '_')}"
%>

<!-- Zoom on iPad doesn't like opening in a new tab. It won't foward the form parameters. -->
<script type="text/javascript">
  function changeTarget() {
    const agent = navigator.userAgent;
    if (/Mozilla\/5\.0 \(Macintosh; Intel Mac OS X \d+_\d+_\d+\) AppleWebKit\/\d+\.\d+\.\d+ \(KHTML, like Gecko\)/.test(agent)) {
      document.getElementById("<%= form_id %>").target = "_self";
    }
  }
</script>

<form id="<%= form_id %>" action="<%= full_url %>" method="post" target="_blank" onsubmit="changeTarget()" >

This file has been truncated. show original

So, that Jupyter app will provide the best guidance for connecting to an HTTP webserver that comes with an app, but it requires some knowledge of how that app does this and passing some of that back to the view using the template directory and the scripts there like with what you see for Jupyter.

zihaddad · December 14, 2023, 8:32pm

this is my view.html.erb file:
view.html.erb.txt (208 Bytes)

this open up
https://OOD/node/compute_node/8192
what get displayed is {“detail”:“Not Found”}
am i missing something or do i have a broken pipe…because i know the app started on the compute node and on port 8192 web server is serving:

travert · December 14, 2023, 9:58pm

It sounds like something is wrong with the URL used looking at the {“detail”:“Not Found”} that you are seeing if the port is correct and open.

Yeah, I’m thinking that this is the most likely issue, that this is starting but the place you are sent is incorrect, and this is likely because the proxy may not be setup to handle this correctly.

Are you seeing any hints in the logs: Logging — Open OnDemand 3.0.3 documentation

jeff.ohrstrom · December 15, 2023, 4:10pm

Try /rnode/<%= host %>/<%= port %> in your view.html.erb instead of /node.

The difference is

Use node when the origin server (your app) knows it’s behind a proxy. It’s hard to find the docs from Gradio on this, but it’s something about mounting the application with a custom path.
Use rnode when the origin server (your app) doesn’t know it’s behind a proxy. This way we’ll re-write urls in apache and send requests to the origin server (your app) as if they’re local like what you have in a browser in a desktop.

That said - I’ve had issues with gradio apps serving assets correctly, so even then, you may still need work.

zihaddad · December 15, 2023, 8:29pm

yes that fixed it … I actually compared the jypyter view.html.erv against code server view.html.erb and found the /node/ vs /rnode/ and voila magic.

I did not know the reason so thanks for the explanation that makes it easy next time we write a new app.
really thanks for the quick support and awesome product.

luvres · January 26, 2024, 2:17am

I’m trying to run an app with Gradio and it seems that the POST methods are not loaded.

Here’s the difference between loading locally:

And with Interactive Apps:

jeff.ohrstrom · January 26, 2024, 4:20pm

Yea I’ve had issues proxying to gradio applications. Is your application able to recognize it’s behind a proxy? That could help - configure the app with the right node url and use it.

luvres · January 26, 2024, 7:04pm

Could you give me an example?

jeff.ohrstrom · January 26, 2024, 7:07pm

It’s app dependent, but here’s how we configure Jupyter to use a different base url. So when that app boots up, it recognizes that the URL paths should have that prefix.

github.com

OSC/bc_osc_jupyter/blob/6104427a4bdd9dbee2df782ca9a5f00177399c98/template/before.sh.erb#L103


      
          umask 077
          cat > "${CONFIG_FILE}" << EOL
          c.KernelSpecManager.ensure_native_kernel = False
          c.NotebookApp.ip = '*'
          <% if context.cluster.match?(/kubernetes/) -%>
          c.NotebookApp.port = 8080
          c.NotebookApp.base_url = '/node/${HOST_CFG}/${PORT_CFG}/'
          c.NotebookApp.terminado_settings = {'shell_command': ['/bin/bash', '--login', '-i']}
          <% else -%>
          c.NotebookApp.port = ${port}
          c.NotebookApp.base_url = '/node/${host}/${port}/'
          <% end -%>
          c.NotebookApp.port_retries = 0
          c.NotebookApp.password = u'sha1:${SALT}:${PASSWORD_SHA1}'
          c.NotebookApp.open_browser = False
          c.NotebookApp.allow_origin = '*'
          c.NotebookApp.notebook_dir = '${NOTEBOOK_ROOT}'
          c.NotebookApp.disable_check_xsrf = True
          EOL
          )

luvres · January 30, 2024, 1:47am

I use Jupyter in several ways with stage parameters, but in the specific case with Gradio the call is like this:

apptainer run --nv --cleanenv \
--env CLI_ARGS="--listen --listen-host ${host} --listen-port ${port}" \
container.sif

I believe the path is through the file view.html.erb

<form action="/rnode/<%= host %>/<%= port %>/" method="get" target="_blank">
  <button class="btn btn-primary" type="submit">
    <i class="fa fa-cogs"></i> Connect to Text Generation Webui
  </button>
</form>

I tried something like:

--env CLI_ARGS="--listen --listen-host /node/${host}/${port}/ --listen-port ${port}"

but it did not work.

I changed the method to “post”, but only “get” works and it doesn’t load the rest of the page which depends on “post”

Topic		Replies	Views
Jupyter App - Connections Failed Get Help	6	541	December 20, 2023
Using nb_conda_kernels to switch envs in Jupyter OOD Get Help question	7	426	October 3, 2022
IGV interactive web server app - Anyone have one up and running on their OOD? Get Help	17	1876	May 19, 2022
Web application failing when trying to access a websocket Get Help	2	521	May 19, 2022
Jupyter timed out waiting for port Get Help ondemand2 , question	8	594	August 15, 2023

OOD app to display interface to a python app

Set working directory to home directory

Start Project Mosaic Server

List loaded modules

Benchmark info

Launch the Project Mosaic Server

Related Topics