Proxy Custom Web Services

Article author
Jillian Rowe
  • Updated

You can proxy different web services using jupyter-server-proxy, which is already installed to your Jupyterhub Instance.

This allows you start any web service and access it in jupyterhub using the port you ran it on as https://MY_JHUB_DOMAIN/user/{username}/proxy/5000/

Note

Different services may have slightly different ways of dealing with URLs. You may have to try a few different combinations before you hit on the correct one.

https://MY_JHUB_DOMAIN/user/{user}/proxy/{port}/

https://MY_JHUB_DOMAIN/user/{user}/proxy/{port}

https://MY_JHUB_DOMAIN/user/{user}/proxy/absolute/{port}/

https://MY_JHUB_DOMAIN/user/{user}/proxy/absolute/{port}

Also be sure to try each combination with and without a trailing /.

If are having trouble finding a free port here is a handy python function used in sphinx-autobuild.

# https://github.com/executablebooks/sphinx-autobuild/blob/master/src/sphinx_autobuild/utils.py
import socket

def find_free_port():
    """Find and return a free port number.
    Shout-out to https://stackoverflow.com/a/45690594/1931274!
    """
    with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as s:
        s.bind(("", 0))
        s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        return s.getsockname()[1]

 

Simple Example

We’ll create a ‘Hello World’ static website and use the python http server to serve it.

mkdir -p dummy_website 
cd dummy_website
echo "<h1>Hello, World!</h1>" > index.html
python -m http.server 3001

 

Make sure to run any servers on host/ip 0.0.0.0 and specify the port.

You may have to switch up the urls depending on how your app deals with trailing slashes and urls.

Then check out -

http://{jhub-url}:{jhub-port}/user/{username}/proxy/3001

https://{jhub-url}:{jhub-port}/user/{user}/proxy/3001/

https://{jhub-url}:{jhub-port}/user/{user}/proxy/absolute/3001/

 

If you’re username is ‘user001’

https://{jhub-url}:{jhub-port}/user/user001/proxy/absolute/3001/

Tip

I’ve had mixed luck running long running bash processes from jupyterhub notebooks. It seems to be kind of hit or miss. If you’re having trouble try putting your commands in a script and run from a terminal.

Run the MLFlow UI

You may need to install some packages.

# If you're running from a jupyterhub cell make sure to include the '!' at the beginning!
! pip install --user jupyter-server-proxy mlflow gunicorn
import os
from random import random, randint
from mlflow import log_metric, log_param, log_artifacts

if __name__ == "__main__":
   # Log a parameter (key-value pair)
    log_param("param1", randint(0, 100))

    # Log a metric; metrics can be updated throughout the run
    log_metric("foo", random())
    log_metric("foo", random() + 1)
    log_metric("foo", random() + 2)

    # Log an artifact (output file)
    if not os.path.exists("outputs"):
        os.makedirs("outputs")
    with open("outputs/test.txt", "w") as f:
        f.write("hello world!")
    log_artifacts("outputs")

! PATH=$HOME/.local/bin:$PATH && mlflow server -w 1
 
[2020-11-09 07:34:49 +0000] [282] [INFO] Starting gunicorn 20.0.4
[2020-11-09 07:34:49 +0000] [282] [INFO] Listening at: http://127.0.0.1:5000 (282)
[2020-11-09 07:34:49 +0000] [282] [INFO] Using worker: sync
[2020-11-09 07:34:49 +0000] [284] [INFO] Booting worker with pid: 284
 

Access the MLFLow UI

You’ll have to use the proxy in order to access the UI. You may have to play around with the URL as it seems that different versions of jupyter-server-proxy accept different urls.

Make sure to try including and excluding the trailing /

https://MY_JHUB_DOMAIN/user/{user}/proxy/5000/

https://MY_JHUB_DOMAIN/user/{user}/proxy/absolute/5000/

If you’re username is ‘admin’ -

https://MY_JHUB_DOMAIN/user/admin/proxy/5000/

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.

Still have questions?

Submit a request