Overview of Jupyterhub on SLURM

Article author
Jillian Rowe
  • Updated

You can access a Jupyterhub instance by using the URL supplied by your administrator in your browser. Login using your username and password.

Run your Notebook as a SLURM Job

Once you’ve logged in you should see a button that says ‘Start Server’. Press that button, and you will be shown a form that corresponds to SLURM submission parameters. Fill it in with your desired submission parameters, and hit submit. A background process will submit a SLURM job, startup a single user notebook, and connect it to the main process running in the hub.

Notebooks on the Login Node

The Jupyterhub instance is configured to only allow notebooks on the compute nodes.

If for any reason you need a Jupyterhub notebook on the login node, such as you want to programatically submit jobs, or spin up a Dask or Apache Spark cluster for your analysis, you will need to manually start a notebook from the login node. You can use the IP address or DNS name, http://MY_JHUB_DOMAIN, along with the port the notebook is running on.

Detailed Instructions

This looks complicated, but it's really not that bad. You need to keep track of the public IP address of the login node, LOGIN, and the IP of the compute, COMPUTE.

First, you will need to create a software environment that has jupyter, and if you want to use RStudio R.

# r-single-cell.yaml
name: r-single-cell
channels:
- bioconda
- conda-forge
dependencies:
- jupyterlab
- notebook
- rstudio
- r
- pip
- pip:
    - jupyter-rsession-proxy
 
Pip is in there twice. First as a dependency and then to tell conda that we want to install pip packages.
 
module load Miniconda3
conda env create -f r-single-cell.yml
# This corresponds to the name
source activate r-single-cell
# I always prefer to specify a filepath
# conda env create -f r-single-cell.yml -p /apps/users/$USER/r-single-cell
# This corresponds to the name
# source activate /apps/users/$USER/r-single-cell
 
To access the notebook, open this file in a browser:
    file:///home/$USER/.local/share/jupyter/runtime/nbserver-17348-open.html
    Or copy and paste one of these URLs:
        http://{IP}:8888/?token=8a7f350e3e19ac27439bd259cb144fbab8970457256a41a6
     or http://127.0.0.1:8888/?token=8a7f350e3e19ac27439bd259cb144fbab8970457256a41a6

Ignore the IP addresses. Usually, those point to private IPs and instead you want the public IP that you used initially.

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.

Still have questions?

Submit a request