Run MATLAB Notebooks with persistence on a JupyterHub (sys-admin nomination required)¶
Table of Contents
Prerequisites¶
Make sure you are registered to the IAM system for INFN-CLOUD https://iam.cloud.infn.it, as described in the Getting Started guide. Only registered users can login into the INFN-CLOUD dashboard https://my.cloud.infn.it.
Access to the INFN-CLOUD dashboard enables users to instantiate a JupyterHub service on a single VM, providing Notebooks with data persistence.
Important
This solution requires the instantiation of a JupytherHub service on top of a newly created virtual machine (VM). You will have complete control, administration rights, on the respective service and VM becoming a service administrator.
Please read the INFN Cloud AUP in order to understand the responsibilities you have in managing this service.
How to deploy and access the JupyterHub service with a MATLAB kernel¶
Step 1 - Connecting and authenticating to the INFN-CLOUD dashboard¶
Connect to the INFN-CLOUD dashboard (https://my.cloud.infn.it/ ):
Press the login button and the IAM authentication page should open
Select “Sign in with INFN CCR-AAI” button and use your AAI credential.
Step 2 - Select and Configure the Jupyter + Matlab (with persistence for Notebooks) service¶
First check that you are working in the intended project, among those you belong to, by looking at the active one in the bottom left of the dashboard (under your name).
Then go over the Jupyter + Matlab (with persistence for Notebooks) card and click on the Configure button.
After that you will have to configure your deployment. The deployment definition window consists of three tabs: “GENERAL”, “AUTHORIZATIONS” and “ADVANCED”. Before continuing please fill the first mandatory field - the “DEPLOYMENT DESCRIPTION”. You will not be able to submit your deployment without it!
“GENERAL” TAB¶
In this tab, all fields have default values but can be changed if desired. You can fill the following fields:
- NUM_CPUS
- Number of virtual CPUs for the VM that will host the Jupyter hub. The default value is 2.
- MEM_SIZE
- Amount of memory for the same VM, expressed in GB. The default value is 4.
- ENABLE_MONITORING
- Option to enable the monitoring of the VM. It is disabled by default.
- JUPYTER_IMAGES
- The image used to create the Jupyter Hub on the VM.
- Default value is “harbor.cloud.infn.it/datacloud-templates/jupyter_matlab”. If you want to build and use your own Jupyter Hub images, it is better to use the default image as a base image and add your customization on top of it.
- JUPYTERLAB_COLLABORATIVE
- Enable the new collaborative editing feature that allows collaboration in real-time between multiple users. It is disabled by default. See JupyterLab documentation for more information.
- JUPYTERLAB_COLLABORATIVE_IMAGE
- The image used to create the Jupyter Hub where it’s possible to host a collaborative Jupyter Lab instance.
- Default image for JupyterLab collaborative feature is “harbor.cloud.infn.it/datacloud-templates/collaborative_matlab”.
- CONTACT_EMAIL
- Email address of the administrator of the VM, it will be used for the certificate of the VM.
- PORTS
- List of additional ports to be opened. By default, and you don’t need to specify them, the deployment will have the following TCP ports accessible: 22 (ssh to the host VM), 3000 (for grafana dashboard), 8888 (for JupyterHub), 8889 (for jupyter collaborative)
“AUTHORIZATIONS” TAB¶
You can decide to authorize INFN Cloud user groups by filling:
- IAM_GROUPS
- user groups that are allowed to access jupyterhub services.
- IAM_ADMIN_GROUPS
- user groups that are allowed to administrate jupyterhub.
Note
INFN Cloud (https://iam.cloud.infn.it) is the IAM identity provider.
“ADVANCED” TAB¶
Advanced parameters can be configured here:
Configure Scheduling
Automatic (Default)
- The system will choose the most suitable cloud provider for the deployment
Manual
A resource provider can be selected from the list of available cloud sites
Deployment creation timeout (minutes)
- If specified the deployment will fail when the timeout is reached
Do not delete the deployment in case of failure
- It will leave the deployment in the dashboard even if it failed. Useful to check the errors in the log.
Send a confirmation email when complete
- The system will send an email to the address provided in the contact_email field under the General tab.
Step 3 - Submitting the deployment¶
After pressing the Submit button, you will be redirected to the list of your deployments from where you can follow the evolution of this new deployment. Please wait for the status to change from “CREATE_IN_PROGRESS” to “CREATE_COMPLETE”
Step 4 - Access your application¶
After the deployment is complete (Status change to “CREATE_COMPLETE”), you can click on the Details button.
A new page will appear with three tabs: “Overview”, “Input Values” and “Output Values”.
Open the “Output values” tab and click on the jupyter_endpoint link.
If this is the first time connecting to this endpoint, the jupyterhub login page should appear asking to Sign-in with OAuth 2.0. After pressing the orange button the IAM login form should open. Login with your INFN-AAI credentials and approve the request for the jh-client.
The home page should now appear. Here it is possible to initialize a jupyterlab instance by filling the following fields:
- desired image: field should be filled with the proper image, the default one can be chosen by clicking on the field and selecting the value that pops-up.
- desired memory size: you can select multiple options depending on your needs for the particular jupyterhub instance; please be aware that the maximum amount depends on the total memory assigned to the machine deployed in the first place.
- GPU: if available, a GPU can be added here.
When all the fields are correctly filled it is possible to start the jupyterhub instance. The Hub homepage should now appear. Click the “Open MATLAB” button in order to start the MATLAB Online instance, needed to run the MATLAB kernel inside the notebook.
The MathWorks login page should now open. Insert as email your AAI username followed by @infn.it domain and click next. The AAI login page should appear where you can use again your AAI credentials. Once logged in please wait for MATLAB Status to be “Complete” and the MATLAB Online instance to open.
Now it is possible to go back to the jupyterhub home page and launch a notebook with a MATLAB Kernel by clicking on the proper button.
