How To: Deploy Working Station for CYGNO Experiment (sys-admin nomination required)

Prerequisites

The user has to be registered in the IAM system for INFN-Cloud https://iam.cloud.infn.it/. Only registered users can login into the INFN-Cloud dashboard https://my.cloud.infn.it (for more details regarding registration please see Getting Started).

For this particular use-case, there is the possibility of accessing already existing services, without the need to be a nominated system administrator. To be able to use these services, however, it is necessary to ask for permission of use from the responsible of the CYGNO experiment, Giovanni Mazzitelli.

User responsabilities

Important

The solution described in this guide consists on the instantiation of a Virtual Machine on the INFN-CLOUD infrastructure. The instantiation of a VM comes with the responsibility of maintaining it and all the services it hosts.

Please read the INFN Cloud AUP in order to understand the responsabilities you have in managing this service.

Deployment of the solution

Note

If you belong to multiple projects, aka multiple IAM-groups, after login into the dashboard, from the upper right corner, select the one to be used for the deployment you intend to perform. Not all solutions are available for all projects. The resources used for the deployment will be accounted to the respective project, and impact on their available quota. See figure below.

../_images/howto1_01.png

After the selection of the project, choose the “Working Station for CYGNO Experiment” button from the solutions available, then press “Configure”. The configuration menu is shown, in which you have to enter the name of the deployment (this field is mandatory). Parameters are splitted in four pages: “General”, “Volumes”, “Authorizations” and “Advanced”.

../_images/howto16_fig1.png

Figure 1: Use-cases panel in the Dashboard

General

The user has to fill the following parameters:

  • number of virtual CPUs for the VM (default 2 vCPUs);
  • amount of memory of the VM (default 4 GB);
  • if you want to enable or to disable the Grafana monitoring;
  • image to be used for the Jupyter server. A default image is provided containing the CYGNO environment
  • if needed, you can specify additional ports, a single one or a range of ports, to open in the VM. By default the following ports are already opened, and you don’t hve to specify them: 22, 3000 (for Grafana), 8888 (for Jupyter).

Important

INFN Cloud has a clearly defined list of open ports (see reference in the Networking section). The opening of any port that is not present in the above mentioned list, must be formally requested through the INFN Cloud ServiceDesk.

../_images/howto16_fig2.png

Figure 2: General configuration Tab

Volumes

The user has to fill the following parameters:

  • path to mount the volume for docker, you can keep the default value “/var/lib/docker/”;
  • path to mount the volume for data. A default value is already provided, “/data”;
  • size of the volume to be use to store docker information.. Default value: 100GB;
  • size of the VM volume to be used to store the data you want to use. Default value: 100GB
../_images/howto16_fig3.png

Figure 3: Volume configuration Tab

Authorizations

The user has to fill the following parameters:

  • IAM Instance for end user authentication;
  • IAM groups to be used for the authorization of the normal end-users;
  • IAM groups for JupyterHub administrators authorization.
../_images/howto16_fig4.png

Figure 4: Authorization configuration Tab

Advanced

In this section you can:

  • set automatic (recommended) or manual (perform a direct submission towards one of the providers available) scheduling.
    • in case manual scheduling is choosed a list of possible resource providers if available and you need to choose one.
  • set deployment creation timeout (in minutes), i.e. amount of time to wait until the deployment should be considered failed, Default value is 720 minutes
  • choose to not delete the deployment in case of failure (in case further debbugging is needed);
  • send, or not, a confirmation email when deployment is complete. Deafult is “send”
../_images/howto16_fig5.png

Figure 5: Advanced configuration Tab

Deployment result

To check the status of the deployment and its details select the “Deployments” button. Here all the user’s deployments are reported with “Deployment identifier”, “Status”, “Creation time”, “Deployed at” and the button “Details”.

../_images/howto16_fig6.png

Figure 6: Deployment creation in progress

../_images/howto16_fig7.png

Figure 7: Deployment completed

Clicking on the “Deployment identifier” or on “Details” button the details of the deployed services are shown:

  • the “Overview” of the deployment;
  • the “Input values” used for the configuration;
  • the “Output values” such as the Grafana and Jupyter dashboard endpoints, node IP and account to be used to access the created VM.
../_images/howto16_fig8.png

Figure 8: Deployment overview

../_images/howto16_fig9.png

Figure 9: Deployment input values

../_images/howto16_fig10.png

Figure 10: Deployment output values

Troubleshooting

In both the cases (auto and manual scheduling) the success of creation depends on the provider resources availability. Otherwise a “no quota” is reported as failure reason.

Extend and customize CYGNO container image

CYGNO image repository

You can find at the following link some useful information about the default images used to deploy CYGNO specifuc solutions and some instructions to make a custom image:

For the particular solution described in this guide the Dockerfile of the defualt image used in the deployment, cygno-lab:v1.0.12-cygno, is available under the lab directory.

  • in the file assets/packages there is the list of packages use by the CYGNO project.
  • in the scripts directory you will have several utilities to install specific software.

How to customize the default image

To create a new customized image, with additional software or custom environment, you have to start from the latest (that can you can find here) release of the default image and create a Dockerfile, like in the example below:

# Start from the base image
FROM dodasts/cygno-lab:<latest_release>

# Add env variables
ENV <variable1>=<value>
ENV <variable2>=<value>

# Install additional packages
RUN && yum -y install \
   <packageA> \
   <packageB> \
   && yum -y clean all \
   && rm -f /tmp/packages

# Copy and execute a custom script
COPY <script.sh> /tmp/
RUN chmod +x ./tmp/<script.sh>
RUN ./tmp/<script.sh>

Then, you can build your image and upload it, for example, on DockerHub

docker build -t <docker_user>/<custom_image>:<version> /<path>/<Dockerfile>/<directory>
docker push <docker_user>/<custom_image>:<version>