Deploy Working Station for CYGNO Experiment (sys-admin nomination required)¶

Table of Contents

Deploy Working Station for CYGNO Experiment (sys-admin nomination required)

Prerequisites ¶

The user has to be registered in the IAM system for INFN-Cloud https://iam.cloud.infn.it/. Only registered users can login into the INFN-Cloud dashboard https://my.cloud.infn.it (for more details regarding registration please see Getting Started).

For this particular use-case, there is the possibility of accessing already existing services, without the need to be a nominated system administrator. To be able to use these services, however, it is necessary to ask for permission of use from the responsible of the CYGNO experiment, Giovanni Mazzitelli.

User responsibilities ¶

Important

The solution described in this guide consists on the instantiation of a Virtual Machine on the INFN-CLOUD infrastructure. The instantiation of a VM comes with the responsibility of maintaining it and all the services it hosts.

Please read the INFN Cloud AUP in order to understand the responsibilities you have in managing this service.

Deployment of the solution ¶

Note

If you belong to multiple projects, aka multiple IAM-groups, after login into the dashboard, from the upper right corner, select the one to be used for the deployment you intend to perform. Not all solutions are available for all projects. The resources used for the deployment will be accounted to the respective project, and impact on their available quota. See figure below.

After the selection of the project, choose the “Working Station for CYGNO Experiment” button from the solutions available, then press “Configure”. The configuration menu is shown, in which you have to enter the name of the deployment (this field is mandatory). Parameters are split in four pages: “General”, “Volumes”, “Authorizations” and “Advanced”.

Figure 1: Use-cases panel in the Dashboard

General ¶

The user has to fill the following parameters:

number of virtual CPUs for the VM (default 2 vCPUs);
amount of memory of the VM (default 4 GB);
if you want to enable or to disable the Grafana monitoring;
image to be used for the Jupyter server. A default image is provided containing the CYGNO environment
- If you want to use your own image or add new dependencies the the default image please follow the Extend and customize CYGNO container image guide
if needed, you can specify additional ports, a single one or a range of ports, to open in the VM. By default the following ports are already opened, and you don’t have to specify them: 22, 3000 (for Grafana), 8888 (for Jupyter).

Important

INFN Cloud has a clearly defined list of open ports (see reference in the Networking section). The opening of any port that is not present in the above mentioned list, must be formally requested through the INFN Cloud ServiceDesk.

../../../_images/deployment_config_general.png

Figure 2: General configuration Tab

Volumes ¶

The user has to fill the following parameters:

path to mount the volume for docker, you can keep the default value “/var/lib/docker/”;
path to mount the volume for data. A default value is already provided, “/data”;
size of the volume to be use to store docker information.. Default value: 100GB;
size of the VM volume to be used to store the data you want to use. Default value: 100GB

../../../_images/deployment_config_volumes.png

Figure 3: Volume configuration Tab

Authorizations ¶

The user has to fill the following parameters:

IAM Instance for end user authentication;
IAM groups to be used for the authorization of the normal end-users;
IAM groups for JupyterHub administrators authorization.

../../../_images/deployment_config_auth.png

Figure 4: Authorization configuration Tab

Advanced ¶

In this section you can:

set automatic (recommended) or manual (perform a direct submission towards one of the providers available) scheduling.
- in case manual scheduling is chosen a list of possible resource providers if available and you need to choose one.
set deployment creation timeout (in minutes), i.e. amount of time to wait until the deployment should be considered failed, Default value is 720 minutes
choose to not delete the deployment in case of failure (in case further debugging is needed);
send, or not, a confirmation email when deployment is complete. Default is “send”

../../../_images/deployment_config_advanced.png

Figure 5: Advanced configuration Tab

Deployment result ¶

To check the status of the deployment and its details select the “Deployments” button. Here all the user’s deployments are reported with “Deployment identifier”, “Status”, “Creation time”, “Deployed at” and the button “Details”.

../../../_images/dashboard_creation_progress.png

Figure 6: Deployment creation in progress

../../../_images/dashboard_creation_complete.png

Figure 7: Deployment completed

Clicking on the “Deployment identifier” or on “Details” button the details of the deployed services are shown:

the “Overview” of the deployment;
the “Input values” used for the configuration;
the “Output values” such as the Grafana and Jupyter dashboard endpoints, node IP and account to be used to access the created VM.

../../../_images/deployment_details_overview.png

Figure 8: Deployment overview

../../../_images/deployment_details_input.png

Figure 9: Deployment input values

../../../_images/deployment_details_output.png

Figure 10: Deployment output values

Troubleshooting ¶

In both the cases (auto and manual scheduling) the success of creation depends on the provider resources availability. Otherwise a “no quota” is reported as failure reason.

Extend and customize CYGNO container image ¶

CYGNO image repository ¶

You can find at the following link some useful information about the default images used to deploy CYGNO specific solutions and some instructions to make a custom image:

Repo: https://github.com/infn-datacloud/dodas-docker-images/tree/master/docker/CYGNO

For the particular solution described in this guide the Dockerfile of the default image used in the deployment, cygno-lab:v1.2.0, is available under the lab directory.

in the file assets/packages there is the list of packages use by the CYGNO project.
in the scripts directory you will have several utilities to install specific software.

How to customize the default image ¶

To create a new customized image, with additional software or custom environment, you have to start from the latest (that can you can find here) release of the default image and create a Dockerfile, like in the example below:

# Start from the base image
FROM harbor.cloud.infn.it/datacloud-templates/cygno-lab:<latest_release>

# Add env variables
ENV <variable1>=<value>
ENV <variable2>=<value>

# Install additional packages
RUN && yum -y install \
   <packageA> \
   <packageB> \
   && yum -y clean all \
   && rm -f /tmp/packages

# Copy and execute a custom script
COPY <script.sh> /tmp/
RUN chmod +x ./tmp/<script.sh>
RUN ./tmp/<script.sh>

Then, you can build your image and upload it, for example, on DockerHub

docker build -t <docker_user>/<custom_image>:<version> /<path>/<Dockerfile>/<directory>
docker push <docker_user>/<custom_image>:<version>