Deploy Working Station for CYGNO Experiment (sys-admin nomination required)

Prerequisites

The user has to be registered in the IAM system for INFN-Cloud https://iam.cloud.infn.it/. Only registered users can login into the INFN-Cloud dashboard https://my.cloud.infn.it (for more details regarding registration please see Getting Started).

For this particular use-case, there is the possibility of accessing already existing services, without the need to be a nominated system administrator. To be able to use these services, however, it is necessary to ask for permission of use from the responsible of the CYGNO experiment, Giovanni Mazzitelli.

User responsibilities

The solution described in this guide consists on the instantiation of a Virtual Machine on the INFN-CLOUD infrastructure. The instantiation of a VM comes with the responsibility of maintaining it and all the services it hosts.

Please read the INFN Cloud AUP in order to understand the responsibilities you have in managing this service.

Deployment of the solution

After the selection of the project, choose the "Working Station for CYGNO Experiment" button from the solutions available, then press "Configure". The configuration menu is shown, in which you have to enter the name of the deployment (this field is mandatory). Parameters are split in four pages: "General", "Volumes", "Authorizations" and "Advanced".

Select either Automatic or Manual scheduling as shown below:

In the first case, the Orchestrator will take care of choosing the best available provider, in the other case it will be performed a direct submission towards one of the providers available, to be selected from the drop-down menu. In the case of manual scheduling, the flavors displayed on the next page will be those offered by the chosen provider.

General

The user has to fill the following parameters:

• if you want to enable or to disable the Grafana monitoring;
• image to be used for the Jupyter server. A default image is provided containing the CYGNO environment
  • If you want to use your own image or add new dependencies the the default image please follow the Extend and customize CYGNO container image guide
• if needed, you can specify additional ports, a single one or a range of ports, to open in the VM. By default the following ports are already opened, and you don't have to specify them: 22, 3000 (for Grafana), 8888 (for Jupyter).

Volumes

The user has to fill the following parameters:

• path to mount the volume for docker, you can keep the default value /var/lib/docker/;
• path to mount the volume for data. A default value is already provided, /data;
• size of the volume to be use to store docker data. Default value: 100GB;
• size of the VM volume to be used to store the data you want to use. Default value: 100GB

Authorizations

The user has to fill the following parameters:

• IAM Instance for end user authentication;
• IAM groups to be used for the authorization of the normal end-users;
• IAM groups for JupyterHub administrators authorization.

Advanced

In this section you can:

• set deployment creation timeout (in minutes), i.e. amount of time to wait until the deployment should be considered failed, Default value is 720 minutes
• choose to not delete the deployment in case of failure (in case further debugging is needed);
• send, or not, a confirmation email when deployment is complete. Default is "send".

Deployment result

To check the status of the deployment and its details select the "Deployments" button. Here all the user's deployments are reported with "Deployment identifier", "Status", "Creation time", "Deployed at" and the button "Details".

If the creation of a deployment fails, an additional option (retry) is introduced in the dropdown menu, allowing the user to resubmit the deployment with the same parameters:

If the deletion of a deployment fails, resulting in the status being set to DELETE_FAILED, the "delete (force)" button is displayed in the list of available actions, allowing the user to force the deletion of the deployment:

Clicking on the "Deployment identifier" or on "Details" button the details of the deployed services are shown:

• the "Overview" of the deployment;
• the "Input values" used for the configuration;
• the "Output values" such as the Grafana and Jupyter dashboard endpoints, node IP and account to be used to access the created VM.

Troubleshooting

In both the cases of automatic and manual scheduling the success of creation depends on the provider resources availability. Otherwise a "no quota" is reported as failure reason.

Extend and customize CYGNO container image

CYGNO image repository

You can find at the following link some useful information about the default images used to deploy CYGNO specific solutions and some instructions to make a custom image:

• Repo: https://github.com/infn-datacloud/dodas-docker-images/tree/master/docker/CYGNO

For the particular solution described in this guide the Dockerfile of the default image used in the deployment, cygno-lab:v1.2.0, is available under the lab directory.

• in the file assets/packages there is the list of packages use by the CYGNO project.
• in the scripts directory you will have several utilities to install specific software.

How to customize the default image

To create a new customized image, with additional software or custom environment, you have to start from the latest (that can you can find here) release of the default image and create a Dockerfile, like in the example below:

# Start from the base image
FROM harbor.cloud.infn.it/datacloud-templates/cygno-lab:<latest_release>

# Add env variables
ENV <variable1>=<value>
ENV <variable2>=<value>

# Install additional packages
RUN && yum -y install \
   <packageA> \
   <packageB> \
   && yum -y clean all \
   && rm -f /tmp/packages

# Copy and execute a custom script
COPY <script.sh> /tmp/
RUN chmod +x ./tmp/<script.sh>
RUN ./tmp/<script.sh>

Then, you can build your image and upload it, for example, on DockerHub

docker build -t <docker_user>/<custom_image>:<version> /<path>/<Dockerfile>/<directory>
docker push <docker_user>/<custom_image>:<version>