How To: Use the INFN Cloud object storage service

Introduction

This guide provides a short description about the INFN Cloud object storage and how to use the centrally managed service based on MinIO-Gateway, https://minio.cloud.infn.it (Minio@INFN-Cloud) The object storage is the persistent storage of INFN Cloud.

Prerequisites

The user has to be registered in the IAM system for INFN-Cloud https://iam.cloud.infn.it/login. Only authorized users can access the storage service, obtaining a personal private bucket and an area under /scratch in readonly mode visible to all registered users.

  • For more details regarding the registration process please see Getting Started. To use this service you don’t need the “system admin” nomination.

The Object Storage

INFN Cloud provides a distributed object storage service, centrally managed, based on Openstack Swift. This storage is the persistent one for INFN Cloud. The object storage is replicated in the two sites of backbone, Bari and CNAF. In this way the redundancy of data is guarantee. There isn’t the backup of data. Only authorized users can access the storage service. Its configuration allows users to have a personal bucket (directory) labeled as the INFN AAI username and an area under /scratch. Data stored in the personal bucket are private, data under /scratch are visible and downloadable by all the registered users. The quota for each user is 200GB.

INFN Cloud uses the Minio-gateway software on Swift to provide a simple web interface to access the storage, create buckets and manage files. It provides also S3 api for the object storage access via code.

The personal bucket can by created by the user accessing the Minio-gateway webui otherwise it is automatically created the first time user instantiates a notebook. Instead the user area under /scratch/<INFN_AAI_username> has to be created by the user.

All the services running notebooks mount in automatic way the personal bucket and the /scratch areas, so they are visible in the notebook file system as posix directories (under the cloud-storage dir).

Login to the service

To log into the Object Storage, follow the endpoint https://minio.cloud.infn.it and click on “Log with OpenID”

login

Figure 1: Authentication in the system

The user logs in to the system via IAM using INFN-AAI credentials

INFN-CCR AAI

Figure 2: Iam homepage

INFN-CCR AAI credential

Figure 3: Using INFN-AAI identity

and then authorizes the client.

Creation of the personal bucket

The user can create the personal bucket clicking on the icon “+” located at the bottom right of the page and then on the icon “create bucket”. The name of the bucket must be the INFN_AAI username.

server configuration

Figure 5: Personal bucket creation

Creation of the /scratch/<username> directory

The user can create his/her directory under /scratch, clicking on the icon “+” located near the “scratch” path on top of the page. The name of the directory must be the INFN_AAI username. A file must be uploaded to this area immediately otherwise the empty area is removed after some minutes.

server filesystem

Figure 6: readonly area in /scratch visible to all registered users

Upload and download files

The user can upload and download files from the personal bucket. He/she can also upload and download files from the “personal scratch area” and download files from the “scratch area” of other users.

For the upload of files use the “+” icon located at the bottom right of the page and then click on the icon “upload file”.

server filesystem

Figure 7: upload file to the “personal scratch area”

For the download of files use the “…” icon located near the name of files and then click on the icon “download file”.

server filesystem

Figure 8: download file from the “scratch area”