Use the INFN Cloud object storage service¶
Table of Contents
Introduction¶
This guide provides a short description about the INFN-Cloud object storage and how to use the centrally managed service based on rados-Gateway, https://s3webui.cloud.infn.it. The object storage is the persistent storage of INFN-Cloud.
Prerequisites¶
The user has to be registered in the IAM system for INFN-Cloud https://iam.cloud.infn.it/login. Only authorized users can access the storage service, obtaining a personal private bucket and an area under /scratch in readonly mode visible to all registered users.
- For more details regarding the registration process please see Getting Started. To use this service you don’t need the “system admin” nomination.
The Object Storage¶
INFN-Cloud provides a distributed object storage service, centrally managed, based on rados-Gateway. This storage is the persistent one for INFN-Cloud. The object storage is replicated in the two sites of backbone, Bari and CNAF. In this way the redundancy of data is guaranteed. There is not a backup of this data. Only authorized users can access the storage service. Its configuration allows users to have a personal bucket labeled as the INFN AAI username and an area under /scratch. Data stored in the personal bucket are private, data under /scratch are visible and downloadable by all the registered users. The quota for each user is 200GB.
INFN-Cloud uses the rados-gateway software to provide access to the storage, create buckets and manage files. It provides also S3 api for the object storage access via code.
The personal bucket can by created by the user accessing the rados-gateway webui otherwise it is automatically created the first time user instantiates a notebook. Instead the user area under /scratch/<INFN_AAI_username> has to be created by the user.
All the services running notebooks mount in automatic way the personal bucket and the /scratch areas, so they are visible in the notebook file system as posix directories (under the cloud-storage dir).
Login to the service¶
To log into the Object Storage, follow the endpoint https://s3webui.cloud.infn.it and click on “Login with INDIGO IAM”
Figure 1: Log in page
The user logs in to the system via IAM using INFN-AAI credentials
Figure 2: IAM login homepage
Figure 3: Using INFN-AAI identity
and then authorizes the client:
Figure 4: Client authorization
Creation of the personal bucket¶
The user can create the personal bucket clicking (1) on the button “Buckets” on the left side menu and then (2) selecting the icon “+ Create Bucket”. The name of the bucket must be the INFN_AAI username.
Figure 5: Personal bucket creation
Creation of the /scratch/<username> directory¶
The user can create his/her directory under /scratch, clicking (1) on the icon “Browser” on the left menu and then (2) on “scratch”.
Figure 6: How to select /scratch area
At this point select the icon “New path”
Figure 7: How to select the New path
Enter the name of the directory (1) that must be the INFN_AAI username and then (2) click the botton to confirm. A file must be uploaded to this area immediately otherwise the empty area is removed.
Figure 8: How to give the name to the new area under /scratch
In the following it is reported how it looks the read only area in /scratch visible to all registered users:
Figure 9: Readonly area in /scratch visible to all registered users
Upload and download files¶
The user can upload and download files from the personal bucket. He/she can also upload and download files from the “personal scratch area” and download files from the “scratch area” of other users.
For the upload of files use the “Upload File” icon then select the file to be uploaded.
Figure 10: Upload file to the “personal scratch area”
To download files, click on the selected files and then click on the “Download” botton.
Figure 11: Download file from the “scratch area”