Deploy Sync&Share aaS (sys-admin nomination required)¶

Sync&Share aaS is offered to INFN Cloud users as a means to easily share documentation and user data among a project or a scientific collaboration. Different tools should be used for scientific data archival and distribution.

Table of Contents

Deploy Sync&Share aaS (sys-admin nomination required)

Prerequisites ¶

The user has to be registered in the IAM system for INFN-CLOUD https://iam.cloud.infn.it/login. Only registered users can login into the INFN-CLOUD dashboard https://my.cloud.infn.it.

At the time being additional authorizations are needed to submit a Sync&Share deployment on INFN Cloud. Please contact the support team in order to be granted access to the INFN-Cloud Object storage backend before launching your application.

Important

The solution described in this guide consists on deployment of a Sync&Share solution on top of a Virtual Machine instantiated on INFN-CLOUD infrastructure. The instantiation of a VM comes with the responsibility of maintaining it and all the services it hosts.

Please read the INFN Cloud AUP in order to understand the responsibilities you have in managing this service.
Production level Sync&Share aaS instances should not be deployed within the infn-catch-all project as the archived data are potentially accessible by extraneous users. If you need to deploy a Sync & Share service for production use make sure to do it within your experiment project or to ask for a dedicated one.

Sync&Share aaS Configuration ¶

Note

If you belong to multiple projects, aka multiple IAM-groups, after login into the dashboard, from the upper right corner, select the one to be used for the deployment you intend to perform. Not all solutions are available for all projects. The resources used for the deployment will be accounted to the respective project, and impact on their available quota. See figure below.

Once done, select the “Sync&Share aaS” card in the service catalog and click on the Configure button.

Figure 1: The main page of the dashboard allows to select the service to be deployed. Move the cursor on the service card and click the Configure button to access the service configuration page.

The configuration parameters are split into three tabs:

Basic configuration tab ¶

Figure 2: Basic configuration tab overview. This form allows to customize Sync&Share aaS service.

The following parameters can be set in order to customize the service:

DOCKER STORAGE SIZE: size of the volume to be mounted in /var/lib/docker
CONTACT EMAIL: contact for receiving notifications (e.g. alerts from the monitoring system)
ADMIN USERNAME: username for admin access (default is admin)
ADMIN PASSWORD: password for admin user. A password is proposed by default, following secure password best-practices.
MONITORING ADMIN USERNAME: username for the admin user of the monitoring service (default is admin)
MONITORING ADMIN PASSWORD: password for the admin user of the monitoring service
BACKUP PASSPHRASE: passphrase used for encrypting the backup data
DATA BUCKET NAME: name of the bucket where data will be stored (max 27 characters)
BACKUP BUCKET NAME: name of the bucket where backup data will be stored (max 27 characters)
IAM URL: URL of the IAM (default: https://iam.cloud.infn.it) for enabling user authentication/authorization (see IAM integration section)
IAM AUTHORIZED GROUP: IAM group whose members will be authorized to access the service
VM FLAVOR: The VM “flavor”: 2 CPU and 4GB RAM for small projects involving a few dozen users, 4 CPUs and 8GB RAM for middle sized projects involving up to hundreds of users.

Important

DATA BUCKET NAME and BACKUP BUCKET NAME should follow the AWS Bucket naming rules. Please find below the link to the AWS documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html

Figure 2b: You can select the VM flavor.

Default values, if provided, can be changed; required inputs are highlighted in red if not provided. The deployment description field has to be filled with an arbitrary label that will identify your service in the list of your deployments (Figure 4).

Implementation (advanced) tab ¶

Here you can choose the type of service and whether to add an IAM group authorized to access the service as admins (applicable only with nextcloud). If there are no special needs, the use of ownCloud is recommended.

Figure 2c: Implementation (advanced) tab

Advanced configuration tab ¶

The Advanced tab allows to configure further properties of the deployment:

you can manually select the provider where the service will be instantiated;
you can set a timeout for the deployment creation;
you can decide to preserve the machine in case of failure in the service configuration (mostly used for debugging purposes).
you can enable or disable an e-mail to be sent when the deployment is completed.

In general, you do not need to change the settings provided by default in the Advanced tab, unless you have specific needs.

Figure 3: Advanced configuration tab.

Clicking on the Submit button, the deployment request is sent to the INFN Cloud Orchestration system. The deployment request can be monitored from the “My deployments” page. As soon as your deployment is ready, you will receive a notification by email.

Figure 4: Deployment list.

Deployment outputs ¶

Clicking on the deployment identifier you can retrieve the details of your deployment (Figure 5) split in three tabs:

Overview tab (Figure 5): overall information about the deployment (status, creation time, provider name, etc.);
Input values tab (Figure 6): the values used to configure the service;
Output values tab (Figure 7): the information needed to access the deployed services.

Figure 5: Deployment details - Overview tab

Figure 6: Deployment details - Input values tab

Figure 7: Deployment details - Output values tab

Access your services ¶

The “Output values” tab, as shown in the previous figure, provides the following information:

the IP and ssh account (that coincides with your AAI username) to access the VM where the services are running
the list of endpoints to access the web UI of the deployed services
- field URL of ownCloud web UI. The login credentials (username/password) are those provided in the configuration phase and are also available in the “Input values” tab;
- field status_service_endpoint: URL of the monitoring dashboard. The login credentials (username/password) are those provided in the configuration phase and are also available in the “Input values” tab;

Manage your service ¶

All the configuration files needed to run the service can be found on the VM in the folder /opt/storageservice:

storageservice
├── .client-iam.json
├── .env
├── docker-compose.yaml
├── oidc.config.php
└── traefik
    └── tls.toml

In particular,

.env file contains a list of environment variables used to customize the service (including the input parameters provided through the configuration form),
oidc.config.php and .client-iam.json files are relevant for the IAM integration (see next paragraph),
traefik/tls.toml file defines some SSL configuration options

In general, you do not need to change any settings in these files unless you have received a specific notification from the INFN Cloud Support Team/Security Incident Team.

Integration with IAM ¶

The service is automatically integrated with IAM by registering a new client dynamically. The information about the client are saved in the .client-iam.json file; in this file you can find the registration access token that can be used to modify (if needed) the client properties in IAM:

{
"client_id": "REDACTED",
"client_secret": "REDACTED",
"client_secret_expires_at": 0,
"client_id_issued_at": 1646126535,
"registration_access_token": "eyJ...REDACTED",
"registration_client_uri": "https://iam.cloud.infn.it/register/b6deeb49-2259-44a7-a8b5-347ffa069159",
"redirect_uris": [
    "https://data.90.147.174.123.myip.cloud.infn.it/apps/openidconnect/redirect"
],
"client_name": "oc-client",
"contacts": [
    "username@infn.it"
],
"token_endpoint_auth_method": "client_secret_basic",
"scope": "openid email profile",
"grant_types": [
    "authorization_code"
],
"response_types": [
    "code"
]
}

From the MitreID Dashboard of IAM you can edit your client as shown in the following screenshots:

Fill in the client id and the registration access token provided in the .client-iam.json file (see above):

Edit the client and save your changes:

How to access your data via cli ¶

ownCloud offers a user-friendly web interface for managing and sharing your data. In some cases, you may need to access your data from your environment, e.g. mounting your storage space on your local machine. Rclone is a powerful command-line tool that allows to manage files on a cloud storage. In this guide you can find some useful tips on how to use Rclone.

How to change the authorized IAM group ¶

ownCloud ¶

If you deployed an ownCloud instance and want to change the name of the IAM group that users must be members of to have access granted, you need to update the file located in /opt/storageservice/oidc.config.php. Here is an example of its content:

<?php
$CONFIG = [
  'http.cookie.samesite' => 'None',
  'openid-connect' => [
        'provider-url' => 'https://iam.cloud.infn.it',
        'client-id' => 'REDACTED',
        'client-secret' => 'REDACTED',
        'loginButtonName' => 'INFN Cloud IAM',
        'auto-provision' => [
                'enabled' => true,
                'email-claim' => 'email',
                'display-name-claim' => 'name',
                'provisioning-claim' => 'groups',
                'provisioning-attribute' => 'users/example',
        ],
        'mode' => 'userid',
        'search-attribute' => 'preferred_username',
  ]
];

The value to be updated to change the group the user authorization is based on is that on line 14, i.e. the value at $CONFIG.openid-connect.auto-provision.provision-attribute that is the users/example string.

Furthermore, to make the change effective, a restart of the service has to be performed:

cd /opt/storageservice/
docker-compose restart || docker compose restart

NextCloud ¶

If you deployed an NextCloud instance and want to change the name of the IAM group that users must be members of to have access granted, you need to modify or add groups through the service’s webui at the link https://data.<ip>.myip.cloud.infn.it/settings/admin/sociallogin, in the section Custom OpenID Connect –> add group mapping

Troubleshooting ¶

Authorization failure ¶

In case the submission of your deployment request is not accepted due to authorization issues, you will get a warning message as shown in the figure below. Please contact the support team in order to be granted access to the backend INFN-Cloud Object storage.

Figure 12: Authorization may be needed to deploy Sync&Share aaS