Exercise 0 1 – BIOI2 – Integrative BIOInformatics platforme

Getting started with the I2BC cluster

About this course | Before the session | About the cluster | Course material | Exercises

Exercise 0 - first steps with the cluster

objective > setup > o1 > o2 > o3 > bonus > recap

Objective 1

Get familiar with storage & computing spaces, when to use each of them and how to access them.

When you start using the cluster, it’s important to distinguish between storage and computing spaces and in what context you should use one or the other.

Storage spaces

Storage spaces are destined for safe, medium- to long-term storage of data, scripts and other files. For this, you should use the “partages” spaces that are also accessible from the cluster using the following paths:

Space Path Quota Usage Accessibility

User /store/USERS/<login> 20Gb For personal (yet professional) use
e.g. administrative papers Only you (read & write)

Team /store/EQUIPES/<team name>
— /store/EQUIPES/<team name>/MEMBERS/<login> <1Tb Shared team space
— Subspace only for you
e.g. shared data, codes, protocoles… Team members only (read & write)
— Read-only for all except you

Platform

Space	Path	Quota	Usage	Accessibility
User	`/store/USERS/<login>`	20Gb	For personal (yet professional) use e.g. administrative papers	Only you (read & write)
Team	`/store/EQUIPES/<team name>` — `/store/EQUIPES/<team name>/MEMBERS/<login>`	<1Tb	Shared team space — Subspace only for you e.g. shared data, codes, protocoles…	Team members only (read & write) — Read-only for all except you
Platform	`/store/plateformes/<project name>`	<5Tb	For collaborations or specific projects, created on demand (ask SICS) e.g. data & scripts for a collaborative project between teams	Read & write for all people who are part of this project

/store/plateformes/<project name>

<5Tb

For collaborations or specific projects, created on demand (ask SICS)

e.g. data & scripts for a collaborative project between teams

Read & write for all people who are part of this project

More information on the intranet

Computing spaces

We regrouped under “Computing spaces” two types of spaces:

the user homes, destined to harbour local installations of software and configuration files, they are specific to the cluster
two temporary spaces, mainly destined to harbour intermediate files and data generated by an analysis

These spaces can be accessed on the cluster using the following paths:

Space Path Quota Usage Accessibility

Home /home/<login> 200Gb Your home directory on the cluster, where you save e.g. config & local files Only you (read & write)

Data work

Space	Path	Quota	Usage	Accessibility
Home	`/home/<login>`	200Gb	Your home directory on the cluster, where you save e.g. config & local files	Only you (read & write)
Data work	`/data/work/I2BC`	20Tb	Shared between all users e.g. for intermediate or temporary files, for testing, can also be a temporary sharing space with other I2BC members	Everyone (read & write)
Scratch local	`/scratchlocal`	<1Tb (variable)	For temporary files, only on the node in question, more rapid because it’s a local space of the node	Read & write only for your files & directories

/data/work/I2BC

20Tb

Shared between all users

e.g. for intermediate or temporary files, for testing, can also be a temporary sharing space with other I2BC members

Everyone (read & write)

Scratch local /scratchlocal <1Tb (variable) For temporary files, only on the node in question, more rapid because it’s a local space of the node Read & write only for your files & directories

Backed-up vs “volatile” spaces

All spaces above are regularly backed up except for Data work and Scratch local spaces. That’s what makes them ideal for harbouring temporary and/or intermediate files and data that are generated during an analysis (generating plenty of intermediate files on backed-up spaces will slow them down and reduce the total storage capacity of the I2BC).

Clean up after yourselves

Both Data work and Scratch local spaces have a limited storage capacity that is shared between all users of the I2BC. There is no automatic cleaning for these spaces so it’s up to you to clean up after yourselves.

Take home message

1) there are 2 types of spaces: storage & computing

2) all /store spaces (“partages”) and the /home are backed up

3) /data/work and /scratchlocal shared spaces useful to save temporary/intermediate files

4) don’t forget to clean up after yourselves