iRODS: quick tutorial

Enabling iRODS on Cineca hpc clusters

1.    Log-in into your own home directory on Marconi  and create a new dir (mkdir) named .irods (note the presence of a dot before irods)

2.    From link, go to point 2) and download the file chain.pem; put this file into the .irods dir

3.    Into the .irods directory edit a file named irods_environment.json and cut and paste the lines listed in the Appendix at the END of this tutorial (insert your Marconi account name)

4.     Start irods by taping iinit

iRODS commands 

iCommands are Unix utilities allowing the iRODS user to make use of a command-line interface to operate on data in the iRODS system.

Basic files and directory commands

1.    To store a file the iRODS current directory use the command: iput  [local file name]

2.    To store a directory iput -r [local directory name]

3.    To list stored files ils -A 

4.    To show current path: ipwd

5.    To create a new directory (collection): imkdir [directory name]

6.    To move among directories: icd [directory name]

More unix-like iRODS commands can be found on-line here.   

Using Metadata in iRODS

We need to insert metadata in order that other people can easily find and access to data by making a search on the DB using the iRODS tool. 
In iRODS, metadata can be used to describe data objects, collections, resources, and users. Metadata are stored as strings in the form of attribute-value-unit (AVU) triples, similar to those found in Resource Description Format (RDF). AVU triples are used for derived metadata (creation date, file size, etc.) and user-defined metadata. In iRODS, the main command line utility for handling metadata is imeta. It is used to determine, modify, list, search by, and delete iRODS metadata. In AIDA a list of metadata is predefined (see below).

1.    To add metadata: 
imeta add -d  [file name] [ATTRIBUTE] [VALUE] [UNIT:optional]
es: imeta add -d  solar.dat  Author “Jones”

2.    To add metadata to a file selection: 
imeta  addw -d <%string% > [ATTRIBUTE] [VALUE] [UNIT:optional]

where % is the jolly character. 
ex: imeta  addw -d <%dat% > Author “Jones”

3.    To list metadata: 
imeta ls -d [file name]

4.    To search on metadata
imeta qu -d <Attribute> = ‘string_to_search'
es: imeta qu -d Author = ‘Jones’
(please remember the space)

5.    Assign "inherit" permission to your home directory (move up with icd..), to give the same permissions as the parent to all subdirectories

  ichmod inherit <your_home_directory name>

6.     For AIDA publication give to the user gpedrazz write permissions using the ichmod command followed by the -r command line option:

ichmod -r write gpedrazz <directory name> 

Other unix-like commands can be found here

List of  AIDA metadata

Author [family name] , University nick name (ex. UNIPI)
Link link to data
Simulation name of simulation (ex.: TURB, KH, Reconnection, … )
Datesim date of creation of the simulation
Datein date of creation of the metadata set
Code (ex. Vlasov, PIC, MHD, …)
Coderef link to a paper presenting the numerical code
Dim Simulation dim (ex. 2D3V, …)
IClink link to Initial condition text file description
BClink link to Boundary conditions text file description
Notelink link to any file text with other important info
Dataform data format (ex. binary, HDF5, CDF, ASCII, PMML)

Further readings 

•    https://irods.org/uploads/2016/06/irods_beginner_training_2016.pdf
•    https://irods.org/documentation/
•    To access iRODS you can use also different APIs and interfaces see: https://irods.org/clients/