Enabling iRODS on Cineca hpc clusters
1. Log-in into your own home directory on Marconi and create a new dir (mkdir) named .irods (note the presence of a dot before irods)
2. From link, go to point 2) and download the file chain.pem; put this file into the .irods dir
3. Into the .irods directory edit a file named irods_environment.json and cut and paste the lines listed in the Appendix at the END of this tutorial (insert your Marconi account name)
4. Start irods by taping iinit
iRODS commands
iCommands are Unix utilities allowing the iRODS user to make use of a command-line interface to operate on data in the iRODS system.
Basic files and directory commands
1. To store a file the iRODS current directory use the command: iput [local file name]
2. To store a directory iput -r [local directory name]
3. To list stored files ils -A
4. To show current path: ipwd
5. To create a new directory (collection): imkdir [directory name]
6. To move among directories: icd [directory name]
More unix-like iRODS commands can be found on-line here.
Using Metadata in iRODS
We need to insert metadata in order that other people can easily find and access to data by making a search on the DB using the iRODS tool.
In iRODS, metadata can be used to describe data objects, collections, resources, and users. Metadata are stored as strings in the form of attribute-value-unit (AVU) triples, similar to those found in Resource Description Format (RDF). AVU triples are used for derived metadata (creation date, file size, etc.) and user-defined metadata. In iRODS, the main command line utility for handling metadata is imeta. It is used to determine, modify, list, search by, and delete iRODS metadata. In AIDA a list of metadata is predefined (see below).
1. To add metadata:
imeta add -d [file name] [ATTRIBUTE] [VALUE] [UNIT:optional]
es: imeta add -d solar.dat Author “Jones”
2. To add metadata to a file selection:
imeta addw -d <%string% > [ATTRIBUTE] [VALUE] [UNIT:optional]
where % is the jolly character.
ex: imeta addw -d <%dat% > Author “Jones”
3. To list metadata:
imeta ls -d [file name]
4. To search on metadata
imeta qu -d <Attribute> = ‘string_to_search'
es: imeta qu -d Author = ‘Jones’
(please remember the space)
5. Assign "inherit" permission to your home directory (move up with icd..), to give the same permissions as the parent to all subdirectories
ichmod inherit <your_home_directory name>
6. For AIDA publication give to the user gpedrazz write permissions using the ichmod command followed by the -r command line option:
ichmod -r write gpedrazz <directory name>
Other unix-like commands can be found here.
List of AIDA metadata
| Author | [family name] , University nick name (ex. UNIPI) |
| Link | link to data |
| Simulation | name of simulation (ex.: TURB, KH, Reconnection, … ) |
| Datesim | date of creation of the simulation |
| Datein | date of creation of the metadata set |
| Code | (ex. Vlasov, PIC, MHD, …) |
| Coderef | link to a paper presenting the numerical code |
| Dim | Simulation dim (ex. 2D3V, …) |
| IClink | link to Initial condition text file description |
| BClink | link to Boundary conditions text file description |
| Notelink | link to any file text with other important info |
| Dataform | data format (ex. binary, HDF5, CDF, ASCII, PMML) |
Further readings
• https://irods.org/uploads/2016/06/irods_beginner_training_2016.pdf
• https://irods.org/documentation/
• To access iRODS you can use also different APIs and interfaces see: https://irods.org/clients/