Tutorial 2 NeuroData

Access organized data on NeuroData

Forgejo is a platform for hosting and managing Git repositories, similar to GitHub and GitLab. It allows users to collaborate on projects, track changes, and manage code efficiently. It is particularly useful for research projects that involve code development, version control, and collaborative work, ensuring that all team members have access to the latest updates and can contribute effectively. The NeuroPoly lab hosts its own Forgejo instance, NeuroData, which contains many datasets organized following the BIDS standard.

sequenceDiagram
   actor User
   participant Browser as NeuroData website
   participant Git as git client
   participant Server as NeuroData repository
   participant Annex as git-annex
   User->>Browser: Sign in on VPN and browse datasets
   Browser-->>User: Repository URL and BIDS tree
   User->>Git: Clone repository
   Git->>Server: Fetch metadata and annex pointers
   Server-->>Git: Repository files
   User->>Annex: git annex init; git annex get
   Annex->>Server: Request selected content
   Server-->>Annex: Imaging data
   Annex-->>User: Local BIDS dataset

Loading

Prerequisites

NeuroData hosts both the imaging data and its associated metadata. While the git protocol can handle the lightweight metadata files efficiently, it is not suitable for large imaging data. Instead, it uses git-annex to index those.

1. Install git.

2. Install git-annex.

3. If not done already, set up your NeuroData account and SSH key: follow Get access (or the intranet initial setup).

Accessing NeuroData

1. Navigate to data.neuro.polymtl.ca (VPN required) and log in with your GE credentials.

2. Select the datasets organization and browse its content.

3. Select the whole-spine dataset.

4. Peruse through the dataset content to familiarize yourself with its structure and files.

Copying the data locally

1. Go back to the main page of the whole-spine repository.

2. Click on the Clone button to copy the repository URL.

3. Open a terminal on your local machine and navigate to the directory where you want to clone the repository.

4. Clone the repository using the URL you copied earlier. For example:

   git clone <repository_url>
   cd whole-spine

5. Navigate to a specific subject and session within the dataset and open the anat folder.

6. Open a metadata file (e.g., sub-01/ses-01/anat/sub-01_ses-01_T1w.json) to inspect its contents. You should see a JSON structure containing various metadata fields related to the imaging session.

7. In the same folder, open an imaging file (e.g., sub-01/ses-01/anat/sub-01_ses-01_T1w.nii.gz) to inspect its contents. The file should contain a single line, a reference to the actual data stored by git-annex.

8. Navigate back to the root of the whole-spine repository.

9. Run the following block of git-annex commands to retrieve the actual imaging data :

   git annex init
   git annex dead here
   git annex get

Next: Part 3 - Find and download data with Neurobagel