diff --git a/_episodes/03-data-management.md b/_episodes/03-data-management.md index 42f9d9d..be0533d 100644 --- a/_episodes/03-data-management.md +++ b/_episodes/03-data-management.md @@ -441,7 +441,8 @@ export SAM_EXPERIMENT=dune --> ## Getting file locations using Rucio -### What is Rucio? +### What is Rucio? + Rucio is the next-generation Data Replica service and is part of DUNE's new Distributed Data Management (DDM) system that is currently in deployment. Rucio has two functions: 1. A rule-based system to get files to Rucio Storage Elements around the world and keep them there. @@ -451,7 +452,7 @@ As of the date of the 2025 tutorial: - The Rucio client is available in CVMFS and Spack - Most DUNE users are now enabled to use it. New users may not automatically be added. -### You will need to authenticate to use read files +### You will need to authenticate to read files > #### For SL7 use justin to get a token {:.callout} diff --git a/_includes/02-storage-spaces.toc.md b/_includes/02-storage-spaces.toc.md index b3fb83a..e80b274 100644 --- a/_includes/02-storage-spaces.toc.md +++ b/_includes/02-storage-spaces.toc.md @@ -18,7 +18,7 @@ - [Tape-backed dCache](#tape-backed-dcache) - [Rucio Storage Elements](#rucio-storage-elements) - [CVMFS](#cvmfs) - - [What is my quota?](#what-is-my-quota?) + - [What is my quota?](#what-is-my-quota) - [Your home area at FNAL](#your-home-area-at-fnal) - [Your home area at CERN](#your-home-area-at-cern) - [The /app/ and /data/ areas at FNAL](#the-/app/-and-/data/-areas-at-fnal) diff --git a/_includes/03-data-management.toc.md b/_includes/03-data-management.toc.md index 23ca7ea..b3c5c90 100644 --- a/_includes/03-data-management.toc.md +++ b/_includes/03-data-management.toc.md @@ -3,20 +3,20 @@ - [Live Notes](#live-notes) - [Introduction](#introduction) - [What we need to do to produce accurate physics results](#what-we-need-to-do-to-produce-accurate-physics-results) - - [How we do it ?](#how-we-do-it-?) + - [How we do it ?](#how-we-do-it-) - [How do I use this.](#how-do-i-use-this.) - [How to find and access official data](#how-to-find-and-access-official-data) -- [Official datasets](#official-datasets) +- [Official datasets ](#Official_Datasets) - [Fast web catalog queries](#fast-web-catalog-queries) - [Command line tools and advanced queries](#command-line-tools-and-advanced-queries) - [metacat web interface](#metacat-web-interface) - [Example of finding reconstructed Monte Carlo](#example-of-finding-reconstructed-monte-carlo) - [you can use the web data catalog to do advanced searches](#you-can-use-the-web-data-catalog-to-do-advanced-searches) -- [What is metacat?](#what-is-metacat?) +- [What is metacat?](#what-is-metacat) - [Find a file in metacat](#find-a-file-in-metacat) - [Example of doing a metacat search](#example-of-doing-a-metacat-search) - [then do queries to find particular groups of files](#then-do-queries-to-find-particular-groups-of-files) - - [What do those fields mean?](#what-do-those-fields-mean?) + - [What do those fields mean?](#what-do-those-fields-mean) - [find out how much raw data there is in a run using the summary option](#find-out-how-much-raw-data-there-is-in-a-run-using-the-summary-option) - [Fast web catalog queries](#fast-web-catalog-queries) - [Command line tools and advanced queries](#command-line-tools-and-advanced-queries) @@ -24,12 +24,12 @@ - [Example of finding reconstructed Monte Carlo](#example-of-finding-reconstructed-monte-carlo) - [you can use the web data catalog to do advanced searches](#you-can-use-the-web-data-catalog-to-do-advanced-searches) - [find out how much data there is in a dataset](#find-out-how-much-data-there-is-in-a-dataset) - - [What describes a dataset?](#what-describes-a-dataset?) - - [What files are in that dataset and how do I use them?](#what-files-are-in-that-dataset-and-how-do-i-use-them?) + - [What describes a dataset?](#what-describes-a-dataset) + - [What files are in that dataset and how do I use them?](#what-files-are-in-that-dataset-and-how-do-i-use-them) - [Finding those files on disk](#finding-those-files-on-disk) - [Getting file locations using Rucio](#getting-file-locations-using-rucio) - - [What is Rucio?](#what-is-rucio?) - - [You will need to authenticate to use read files](#you-will-need-to-authenticate-to-use-read-files) + - [What is Rucio?](#what-is-rucio) + - [You will need to authenticate to read files](#you-will-need-to-authenticate-to-read-files) - [Accessing `rucio` and `justin` require a bit more](#accessing-`rucio`-and-`justin`-require-a-bit-more) - [getting a token for xroot access in AL9](#getting-a-token-for-xroot-access-in-al9) - [finding a file](#finding-a-file) diff --git a/_includes/03.2-UPS.toc.md b/_includes/03.2-UPS.toc.md index b395d93..e6ca5d9 100644 --- a/_includes/03.2-UPS.toc.md +++ b/_includes/03.2-UPS.toc.md @@ -1,3 +1,3 @@ ## Table of Contents -- [What is UPS and why do we need it?](#what-is-ups-and-why-do-we-need-it?) +- [What is UPS and why do we need it?](#what-is-ups-and-why-do-we-need-it) - [UPS basic commands](#ups-basic-commands) \ No newline at end of file diff --git a/_includes/04-Spack.toc.md b/_includes/04-Spack.toc.md index bab7abd..7066fcf 100644 --- a/_includes/04-Spack.toc.md +++ b/_includes/04-Spack.toc.md @@ -1,5 +1,5 @@ ## Table of Contents -- [What is Spack and why do we need it?](#what-is-spack-and-why-do-we-need-it?) +- [What is Spack and why do we need it?](#what-is-spack-and-why-do-we-need-it) - [Minimal spack for root analysis and file access](#minimal-spack-for-root-analysis-and-file-access) - [A more flexible environment with more packages but you have to make choices of versions](#a-more-flexible-environment-with-more-packages-but-you-have-to-make-choices-of-versions) - [Spack basic commands](#spack-basic-commands) \ No newline at end of file diff --git a/code/tocgen.py b/code/tocgen.py index 8c50517..7d4831d 100644 --- a/code/tocgen.py +++ b/code/tocgen.py @@ -10,7 +10,8 @@ def dealwithline(input,toclines): dealwithline(newlines,toclines) continue # remove any labels - line = line.split('