{2023.06,2023a} PyTorch-bundle v2.1.2 by trz42 · Pull Request #400 · NorESSI/software-layer

trz42 · 2024-06-09T18:17:57Z

Bundle for PyTorch, CPU-only.

SPDX license identifier:

Missing packages:

nessi-bot · 2024-06-09T18:18:01Z

Instance AWS-MC-NESSI is configured to build for:

architectures: x86_64/generic, x86_64/intel/skylake_avx512, x86_64/amd/zen2, aarch64/generic
repositories: nessi-2023.06-swl-deb11, nessi-2023.06-cl, nessi-2023.06-swl-deb10

nessi-bot · 2024-06-09T18:18:01Z

Instance eX3-NESSI is configured to build for:

architectures: x86_64/amd/zen2, aarch64/generic
repositories: nessi-2023.06-cl, nessi-2023.06-swl-deb11, nessi-2023.06-swl-deb10

nessi-bot · 2024-06-09T18:18:01Z

Instance Fram-NESSI is configured to build for:

architectures: x86_64/generic, x86_64/intel/broadwell
repositories: nessi-2023.06-swl-deb11, nessi-2023.06-swl-deb10, nessi-2023.06-cl

nessi-bot · 2024-06-09T18:18:02Z

Instance Saga-NESSI is configured to build for:

architectures: x86_64/intel/skylake_avx512, x86_64/intel/broadwell, x86_64/generic
repositories: nessi-2023.06-cl, nessi-2023.06-swl-deb10, nessi-2023.06-swl-deb11

trz42 · 2024-06-09T18:18:38Z

Just an initial test...

bot: build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic
bot: build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2

nessi-bot · 2024-06-09T18:18:41Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:18:41Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 218048, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- submitted job 218049, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)

nessi-bot · 2024-06-09T18:18:41Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:18:41Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:18:45Z

New job on instance eX3-NESSI for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/pilot.nessi.no/jobs/2024.06/pr_400/218048

date	job status	comment
Jun 09 06:18:44 PM UTC 2024	submitted	job id `218048` awaits release by job manager
Jun 09 06:19:20 PM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 06:20:24 PM UTC 2024	running	job `218048` is running
Jun 09 06:49:03 PM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-218048.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 06:49:03 PM UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-218048.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-09T18:18:49Z

New job on instance eX3-NESSI for architecture x86_64-amd-zen2 for repository nessi-2023.06-swl-deb11 in job dir /home/thomarob/pilot.nessi.no/jobs/2024.06/pr_400/218049

failed while pulling container

FATAL:   While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: writing blob: rename .../pilot.nessi.no/jobs/2024.06/pr_400/event_b22a0570-268c-11ef-869d-ca261f0efebd/run_001/linux_x86_64_amd_zen2/nessi-2023.06-swl-deb11/singularity_tmpdir/bundle-temp-1718401796/oci-put-blob2092045356 .../pilot.nessi.no/jobs/2024.06/pr_400/event_b22a0570-268c-11ef-869d-ca261f0efebd/run_001/linux_x86_64_amd_zen2/nessi-2023.06-swl-deb11/singularity_tmpdir/bundle-temp-1718401796/blobs/sha256/f2f58072e9ed1aa1b0143341c5ee83815c00ce47548309fa240155067ab0e698: input/output error

date	job status	comment
Jun 09 06:18:48 PM UTC 2024	submitted	job id `218049` awaits release by job manager
Jun 09 06:19:22 PM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 06:20:25 PM UTC 2024	running	job `218049` is running
Jun 09 06:21:27 PM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-218049.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 06:21:27 PM UTC 2024	test result	😁 FAILURE (click triangle for details) Reason Failed for unknown reason Details ✅ job output file `slurm-218049.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-06-09T18:33:27Z

EGT...

bot: build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2

nessi-bot · 2024-06-09T18:33:29Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:33:29Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- submitted job 218117, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)

nessi-bot · 2024-06-09T18:33:29Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:33:29Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:33:33Z

New job on instance eX3-NESSI for architecture x86_64-amd-zen2 for repository nessi-2023.06-swl-deb11 in job dir /home/thomarob/pilot.nessi.no/jobs/2024.06/pr_400/218117

date	job status	comment
Jun 09 06:33:32 PM UTC 2024	submitted	job id `218117` awaits release by job manager
Jun 09 06:33:42 PM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 06:34:46 PM UTC 2024	running	job `218117` is running
Jun 09 06:35:48 PM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-218117.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 06:35:48 PM UTC 2024	test result	😁 FAILURE (click triangle for details) Reason Failed for unknown reason Details ✅ job output file `slurm-218117.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-06-09T18:38:13Z

Fall back to AWS...

bot: build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell
bot: build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic

nessi-bot · 2024-06-09T18:38:16Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- submitted job 12393, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- submitted job 12394, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- submitted job 12395, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)

nessi-bot · 2024-06-09T18:38:16Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:38:16Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- submitted job 5816262, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:38:16Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- submitted job 11761274, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-06-09T18:38:20Z

New job on instance Fram-NESSI for architecture x86_64-intel-broadwell for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.06/pr_400/5816262

date	job status	comment
Jun 09 18:38:19 UTC 2024	submitted	job id `5816262` awaits release by job manager
Jun 09 18:39:01 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 18:40:06 UTC 2024	running	job `5816262` is running
Jun 09 19:42:45 UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-5816262.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 19:42:45 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-5816262.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-09T18:38:21Z

New job on instance AWS-MC-NESSI for architecture aarch64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12393

date	job status	comment
Jun 09 18:38:20 UTC 2024	submitted	job id `12393` awaits release by job manager
Jun 09 18:39:21 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 18:43:57 UTC 2024	running	job `12393` is running
Jun 09 19:37:47 UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-12393.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 19:37:47 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12393.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-09T18:38:25Z

New job on instance AWS-MC-NESSI for architecture x86_64-amd-zen2 for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12394

date	job status	comment
Jun 09 18:38:24 UTC 2024	submitted	job id `12394` awaits release by job manager
Jun 09 18:39:23 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 18:44:01 UTC 2024	running	job `12394` is running
Jun 09 20:07:45 UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-12394.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 20:07:45 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12394.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-09T18:38:28Z

New job on instance Saga-NESSI for architecture x86_64-intel-skylake_avx512 for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.06/pr_400/11761274

date	job status	comment
Jun 09 06:38:27 PM UTC 2024	submitted	job id `11761274` awaits release by job manager
Jun 09 06:38:50 PM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 06:39:55 PM UTC 2024	running	job `11761274` is running
Jun 09 08:28:58 PM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-11761274.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 08:28:58 PM UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-11761274.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-09T18:38:29Z

New job on instance AWS-MC-NESSI for architecture x86_64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12395

failed with several failing tests such as

=================================== FAILURES ===================================
___ test_decode_jpeg[None-ImageReadMode.UNCHANGED-grace_hopper_517x606.jpg] ____
test/test_image.py:94: in test_decode_jpeg
    img_ljpeg = decode_image(data, mode=mode)
/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u/lib/python3.11/site-packages/torchvision/io/image.py:236: in decode_image
    output = torch.ops.image.decode_image(input, mode.value)
/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/PyTorch/2.1.2-foss-2023a/lib/python3.11/site-packages/torch/_ops.py:692: in __call__
    return self._op(*args, **kwargs or {})
E   RuntimeError: decode_jpeg: torchvision not compiled with libjpeg support

this might come from the following lines in PyTorchbundle/2.1.2/foss-2023a/torchvision/vision-0.16.2/torchvision/csrc/io/image/cpu/decode_jpeg.cpp

#if !JPEG_FOUND
torch::Tensor decode_jpeg(const torch::Tensor& data, ImageReadMode mode) {
  TORCH_CHECK(
      false, "decode_jpeg: torchvision not compiled with libjpeg support");
}

relevant eb log lines should be

== installing extension torchvision 0.16.2 (5/8)...
  >> defining build environment for foss/2023a toolchain
  >> running command:
        [started at: 2024-06-09 19:33:25]
        [working dir: /tmp/nessibot/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchvision]
        [output logged in /tmp/eb-8peuouyv/eb-q9t8b76x/easybuild-run_cmd-euocj7xl.log]
        tar xzf /project/def-nessi/nessibot/shared_fs_path/easybuild/sources/p/PyTorch-bundle/extensions/torchvision-0.16.2.tar.gz
  >> command completed: exit 0, ran in < 1s
  >> applying patch torchvision-0.16.2_ffmpeg-6.0-fix.patch
  >> applying patch torchvision-0.16.2_quantized_tol.patch
==      configuring...
==      building...
==      testing...
  >> running command:
        [started at: 2024-06-09 19:33:25]
        [working dir: /tmp/nessibot/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchvision/vision-0.16.2]
        [output logged in /tmp/eb-8peuouyv/eb-q9t8b76x/easybuild-run_cmd-x4labmor.log]
        export PYTHONPATH=/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u/lib/python3.11/site-packages:$PYTHONPATH &&  /cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/Python/3.11.3-GCCcore-12.3.0/bin/python -m pip install --prefix=/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u -v --verbose  --no-deps  --ignore-installed  --no-index  --no-build-isolation  .
  >> command completed: exit 0, ran in 00h01m24s
  >> running command:
        [started at: 2024-06-09 19:34:50]
        [working dir: /tmp/nessibot/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchvision/vision-0.16.2]
        [output logged in /tmp/eb-8peuouyv/eb-q9t8b76x/easybuild-run_cmd-10b99cmb.log]
        export PYTHONPATH=/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u/lib/python3.11/site-packages:$PYTHONPATH &&  pytest -m "not xfail" -k "not test_frame_reading_mem_vs_file"" and not test_antialias_default_warning"
  >> command completed: exit 1, ran in 00h22m29s
==      ... (took 23 mins 58 secs)
== ... (took 50 mins 26 secs)
== FAILED: Installation ended unsuccessfully (build directory: /tmp/nessibot/easybuild/build/PyTorchbundle/2.1.2/foss-2023a): build failed (first 300 chars): cmd "export PYTHONPATH=/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u/lib/python3.11/site-packages:$PYTHONPATH &&  pytest -m "not xfail" -k "not test_frame_reading_mem_vs_file"" and not test_antialias_default_warning" " exited with exit code 1 and output:

build command for torchvision and selected output (from )

# output for command: export PYTHONPATH=/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u/lib/python3.11/site-packages:$PYTHONPATH &&  /cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/Python/3.11.3-GCCcore-12.3.0/bin/python -m pip install --prefix=/tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u -v --verbose  --no-deps  --ignore-installed  --no-index  --no-build-isolation  .

and

  Building wheel torchvision-0.16.2
  Compiling extensions with following flags:
    FORCE_CUDA: False
    FORCE_MPS: False
    DEBUG: False
    TORCHVISION_USE_PNG: True
    TORCHVISION_USE_JPEG: True
    TORCHVISION_USE_NVJPEG: True
    TORCHVISION_USE_FFMPEG: True
    TORCHVISION_USE_VIDEO_CODEC: True
    NVCC_FLAGS:
  Compiling with debug mode OFF
  Found PNG library
  Building torchvision with PNG image support
    libpng version: 1.6.39
    libpng include path: /cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/libpng/1.6.39-GCCcore-12.3.0/include/libpng16
  Running build on conda-build: False
  Running build on conda: False
  Building torchvision without JPEG image support
  Building torchvision without NVJPEG image support
  Building torchvision with ffmpeg support
    ffmpeg version: b'ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers\nbuilt with gcc 12.3.0 (GCC)\nconfiguration: --prefix=/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/FFmpeg/6.0-GCCcore-12.3.0 --enable-pic --enable-shared --enable-gpl --enable-version3 --enable-nonfree --cc=gcc --cxx=g++ --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libfreetype --enable-fontconfig --enable-libfribidi --enable-sdl2\nlibavutil      58.  2.100 / 58.  2.100\nlibavcodec     60.  3.100 / 60.  3.100\nlibavformat    60.  3.100 / 60.  3.100\nlibavdevice    60.  1.100 / 60.  1.100\nlibavfilter     9.  3.100 /  9.  3.100\nlibswscale      7.  1.100 /  7.  1.100\nlibswresample   4. 10.100 /  4. 10.100\nlibpostproc    57.  1.100 / 57.  1.100\n'
    ffmpeg include path: ['/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/FFmpeg/6.0-GCCcore-12.3.0/include']
    ffmpeg library_dir: ['/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/FFmpeg/6.0-GCCcore-12.3.0/lib']
  Building torchvision without video codec support

analysing the shared library image.so (shortened output) ... libpng as well as libjpeg-turbo are in the RPATH value, but only libpng16.so.16 is listed as NEEDED

nessibot@x86-64-generic-node2 /home/nessibot/pilot.nessi.no/eessi-bot-software-layer $ readelf -d /tmp/eb-8peuouyv/eb-q9t8b76x/tmpf4ry0q2u/lib/python3.11/site-packages/torchvision/image.so | tr ':' '\n'

Dynamic section at offset 0x1eb70 contains 33 entries

  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library [libpng16.so.16]
 0x0000000000000001 (NEEDED)             Shared library [libc10.so]
 0x0000000000000001 (NEEDED)             Shared library [libtorch.so]
 0x0000000000000001 (NEEDED)             Shared library [libtorch_cpu.so]
 0x0000000000000001 (NEEDED)             Shared library [libtorch_python.so]
 0x0000000000000001 (NEEDED)             Shared library [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000f (RPATH)              Library rpath
 [...
/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/libpng/1.6.39-GCCcore-12.3.0/lib
...
/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/PyTorch/2.1.2-foss-2023a/lib/python3.11/site-packages/torch/lib
...
/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/generic/software/libjpeg-turbo/2.1.5.1-GCCcore-12.3.0/lib
]

date	job status	comment
Jun 09 18:38:28 UTC 2024	submitted	job id `12395` awaits release by job manager
Jun 09 18:39:25 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 09 18:45:14 UTC 2024	running	job `12395` is running
Jun 09 20:15:42 UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-12395.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 09 20:15:42 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12395.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-11T04:25:09Z

New job on instance AWS-MC-NESSI for architecture x86_64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12531

date	job status	comment
Jun 11 04:25:08 UTC 2024	submitted	job id `12531` awaits release by job manager
Jun 11 04:25:53 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 04:26:59 UTC 2024	running	job `12531` is running
Jun 11 05:26:22 UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-12531.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 11 05:26:22 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12531.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-06-11T06:51:38Z

Added TORCHVISION_* to preinstallopts (fixed syntax) ... next try ...

bot: build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell
bot: build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake
bot: build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic

nessi-bot · 2024-06-11T06:51:42Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- submitted job 12532, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- submitted job 12533, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- submitted job 12534, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)

nessi-bot · 2024-06-11T06:51:42Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 218750, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T06:51:42Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- submitted job 5816847, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T06:51:42Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- submitted job 11772029, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T06:51:47Z

New job on instance Fram-NESSI for architecture x86_64-intel-broadwell for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.06/pr_400/5816847

date	job status	comment
Jun 11 06:51:46 UTC 2024	submitted	job id `5816847` awaits release by job manager
Jun 11 06:52:25 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 08:27:34 UTC 2024	finished	🤷 UNKNOWN (click triangle for details) Job results file `_bot_job5816847.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Jun 11 08:27:34 UTC 2024	test result	🤷 UNKNOWN (click triangle for details) Job test file `_bot_job5816847.test` does not exist in job directory, or parsing it failed.

nessi-bot · 2024-06-11T06:51:47Z

New job on instance eX3-NESSI for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/pilot.nessi.no/jobs/2024.06/pr_400/218750

failed with Illegal instruction

== 2024-06-11 09:08:39,431 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/tools/build_log.py:111 in caller_info): cmd "export PYTHONPATH=/tmp/eb-281juo7k/eb-txrx6l7i/tmpzsc___aj/lib/python3.11/site-packages:$PYTHONPATH &&  pytest test/torchtext_unittest -
k "not test_vocab_from_raw_text_file"" and not test_get_tokenizer_moses"" and not test_get_tokenizer_spacy"" and not test_download_charngram_vectors" " exited with exit code -4 and output:
Fatal Python error: Illegal instruction

Current thread 0x000040000002c980 (most recent call first):
  File "/tmp/thomarob/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchtext/text-0.16.2/test/torchtext_unittest/test_transforms.py", line 1268 in TestMaskTransform
  File "/tmp/thomarob/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchtext/text-0.16.2/test/torchtext_unittest/test_transforms.py", line 1255 in <module>
  File "/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/aarch64/generic/software/Python-bundle-PyPI/2023.06-GCCcore-12.3.0/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 178 in exec_module

date	job status	comment
Jun 11 06:51:47 AM UTC 2024	submitted	job id `218750` awaits release by job manager
Jun 11 06:52:15 AM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 06:53:17 AM UTC 2024	running	job `218750` is running
Jun 11 07:24:06 AM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-218750.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 11 07:24:06 AM UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-218750.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-11T06:51:48Z

New job on instance AWS-MC-NESSI for architecture aarch64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12532

failed with Segmentation fault

== 2024-06-11 07:27:04,241 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/tools/build_log.py:111 in caller_info): cmd "export PYTHONPATH=/tmp/eb-idhzkgrh/eb-hhf6iac_/tmpdysc5db5/lib/python3.11/site-packages:$PYTHONPATH &&  pytest test/torchtext_unittest -k "not test_vocab_from_raw_text_file"" and not test_get_tokenizer_moses"" and not test_get_tokenizer_spacy"" and not test_download_charngram_vectors" " exited with exit code -11 and output:
Fatal Python error: Segmentation fault

Current thread 0x0000400006965900 (most recent call first):
  File "/tmp/nessibot/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchtext/text-0.16.2/test/torchtext_unittest/test_transforms.py", line 1268 in TestMaskTransform
  File "/tmp/nessibot/easybuild/build/PyTorchbundle/2.1.2/foss-2023a/torchtext/text-0.16.2/test/torchtext_unittest/test_transforms.py", line 1255 in <module>
  File "/cvmfs/pilot.nessi.no/versions/2023.06/software/linux/aarch64/generic/software/Python-bundle-PyPI/2023.06-GCCcore-12.3.0/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 178 in exec_module

date	job status	comment
Jun 11 06:51:47 UTC 2024	submitted	job id `12532` awaits release by job manager
Jun 11 06:52:34 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 06:57:41 UTC 2024	running	job `12532` is running
Jun 11 07:49:12 UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-12532.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 11 07:49:12 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12532.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

nessi-bot · 2024-06-11T06:51:51Z

New job on instance AWS-MC-NESSI for architecture x86_64-amd-zen2 for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12533

date	job status	comment
Jun 11 06:51:51 UTC 2024	submitted	job id `12533` awaits release by job manager
Jun 11 06:52:36 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 06:58:45 UTC 2024	running	job `12533` is running
Jun 11 08:54:51 UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-12533.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-amd-zen2-1718095448.tar.gz` size: 10 MiB (11026736 bytes) entries: 1487 modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all `PyTorch-bundle/2.1.2-foss-2023a.lua` software under 2023.06/software/linux/x86_64/amd/zen2/software `PyTorch-bundle/2.1.2-foss-2023a` other under 2023.06/software/linux/x86_64/amd/zen2 `2023.06/init/easybuild/eb_hooks.py`
Jun 11 08:54:51 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12533.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`
Jun 11 18:44:30 UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-amd-zen2-1718095448.tar.gz` to S3 bucket succeeded

nessi-bot · 2024-06-11T06:51:55Z

New job on instance AWS-MC-NESSI for architecture x86_64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12534

date	job status	comment
Jun 11 06:51:54 UTC 2024	submitted	job id `12534` awaits release by job manager
Jun 11 06:52:38 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 06:58:48 UTC 2024	running	job `12534` is running
Jun 11 08:55:55 UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-12534.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-generic-1718095679.tar.gz` size: 10 MiB (11004382 bytes) entries: 1487 modules under 2023.06/software/linux/x86_64/generic/modules/all `PyTorch-bundle/2.1.2-foss-2023a.lua` software under 2023.06/software/linux/x86_64/generic/software `PyTorch-bundle/2.1.2-foss-2023a` other under 2023.06/software/linux/x86_64/generic `2023.06/init/easybuild/eb_hooks.py`
Jun 11 08:55:55 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12534.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`
Jun 11 18:44:52 UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-generic-1718095679.tar.gz` to S3 bucket succeeded

nessi-bot · 2024-06-11T06:52:04Z

New job on instance Saga-NESSI for architecture x86_64-intel-skylake_avx512 for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.06/pr_400/11772029

date	job status	comment
Jun 11 06:52:03 AM UTC 2024	submitted	job id `11772029` awaits release by job manager
Jun 11 06:52:36 AM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 07:26:43 AM UTC 2024	running	job `11772029` is running
Jun 11 02:25:50 PM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-11772029.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 11 02:25:50 PM UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-11772029.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-06-11T08:15:21Z

Use Saga for broadwell too...

bot: build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell

nessi-bot · 2024-06-11T08:15:25Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T08:15:25Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T08:15:25Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T08:15:25Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- submitted job 11772440, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)

nessi-bot · 2024-06-11T08:15:55Z

New job on instance Saga-NESSI for architecture x86_64-intel-broadwell for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.06/pr_400/11772440

date	job status	comment
Jun 11 08:15:54 AM UTC 2024	submitted	job id `11772440` awaits release by job manager
Jun 11 08:16:12 AM UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 08:17:16 AM UTC 2024	running	job `11772440` is running
Jun 11 10:01:24 PM UTC 2024	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-11772440.out` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts Details No artefacts were created or found.
Jun 11 10:01:24 PM UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-11772440.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-06-11T15:33:48Z

Try building for Skylake on AWS...

bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake_avx512

nessi-bot · 2024-06-11T15:33:51Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake_avx512 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512 resulted in:
- submitted job 12554, for details & status see {2023.06,2023a} PyTorch-bundle v2.1.2 #400 (comment)

nessi-bot · 2024-06-11T15:33:51Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake_avx512 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T15:33:51Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake_avx512 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T15:33:52Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake_avx512 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake_avx512 resulted in:
- no jobs were submitted

nessi-bot · 2024-06-11T15:33:55Z

New job on instance AWS-MC-NESSI for architecture x86_64-intel-skylake_avx512 for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.06/pr_400/12554

date	job status	comment
Jun 11 15:33:54 UTC 2024	submitted	job id `12554` awaits release by job manager
Jun 11 15:34:43 UTC 2024	released	job awaits launch by Slurm scheduler
Jun 11 15:40:59 UTC 2024	running	job `12554` is running
Jun 11 17:41:29 UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-12554.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1718127189.tar.gz` size: 10 MiB (11025824 bytes) entries: 1487 modules under 2023.06/software/linux/x86_64/intel/skylake_avx512/modules/all `PyTorch-bundle/2.1.2-foss-2023a.lua` software under 2023.06/software/linux/x86_64/intel/skylake_avx512/software `PyTorch-bundle/2.1.2-foss-2023a` other under 2023.06/software/linux/x86_64/intel/skylake_avx512 `2023.06/init/easybuild/eb_hooks.py`
Jun 11 17:41:29 UTC 2024	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-12554.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`
Jun 11 18:45:13 UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1718127189.tar.gz` to S3 bucket succeeded

…into nessi-2023.06-PyTorch-bundle-2.1.2-foss-2023a

trz42 · 2024-06-11T18:43:01Z

Checklist before starting deployment (setting bot:deploy label):

Check if the SPDX license identifier is provided
- skipped
Check whether builds for all required architectures succeed (SUCCESS message + reasonably sized tarball)
- different failures (Segmentation fault / Illegal instruction) when building for aarch64/generic on (AWS / eX3)
- since existing HPC systems in Sigma2/NRIS don't have aarch64/generic we skip it for the time being
- !!! Broadwell is missing !!!
Check if the PR is up-to-date with the target branch nessi.no-2023.06 in the repository (if not what are the differences)
Assess if all requested changes are sound (checking files changed on GitHub.com)
Verify that all easyconfig/s being built are included with the EB version used (if not why not)
Review changes (if any) needed to get the build(s) succeed (common changes for all architectures, changes for a single architecture, changes because of build environment specifics, etc.)

{2023.06,2023a} PyTorch-bundle v2.1.2

8191ee2

trz42 added the NESSI/2023.06 label Jun 9, 2024

use tmp for host-injections

c8c59c2

fix shell syntax for adding exports

2ef5b5a

Merge branch 'nessi.no-2023.06' of github-trz:NorESSI/software-layer …

e2d67f0

…into nessi-2023.06-PyTorch-bundle-2.1.2-foss-2023a

trz42 added bot:deploy DONT_INGEST and removed bot:deploy labels Jun 11, 2024

Conversation

trz42 commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024

Uh oh!

trz42 commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trz42 commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trz42 commented Jun 9, 2024

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited by trz42

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited

Loading

nessi-bot bot commented Jun 9, 2024 •

edited by trz42

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited by trz42

Loading

nessi-bot bot commented Jun 11, 2024 •

edited by trz42

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading

nessi-bot bot commented Jun 11, 2024 •

edited

Loading