OSPool Resource Status

Refreshes every 5 minutes
78
Resources
62
All Passing
5
Failing
11
Warnings
0
Not Available
Resource Last Updated Glideins Singularity SIF CVMFS Pelican Probes
AGLT2 4 hours ago 51
100.0%
100.0%
90.2%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
AGLT2 — probe output
pass kernel
5.14.0-687.25.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/execute/dir_3661946/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3661946 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T3_US_OSG GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.HCC_US_Michigan_gate02.idtoken.idtoken GLIDEIN_ClusterId=662995 GLIDEIN_Country=US GLIDEIN_DUNESite=US_Michigan GLIDEIN_Entry_Name=HCC_US_Michigan_gate02 GLIDEIN_Entry_Signature=e9d22badbbcf69838cbc558e5266c4b8d7ae02fd GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=gate02.aglt2.org gate02.aglt2.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_L4vrcd GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_Nickname=AGLT2 GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=AGLT2 GLIDEIN_SEs=head01.aglt2.org GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=Michigan GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=Michigan PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_3509876_291887568@c-211-23.aglt2.org$ "; unset PROMPT_COMMAND PS1=slot1@glidein_3509876_291887568@c-211-23.aglt2.org$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/var/lib/condor/execute/dir_3509868/glide_Q62oAU/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_3509868/glide_Q62oAU/execute/dir_3661946/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
1505.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg_sys-lv_condor 1707060344 41319424 1578953236 3% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.012ms pmtu 1500 1: no reply 2: no reply 3: no reply 4: chi-710n-mx1-lo0-229.aglt2.org 6.190ms 5: no reply 6: site--uwmadison-se306.lhcone.es.net 9.705ms 7: perfsonar02.hep.wisc.edu 9.639ms reached Resume: pmtu 1500 hops 7 back 8
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 789087)
AMNH-Huxley-CE1 4 hours ago 228
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
AMNH-Huxley-CE1 — probe output
pass kernel
4.18.0-553.144.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch_space/glide_CiiNjX/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch_space/glide_CiiNjX/execute/dir_3777447/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3777447 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch_space/glide_CiiNjX/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch_space/glide_CiiNjX/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_AMNH-Huxley-CE1.idtoken.idtoken GLIDEIN_CPUS=2 GLIDEIN_ClusterId=691354 GLIDEIN_Entry_Name=OSG_US_AMNH-Huxley-CE1 GLIDEIN_Entry_Signature=508c8e4f08165e30a1c8c062eff330fa0bd4272e GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=amnh-huxley-ce1.svc.osg-htc.org amnh-huxley-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_OlknUp GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=6 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=AMNH-Huxley-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-2.osg-htc.org GLIDEIN_Signature=4fff592f8ad411133f59628f88c182b905a98080 GLIDEIN_Site=AMNH GLIDEIN_Tmp_Dir=/scratch_space/glide_CiiNjX/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch_space/glide_CiiNjX/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch_space/glide_CiiNjX/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=amnh-huxley-ce1.svc.osg-htc.org OSG_SITE_NAME=AMNH-Huxley-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_2@glidein_3930655_181121550@huxley-n0010$ "; unset PROMPT_COMMAND PS1=slot1_2@glidein_3930655_181121550@huxley-n0010$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch_space/glide_CiiNjX/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch_space/glide_CiiNjX/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/scratch_space/glide_CiiNjX/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch_space/glide_CiiNjX/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch_space/glide_CiiNjX/execute/dir_3777447/scratch _CONDOR_SLOT=slot1_2 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
72.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 117162092 40867884 76294208 35% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: huxley-head-new 0.145ms 1: huxley-head-new 0.196ms 2: 216.73.255.65 8.984ms 3: 216.73.255.4 0.490ms 4: 216.73.240.230 0.617ms 5: 199.109.14.49 0.858ms 6: syr-57c3-nyc32-55a1.nysernet.net 6.582ms 7: buf-55a1-syr-57c3.nysernet.net 9.670ms 8: buf-9208-buf-55a1.nysernet.net 9.137ms 9: i2-clev-buf-9208.nysernet.net 15.243ms 10: fourhundredge-0-0-0-2.4079.core2.eqch.net.internet2.edu 20.477ms asymm 12 11: fourhundredge-0-0-0-10.4079.core1.star.net.internet2.edu 21.062ms 12: 144.92.233.112 22.667ms 13: perfsonar02.hep.wisc.edu 22.670ms reached Resume: pmtu 1500 hops 13 back 14
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1007)
Alabama-CHPC-CE1 4 hours ago 50
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Alabama-CHPC-CE1 — probe output
pass kernel
4.18.0-372.9.1.el8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_WEikm7/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_WEikm7/execute/dir_3760431/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3760431 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/tmp/glide_WEikm7/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_WEikm7/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Alabama-CHPC-CE1_mc.idtoken.idtoken GLIDEIN_CPUS=16 GLIDEIN_ClusterId=6216633 GLIDEIN_Entry_Name=OSG_US_Alabama-CHPC-CE1_mc GLIDEIN_Entry_Signature=6c36f8cebb3701111fd3dc8fec385072c4c5377c GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=ua-hpc-ce1.svc.osg-htc.org ua-hpc-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_ezfsoD GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Alabama-CHPC-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Alabama-CHPC GLIDEIN_Tmp_Dir=/tmp/glide_WEikm7/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/tmp/glide_WEikm7/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/tmp/glide_WEikm7/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ua-hpc-ce1.svc.osg-htc.org OSG_SITE_NAME=Alabama-CHPC-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_8@glidein_220029_257100324@chpc-compute001.cm.cluster$ "; unset PROMPT_COMMAND PS1=slot1_8@glidein_220029_257100324@chpc-compute001.cm.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_WEikm7/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/tmp/glide_WEikm7/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/tmp/glide_WEikm7/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_WEikm7/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_WEikm7/execute/dir_3760431/scratch _CONDOR_SLOT=slot1_8 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
13897.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda7 15418689536 845759816 14572929720 6% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: chpc-head001.cm.cluster 0.073ms 1: chpc-head001.cm.cluster 0.120ms 2: 10.8.96.2 1.913ms 3: no reply 4: 192.168.9.17 3.078ms asymm 5 5: no reply 6: EDGE-ASR9K-PE-9.oit.ua.edu 2.768ms 7: 192.168.9.14 4.094ms 8: 192.168.13.252 6.585ms 9: TCL-ASRK9-R1-4001.oit.ua.edu 5.182ms 10: 130.160.15.233 10.739ms 11: 365dc-to-56mar-re.sox.net 15.412ms asymm 10 12: i2-to-sox-re.sidco.sox.net 14.474ms asymm 11 13: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 25.634ms asymm 18 14: fourhundredge-0-0-0-23.4079.core2.cinc.net.internet2.edu 26.458ms asymm 17 15: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 25.454ms asymm 16 16: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 24.417ms asymm 15 17: 144.92.233.112 27.288ms asymm 16 18: perfsonar02.hep.wisc.edu 27.199ms reached Resume: pmtu 1500 hops 18 back 18
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 700)
BNL-SDCC-SPOOLCE01 4 hours ago 4
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
BNL-SDCC-SPOOLCE01 — probe output
pass kernel
5.14.0-611.5.1.el9_7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/execute/dir_2877240/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2877240 CONTAINERSW_FULL_VERSION=apptainer version 1.4.4-1.el9 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.4-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.DUNE_US_BNL_spoolce01.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6712557 GLIDEIN_Country=US GLIDEIN_DUNESite=US_BNL GLIDEIN_Entry_Name=DUNE_US_BNL_spoolce01 GLIDEIN_Entry_Signature=a990dc8617e74c9ced7553fec24f7eff34ce2d74 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=spoolce01.sdcc.bnl.gov spoolce01.sdcc.bnl.gov:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_064rcU GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=BNL-SDCC-SPOOLCE01 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=BNL-SDCC GLIDEIN_Tmp_Dir=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.4-1.el9 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.4-1.el9 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.4-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=BNL-SDCC PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_1@glidein_2715085_164937183@spool1597.sdcc.bnl.gov$ "; unset PROMPT_COMMAND PS1=slot1_1@glidein_2715085_164937183@spool1597.sdcc.bnl.gov$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.4-1.el9 SINGULARITY_IMAGES_DICT=default:/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.4-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/home/condor/local/sdcc/execute/dir_2714966/glide_ZNolKu/execute/dir_2877240/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
5997.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/sysvg-scratch 7048416776 404859376 6288262168 7% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.015ms pmtu 1500 1: nr0725-p1-l43.s151-0.sdcc.bnl.gov 4.295ms 1: nr0725-p1-l43.s151-0.sdcc.bnl.gov 0.557ms 2: 2620:0:210:e014::4 0.287ms 3: 2620:0:210:e013::231 0.330ms 4: fddd:7d33:c9b4:ff::51 0.978ms asymm 5 5: fddd:7d33:c9b4:ff::43 0.488ms 6: no reply 7: no reply 8: newy32aoa-cr6--wash-bb-e.igp.es.net 21.303ms asymm 16 9: wash-cr6--eqxdc4-bb-d.igp.es.net 21.250ms asymm 15 10: eqxdc4-cr6--eqxch2-bb-e.igp.es.net 21.371ms asymm 14 11: eqxch2-cr6--star-bb-c.igp.es.net 21.314ms asymm 13 12: no reply 13: site--uwmadison-se306.lhcone.es.net 25.315ms 14: perfsonar02.hep.wisc.edu 25.234ms reached Resume: pmtu 1500 hops 14 back 15
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 9878)
BNL-SDCC-SPOOLCE02 4 hours ago 4
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
BNL-SDCC-SPOOLCE02 — probe output
pass kernel
5.14.0-611.5.1.el9_7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/execute/dir_722426/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=722426 CONTAINERSW_FULL_VERSION=apptainer version 1.4.4-1.el9 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.4-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.DUNE_US_BNL_spoolce02.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6054897 GLIDEIN_Country=US GLIDEIN_DUNESite=US_BNL GLIDEIN_Entry_Name=DUNE_US_BNL_spoolce02 GLIDEIN_Entry_Signature=4935792c2a18c011e6b1bc7c30625cc1614d672c GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=spoolce02.sdcc.bnl.gov spoolce02.sdcc.bnl.gov:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_EyKRjv GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=BNL-SDCC-SPOOLCE02 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=BNL-SDCC GLIDEIN_Tmp_Dir=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.4-1.el9 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.4-1.el9 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.4-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=BNL-SDCC PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_1@glidein_1697815_37041030@spool1601.sdcc.bnl.gov$ "; unset PROMPT_COMMAND PS1=slot1_1@glidein_1697815_37041030@spool1601.sdcc.bnl.gov$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.4-1.el9 SINGULARITY_IMAGES_DICT=default:/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.4-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/home/condor/local/sdcc/execute/dir_1697645/glide_TBdVOm/execute/dir_722426/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
6057.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/sysvg-scratch 7048416776 341323672 6351797872 6% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.013ms pmtu 1500 1: nr0725-p1-l43.s151-0.sdcc.bnl.gov 0.372ms 1: nr0725-p1-l43.s151-0.sdcc.bnl.gov 0.361ms 2: 2620:0:210:e014::4 0.141ms 3: 2620:0:210:e023::229 0.186ms 4: fddd:7d33:c9b4:ff::51 0.321ms asymm 5 5: fddd:7d33:c9b4:ff::43 0.601ms 6: no reply 7: no reply 8: newy32aoa-cr6--wash-bb-e.igp.es.net 21.335ms asymm 16 9: wash-cr6--eqxdc4-bb-d.igp.es.net 21.253ms asymm 15 10: eqxdc4-cr6--eqxch2-bb-e.igp.es.net 21.358ms asymm 14 11: eqxch2-cr6--star-bb-c.igp.es.net 21.261ms asymm 13 12: no reply 13: site--uwmadison-se306.lhcone.es.net 25.260ms 14: perfsonar02.hep.wisc.edu 25.338ms reached Resume: pmtu 1500 hops 14 back 15
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 9878)
Boise-State-Borah-CE1 4 hours ago 8
0.0%
0.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Boise-State-Borah-CE1 — probe output
warning kernel
3.10.0-1160.119.1.el7.x86_64
pass env
APPTAINER_CACHEDIR=/local/osg01/glide_zM5LXe/execute/dir_64379 BASH_ENV=/cm/local/apps/environment-modules/4.4.0//init/bash BASH_FUNC_module%%=() { unset _mlshdbg; if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then case "$-" in *v*x*) set +vx; _mlshdbg='vx' ;; *v*) set +v; _mlshdbg='v' ;; *x*) set +x; _mlshdbg='x' ;; *) _mlshdbg='' ;; esac; fi; unset _mlre _mlIFS; if [ -n "${IFS+x}" ]; then _mlIFS=$IFS; fi; IFS=' '; for _mlv in ${MODULES_RUN_QUARANTINE:-}; do if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' "; fi; _mlrv="MODULES_RUNENV_${_mlv}"; _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' "; fi; done; if [ -n "${_mlre:-}" ]; then eval `eval ${_mlre}/usr/bin/tclsh /cm/local/apps/environment-modules/4.4.0/libexec/modulecmd.tcl bash '"$@"'`; else eval `/usr/bin/tclsh /cm/local/apps/environment-modules/4.4.0/libexec/modulecmd.tcl bash "$@"`; fi; _mlstatus=$?; if [ -n "${_mlIFS+x}" ]; then IFS=$_mlIFS; else unset IFS; fi; unset _mlre _mlv _mlrv _mlIFS; if [ -n "${_mlshdbg:-}" ]; then set -$_mlshdbg; fi; unset _mlshdbg; return $_mlstatus } BASH_FUNC_module()=() { unset _mlshdbg; if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then case "$-" in *v*x*) set +vx; _mlshdbg='vx' ;; *v*) set +v; _mlshdbg='v' ;; *x*) set +x; _mlshdbg='x' ;; *) _mlshdbg='' ;; esac; fi; unset _mlre _mlIFS; if [ -n "${IFS+x}" ]; then _mlIFS=$IFS; fi; IFS=' '; for _mlv in ${MODULES_RUN_QUARANTINE:-}; do if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' "; fi; _mlrv="MODULES_RUNENV_${_mlv}"; _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' "; fi; done; if [ -n "${_mlre:-}" ]; then eval `eval ${_mlre}/usr/bin/tclsh /cm/local/apps/environment-modules/4.4.0/libexec/modulecmd.tcl bash '"$@"'`; else eval `/usr/bin/tclsh /cm/local/apps/environment-modules/4.4.0/libexec/modulecmd.tcl bash "$@"`; fi; _mlstatus=$?; if [ -n "${_mlIFS+x}" ]; then IFS=$_mlIFS; else unset IFS; fi; unset _mlre _mlv _mlrv _mlIFS; if [ -n "${_mlshdbg:-}" ]; then set -$_mlshdbg; fi; unset _mlshdbg; return $_mlstatus } BASH_FUNC_switchml%%=() { typeset swfound=1; if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then typeset swname='main'; if [ -e /cm/local/apps/environment-modules/4.4.0//libexec/modulecmd.tcl ]; then typeset swfound=0; unset MODULES_USE_COMPAT_VERSION; fi; else typeset swname='compatibility'; if [ -e /cm/local/apps/environment-modules/4.4.0//libexec/modulecmd-compat ]; then typeset swfound=0; MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION; fi; fi; if [ $swfound -eq 0 ]; then echo "Switching to Modules $swname version"; source /cm/local/apps/environment-modules/4.4.0//init/bash; else echo "Cannot switch to Modules $swname version, command not found"; return 1; fi } BASH_FUNC_switchml()=() { typeset swfound=1; if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then typeset swname='main'; if [ -e /cm/local/apps/environment-modules/4.4.0//libexec/modulecmd.tcl ]; then typeset swfound=0; unset MODULES_USE_COMPAT_VERSION; fi; else typeset swname='compatibility'; if [ -e /cm/local/apps/environment-modules/4.4.0//libexec/modulecmd-compat ]; then typeset swfound=0; MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION; fi; fi; if [ $swfound -eq 0 ]; then echo "Switching to Modules $swname version"; source /cm/local/apps/environment-modules/4.4.0//init/bash; else echo "Cannot switch to Modules $swname version, command not found"; return 1; fi } BASH_FUNC_which%%=() { ( alias; eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot $@ } BATCH_SYSTEM=HTCondor BLAHPD_LOCATION=/bsuhome/osg01/bosco/glite CMD_WLM_CLUSTER_NAME=slurm CONDORCE_COLLECTOR_HOST=boise-state-borah-ce1.svc.osg-htc.org:9619 CONDOR_CONFIG=/local/osg01/glide_zM5LXe/condor_config CONDOR_PARENT_ID=gpu111:64379:1785588766 CONDOR_PROCD_ADDRESS=/local/osg01/glide_zM5LXe/log/procd_address CONDOR_PROCD_ADDRESS_BASE=/local/osg01/glide_zM5LXe/log/procd_address CONTAINERSW_FULL_VERSION=singularity version 3.7.3 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/cm/shared/apps/singularity/3.7.3/bin/singularity CONTAINERSW_VERSION=3.7.3 CPATH=/cm/shared/apps/slurm/current/include CPATH_modshare=/cm/shared/apps/slurm/current/include:1 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1022/bus DEBUGINFOD_URLS=https://debuginfod.centos.org/ ENABLE_LMOD=0 ENV=/cm/local/apps/environment-modules/4.4.0//init/profile.sh ENVIRONMENT=BATCH GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/local/osg01/glide_zM5LXe/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Boise-State-Borah-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=765096 GLIDEIN_DEBUG_OPTIONS= GLIDEIN_Entry_Name=OSG_US_Boise-State-Borah-CE1 GLIDEIN_Entry_Signature=e6061db77e73f91b9f808579b884ceddabe549c8 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=boise-state-borah-ce1.svc.osg-htc.org boise-state-borah-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_ScPp1U GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Boise-State-Borah-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=Boise-State-Borah GLIDEIN_Tmp_Dir=/local/osg01/glide_zM5LXe/tmp GLIDEIN_UUID=91d8f722-8163-45c9-9437-24f21ae30cbb GLITE_LOCATION=/bsuhome/osg01/bosco/glite GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GSI_AUTHZ_CONF=/dev/null GWMS_CONTAINERSW_FULL_VERSION=singularity version 3.7.3 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/cm/shared/apps/singularity/3.7.3/bin/singularity GWMS_CONTAINERSW_VERSION=3.7.3 GWMS_PATH=/local/osg01/glide_zM5LXe/.gwms.d/bin: GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/cm/shared/apps/singularity/3.7.3/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=3.7.3 HAS_UNPRIVILEGED_USER_NAMESPACES=unavailable HISTCONTROL=ignoredups HISTSIZE=1000 HOLD_GRACE_TIME=0 HOME=/tmp/osg01/glide_zM5LXe/execute/dir_64379 HOSTNAME=gpu111 JOB_TOKENS=/var/lib/gwms-factory/server-credentials/entry_OSG_US_Boise-State-Borah-CE1/tokens.tgz JULIA_NUM_THREADS=1 LANG=C LD_LIBRARY_PATH=/cm/shared/apps/python/3.9.7/lib:/cm/shared/apps/slurm/current/lib64/slurm:/cm/shared/apps/slurm/current/lib64:/cm/local/apps/gcc/9.2.0/lib:/cm/local/apps/gcc/9.2.0/lib64 LD_LIBRARY_PATH_modshare=/cm/shared/apps/slurm/current/lib64:1:/cm/local/apps/gcc/9.2.0/lib:1:/cm/shared/apps/python/3.9.7/lib:1:/cm/shared/apps/slurm/current/lib64/slurm:1:/cm/local/apps/gcc/9.2.0/lib64:1 LESSOPEN=||/usr/bin/lesspipe.sh %s LIBRARY_PATH=/cm/shared/apps/slurm/current/lib64/slurm:/cm/shared/apps/slurm/current/lib64 LIBRARY_PATH_modshare=/cm/shared/apps/slurm/current/lib64:1:/cm/shared/apps/slurm/current/lib64/slurm:1 LOADEDMODULES=gcc/9.2.0:slurm/slurm/23.02.7:python/3.9.7 LOADEDMODULES_modshare=python/3.9.7:1:slurm/slurm/23.02.7:1:gcc/9.2.0:1 LOGNAME=osg01 MAIL=/var/spool/mail/osg01 MANPATH=/cm/shared/apps/slurm/current/man:: MANPATH_modshare=:1:/cm/shared/apps/slurm/current/man:1 MKL_NUM_THREADS=1 MODULEPATH=/cm/local/modulefiles:/cm/shared/modulefiles MODULESHOME=/cm/local/apps/environment-modules/4.4.0/ MODULES_CMD=/cm/local/apps/environment-modules/4.4.0/libexec/modulecmd.tcl NODE_COUNT=4 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_APP=UNAVAILABLE OSG_DATA=UNAVAILABLE OSG_GRID=/root OSG_HOSTNAME=boise-state-borah-ce1.svc.osg-htc.org OSG_SITE_NAME=Boise-State-Borah-CE1 OSG_WN_TMP=/local/osg01/glide_zM5LXe/execute/dir_64379/.local-tmp PATH=/local/osg01/glide_zM5LXe/.gwms.d/bin::/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/cm/shared/apps/python/3.9.7/bin:/cm/shared/apps/slurm/current/sbin:/cm/shared/apps/slurm/current/bin:/cm/local/apps/gcc/9.2.0/bin:/cm/local/apps/environment-modules/4.4.0//bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/sbin:/sbin:/usr/sbin:/cm/local/apps/environment-modules/4.4.0/bin:/opt/dell/srvadmin/sbin:/bsuhome/osg01/.local/bin:/bsuhome/osg01/bin PATH_modshare=/cm/local/apps/environment-modules/4.4.0//bin:1:/opt/dell/srvadmin/sbin:1:/usr/bin:1:/usr/local/bin:1:/cm/shared/apps/slurm/current/bin:1:/cm/shared/apps/python/3.9.7/bin:1:/cm/shared/apps/slurm/current/sbin:1:/cm/local/apps/environment-modules/4.4.0/bin:1:/bsuhome/osg01/bin:1:/sbin:1:/bsuhome/osg01/.local/bin:1:/usr/sbin:1:/usr/local/sbin:1:/cm/local/apps/gcc/9.2.0/bin:1 PREEMPT_GRACE_TIME=10000000 PWD=/tmp/osg01/glide_zM5LXe/execute/dir_64379 PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SELINUX_LEVEL_REQUESTED= SELINUX_ROLE_REQUESTED= SELINUX_USE_CURRENT_RANGE= SHELL=/bin/bash SHLVL=6 SINGULARITY_CACHEDIR=/local/osg01/glide_zM5LXe/execute/dir_64379 SINGULARITY_FULL_VERSION=singularity version 3.7.3 SINGULARITY_IMAGES_DICT=default:/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_MODE=privileged SINGULARITY_PATH=/cm/shared/apps/singularity/3.7.3/bin/singularity SINGULARITY_VERSION=3.7.3 SLOTS_LAYOUT=partitionable SLURMD_NODENAME=gpu111 SLURM_CLUSTER_NAME=slurm SLURM_CONF=/cm/shared/apps/slurm/var/etc/slurm/slurm.conf SLURM_CPUS_ON_NODE=4 SLURM_GTIDS=0 SLURM_JOBID=3144434 SLURM_JOB_ACCOUNT=osg01 SLURM_JOB_CPUS_PER_NODE=4 SLURM_JOB_END_TIME=1785846628 SLURM_JOB_GID=1022 SLURM_JOB_ID=3144434 SLURM_JOB_NAME=bl_XGjFuj SLURM_JOB_NODELIST=gpu111 SLURM_JOB_NUM_NODES=1 SLURM_JOB_PARTITION=osg SLURM_JOB_QOS=normal SLURM_JOB_START_TIME=1785587428 SLURM_JOB_UID=1022 SLURM_JOB_USER=osg01 SLURM_LOCALID=0 SLURM_MEM_PER_NODE=32768 SLURM_NNODES=1 SLURM_NODEID=0 SLURM_NODELIST=gpu111 SLURM_NODE_ALIASES=(null) SLURM_NPROCS=4 SLURM_NTASKS=4 SLURM_NTASKS_PER_NODE=4 SLURM_PRIO_PROCESS=0 SLURM_PROCID=0 SLURM_SUBMIT_DIR=/bsuhome/osg01/bosco/sandbox/e8ed/e8ede698/boise-state-borah-ce1.svc.osg-htc.org_9619_boise-state-borah-ce1.svc.osg-htc.org_23287.0_1785587417 SLURM_SUBMIT_HOST=borah-login SLURM_TASKS_PER_NODE=4 SLURM_TASK_PID=22164 SLURM_TOPOLOGY_ADDR=gpu111 SLURM_TOPOLOGY_ADDR_PATTERN=node SLURM_WORKING_CLUSTER=slurm:borah-hn001:6817:9984:109 SQUEUE_SORT=t SSH_CLIENT=128.105.82.94 17128 22 SSH_CONNECTION=128.105.82.94 17128 132.178.231.141 22 TEMP=/local/osg01/glide_zM5LXe/execute/dir_64379 TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/local/osg01/glide_zM5LXe/execute/dir_64379 TMPDIR=/local/osg01/glide_zM5LXe/execute/dir_64379/.local-tmp TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM TZ=:/etc/localtime USER=osg01 XDG_RUNTIME_DIR=/run/user/1022 XDG_SESSION_ID=346 _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_ANCESTOR_48901=48907:1785587715:942534855 _CONDOR_ANCESTOR_48907=64379:1785588765:273980711 _CONDOR_ANCESTOR_64379=64836:1785588767:2782214636 _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/local/osg01/glide_zM5LXe/main/condor/bin _CONDOR_CHIRP_CONFIG=/local/osg01/glide_zM5LXe/execute/dir_64379/.chirp.config _CONDOR_CREDS=/local/osg01/glide_zM5LXe/execute/dir_64379/.condor_creds _CONDOR_EXECUTE=/local/osg01/glide_zM5LXe/execute _CONDOR_JOB_AD=/local/osg01/glide_zM5LXe/execute/dir_64379/.job.ad _CONDOR_JOB_IWD=/local/osg01/glide_zM5LXe/execute/dir_64379 _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/local/osg01/glide_zM5LXe/execute/dir_64379/.machine.ad _CONDOR_SCRATCH_DIR=/local/osg01/glide_zM5LXe/execute/dir_64379 _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/local/osg01/glide_zM5LXe/execute/dir_64379/.job_wrapper_failure _LMFILES_=/cm/local/modulefiles/gcc/9.2.0:/cm/local/modulefiles/slurm/slurm/23.02.7:/cm/shared/modulefiles/python/3.9.7 _LMFILES__modshare=/cm/shared/modulefiles/python/3.9.7:1:/cm/local/modulefiles/slurm/slurm/23.02.7:1:/cm/local/modulefiles/gcc/9.2.0:1 which_declare=declare -f
pass df
361.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p4 382609888 3192932 379416956 1% /tmp
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: borah-hn001.cm.cluster 0.274ms 1: borah-hn001.cm.cluster 0.164ms 2: l3-c3-nexus-01-10-32-4-2.boisestate.edu 0.483ms 3: l3-dc-10-32-0-9.boisestate.edu 8.092ms asymm 4 4: fw-5450-ae1-616.boisestate.edu 8.370ms 5: l3-101mec-204-sci-dmz-01-po53-500.boisestate.edu 9.155ms 6: 74.118.19.33 9.184ms 7: 74.118.19.130 16.773ms 8: hundredge-0-0-0-24.4007.core1.salt.net.internet2.edu 17.959ms 9: fourhundredge-0-0-0-0.4079.core1.denv.net.internet2.edu 52.307ms asymm 17 10: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 51.448ms asymm 16 11: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 50.917ms asymm 15 12: fourhundredge-0-0-0-21.4079.core1.chic.net.internet2.edu 51.143ms asymm 14 13: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 50.925ms 14: 144.92.233.112 53.429ms 15: perfsonar02.hep.wisc.edu 53.336ms reached Resume: pmtu 1500 hops 15 back 16
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1022)
CHTC-DEV-PROD-EP-mwestphall 10 hours ago 1
100.0%
0.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
CHTC-DEV-PROD-EP-mwestphall — probe output
pass kernel
5.14.0-708.el9.aarch64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-iXKR2s/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-iXKR2s/execute/dir_7706 BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=7706 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=CHTC PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_1@CHTC-DEV-PROD-EP-mwestphall.50c8ae7ed0c0$ "; unset PROMPT_COMMAND PS1=slot1_1@CHTC-DEV-PROD-EP-mwestphall.50c8ae7ed0c0$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv SINGULARITY_CONTAINER=/pilot/osgvo-pilot-iXKR2s/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-iXKR2s/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-iXKR2s/execute/dir_7706 _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure
pass df
74.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on overlay 263112772 172099316 77575300 69% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.038ms 1: _gateway 0.052ms 2: 128.104.100.2 0.365ms 3: 146.151.164.1 0.399ms 4: perfsonar02.hep.wisc.edu 0.332ms reached Resume: pmtu 1500 hops 4 back 4
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 1000)
CHTC-Spark-CE1 4 hours ago 23
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
CHTC-Spark-CE1 — probe output
pass kernel
5.14.0-511.el9.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/lib/condor/execute/osg01/glide_6V0vDd/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/var/lib/condor/execute/osg01/glide_6V0vDd/execute/dir_1385953/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=1385953 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/osg01/glide_6V0vDd/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_CHTC-Spark-CE1.idtoken.idtoken GLIDEIN_CPUS=64 GLIDEIN_ClusterId=6054014 GLIDEIN_Entry_Name=OSG_US_CHTC-Spark-CE1 GLIDEIN_Entry_Signature=0d99fde496828b5f23979f237510d39f07522370 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=chtc-spark-ce1.svc.osg-htc.org chtc-spark-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_hTzbGK GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=2 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=CHTC-Spark-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=51c806521342f21128eec483be00d289d526592b GLIDEIN_Site=CHTC-Spark GLIDEIN_Tmp_Dir=/var/lib/condor/execute/osg01/glide_6V0vDd/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=chtc-spark-ce1.svc.osg-htc.org OSG_SITE_NAME=CHTC-Spark-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_53@glidein_3305710_296069040@spark-a004.chtc.wisc.edu$ "; unset PROMPT_COMMAND PS1=slot1_53@glidein_3305710_296069040@spark-a004.chtc.wisc.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/lib/condor/execute/osg01/glide_6V0vDd/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/var/lib/condor/execute/osg01/glide_6V0vDd/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/osg01/glide_6V0vDd/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/osg01/glide_6V0vDd/execute/dir_1385953/scratch _CONDOR_SLOT=slot1_53 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
464.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sdb1 921816860 388027372 486888652 45% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 4092 1: _gateway 0.086ms 1: _gateway 0.077ms 2: _gateway 0.065ms pmtu 1500 2: 128.104.100.3 0.216ms 3: 146.151.164.9 0.292ms 4: perfsonar02.hep.wisc.edu 0.248ms reached Resume: pmtu 1500 hops 4 back 4
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 4977)
CHTC-ce2000 4 hours ago 10
100.0%
100.0%
50.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
CHTC-ce2000 — probe output
pass kernel
6.12.0-211.34.1.el10_2.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/execute/dir_279170/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=279170 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el10_2 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/bin/apptainer CONTAINERSW_VERSION=1.5.3-1.el10_2 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 FTP_PROXY=http://squid-cs-2360.chtc.wisc.edu:3128 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_CHTC-ce2000.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=6784999 GLIDEIN_Entry_Name=OSG_US_CHTC-ce2000 GLIDEIN_Entry_Signature=74e823315c4f232d38a9a12e0c570d744f5eeed1 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=ce2000.chtc.wisc.edu ce2000.chtc.wisc.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osgvopilot_26hzXt GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=squid-cs-2360.chtc.wisc.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=CHTC-ce2000 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=CHTC GLIDEIN_Tmp_Dir=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el10_2 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-1.el10_2 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-1.el10_2 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ce2000.chtc.wisc.edu OSG_SITE_NAME=CHTC-ce2000 OSG_SQUID_LOCATION=squid-cs-2360.chtc.wisc.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_3@glidein_209116_653193912@e2016.chtc.wisc.edu$ "; unset PROMPT_COMMAND PS1=slot1_3@glidein_209116_653193912@e2016.chtc.wisc.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-1.el10_2 SINGULARITY_IMAGES_DICT=default:/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/bin/apptainer SINGULARITY_VERSION=1.5.3-1.el10_2 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/slot1/dir_209100/scratch/glide_urV0Hx/execute/dir_279170/scratch _CONDOR_SLOT=slot1_3 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=http://squid-cs-2360.chtc.wisc.edu:3128
pass df
54.6 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/condor_vg-slot1_6+7895_272 66002476 8722980 57263112 14% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.010ms pmtu 1500 1: 2607:f388:2200:100::2 0.693ms 1: 2607:f388:2200:100::2 0.498ms 2: 2607:f388:0:33f::2 0.321ms 3: perfsonar02.hep.wisc.edu 2.188ms reached Resume: pmtu 1500 hops 3 back 3
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy http://squid-cs-2360.chtc.wisc.edu:3128
pass user
osgvopilot (uid 4984)
CHTC-ce2001 4 hours ago 10
100.0%
100.0%
60.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
CHTC-ce2001 — probe output
pass kernel
5.14.0-620.el9.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/images/htc__debian__13.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__debian__13.sif APPTAINER_TMPDIR=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/execute/dir_2618822/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2618822 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 FTP_PROXY=http://squid-wid.chtc.wisc.edu:3128 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_CHTC-ce2001.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=561423 GLIDEIN_Entry_Name=OSG_US_CHTC-ce2001 GLIDEIN_Entry_Signature=b022e978485ad8d00b2074eb2df48cafa04c3f25 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=ce2001.chtc.wisc.edu ce2001.chtc.wisc.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osgvopilot_zAug35 GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=CHTC-ce2001 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=CHTC GLIDEIN_Tmp_Dir=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ce2000.chtc.wisc.edu OSG_SITE_NAME=CHTC-ce2000 OSG_SQUID_LOCATION=squid-cs-2360.chtc.wisc.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_7@glidein_819242_480111095@bartels0001.chtc.wisc.edu$ "; unset PROMPT_COMMAND PS1=slot1_7@glidein_819242_480111095@bartels0001.chtc.wisc.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/images/htc__debian__13.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/images/htc__debian__13.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__debian__13.sif SINGULARITY_PATH=/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/slot1/dir_819240/scratch/glide_igr27f/execute/dir_2618822/scratch _CONDOR_SLOT=slot1_7 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=http://squid-wid.chtc.wisc.edu:3128
pass df
1028.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p1 1537155984 380794000 1078204980 27% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.005ms pmtu 1500 1: 2607:f388:107c:501::2 0.575ms 1: 2607:f388:107c:501::2 0.433ms 2: 2607:f388:0:377::2 0.298ms 3: perfsonar02.hep.wisc.edu 0.292ms reached Resume: pmtu 1500 hops 3 back 3
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy http://squid-wid.chtc.wisc.edu:3128
pass user
osgvopilot (uid 4984)
Cincinnati-ARC-OSG-CE1 4 hours ago 8
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Cincinnati-ARC-OSG-CE1 — probe output
pass kernel
4.18.0-553.30.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch-local/glide_bnZ6JF/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch-local/glide_bnZ6JF/execute/dir_3653634/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3653634 CONTAINERSW_FULL_VERSION=apptainer version 1.3.6-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.3.6-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch-local/glide_bnZ6JF/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Cincinnati-ARC-OSG-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6212173 GLIDEIN_Entry_Name=OSG_US_Cincinnati-ARC-OSG-CE1 GLIDEIN_Entry_Signature=26385b96fe80cbc1cb861b5a6368a5cee729d44f GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=cincinnati-arc-osg-ce1.svc.opensciencegrid.org cincinnati-arc-osg-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_7UcUQO GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Cincinnati-ARC-OSG-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Cincinnati-ARC-OSG GLIDEIN_Tmp_Dir=/scratch-local/glide_bnZ6JF/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.3.6-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.3.6-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.3.6-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=cincinnati-arc-osg-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Cincinnati-ARC-OSG-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_4@glidein_2852670_78385846@compute-4.localdomain$ "; unset PROMPT_COMMAND PS1=slot1_4@glidein_2852670_78385846@compute-4.localdomain$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch-local/glide_bnZ6JF/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.3.6-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch-local/glide_bnZ6JF/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.3.6-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch-local/glide_bnZ6JF/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch-local/glide_bnZ6JF/execute/dir_3653634/scratch _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
291.2 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/localscratch-lv_local_scratch 460365864 131573800 305333248 31% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: arc-osg 0.226ms 1: arc-osg 0.250ms 2: msbb-00g95-hpc.manage.uc.edu 0.458ms asymm 3 3: 10.0.26.17 0.734ms 4: 10.0.26.253 2.287ms 5: 10.0.0.2 1.312ms 6: 129.137.1.3 1.963ms 7: 129.137.1.5 2.743ms 8: cncne-r4-et-0-2-0s812.core.oar.net 6.381ms asymm 9 9: cncno-r5-et-4-3-0s735.core.oar.net 2.846ms asymm 8 10: hundredge-0-0-0-25.1260.core1.cinc.net.internet2.edu 4.442ms asymm 9 11: fourhundredge-0-0-0-22.4079.core2.cinc.net.internet2.edu 10.284ms asymm 14 12: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 9.106ms asymm 13 13: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 10.993ms asymm 12 14: 144.92.233.112 11.715ms asymm 13 15: perfsonar02.hep.wisc.edu 11.884ms reached Resume: pmtu 1500 hops 15 back 15
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1001)
Clemson-Palmetto-CE1 4 hours ago 271
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Clemson-Palmetto-CE1 — probe output
pass kernel
4.18.0-553.123.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/local_scratch/slurm.14859201/glide_OBwq3Z/images/htc__debian__13.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__debian__13.sif APPTAINER_TMPDIR=/local_scratch/slurm.14859201/glide_OBwq3Z/execute/dir_2578715/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2578715 CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-3.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.5-3.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/local_scratch/slurm.14859201/glide_OBwq3Z/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Clemson-Palmetto-CE1.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=757030 GLIDEIN_Entry_Name=OSG_US_Clemson-Palmetto-CE1 GLIDEIN_Entry_Signature=e4964bc2d8ea3ce4bbe3f8509a146e0bda8622b5 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=clemson-palmetto-ce1.svc.opensciencegrid.org clemson-palmetto-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg001_AzOyzX GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Clemson-Palmetto-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-2.osg-htc.org GLIDEIN_Signature=6295e4e4a537bce420081a31b816d769daa1eb6c GLIDEIN_Site=Clemson-Palmetto GLIDEIN_Tmp_Dir=/local_scratch/slurm.14859201/glide_OBwq3Z/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-3.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.5-3.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.5-3.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=clemson-palmetto-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Clemson-Palmetto-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_7@glidein_1478683_188797984@node1868.palmetto.clemson.edu$ "; unset PROMPT_COMMAND PS1=slot1_7@glidein_1478683_188797984@node1868.palmetto.clemson.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/local_scratch/slurm.14859201/glide_OBwq3Z/images/htc__debian__13.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.5-3.el8 SINGULARITY_IMAGES_DICT=default:/local_scratch/slurm.14859201/glide_OBwq3Z/images/htc__debian__13.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__debian__13.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.5-3.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/local_scratch/slurm.14859201/glide_OBwq3Z/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/local_scratch/slurm.14859201/glide_OBwq3Z/execute/dir_2578715/scratch _CONDOR_SLOT=slot1_7 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
365.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda6 433644668 50030108 383614560 12% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: 10.125.0.1 5.836ms 1: 10.125.0.1 28.048ms 2: 205-186-62-81.generic.c-light.net 0.458ms 3: xe-1-1-1-816-t01-cle.culr.net 7.040ms 4: xe-1-1-1-816-t01-sox.culr.net 11.801ms asymm 9 5: sidco-to-56mar-al2s.sox.net 17.420ms asymm 11 6: i2-to-sox-re.sidco.sox.net 18.709ms asymm 9 7: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 25.851ms asymm 12 8: fourhundredge-0-0-0-21.4079.core2.cinc.net.internet2.edu 25.436ms asymm 11 9: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 25.519ms asymm 10 10: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 26.698ms asymm 9 11: 144.92.233.112 27.823ms asymm 10 12: perfsonar02.hep.wisc.edu 27.788ms reached Resume: pmtu 9000 hops 12 back 12
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg001 (uid 527198)
Colgate-CCARE-EP 5 hours ago 2
100.0%
100.0%
0.0%
100.0%
all
Colgate-CCARE-EP — probe output
warning all
Job failed with exit code 0; no results transferred
Duke-NCShare-CE1 4 hours ago 23
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Duke-NCShare-CE1 — probe output
pass kernel
6.8.0-111-generic
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_VOkEB5/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/glide_VOkEB5/execute/dir_2327105/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2327105 CONTAINERSW_FULL_VERSION=apptainer version 1.4.5 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.5 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_VOkEB5/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Duke-NCShare-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6781005 GLIDEIN_Entry_Name=OSG_US_Duke-NCShare-CE1 GLIDEIN_Entry_Signature=1883db760b0cddb6ed9dd4fc7124debdb8642fcd GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=duke-ncshare-ce1.svc.osg-htc.org duke-ncshare-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_nosg1_a3sEYI GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=4 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Duke-NCShare-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Duke-NCShare GLIDEIN_Tmp_Dir=/scratch/glide_VOkEB5/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.5 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.5 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.5 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=duke-ncshare-ce1.svc.osg-htc.org OSG_SITE_NAME=Duke-NCShare-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_1582404_984296866@compute-07$ "; unset PROMPT_COMMAND PS1=slot1@glidein_1582404_984296866@compute-07$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_VOkEB5/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.5 SINGULARITY_IMAGES_DICT=default:/scratch/glide_VOkEB5/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.5 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_VOkEB5/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_VOkEB5/execute/dir_2327105/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
729.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/ubuntu--vg-lstor--lv 824572824 17497200 765116200 3% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.324ms asymm 2 1: _gateway 0.252ms asymm 2 2: rtp-gw-to-scidmz-gw-1.ncren.net 0.699ms 3: bundle-ether250.613.core1.rale.net.internet2.edu 3.645ms 4: fourhundredge-0-0-0-6.4079.core1.ashb.net.internet2.edu 21.521ms asymm 10 5: fourhundredge-0-0-0-1.4079.core1.clev.net.internet2.edu 22.786ms asymm 9 6: fourhundredge-0-0-0-2.4079.core1.eqch.net.internet2.edu 21.188ms asymm 8 7: fourhundredge-0-0-0-6.4079.core1.star.net.internet2.edu 22.760ms 8: 144.92.233.112 23.687ms 9: perfsonar02.hep.wisc.edu 23.768ms reached Resume: pmtu 1500 hops 9 back 10
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
nosg1 (uid 3000012)
FANDM-ITS-CE1 4 hours ago 32
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
FANDM-ITS-CE1 — probe output
pass kernel
4.18.0-553.50.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_LKWLws/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/glide_LKWLws/execute/dir_3803613/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3803613 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_LKWLws/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_FANDM-ITS-CE1.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=6507028 GLIDEIN_Entry_Name=OSG_US_FANDM-ITS-CE1 GLIDEIN_Entry_Signature=6060e0492587dc15606f9fc7025a13313c4b9f38 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=fandm-its-ce1.svc.opensciencegrid.org fandm-its-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_Cns1eO GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=FANDM-ITS-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=FANDM-ITS GLIDEIN_Tmp_Dir=/scratch/glide_LKWLws/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=fandm-its-ce1.svc.opensciencegrid.org OSG_SITE_NAME=FANDM-ITS-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_2@glidein_2051246_186826135@n05.cluster$ "; unset PROMPT_COMMAND PS1=slot1_2@glidein_2051246_186826135@n05.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_LKWLws/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/glide_LKWLws/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_LKWLws/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_LKWLws/execute/dir_3803613/scratch _CONDOR_SLOT=slot1_2 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
999.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p5 1092553500 44673148 1047880352 5% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: cv01.cluster 0.169ms 1: cv01.cluster 0.202ms 2: 10.68.128.1 3.726ms 3: 10.68.0.9 1.237ms 4: 10.68.0.14 0.454ms 5: 10.255.255.25 1.118ms 6: 10.255.255.34 1.257ms asymm 5 7: 164.153.140.76 2.829ms asymm 6 8: 172.24.1.20 10.183ms asymm 7 9: hundredge-0-0-0-25.68.core1.pitt.net.internet2.edu 13.514ms asymm 8 10: fourhundredge-0-0-0-3.4079.core1.clev.net.internet2.edu 21.749ms asymm 13 11: fourhundredge-0-0-0-2.4079.core1.eqch.net.internet2.edu 22.015ms asymm 12 12: fourhundredge-0-0-0-6.4079.core1.star.net.internet2.edu 22.218ms asymm 11 13: 144.92.233.112 22.604ms asymm 12 14: perfsonar02.hep.wisc.edu 23.040ms reached Resume: pmtu 1500 hops 14 back 14
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1004)
FIU-IRCC-CE1 4 hours ago 16
0.0%
0.0%
87.5%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
FIU-IRCC-CE1 — probe output
warning kernel
3.10.0-1127.el7.x86_64
pass env
APPTAINER_CACHEDIR=/scratch/osg/glide_mTQIp8/execute/dir_12549 BASH_ENV=/home/share/Modules/4.1.3/init/bash BASH_FUNC_module%%=() { unset _mlre _mlIFS _mlshdbg; if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then case "$-" in *v*x*) set +vx; _mlshdbg='vx' ;; *v*) set +v; _mlshdbg='v' ;; *x*) set +x; _mlshdbg='x' ;; *) _mlshdbg='' ;; esac; fi; if [ -n "${IFS+x}" ]; then _mlIFS=$IFS; fi; IFS=' '; for _mlv in ${MODULES_RUN_QUARANTINE:-}; do if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' "; fi; _mlrv="MODULES_RUNENV_${_mlv}"; _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' "; fi; done; if [ -n "${_mlre:-}" ]; then eval `eval ${_mlre}/usr/bin/tclsh /home/share/Modules/4.1.3/libexec/modulecmd.tcl bash '"$@"'`; else eval `/usr/bin/tclsh /home/share/Modules/4.1.3/libexec/modulecmd.tcl bash "$@"`; fi; _mlstatus=$?; if [ -n "${_mlIFS+x}" ]; then IFS=$_mlIFS; else unset IFS; fi; if [ -n "${_mlshdbg:-}" ]; then set -$_mlshdbg; fi; unset _mlre _mlv _mlrv _mlIFS _mlshdbg; return $_mlstatus } BASH_FUNC_module()=() { unset _mlre _mlIFS _mlshdbg; if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then case "$-" in *v*x*) set +vx; _mlshdbg='vx' ;; *v*) set +v; _mlshdbg='v' ;; *x*) set +x; _mlshdbg='x' ;; *) _mlshdbg='' ;; esac; fi; if [ -n "${IFS+x}" ]; then _mlIFS=$IFS; fi; IFS=' '; for _mlv in ${MODULES_RUN_QUARANTINE:-}; do if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' "; fi; _mlrv="MODULES_RUNENV_${_mlv}"; _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' "; fi; done; if [ -n "${_mlre:-}" ]; then eval `eval ${_mlre}/usr/bin/tclsh /home/share/Modules/4.1.3/libexec/modulecmd.tcl bash '"$@"'`; else eval `/usr/bin/tclsh /home/share/Modules/4.1.3/libexec/modulecmd.tcl bash "$@"`; fi; _mlstatus=$?; if [ -n "${_mlIFS+x}" ]; then IFS=$_mlIFS; else unset IFS; fi; if [ -n "${_mlshdbg:-}" ]; then set -$_mlshdbg; fi; unset _mlre _mlv _mlrv _mlIFS _mlshdbg; return $_mlstatus } BASH_FUNC_switchml%%=() { typeset swfound=1; if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then typeset swname='main'; if [ -e /home/share/Modules/4.1.3/libexec/modulecmd.tcl ]; then typeset swfound=0; unset MODULES_USE_COMPAT_VERSION; fi; else typeset swname='compatibility'; if [ -e /home/share/Modules/4.1.3/libexec/modulecmd-compat ]; then typeset swfound=0; MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION; fi; fi; if [ $swfound -eq 0 ]; then echo "Switching to Modules $swname version"; source /home/share/Modules/4.1.3/init/bash; else echo "Cannot switch to Modules $swname version, command not found"; return 1; fi } BASH_FUNC_switchml()=() { typeset swfound=1; if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then typeset swname='main'; if [ -e /home/share/Modules/4.1.3/libexec/modulecmd.tcl ]; then typeset swfound=0; unset MODULES_USE_COMPAT_VERSION; fi; else typeset swname='compatibility'; if [ -e /home/share/Modules/4.1.3/libexec/modulecmd-compat ]; then typeset swfound=0; MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION; fi; fi; if [ $swfound -eq 0 ]; then echo "Switching to Modules $swname version"; source /home/share/Modules/4.1.3/init/bash; else echo "Cannot switch to Modules $swname version, command not found"; return 1; fi } BASH_FUNC_which%%=() { ( alias; eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot "$@" } BATCH_SYSTEM=HTCondor BLAHPD_LOCATION=/home/osg/bosco/glite CMAKE_PREFIX_PATH=/home/share/slurm/current CMAKE_PREFIX_PATH_modshare=/home/share/slurm/current:1 CONDORCE_COLLECTOR_HOST=fiu-ircc-ce1.svc.osg-htc.org:9619 CONDOR_CONFIG=/scratch/osg/glide_mTQIp8/condor_config CONDOR_PARENT_ID=n110:12549:1785588772 CONDOR_PROCD_ADDRESS=/scratch/osg/glide_mTQIp8/log/procd_address CONDOR_PROCD_ADDRESS_BASE=/scratch/osg/glide_mTQIp8/log/procd_address CPATH=/home/share/slurm/current/include CPATH_modshare=/home/share/slurm/current/include:1 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 ENV=/home/share/Modules/4.1.3/init/profile.sh ENVIRONMENT=BATCH GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osg/glide_mTQIp8/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_FIU-IRCC.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6061624 GLIDEIN_DEBUG_OPTIONS= GLIDEIN_Entry_Name=OSG_US_FIU-IRCC GLIDEIN_Entry_Signature=cd7c7145535427968bd558a170c08a4df4f94a9a GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=fiu-ircc-ce1.svc.osg-htc.org fiu-ircc-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_LK4lDm GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=FIU-IRCC-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=FIU-IRCC GLIDEIN_Tmp_Dir=/scratch/osg/glide_mTQIp8/tmp GLIDEIN_UUID=909e16fd-d06f-4e6c-aa41-2c9afd5d6d99 GLITE_LOCATION=/home/osg/bosco/glite GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GSI_AUTHZ_CONF=/dev/null GWMS_PATH=/scratch/osg/glide_mTQIp8/.gwms.d/bin: GWMS_SINGULARITY_STATUS=PREFERRED HAS_SINGULARITY=False HAS_UNPRIVILEGED_USER_NAMESPACES=unavailable HISTCONTROL=ignoredups HISTSIZE=1000 HOLD_GRACE_TIME=0 HOME=/scratch/osg/glide_mTQIp8/execute/dir_12549 HOSTNAME=n110 JOB_TOKENS=/var/lib/gwms-factory/server-credentials/entry_OSG_US_FIU-IRCC/tokens.tgz JULIA_NUM_THREADS=1 LANG=C LD_LIBRARY_PATH=/home/share/slurm/current/lib LD_LIBRARY_PATH_modshare=/home/share/slurm/current/lib:1 LESSOPEN=||/usr/bin/lesspipe.sh %s LIBRARY_PATH=/home/share/slurm/current/lib LIBRARY_PATH_modshare=/home/share/slurm/current/lib:1 LOADEDMODULES=slurm LOADEDMODULES_modshare=slurm:1 LOGNAME=osg MAIL=/var/spool/mail/osg MANPATH=/home/share/Modules/4.1.3/share/man MKL_NUM_THREADS=1 MODULEPATH=/home/share/Modules/4.1.3/modulefiles/linux-centos7-x86_64 MODULEPATH_modshare=/home/share/Modules/4.1.3/modulefiles/linux-centos7-x86_64:1 MODULESHOME=/home/share/Modules/4.1.3 MODULES_CMD=/home/share/Modules/4.1.3/libexec/modulecmd.tcl NODE_COUNT=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_APP=UNAVAILABLE OSG_DATA=UNAVAILABLE OSG_GRID=/cvmfs/oasis.opensciencegrid.org/osg-software/osg-wn-client/current/el7-x86_64/ OSG_HOSTNAME=fiu-ircc-ce1.svc.osg-htc.org OSG_SITE_NAME=FIU-IRCC-CE1 OSG_WN_TMP=/scratch/osg/glide_mTQIp8/execute/dir_12549/.local-tmp PATH=/scratch/osg/glide_mTQIp8/.gwms.d/bin::/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/share/slurm/current/bin:/home/share/Modules/4.1.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/osg/bin PATH_modshare=/usr/sbin:1:/home/share/Modules/4.1.3/bin:1:/usr/bin:1:/home/share/slurm/current/bin:1:/usr/local/sbin:1:/usr/local/bin:1 PREEMPT_GRACE_TIME=10000000 PWD=/scratch/osg/glide_mTQIp8/execute/dir_12549 PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SHELL=/bin/bash SHLVL=6 SINGULARITY_CACHEDIR=/scratch/osg/glide_mTQIp8/execute/dir_12549 SLOTS_LAYOUT=fixed SLURMD_NODENAME=n110 SLURM_CLUSTER_NAME=fiuhpcslurm SLURM_CONF=/home/share/slurm/22.05.6/etc/slurm.conf SLURM_CPUS_ON_NODE=2 SLURM_GTIDS=0 SLURM_JOBID=3574044 SLURM_JOB_ACCOUNT=osg SLURM_JOB_CPUS_PER_NODE=2 SLURM_JOB_GID=109 SLURM_JOB_ID=3574044 SLURM_JOB_NAME=bl_AFrYGt SLURM_JOB_NODELIST=n110 SLURM_JOB_NUM_NODES=1 SLURM_JOB_PARTITION=osg SLURM_JOB_QOS=osg SLURM_JOB_UID=84159 SLURM_JOB_USER=osg SLURM_LOCALID=0 SLURM_MEM_PER_NODE=8192 SLURM_NNODES=1 SLURM_NODEID=0 SLURM_NODELIST=n110 SLURM_NODE_ALIASES=(null) SLURM_NPROCS=1 SLURM_NTASKS=1 SLURM_NTASKS_PER_NODE=1 SLURM_PRIO_PROCESS=0 SLURM_PROCID=0 SLURM_SUBMIT_DIR=/home/osg/bosco/sandbox/a8f4/a8f49119/fiu-ircc-ce1.svc.osg-htc.org_9619_fiu-ircc-ce1.svc.osg-htc.org_51494.0_1785588049 SLURM_SUBMIT_HOST=hpcosgce1.panther.net SLURM_TASKS_PER_NODE=1 SLURM_TASK_PID=977 SLURM_TOPOLOGY_ADDR=n110 SLURM_TOPOLOGY_ADDR_PATTERN=node SLURM_WORKING_CLUSTER=fiuhpcslurm:ms1:6817:9728:101 SSH_CLIENT=128.105.82.94 37220 22 SSH_CONNECTION=128.105.82.94 37220 131.94.55.228 22 S_COLORS=auto TEMP=/scratch/osg/glide_mTQIp8/execute/dir_12549 TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/scratch/osg/glide_mTQIp8/execute/dir_12549 TMPDIR=/scratch/osg/glide_mTQIp8/execute/dir_12549/.local-tmp TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM TZ=:/etc/localtime USER=osg _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_ANCESTOR_12549=17240:1785588773:30247606 _CONDOR_ANCESTOR_17690=17775:1785588405:951133400 _CONDOR_ANCESTOR_17775=12549:1785588764:2139749583 _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osg/glide_mTQIp8/main/condor/bin _CONDOR_CHIRP_CONFIG=/scratch/osg/glide_mTQIp8/execute/dir_12549/.chirp.config _CONDOR_CREDS=/scratch/osg/glide_mTQIp8/execute/dir_12549/.condor_creds _CONDOR_EXECUTE=/scratch/osg/glide_mTQIp8/execute _CONDOR_JOB_AD=/scratch/osg/glide_mTQIp8/execute/dir_12549/.job.ad _CONDOR_JOB_IWD=/scratch/osg/glide_mTQIp8/execute/dir_12549 _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/scratch/osg/glide_mTQIp8/execute/dir_12549/.machine.ad _CONDOR_SCRATCH_DIR=/scratch/osg/glide_mTQIp8/execute/dir_12549 _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/scratch/osg/glide_mTQIp8/execute/dir_12549/.job_wrapper_failure _LMFILES_=/home/share/Modules/4.1.3/modulefiles/linux-centos7-x86_64/slurm _LMFILES__modshare=/home/share/Modules/4.1.3/modulefiles/linux-centos7-x86_64/slurm:1 which_declare=declare -f
pass df
38761.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on 192.168.232.21@tcp:192.168.232.22@tcp:/scratch 50398964480 9243378376 40644555584 19% /scratch
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: hpcosgce1.panther.net 0.154ms 1: hpcosgce1.panther.net 0.158ms 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 84159)
FSU-HNPGRID 4 hours ago 126
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
FSU-HNPGRID — probe output
pass kernel
5.14.0-687.29.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_eqQhEA/images/htc__ubuntu__24.04.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__ubuntu__24.04.sif APPTAINER_TMPDIR=/scratch/glide_eqQhEA/execute/dir_984998/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=984998 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_eqQhEA/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.GLUEX_US_FSU-HNPGRID.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6217453 GLIDEIN_Entry_Name=GLUEX_US_FSU-HNPGRID GLIDEIN_Entry_Signature=e814e8c8b983255b17157c0ff4185ca0fd7a16e0 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=fsu-hnpgrid-ce1.svc.osg-htc.org fsu-hnpgrid-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_njr4KH GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=8 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=FSU-HNPGRID GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=OSG_US_FSU_HNPGRID GLIDEIN_Tmp_Dir=/scratch/glide_eqQhEA/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=fsu-hnpgrid-ce1.svc.osg-htc.org OSG_SITE_NAME=FSU-HNPGRID PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_4@glidein_445005_167096586@scigrid6.physics.fsu.edu$ "; unset PROMPT_COMMAND PS1=slot1_4@glidein_445005_167096586@scigrid6.physics.fsu.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_eqQhEA/images/htc__ubuntu__24.04.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/glide_eqQhEA/images/htc__ubuntu__24.04.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__ubuntu__24.04.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_eqQhEA/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_eqQhEA/execute/dir_984998/scratch _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
2180.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 3900228936 1613844468 2286384468 42% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: bvi100411551.dsl-pe.coas-physics.net.fsu.edu 0.620ms 1: bvi100411551.dsl-pe.coas-physics.net.fsu.edu 1.077ms 2: no reply 3: no reply 4: no reply 5: illegally-used-at.fsu 1.375ms 6: 128.186.253.197 1.768ms 7: ae22.1901.rsb-br.net.fsu.edu 2.383ms 8: tlh-flrcore-asr9010-1-te0203-1902.net.flrnet.org 5.419ms 9: tlh-bb-rtr-2-fh0001-1.rtr.net.flrnet.org 6.897ms asymm 11 10: jax-bb-rtr-1-fh0001-1.rtr.net.flrnet.org 8.374ms 11: prov-i2-atla-renet-1070.net.flrnet.org 8.312ms 12: fourhundredge-0-0-0-6.4079.core1.atla.net.internet2.edu 30.319ms asymm 24 13: fourhundredge-0-0-0-23.4079.core2.atla.net.internet2.edu 30.536ms asymm 23 14: fourhundredge-0-0-0-4.4079.core2.nash.net.internet2.edu 30.061ms asymm 22 15: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 30.808ms asymm 21 16: fourhundredge-0-0-0-21.4079.core2.cinc.net.internet2.edu 30.296ms asymm 20 17: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 31.121ms asymm 19 18: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 31.458ms 19: 144.92.233.112 32.928ms 20: perfsonar02.hep.wisc.edu 33.803ms reached Resume: pmtu 1500 hops 20 back 21
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 9004)
GSU-ACIDS 4 hours ago 10
40.0%
40.0%
0.0%
60.0%
kernel env df firewall mtu nrp_images http_proxy user
GSU-ACIDS — probe output
pass kernel
6.8.0-111-generic
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/users/svc-osgaccnt/glide_q9pnIx/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/users/svc-osgaccnt/glide_q9pnIx/execute/dir_541945/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=541945 CONTAINERSW_FULL_VERSION=apptainer version 1.4.1 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/sysapps/ubuntu-applications/apptainer/1.4.1/install/bin/apptainer CONTAINERSW_VERSION=1.4.1 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/users/svc-osgaccnt/glide_q9pnIx/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_GSU-ACIDS.idtoken.idtoken GLIDEIN_CPUS=64 GLIDEIN_ClusterId=6210826 GLIDEIN_Entry_Name=OSG_US_GSU-ACIDS GLIDEIN_Entry_Signature=f437dfef487f3640d198bd64e34c495282a9e263 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=gsu-acids-ce1.svc.opensciencegrid.org gsu-acids-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_svc-osgaccnt_uOlIW6 GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=arcosg04.rs.gsu.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=GSU-ACIDS GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=51c806521342f21128eec483be00d289d526592b GLIDEIN_Site=GSU-ACIDS GLIDEIN_Tmp_Dir=/scratch/users/svc-osgaccnt/glide_q9pnIx/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.1 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/sysapps/ubuntu-applications/apptainer/1.4.1/install/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.1 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/sysapps/ubuntu-applications/apptainer/1.4.1/install/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.1 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=gsu-acids-ce1.svc.opensciencegrid.org OSG_SITE_NAME=GSU-ACIDS OSG_SQUID_LOCATION=arcosg04.rs.gsu.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_11@glidein_3307971_76657785@acidsgn007.rs.gsu.edu$ "; unset PROMPT_COMMAND PS1=slot1_11@glidein_3307971_76657785@acidsgn007.rs.gsu.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/users/svc-osgaccnt/glide_q9pnIx/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.1 SINGULARITY_IMAGES_DICT=default:/scratch/users/svc-osgaccnt/glide_q9pnIx/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/sysapps/ubuntu-applications/apptainer/1.4.1/install/bin/apptainer SINGULARITY_VERSION=1.4.1 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/users/svc-osgaccnt/glide_q9pnIx/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/users/svc-osgaccnt/glide_q9pnIx/execute/dir_541945/scratch _CONDOR_SLOT=slot1_11 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
184897.9 GB available Filesystem 1024-blocks Used Available Capacity Mounted on beegfs_nodev 270018342912 76138836480 193879506432 29% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.245.10.2 0.541ms 1: 10.245.10.3 0.601ms 2: rtcdfdrs-rtlsdc02rs-gw.gsu.edu 0.466ms 3: fwcdfxrs-rtcdfars-rtcdfdrs-gw.gsu.edu 0.401ms 4: fwcdfrs-rtcdfdrs.gsunet 0.890ms 5: rtcdfdedge-rtcdfdrs-gw.gsu.edu 0.748ms 6: SOX-10G-Secondary-TO-GSU.gsu.edu 4.227ms asymm 7 7: 365dc-to-56mar-re.sox.net 7.853ms asymm 8 8: i2-to-sox-re.sidco.sox.net 10.336ms asymm 9 9: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 20.554ms asymm 16 10: fourhundredge-0-0-0-22.4079.core2.cinc.net.internet2.edu 20.203ms asymm 15 11: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 20.030ms asymm 14 12: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 21.559ms asymm 13 13: 144.92.233.112 21.833ms asymm 14 14: perfsonar02.hep.wisc.edu 21.792ms reached Resume: pmtu 1500 hops 14 back 16
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy arcosg04.rs.gsu.edu:3128
pass user
svc-osgaccnt (uid 330104)
GSU-Adonis-EP 4 hours ago 1
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
GSU-Adonis-EP — probe output
pass kernel
6.8.0-136-lowlatency
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-NXOXSC/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-NXOXSC/execute/dir_599503 BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=599503 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=GSU-Adonis OSG_SQUID_LOCATION=http://arcosg04.rs.gsu.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_5@GSU-Adonis-EP.osg-endpoint-74b7c7597f-nwk2d$ "; unset PROMPT_COMMAND PS1=slot1_5@GSU-Adonis-EP.osg-endpoint-74b7c7597f-nwk2d$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv SINGULARITY_CONTAINER=/pilot/osgvo-pilot-NXOXSC/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-NXOXSC/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el8 TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-NXOXSC/execute/dir_599503 _CONDOR_SLOT=slot1_5 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure http_proxy=http://arcosg04.rs.gsu.edu:3128
pass df
35.9 GB available Filesystem 1024-blocks Used Available Capacity Mounted on tmpfs 67108864 29513508 37595356 44% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1430 1?: [LOCALHOST] pmtu 1430 1: 10-245-19-162.prometheus-prometheus-node-exporter.monitoring.svc.cluster.local 0.051ms 1: 10-245-19-162.prometheus-prometheus-node-exporter.monitoring.svc.cluster.local 0.046ms 2: 10.245.18.3 0.342ms 3: rtcdfars-rtlsdc01rs-gw.gsu.edu 0.489ms 4: fwcdfxrs-rtcdfars-rtcdfdrs-gw.gsu.edu 0.126ms 5: fwcdfrs-rtcdfdrs.gsunet 0.745ms 6: rtcdfaedge-rtcdfars-gw.gsu.edu 0.475ms 7: SOX-10G-Primary-TO-GSU.gsu.edu 3.283ms 8: 365dc-to-56mar-re.sox.net 7.883ms asymm 9 9: i2-to-sox-re.sidco.sox.net 7.774ms asymm 10 10: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 20.278ms asymm 17 11: fourhundredge-0-0-0-22.4079.core2.cinc.net.internet2.edu 22.057ms asymm 16 12: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 19.023ms asymm 15 13: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 19.232ms asymm 14 14: 144.92.233.112 21.672ms asymm 15 15: perfsonar02.hep.wisc.edu 21.627ms reached Resume: pmtu 1430 hops 15 back 17
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy http://arcosg04.rs.gsu.edu:3128
pass user
osg (uid 1000)
GWU-Viper-CE1 4 hours ago 12
100.0%
91.7%
0.0%
91.7%
kernel env df firewall mtu nrp_images http_proxy user
GWU-Viper-CE1 — probe output
pass kernel
5.14.0-570.52.1.el9_6.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/osg/dir_2033965/glide_MEr7yL/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/osg/dir_2033965/glide_MEr7yL/execute/dir_1851981/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=1851981 CONTAINERSW_FULL_VERSION=apptainer version 1.4.3-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.3-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osg/dir_2033965/glide_MEr7yL/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_GWU-Viper-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6704225 GLIDEIN_Entry_Name=OSG_US_GWU-Viper-CE1 GLIDEIN_Entry_Signature=e6878f94a4b8dc74613fb84a8f0961e480c20bc0 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=gwu-viper-ce1.svc.osg-htc.org gwu-viper-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_iCCoSF GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=GWU-Viper-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=abb0f8c777e8d5f9bcf1379ef7c8ea262d98ef55 GLIDEIN_Site=GWU-Viper GLIDEIN_Tmp_Dir=/scratch/osg/dir_2033965/glide_MEr7yL/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.3-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.3-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.3-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=gwu-viper-ce1.svc.osg-htc.org OSG_SITE_NAME=GWU-Viper-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_3@glidein_2033968_493120813@node001$ "; unset PROMPT_COMMAND PS1=slot1_3@glidein_2033968_493120813@node001$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/osg/dir_2033965/glide_MEr7yL/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.3-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/osg/dir_2033965/glide_MEr7yL/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.3-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osg/dir_2033965/glide_MEr7yL/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osg/dir_2033965/glide_MEr7yL/execute/dir_1851981/scratch _CONDOR_SLOT=slot1_3 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
818.4 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 960302824 53299656 858148708 6% /srv
fail firewall
Firewall blocked connections to port(s): [2880] http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: FAILED (timed out) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.005ms pmtu 1500 1: 2620:118:500f:2b8a::1 3.644ms 1: 2620:118:500f:2b8a::1 2.937ms 2: no reply 3: no reply 4: no reply 5: fourhundredge-0-0-0-10.4079.core1.star.net.internet2.edu 17.454ms asymm 6 6: 2607:f388:0:2204:: 18.773ms asymm 7 7: perfsonar02.hep.wisc.edu 18.706ms reached Resume: pmtu 1500 hops 7 back 9
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1002)
Hawaii-Koa-CE1 4 hours ago 69
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Hawaii-Koa-CE1 — probe output
pass kernel
5.14.0-687.17.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/mnt/nfs/scratch/osg01/glide_HyupT2/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/mnt/nfs/scratch/osg01/glide_HyupT2/execute/dir_3166253/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3166253 CONTAINERSW_FULL_VERSION=apptainer version 1.5.1-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.1-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/mnt/nfs/scratch/osg01/glide_HyupT2/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Hawaii-Koa-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=697909 GLIDEIN_Entry_Name=OSG_US_Hawaii-Koa-CE1 GLIDEIN_Entry_Signature=7f924b841339f8e5c07b57deecb7892f5659b602 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=hawaii-koa-ce1.svc.osg-htc.org hawaii-koa-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_kkiMaN GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=3 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Hawaii-Koa-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=Hawaii-Koa GLIDEIN_Tmp_Dir=/mnt/nfs/scratch/osg01/glide_HyupT2/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.1-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.1-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.1-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=hawaii-koa-ce1.svc.osg-htc.org OSG_SITE_NAME=Hawaii-Koa-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_397794_101334420@cn-03-23-05.mana.ci.its.hawaii.edu$ "; unset PROMPT_COMMAND PS1=slot1@glidein_397794_101334420@cn-03-23-05.mana.ci.its.hawaii.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/mnt/nfs/scratch/osg01/glide_HyupT2/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.1-1.el9 SINGULARITY_IMAGES_DICT=default:/mnt/nfs/scratch/osg01/glide_HyupT2/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.1-1.el9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/mnt/nfs/scratch/osg01/glide_HyupT2/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/mnt/nfs/scratch/osg01/glide_HyupT2/execute/dir_3166253/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
9615.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on 10.0.0.99:/datastore/scratch/osg01/glide_HyupT2/execute/dir_3166253 10585137152 503129088 10082008064 5% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: _gateway 0.239ms 1: _gateway 0.267ms 2: et-1-2-0-2500-scidmz-re0.uhnet.net 0.342ms 3: et-0-2-0-35-ohelo-re0.uhnet.net 0.395ms 4: internet2-uhm-losa.uhnet.net 52.551ms 5: fourhundredge-0-0-0-2.4079.core2.salt.net.internet2.edu 97.657ms asymm 13 6: fourhundredge-0-0-0-0.4079.core2.denv.net.internet2.edu 97.335ms asymm 12 7: fourhundredge-0-0-0-0.4079.core2.kans.net.internet2.edu 95.551ms asymm 11 8: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 97.319ms asymm 10 9: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 96.495ms 10: 144.92.233.112 99.074ms 11: perfsonar02.hep.wisc.edu 98.330ms reached Resume: pmtu 9000 hops 11 back 12
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 40002)
IRISHEP-SSL-RIVER-DEV-BACKFILL 4 hours ago 64
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
IRISHEP-SSL-RIVER-DEV-BACKFILL — probe output
pass kernel
5.14.0-687.26.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-UGOIv3/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-UGOIv3/execute/dir_587534/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=587534 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=IRISHEP-SSL-UCHICAGO OSG_SQUID_LOCATION=squid.grid.uchicago.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_19@IRISHEP-SSL-RIVER-DEV-BACKFILL.river-c044.ssl-hep.org$ "; unset PROMPT_COMMAND PS1=slot1_19@IRISHEP-SSL-RIVER-DEV-BACKFILL.river-c044.ssl-hep.org$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-UGOIv3/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-UGOIv3/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-UGOIv3/execute/dir_587534/scratch _CONDOR_SLOT=slot1_19 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=squid.grid.uchicago.edu:3128
pass df
285.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 765870024 427526888 299365652 59% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] 0.011ms pmtu 1500 1: 2605:9a00:10:200c::2 0.247ms 1: 2605:9a00:10:200c::2 0.311ms 2: hinds-a-to-hinds-b.sci.uchicago.edu 0.203ms asymm 1 3: hinds-b-to-sb.sci.uchicago.edu 0.383ms asymm 2 4: northsci-to-southsci.sci.uchicago.edu 0.328ms asymm 2 5: site--uwmadison-se306.lhcone.es.net 4.854ms 6: star-cr6--uwmadison-se306.lhcone.es.net 0.903ms asymm 4 7: site--uwmadison-se306.lhcone.es.net 4.854ms asymm 5 8: perfsonar02.hep.wisc.edu 4.834ms reached Resume: pmtu 1500 hops 8 back 7
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy squid.grid.uchicago.edu:3128
pass user
nobody (uid 65534)
IU-Jetstream2-Backfill 4 hours ago 49
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
IU-Jetstream2-Backfill — probe output
pass kernel
6.12.85+deb13-cloud-amd64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/alphafold3 APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-6VAhzv/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-6VAhzv/execute/dir_7330/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=7330 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=Pervasive Technology Institute PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_1@IU-Jetstream2-Backfill.blue-68b8958644-2jgwd$ "; unset PROMPT_COMMAND PS1=slot1_1@IU-Jetstream2-Backfill.blue-68b8958644-2jgwd$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/alphafold3 SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-6VAhzv/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-6VAhzv/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-6VAhzv/execute/dir_7330/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
177.4 GB available Filesystem 1024-blocks Used Available Capacity Mounted on overlay 206376368 11896588 186054156 7% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=8850 1?: [LOCALHOST] pmtu 8850 1: _gateway 0.072ms 1: _gateway 0.035ms 2: host-192-168-0-1.js2local 0.987ms 3: host-192-168-3-22.js2local 0.871ms asymm 2 4: 149.165.159.1 0.357ms asymm 3 5: 172.16.11.140 0.512ms asymm 4 6: e-12-1.leafc1.bldc.net.rt.iu.edu 0.663ms asymm 5 7: 10.9.213.9 2.558ms asymm 6 8: et-4-2-3.2012.rtr.ictc.indiana.gigapop.net 2.431ms asymm 7 9: ae-9.0.rtr.ll.indiana.gigapop.net 2.823ms asymm 8 10: ae-7.1.rtr2.chic.indiana.gigapop.net 5.867ms asymm 9 11: ae-7.1.rtr2.chic.indiana.gigapop.net 5.507ms asymm 9 12: 144.92.233.118 23.062ms asymm 10 13: perfsonar02.hep.wisc.edu 22.915ms reached Resume: pmtu 8850 hops 13 back 11
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 1000)
JSU-Tigers-CE1 7 hours ago 1
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
JSU-Tigers-CE1 — probe output
pass kernel
5.14.0-701.el9.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/execute/dir_3853894/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3853894 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.3-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_JSU-Tigers-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6507558 GLIDEIN_Entry_Name=OSG_US_JSU-Tigers-CE1 GLIDEIN_Entry_Signature=6d7d83c08bb7f70fc9858beb368cc150b6a912e8 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=jsu-tigers-ce1.svc.opensciencegrid.org jsu-tigers-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_Ls4qaI GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=JSU-Tigers-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=JSU-Tigers GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=jsu-tigers-ce1.svc.opensciencegrid.org OSG_SITE_NAME=JSU-Tigers-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_3697413_227982062@chtc.ep0.jsums.edu$ "; unset PROMPT_COMMAND PS1=slot1@glidein_3697413_227982062@chtc.ep0.jsums.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-1.el9 SINGULARITY_IMAGES_DICT=default:/var/lib/condor/execute/dir_3697411/glide_lxEuHT/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.3-1.el9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_3697411/glide_lxEuHT/execute/dir_3853894/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
1675.9 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/condor_vg-condor_lv 1844816532 1697136 1757351596 1% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.340ms 1: _gateway 0.327ms 2: 173.235.100.112 1.632ms 3: 173.235.100.7 9.336ms asymm 2 4: 173.235.100.17 19.034ms asymm 3 5: fourhundredge-0-0-0-23.4079.core2.atla.net.internet2.edu 35.389ms asymm 14 6: fourhundredge-0-0-0-4.4079.core2.nash.net.internet2.edu 36.366ms asymm 13 7: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 35.916ms asymm 12 8: fourhundredge-0-0-0-22.4079.core2.cinc.net.internet2.edu 35.583ms asymm 11 9: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 65.148ms asymm 10 10: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 38.762ms asymm 9 11: 144.92.233.112 34.755ms asymm 10 12: perfsonar02.hep.wisc.edu 34.577ms reached Resume: pmtu 1500 hops 12 back 12
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 4977)
KENT-STATE-RESEARCH-EP 4 hours ago 4
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
KENT-STATE-RESEARCH-EP — probe output
pass kernel
5.14.0-729.el9.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-pZsJJA/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-pZsJJA/execute/dir_71340/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=71340 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=Kent_State_Research PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_52@KENT-STATE-RESEARCH-EP.6a939cfae9dc$ "; unset PROMPT_COMMAND PS1=slot1_52@KENT-STATE-RESEARCH-EP.6a939cfae9dc$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-pZsJJA/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-pZsJJA/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-pZsJJA/execute/dir_71340/scratch _CONDOR_SLOT=slot1_52 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
41300.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg--main-home 45986104216 370494296 43307048616 1% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.339ms 1: _gateway 0.302ms 2: gw2.research.cs.kent.edu 1.020ms 3: 10.255.255.1 0.905ms 4: akrnq-r4-et-0-2-1s3112.core.oar.net 1.050ms 5: yntww-r5-et-1-0-0s100.core.oar.net 2.539ms 6: clevb-r5-et-5-1-0s100.core.oar.net 4.376ms 7: clevb-r7-et-0-0-4s100.core.oar.net 4.440ms 8: clmbn-r4-et-2-1-2s100.core.oar.net 8.029ms 9: cncne-r4-et-0-0-4s100.core.oar.net 10.294ms 10: cncno-r5-et-4-3-0s735.core.oar.net 10.548ms 11: hundredge-0-0-0-25.1260.core1.cinc.net.internet2.edu 11.753ms asymm 10 12: fourhundredge-0-0-0-21.4079.core2.cinc.net.internet2.edu 15.729ms 13: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 16.071ms asymm 11 14: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 15.988ms asymm 10 15: 144.92.233.112 17.199ms asymm 11 16: perfsonar02.hep.wisc.edu 17.218ms reached Resume: pmtu 1500 hops 16 back 13
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg (uid 1000)
KSU-Beocat-CE1 4 hours ago 400
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
KSU-Beocat-CE1 — probe output
pass kernel
5.14.0-611.54.1.el9_7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_lCnT1G/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_lCnT1G/execute/dir_2704455/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2704455 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.3-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_lCnT1G/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_KSU-Beocat-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=697942 GLIDEIN_Entry_Name=OSG_US_KSU-Beocat-CE1 GLIDEIN_Entry_Signature=01a0c18993aff14c739c70de4b9dc0fd8eaa7f22 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=ksu-beocat-ce1.svc.opensciencegrid.org ksu-beocat-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osgsubmit_lgp8nf GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=5 GLIDEIN_Proxy_URL=squid.beocat.ksu.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=KSU-Beocat-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=BEOCAT-SLATE GLIDEIN_Tmp_Dir=/tmp/glide_lCnT1G/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ksu-beocat-ce1.svc.opensciencegrid.org OSG_SITE_NAME=KSU-Beocat-CE1 OSG_SQUID_LOCATION=squid.beocat.ksu.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_3@glidein_4162374_397433120@warlock41.beocat.ksu.edu$ "; unset PROMPT_COMMAND PS1=slot1_3@glidein_4162374_397433120@warlock41.beocat.ksu.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_lCnT1G/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-1.el9 SINGULARITY_IMAGES_DICT=default:/tmp/glide_lCnT1G/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.3-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_lCnT1G/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_lCnT1G/execute/dir_2704455/scratch _CONDOR_SLOT=slot1_3 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
672.4 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/rlm_warlock41-root 931372720 226268912 705103808 25% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: no reply 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 9000 Resume: pmtu 9000
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy squid.beocat.ksu.edu:3128
pass user
osgsubmit (uid 3838)
LIGO-WA-CE1 4 hours ago 19
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
LIGO-WA-CE1 — probe output
pass kernel
4.18.0-553.137.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/execute/dir_3536614/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3536614 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a25bfcc67a94e12a2a653601c5374a48a93b86c4 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=790840aafc78761648fb614a0f2c0c7cf0c4b4e6 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.LIGO_US_Caltech_ldas_grid-wa_6C.idtoken.idtoken GLIDEIN_CPUS=6 GLIDEIN_ClusterId=6682659 GLIDEIN_Entry_Name=LIGO_US_Caltech_ldas_grid-wa_6C GLIDEIN_Entry_Signature=d5b36aafad2293c310b3721c03dff5beb58b80a9 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=ligo-hanford-ce1.svc.osg-htc.org ligo-hanford-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_XBqbgh GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=5 GLIDEIN_Proxy_URL=frontier.dcs.ligo-wa.caltech.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=LIGO-WA-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=4065df84057e3fa67e8d166e5370d05f03e77e54 GLIDEIN_Site=LIGO-WA GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ligo-hanford-ce1.svc.osg-htc.org OSG_SITE_NAME=LIGO-WA-CE1 OSG_SQUID_LOCATION=frontier.dcs.ligo-wa.caltech.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_2@glidein_2880573_179437492@node050.dcs.ligo-wa.caltech.edu$ "; unset PROMPT_COMMAND PS1=slot1_2@glidein_2880573_179437492@node050.dcs.ligo-wa.caltech.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_2880518/scratch/glide_HER8RX/execute/dir_3536614/scratch _CONDOR_SLOT=slot1_2 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
65.2 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/node050-slot1_1+2620437_5547 72195192 3820736 68358072 6% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: pluto.dcs.ligo-wa.caltech.edu 0.207ms 1: pluto.dcs.ligo-wa.caltech.edu 0.200ms 2: 198.129.208.1 0.341ms asymm 3 3: no reply 4: bois-cr6--salt-bb-c.igp.es.net 43.731ms asymm 15 5: salt-cr6--denv-bb-b.igp.es.net 43.711ms asymm 14 6: denv-cr6--kans-bb-e.igp.es.net 43.681ms asymm 13 7: kans-cr6--chic-bb-d.igp.es.net 43.644ms asymm 12 8: chic-cr6--star-bb-e.igp.es.net 43.690ms asymm 11 9: no reply 10: site--uwmadison-se305.ip.es.net 47.536ms asymm 11 11: perfsonar02.hep.wisc.edu 54.107ms reached Resume: pmtu 9000 hops 11 back 11
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy frontier.dcs.ligo-wa.caltech.edu:3128
pass user
osg01 (uid 9991)
LSUHSC-Tigerfish-CE2 4 hours ago 3
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
LSUHSC-Tigerfish-CE2 — probe output
pass kernel
4.18.0-240.15.1.el8_3.x86_64
pass env
APPTAINER_CACHEDIR=/srv/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2661088 CONTAINERSW_FULL_VERSION=singularity version 3.7.4-1.el8 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/singularity CONTAINERSW_VERSION=3.7.4-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/mnt/beegfs/scratch/glide_7OkMy4/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_LSUHSC-Tigerfish-CE2.idtoken.idtoken GLIDEIN_CPUS=40 GLIDEIN_ClusterId=6216758 GLIDEIN_Entry_Name=OSG_US_LSUHSC-Tigerfish-CE2 GLIDEIN_Entry_Signature=2de3309c066653b2d70c87b074a7dc8393591650 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=lsuhsc-tf-ce2.svc.opensciencegrid.org lsuhsc-tf-ce2.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_CCF6Fw GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=LSUHSC-Tigerfish-CE2 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=LSUHSC-Tigerfish GLIDEIN_Tmp_Dir=/mnt/beegfs/scratch/glide_7OkMy4/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=singularity version 3.7.4-1.el8 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/singularity GWMS_CONTAINERSW_VERSION=3.7.4-1.el8 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=3.7.4-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=lsuhsc-tf-ce2.svc.opensciencegrid.org OSG_SITE_NAME=LSUHSC-Tigerfish-CE2 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_1@glidein_2253483_70575216@node13.cluster$ "; unset PROMPT_COMMAND PS1=slot1_1@glidein_2253483_70575216@node13.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/mnt/beegfs/scratch/glide_7OkMy4/execute/dir_2661088:/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_COMMAND=exec SINGULARITY_CONTAINER=/mnt/beegfs/scratch/glide_7OkMy4/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=singularity version 3.7.4-1.el8 SINGULARITY_IMAGES_DICT=default:/mnt/beegfs/scratch/glide_7OkMy4/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/singularity SINGULARITY_TMPDIR=/mnt/beegfs/scratch/glide_7OkMy4/execute/dir_2661088/scratch SINGULARITY_VERSION=3.7.4-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/mnt/beegfs/scratch/glide_7OkMy4/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/mnt/beegfs/scratch/glide_7OkMy4/execute/dir_2661088/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
350680.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on beegfs_nodev 596209287168 228494623744 367714663424 39% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: tigerfish.cluster 0.134ms 1: tigerfish.cluster 0.219ms 2: 155.58.254.1 0.784ms 3: rcb-bdr-r1.lsuhsc.edu 1.015ms 4: 208.100.102.142 1.233ms 5: 10.250.25.153 7.445ms asymm 13 6: 10.250.24.121 7.431ms asymm 12 7: 10.250.25.69 7.415ms asymm 11 8: 10.250.24.117 7.213ms asymm 10 9: 10.250.27.69 7.488ms 10: 10.250.27.70 7.686ms asymm 9 11: 10.250.27.102 7.416ms asymm 8 12: 208.100.66.98 9.432ms asymm 9 13: fourhundredge-0-0-0-2.4079.core1.houh.net.internet2.edu 36.274ms asymm 20 14: fourhundredge-0-0-0-2.4079.core1.hous.net.internet2.edu 34.107ms asymm 19 15: fourhundredge-0-0-0-0.4079.core2.dall.net.internet2.edu 34.810ms asymm 18 16: fourhundredge-0-0-0-2.4079.core2.kans.net.internet2.edu 36.330ms asymm 17 17: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 34.820ms asymm 16 18: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 36.443ms asymm 15 19: 144.92.233.112 37.000ms asymm 16 20: perfsonar02.hep.wisc.edu 37.102ms reached Resume: pmtu 1500 hops 20 back 18
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 100017)
Lafayette-Firebird-CE1 4 hours ago 26
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Lafayette-Firebird-CE1 — probe output
pass kernel
5.14.0-611.55.1.el9_7.0.3.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/osg01/glide_AfbEwb/images/htc__rocky__8.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__8.sif APPTAINER_TMPDIR=/scratch/osg01/glide_AfbEwb/execute/dir_76305/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=76305 CONTAINERSW_FULL_VERSION=apptainer version 1.5.0-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.0-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osg01/glide_AfbEwb/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Lafayette-firebird-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6218993 GLIDEIN_Entry_Name=OSG_US_Lafayette-firebird-CE1 GLIDEIN_Entry_Signature=a0763be49e1a961a3d2fc61c1982ac5218e47799 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=lafayette-firebird-ce1.svc.osg-htc.org lafayette-firebird-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_GOCo1I GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=4 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Lafayette-Firebird-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Lafayette-Firebird GLIDEIN_Tmp_Dir=/scratch/osg01/glide_AfbEwb/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.0-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.0-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.0-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=lafayette-firebird-ce1.svc.osg-htc.org OSG_SITE_NAME=Lafayette-Firebird-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_1@glidein_4023390_92592240@node01.cluster$ "; unset PROMPT_COMMAND PS1=slot1_1@glidein_4023390_92592240@node01.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/osg01/glide_AfbEwb/images/htc__rocky__8.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.0-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/osg01/glide_AfbEwb/images/htc__rocky__8.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__8.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.0-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osg01/glide_AfbEwb/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osg01/glide_AfbEwb/execute/dir_76305/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
7021.9 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/disk0-scratch 7439005696 75985500 7363020196 2% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: firebird.cluster 3.324ms 1: firebird.cluster 1.942ms 2: fw-comsvc-colo.lafayette.edu 4.457ms 3: 139.147.1.113 5.098ms asymm 4 4: 172.31.0.33 3.674ms 5: 172.31.0.3 4.562ms 6: gw0-g0-3-ibgp.prd.lafayette.edu 4.900ms asymm 5 7: 164.153.140.160 10.070ms asymm 6 8: no reply 9: hundredge-0-0-0-25.68.core1.pitt.net.internet2.edu 18.461ms asymm 8 10: fourhundredge-0-0-0-3.4079.core1.clev.net.internet2.edu 24.277ms asymm 13 11: fourhundredge-0-0-0-2.4079.core1.eqch.net.internet2.edu 29.441ms asymm 12 12: fourhundredge-0-0-0-6.4079.core1.star.net.internet2.edu 24.866ms asymm 11 13: 144.92.233.112 28.365ms asymm 12 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 200079)
Langston-Lion-CE1 3 hours ago 2
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Langston-Lion-CE1 — probe output
pass kernel
5.14.0-570.49.1.el9_6.x86_64
pass env
APPTAINER_CACHEDIR=/srv/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3323918 CONTAINERSW_FULL_VERSION=singularity-ce version 4.3.3-3.el9 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/singularity CONTAINERSW_VERSION=4.3.3-3.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osgccre/glide_tEwme4/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Langston-Lion-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6783182 GLIDEIN_Entry_Name=OSG_US_Langston-Lion-CE1 GLIDEIN_Entry_Signature=cd4e9acc65e1f683a1b67efb2e0f061fd45e737d GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=langston-lion-ce1.svc.osg-htc.org langston-lion-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osgccre_058Mrc GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Langston-Lion-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Langston-Lion GLIDEIN_Tmp_Dir=/scratch/osgccre/glide_tEwme4/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=singularity-ce version 4.3.3-3.el9 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/singularity GWMS_CONTAINERSW_VERSION=4.3.3-3.el9 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=4.3.3-3.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=langston-lion-ce1.svc.osg-htc.org OSG_SITE_NAME=Langston-Lion-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_2227509_586728804@c000.ccre.langston.edu$ "; unset PROMPT_COMMAND PS1=slot1@glidein_2227509_586728804@c000.ccre.langston.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/scratch/osgccre/glide_tEwme4/execute/dir_3323918:/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_COMMAND=exec SINGULARITY_CONTAINER=/scratch/osgccre/glide_tEwme4/images/htc__rocky__8.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=singularity-ce version 4.3.3-3.el9 SINGULARITY_IMAGES_DICT=default:/scratch/osgccre/glide_tEwme4/images/htc__rocky__8.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__8.sif SINGULARITY_PATH=/usr/bin/singularity SINGULARITY_TMPDIR=/scratch/osgccre/glide_tEwme4/execute/dir_3323918/scratch SINGULARITY_VERSION=4.3.3-3.el9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osgccre/glide_tEwme4/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osgccre/glide_tEwme4/execute/dir_3323918/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
885.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 936705024 7982820 928722204 1% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: lionvmhost.ccre.langston.edu 0.142ms 1: lionvmhost.ccre.langston.edu 0.130ms 2: 164.58.29.113 0.529ms 3: 198.71.46.45 2.126ms asymm 4 4: fourhundredge-0-0-0-9.302.core1.tuls.net.internet2.edu 4.355ms 5: fourhundredge-0-0-0-2.4079.core1.kans.net.internet2.edu 17.404ms asymm 11 6: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 18.753ms asymm 10 7: fourhundredge-0-0-0-22.4079.core1.chic.net.internet2.edu 18.430ms asymm 9 8: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 17.493ms 9: 144.92.233.112 20.550ms 10: perfsonar02.hep.wisc.edu 20.466ms reached Resume: pmtu 1500 hops 10 back 11
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osgccre (uid 20006)
MI-HORUS-CE2 4 hours ago 801
100.0%
43.4%
50.4%
43.4%
kernel env df firewall mtu nrp_images http_proxy user
MI-HORUS-CE2 — probe output
pass kernel
5.14.0-427.42.1.el9_4.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_p2k4ZM/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/glide_p2k4ZM/execute/dir_3696394/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3696394 CONTAINERSW_FULL_VERSION=apptainer version 1.4.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_p2k4ZM/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_MI-HORUS-CE2.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=696998 GLIDEIN_Entry_Name=OSG_US_MI-HORUS-CE2 GLIDEIN_Entry_Signature=6c7e977ce1e40ee42796d4e7c7d567455deb5b3d GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=mi-horus-ce2.svc.osg-htc.org mi-horus-ce2.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_N2sZbf GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=2 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=MI-HORUS-CE2 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=MI-HORUS GLIDEIN_Tmp_Dir=/scratch/glide_p2k4ZM/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.2-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=mi-horus-ce2.svc.osg-htc.org OSG_SITE_NAME=MI-HORUS-CE2 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_4045348_111823095@wsu-lc02.osris.org$ "; unset PROMPT_COMMAND PS1=slot1@glidein_4045348_111823095@wsu-lc02.osris.org$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_p2k4ZM/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.2-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/glide_p2k4ZM/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.2-1.el9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_p2k4ZM/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_p2k4ZM/execute/dir_3696394/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
2760.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg_scratch-lv_scratch 3124098648 229482220 2894616428 8% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: wsu-rtr-fe.osris.org 0.264ms 1: wsu-rtr-fe.osris.org 0.208ms 2: 10.0.8.17 0.218ms 3: 10.0.8.5 0.240ms 4: ae1x160.dtrt-wsudc-c1.mich.net 36.391ms 5: et-3-1-0x3.gdrp-eberhard-c1.mich.net 38.293ms 6: ae100x0.gdrp-gvsu-eberhard-c1.mich.net 37.990ms 7: ae102x0.chcg-lumen-600w-c1.mich.net 39.490ms asymm 8 8: 192.122.182.34 43.363ms asymm 9 9: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 45.721ms 10: 144.92.233.112 46.242ms 11: perfsonar02.hep.wisc.edu 47.699ms reached Resume: pmtu 1500 hops 11 back 12
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 100194)
MWT2 4 hours ago 270
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
MWT2 — probe output
pass kernel
5.14.0-687.25.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch.local/condor/execute/dir_3066075/glide_omNowV/images/htc__debian__13.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__debian__13.sif APPTAINER_TMPDIR=/scratch.local/condor/execute/dir_3066075/glide_omNowV/execute/dir_2438172/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2438172 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/bin/apptainer CONTAINERSW_VERSION=1.5.3-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T3_US_OSG GLIDEIN_CONDOR_TOKEN=/scratch.local/condor/execute/dir_3066075/glide_omNowV/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.Engage_US_MWT2_uct2_gk02_condce_mcore.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=6057251 GLIDEIN_Country=US GLIDEIN_DUNESite=US_UChicago GLIDEIN_Entry_Name=Engage_US_MWT2_uct2_gk02_condce_mcore GLIDEIN_Entry_Signature=3e6fe863e54568d80375334f892b58b8ea73a19a GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=uct2-gk02.mwt2.org uct2-gk02.mwt2.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_Pilgox GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=2 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=MWT2 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=UChicago GLIDEIN_Tmp_Dir=/scratch.local/condor/execute/dir_3066075/glide_omNowV/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=uct2-gk02.mwt2.org OSG_SITE_NAME=MWT2_CE_UC OSG_SQUID_LOCATION=uct2-squid.mwt2.org:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_7@glidein_3066077_10339768@mwt2-c085.campuscluster.illinois.edu$ "; unset PROMPT_COMMAND PS1=slot1_7@glidein_3066077_10339768@mwt2-c085.campuscluster.illinois.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch.local/condor/execute/dir_3066075/glide_omNowV/images/htc__debian__13.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch.local/condor/execute/dir_3066075/glide_omNowV/images/htc__debian__13.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__debian__13.sif SINGULARITY_PATH=/bin/apptainer SINGULARITY_VERSION=1.5.3-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch.local/condor/execute/dir_3066075/glide_omNowV/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch.local/condor/execute/dir_3066075/glide_omNowV/execute/dir_2438172/scratch _CONDOR_SLOT=slot1_7 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
2132.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda 2767142360 390976472 2235574784 15% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] 0.014ms pmtu 9000 1: vl-1675-cc-core.gw.ncsa.edu 0.266ms 1: vl-1675-cc-core.gw.ncsa.edu 0.254ms 2: 2620:0:c80:1::216 0.290ms 3: ncsa-07-br.gw.ncsa.edu 0.482ms 4: no reply 5: no reply 6: star-cr6--uwmadison-se306.lhcone.es.net 3.100ms 7: site--uwmadison-se306.lhcone.es.net 6.934ms 8: perfsonar02.hep.wisc.edu 6.859ms reached Resume: pmtu 9000 hops 8 back 9
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy uct2-squid.mwt2.org:3128
pass user
osg (uid 21039)
Missouri-Hellbender-CE1 4 hours ago 379
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Missouri-Hellbender-CE1 — probe output
pass kernel
4.18.0-553.141.2.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/local/scratch/glide_2Q0G7I/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/local/scratch/glide_2Q0G7I/execute/dir_176596/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=176596 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-3.el8 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.3-3.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/local/scratch/glide_2Q0G7I/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Missouri-Hellbender-CE1.idtoken.idtoken GLIDEIN_CPUS=2 GLIDEIN_ClusterId=6785286 GLIDEIN_Entry_Name=OSG_US_Missouri-Hellbender-CE1 GLIDEIN_Entry_Signature=7c15ab6cb15c1d219c0e97a491e0125e7e1d3dec GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=missouri-hellbender-ce1.svc.opensciencegrid.org missouri-hellbender-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_s2c_osg_01_snttnl GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Missouri-Hellbender-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Missouri-Hellbender GLIDEIN_Tmp_Dir=/local/scratch/glide_2Q0G7I/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-3.el8 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-3.el8 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-3.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=missouri-hellbender-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Missouri-Hellbender-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_2@glidein_3296314_352173632@c219.mgmt.hellbender$ "; unset PROMPT_COMMAND PS1=slot1_2@glidein_3296314_352173632@c219.mgmt.hellbender$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/local/scratch/glide_2Q0G7I/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-3.el8 SINGULARITY_IMAGES_DICT=default:/local/scratch/glide_2Q0G7I/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.3-3.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/local/scratch/glide_2Q0G7I/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/local/scratch/glide_2Q0G7I/execute/dir_176596/scratch _CONDOR_SLOT=slot1_2 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
899.2 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme1n1p1 1562049324 619161832 942887492 40% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: rcore2-hellbender-mgmt-v348.rnet.missouri.edu 0.792ms 1: rcore2-hellbender-mgmt-v348.rnet.missouri.edu 0.493ms 2: ncore-agg-rnet.bb.missouri.edu 0.633ms 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
s2c_osg_01 (uid 100753443)
NCSU-OSG-CE1 3 hours ago 2
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
NCSU-OSG-CE1 — probe output
pass kernel
5.14.0-570.23.1.el9_6.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_YZ84cd/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/glide_YZ84cd/execute/dir_2243177/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2243177 CONTAINERSW_FULL_VERSION=apptainer version 1.4.1-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.1-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_YZ84cd/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_NCSU-OSG-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6060635 GLIDEIN_Entry_Name=OSG_US_NCSU-OSG-CE1 GLIDEIN_Entry_Signature=955c37f87b542e11df4c8f9f0d3b9ddcd98b02c6 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=ncsu-osg-ce1.svc.osg-htc.org ncsu-osg-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_OSG01_YbPayJ GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=NCSU-OSG-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=NCSU-OSG GLIDEIN_Tmp_Dir=/scratch/glide_YZ84cd/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.1-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.1-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.1-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ncsu-osg-ce1.svc.osg-htc.org OSG_SITE_NAME=NCSU-OSG-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_1909478_55754244@vcledch0201.hpc.ncsu.edu$ "; unset PROMPT_COMMAND PS1=slot1@glidein_1909478_55754244@vcledch0201.hpc.ncsu.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_YZ84cd/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.1-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/glide_YZ84cd/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.1-1.el9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_YZ84cd/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_YZ84cd/execute/dir_2243177/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
3471.1 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p7 3671331348 31623908 3639707440 1% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: vl2140-dc2-6509-1-fwsm-2.ncstate.net 9.138ms 1: vl2140-dc2-6509-1-fwsm-2.ncstate.net 9.733ms 2: 10.250.142.110 10.184ms asymm 3 3: 10.250.133.5 9.669ms 4: 128.109.1.245 6.656ms 5: rtp-gw-to-ws-gw.ncren.net 18.248ms 6: bundle-ether250.613.core1.rale.net.internet2.edu 19.726ms 7: fourhundredge-0-0-0-6.4079.core1.ashb.net.internet2.edu 39.665ms asymm 13 8: fourhundredge-0-0-0-1.4079.core1.clev.net.internet2.edu 40.680ms asymm 12 9: fourhundredge-0-0-0-2.4079.core1.eqch.net.internet2.edu 40.309ms asymm 11 10: fourhundredge-0-0-0-6.4079.core1.star.net.internet2.edu 39.606ms 11: 144.92.233.112 41.792ms 12: perfsonar02.hep.wisc.edu 41.528ms reached Resume: pmtu 1500 hops 12 back 13
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
OSG01 (uid 1002)
NMSU-Discovery-CE1 4 hours ago 84
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
NMSU-Discovery-CE1 — probe output
pass kernel
4.18.0-553.92.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/fs1/osg/osg01/glide_hTJHfo/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/fs1/osg/osg01/glide_hTJHfo/execute/dir_695948/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=695948 CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-2.el8 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.5-2.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/fs1/osg/osg01/glide_hTJHfo/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_NMSU-Discovery-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=6507456 GLIDEIN_Entry_Name=OSG_US_NMSU-Discovery-CE1 GLIDEIN_Entry_Signature=8358b40f3525fe754120b33ab95eac2d518aae4f GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=nmsu-discovery-ce1.svc.osg-htc.org nmsu-discovery-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_hucy7k GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=192.168.211.16:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=NMSU-Discovery-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=NMSU_DISCOVERY GLIDEIN_Tmp_Dir=/fs1/osg/osg01/glide_hTJHfo/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-2.el8 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.5-2.el8 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.5-2.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=nmsu-discovery-ce1.svc.osg-htc.org OSG_SITE_NAME=NMSU-Discovery-CE1 OSG_SQUID_LOCATION=192.168.211.16:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_4@glidein_1553297_613257390@discovery-c36.cluster.local$ "; unset PROMPT_COMMAND PS1=slot1_4@glidein_1553297_613257390@discovery-c36.cluster.local$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/fs1/osg/osg01/glide_hTJHfo/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.5-2.el8 SINGULARITY_IMAGES_DICT=default:/fs1/osg/osg01/glide_hTJHfo/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.5-2.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/fs1/osg/osg01/glide_hTJHfo/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/fs1/osg/osg01/glide_hTJHfo/execute/dir_695948/scratch _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
33971.1 GB available Filesystem 1024-blocks Used Available Capacity Mounted on fs1 37580963840 1959673856 35621289984 6% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.88.37.254 28.706ms 1: 10.88.37.254 29.271ms 2: dc-core-vlan-88-dc.nmsu.edu 25.970ms 3: 10.123.129.137 20.992ms asymm 4 4: 10.123.128.49 21.953ms asymm 6 5: 10.123.128.45 23.213ms asymm 7 6: 192.65.78.193 22.904ms 7: abqg-rgon.checs.net 28.285ms 8: 208.77.77.233 29.187ms asymm 7 9: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 62.225ms asymm 15 10: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 71.420ms asymm 14 11: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 70.335ms asymm 14 12: fourhundredge-0-0-0-22.4079.core1.chic.net.internet2.edu 69.423ms asymm 13 13: 144.92.233.112 68.991ms 14: perfsonar02.hep.wisc.edu 58.937ms reached Resume: pmtu 1500 hops 14 back 15
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 192.168.211.16:3128
pass user
osg01 (uid 3001)
NWICG_NDCMS 4 hours ago 26
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
NWICG_NDCMS — probe output
pass kernel
5.14.0-611.55.1.el9_7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/cms,/users APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/var/condor/execute/dir_3025732/glide_gO7E7U/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/var/condor/execute/dir_3025732/glide_gO7E7U/execute/dir_3279035/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3279035 CONTAINERSW_FULL_VERSION=apptainer version 1.5.0-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/bin/apptainer CONTAINERSW_VERSION=1.5.0-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T3_US_NotreDame GLIDEIN_CONDOR_TOKEN=/var/condor/execute/dir_3025732/glide_gO7E7U/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.CMSHTPC_T3_US_NotreDame_deepthought.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=6715394 GLIDEIN_Country=US GLIDEIN_DUNESite=US_NotreDame GLIDEIN_Entry_Name=CMSHTPC_T3_US_NotreDame_deepthought GLIDEIN_Entry_Signature=dcd1b932072b203c6aef6d12ee825f820a438719 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=deepthought.crc.nd.edu deepthought.crc.nd.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_GS6I4S GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=3 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=NWICG_NDCMS GLIDEIN_SEs=deepthought.crc.nd.edu GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=NotreDame GLIDEIN_Tmp_Dir=/var/condor/execute/dir_3025732/glide_gO7E7U/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.0-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.0-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.0-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=deepthought.crc.nd.edu OSG_SITE_NAME=NWICG_NDCMS OSG_SQUID_LOCATION=eddie.crc.nd.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_5@glidein_3025747_127654202@d12chas580.crc.nd.edu$ "; unset PROMPT_COMMAND PS1=slot1_5@glidein_3025747_127654202@d12chas580.crc.nd.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/cms,/users SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/var/condor/execute/dir_3025732/glide_gO7E7U/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.0-1.el9 SINGULARITY_IMAGES_DICT=default:/var/condor/execute/dir_3025732/glide_gO7E7U/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/bin/apptainer SINGULARITY_VERSION=1.5.0-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/condor/execute/dir_3025732/glide_gO7E7U/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/condor/execute/dir_3025732/glide_gO7E7U/execute/dir_3279035/scratch _CONDOR_SLOT=slot1_5 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
1795.4 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 1944975360 62326896 1882648464 4% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: r64cisco9508-85-secondary.crc.nd.edu 0.515ms 1: r64cisco9508-85-secondary.crc.nd.edu 0.476ms 2: crc-def2compute-1.gw.nd.edu 0.521ms 3: 172.21.0.25 0.366ms asymm 4 4: man-pa-node-vip.fw.nd.edu 0.620ms 5: jjbr1-nside.gw.nd.edu 8.474ms 6: et-10-0-2.403.rtr.ll.indiana.gigapop.net 13.870ms 7: ae-7.1.rtr2.chic.indiana.gigapop.net 16.940ms 8: 144.92.233.118 34.375ms 9: perfsonar02.hep.wisc.edu 34.079ms reached Resume: pmtu 1500 hops 9 back 9
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy eddie.crc.nd.edu:3128
pass user
osg (uid 1005005)
Nebraska 4 hours ago 19
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Nebraska — probe output
pass kernel
5.14.0-687.29.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/var/lib/condor/execute/dir_2010569,/cvmfs,/etc/cvmfs,/etc/grid-security,/var/lib/cvmfs,/var/lib/sss/pipes,/var/lib/sss/mc,/etc/nsswitch.conf,/dev/fuse,/dev/log APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=rocky:9 APPTAINER_TMPDIR=/var/lib/condor/execute/dir_2010569/glide_aVTpaT/execute/dir_952341/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=952341 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-3.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/testing/bin/apptainer CONTAINERSW_VERSION=1.5.3-3.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T2_US_Nebraska GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_2010569/glide_aVTpaT/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.CMS_T2_US_Nebraska_Red_test_apptainer.idtoken.idtoken GLIDEIN_CPUS=32 GLIDEIN_ClusterId=6771535 GLIDEIN_Country=US GLIDEIN_ESTIMATED_CPUS=36 GLIDEIN_Entry_Name=CMS_T2_US_Nebraska_Red_test_apptainer GLIDEIN_Entry_Signature=275f6d1c4b41a632c43c5c3b9123ddd30a82e489 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=red.unl.edu red.unl.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_BwAAAw GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=red-squid.unl.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Nebraska GLIDEIN_SE_VONAME_LOWERCASE=srm://srm.unl.edu:8443/srm/v2/server?SFN=/mnt/hadoop/user/ GLIDEIN_SEs=srm.unl.edu GLIDEIN_SINGULARITY_BINARY_OVERRIDE=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/testing/bin/apptainer GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=8a65fb1cceb2c98bec9e9c523e155eb72d059921 GLIDEIN_Site=Nebraska GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_2010569/glide_aVTpaT/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-3.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/testing/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-3.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/testing/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-3.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=red.unl.edu OSG_SITE_NAME=Red OSG_SQUID_LOCATION=red-squid.unl.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_23@glidein_15_140314962@red-c9519.unl.edu$ "; unset PROMPT_COMMAND PS1=slot1_23@glidein_15_140314962@red-c9519.unl.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/var/lib/condor/execute/dir_2010569,/cvmfs,/etc/cvmfs,/etc/grid-security,/var/lib/cvmfs,/var/lib/sss/pipes,/var/lib/sss/mc,/etc/nsswitch.conf,/dev/fuse,/dev/log SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-3.el8 SINGULARITY_IMAGES_DICT=default:/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_MODE=unprivileged SINGULARITY_NAME=rocky:9 SINGULARITY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/testing/bin/apptainer SINGULARITY_VERSION=1.5.3-3.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_2010569/glide_aVTpaT/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_2010569/glide_aVTpaT/execute/dir_952341/scratch _CONDOR_SLOT=slot1_23 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=red-squid.unl.edu:3128
pass df
1572.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg1-lv_var 1925047904 276686692 1648361212 15% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] 0.015ms pmtu 9000 1: 2600:900:6:1102::28 0.593ms 1: 2600:900:6:1102::28 0.620ms 2: 2600:900:6:1048::1 0.874ms 3: no reply 4: chic-cr6--star-bb-e.igp.es.net 16.004ms asymm 6 5: no reply 6: site--uwmadison-se306.lhcone.es.net 20.062ms 7: perfsonar02.hep.wisc.edu 19.800ms reached Resume: pmtu 9000 hops 7 back 8
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy red-squid.unl.edu:3128
pass user
osg (uid 32016)
ORU-Titan-CE1 4 hours ago 20
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
ORU-Titan-CE1 — probe output
pass kernel
4.18.0-553.40.1.el8_10.x86_64
pass env
APPTAINER_CACHEDIR=/srv/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=578633 CONTAINERSW_FULL_VERSION=singularity version 3.7.1-5.1.ohpc.2.1 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/opt/ohpc/pub/libs/singularity/3.7.1/bin/singularity CONTAINERSW_VERSION=3.7.1-5.1.ohpc.2.1 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osgsubmit/glide_KgFo4a/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_ORU-Titan-CE1.idtoken.idtoken GLIDEIN_CPUS=16 GLIDEIN_ClusterId=663871 GLIDEIN_Entry_Name=OSG_US_ORU-Titan-CE1 GLIDEIN_Entry_Signature=00b24cfeb47ffe330523342e80f27f1385c9b4bd GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=oru-titan-ce1.svc.opensciencegrid.org oru-titan-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osgsubmit_uZy6p9 GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=3 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=ORU-Titan-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=ORU-Titan GLIDEIN_Tmp_Dir=/scratch/osgsubmit/glide_KgFo4a/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=singularity version 3.7.1-5.1.ohpc.2.1 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/opt/ohpc/pub/libs/singularity/3.7.1/bin/singularity GWMS_CONTAINERSW_VERSION=3.7.1-5.1.ohpc.2.1 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/opt/ohpc/pub/libs/singularity/3.7.1/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=3.7.1-5.1.ohpc.2.1 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=oru-titan-ce1.svc.opensciencegrid.org OSG_SITE_NAME=ORU-Titan-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_1@glidein_313779_211158084@c285.orca.oru.edu$ "; unset PROMPT_COMMAND PS1=slot1_1@glidein_313779_211158084@c285.orca.oru.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/scratch/osgsubmit/glide_KgFo4a/execute/dir_578633:/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_COMMAND=exec SINGULARITY_CONTAINER=/scratch/osgsubmit/glide_KgFo4a/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=singularity version 3.7.1-5.1.ohpc.2.1 SINGULARITY_IMAGES_DICT=default:/scratch/osgsubmit/glide_KgFo4a/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/opt/ohpc/pub/libs/singularity/3.7.1/bin/singularity SINGULARITY_TMPDIR=/scratch/osgsubmit/glide_KgFo4a/execute/dir_578633/scratch SINGULARITY_VERSION=3.7.1-5.1.ohpc.2.1 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osgsubmit/glide_KgFo4a/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osgsubmit/glide_KgFo4a/execute/dir_578633/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
461156.6 GB available Filesystem 1024-blocks Used Available Capacity Mounted on beegfs_nodev 777314222592 293756463104 483557759488 38% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.245ms asymm 2 1: _gateway 0.290ms asymm 2 2: 156.110.41.145 0.594ms 3: 156.110.31.225 1.183ms 4: 198.71.46.45 1.221ms asymm 5 5: fourhundredge-0-0-0-9.302.core1.tuls.net.internet2.edu 4.523ms 6: fourhundredge-0-0-0-2.4079.core1.kans.net.internet2.edu 17.271ms asymm 12 7: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 17.622ms asymm 11 8: fourhundredge-0-0-0-21.4079.core1.chic.net.internet2.edu 16.341ms asymm 10 9: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 17.937ms 10: 144.92.233.112 19.713ms 11: perfsonar02.hep.wisc.edu 19.604ms reached Resume: pmtu 1500 hops 11 back 12
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osgsubmit (uid 50149)
PDX-Coeus-CE1 38 mins ago 8
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user gpu
PDX-Coeus-CE1 — probe output
pass kernel
5.14.0-687.31.1.el9_8.0.1.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_QuuBvr/images/htc__ubuntu__24.04.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__ubuntu__24.04.sif APPTAINER_TMPDIR=/tmp/glide_QuuBvr/execute/dir_874384/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=874384 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/tmp/glide_QuuBvr/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_QuuBvr/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_PDX-Coeus-CE1_compute.idtoken.idtoken GLIDEIN_CPUS=20 GLIDEIN_ClusterId=663008 GLIDEIN_Entry_Name=OSG_US_PDX-Coeus-CE1_compute GLIDEIN_Entry_Signature=c3c6024705540d70daa482f368bf6d431f55712d GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=pdx-coeus-ce1.svc.osg-htc.org pdx-coeus-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_Tt1e4B GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=coeus.cluster:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=PDX-Coeus-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=PDX-Coeus GLIDEIN_Tmp_Dir=/tmp/glide_QuuBvr/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/tmp/glide_QuuBvr/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/tmp/glide_QuuBvr/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=pdx-coeus-ce1.svc.osg-htc.org OSG_SITE_NAME=PDX-Coeus-CE1 OSG_SQUID_LOCATION=coeus.cluster:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_2@glidein_628813_142987000@compute032$ "; unset PROMPT_COMMAND PS1=slot1_2@glidein_628813_142987000@compute032$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_QuuBvr/images/htc__ubuntu__24.04.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/tmp/glide_QuuBvr/images/htc__ubuntu__24.04.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__ubuntu__24.04.sif SINGULARITY_PATH=/tmp/glide_QuuBvr/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_QuuBvr/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_QuuBvr/execute/dir_874384/scratch _CONDOR_SLOT=slot1_2 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
123.4 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 195264552 65859056 129405496 34% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: coeus.cluster 0.928ms 1: coeus.cluster 0.921ms 2: 131.252.42.10 1.710ms asymm 3 3: 131.252.42.10 1.573ms asymm 4 4: 131.252.5.19 1.737ms 5: no reply 6: 131.252.5.213 1.656ms 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy coeus.cluster:3128
pass user
osg01 (uid 10262)
pass gpu
Sat Aug 1 09:47:44 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA A40 On | 00000000:01:00.0 Off | 0 | | 0% 25C P8 24W / 300W | 4MiB / 46068MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 1 NVIDIA A40 On | 00000000:41:00.0 Off | 0 | | 0% 25C P8 24W / 300W | 4MiB / 46068MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 2 NVIDIA A40 On | 00000000:81:00.0 Off | 0 | | 0% 24C P8 24W / 300W | 4MiB / 46068MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ | 3 NVIDIA A40 On | 00000000:C1:00.0 Off | 0 | | 0% 25C P8 23W / 300W | 4MiB / 46068MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+
PDX-Orca-CE1 4 hours ago 6
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
PDX-Orca-CE1 — probe output
pass kernel
5.14.0-687.29.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_MLZGRz/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_MLZGRz/execute/dir_1284285/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=1284285 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/tmp/glide_MLZGRz/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_MLZGRz/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_PDX-Orca-CE1.idtoken.idtoken GLIDEIN_CPUS=32 GLIDEIN_ClusterId=6054633 GLIDEIN_Entry_Name=OSG_US_PDX-Orca-CE1 GLIDEIN_Entry_Signature=ae9356dec30dff3ec2bf656f8d66cb3c3a74eb8a GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=pdx-orca-ce1.svc.osg-htc.org pdx-orca-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_xFktVG GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=PDX-Orca-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=PDX-Orca GLIDEIN_Tmp_Dir=/tmp/glide_MLZGRz/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/tmp/glide_MLZGRz/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/tmp/glide_MLZGRz/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=pdx-orca-ce1.svc.osg-htc.org OSG_SITE_NAME=PDX-Orca-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_13@glidein_2890619_750031800@orcaga21$ "; unset PROMPT_COMMAND PS1=slot1_13@glidein_2890619_750031800@orcaga21$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_MLZGRz/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/tmp/glide_MLZGRz/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/tmp/glide_MLZGRz/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_MLZGRz/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_MLZGRz/execute/dir_1284285/scratch _CONDOR_SLOT=slot1_13 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
244.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sdb1 460367788 180715952 256192924 42% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 9000 1: _gateway 0.815ms asymm 2 1: _gateway 0.642ms asymm 2 2: _gateway 0.731ms asymm 3 3: 10.252.100.1 0.151ms pmtu 1500 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 10262)
Purdue-Anvil-CE1 4 hours ago 22
100.0%
81.8%
0.0%
81.8%
kernel env df firewall mtu nrp_images http_proxy user
Purdue-Anvil-CE1 — probe output
pass kernel
4.18.0-553.81.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_HCaZN9/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_HCaZN9/execute/dir_2502760/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2502760 CONTAINERSW_FULL_VERSION=apptainer version 1.4.3-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.3-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_HCaZN9/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Purdue-Anvil-CE1.idtoken.idtoken GLIDEIN_CPUS=128 GLIDEIN_ClusterId=692450 GLIDEIN_Entry_Name=OSG_US_Purdue-Anvil-CE1 GLIDEIN_Entry_Signature=5768c7c9b0a85a409b06050ec4702e79e03d4327 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=purdue-anvil-ce1.svc.osg-htc.org purdue-anvil-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_x-ob_Z94HfO GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Purdue-Anvil-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-2.osg-htc.org GLIDEIN_Signature=831b80533b1b88ec87a4ed92f0155e0403abb177 GLIDEIN_Site=Purdue-Anvil GLIDEIN_Tmp_Dir=/tmp/glide_HCaZN9/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.3-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.3-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.3-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=purdue-anvil-ce1.svc.osg-htc.org OSG_SITE_NAME=Purdue-Anvil-CE1 OSG_SQUID_LOCATION=squid1.rcac.purdue.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_15@glidein_3272655_452223910@a121.anvil.rcac.purdue.edu$ "; unset PROMPT_COMMAND PS1=slot1_15@glidein_3272655_452223910@a121.anvil.rcac.purdue.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_HCaZN9/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.3-1.el8 SINGULARITY_IMAGES_DICT=default:/tmp/glide_HCaZN9/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.3-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_HCaZN9/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_HCaZN9/execute/dir_2502760/scratch _CONDOR_SLOT=slot1_15 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
179.1 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda2 374917264 187086608 187830656 50% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=4092 1?: [LOCALHOST] pmtu 4092 1: 172.17.3.179 0.417ms 1: 172.17.3.179 0.417ms 2: math-g190-ar7280dr3a-01-et7-1.tcom.purdue.edu 5.880ms 3: math-g190-ar7280dr3a-01-vl2500.tcom.purdue.edu 0.311ms 4: math-g190-ar7280dr3a-02-et54-1.tcom.purdue.edu 0.328ms asymm 3 5: research-gigapop-ll-lhcone-400g.tcom.purdue.edu 2.147ms asymm 4 6: lo-0.2.rtr2.chic.indiana.gigapop.net 5.671ms asymm 5 7: no reply 8: esnet-lhc1-uwmadison.es.net 5.569ms asymm 7 9: uwmadison-lhc1-esnet.es.net 9.362ms asymm 8 10: perfsonar02.hep.wisc.edu 9.379ms reached Resume: pmtu 4092 hops 10 back 10
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
fail http_proxy
Failed to download http://ospool-network-test.osg.chtc.io/index.html via proxy squid1.rcac.purdue.edu:3128: <urlopen error [Errno 101] Network is unreachable>
pass user
x-ob (uid 7941029)
Purdue-Geddes 4 hours ago 10
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Purdue-Geddes — probe output
pass kernel
4.18.0-553.22.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-HWF2DG/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-HWF2DG/execute/dir_831824 BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=831824 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=Purdue Geddes OSG_SQUID_LOCATION=http://squid1.rcac.purdue.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_4@Purdue-Geddes.osg-geddes-backfill-a-8$ "; unset PROMPT_COMMAND PS1=slot1_4@Purdue-Geddes.osg-geddes-backfill-a-8$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv SINGULARITY_CONTAINER=/pilot/osgvo-pilot-HWF2DG/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-HWF2DG/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-HWF2DG/execute/dir_831824 _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure http_proxy=http://squid1.rcac.purdue.edu:3128
pass df
73.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/rbd2 123268672 46685568 76566720 38% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=8950 1?: [LOCALHOST] pmtu 8950 1: 172-21-160-11.rancher-monitoring-prometheus-node-exporter.cattle-monitoring-system.svc.cluster.local 0.085ms 1: 172-21-160-11.rancher-monitoring-prometheus-node-exporter.cattle-monitoring-system.svc.cluster.local 0.201ms 2: math-g109-dz9264f-01-vlan1160.tcom.purdue.edu 0.398ms 3: math-g190-c7710-01-eth-3-15.tcom.purdue.edu 0.254ms 4: math-g190-ar7280dr3a-01-vl2500.tcom.purdue.edu 0.477ms 5: math-g190-ar7280dr3a-02-et54-1.tcom.purdue.edu 0.388ms asymm 4 6: research-gigapop-ll-lhcone-400g.tcom.purdue.edu 2.285ms asymm 5 7: lo-0.2.rtr2.chic.indiana.gigapop.net 5.717ms asymm 6 8: no reply 9: esnet-lhc1-uwmadison.es.net 5.747ms asymm 8 10: uwmadison-lhc1-esnet.es.net 9.740ms asymm 9 11: perfsonar02.hep.wisc.edu 9.577ms reached Resume: pmtu 8950 hops 11 back 11
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
fail http_proxy
Failed to download http://ospool-network-test.osg.chtc.io/index.html via proxy http://squid1.rcac.purdue.edu:3128: <urlopen error [Errno 101] Network is unreachable>
pass user
osg (uid 1000)
Rhodes-HPC-CE1 4 hours ago 10
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Rhodes-HPC-CE1 — probe output
pass kernel
5.14.0-687.12.1.el9_8.0.1.x86_64
pass env
APPTAINER_CACHEDIR=/srv/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=840179 CONTAINERSW_FULL_VERSION=singularity-ce version 4.4.1-1.el9 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/singularity CONTAINERSW_VERSION=4.4.1-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_SXlvEx/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Rhodes-HPC-CE1.idtoken.idtoken GLIDEIN_CPUS=48 GLIDEIN_ClusterId=6784824 GLIDEIN_Entry_Name=OSG_US_Rhodes-HPC-CE1 GLIDEIN_Entry_Signature=bc2f8e148f920af651dc8e9f0541b83931ad522c GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=rhodes-hpc-ce1.svc.osg-htc.org rhodes-hpc-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_0XyEcv GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=10.0.0.1:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Rhodes-HPC-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Rhodes-HPC GLIDEIN_Tmp_Dir=/scratch/glide_SXlvEx/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=singularity-ce version 4.4.1-1.el9 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/singularity GWMS_CONTAINERSW_VERSION=4.4.1-1.el9 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=4.4.1-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=rhodes-hpc-ce1.svc.osg-htc.org OSG_SITE_NAME=Rhodes-HPC-CE1 OSG_SQUID_LOCATION=10.0.0.1:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_16@glidein_27086_57244572@compute30$ "; unset PROMPT_COMMAND PS1=slot1_16@glidein_27086_57244572@compute30$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/scratch/glide_SXlvEx/execute/dir_840179:/srv,/etc/hosts,/etc/localtime,/cvmfs:/cvmfs:ro SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_COMMAND=exec SINGULARITY_CONTAINER=/scratch/glide_SXlvEx/images/htc__rocky__8.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=singularity-ce version 4.4.1-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/glide_SXlvEx/images/htc__rocky__8.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__8.sif SINGULARITY_PATH=/usr/bin/singularity SINGULARITY_TMPDIR=/scratch/glide_SXlvEx/execute/dir_840179/scratch SINGULARITY_VERSION=4.4.1-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_SXlvEx/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_SXlvEx/execute/dir_840179/scratch _CONDOR_SLOT=slot1_16 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
371.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 499862048 110308372 389553676 23% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: lotus 0.190ms 1: lotus 0.142ms 2: 10.70.0.1 0.149ms 3: xg-1-1-35-3909-soag12.hornlake.tn.malt.comcast.net 1.219ms 4: no reply 5: no reply 6: be-288-ar01.hornlake.tn.malt.comcast.net 1.855ms 7: 162.151.119.229 1.116ms 8: be-34922-cs02.dallas.tx.ibone.comcast.net 9.984ms 9: be-3311-pe11.1950stemmons.tx.ibone.comcast.net 10.441ms 10: dls-b7-link.ip.twelve99.net 10.933ms 11: dls-b23-link.ip.twelve99.net 11.466ms 12: kanc-bb2-link.ip.twelve99.net 26.269ms 13: chi-bb2-link.ip.twelve99.net 27.048ms asymm 10 14: chi-b24-link.ip.twelve99.net 26.462ms asymm 13 15: wiscnet-ic-308898.ip.twelve99-cust.net 26.808ms asymm 13 16: r-uwmilwaukee-ems-ae1-3473.uwsys.net 30.250ms asymm 14 17: r-uwmadison-cssc-ae4-3455.uwsys.net 32.874ms asymm 15 18: 143.235.41.129 33.341ms asymm 16 19: perfsonar02.hep.wisc.edu 33.693ms reached Resume: pmtu 1500 hops 19 back 18
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 10.0.0.1:3128
pass user
osg01 (uid 10000)
SDSC-PRP-OSPool-Provisioner 4 hours ago 2152
100.0%
100.0%
98.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user gpu
SDSC-PRP-OSPool-Provisioner — probe output
pass kernel
6.8.0-124-generic
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-6j1lp0/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-6j1lp0/execute/dir_459149/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=459149 CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-3.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.3-3.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.3-3.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.3-3.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.3-3.el8 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=SDSC-PRP OSG_SQUID_LOCATION=10.110.90.167:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1@SDSC-PRP-OSPool-Provisioner.osg-direct-6a490096-00ea92-5vtkg$ "; unset PROMPT_COMMAND PS1=slot1@SDSC-PRP-OSPool-Provisioner.osg-direct-6a490096-00ea92-5vtkg$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-6j1lp0/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.3-3.el8 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-6j1lp0/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.3-3.el8 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-6j1lp0/execute/dir_459149/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=10.110.90.167:3128
pass df
263.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on overlay 959218776 633803824 276615620 70% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=8930 1?: [LOCALHOST] pmtu 8930 1: 192.168.103.131 0.181ms 1: 192.168.103.131 0.225ms 2: 192.69.103.190 0.751ms 3: mghpcc-re-nox-mghpcc-gw1.nox.org 1.360ms 4: nox-mghpcc-gw1-i2-re-chic.nox.org 6.400ms 5: fourhundredge-0-0-0-2.4079.core2.clev.net.internet2.edu 18.785ms asymm 9 6: fourhundredge-0-0-0-2.4079.core2.eqch.net.internet2.edu 20.109ms asymm 8 7: fourhundredge-0-0-0-10.4079.core1.star.net.internet2.edu 18.896ms 8: 144.92.233.112 20.939ms 9: perfsonar02.hep.wisc.edu 20.882ms reached Resume: pmtu 8930 hops 9 back 10
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 10.110.90.167:3128
pass user
osg (uid 1000)
pass gpu
Sat Aug 1 12:57:41 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.173.02 Driver Version: 580.173.02 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100-SXM2-32GB Off | 00000000:07:00.0 Off | 0 | | N/A 37C P0 45W / 300W | 4MiB / 32768MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+
SIUE-Kyna-CE1 15 mins ago 2
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
SIUE-Kyna-CE1 — probe output
pass kernel
6.8.0-111-generic
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/osg/glide_VR4fGM/images/htc__debian__13.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__debian__13.sif APPTAINER_TMPDIR=/scratch/osg/glide_VR4fGM/execute/dir_4125105/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=4125105 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/osg/glide_VR4fGM/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osg/glide_VR4fGM/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_SIUE-Kyna-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6219929 GLIDEIN_Entry_Name=OSG_US_SIUE-Kyna-CE1 GLIDEIN_Entry_Signature=bded58c437163399b6fd7bc18aa81b2b7ed1d327 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=siue-kyna-ce1.svc.osg-htc.org siue-kyna-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_WKOWtC GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=SIUE-Kyna-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=SIUE-Kyna GLIDEIN_Tmp_Dir=/scratch/osg/glide_VR4fGM/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/osg/glide_VR4fGM/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/osg/glide_VR4fGM/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv HTTPS_PROXY=http://10.10.44.230:8080 HTTP_PROXY=http://10.10.44.230:8080 JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NO_PROXY=::1,127.0.0.1,localhost,10.0.0.0/8,*.hpc.siue.edu NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=siue-kyna-ce1.svc.osg-htc.org OSG_SITE_NAME=SIUE-Kyna-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_4092617_196575855@cn11.hpc.siue.edu$ "; unset PROMPT_COMMAND PS1=slot1@glidein_4092617_196575855@cn11.hpc.siue.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/osg/glide_VR4fGM/images/htc__debian__13.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/osg/glide_VR4fGM/images/htc__debian__13.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__debian__13.sif SINGULARITY_PATH=/scratch/osg/glide_VR4fGM/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osg/glide_VR4fGM/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osg/glide_VR4fGM/execute/dir_4125105/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=http://10.10.44.230:8080 https_proxy=http://10.10.44.230:8080 no_proxy=::1,127.0.0.1,localhost,10.0.0.0/8,*.hpc.siue.edu
pass df
1641.1 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vgroot-lvroot 1843643996 29083124 1720835176 2% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.10.44.252 0.641ms 1: 10.10.44.252 0.222ms 2: 146.163.142.254 0.551ms 3: vdom-root120.net.siue.edu 0.625ms 4: dh0122-r1.net.siue.edu 1.969ms 5: mren-router.net.siue.edu 17.177ms 6: hundredge-0-0-0-28.1136.core1.star.net.internet2.edu 10.457ms 7: 144.92.233.112 12.155ms 8: perfsonar02.hep.wisc.edu 12.003ms reached Resume: pmtu 1500 hops 8 back 9
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy http://10.10.44.230:8080
pass user
osg (uid 64034)
SLU-Libra-CE1 3 hours ago 2
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
SLU-Libra-CE1 — probe output
pass kernel
5.14.0-611.55.1.el9_7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/osg01/glide_A5dY90/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/osg01/glide_A5dY90/execute/dir_2322243/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2322243 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/osg01/glide_A5dY90/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osg01/glide_A5dY90/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_SLU-Libra-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=663176 GLIDEIN_Entry_Name=OSG_US_SLU-Libra-CE1 GLIDEIN_Entry_Signature=d00303639042bdca2037435a79db90f33981c26c GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=slu-libra-ce1.svc.osg-htc.org slu-libra-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_GyXUCo GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=SLU-Libra-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=SLU-Libra GLIDEIN_Tmp_Dir=/scratch/osg01/glide_A5dY90/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/osg01/glide_A5dY90/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/osg01/glide_A5dY90/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=slu-libra-ce1.svc.osg-htc.org OSG_SITE_NAME=SLU-Libra-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_2019499_61485710@comp01$ "; unset PROMPT_COMMAND PS1=slot1@glidein_2019499_61485710@comp01$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/osg01/glide_A5dY90/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/osg01/glide_A5dY90/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/scratch/osg01/glide_A5dY90/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osg01/glide_A5dY90/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osg01/glide_A5dY90/execute/dir_2322243/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
95608.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on 10.149.6.1:/data01/scratch/osg01/glide_A5dY90/execute/dir_2322243 120461613056 20208834560 100252778496 17% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: head01.cm.cluster 0.195ms 1: head01.cm.cluster 0.192ms 2: 10.108.121.254 16.899ms 3: no reply 4: 10.248.252.1 1.260ms 5: 10.248.101.30 17.139ms 6: 10.248.100.20 24.916ms 7: 165.134.223.27 25.241ms 8: fourhundredge-0-0-0-10.23.core2.kans.net.internet2.edu 37.716ms 9: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 43.906ms asymm 11 10: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 44.632ms 11: 144.92.233.112 47.106ms 12: perfsonar02.hep.wisc.edu 44.633ms reached Resume: pmtu 1500 hops 12 back 13
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1001)
SU-ITS-CE2 4 hours ago 151
100.0%
0.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
SU-ITS-CE2 — probe output
warning kernel
3.10.0-1160.66.1.el7.x86_64
pass env
APPTAINER_CACHEDIR=/srv BATCH_SYSTEM=HTCondor CONTAINERSW_FULL_VERSION=singularity version 3.6.0-rc.4 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/bin/singularity CONTAINERSW_VERSION=3.6.0-rc.4 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T3_US_OSG GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_37411/glide_QSFOPw/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.Glow_US_Syracuse_condor-ce2_whole.idtoken.idtoken GLIDEIN_CPUS=40 GLIDEIN_ClusterId=558721 GLIDEIN_Country=US GLIDEIN_ESTIMATED_CPUS=40 GLIDEIN_Entry_Name=Glow_US_Syracuse_condor-ce2_whole GLIDEIN_Entry_Signature=7b1ccc0140eeeee557a00417483a9bc17d093937 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=its-condor-ce2.syr.edu its-condor-ce2.syr.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_nobody_QPrwso GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=10.5.0.41:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=SU-ITS-CE2 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-2.osg-htc.org GLIDEIN_Signature=831b80533b1b88ec87a4ed92f0155e0403abb177 GLIDEIN_Site=SU-ITS GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_37411/glide_QSFOPw/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=singularity version 3.6.0-rc.4 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/bin/singularity GWMS_CONTAINERSW_VERSION=3.6.0-rc.4 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=3.6.0-rc.4 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=unavailable HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=its-condor-ce2.syr.edu OSG_SITE_NAME=SU-ITS-CE2 OSG_SQUID_LOCATION=10.5.0.41:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_29@glidein_37413_159195950@CRUSH-OSG-C7-10-5-174-107$ "; unset PROMPT_COMMAND PS1=slot1_29@glidein_37413_159195950@CRUSH-OSG-C7-10-5-174-107$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_CACHEDIR=/srv SINGULARITY_COMMAND=exec SINGULARITY_CONTAINER=/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=singularity version 3.6.0-rc.4 SINGULARITY_IMAGES_DICT=default:/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_MODE=privileged SINGULARITY_NAME=rocky:9 SINGULARITY_PATH=/bin/singularity SINGULARITY_TMPDIR=/var/lib/condor/execute/dir_37411/glide_QSFOPw/execute/dir_118297 SINGULARITY_VERSION=3.6.0-rc.4 SLOTS_LAYOUT=partitionable TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_37411/glide_QSFOPw/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_37411/glide_QSFOPw/execute/dir_118297 _CONDOR_SLOT=slot1_29 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure
pass df
123.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/centos-root 206542340 76759500 129782840 38% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.5.255.253 0.553ms 1: 10.5.255.253 0.623ms 2: 10.30.29.35 0.360ms 3: 10.30.29.44 0.643ms asymm 2 4: 128.230.61.33 1.501ms asymm 3 5: syr-9208-su.nysernet.net 0.981ms asymm 4 6: buf-9208-syr-9208.nysernet.net 48.360ms asymm 5 7: i2-clev-buf-9208.nysernet.net 10.359ms asymm 6 8: fourhundredge-0-0-0-2.4079.core2.eqch.net.internet2.edu 16.207ms asymm 9 9: fourhundredge-0-0-0-10.4079.core1.star.net.internet2.edu 17.128ms asymm 8 10: 144.92.233.112 18.126ms asymm 9 11: perfsonar02.hep.wisc.edu 17.794ms reached Resume: pmtu 1500 hops 11 back 11
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 10.5.0.41:3128
pass user
nobody (uid 99)
SU-ITS-CE3 4 hours ago 106
100.0%
0.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
SU-ITS-CE3 — probe output
warning kernel
3.10.0-1160.66.1.el7.x86_64
pass env
APPTAINER_CACHEDIR=/srv BATCH_SYSTEM=HTCondor CONTAINERSW_FULL_VERSION=singularity version 3.6.0-rc.4 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/bin/singularity CONTAINERSW_VERSION=3.6.0-rc.4 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T3_US_OSG GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_52006/glide_JhRgpK/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.Glow_US_Syracuse_condor-ce3_whole.idtoken.idtoken GLIDEIN_CPUS=64 GLIDEIN_ClusterId=691488 GLIDEIN_Country=US GLIDEIN_ESTIMATED_CPUS=40 GLIDEIN_Entry_Name=Glow_US_Syracuse_condor-ce3_whole GLIDEIN_Entry_Signature=38c8751d0024bd320eb88169e1f71c4f3c7b19e4 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=its-condor-ce3.syr.edu its-condor-ce3.syr.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_nobody_Ezk21o GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=10.5.0.41:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=SU-ITS-CE3 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-2.osg-htc.org GLIDEIN_Signature=4fff592f8ad411133f59628f88c182b905a98080 GLIDEIN_Site=SU-ITS GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_52006/glide_JhRgpK/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=singularity version 3.6.0-rc.4 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/bin/singularity GWMS_CONTAINERSW_VERSION=3.6.0-rc.4 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/bin/singularity GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=3.6.0-rc.4 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=unavailable HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=its-condor-ce3.syr.edu OSG_SITE_NAME=SU-ITS-CE3 OSG_SQUID_LOCATION=10.5.0.41:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_7@glidein_52008_416491038@CRUSH-OSG-C7-10-5-172-252$ "; unset PROMPT_COMMAND PS1=slot1_7@glidein_52008_416491038@CRUSH-OSG-C7-10-5-172-252$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_CACHEDIR=/srv SINGULARITY_COMMAND=exec SINGULARITY_CONTAINER=/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=singularity version 3.6.0-rc.4 SINGULARITY_IMAGES_DICT=default:/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_MODE=privileged SINGULARITY_NAME=rocky:9 SINGULARITY_PATH=/bin/singularity SINGULARITY_TMPDIR=/var/lib/condor/execute/dir_52006/glide_JhRgpK/execute/dir_59387 SINGULARITY_VERSION=3.6.0-rc.4 SLOTS_LAYOUT=partitionable TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_52006/glide_JhRgpK/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_52006/glide_JhRgpK/execute/dir_59387 _CONDOR_SLOT=slot1_7 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure
pass df
85.4 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/centos-root 206542340 116951748 89590592 57% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.5.255.253 0.457ms 1: 10.5.255.254 0.492ms 2: 10.30.29.35 0.215ms 3: 10.30.29.44 1.032ms asymm 2 4: 128.230.61.33 1.461ms asymm 3 5: syr-9208-su.nysernet.net 0.959ms asymm 4 6: buf-9208-syr-9208.nysernet.net 4.302ms asymm 5 7: i2-clev-buf-9208.nysernet.net 9.678ms asymm 6 8: fourhundredge-0-0-0-2.4079.core2.eqch.net.internet2.edu 16.146ms asymm 9 9: fourhundredge-0-0-0-10.4079.core1.star.net.internet2.edu 15.155ms asymm 8 10: 144.92.233.112 17.813ms asymm 9 11: perfsonar02.hep.wisc.edu 17.824ms reached Resume: pmtu 1500 hops 11 back 11
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 10.5.0.41:3128
pass user
nobody (uid 99)
SYRA-PATH-EP 5 hours ago 1
100.0%
100.0%
100.0%
100.0%
all
SYRA-PATH-EP — probe output
warning all
Job failed with exit code 0; no results transferred
Swan-CE1 4 hours ago 15
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Swan-CE1 — probe output
pass kernel
4.18.0-553.123.2.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_P8f5An/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/glide_P8f5An/execute/dir_3774819/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3774819 CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-3.el8 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.5-3.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T2_US_Nebraska GLIDEIN_CONDOR_TOKEN=/scratch/glide_P8f5An/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.HCC_US_Omaha_swan.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6716063 GLIDEIN_Country=US GLIDEIN_DUNESite=US_Swan GLIDEIN_Entry_Name=HCC_US_Omaha_swan GLIDEIN_Entry_Signature=aee75d517c6d6bd9340a1971104a714f5299ee33 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=swan-gw1.unl.edu swan-gw1.unl.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_sGW24t GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Swan-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Swan GLIDEIN_Tmp_Dir=/scratch/glide_P8f5An/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-3.el8 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.5-3.el8 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.5-3.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=unavailable HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=swan-gw1.unl.edu OSG_SITE_NAME=Swan OSG_SQUID_LOCATION=red-squid.unl.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_3482277_577395480@c1513.swan.hcc.unl.edu$ "; unset PROMPT_COMMAND PS1=slot1@glidein_3482277_577395480@c1513.swan.hcc.unl.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_P8f5An/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.5-3.el8 SINGULARITY_IMAGES_DICT=default:/scratch/glide_P8f5An/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.5-3.el8 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TERM=vt220 TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_P8f5An/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_P8f5An/execute/dir_3774819/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
3525.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/md3 3748646536 51647792 3696998744 2% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: c3838.pki.hcc.unl.edu 0.134ms 1: _gateway 6.765ms 1: c3838.pki.hcc.unl.edu 0.129ms 2: c3838.pki.hcc.unl.edu 0.070ms asymm 1 3: hcc-pki-mlxe.unl.edu 0.392ms asymm 2 4: 129.93.241.41 0.547ms asymm 3 5: 129.93.6.29 2.406ms asymm 4 6: 10.161.1.14 2.948ms asymm 5 7: fourhundredge-0-0-0-10.1410.core2.kans.net.internet2.edu 9.028ms asymm 6 8: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 47.256ms asymm 9 9: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 20.080ms asymm 8 10: 144.92.233.112 20.392ms asymm 9 11: perfsonar02.hep.wisc.edu 17.323ms reached Resume: pmtu 1500 hops 11 back 10
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy red-squid.unl.edu:3128
pass user
osg (uid 32016)
Swarthmore-Firebird-CE1 4 hours ago 49
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Swarthmore-Firebird-CE1 — probe output
pass kernel
5.14.0-611.55.1.el9_7.0.3.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/osg01/glide_KOO0Cz/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/osg01/glide_KOO0Cz/execute/dir_1649116/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=1649116 CONTAINERSW_FULL_VERSION=apptainer version 1.5.0-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.0-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/osg01/glide_KOO0Cz/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Swarthmore-firebird-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=697737 GLIDEIN_Entry_Name=OSG_US_Swarthmore-firebird-CE1 GLIDEIN_Entry_Signature=6f861edc0a5b4de080f55fe261a21e9ae2acee4f GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=swarthmore-firebird-ce1.svc.osg-htc.org swarthmore-firebird-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_JN83be GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Swarthmore-Firebird-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins3@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=Swarthmore-Firebird GLIDEIN_Tmp_Dir=/scratch/osg01/glide_KOO0Cz/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.0-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.0-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.0-1.el9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=swarthmore-firebird-ce1.svc.osg-htc.org OSG_SITE_NAME=Swarthmore-Firebird-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_2@glidein_14997_206527442@node07.cluster$ "; unset PROMPT_COMMAND PS1=slot1_2@glidein_14997_206527442@node07.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/osg01/glide_KOO0Cz/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.0-1.el9 SINGULARITY_IMAGES_DICT=default:/scratch/osg01/glide_KOO0Cz/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.0-1.el9 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/osg01/glide_KOO0Cz/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/osg01/glide_KOO0Cz/execute/dir_1649116/scratch _CONDOR_SLOT=slot1_2 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
765.2 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/disk0-scratch 877647156 75281408 802365748 9% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: firebird.cluster 2.917ms 1: firebird.cluster 2.539ms 2: fw-comsvc-colo.lafayette.edu 3.562ms 3: 139.147.1.113 2.818ms asymm 4 4: 172.31.0.33 4.141ms 5: 172.31.0.3 4.010ms 6: gw0-g0-3-ibgp.prd.lafayette.edu 4.076ms asymm 5 7: 164.153.140.160 8.004ms asymm 6 8: no reply 9: hundredge-0-0-0-25.68.core1.pitt.net.internet2.edu 18.205ms asymm 8 10: fourhundredge-0-0-0-3.4079.core1.clev.net.internet2.edu 24.825ms asymm 13 11: fourhundredge-0-0-0-2.4079.core1.eqch.net.internet2.edu 25.642ms asymm 12 12: fourhundredge-0-0-0-6.4079.core1.star.net.internet2.edu 27.281ms asymm 11 13: 144.92.233.112 27.053ms asymm 12 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 200079)
TCNJ-ELSA-CE2 4 hours ago 2
100.0%
100.0%
0.0%
100.0%
gpu
TCNJ-ELSA-CE2 — probe output
pass gpu
Sat Aug 1 08:52:49 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.159.04 Driver Version: 580.159.04 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce GTX 1080 Ti On | 00000000:1A:00.0 Off | N/A | | 23% 16C P8 8W / 250W | 3MiB / 11264MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+
UA-LR-ITS-EP 4 hours ago 29
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UA-LR-ITS-EP — probe output
pass kernel
4.18.0-553.6.1.el8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-3mA2Vz/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-3mA2Vz/execute/dir_205563/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=205563 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=UA Little Rock ITS OSG_SQUID_LOCATION=144.167.99.254:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_8@UA-LR-ITS-EP.a56a55e758c3$ "; unset PROMPT_COMMAND PS1=slot1_8@UA-LR-ITS-EP.a56a55e758c3$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-3mA2Vz/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-3mA2Vz/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-3mA2Vz/execute/dir_205563/scratch _CONDOR_SLOT=slot1_8 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=144.167.99.254:3128
pass df
10.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on overlay 132653920 114509800 11332884 91% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.063ms 1: _gateway 0.085ms 2: 144.167.99.3 0.196ms 3: FH213-Firewall-Campus.net.ualr.edu 8.035ms 4: rtr-fh213-intf2-0.mgt.ualr.edu 1.096ms 5: 208.90.110.230 8.382ms 6: 208.90.110.230 8.216ms asymm 5 7: no reply 8: no reply 9: no reply 10: 208.90.110.99 20.478ms 11: 208.90.110.99 19.864ms asymm 10 12: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 33.485ms asymm 16 13: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 35.011ms asymm 16 14: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 33.538ms 15: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 36.038ms asymm 14 16: perfsonar02.hep.wisc.edu 36.122ms reached Resume: pmtu 1500 hops 16 back 17
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 144.167.99.254:3128
fail user
uid 1000 does not map to a user
UAH-Voyager-CE1 4 hours ago 31
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UAH-Voyager-CE1 — probe output
pass kernel
5.14.0-503.40.1.el9_5.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/data001/OSG/glide_Vtecaf/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/data001/OSG/glide_Vtecaf/execute/dir_2705469/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2705469 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/data001/OSG/glide_Vtecaf/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/data001/OSG/glide_Vtecaf/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UAH-Voyager-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6060428 GLIDEIN_Entry_Name=OSG_US_UAH-Voyager-CE1 GLIDEIN_Entry_Signature=1c83400d47aa2e2cd09854a5796c94c1dc855583 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=uah-voyager-ce1.svc.osg-htc.org uah-voyager-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_BSqWxn GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UAH-Voyager-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=UAH-Voyager GLIDEIN_Tmp_Dir=/data001/OSG/glide_Vtecaf/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/data001/OSG/glide_Vtecaf/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/data001/OSG/glide_Vtecaf/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=uah-voyager-ce1.svc.osg-htc.org OSG_SITE_NAME=UAH-Voyager-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_239521_262310580@node019$ "; unset PROMPT_COMMAND PS1=slot1@glidein_239521_262310580@node019$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/data001/OSG/glide_Vtecaf/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/data001/OSG/glide_Vtecaf/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/data001/OSG/glide_Vtecaf/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/data001/OSG/glide_Vtecaf/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/data001/OSG/glide_Vtecaf/execute/dir_2705469/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
374580.2 GB available Filesystem 1024-blocks Used Available Capacity Mounted on 172.17.170.1:/zpool/raid001/data001/OSG/glide_Vtecaf/execute/dir_2705469 393108115456 332316672 392775798784 1% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: stor001 0.281ms 1: stor001 0.350ms 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 50004)
UCMerced-Pinnacles-CE1 4 hours ago 62
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UCMerced-Pinnacles-CE1 — probe output
pass kernel
5.14.0-687.24.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/localscratch/osg01/glide_innouR/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/localscratch/osg01/glide_innouR/execute/dir_3016311/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3016311 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/localscratch/osg01/glide_innouR/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/localscratch/osg01/glide_innouR/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UCMerced-Pinnacles-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=696038 GLIDEIN_Entry_Name=OSG_US_UCMerced-Pinnacles-CE1 GLIDEIN_Entry_Signature=358ba5c80e0fd1b6a1d6964a5d964c02bca5c0e6 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=ucmerced-pinnacles-ce1.svc.opensciencegrid.org ucmerced-pinnacles-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_suISj5 GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=4 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UCMerced-Pinnacles-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=UCMerced-Pinnacles GLIDEIN_Tmp_Dir=/localscratch/osg01/glide_innouR/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/localscratch/osg01/glide_innouR/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/localscratch/osg01/glide_innouR/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ucmerced-pinnacles-ce1.svc.opensciencegrid.org OSG_SITE_NAME=UCMerced-Pinnacles-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_4@glidein_2751820_205938330@node080.cluster$ "; unset PROMPT_COMMAND PS1=slot1_4@glidein_2751820_205938330@node080.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/localscratch/osg01/glide_innouR/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/localscratch/osg01/glide_innouR/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/localscratch/osg01/glide_innouR/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/localscratch/osg01/glide_innouR/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/localscratch/osg01/glide_innouR/execute/dir_3016311/scratch _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
604.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme1n1p1 937233280 303156912 634076368 33% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: pinnacles.cluster 0.463ms 1: pinnacles.cluster 0.253ms 2: 169.236.74.1 0.959ms 3: 10.7.1.61 1.267ms 4: pa-core-0-2.ucmerced.edu 1.446ms 5: 10.253.1.2 2.978ms 6: sacr2-agg-01--ucm--10g--01.cenic.net 7.048ms asymm 11 7: hpr-emvl1-agg-01--sacr2-agg-01--100g.cenic.net 7.316ms asymm 10 8: hpr-svl-agg10--emvl1-agg-01--100g.cenic.net 6.987ms asymm 9 9: hpr-internet2r&e--svl-agg10-100ge.cenic.net 10.676ms asymm 10 10: fourhundredge-0-0-0-23.4079.core2.sunn.net.internet2.edu 60.112ms asymm 22 11: fourhundredge-0-0-0-1.4079.core2.sacr.net.internet2.edu 60.681ms asymm 21 12: fourhundredge-0-0-0-1.4079.core2.salt.net.internet2.edu 58.583ms asymm 20 13: fourhundredge-0-0-0-0.4079.core2.denv.net.internet2.edu 60.302ms asymm 19 14: fourhundredge-0-0-0-0.4079.core2.kans.net.internet2.edu 58.440ms asymm 18 15: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 57.680ms asymm 17 16: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 60.056ms 17: 144.92.233.112 61.453ms 18: perfsonar02.hep.wisc.edu 61.244ms reached Resume: pmtu 1500 hops 18 back 19
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 100567)
UCR-HPCC-CE1 4 hours ago 1
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UCR-HPCC-CE1 — probe output
pass kernel
4.18.0-553.144.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_sFrAIm/images/htc__rocky__8.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__8.sif APPTAINER_TMPDIR=/scratch/glide_sFrAIm/execute/dir_1532698/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=1532698 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/glide_sFrAIm/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_sFrAIm/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UCR-HPCC-CE1.idtoken.idtoken GLIDEIN_CPUS=10 GLIDEIN_ClusterId=563818 GLIDEIN_Entry_Name=OSG_US_UCR-HPCC-CE1 GLIDEIN_Entry_Signature=0e4d42842aca1650a2c38652f0f820372650f429 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=ucr-hpcc-ce1.svc.osg-htc.org ucr-hpcc-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_wQbWkQ GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UCR-HPCC-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=UCR-HPCC GLIDEIN_Tmp_Dir=/scratch/glide_sFrAIm/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/glide_sFrAIm/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/glide_sFrAIm/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=ucr-hpcc-ce1.svc.osg-htc.org OSG_SITE_NAME=UCR-HPCC-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_1@glidein_1470011_431239282@c05.ib.hpcc.ucr.edu$ "; unset PROMPT_COMMAND PS1=slot1_1@glidein_1470011_431239282@c05.ib.hpcc.ucr.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_sFrAIm/images/htc__rocky__8.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/glide_sFrAIm/images/htc__rocky__8.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__8.sif SINGULARITY_PATH=/scratch/glide_sFrAIm/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_sFrAIm/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_sFrAIm/execute/dir_1532698/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
439.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda6 465561308 4436712 461124596 1% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: nat-clust.clust.hpcc.ucr.edu 0.126ms 1: nat-clust.clust.hpcc.ucr.edu 0.109ms 2: d-genom-1120d-1-v31.ucr.edu 0.740ms 3: 172.23.254.250 1.358ms 4: 172.30.254.9 1.006ms 5: 172.30.254.8 1.093ms 6: b-som-bn-1-v2020.ucr.edu 1.942ms asymm 7 7: 137.164.26.128 12.835ms asymm 13 8: hpr-i2--lax-agg10.cenic.net 15.958ms asymm 14 9: fourhundredge-0-0-0-2.4079.core2.salt.net.internet2.edu 59.657ms asymm 23 10: fourhundredge-0-0-0-0.4079.core2.denv.net.internet2.edu 59.378ms asymm 22 11: fourhundredge-0-0-0-0.4079.core2.kans.net.internet2.edu 60.443ms asymm 21 12: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 60.010ms asymm 20 13: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 60.911ms asymm 19 14: 144.92.233.112 61.949ms asymm 20 15: perfsonar02.hep.wisc.edu 62.205ms reached Resume: pmtu 1500 hops 15 back 22
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 4542)
UChicago_Pile_Backfill 4 hours ago 14
100.0%
100.0%
64.3%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UChicago_Pile_Backfill — probe output
pass kernel
4.18.0-553.144.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-FYgE7w/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-FYgE7w/execute/dir_146174/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=146174 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=UChicago_Pile OSG_SQUID_LOCATION=head04.pile.uchicago.edu:30244 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_16@UChicago-Pile-Backfill.c014.pile.uchicago.edu$ "; unset PROMPT_COMMAND PS1=slot1_16@UChicago-Pile-Backfill.c014.pile.uchicago.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-FYgE7w/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-FYgE7w/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-FYgE7w/execute/dir_146174/scratch _CONDOR_SLOT=slot1_16 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=head04.pile.uchicago.edu:30244
pass df
3237.6 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 3841429688 251322164 3394898872 7% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: no reply 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 9000 Resume: pmtu 9000
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy head04.pile.uchicago.edu:30244
pass user
nobody (uid 65534)
UConn-OSG 4 hours ago 88
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UConn-OSG — probe output
pass kernel
4.18.0-553.144.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/local/execute/dir_58746/glide_gta5tX/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/local/execute/dir_58746/glide_gta5tX/execute/dir_4117962/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=4117962 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/local/execute/dir_58746/glide_gta5tX/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UConn_gluskap_op.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=6058576 GLIDEIN_Country=US GLIDEIN_DUNESite=US_UConn GLIDEIN_Entry_Name=OSG_US_UConn_gluskap_op GLIDEIN_Entry_Signature=e18b4366ac8a1e6e1a594bc924151b87ee76b6c9 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=gluskap.phys.uconn.edu gluskap.phys.uconn.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_prod04_7HqOtG GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=2 GLIDEIN_Proxy_URL=gryphn.phys.uconn.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UConn-OSG GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=UConn GLIDEIN_Tmp_Dir=/local/execute/dir_58746/glide_gta5tX/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=gluskap.phys.uconn.edu OSG_SITE_NAME=UConn-OSG_CE OSG_SQUID_LOCATION=gryphn.phys.uconn.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_8@glidein_58748_324218994@nod55.phys.uconn.edu$ "; unset PROMPT_COMMAND PS1=slot1_8@glidein_58748_324218994@nod55.phys.uconn.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/local/execute/dir_58746/glide_gta5tX/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/local/execute/dir_58746/glide_gta5tX/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/local/execute/dir_58746/glide_gta5tX/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/local/execute/dir_58746/glide_gta5tX/execute/dir_4117962/scratch _CONDOR_SLOT=slot1_8 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
830.9 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 937231232 65926544 871304688 8% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.340ms 1: _gateway 0.439ms 2: 137.99.254.89 0.453ms 3: mx-msb-core-ae39.net.uconn.edu 0.367ms 4: vss-core-border-inside-po52-640.net.uconn.edu 1.153ms 5: fw-msb-i-lag1.666.net.uconn.edu 8.609ms 6: vrf-border-outside-po21-664.net.uconn.edu 9.379ms 7: mx-hbl-border-outside-ae51-628.net.uconn.edu 8.899ms 8: rt1-064hhh-57b16d-be-706-634.net.cen.ct.gov 9.779ms 9: 10.172.96.168 10.052ms 10: bundle-ether2.101.core1.hart2.net.internet2.edu 13.255ms 11: fourhundredge-0-0-0-0.4079.core1.alba.net.internet2.edu 32.282ms asymm 17 12: fourhundredge-0-0-0-2.4079.core2.clev.net.internet2.edu 32.536ms asymm 16 13: fourhundredge-0-0-0-2.4079.core2.eqch.net.internet2.edu 32.519ms asymm 15 14: fourhundredge-0-0-0-10.4079.core1.star.net.internet2.edu 33.852ms 15: 144.92.233.112 34.875ms 16: perfsonar02.hep.wisc.edu 34.793ms reached Resume: pmtu 1500 hops 16 back 17
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy gryphn.phys.uconn.edu:3128
pass user
prod04 (uid 7904)
UIUC-TGI-RAILS-CE1 4 hours ago 20
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UIUC-TGI-RAILS-CE1 — probe output
pass kernel
4.18.0-553.105.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/projects/bcxl/svcosguser/glide_s7pBG1/images/htc__rocky__9-cuda-12.6.0.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9-cuda-12.6.0.sif APPTAINER_TMPDIR=/projects/bcxl/svcosguser/glide_s7pBG1/execute/dir_3346150/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3346150 CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-2.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.4.5-2.el8 CUBACORES=1 CUDA_VERSION=12.6.0 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=gpu GLIDECLIENT_Group_Signature=110bdb2d12be2d6d01c3e4f48d3c6823b1a1f1be GLIDECLIENT_Name=OSPool.gpu GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/projects/bcxl/svcosguser/glide_s7pBG1/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UIUC-TGI-RAILS-CE1_gpu.idtoken.idtoken GLIDEIN_CPUS=24 GLIDEIN_ClusterId=696468 GLIDEIN_Entry_Name=OSG_US_UIUC-TGI-RAILS-CE1_gpu GLIDEIN_Entry_Signature=50a2290e813f4853df9c286fc704daaf3e969227 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_GPUS=1 GLIDEIN_Gatekeeper=uiuc-tgi-rails-ce1.svc.osg-htc.org uiuc-tgi-rails-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_svcosguser_6uaAM4 GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UIUC-TGI-RAILS-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=UIUC-TGI-RAILS GLIDEIN_Tmp_Dir=/projects/bcxl/svcosguser/glide_s7pBG1/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.4.5-2.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.4.5-2.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.4.5-2.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/.singularity.d/libs LIBRARY_PATH=/usr/local/cuda/lib64/stubs MKL_NUM_THREADS=1 NCCL_VERSION=2.22.3 NUMEXPR_NUM_THREADS=1 NVARCH=x86_64 NVIDIA_DRIVER_CAPABILITIES=compute,utility NVIDIA_PRODUCT_NAME=CUDA NVIDIA_REQUIRE_CUDA=cuda>=12.6 brand=unknown,driver>=470,driver<471 brand=grid,driver>=470,driver<471 brand=tesla,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=vapps,driver>=470,driver<471 brand=vpc,driver>=470,driver<471 brand=vcs,driver>=470,driver<471 brand=vws,driver>=470,driver<471 brand=cloudgaming,driver>=470,driver<471 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 NVIDIA_VISIBLE_DEVICES=none NV_CUDA_CUDART_DEV_VERSION=12.6.37-1 NV_CUDA_CUDART_VERSION=12.6.37-1 NV_CUDA_LIB_VERSION=12.6.0-1 NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-6-12.6.0-1 NV_CUDA_NSIGHT_COMPUTE_VERSION=12.6.0-1 NV_CUDNN_PACKAGE=libcudnn9-cuda-12-9.3.0.75-1 NV_CUDNN_PACKAGE_DEV=libcudnn9-devel-cuda-12-9.3.0.75-1 NV_CUDNN_VERSION=9.3.0.75-1 NV_LIBCUBLAS_DEV_VERSION=12.6.0.22-1 NV_LIBCUBLAS_VERSION=12.6.0.22-1 NV_LIBNCCL_DEV_PACKAGE=libnccl-devel-2.22.3-1+cuda12.6 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-devel NV_LIBNCCL_DEV_PACKAGE_VERSION=2.22.3-1 NV_LIBNCCL_PACKAGE=libnccl-2.22.3-1+cuda12.6 NV_LIBNCCL_PACKAGE_NAME=libnccl NV_LIBNCCL_PACKAGE_VERSION=2.22.3-1 NV_LIBNCCL_VERSION=2.22.3 NV_LIBNPP_DEV_PACKAGE=libnpp-devel-12-6-12.3.1.23-1 NV_LIBNPP_DEV_VERSION=12.3.1.23-1 NV_LIBNPP_PACKAGE=libnpp-12-6-12.3.1.23-1 NV_LIBNPP_VERSION=12.3.1.23-1 NV_NVML_DEV_VERSION=12.6.37-1 NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-6-12.6.37-1 NV_NVPROF_VERSION=12.6.37-1 NV_NVTX_VERSION=12.6.37-1 OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=uiuc-tgi-rails-ce1.svc.osg-htc.org OSG_SITE_NAME=UIUC-TGI-RAILS-CE1 PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_8@glidein_3103123_46364140@rails10$ "; unset PROMPT_COMMAND PS1=slot1_8@glidein_3103123_46364140@rails10$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/projects/bcxl/svcosguser/glide_s7pBG1/images/htc__rocky__9-cuda-12.6.0.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.4.5-2.el8 SINGULARITY_IMAGES_DICT=default:/projects/bcxl/svcosguser/glide_s7pBG1/images/htc__rocky__9-cuda-12.6.0.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9-cuda-12.6.0.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.4.5-2.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/projects/bcxl/svcosguser/glide_s7pBG1/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/projects/bcxl/svcosguser/glide_s7pBG1/execute/dir_3346150/scratch _CONDOR_SLOT=slot1_8 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
74825.1 GB available Filesystem 1024-blocks Used Available Capacity Mounted on pool1.railsvast.internal.ncsa.edu:/projects/bcxl/svcosguser/glide_s7pBG1/execute/dir_3346150 80530274304 2070462464 78459811840 3% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: tgi-rails-pub-gw.gw.ncsa.edu 0.244ms 1: tgi-rails-pub-gw.gw.ncsa.edu 0.226ms 2: npcf-core-east-tgi-rails-data.gw.ncsa.edu 0.256ms 3: npcf-exit-east-npcf-core-east.gw.ncsa.edu 0.379ms 4: no reply 5: 72.36.80.50 20.964ms 6: perfsonar02.hep.wisc.edu 20.218ms reached Resume: pmtu 9000 hops 6 back 6
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
svcosguser (uid 72387)
UND-Talon-CE1 3 hours ago 36
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UND-Talon-CE1 — probe output
pass kernel
5.14.0-427.42.1.el9_4.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_lMdhSz/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_lMdhSz/execute/dir_2598278/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2598278 CONTAINERSW_FULL_VERSION=apptainer version 1.1.9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/cm/local/apps/apptainer/current/bin/apptainer CONTAINERSW_VERSION=1.1.9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_lMdhSz/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UND-Talon-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=663239 GLIDEIN_Entry_Name=OSG_US_UND-Talon-CE1 GLIDEIN_Entry_Signature=9c6193728dfd0568915894e38c2e11469ed41570 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=und-talon-ce1.svc.opensciencegrid.org und-talon-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_glEtAW GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UND-Talon-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=UND-Talon GLIDEIN_Tmp_Dir=/tmp/glide_lMdhSz/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.1.9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/cm/local/apps/apptainer/current/bin/apptainer GWMS_CONTAINERSW_VERSION=1.1.9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/cm/local/apps/apptainer/current/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.1.9 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=und-talon-ce1.svc.opensciencegrid.org OSG_SITE_NAME=UND-Talon-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_180926_608567910@talon08.cm.cluster$ "; unset PROMPT_COMMAND PS1=slot1@glidein_180926_608567910@talon08.cm.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_lMdhSz/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.1.9 SINGULARITY_IMAGES_DICT=default:/tmp/glide_lMdhSz/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/cm/local/apps/apptainer/current/bin/apptainer SINGULARITY_VERSION=1.1.9 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_lMdhSz/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_lMdhSz/execute/dir_2598278/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
766.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 920325060 117111996 803213064 13% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.128.128.1 7.987ms 1: 10.128.128.1 7.845ms 2: 172.29.10.26 8.306ms 3: 172.29.10.17 7.991ms asymm 4 4: 10.228.6.253 0.903ms asymm 5 5: w.av25.3/22/1.ndus.edu 9.264ms asymm 6 6: 165.234.8.181 15.630ms 7: 134.129.0.37 16.242ms 8: 165.234.21.1 26.841ms asymm 9 9: telecomb-gr-01-hu0-5-0-3.northernlights.gigapop.net 34.512ms asymm 10 10: r-minneapolis-511-ae0-110.uwsys.net 34.161ms asymm 11 11: r-uweauclaire-centennial-ae0-3470.uwsys.net 36.707ms asymm 17 12: r-uwmadison-animal-ae3-3507.uwsys.net 38.350ms asymm 13 13: r-uwmadison-cssc-ae3-3439.uwsys.net 45.752ms asymm 14 14: 143.235.41.129 39.857ms asymm 15 15: perfsonar02.hep.wisc.edu 37.692ms reached Resume: pmtu 1500 hops 15 back 15
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 1454600096)
UNL-PATH-EP 5 hours ago 24
100.0%
100.0%
100.0%
100.0%
all
UNL-PATH-EP — probe output
warning all
Job failed with exit code 0; no results transferred
USC-CARC-Artemis-Backfill 4 hours ago 35
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
USC-CARC-Artemis-Backfill — probe output
pass kernel
4.18.0-372.16.1.el8_6.0.1.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-SANEJw/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-SANEJw/execute/dir_55220/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=55220 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=Center for Advanced Research Computing OSG_SQUID_LOCATION=68.181.11.8:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_3@USC-CARC-Artemis-Backfill.f016358ace78$ "; unset PROMPT_COMMAND PS1=slot1_3@USC-CARC-Artemis-Backfill.f016358ace78$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/pilot/osgvo-pilot-SANEJw/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-SANEJw/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-SANEJw/execute/dir_55220/scratch _CONDOR_SLOT=slot1_3 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=68.181.11.8:3128
warning df
4.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on tmpfs 65980400 61494820 4485580 94% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: host.containers.internal 0.051ms 1: host.containers.internal 0.059ms 2: artemis-osg 13.797ms 3: 68.181.11.126 0.454ms 4: 68.181.25.17 8.446ms 5: 10.250.9.113 8.213ms 6: 207.151.53.251 8.621ms 7: hpr--lax-agg10--csuf--10g--01.cenic.net 8.912ms 8: hpr-i2--lax-agg10.cenic.net 27.819ms 9: fourhundredge-0-0-0-2.4079.core2.salt.net.internet2.edu 55.941ms asymm 17 10: fourhundredge-0-0-0-0.4079.core2.denv.net.internet2.edu 58.957ms asymm 16 11: fourhundredge-0-0-0-0.4079.core2.kans.net.internet2.edu 63.965ms asymm 15 12: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 55.972ms asymm 14 13: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 56.289ms 14: 144.92.233.112 57.463ms 15: perfsonar02.hep.wisc.edu 57.463ms reached Resume: pmtu 1500 hops 15 back 16
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy 68.181.11.8:3128
fail user
uid 1003 does not map to a user
USCMS-FNAL-WC1 4 hours ago 1
0.0%
0.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
USCMS-FNAL-WC1 — probe output
pass kernel
5.14.0-687.26.1.el9_8.x86_64
pass env
APPTAINER_CACHEDIR=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch BATCH_SYSTEM=HTCondor CONDORCE_COLLECTOR_HOST=cmsosgce-opp1.fnal.gov:9619 CONDOR_CONFIG=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/condor_config CONDOR_PARENT_ID=osg-59472-0-cmswn2426:32618:1785588765 CONDOR_PROCD_ADDRESS=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/log/procd_address CONDOR_PROCD_ADDRESS_BASE=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/log/procd_address CONDOR_STARTER_PID=32618 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 FERMIHTC_APPTAINER_ARGS=--pid --ipc --contain FERMIHTC_APPTAINER_BIN=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/current/bin/apptainer FERMIHTC_APPTAINER_BIND=/cvmfs /etc/hosts /etc/grid-security FERMIHTC_APPTAINER_CMD=exec FERMIHTC_APPTAINER_PWD=/srv GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.CMSHTPC_T1_US_FNAL_condce_opp1_whole.idtoken.idtoken GLIDEIN_CPUS=128 GLIDEIN_ClusterId=663126 GLIDEIN_Country=US GLIDEIN_DEBUG_OPTIONS= GLIDEIN_DUNESite=US_FNAL-T1 GLIDEIN_ESTIMATED_CPUS=64 GLIDEIN_Entry_Name=CMSHTPC_T1_US_FNAL_condce_opp1_whole GLIDEIN_Entry_Signature=25ec5309a48a9bb5a40702a0c7160eb9ffc90534 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=cmsosgce-opp1.fnal.gov cmsosgce-opp1.fnal.gov:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_MQ0bmi GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_ProcId=4 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=USCMS-FNAL-WC1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=FNAL GLIDEIN_Tmp_Dir=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/tmp GLIDEIN_UUID=404005d7-69b8-44b7-ba1f-5e1b4f2fc260 GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GSI_AUTHZ_CONF=/dev/null GWMS_PATH=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/.gwms.d/bin: GWMS_SINGULARITY_STATUS=PREFERRED HAS_SINGULARITY=False HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch HOSTNAME=osg-59472-0-cmswn2426.fnal.gov JOB_TOKENS=/var/lib/gwms-factory/server-credentials/entry_CMSHTPC_T1_US_FNAL_condce_opp1_whole/tokens.tgz JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OLDPWD=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/logs/404005d7-69b8-44b7-ba1f-5e1b4f2fc260 OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_APP=/cvmfs/oasis.opensciencegrid.org OSG_DATA=UNAVAILABLE OSG_GRID=/etc/osg/wn-client/ OSG_HOSTNAME=cmsosgce-opp1.fnal.gov OSG_SITE_NAME=USCMS-FNAL-WC1-OPP1 OSG_SITE_READ= OSG_SITE_WRITE= OSG_SQUID_LOCATION=squid.fnal.gov:3128 OSG_WN_TMP=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch/.local-tmp PATH=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/.gwms.d/bin::/bin:/usr/bin:/sbin:/usr/sbin PREEMPT_GRACE_TIME=10000000 PWD=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SHLVL=0 SINGULARITY_CACHEDIR=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch SLOTS_LAYOUT=partitionable TEMP=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch TMPDIR=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch/.local-tmp TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM TZ=:/etc/localtime _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_ANCESTOR_29107=29110:1785588287:4021522845 _CONDOR_ANCESTOR_29110=32618:1785588764:3263430595 _CONDOR_ANCESTOR_32618=32987:1785588766:99777211 _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/main/condor/bin _CONDOR_CHIRP_CONFIG=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/user/.chirp.config _CONDOR_CREDS=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch/.condor_creds _CONDOR_EXECUTE=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute _CONDOR_JOB_AD=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch/.job.ad _CONDOR_JOB_IWD=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch _CONDOR_SLOT=slot1_1 _CONDOR_WRAPPER_ERROR_FILE=/storage/local/data1/condor/execute/dir_3515990/glide_GTrTlT/execute/dir_32618/scratch/.job_wrapper_failure _JAVA_OPTIONS=-Xmx1024m
pass df
6944.6 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vgData-lv_data1 7751275948 78573864 7281984236 2% /storage/local/data1/condor/execute/dir_3515990
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] 0.008ms pmtu 9000 1: r-cms-fcc2-2-vlan191.fnal.gov 1.338ms 1: r-cms-fcc2-2-vlan191.fnal.gov 1.222ms 2: r-border-fcc2-1-2513-1.fnal.gov 0.290ms 3: no reply 4: fnalgcc-cr6--chic-bb-e.igp.es.net 1.969ms asymm 8 5: chic-cr6--star-bb-e.igp.es.net 1.857ms asymm 7 6: no reply 7: site--uwmadison-se306.lhcone.es.net 5.817ms 8: perfsonar02.hep.wisc.edu 5.684ms reached Resume: pmtu 9000 hops 8 back 9
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy squid.fnal.gov:3128
pass user
osg (uid 12740)
USD-Lawrence-CE1 4 hours ago 43
100.0%
0.0%
0.0%
0.0%
kernel env df firewall mtu nrp_images http_proxy user
USD-Lawrence-CE1 — probe output
pass kernel
4.18.0-372.26.1.el8_6.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/job_1239903/glide_JYdamr/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/job_1239903/glide_JYdamr/execute/dir_3957445/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3957445 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/job_1239903/glide_JYdamr/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/job_1239903/glide_JYdamr/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_USD-Lawrence-CE1.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=761681 GLIDEIN_Entry_Name=OSG_US_USD-Lawrence-CE1 GLIDEIN_Entry_Signature=95c044527f102727b9d521196615b08f8076b684 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=usd-lawrence-ce1.svc.opensciencegrid.org usd-lawrence-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01@usd.local_o4pK5Z GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=4 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=USD-Lawrence-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=USD-Lawrence GLIDEIN_Tmp_Dir=/scratch/job_1239903/glide_JYdamr/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/job_1239903/glide_JYdamr/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/job_1239903/glide_JYdamr/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=usd-lawrence-ce1.svc.opensciencegrid.org OSG_SITE_NAME=USD-Lawrence-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_6@glidein_3740893_230357543@node072.lawrence$ "; unset PROMPT_COMMAND PS1=slot1_6@glidein_3740893_230357543@node072.lawrence$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/job_1239903/glide_JYdamr/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/job_1239903/glide_JYdamr/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/scratch/job_1239903/glide_JYdamr/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/job_1239903/glide_JYdamr/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/job_1239903/glide_JYdamr/execute/dir_3957445/scratch _CONDOR_SLOT=slot1_6 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
186.2 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda6 207528956 12268696 195260260 6% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: login01.lawrence 1.842ms 1: login01.lawrence 1.601ms 2: 206.209.0.250 2.868ms 3: 206.176.0.181 3.686ms 4: 206.71.32.5 2.305ms 5: 206.71.32.241 2.749ms 6: 206.71.32.162 13.592ms 7: fourhundredge-0-0-0-0.4079.core1.chic.net.internet2.edu 22.507ms asymm 9 8: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 25.479ms 9: 144.92.233.112 27.235ms 10: perfsonar02.hep.wisc.edu 24.401ms reached Resume: pmtu 1500 hops 10 back 13
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01@usd.local (uid 735875764)
UTC-Epyc 4 hours ago 6
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UTC-Epyc — probe output
pass kernel
5.14.0-687.5.4.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/mnt/internal/osg01/glide_X4s6TG/images/htc__rocky__9-cuda-12.6.0.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9-cuda-12.6.0.sif APPTAINER_TMPDIR=/mnt/internal/osg01/glide_X4s6TG/execute/dir_2818965/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2818965 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/mnt/internal/osg01/glide_X4s6TG/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VERSION=12.6.0 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=gpu GLIDECLIENT_Group_Signature=110bdb2d12be2d6d01c3e4f48d3c6823b1a1f1be GLIDECLIENT_Name=OSPool.gpu GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/mnt/internal/osg01/glide_X4s6TG/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UTC-Epyc.idtoken.idtoken GLIDEIN_CPUS=64 GLIDEIN_ClusterId=693143 GLIDEIN_Entry_Name=OSG_US_UTC-Epyc GLIDEIN_Entry_Signature=5f341e23b7e6d97f1a16a622440483dbdee2cbd8 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_GPUS=1 GLIDEIN_Gatekeeper=utc-epyc-ce1.svc.opensciencegrid.org utc-epyc-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_xOOFxX GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UTC-Epyc GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=UTC-Epyc GLIDEIN_Tmp_Dir=/mnt/internal/osg01/glide_X4s6TG/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/mnt/internal/osg01/glide_X4s6TG/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/mnt/internal/osg01/glide_X4s6TG/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/.singularity.d/libs LIBRARY_PATH=/usr/local/cuda/lib64/stubs MKL_NUM_THREADS=1 NCCL_VERSION=2.22.3 NUMEXPR_NUM_THREADS=1 NVARCH=x86_64 NVIDIA_DRIVER_CAPABILITIES=compute,utility NVIDIA_PRODUCT_NAME=CUDA NVIDIA_REQUIRE_CUDA=cuda>=12.6 brand=unknown,driver>=470,driver<471 brand=grid,driver>=470,driver<471 brand=tesla,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=vapps,driver>=470,driver<471 brand=vpc,driver>=470,driver<471 brand=vcs,driver>=470,driver<471 brand=vws,driver>=470,driver<471 brand=cloudgaming,driver>=470,driver<471 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 NVIDIA_VISIBLE_DEVICES=none NV_CUDA_CUDART_DEV_VERSION=12.6.37-1 NV_CUDA_CUDART_VERSION=12.6.37-1 NV_CUDA_LIB_VERSION=12.6.0-1 NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-6-12.6.0-1 NV_CUDA_NSIGHT_COMPUTE_VERSION=12.6.0-1 NV_CUDNN_PACKAGE=libcudnn9-cuda-12-9.3.0.75-1 NV_CUDNN_PACKAGE_DEV=libcudnn9-devel-cuda-12-9.3.0.75-1 NV_CUDNN_VERSION=9.3.0.75-1 NV_LIBCUBLAS_DEV_VERSION=12.6.0.22-1 NV_LIBCUBLAS_VERSION=12.6.0.22-1 NV_LIBNCCL_DEV_PACKAGE=libnccl-devel-2.22.3-1+cuda12.6 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-devel NV_LIBNCCL_DEV_PACKAGE_VERSION=2.22.3-1 NV_LIBNCCL_PACKAGE=libnccl-2.22.3-1+cuda12.6 NV_LIBNCCL_PACKAGE_NAME=libnccl NV_LIBNCCL_PACKAGE_VERSION=2.22.3-1 NV_LIBNCCL_VERSION=2.22.3 NV_LIBNPP_DEV_PACKAGE=libnpp-devel-12-6-12.3.1.23-1 NV_LIBNPP_DEV_VERSION=12.3.1.23-1 NV_LIBNPP_PACKAGE=libnpp-12-6-12.3.1.23-1 NV_LIBNPP_VERSION=12.3.1.23-1 NV_NVML_DEV_VERSION=12.6.37-1 NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-6-12.6.37-1 NV_NVPROF_VERSION=12.6.37-1 NV_NVTX_VERSION=12.6.37-1 OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=utc-epyc-ce1.svc.opensciencegrid.org OSG_SITE_NAME=UTC-Epyc PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_43@glidein_2546327_386162231@epyc11$ "; unset PROMPT_COMMAND PS1=slot1_43@glidein_2546327_386162231@epyc11$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/mnt/internal/osg01/glide_X4s6TG/images/htc__rocky__9-cuda-12.6.0.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/mnt/internal/osg01/glide_X4s6TG/images/htc__rocky__9-cuda-12.6.0.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9-cuda-12.6.0.sif SINGULARITY_PATH=/mnt/internal/osg01/glide_X4s6TG/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/mnt/internal/osg01/glide_X4s6TG/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/mnt/internal/osg01/glide_X4s6TG/execute/dir_2818965/scratch _CONDOR_SLOT=slot1_43 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
480.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p1 976284608 472140652 504143956 49% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.759ms 1: _gateway 0.590ms 2: 10.249.200.157 8.072ms 3: 192.30.208.38 8.251ms 4: no reply 5: no reply 6: no reply 7: perfsonar02.hep.wisc.edu 38.112ms reached Resume: pmtu 1500 hops 7 back 16
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 300001)
UW-IT-CE1 4 hours ago 2
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UW-IT-CE1 — probe output
pass kernel
4.18.0-553.126.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_iuXYFd/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_iuXYFd/execute/dir_84626/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=84626 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_iuXYFd/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UW-IT-CE1.idtoken.idtoken GLIDEIN_CPUS=40 GLIDEIN_ClusterId=6505945 GLIDEIN_Entry_Name=OSG_US_UW-IT-CE1 GLIDEIN_Entry_Signature=f4073561b0f22ba9fcd96332ec6efdedba8ae887 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=uw-it-ce1.svc.osg-htc.org uw-it-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_oscigrid_OFSIDL GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UW-IT-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=UW-IT GLIDEIN_Tmp_Dir=/tmp/glide_iuXYFd/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=uw-it-ce1.svc.osg-htc.org OSG_SITE_NAME=UW-IT-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_4@glidein_3425_286617056@g3037.hyak.local$ "; unset PROMPT_COMMAND PS1=slot1_4@glidein_3425_286617056@g3037.hyak.local$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_iuXYFd/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/tmp/glide_iuXYFd/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_iuXYFd/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_iuXYFd/execute/dir_84626/scratch _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure http_proxy=http://klone-dip1:3128 no_proxy=localhost,hyak.local,hyakm.washington.edu
pass df
286.6 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 374896792 74330348 300566444 20% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: klone-dip1.hyak.local 0.248ms 1: klone-dip1.hyak.local 0.204ms 2: ae3--20.uwar-uwtc-1.infra.washington.edu 0.529ms 3: lo0--20.uwcr-ads-2.infra.washington.edu 0.721ms 4: ae4--308.uwcr-ads-2.infra.washington.edu 0.559ms 5: no reply 6: ae16--500.icar-sttl2-2.infra.pnw-gigapop.net 0.645ms 7: lo0--210.icar-sttl1-1.infra.pnw-gigapop.net 0.896ms 8: 209.124.181.167 2.960ms 9: fourhundredge-0-0-0-17.4079.core1.seat.net.internet2.edu 52.693ms asymm 19 10: fourhundredge-0-0-0-0.4079.core1.salt.net.internet2.edu 52.286ms asymm 18 11: fourhundredge-0-0-0-0.4079.core1.denv.net.internet2.edu 51.923ms asymm 17 12: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 50.641ms asymm 16 13: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 51.376ms asymm 15 14: fourhundredge-0-0-0-23.4079.core1.chic.net.internet2.edu 50.342ms 15: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 51.437ms asymm 13 16: 144.92.233.112 53.238ms asymm 14 17: perfsonar02.hep.wisc.edu 53.306ms reached Resume: pmtu 1500 hops 17 back 16
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy http://klone-dip1:3128
pass user
oscigrid (uid 1255596)
UWEC-BOSE-CE1 4 hours ago 4
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UWEC-BOSE-CE1 — probe output
pass kernel
4.18.0-348.el8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/local/scratch/job_388949/glide_aL6SjC/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/local/scratch/job_388949/glide_aL6SjC/execute/dir_268681/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=268681 CONTAINERSW_FULL_VERSION=apptainer version 1.2.5 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/data/software/default/apptainer/1.2.5/bin/apptainer CONTAINERSW_VERSION=1.2.5 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/local/scratch/job_388949/glide_aL6SjC/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UWEC-BOSE-CE1.idtoken.idtoken GLIDEIN_CONTAINER_ENV=keepall GLIDEIN_CPUS=16 GLIDEIN_ClusterId=6061629 GLIDEIN_Entry_Name=OSG_US_UWEC-BOSE-CE1 GLIDEIN_Entry_Signature=e70ec0b69b831c029ef16844f638da691828d64e GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=uwec-bose-ce1.svc.osg-htc.org uwec-bose-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_kdZbaz GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UWEC-BOSE-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=UWEC-BOSE GLIDEIN_Tmp_Dir=/local/scratch/job_388949/glide_aL6SjC/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.2.5 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/data/software/default/apptainer/1.2.5/bin/apptainer GWMS_CONTAINERSW_VERSION=1.2.5 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/data/software/default/apptainer/1.2.5/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.2.5 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=uwec-bose-ce1.svc.osg-htc.org OSG_SITE_NAME=UWEC-BOSE-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_7@glidein_238409_197422425@cn13.head.bose.hpc.cluster$ "; unset PROMPT_COMMAND PS1=slot1_7@glidein_238409_197422425@cn13.head.bose.hpc.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIN=/data/software/default/apptainer/1.2.5/bin SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/local/scratch/job_388949/glide_aL6SjC/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.2.5 SINGULARITY_IMAGES_DICT=default:/local/scratch/job_388949/glide_aL6SjC/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/data/software/default/apptainer/1.2.5/bin/apptainer SINGULARITY_VERSION=1.2.5 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/local/scratch/job_388949/glide_aL6SjC/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/local/scratch/job_388949/glide_aL6SjC/execute/dir_268681/scratch _CONDOR_SLOT=slot1_7 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
787.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda8 837052400 11058932 825993468 2% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 7.908ms 1: _gateway 7.757ms 2: r-uweauclaire-cvtc-ae2-0.uwsys.net 8.188ms 3: r-uweauclaire-centennial-et-0-1-9-3503.uwsys.net 12.123ms asymm 8 4: r-uwmadison-animal-ae3-3507.uwsys.net 4.487ms 5: r-uwmadison-cssc-ae3-3439.uwsys.net 23.885ms 6: 143.235.41.129 11.980ms 7: perfsonar02.hep.wisc.edu 11.854ms reached Resume: pmtu 1500 hops 7 back 6
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 2011)
UWL-Test-EP 4 hours ago 14
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UWL-Test-EP — probe output
pass kernel
5.14.0-687.29.1.el9_8.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/pilot/osgvo-pilot-HOIkom/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/pilot/osgvo-pilot-HOIkom/execute/dir_56326 BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=56326 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el9 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_REQUIRED_OS=any GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el9 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el9 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el9 HAS_SINGULARITY=True HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_SITE_NAME=UW-La Crosse CSCPE PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PROMPT_COMMAND=PS1="slot1_79@UWL-Test-EP.7b95c4940045$ "; unset PROMPT_COMMAND PS1=slot1_79@UWL-Test-EP.7b95c4940045$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv SINGULARITY_CONTAINER=/pilot/osgvo-pilot-HOIkom/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el9 SINGULARITY_IMAGES_DICT=default:/pilot/osgvo-pilot-HOIkom/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el9 TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/usr/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/pilot/osgvo-pilot-HOIkom/execute/dir_56326 _CONDOR_SLOT=slot1_79 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure
pass df
375.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p3 971579892 578316864 393263028 60% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: _gateway 0.059ms 1: _gateway 0.059ms 2: 138.49.30.1 1.805ms asymm 3 3: 10.159.96.1 0.635ms 4: 138.49.0.2 0.814ms asymm 5 5: r-uwlacrosse-wing-ae3-182.uwsys.net 0.736ms 6: r-uwlacrosse-wing-ae3-182.uwsys.net 7.994ms asymm 5 7: r-uwmadison-animal-et-0-1-4-3504.uwsys.net 3.751ms asymm 6 8: 143.235.41.129 3.668ms 9: perfsonar02.hep.wisc.edu 3.564ms reached Resume: pmtu 1500 hops 9 back 8
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
fail user
uid 1000 does not map to a user
UWM-Mortimer-CE1 4 hours ago 51
100.0%
100.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
UWM-Mortimer-CE1 — probe output
pass kernel
4.18.0-553.143.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/tmp/glide_6kCkfO/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/tmp/glide_6kCkfO/execute/dir_732614/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=732614 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=privileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/tmp/glide_6kCkfO/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_UWM-Mortimer-CE1.idtoken.idtoken GLIDEIN_CPUS=4 GLIDEIN_ClusterId=662569 GLIDEIN_Entry_Name=OSG_US_UWM-Mortimer-CE1 GLIDEIN_Entry_Signature=102380c29173c49dfadd1a064cb0df8f225f2e97 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=uwm-mortimer-ce1.svc.osg-htc.org uwm-mortimer-ce1.svc.osg-htc.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_40AObs GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/privileged GLIDEIN_ProcId=5 GLIDEIN_Proxy_URL=uwm-proxy.uwm.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=UWM-Mortimer-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=UWM-Mortimer GLIDEIN_Tmp_Dir=/tmp/glide_6kCkfO/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=privileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=privileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=uwm-mortimer-ce1.svc.osg-htc.org OSG_SITE_NAME=UWM-Mortimer-CE1 OSG_SQUID_LOCATION=uwm-proxy.uwm.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_4@glidein_3238198_1088430465@execute-141.mortimer.hpc.uwm.edu$ "; unset PROMPT_COMMAND PS1=slot1_4@glidein_3238198_1088430465@execute-141.mortimer.hpc.uwm.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/tmp/glide_6kCkfO/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/tmp/glide_6kCkfO/images/htc__rocky__9.sif SINGULARITY_MODE=privileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/tmp/glide_6kCkfO/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/tmp/glide_6kCkfO/execute/dir_732614/scratch _CONDOR_SLOT=slot1_4 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
332.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup-tmp 366991360 18076172 348915188 5% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=9000 1?: [LOCALHOST] pmtu 9000 1: _gateway 46.959ms 1: _gateway 47.499ms 2: no reply 3: r-uwmilwaukee-ems-ae9-160.uwsys.net 56.028ms 4: r-uwmadison-cssc-ae4-3455.uwsys.net 9.059ms 5: 143.235.41.129 48.522ms 6: perfsonar02.hep.wisc.edu 48.882ms reached Resume: pmtu 9000 hops 6 back 7
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy uwm-proxy.uwm.edu:3128
pass user
osg01 (uid 2606)
Utah-Granite-CE1 4 hours ago 98
100.0%
96.9%
0.0%
96.9%
kernel env df firewall mtu nrp_images http_proxy user
Utah-Granite-CE1 — probe output
pass kernel
4.18.0-553.144.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/local/osguserg/1680777/glide_BRkKa8/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/local/osguserg/1680777/glide_BRkKa8/execute/dir_3519621/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3519621 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/local/osguserg/1680777/glide_BRkKa8/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/local/osguserg/1680777/glide_BRkKa8/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Utah-Granite-CE1.idtoken.idtoken GLIDEIN_CPUS=16 GLIDEIN_ClusterId=6053160 GLIDEIN_Entry_Name=OSG_US_Utah-Granite-CE1 GLIDEIN_Entry_Signature=c6c631f84315aa50949180af8f18da6c0d0b7de3 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=utah-granite-ce1.svc.opensciencegrid.org utah-granite-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osguserg_IhTxNZ GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=1 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Utah-Granite-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins2@gfactory-1.osg-htc.org GLIDEIN_Signature=51c806521342f21128eec483be00d289d526592b GLIDEIN_Site=Utah-Granite GLIDEIN_Tmp_Dir=/scratch/local/osguserg/1680777/glide_BRkKa8/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/local/osguserg/1680777/glide_BRkKa8/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/local/osguserg/1680777/glide_BRkKa8/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=utah-granite-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Utah-Granite-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_16@glidein_2273260_383011264@grn047.int.chpc.utah.edu$ "; unset PROMPT_COMMAND PS1=slot1_16@glidein_2273260_383011264@grn047.int.chpc.utah.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/local/osguserg/1680777/glide_BRkKa8/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/local/osguserg/1680777/glide_BRkKa8/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/scratch/local/osguserg/1680777/glide_BRkKa8/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/local/osguserg/1680777/glide_BRkKa8/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/local/osguserg/1680777/glide_BRkKa8/execute/dir_3519621/scratch _CONDOR_SLOT=slot1_16 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
5390.0 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg_peaks_crypt-lv_scratch 6047787008 395942928 5651844080 7% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.242.160.2 0.082ms 1: 10.242.160.2 0.172ms 2: rt-inscc-ddc-core-101.chpc.utah.edu 1.219ms asymm 3 3: rt-park-inscc-ul.chpc.utah.edu 1.242ms asymm 4 4: 155.99.130.13 1.292ms asymm 5 5: 155.99.130.58 1.903ms asymm 6 6: 199.104.93.30 1.803ms 7: 199.104.93.21 2.538ms 8: 140.197.253.138 3.650ms 9: 140.197.248.40 3.010ms asymm 11 10: lv3-beibr-b-182-int.uen.net 2.376ms 11: fourhundredge-0-0-0-8.1013.core1.salt.net.internet2.edu 5.311ms 12: fourhundredge-0-0-0-0.4079.core1.denv.net.internet2.edu 38.087ms asymm 20 13: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 35.850ms asymm 19 14: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 37.316ms asymm 18 15: fourhundredge-0-0-0-21.4079.core1.chic.net.internet2.edu 36.683ms asymm 17 16: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 37.042ms 17: 144.92.233.112 39.247ms 18: perfsonar02.hep.wisc.edu 39.433ms reached Resume: pmtu 1500 hops 18 back 19
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osguserg (uid 67349)
Utah-Kingspeak-CE1 4 hours ago 88
100.0%
87.5%
0.0%
87.5%
kernel env df firewall mtu nrp_images http_proxy user
Utah-Kingspeak-CE1 — probe output
pass kernel
4.18.0-553.141.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/local/osguserk/26958304/glide_9mEkMA/images/htc__debian__13.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__debian__13.sif APPTAINER_TMPDIR=/scratch/local/osguserk/26958304/glide_9mEkMA/execute/dir_2031564/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2031564 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/local/osguserk/26958304/glide_9mEkMA/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/local/osguserk/26958304/glide_9mEkMA/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Utah-Kingspeak-CE1.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=695153 GLIDEIN_Entry_Name=OSG_US_Utah-Kingspeak-CE1 GLIDEIN_Entry_Signature=ecaded7ed097f4915f59b9ddc943fdfc58c99a35 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=utah-kingspeak-ce1.svc.opensciencegrid.org utah-kingspeak-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osguserk_76h4Zx GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=3 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Utah-Kingspeak-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins5@gfactory-2.osg-htc.org GLIDEIN_Signature=6123f2f772465d55c71e14646cc81ad4f2b1f29a GLIDEIN_Site=Utah-Kingspeak GLIDEIN_Tmp_Dir=/scratch/local/osguserk/26958304/glide_9mEkMA/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/local/osguserk/26958304/glide_9mEkMA/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/local/osguserk/26958304/glide_9mEkMA/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=utah-kingspeak-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Utah-Kingspeak-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_6@glidein_1207291_334853467@kp164.ipoib.kingspeak.peaks$ "; unset PROMPT_COMMAND PS1=slot1_6@glidein_1207291_334853467@kp164.ipoib.kingspeak.peaks$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/local/osguserk/26958304/glide_9mEkMA/images/htc__debian__13.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/local/osguserk/26958304/glide_9mEkMA/images/htc__debian__13.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__debian__13.sif SINGULARITY_PATH=/scratch/local/osguserk/26958304/glide_9mEkMA/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/local/osguserk/26958304/glide_9mEkMA/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/local/osguserk/26958304/glide_9mEkMA/execute/dir_2031564/scratch _CONDOR_SLOT=slot1_6 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
1754.6 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg_peaks_crypt-lv_scratch 1935770116 95890968 1839879148 5% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=2044 1?: [LOCALHOST] pmtu 2044 1: 10.242.154.2 0.115ms 1: 10.242.154.2 0.072ms 2: rt-inscc-ddc-core-101.chpc.utah.edu 1.423ms asymm 3 3: rt-park-inscc-ul.chpc.utah.edu 1.347ms asymm 4 4: 155.99.130.13 1.206ms asymm 5 5: 155.99.130.58 1.915ms asymm 6 6: 199.104.93.30 2.103ms 7: no reply 8: 140.197.253.138 3.735ms 9: 140.197.248.40 3.958ms asymm 11 10: lv3-beibr-b-182-int.uen.net 3.572ms 11: fourhundredge-0-0-0-8.1013.core1.salt.net.internet2.edu 5.369ms 12: fourhundredge-0-0-0-0.4079.core1.denv.net.internet2.edu 38.240ms asymm 20 13: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 38.361ms asymm 19 14: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 38.633ms asymm 18 15: fourhundredge-0-0-0-22.4079.core1.chic.net.internet2.edu 37.837ms asymm 17 16: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 37.202ms 17: 144.92.233.112 39.514ms 18: perfsonar02.hep.wisc.edu 40.059ms reached Resume: pmtu 2044 hops 18 back 19
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osguserk (uid 67347)
Utah-Lonepeak-CE1 4 hours ago 6
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Utah-Lonepeak-CE1 — probe output
pass kernel
4.18.0-553.144.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/local/osguserl/9381577/glide_3eoyxD/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/local/osguserl/9381577/glide_3eoyxD/execute/dir_3081724/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=3081724 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/local/osguserl/9381577/glide_3eoyxD/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/local/osguserl/9381577/glide_3eoyxD/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Utah-Lonepeak-CE1.idtoken.idtoken GLIDEIN_CPUS=8 GLIDEIN_ClusterId=6713118 GLIDEIN_Entry_Name=OSG_US_Utah-Lonepeak-CE1 GLIDEIN_Entry_Signature=f1f89e7407ee6a1816e04af543df5f673b303f81 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=utah-lonepeak-ce1.svc.opensciencegrid.org utah-lonepeak-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osguserl_bmWVpO GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Utah-Lonepeak-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Utah-Lonepeak GLIDEIN_Tmp_Dir=/scratch/local/osguserl/9381577/glide_3eoyxD/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/local/osguserl/9381577/glide_3eoyxD/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/local/osguserl/9381577/glide_3eoyxD/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=utah-lonepeak-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Utah-Lonepeak-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_8@glidein_2658930_45426656@lp099.lonepeak.peaks$ "; unset PROMPT_COMMAND PS1=slot1_8@glidein_2658930_45426656@lp099.lonepeak.peaks$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/local/osguserl/9381577/glide_3eoyxD/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/local/osguserl/9381577/glide_3eoyxD/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/scratch/local/osguserl/9381577/glide_3eoyxD/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/local/osguserl/9381577/glide_3eoyxD/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/local/osguserl/9381577/glide_3eoyxD/execute/dir_3081724/scratch _CONDOR_SLOT=slot1_8 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
439.7 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg_peaks_crypt-lv_scratch 518484480 57402784 461081696 12% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: gw-peaks-2.chpc.utah.edu 0.152ms 1: gw-peaks-2.chpc.utah.edu 0.196ms 2: rt-inscc-ddc-core-101.chpc.utah.edu 1.399ms asymm 3 3: rt-park-inscc-ul.chpc.utah.edu 1.231ms asymm 4 4: 155.99.130.13 0.773ms asymm 5 5: 155.99.130.58 1.466ms asymm 6 6: 199.104.93.30 1.366ms 7: no reply 8: 140.197.253.138 3.597ms 9: 140.197.248.40 3.730ms asymm 11 10: lv3-beibr-b-182-int.uen.net 2.562ms 11: fourhundredge-0-0-0-8.1013.core1.salt.net.internet2.edu 5.262ms 12: fourhundredge-0-0-0-0.4079.core1.denv.net.internet2.edu 37.589ms asymm 20 13: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 36.944ms asymm 19 14: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 38.312ms asymm 18 15: fourhundredge-0-0-0-23.4079.core1.chic.net.internet2.edu 36.666ms asymm 17 16: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 37.708ms 17: 144.92.233.112 39.297ms 18: perfsonar02.hep.wisc.edu 39.311ms reached Resume: pmtu 1500 hops 18 back 19
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osguserl (uid 67346)
Utah-Notchpeak-CE1 4 hours ago 100
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
Utah-Notchpeak-CE1 — probe output
pass kernel
4.18.0-553.141.1.el8_10.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/local/osgusern/14417598/glide_cogpzJ/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/local/osgusern/14417598/glide_cogpzJ/execute/dir_1831802/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=1831802 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/scratch/local/osgusern/14417598/glide_cogpzJ/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/local/osgusern/14417598/glide_cogpzJ/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_Utah-Notchpeak-CE1.idtoken.idtoken GLIDEIN_CPUS=16 GLIDEIN_ClusterId=6783104 GLIDEIN_Entry_Name=OSG_US_Utah-Notchpeak-CE1 GLIDEIN_Entry_Signature=79b279a8ed4d14042be536907884af467a3b8752 GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=utah-notchpeak-ce1.svc.opensciencegrid.org utah-notchpeak-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osgusern_s42vXM GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=2 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=Utah-Notchpeak-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=Utah-Notchpeak GLIDEIN_Tmp_Dir=/scratch/local/osgusern/14417598/glide_cogpzJ/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/scratch/local/osgusern/14417598/glide_cogpzJ/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/scratch/local/osgusern/14417598/glide_cogpzJ/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=utah-notchpeak-ce1.svc.opensciencegrid.org OSG_SITE_NAME=Utah-Notchpeak-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_6@glidein_1182306_503231616@notch200.ipoib.int.chpc.utah.edu$ "; unset PROMPT_COMMAND PS1=slot1_6@glidein_1182306_503231616@notch200.ipoib.int.chpc.utah.edu$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/etc/OpenCL/vendors SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/local/osgusern/14417598/glide_cogpzJ/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/scratch/local/osgusern/14417598/glide_cogpzJ/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/scratch/local/osgusern/14417598/glide_cogpzJ/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/local/osgusern/14417598/glide_cogpzJ/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/local/osgusern/14417598/glide_cogpzJ/execute/dir_1831802/scratch _CONDOR_SLOT=slot1_6 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
3510.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/vg_peaks_crypt-lv_scratch 3838014464 157160900 3680853564 5% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: 10.242.154.2 0.073ms 1: 10.242.154.2 0.088ms 2: rt-inscc-ddc-core-101.chpc.utah.edu 1.167ms asymm 3 3: 10.2.0.1 2.412ms asymm 4 4: 155.99.130.11 1.895ms asymm 5 5: 155.99.130.64 1.244ms asymm 6 6: 199.104.93.34 1.915ms 7: 199.104.93.21 3.072ms 8: 140.197.253.138 3.365ms 9: 140.197.248.40 3.725ms asymm 11 10: lv3-beibr-b-182-int.uen.net 3.044ms 11: fourhundredge-0-0-0-8.1013.core1.salt.net.internet2.edu 6.596ms 12: fourhundredge-0-0-0-0.4079.core1.denv.net.internet2.edu 37.027ms asymm 20 13: fourhundredge-0-0-0-0.4079.core1.kans.net.internet2.edu 37.286ms asymm 19 14: fourhundredge-0-0-0-3.4079.core2.chic.net.internet2.edu 38.611ms asymm 18 15: fourhundredge-0-0-0-22.4079.core1.chic.net.internet2.edu 38.595ms asymm 17 16: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 38.553ms 17: 144.92.233.112 39.551ms 18: perfsonar02.hep.wisc.edu 39.848ms reached Resume: pmtu 1500 hops 18 back 19
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osgusern (uid 67354)
VU-AUGIE-CE1 4 hours ago 2
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
VU-AUGIE-CE1 — probe output
pass kernel
5.15.0-57-generic
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/scratch/glide_6EEOgx/images/htc__rocky__9.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__rocky__9.sif APPTAINER_TMPDIR=/scratch/glide_6EEOgx/execute/dir_4151343/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=4151343 CONTAINERSW_FULL_VERSION=apptainer version 1.1.8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/usr/bin/apptainer CONTAINERSW_VERSION=1.1.8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/scratch/glide_6EEOgx/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.VU-AUGIE-CE1.idtoken.idtoken GLIDEIN_CPUS=64 GLIDEIN_ClusterId=6703130 GLIDEIN_Entry_Name=VU-AUGIE-CE1 GLIDEIN_Entry_Signature=9687ab80df8e4074dc12f68d921550e68cde599e GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=vu-augie-ce1.svc.opensciencegrid.org vu-augie-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_af9VNE GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=augie.villanova.edu:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=VU-AUGIE-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-1.osg-htc.org GLIDEIN_Signature=abb0f8c777e8d5f9bcf1379ef7c8ea262d98ef55 GLIDEIN_Site=VU-AUGIE GLIDEIN_Tmp_Dir=/scratch/glide_6EEOgx/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.1.8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/usr/bin/apptainer GWMS_CONTAINERSW_VERSION=1.1.8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/usr/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.1.8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=vu-augie-ce1.svc.opensciencegrid.org OSG_SITE_NAME=VU-AUGIE-CE1 OSG_SQUID_LOCATION=augie.villanova.edu:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1_8@glidein_1782832_411244288@n001.cluster.com$ "; unset PROMPT_COMMAND PS1=slot1_8@glidein_1782832_411244288@n001.cluster.com$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/scratch/glide_6EEOgx/images/htc__rocky__9.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.1.8 SINGULARITY_IMAGES_DICT=default:/scratch/glide_6EEOgx/images/htc__rocky__9.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__rocky__9.sif SINGULARITY_PATH=/usr/bin/apptainer SINGULARITY_VERSION=1.1.8 SLOTS_LAYOUT=partitionable TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/scratch/glide_6EEOgx/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/scratch/glide_6EEOgx/execute/dir_4151343/scratch _CONDOR_SLOT=slot1_8 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
359.3 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/nvme0n1p4 760688044 383976032 376712012 51% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: clusterhn.cluster.pssclabs.com 0.181ms 1: clusterhn.cluster.pssclabs.com 0.239ms 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy augie.villanova.edu:3128
pass user
osg01 (uid 1052)
WISC-PATH-EP 5 hours ago 3
100.0%
100.0%
100.0%
100.0%
all
WISC-PATH-EP — probe output
warning all
Job failed with exit code 0; no results transferred
WSU-GRID-CE1 5 hours ago 8
100.0%
87.5%
100.0%
87.5%
all
WSU-GRID-CE1 — probe output
warning all
Job failed with exit code 0; no results transferred
WY-MedicineBow-CE1 4 hours ago 50
100.0%
100.0%
0.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
WY-MedicineBow-CE1 — probe output
pass kernel
5.14.0-611.55.1.el9_7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime APPTAINER_CACHEDIR=/srv/scratch APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/project/openscigrid/glide_mMHMby/images/htc__almalinux__10.sif APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=htc__almalinux__10.sif APPTAINER_TMPDIR=/project/openscigrid/glide_mMHMby/execute/dir_2811271/scratch BATCH_SYSTEM=HTCondor CONDOR_STARTER_PID=2811271 CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/project/openscigrid/glide_mMHMby/main/condor/usr/libexec/apptainer CONTAINERSW_VERSION=1.5.2-1.el8 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CONDOR_TOKEN=/project/openscigrid/glide_mMHMby/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.OSG_US_WY-MedicineBow-CE1.idtoken.idtoken GLIDEIN_CPUS=1 GLIDEIN_ClusterId=6785311 GLIDEIN_Entry_Name=OSG_US_WY-MedicineBow-CE1 GLIDEIN_Entry_Signature=39fbc9dca72f8853809a9eb642ff5f5dc0045d4e GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=Tiger-Prod GLIDEIN_Gatekeeper=wy-medicinebow-ce1.svc.opensciencegrid.org wy-medicinebow-ce1.svc.opensciencegrid.org:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg01_thSBLl GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=0 GLIDEIN_Proxy_URL=None GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=WY-MedicineBow-CE1 GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=schedd_glideins4@gfactory-1.osg-htc.org GLIDEIN_Signature=d3f9636fb96fc9b1fa9540f32b8684dc0c6bce44 GLIDEIN_Site=WY-MedicineBow GLIDEIN_Tmp_Dir=/project/openscigrid/glide_mMHMby/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.5.2-1.el8 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/project/openscigrid/glide_mMHMby/main/condor/usr/libexec/apptainer GWMS_CONTAINERSW_VERSION=1.5.2-1.el8 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/project/openscigrid/glide_mMHMby/main/condor/usr/libexec/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.5.2-1.el8 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=wy-medicinebow-ce1.svc.opensciencegrid.org OSG_SITE_NAME=WY-MedicineBow-CE1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_2665729_139449951@mbcpu-001.cluster$ "; unset PROMPT_COMMAND PS1=slot1@glidein_2665729_139449951@mbcpu-001.cluster$ PWD=/srv/scratch PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime SINGULARITY_CACHEDIR=/srv/scratch SINGULARITY_CONTAINER=/project/openscigrid/glide_mMHMby/images/htc__almalinux__10.sif SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.5.2-1.el8 SINGULARITY_IMAGES_DICT=default:/project/openscigrid/glide_mMHMby/images/htc__almalinux__10.sif SINGULARITY_MODE=unprivileged SINGULARITY_NAME=htc__almalinux__10.sif SINGULARITY_PATH=/project/openscigrid/glide_mMHMby/main/condor/usr/libexec/apptainer SINGULARITY_VERSION=1.5.2-1.el8 SLOTS_LAYOUT=fixed TEMP=/srv/scratch TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv/scratch TMPDIR=/srv/scratch TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/project/openscigrid/glide_mMHMby/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/user/.chirp.config _CONDOR_CREDS=/srv/scratch/.condor_creds _CONDOR_JOB_AD=/srv/scratch/.job.ad _CONDOR_JOB_IWD=/srv/scratch _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/scratch/.machine.ad _CONDOR_SCRATCH_DIR=/srv/scratch _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/project/openscigrid/glide_mMHMby/execute/dir_2811271/scratch _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/scratch/.job_wrapper_failure
pass df
4805.8 GB available Filesystem 1024-blocks Used Available Capacity Mounted on vast-eth:/cluster/medbow/project/openscigrid/glide_mMHMby/execute/dir_2811271 5368709120 329490432 5039218688 7% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: no reply 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1500 Resume: pmtu 1500
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
not_available http_proxy
Neither http_proxy nor OSG_SQUID_LOCATION is set
pass user
osg01 (uid 5720062)
uprm-cms 2 hours ago 20
100.0%
0.0%
100.0%
100.0%
kernel env df firewall mtu nrp_images http_proxy user
uprm-cms — probe output
warning kernel
3.10.0-1160.105.1.el7.x86_64
pass env
APPTAINER_APPNAME= APPTAINER_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs APPTAINER_CACHEDIR=/srv APPTAINER_COMMAND=exec APPTAINER_CONTAINER=/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 APPTAINER_ENVIRONMENT=/.singularity.d/env/91-environment.sh APPTAINER_NAME=rocky:9 APPTAINER_TMPDIR=/var/lib/condor/execute/dir_3089036/glide_eywo8l/execute/dir_3136157 BATCH_SYSTEM=HTCondor CONTAINERSW_FULL_VERSION=apptainer version 1.2.5-1.el7 CONTAINERSW_MODE=unprivileged CONTAINERSW_PATH=/bin/apptainer CONTAINERSW_VERSION=1.2.5-1.el7 CUBACORES=1 CUDA_VISIBLE_DEVICES=10000 GLIDECLIENT_Group=main GLIDECLIENT_Group_Signature=a74fe127f5696b642ae33d713cd1f96e62462e11 GLIDECLIENT_Name=OSPool.main GLIDECLIENT_OSG_VO=OSG GLIDECLIENT_Signature=e0f03b4feb1ae0f8341a6157712e8e8d892caac2 GLIDEIN_BLACKHOLE=False GLIDEIN_CMSSite=T3_US_PuertoRico GLIDEIN_CONDOR_TOKEN=/var/lib/condor/execute/dir_3089036/glide_eywo8l/cred.d/idtokens/credential_callback_IdTokenGenerator_63XUYJG2.CMS_T3_US_PuertoRico_UPRM.idtoken.idtoken GLIDEIN_ClusterId=555292 GLIDEIN_Country=US GLIDEIN_DUNESite=US_PuertoRico GLIDEIN_Entry_Name=CMS_T3_US_PuertoRico_UPRM GLIDEIN_Entry_Signature=92233fe3dea2247ccff874d9495979b62dd7117a GLIDEIN_Expose_Grid_Env=True GLIDEIN_Factory=NRP-Prod GLIDEIN_Gatekeeper=cms-grid0.hep.uprm.edu cms-grid0.hep.uprm.edu:9619 GLIDEIN_Glexec_Use=NEVER GLIDEIN_GridType=condor GLIDEIN_LOCAL_TMP_DIR=/tmp/glide_osg_zIimpx GLIDEIN_Monitoring_Enabled=False GLIDEIN_NODES=1 GLIDEIN_Name=gfactory_instance GLIDEIN_OVERLOAD_ENABLED=False GLIDEIN_PROVIDES=singularity/unprivileged GLIDEIN_ProcId=5 GLIDEIN_Proxy_URL=squid-0-0-0.local:3128 GLIDEIN_REQUIRED_OS=any GLIDEIN_REQUIRE_VOMS=False GLIDEIN_ResourceName=uprm-cms GLIDEIN_SEs=cms-se.hep.uprm.edu GLIDEIN_SINGULARITY_BINDPATH=/alphafold3,/mnt/alphafold,/mnt/alphafold/alphafold3_7a4a2f,/scitech/nas/scratch-90-days/alphafold3 GLIDEIN_Schedd=gfactory-2.osg-htc.org GLIDEIN_Signature=a27902e12b0688aace73b9e2a07ae961f9769aa5 GLIDEIN_Site=PuertoRico GLIDEIN_Tmp_Dir=/var/lib/condor/execute/dir_3089036/glide_eywo8l/tmp GOMAXPROCS=1 GPU_DEVICE_ORDINAL=10000 GWMS_CONTAINERSW_FULL_VERSION=apptainer version 1.2.5-1.el7 GWMS_CONTAINERSW_MODE=unprivileged GWMS_CONTAINERSW_PATH=/bin/apptainer GWMS_CONTAINERSW_VERSION=1.2.5-1.el7 GWMS_SINGULARITY_MODE=unprivileged GWMS_SINGULARITY_PATH=/bin/apptainer GWMS_SINGULARITY_STATUS=PREFERRED GWMS_SINGULARITY_VERSION=1.2.5-1.el7 HAS_SINGULARITY=True HAS_UNPRIVILEGED_USER_NAMESPACES=enabled HOLD_GRACE_TIME=0 HOME=/srv JULIA_NUM_THREADS=1 LANG=C LC_CTYPE=C.UTF-8 LD_LIBRARY_PATH=/.singularity.d/libs MKL_NUM_THREADS=1 NUMEXPR_NUM_THREADS=1 NVIDIA_VISIBLE_DEVICES=none OMP_NUM_THREADS=1 OMP_THREAD_LIMIT=1 OPENBLAS_NUM_THREADS=1 OSG_HOSTNAME=cms-grid0.hep.uprm.edu OSG_SITE_NAME=uprm-cms-ce OSG_SQUID_LOCATION=squid-0-0-0.local:3128 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PREEMPT_GRACE_TIME=10000000 PROMPT_COMMAND=PS1="slot1@glidein_3089038_290694101@sandy-0-2.local$ "; unset PROMPT_COMMAND PS1=slot1@glidein_3089038_290694101@sandy-0-2.local$ PWD=/srv PYTHON_CPU_COUNT=1 ROOT_MAX_THREADS=1 SINGULARITY_BIND=/srv,/etc/hosts,/etc/localtime,/cvmfs SINGULARITY_CACHEDIR=/srv SINGULARITY_CONTAINER=/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_ENVIRONMENT=/.singularity.d/env/91-environment.sh SINGULARITY_FULL_VERSION=apptainer version 1.2.5-1.el7 SINGULARITY_IMAGES_DICT=default:/cvmfs/singularity.opensciencegrid.org/htc/rocky:9 SINGULARITY_MODE=unprivileged SINGULARITY_NAME=rocky:9 SINGULARITY_PATH=/bin/apptainer SINGULARITY_VERSION=1.2.5-1.el7 SLOTS_LAYOUT=fixed TEMP=/srv TF_LOOP_PARALLEL_ITERATIONS=1 TF_NUM_THREADS=1 TMP=/srv TMPDIR=/srv TRUST_DOMAIN=cm-1.ospool.osg-htc.org:9618?sock=collectorRANDOM _CHIRP_DELAYED_UPDATE_PREFIX=Chirp* _CONDOR_AssignedGPUs=10000 _CONDOR_BIN=/var/lib/condor/execute/dir_3089036/glide_eywo8l/main/condor/bin _CONDOR_CHIRP_CONFIG=/srv/.chirp.config _CONDOR_CREDS=/srv/.condor_creds _CONDOR_JOB_AD=/srv/.job.ad _CONDOR_JOB_IWD=/srv _CONDOR_JOB_PIDS= _CONDOR_MACHINE_AD=/srv/.machine.ad _CONDOR_SCRATCH_DIR=/srv _CONDOR_SCRATCH_DIR_OUTSIDE_CONTAINER=/var/lib/condor/execute/dir_3089036/glide_eywo8l/execute/dir_3136157 _CONDOR_SLOT=slot1 _CONDOR_WRAPPER_ERROR_FILE=/srv/.job_wrapper_failure
pass df
153.5 GB available Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3 206292968 34825380 160965444 18% /srv
pass firewall
All ports reachable on ospool-network-test.osg.chtc.io http://ospool-network-test.osg.chtc.io:80: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:1094: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:2880: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8000: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:8443: OK (HTTP 200) http://ospool-network-test.osg.chtc.io:27017: OK (HTTP 200)
pass mtu
pmtu=1500 1?: [LOCALHOST] pmtu 1500 1: cms-hn.local 0.237ms 1: cms-hn.local 0.264ms 2: 136.145.77.1 0.512ms 3: 136.145.230.114 0.472ms 4: R8500.uprm.edu 0.632ms 5: hub787-rum-a.upr.edu 5.052ms 6: et-0-0-1-819.rt04.be.ampath.net 34.998ms 7: et-0-0-1-67.rt05.bb.ampath.net 35.073ms 8: hundredge-0-0-0-26.1804.core1.jack.net.internet2.edu 44.852ms 9: fourhundredge-0-0-0-6.4079.core1.atla.net.internet2.edu 67.334ms asymm 21 10: fourhundredge-0-0-0-22.4079.core2.atla.net.internet2.edu 67.981ms asymm 20 11: fourhundredge-0-0-0-4.4079.core2.nash.net.internet2.edu 68.494ms asymm 19 12: fourhundredge-0-0-0-1.4079.core1.cinc.net.internet2.edu 68.845ms asymm 18 13: fourhundredge-0-0-0-23.4079.core2.cinc.net.internet2.edu 68.038ms asymm 17 14: fourhundredge-0-0-0-4.4079.core1.chic.net.internet2.edu 67.181ms asymm 16 15: fourhundredge-0-0-0-4.4079.core1.star.net.internet2.edu 67.408ms 16: 144.92.233.112 69.448ms 17: perfsonar02.hep.wisc.edu 69.318ms reached Resume: pmtu 1500 hops 17 back 18
pass nrp_images
Downloaded https://ospool-images-web.nrp-nautilus.io/v2/x86_64/htc__rocky__9/latest.txt successfully 20260526T053001Z.sif
pass http_proxy
Downloaded http://ospool-network-test.osg.chtc.io/index.html successfully via proxy squid-0-0-0.local:3128
pass user
osg (uid 5458)