Upload folder using huggingface_hub
Browse files
driver-logs-next-campaign/README.txt
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Driver logs, next-top-k / new-Tier-B campaign (2026-09-16 to 09-21).
|
| 2 |
+
|
| 3 |
+
next2.log first pod: Tier A probe, timing, memory, ECHO traces
|
| 4 |
+
next3.log second pod after preemption: ultraep+moonep traces, all Tier B
|
| 5 |
+
pvc_mirror.log nested PVC state mirror
|
| 6 |
+
build.log environment build on the second pod
|
| 7 |
+
|
| 8 |
+
The campaign spans two pods: the first was preempted during the Tier A trace
|
| 9 |
+
stage. 29 completion markers were restored from the PVC mirror and no
|
| 10 |
+
measurement work was repeated.
|
driver-logs-next-campaign/build.log
ADDED
|
@@ -0,0 +1,175 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
==============================================
|
| 2 |
+
MoonEP B200 build
|
| 3 |
+
Workspace: /workspace (local overlay)
|
| 4 |
+
Wheel cache: /pvcdata/aiinfra/b-aghosh/wheels (PVC)
|
| 5 |
+
TE branch: cutlass_device_grouped_gemm archs=100a
|
| 6 |
+
==============================================
|
| 7 |
+
|
| 8 |
+
[1/6] System dependencies...
|
| 9 |
+
debconf: unable to initialize frontend: Dialog
|
| 10 |
+
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79, <STDIN> line 18.)
|
| 11 |
+
debconf: falling back to frontend: Readline
|
| 12 |
+
debconf: unable to initialize frontend: Readline
|
| 13 |
+
debconf: (This frontend requires a controlling tty.)
|
| 14 |
+
debconf: falling back to frontend: Teletype
|
| 15 |
+
dpkg-preconfigure: unable to re-open stdin:
|
| 16 |
+
(Reading database ...
|
| 17 |
+
Preparing to unpack .../libpython3-dev_3.12.3-0ubuntu2.1_amd64.deb ...
|
| 18 |
+
Unpacking libpython3-dev:amd64 (3.12.3-0ubuntu2.1) over (3.12.3-0ubuntu2) ...
|
| 19 |
+
Preparing to unpack .../python3-dev_3.12.3-0ubuntu2.1_amd64.deb ...
|
| 20 |
+
Unpacking python3-dev (3.12.3-0ubuntu2.1) over (3.12.3-0ubuntu2) ...
|
| 21 |
+
Preparing to unpack .../python3-venv_3.12.3-0ubuntu2.1_amd64.deb ...
|
| 22 |
+
Unpacking python3-venv (3.12.3-0ubuntu2.1) over (3.12.3-0ubuntu2) ...
|
| 23 |
+
Preparing to unpack .../python3-minimal_3.12.3-0ubuntu2.1_amd64.deb ...
|
| 24 |
+
Unpacking python3-minimal (3.12.3-0ubuntu2.1) over (3.12.3-0ubuntu2) ...
|
| 25 |
+
Setting up python3-minimal (3.12.3-0ubuntu2.1) ...
|
| 26 |
+
(Reading database ...
|
| 27 |
+
Preparing to unpack .../00-python3_3.12.3-0ubuntu2.1_amd64.deb ...
|
| 28 |
+
running python pre-rtupdate hooks for python3.12...
|
| 29 |
+
Unpacking python3 (3.12.3-0ubuntu2.1) over (3.12.3-0ubuntu2) ...
|
| 30 |
+
Preparing to unpack .../01-libpython3-stdlib_3.12.3-0ubuntu2.1_amd64.deb ...
|
| 31 |
+
Unpacking libpython3-stdlib:amd64 (3.12.3-0ubuntu2.1) over (3.12.3-0ubuntu2) ...
|
| 32 |
+
Preparing to unpack .../02-ca-certificates_20260601~24.04.1_all.deb ...
|
| 33 |
+
Unpacking ca-certificates (20260601~24.04.1) over (20240203) ...
|
| 34 |
+
Selecting previously unselected package libicu74:amd64.
|
| 35 |
+
Preparing to unpack .../03-libicu74_74.2-1ubuntu3.1_amd64.deb ...
|
| 36 |
+
Unpacking libicu74:amd64 (74.2-1ubuntu3.1) ...
|
| 37 |
+
Selecting previously unselected package libxml2:amd64.
|
| 38 |
+
Preparing to unpack .../04-libxml2_2.9.14+dfsg-1.3ubuntu3.8_amd64.deb ...
|
| 39 |
+
Unpacking libxml2:amd64 (2.9.14+dfsg-1.3ubuntu3.8) ...
|
| 40 |
+
Selecting previously unselected package libuv1t64:amd64.
|
| 41 |
+
Preparing to unpack .../05-libuv1t64_1.48.0-1.1build1_amd64.deb ...
|
| 42 |
+
Unpacking libuv1t64:amd64 (1.48.0-1.1build1) ...
|
| 43 |
+
Preparing to unpack .../06-wget_1.21.4-1ubuntu4.5_amd64.deb ...
|
| 44 |
+
Unpacking wget (1.21.4-1ubuntu4.5) over (1.21.4-1ubuntu4.1) ...
|
| 45 |
+
Selecting previously unselected package libarchive13t64:amd64.
|
| 46 |
+
Preparing to unpack .../07-libarchive13t64_3.7.2-2ubuntu0.8_amd64.deb ...
|
| 47 |
+
Unpacking libarchive13t64:amd64 (3.7.2-2ubuntu0.8) ...
|
| 48 |
+
Selecting previously unselected package libjsoncpp25:amd64.
|
| 49 |
+
Preparing to unpack .../08-libjsoncpp25_1.9.5-6build1_amd64.deb ...
|
| 50 |
+
Unpacking libjsoncpp25:amd64 (1.9.5-6build1) ...
|
| 51 |
+
Selecting previously unselected package librhash0:amd64.
|
| 52 |
+
Preparing to unpack .../09-librhash0_1.4.3-3build1_amd64.deb ...
|
| 53 |
+
Unpacking librhash0:amd64 (1.4.3-3build1) ...
|
| 54 |
+
Selecting previously unselected package cmake-data.
|
| 55 |
+
Preparing to unpack .../10-cmake-data_3.28.3-1build7_all.deb ...
|
| 56 |
+
Unpacking cmake-data (3.28.3-1build7) ...
|
| 57 |
+
Selecting previously unselected package cmake.
|
| 58 |
+
Preparing to unpack .../11-cmake_3.28.3-1build7_amd64.deb ...
|
| 59 |
+
Unpacking cmake (3.28.3-1build7) ...
|
| 60 |
+
Preparing to unpack .../12-git-man_1%3a2.43.0-1ubuntu7.3_all.deb ...
|
| 61 |
+
Unpacking git-man (1:2.43.0-1ubuntu7.3) over (1:2.43.0-1ubuntu7.2) ...
|
| 62 |
+
Preparing to unpack .../13-git_1%3a2.43.0-1ubuntu7.3_amd64.deb ...
|
| 63 |
+
Unpacking git (1:2.43.0-1ubuntu7.3) over (1:2.43.0-1ubuntu7.2) ...
|
| 64 |
+
Setting up libpython3-dev:amd64 (3.12.3-0ubuntu2.1) ...
|
| 65 |
+
Setting up wget (1.21.4-1ubuntu4.5) ...
|
| 66 |
+
Setting up libuv1t64:amd64 (1.48.0-1.1build1) ...
|
| 67 |
+
Setting up ca-certificates (20260601~24.04.1) ...
|
| 68 |
+
debconf: unable to initialize frontend: Dialog
|
| 69 |
+
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.)
|
| 70 |
+
debconf: falling back to frontend: Readline
|
| 71 |
+
debconf: unable to initialize frontend: Readline
|
| 72 |
+
debconf: (This frontend requires a controlling tty.)
|
| 73 |
+
debconf: falling back to frontend: Teletype
|
| 74 |
+
Updating certificates in /etc/ssl/certs...
|
| 75 |
+
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
|
| 76 |
+
14 added, 39 removed; done.
|
| 77 |
+
Setting up libjsoncpp25:amd64 (1.9.5-6build1) ...
|
| 78 |
+
Setting up libicu74:amd64 (74.2-1ubuntu3.1) ...
|
| 79 |
+
Setting up librhash0:amd64 (1.4.3-3build1) ...
|
| 80 |
+
Setting up git-man (1:2.43.0-1ubuntu7.3) ...
|
| 81 |
+
Setting up cmake-data (3.28.3-1build7) ...
|
| 82 |
+
Setting up libxml2:amd64 (2.9.14+dfsg-1.3ubuntu3.8) ...
|
| 83 |
+
Setting up libpython3-stdlib:amd64 (3.12.3-0ubuntu2.1) ...
|
| 84 |
+
Setting up python3 (3.12.3-0ubuntu2.1) ...
|
| 85 |
+
running python rtupdate hooks for python3.12...
|
| 86 |
+
running python post-rtupdate hooks for python3.12...
|
| 87 |
+
Setting up python3-venv (3.12.3-0ubuntu2.1) ...
|
| 88 |
+
Setting up python3-dev (3.12.3-0ubuntu2.1) ...
|
| 89 |
+
Setting up git (1:2.43.0-1ubuntu7.3) ...
|
| 90 |
+
Setting up libarchive13t64:amd64 (3.7.2-2ubuntu0.8) ...
|
| 91 |
+
Setting up cmake (3.28.3-1build7) ...
|
| 92 |
+
Processing triggers for libc-bin (2.39-0ubuntu8.4) ...
|
| 93 |
+
Processing triggers for ca-certificates (20260601~24.04.1) ...
|
| 94 |
+
Updating certificates in /etc/ssl/certs...
|
| 95 |
+
0 added, 0 removed; done.
|
| 96 |
+
Running hooks in /etc/ca-certificates/update.d...
|
| 97 |
+
done.
|
| 98 |
+
|
| 99 |
+
[2/6] Python deps...
|
| 100 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 101 |
+
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
|
| 102 |
+
cudf 25.4.0 requires pyarrow<20.0.0a0,>=14.0.0; platform_machine == "x86_64", but you have pyarrow 25.0.1 which is incompatible.
|
| 103 |
+
pylibcudf 25.4.0 requires pyarrow<20.0.0a0,>=14.0.0; platform_machine == "x86_64", but you have pyarrow 25.0.1 which is incompatible.
|
| 104 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 105 |
+
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
|
| 106 |
+
cudf 25.4.0 requires pyarrow<20.0.0a0,>=14.0.0; platform_machine == "x86_64", but you have pyarrow 25.0.1 which is incompatible.
|
| 107 |
+
cupy-cuda12x 13.3.0 requires numpy<2.3,>=1.22, but you have numpy 2.5.3 which is incompatible.
|
| 108 |
+
thinc 8.2.5 requires numpy<2.0.0,>=1.19.0; python_version >= "3.9", but you have numpy 2.5.3 which is incompatible.
|
| 109 |
+
scipy 1.15.3 requires numpy<2.5,>=1.23.5, but you have numpy 2.5.3 which is incompatible.
|
| 110 |
+
numba 0.59.1 requires numpy<1.27,>=1.22, but you have numpy 2.5.3 which is incompatible.
|
| 111 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 112 |
+
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
|
| 113 |
+
ml-dtypes 0.6.0 requires numpy>=2.0.0, but you have numpy 1.26.4 which is incompatible.
|
| 114 |
+
cudf 25.4.0 requires pyarrow<20.0.0a0,>=14.0.0; platform_machine == "x86_64", but you have pyarrow 25.0.1 which is incompatible.
|
| 115 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 116 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 117 |
+
NVSHMEM_DIR=/usr/local/lib/python3.12/dist-packages/nvidia/nvshmem
|
| 118 |
+
NCCL_DIR=/workspace/.nccl-root -> /usr/lib/x86_64-linux-gnu/libnccl.so.2.27.3
|
| 119 |
+
|
| 120 |
+
[3/6] TransformerEngine (cutlass_device_grouped_gemm)...
|
| 121 |
+
cache HIT transformer_engine -> transformer_engine-2.10.0.dev0+10f6128d-cp312-cp312-linux_x86_64.whl
|
| 122 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 123 |
+
OK TE: 2.10.0.dev0+10f6128d
|
| 124 |
+
|
| 125 |
+
[4/6] MoonEP (megatron-ddp-symmetric-params, editable)...
|
| 126 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 127 |
+
OK MoonEP: /workspace/MoonEP/moonep/__init__.py
|
| 128 |
+
|
| 129 |
+
[5/6] Megatron-MoE-Echo (moonep_backend, editable)...
|
| 130 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 131 |
+
|
| 132 |
+
[6/6] DeepEP (hybrid-ep @ 34152ae; REQUIRED for echo + ultraep)...
|
| 133 |
+
DeepEP pinned to: 34152ae Quick fix about the corner case (#602)
|
| 134 |
+
Obtaining file:///workspace/DeepEP
|
| 135 |
+
Checking if build backend supports build_editable: started
|
| 136 |
+
Checking if build backend supports build_editable: finished with status 'done'
|
| 137 |
+
Preparing editable metadata (pyproject.toml): started
|
| 138 |
+
Preparing editable metadata (pyproject.toml): finished with status 'done'
|
| 139 |
+
Requirement already satisfied: pynvml in /usr/local/lib/python3.12/dist-packages (from deep_ep==1.2.1+34152ae) (12.0.0)
|
| 140 |
+
Requirement already satisfied: nvidia-ml-py<13.0.0a0,>=12.0.0 in /usr/local/lib/python3.12/dist-packages (from pynvml->deep_ep==1.2.1+34152ae) (12.575.51)
|
| 141 |
+
Building wheels for collected packages: deep_ep
|
| 142 |
+
Building editable for deep_ep (pyproject.toml): started
|
| 143 |
+
Building editable for deep_ep (pyproject.toml): still running...
|
| 144 |
+
Building editable for deep_ep (pyproject.toml): still running...
|
| 145 |
+
Building editable for deep_ep (pyproject.toml): finished with status 'done'
|
| 146 |
+
Created wheel for deep_ep: filename=deep_ep-1.2.1+34152ae-0.editable-cp312-cp312-linux_x86_64.whl size=3700 sha256=0bc77036691a62a9593f5d26a5e402d16e147e5b937ce8fcbca6f9860b2c4dc8
|
| 147 |
+
Stored in directory: /tmp/pip-ephem-wheel-cache-ij1vi2vc/wheels/0e/b1/6f/ea438df43e3844fdaa802f7af06ae674f98b0a31161a829e45
|
| 148 |
+
Successfully built deep_ep
|
| 149 |
+
Installing collected packages: deep_ep
|
| 150 |
+
Successfully installed deep_ep-1.2.1+34152ae
|
| 151 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 152 |
+
OK DeepEP: /workspace/DeepEP/deep_ep/__init__.py HybridEPBuffer: True
|
| 153 |
+
|
| 154 |
+
==============================================
|
| 155 |
+
Building ultra_ep ...
|
| 156 |
+
Obtaining file:///workspace/UltraEP
|
| 157 |
+
Checking if build backend supports build_editable: started
|
| 158 |
+
Checking if build backend supports build_editable: finished with status 'done'
|
| 159 |
+
Preparing editable metadata (pyproject.toml): started
|
| 160 |
+
Preparing editable metadata (pyproject.toml): finished with status 'done'
|
| 161 |
+
Building wheels for collected packages: ultra_ep
|
| 162 |
+
Building editable for ultra_ep (pyproject.toml): started
|
| 163 |
+
Building editable for ultra_ep (pyproject.toml): still running...
|
| 164 |
+
Building editable for ultra_ep (pyproject.toml): finished with status 'done'
|
| 165 |
+
Created wheel for ultra_ep: filename=ultra_ep-1.0.0+94cab09-0.editable-cp312-cp312-linux_x86_64.whl size=3660 sha256=c9ed850fe14d993aea667adb8c5ffe41f25af603a2f9b4a27d5b4f4112c2d476
|
| 166 |
+
Stored in directory: /tmp/pip-ephem-wheel-cache-30o2dnv4/wheels/96/90/1f/6a4274963e828e0935457badbf63e962cf21f548787e364b32
|
| 167 |
+
Successfully built ultra_ep
|
| 168 |
+
Installing collected packages: ultra_ep
|
| 169 |
+
Successfully installed ultra_ep-1.0.0+94cab09
|
| 170 |
+
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
|
| 171 |
+
PyTorch: 2.8.0a0+5228986c39.nv25.06 CUDA: 12.9
|
| 172 |
+
GPU0: NVIDIA B200 sm_100
|
| 173 |
+
OK all imports (incl. DeepEP)
|
| 174 |
+
OK ultra_ep
|
| 175 |
+
Build complete.
|
driver-logs-next-campaign/next3.log
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
driver-logs-next-campaign/pvc_mirror.log
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[2026-09-21T05:16:14+00:00] mirroring nested resume state
|
| 2 |
+
from : /workspace/granularity_next_b200
|
| 3 |
+
to : /pvcdata/aiinfra/b-aghosh/granularity_state/granularity_next_b200
|
| 4 |
+
every: 300s
|
| 5 |
+
|
| 6 |
+
to resume on a new pod:
|
| 7 |
+
mkdir -p /workspace/granularity_next_b200 && cp -r /pvcdata/aiinfra/b-aghosh/granularity_state/granularity_next_b200/. /workspace/granularity_next_b200/ # -r not -a: NFS rejects -a's chmod and cp aborts
|
| 8 |
+
./granularity/run_next_topk_and_models.sh --out /workspace/granularity_next_b200 --hf-repo <repo>
|
| 9 |
+
(resume is the default; completed steps are skipped)
|
| 10 |
+
[2026-09-21T05:17:10+00:00] pass 1: 29 marker(s) across 10 sub-root(s), 2433 file(s) mirrored, 0 pruned
|
| 11 |
+
[2026-09-21T05:23:05+00:00] pass 2: 29 marker(s) across 10 sub-root(s), 2433 file(s) mirrored, 0 pruned
|
| 12 |
+
[2026-09-21T05:29:01+00:00] pass 3: 29 marker(s) across 10 sub-root(s), 2433 file(s) mirrored, 0 pruned
|
| 13 |
+
[2026-09-21T05:35:00+00:00] pass 4: 29 marker(s) across 10 sub-root(s), 2433 file(s) mirrored, 0 pruned
|
| 14 |
+
[2026-09-21T05:40:57+00:00] pass 5: 29 marker(s) across 10 sub-root(s), 2435 file(s) mirrored, 0 pruned
|
| 15 |
+
[2026-09-21T05:46:54+00:00] pass 6: 29 marker(s) across 10 sub-root(s), 2439 file(s) mirrored, 0 pruned
|
| 16 |
+
[2026-09-21T05:52:51+00:00] pass 7: 29 marker(s) across 10 sub-root(s), 2445 file(s) mirrored, 0 pruned
|
| 17 |
+
[2026-09-21T05:58:50+00:00] pass 8: 30 marker(s) across 10 sub-root(s), 2453 file(s) mirrored, 0 pruned
|
| 18 |
+
[2026-09-21T06:04:47+00:00] pass 9: 30 marker(s) across 10 sub-root(s), 2461 file(s) mirrored, 0 pruned
|
| 19 |
+
[2026-09-21T06:10:44+00:00] pass 10: 30 marker(s) across 10 sub-root(s), 2473 file(s) mirrored, 0 pruned
|
| 20 |
+
[2026-09-21T06:16:42+00:00] pass 11: 30 marker(s) across 10 sub-root(s), 2481 file(s) mirrored, 0 pruned
|
| 21 |
+
[2026-09-21T06:22:41+00:00] pass 12: 31 marker(s) across 11 sub-root(s), 2502 file(s) mirrored, 0 pruned
|
| 22 |
+
[2026-09-21T06:28:40+00:00] pass 13: 33 marker(s) across 11 sub-root(s), 2525 file(s) mirrored, 0 pruned
|
| 23 |
+
[2026-09-21T06:34:40+00:00] pass 14: 35 marker(s) across 11 sub-root(s), 2538 file(s) mirrored, 0 pruned
|
| 24 |
+
[2026-09-21T06:40:39+00:00] pass 15: 36 marker(s) across 11 sub-root(s), 2556 file(s) mirrored, 0 pruned
|
| 25 |
+
[2026-09-21T06:46:39+00:00] pass 16: 38 marker(s) across 12 sub-root(s), 2578 file(s) mirrored, 0 pruned
|
| 26 |
+
[2026-09-21T06:52:39+00:00] pass 17: 40 marker(s) across 12 sub-root(s), 2600 file(s) mirrored, 0 pruned
|
| 27 |
+
[2026-09-21T06:58:40+00:00] pass 18: 41 marker(s) across 12 sub-root(s), 2620 file(s) mirrored, 0 pruned
|
| 28 |
+
[2026-09-21T07:04:43+00:00] pass 19: 43 marker(s) across 13 sub-root(s), 2642 file(s) mirrored, 0 pruned
|
| 29 |
+
[2026-09-21T07:10:48+00:00] pass 20: 45 marker(s) across 13 sub-root(s), 2837 file(s) mirrored, 0 pruned
|
| 30 |
+
[2026-09-21T07:16:57+00:00] pass 21: 46 marker(s) across 14 sub-root(s), 2939 file(s) mirrored, 0 pruned
|
| 31 |
+
[2026-09-21T07:23:06+00:00] pass 22: 47 marker(s) across 14 sub-root(s), 2947 file(s) mirrored, 0 pruned
|
| 32 |
+
[2026-09-21T07:29:13+00:00] pass 23: 48 marker(s) across 14 sub-root(s), 2955 file(s) mirrored, 0 pruned
|
| 33 |
+
[2026-09-21T07:35:22+00:00] pass 24: 49 marker(s) across 15 sub-root(s), 2975 file(s) mirrored, 0 pruned
|
| 34 |
+
[2026-09-21T07:41:31+00:00] pass 25: 51 marker(s) across 15 sub-root(s), 2998 file(s) mirrored, 0 pruned
|
| 35 |
+
[2026-09-21T07:47:41+00:00] pass 26: 53 marker(s) across 15 sub-root(s), 3018 file(s) mirrored, 0 pruned
|
| 36 |
+
[2026-09-21T07:53:52+00:00] pass 27: 54 marker(s) across 15 sub-root(s), 3032 file(s) mirrored, 0 pruned
|
| 37 |
+
[2026-09-21T08:00:07+00:00] pass 28: 56 marker(s) across 16 sub-root(s), 3191 file(s) mirrored, 0 pruned
|
| 38 |
+
[2026-09-21T08:06:23+00:00] pass 29: 58 marker(s) across 17 sub-root(s), 3340 file(s) mirrored, 0 pruned
|
| 39 |
+
[2026-09-21T08:12:38+00:00] pass 30: 59 marker(s) across 17 sub-root(s), 3348 file(s) mirrored, 0 pruned
|
| 40 |
+
[2026-09-21T08:18:55+00:00] pass 31: 60 marker(s) across 17 sub-root(s), 3356 file(s) mirrored, 0 pruned
|
| 41 |
+
[2026-09-21T08:25:13+00:00] pass 32: 61 marker(s) across 18 sub-root(s), 3375 file(s) mirrored, 0 pruned
|
| 42 |
+
[2026-09-21T08:31:31+00:00] pass 33: 63 marker(s) across 18 sub-root(s), 3394 file(s) mirrored, 0 pruned
|
| 43 |
+
[2026-09-21T08:37:52+00:00] pass 34: 65 marker(s) across 18 sub-root(s), 3413 file(s) mirrored, 0 pruned
|
| 44 |
+
[2026-09-21T08:44:11+00:00] pass 35: 67 marker(s) across 19 sub-root(s), 3430 file(s) mirrored, 0 pruned
|
| 45 |
+
[2026-09-21T08:50:35+00:00] pass 36: 69 marker(s) across 19 sub-root(s), 3633 file(s) mirrored, 0 pruned
|
| 46 |
+
[2026-09-21T08:57:02+00:00] pass 37: 70 marker(s) across 20 sub-root(s), 3739 file(s) mirrored, 0 pruned
|
| 47 |
+
[2026-09-21T09:03:30+00:00] pass 38: 71 marker(s) across 20 sub-root(s), 3747 file(s) mirrored, 0 pruned
|
| 48 |
+
[2026-09-21T09:09:57+00:00] pass 39: 72 marker(s) across 20 sub-root(s), 3755 file(s) mirrored, 0 pruned
|
| 49 |
+
[2026-09-21T09:16:25+00:00] pass 40: 73 marker(s) across 20 sub-root(s), 3764 file(s) mirrored, 0 pruned
|
| 50 |
+
[2026-09-21T09:22:55+00:00] pass 41: 73 marker(s) across 20 sub-root(s), 3764 file(s) mirrored, 0 pruned
|