feat: translations integration tests #210
Open
stage-taskcluster / translations-dataset-opus-tldr-pages_v2023-08-29-en
succeeded
Jan 27, 2025 in 9m 41s
Stage-TC (issue_comment)
Fetch a monolingual opus dataset for {trg_locale}
Details
View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster
Task Status
Started: 2025-01-27T12:11:37.000Z
Resolved: 2025-01-27T12:14:53.736Z
Task Execution Time: 3 minutes, 16 seconds, 736 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0
Artifacts
- public/build/tldr-pages_v2023-08-29.en.zst
- public/logs/live_backing.log
- public/logs/live.log
[taskcluster 2025-01-27 12:11:37.096Z] Task ID: MpMjzbh7RwyhdJXNOSQR0Q
[taskcluster 2025-01-27 12:11:37.096Z] Worker ID: 6177806456876353906
[taskcluster 2025-01-27 12:11:37.096Z] Worker Group: us-west1-b
[taskcluster 2025-01-27 12:11:37.096Z] Worker Node Type: projects/474530013412/machineTypes/n2-highmem-32
[taskcluster 2025-01-27 12:11:37.096Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2025-01-27 12:11:37.096Z] Worker Version: 38.0.5
[taskcluster 2025-01-27 12:11:37.096Z] Public IP: 34.168.114.12
[taskcluster 2025-01-27 12:11:37.096Z] Hostname: translations-1-b-linux-large-gcp-eefvi9nvsh-sf7cg9qaapq
[taskcluster 2025-01-27 12:11:37.096Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-LaTmU8ILTKmjq-SGho18mA" -> /builds/worker/checkouts
[taskcluster 2025-01-27 12:11:46.286Z] Downloading artifact "public/image.tar.zst" from task ID: DUNjdZeFTIScNP4MGcgXGQ.
[taskcluster 2025-01-27 12:11:51.288Z] Download Progress: 43.54%
[taskcluster 2025-01-27 12:11:56.288Z] Download Progress: 93.97%
[taskcluster 2025-01-27 12:11:56.698Z] Downloaded artifact successfully.
[taskcluster 2025-01-27 12:11:56.698Z] Downloaded 761.177 mb
[taskcluster 2025-01-27 12:11:56.699Z] Decompressing downloaded image
[taskcluster 2025-01-27 12:12:03.439Z] Loading docker image from downloaded archive.
[taskcluster 2025-01-27 12:12:42.236Z] Image 'public/image.tar.zst' from task 'DUNjdZeFTIScNP4MGcgXGQ' loaded. Using image ID sha256:a6cdc387f7c79fe4f5247374a546bc03afc39e06ae273fa8d16a68cb6a594d1f.
[taskcluster 2025-01-27 12:12:43.639Z] === Task Starting ===
[setup 2025-01-27T12:12:49.835Z] run-task started in /builds/worker
...(146 lines hidden)...
[task 2025-01-27T12:13:11.106Z] Downloading scikit_learn-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
[task 2025-01-27T12:13:11.225Z] Collecting scipy==1.14.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 110))
[task 2025-01-27T12:13:11.236Z] Downloading scipy-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
[task 2025-01-27T12:13:11.298Z] Collecting sentencepiece==0.1.99 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 114))
[task 2025-01-27T12:13:11.309Z] Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.7 kB)
[task 2025-01-27T12:13:11.328Z] Collecting simalign==0.4 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 116))
[task 2025-01-27T12:13:11.338Z] Downloading simalign-0.4-py3-none-any.whl.metadata (6.2 kB)
[task 2025-01-27T12:13:11.359Z] Collecting six==1.16.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 118))
[task 2025-01-27T12:13:11.368Z] Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
[task 2025-01-27T12:13:11.392Z] Collecting sympy==1.12.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 122))
[task 2025-01-27T12:13:11.402Z] Downloading sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
[task 2025-01-27T12:13:11.422Z] Collecting tabulate==0.9.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 124))
[task 2025-01-27T12:13:11.431Z] Downloading tabulate-0.9.0-py3-none-any.whl.metadata (34 kB)
[task 2025-01-27T12:13:11.452Z] Collecting threadpoolctl==3.5.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 126))
[task 2025-01-27T12:13:11.461Z] Downloading threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB)
[task 2025-01-27T12:13:11.626Z] Collecting tokenizers==0.19.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 128))
[task 2025-01-27T12:13:11.636Z] Downloading tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
[task 2025-01-27T12:13:11.687Z] Collecting torch==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:11.697Z] Downloading torch-2.2.2-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
[task 2025-01-27T12:13:11.744Z] Collecting tqdm==4.66.4 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 132))
[task 2025-01-27T12:13:11.754Z] Downloading tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
[task 2025-01-27T12:13:11.802Z] Collecting transformers==4.42.3 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 137))
[task 2025-01-27T12:13:11.813Z] Downloading transformers-4.42.3-py3-none-any.whl.metadata (43 kB)
[task 2025-01-27T12:13:12.089Z] Collecting typing-extensions==4.12.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 139))
[task 2025-01-27T12:13:12.098Z] Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
[task 2025-01-27T12:13:12.115Z] Collecting typo==0.1.5 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 143))
[task 2025-01-27T12:13:12.124Z] Downloading typo-0.1.5.tar.gz (7.0 kB)
[task 2025-01-27T12:13:12.133Z] Preparing metadata (setup.py): started
[task 2025-01-27T12:13:12.311Z] Preparing metadata (setup.py): finished with status 'done'
[task 2025-01-27T12:13:12.343Z] Collecting urllib3==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 145))
[task 2025-01-27T12:13:12.352Z] Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
[task 2025-01-27T12:13:12.374Z] Collecting wcwidth==0.2.13 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 147))
[task 2025-01-27T12:13:12.383Z] Downloading wcwidth-0.2.13-py2.py3-none-any.whl.metadata (14 kB)
[task 2025-01-27T12:13:12.403Z] Collecting zhon==2.0.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 149))
[task 2025-01-27T12:13:12.412Z] Downloading zhon-2.0.2-py3-none-any.whl.metadata (3.4 kB)
[task 2025-01-27T12:13:12.629Z] Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.638Z] Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:12.658Z] Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.667Z] Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:12.689Z] Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.698Z] Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:12.718Z] Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.727Z] Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:12.748Z] Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.757Z] Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:12.776Z] Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.785Z] Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:12.806Z] Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.815Z] Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:12.835Z] Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.844Z] Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:12.864Z] Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.872Z] Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:12.892Z] Collecting nvidia-nccl-cu12==2.19.3 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.901Z] Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl.metadata (1.8 kB)
[task 2025-01-27T12:13:12.921Z] Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.931Z] Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)
[task 2025-01-27T12:13:12.951Z] Collecting triton==2.2.0 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:12.961Z] Downloading triton-2.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.4 kB)
[task 2025-01-27T12:13:13.071Z] Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:13.079Z] Downloading nvidia_nvjitlink_cu12-12.8.61-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
[task 2025-01-27T12:13:13.116Z] Downloading blessed-1.20.0-py2.py3-none-any.whl (58 kB)
[task 2025-01-27T12:13:13.127Z] Downloading certifi-2024.6.2-py3-none-any.whl (164 kB)
[task 2025-01-27T12:13:13.145Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2025-01-27T12:13:13.156Z] Downloading click-8.1.7-py3-none-any.whl (97 kB)
[task 2025-01-27T12:13:13.167Z] Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
[task 2025-01-27T12:13:13.179Z] Downloading enlighten-1.10.1-py2.py3-none-any.whl (53 kB)
[task 2025-01-27T12:13:13.190Z] Downloading filelock-3.15.4-py3-none-any.whl (16 kB)
[task 2025-01-27T12:13:13.201Z] Downloading fsspec-2024.6.1-py3-none-any.whl (177 kB)
[task 2025-01-27T12:13:13.214Z] Downloading hanzidentifier-1.2.0-py3-none-any.whl (4.8 kB)
[task 2025-01-27T12:13:13.226Z] Downloading huggingface_hub-0.23.4-py3-none-any.whl (402 kB)
[task 2025-01-27T12:13:13.239Z] Downloading idna-3.7-py3-none-any.whl (66 kB)
[task 2025-01-27T12:13:13.251Z] Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
[task 2025-01-27T12:13:13.262Z] Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
[task 2025-01-27T12:13:13.275Z] Downloading latexcodec-3.0.0-py3-none-any.whl (18 kB)
[task 2025-01-27T12:13:13.286Z] Downloading lxml-5.3.0-cp310-cp310-manylinux_2_28_x86_64.whl (5.0 MB)
[task 2025-01-27T12:13:13.316Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 180.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.325Z] Downloading MarkupSafe-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
[task 2025-01-27T12:13:13.336Z] Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
[task 2025-01-27T12:13:13.343Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 79.9 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.355Z] Downloading mtdata-0.4.1-py3-none-any.whl (819 kB)
[task 2025-01-27T12:13:13.363Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.3/819.3 kB 129.1 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.372Z] Downloading networkx-3.3-py3-none-any.whl (1.7 MB)
[task 2025-01-27T12:13:13.382Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 204.0 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.391Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2025-01-27T12:13:13.466Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 250.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.479Z] Downloading OpenCC-1.1.9-cp310-cp310-manylinux2014_x86_64.whl (1.7 MB)
[task 2025-01-27T12:13:13.489Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 193.1 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.498Z] Downloading packaging-24.1-py3-none-any.whl (53 kB)
[task 2025-01-27T12:13:13.511Z] Downloading portalocker-2.3.0-py2.py3-none-any.whl (15 kB)
[task 2025-01-27T12:13:13.524Z] Downloading prefixed-0.7.1-py2.py3-none-any.whl (13 kB)
[task 2025-01-27T12:13:13.535Z] Downloading psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
[task 2025-01-27T12:13:13.548Z] Downloading pybtex-0.24.0-py2.py3-none-any.whl (561 kB)
[task 2025-01-27T12:13:13.557Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 561.4/561.4 kB 65.0 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.566Z] Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
[task 2025-01-27T12:13:13.574Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 94.4 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.585Z] Downloading regex-2024.5.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (775 kB)
[task 2025-01-27T12:13:13.593Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 775.1/775.1 kB 119.1 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.602Z] Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2025-01-27T12:13:13.613Z] Downloading ruamel.yaml-0.18.6-py3-none-any.whl (117 kB)
[task 2025-01-27T12:13:13.625Z] Downloading ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (526 kB)
[task 2025-01-27T12:13:13.632Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 526.7/526.7 kB 69.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.642Z] Downloading sacrebleu-2.4.2-py3-none-any.whl (106 kB)
[task 2025-01-27T12:13:13.657Z] Downloading sacremoses-0.1.1-py3-none-any.whl (897 kB)
[task 2025-01-27T12:13:13.665Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 897.5/897.5 kB 121.8 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.677Z] Downloading safetensors-0.4.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
[task 2025-01-27T12:13:13.689Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 111.1 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.698Z] Downloading scikit_learn-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.3 MB)
[task 2025-01-27T12:13:13.753Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 252.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.766Z] Downloading scipy-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.1 MB)
[task 2025-01-27T12:13:13.968Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.1/41.1 MB 206.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:13.979Z] Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
[task 2025-01-27T12:13:14.036Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 19.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:14.045Z] Downloading simalign-0.4-py3-none-any.whl (8.1 kB)
[task 2025-01-27T12:13:14.056Z] Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
[task 2025-01-27T12:13:14.069Z] Downloading sympy-1.12.1-py3-none-any.whl (5.7 MB)
[task 2025-01-27T12:13:14.097Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 222.0 MB/s eta 0:00:00
[task 2025-01-27T12:13:14.106Z] Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
[task 2025-01-27T12:13:14.118Z] Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
[task 2025-01-27T12:13:14.136Z] Downloading tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
[task 2025-01-27T12:13:14.155Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 209.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:14.167Z] Downloading torch-2.2.2-cp310-cp310-manylinux1_x86_64.whl (755.5 MB)
[task 2025-01-27T12:13:19.260Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 755.5/755.5 MB 80.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:19.269Z] Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
[task 2025-01-27T12:13:19.319Z] Downloading transformers-4.42.3-py3-none-any.whl (9.3 MB)
[task 2025-01-27T12:13:19.377Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 MB 165.8 MB/s eta 0:00:00
[task 2025-01-27T12:13:19.386Z] Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
[task 2025-01-27T12:13:19.397Z] Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
[task 2025-01-27T12:13:19.409Z] Downloading wcwidth-0.2.13-py2.py3-none-any.whl (34 kB)
[task 2025-01-27T12:13:19.420Z] Downloading zhon-2.0.2-py3-none-any.whl (83 kB)
[task 2025-01-27T12:13:19.435Z] Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
[task 2025-01-27T12:13:22.314Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 113.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.325Z] Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
[task 2025-01-27T12:13:22.393Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 216.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.404Z] Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
[task 2025-01-27T12:13:22.524Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 199.3 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.536Z] Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
[task 2025-01-27T12:13:22.565Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 21.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.577Z] Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
[task 2025-01-27T12:13:27.697Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 77.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:27.709Z] Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
[task 2025-01-27T12:13:28.447Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 165.3 MB/s eta 0:00:00
[task 2025-01-27T12:13:28.459Z] Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
[task 2025-01-27T12:13:28.743Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 200.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:28.759Z] Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
[task 2025-01-27T12:13:29.625Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 143.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:29.639Z] Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
[task 2025-01-27T12:13:30.847Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 162.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:30.858Z] Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)
[task 2025-01-27T12:13:32.008Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 MB 144.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:32.017Z] Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
[task 2025-01-27T12:13:32.034Z] Downloading triton-2.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (167.9 MB)
[task 2025-01-27T12:13:33.115Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.9/167.9 MB 155.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:33.127Z] Downloading nvidia_nvjitlink_cu12-12.8.61-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (39.2 MB)
[task 2025-01-27T12:13:33.326Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.2/39.2 MB 199.0 MB/s eta 0:00:00
[task 2025-01-27T12:13:40.746Z] Building wheels for collected packages: typo, opustrainer
[task 2025-01-27T12:13:40.747Z] Building wheel for typo (setup.py): started
[task 2025-01-27T12:13:40.961Z] Building wheel for typo (setup.py): finished with status 'done'
[task 2025-01-27T12:13:40.962Z] Created wheel for typo: filename=typo-0.1.5-py3-none-any.whl size=6878 sha256=4c340de47e1b8ff40b93d2ffc0579b708dbc507ba2b17c74359200fb5a5c1703
[task 2025-01-27T12:13:40.962Z] Stored in directory: /builds/worker/.cache/pip/wheels/2e/2f/73/60e0ce42d1375a386b9171a37cd5536e173ad950a98e7dc6b1
[task 2025-01-27T12:13:40.969Z] Building wheel for opustrainer (pyproject.toml): started
[task 2025-01-27T12:13:41.231Z] Building wheel for opustrainer (pyproject.toml): finished with status 'done'
[task 2025-01-27T12:13:41.232Z] Created wheel for opustrainer: filename=opustrainer-0.3-py3-none-any.whl size=45114 sha256=5ec39bb273da3437957daea51af3d96512a66ef0d1450a18396bc6224b0511d8
[task 2025-01-27T12:13:41.232Z] Stored in directory: /builds/worker/.cache/pip/wheels/09/01/1c/d82c1698dc71385b0bf6eae4f433e85a02f89f99eec2e21072
[task 2025-01-27T12:13:41.235Z] Successfully built typo opustrainer
[task 2025-01-27T12:13:41.862Z] Installing collected packages: wcwidth, typo, sentencepiece, prefixed, portalocker, opencc, mpmath, zhon, urllib3, typing-extensions, tqdm, threadpoolctl, tabulate, sympy, six, safetensors, ruamel-yaml-clib, regex, pyyaml, psutil, packaging, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, markupsafe, lxml, latexcodec, joblib, idna, fsspec, filelock, colorama, click, charset-normalizer, certifi, triton, scipy, sacremoses, sacrebleu, ruamel-yaml, requests, pybtex, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, hanzidentifier, blessed, scikit-learn, opustrainer, nvidia-cusolver-cu12, huggingface-hub, enlighten, torch, tokenizers, mtdata, transformers, simalign
[task 2025-01-27T12:14:42.736Z] Successfully installed blessed-1.20.0 certifi-2024.6.2 charset-normalizer-3.3.2 click-8.1.7 colorama-0.4.6 enlighten-1.10.1 filelock-3.15.4 fsspec-2024.6.1 hanzidentifier-1.2.0 huggingface-hub-0.23.4 idna-3.7 jinja2-3.1.4 joblib-1.4.2 latexcodec-3.0.0 lxml-5.3.0 markupsafe-2.1.5 mpmath-1.3.0 mtdata-0.4.1 networkx-3.3 numpy-1.26.4 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.8.61 nvidia-nvtx-cu12-12.1.105 opencc-1.1.9 opustrainer-0.3 packaging-24.1 portalocker-2.3.0 prefixed-0.7.1 psutil-6.0.0 pybtex-0.24.0 pyyaml-6.0.1 regex-2024.5.15 requests-2.31.0 ruamel-yaml-0.18.6 ruamel-yaml-clib-0.2.8 sacrebleu-2.4.2 sacremoses-0.1.1 safetensors-0.4.3 scikit-learn-1.5.0 scipy-1.14.0 sentencepiece-0.1.99 simalign-0.4 six-1.16.0 sympy-1.12.1 tabulate-0.9.0 threadpoolctl-3.5.0 tokenizers-0.19.1 torch-2.2.2 tqdm-4.66.4 transformers-4.42.3 triton-2.2.0 typing-extensions-4.12.2 typo-0.1.5 urllib3-2.2.2 wcwidth-0.2.13 zhon-2.0.2
[task 2025-01-27T12:14:43.628Z] [download-mono] Dataset: opus_tldr-pages/v2023-08-29
[task 2025-01-27T12:14:43.628Z] [download-mono] Language: en
[task 2025-01-27T12:14:43.628Z] [download-mono] Max Sentences: 500
[task 2025-01-27T12:14:43.628Z] [download-mono] Mininmum Fluency Threshold: 0.8
[task 2025-01-27T12:14:43.628Z] [download-mono] Artifacts: /builds/worker/artifacts
[task 2025-01-27T12:14:43.628Z] [download-mono] File Destination: /builds/worker/artifacts/tldr-pages_v2023-08-29.en.zst
[task 2025-01-27T12:14:43.628Z] [download-mono] Downloading OPUS monolingual data
[task 2025-01-27T12:14:43.628Z] [download-mono] https://object.pouta.csc.fi/OPUS-tldr-pages/v2023-08-29/mono/en.txt.gz
[task 2025-01-27T12:14:43.628Z] [download-mono] URL: https://object.pouta.csc.fi/OPUS-tldr-pages/v2023-08-29/mono/en.txt.gz
[task 2025-01-27T12:14:45.760Z] [downloads] Download size: 446,937 bytes
[task 2025-01-27T12:14:45.815Z] [downloads] 5% downloaded (24576/446937 bytes)
[task 2025-01-27T12:14:45.975Z] [downloads] 11% downloaded (49152/446937 bytes)
[task 2025-01-27T12:14:45.978Z] [downloads] 16% downloaded (73728/446937 bytes)
[task 2025-01-27T12:14:45.979Z] [downloads] 20% downloaded (90112/446937 bytes)
[task 2025-01-27T12:14:46.137Z] [downloads] 26% downloaded (114688/446937 bytes)
[task 2025-01-27T12:14:46.139Z] [downloads] 31% downloaded (139264/446937 bytes)
[task 2025-01-27T12:14:46.141Z] [downloads] 37% downloaded (163840/446937 bytes)
[task 2025-01-27T12:14:46.143Z] [downloads] 40% downloaded (180224/446937 bytes)
[task 2025-01-27T12:14:46.245Z] [downloads] 46% downloaded (204800/446937 bytes)
[task 2025-01-27T12:14:46.247Z] [downloads] 51% downloaded (229376/446937 bytes)
[task 2025-01-27T12:14:46.299Z] [downloads] 57% downloaded (253952/446937 bytes)
[task 2025-01-27T12:14:46.301Z] [downloads] 60% downloaded (270336/446937 bytes)
[task 2025-01-27T12:14:46.303Z] [downloads] 66% downloaded (294912/446937 bytes)
[task 2025-01-27T12:14:46.305Z] [downloads] 71% downloaded (319488/446937 bytes)
[task 2025-01-27T12:14:46.308Z] [downloads] 75% downloaded (335872/446937 bytes)
[task 2025-01-27T12:14:46.313Z] [downloads] 81% downloaded (360448/446937 bytes)
[task 2025-01-27T12:14:46.317Z] [downloads] 86% downloaded (385024/446937 bytes)
[task 2025-01-27T12:14:46.320Z] [downloads] 92% downloaded (409600/446937 bytes)
[task 2025-01-27T12:14:46.406Z] [downloads] 95% downloaded (425984/446937 bytes)
[task 2025-01-27T12:14:46.409Z] [downloads] 100% downloaded - Download finished.
[taskcluster 2025-01-27 12:14:53.185Z] === Task Finished ===
[taskcluster 2025-01-27 12:14:53.298Z] Successful task run with exit code: 0 completed in 196.203 seconds
Loading