feat: translations integration tests #210
+1,005
−69
Open
stage-taskcluster / translations-dataset-news-crawl-news_2008-ru
succeeded
Jan 27, 2025 in 9m 52s
Stage-TC (issue_comment)
Fetch news-crawl dataset for {src_locale}
Details
View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster
Task Status
Started: 2025-01-27T12:11:51.658Z
Resolved: 2025-01-27T12:15:02.460Z
Task Execution Time: 3 minutes, 10 seconds, 802 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0
Artifacts
- public/build/news_2008.ru.zst
- public/logs/live_backing.log
- public/logs/live.log
[taskcluster 2025-01-27 12:11:51.785Z] Task ID: ZD8V6jtgSL2eqyYtfvjysg
[taskcluster 2025-01-27 12:11:51.786Z] Worker ID: 8157186502250121586
[taskcluster 2025-01-27 12:11:51.786Z] Worker Group: us-central1-f
[taskcluster 2025-01-27 12:11:51.786Z] Worker Node Type: projects/474530013412/machineTypes/n2-highmem-32
[taskcluster 2025-01-27 12:11:51.786Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2025-01-27 12:11:51.786Z] Worker Version: 38.0.5
[taskcluster 2025-01-27 12:11:51.786Z] Public IP: 34.29.26.36
[taskcluster 2025-01-27 12:11:51.786Z] Hostname: translations-1-b-linux-large-gcp-skxvkah8qqu9vbheynzqkq
[taskcluster 2025-01-27 12:11:51.786Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-LaTmU8ILTKmjq-SGho18mA" -> /builds/worker/checkouts
[taskcluster 2025-01-27 12:11:54.225Z] Downloading artifact "public/image.tar.zst" from task ID: DUNjdZeFTIScNP4MGcgXGQ.
[taskcluster 2025-01-27 12:11:59.227Z] Download Progress: 34.10%
[taskcluster 2025-01-27 12:12:04.227Z] Download Progress: 69.10%
[taskcluster 2025-01-27 12:12:08.671Z] Downloaded artifact successfully.
[taskcluster 2025-01-27 12:12:08.672Z] Downloaded 761.177 mb
[taskcluster 2025-01-27 12:12:08.673Z] Decompressing downloaded image
[taskcluster 2025-01-27 12:12:15.117Z] Loading docker image from downloaded archive.
[taskcluster 2025-01-27 12:12:52.020Z] Image 'public/image.tar.zst' from task 'DUNjdZeFTIScNP4MGcgXGQ' loaded. Using image ID sha256:a6cdc387f7c79fe4f5247374a546bc03afc39e06ae273fa8d16a68cb6a594d1f.
[taskcluster 2025-01-27 12:12:52.443Z] === Task Starting ===
[setup 2025-01-27T12:13:00.454Z] run-task started in /builds/worker
...(146 lines hidden)...
[task 2025-01-27T12:13:19.409Z] Downloading scikit_learn-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
[task 2025-01-27T12:13:19.529Z] Collecting scipy==1.14.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 110))
[task 2025-01-27T12:13:19.542Z] Downloading scipy-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
[task 2025-01-27T12:13:19.607Z] Collecting sentencepiece==0.1.99 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 114))
[task 2025-01-27T12:13:19.621Z] Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.7 kB)
[task 2025-01-27T12:13:19.659Z] Collecting simalign==0.4 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 116))
[task 2025-01-27T12:13:19.672Z] Downloading simalign-0.4-py3-none-any.whl.metadata (6.2 kB)
[task 2025-01-27T12:13:19.695Z] Collecting six==1.16.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 118))
[task 2025-01-27T12:13:19.707Z] Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
[task 2025-01-27T12:13:19.734Z] Collecting sympy==1.12.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 122))
[task 2025-01-27T12:13:19.748Z] Downloading sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
[task 2025-01-27T12:13:19.770Z] Collecting tabulate==0.9.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 124))
[task 2025-01-27T12:13:19.782Z] Downloading tabulate-0.9.0-py3-none-any.whl.metadata (34 kB)
[task 2025-01-27T12:13:19.805Z] Collecting threadpoolctl==3.5.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 126))
[task 2025-01-27T12:13:19.817Z] Downloading threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB)
[task 2025-01-27T12:13:19.985Z] Collecting tokenizers==0.19.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 128))
[task 2025-01-27T12:13:19.999Z] Downloading tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
[task 2025-01-27T12:13:20.052Z] Collecting torch==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:20.067Z] Downloading torch-2.2.2-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
[task 2025-01-27T12:13:20.119Z] Collecting tqdm==4.66.4 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 132))
[task 2025-01-27T12:13:20.132Z] Downloading tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
[task 2025-01-27T12:13:20.185Z] Collecting transformers==4.42.3 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 137))
[task 2025-01-27T12:13:20.199Z] Downloading transformers-4.42.3-py3-none-any.whl.metadata (43 kB)
[task 2025-01-27T12:13:20.467Z] Collecting typing-extensions==4.12.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 139))
[task 2025-01-27T12:13:20.480Z] Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
[task 2025-01-27T12:13:20.516Z] Collecting typo==0.1.5 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 143))
[task 2025-01-27T12:13:20.530Z] Downloading typo-0.1.5.tar.gz (7.0 kB)
[task 2025-01-27T12:13:20.538Z] Preparing metadata (setup.py): started
[task 2025-01-27T12:13:20.703Z] Preparing metadata (setup.py): finished with status 'done'
[task 2025-01-27T12:13:20.736Z] Collecting urllib3==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 145))
[task 2025-01-27T12:13:20.749Z] Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
[task 2025-01-27T12:13:20.773Z] Collecting wcwidth==0.2.13 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 147))
[task 2025-01-27T12:13:20.785Z] Downloading wcwidth-0.2.13-py2.py3-none-any.whl.metadata (14 kB)
[task 2025-01-27T12:13:20.806Z] Collecting zhon==2.0.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 149))
[task 2025-01-27T12:13:20.819Z] Downloading zhon-2.0.2-py3-none-any.whl.metadata (3.4 kB)
[task 2025-01-27T12:13:21.032Z] Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.045Z] Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:21.066Z] Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.078Z] Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:21.101Z] Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.113Z] Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:21.135Z] Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.148Z] Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:21.170Z] Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.183Z] Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:21.204Z] Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.217Z] Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:21.239Z] Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.251Z] Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
[task 2025-01-27T12:13:21.276Z] Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.288Z] Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:21.311Z] Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.324Z] Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
[task 2025-01-27T12:13:21.346Z] Collecting nvidia-nccl-cu12==2.19.3 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.359Z] Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl.metadata (1.8 kB)
[task 2025-01-27T12:13:21.380Z] Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.393Z] Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)
[task 2025-01-27T12:13:21.415Z] Collecting triton==2.2.0 (from torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.428Z] Downloading triton-2.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.4 kB)
[task 2025-01-27T12:13:21.553Z] Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.2.2->-r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 130))
[task 2025-01-27T12:13:21.566Z] Downloading nvidia_nvjitlink_cu12-12.8.61-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
[task 2025-01-27T12:13:21.605Z] Downloading blessed-1.20.0-py2.py3-none-any.whl (58 kB)
[task 2025-01-27T12:13:21.621Z] Downloading certifi-2024.6.2-py3-none-any.whl (164 kB)
[task 2025-01-27T12:13:21.649Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2025-01-27T12:13:21.667Z] Downloading click-8.1.7-py3-none-any.whl (97 kB)
[task 2025-01-27T12:13:21.682Z] Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
[task 2025-01-27T12:13:21.697Z] Downloading enlighten-1.10.1-py2.py3-none-any.whl (53 kB)
[task 2025-01-27T12:13:21.711Z] Downloading filelock-3.15.4-py3-none-any.whl (16 kB)
[task 2025-01-27T12:13:21.726Z] Downloading fsspec-2024.6.1-py3-none-any.whl (177 kB)
[task 2025-01-27T12:13:21.744Z] Downloading hanzidentifier-1.2.0-py3-none-any.whl (4.8 kB)
[task 2025-01-27T12:13:21.759Z] Downloading huggingface_hub-0.23.4-py3-none-any.whl (402 kB)
[task 2025-01-27T12:13:21.785Z] Downloading idna-3.7-py3-none-any.whl (66 kB)
[task 2025-01-27T12:13:21.799Z] Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
[task 2025-01-27T12:13:21.814Z] Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
[task 2025-01-27T12:13:21.832Z] Downloading latexcodec-3.0.0-py3-none-any.whl (18 kB)
[task 2025-01-27T12:13:21.847Z] Downloading lxml-5.3.0-cp310-cp310-manylinux_2_28_x86_64.whl (5.0 MB)
[task 2025-01-27T12:13:21.893Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 113.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:21.906Z] Downloading MarkupSafe-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
[task 2025-01-27T12:13:21.922Z] Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
[task 2025-01-27T12:13:21.929Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 87.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:21.945Z] Downloading mtdata-0.4.1-py3-none-any.whl (819 kB)
[task 2025-01-27T12:13:21.952Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.3/819.3 kB 145.8 MB/s eta 0:00:00
[task 2025-01-27T12:13:21.967Z] Downloading networkx-3.3-py3-none-any.whl (1.7 MB)
[task 2025-01-27T12:13:21.976Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 223.9 MB/s eta 0:00:00
[task 2025-01-27T12:13:21.989Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2025-01-27T12:13:22.094Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 176.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.108Z] Downloading OpenCC-1.1.9-cp310-cp310-manylinux2014_x86_64.whl (1.7 MB)
[task 2025-01-27T12:13:22.125Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 102.9 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.138Z] Downloading packaging-24.1-py3-none-any.whl (53 kB)
[task 2025-01-27T12:13:22.154Z] Downloading portalocker-2.3.0-py2.py3-none-any.whl (15 kB)
[task 2025-01-27T12:13:22.169Z] Downloading prefixed-0.7.1-py2.py3-none-any.whl (13 kB)
[task 2025-01-27T12:13:22.186Z] Downloading psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
[task 2025-01-27T12:13:22.204Z] Downloading pybtex-0.24.0-py2.py3-none-any.whl (561 kB)
[task 2025-01-27T12:13:22.211Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 561.4/561.4 kB 69.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.224Z] Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
[task 2025-01-27T12:13:22.231Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 123.3 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.245Z] Downloading regex-2024.5.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (775 kB)
[task 2025-01-27T12:13:22.252Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 775.1/775.1 kB 140.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.264Z] Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2025-01-27T12:13:22.279Z] Downloading ruamel.yaml-0.18.6-py3-none-any.whl (117 kB)
[task 2025-01-27T12:13:22.298Z] Downloading ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (526 kB)
[task 2025-01-27T12:13:22.303Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 526.7/526.7 kB 84.4 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.317Z] Downloading sacrebleu-2.4.2-py3-none-any.whl (106 kB)
[task 2025-01-27T12:13:22.335Z] Downloading sacremoses-0.1.1-py3-none-any.whl (897 kB)
[task 2025-01-27T12:13:22.343Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 897.5/897.5 kB 123.4 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.359Z] Downloading safetensors-0.4.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
[task 2025-01-27T12:13:22.368Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 170.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.388Z] Downloading scikit_learn-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.3 MB)
[task 2025-01-27T12:13:22.468Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 172.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.482Z] Downloading scipy-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.1 MB)
[task 2025-01-27T12:13:22.682Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.1/41.1 MB 207.8 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.704Z] Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
[task 2025-01-27T12:13:22.758Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 20.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.771Z] Downloading simalign-0.4-py3-none-any.whl (8.1 kB)
[task 2025-01-27T12:13:22.787Z] Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
[task 2025-01-27T12:13:22.802Z] Downloading sympy-1.12.1-py3-none-any.whl (5.7 MB)
[task 2025-01-27T12:13:22.833Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 199.2 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.846Z] Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
[task 2025-01-27T12:13:22.860Z] Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
[task 2025-01-27T12:13:22.876Z] Downloading tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
[task 2025-01-27T12:13:22.897Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 181.3 MB/s eta 0:00:00
[task 2025-01-27T12:13:22.913Z] Downloading torch-2.2.2-cp310-cp310-manylinux1_x86_64.whl (755.5 MB)
[task 2025-01-27T12:13:27.995Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 755.5/755.5 MB 78.9 MB/s eta 0:00:00
[task 2025-01-27T12:13:28.009Z] Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
[task 2025-01-27T12:13:28.026Z] Downloading transformers-4.42.3-py3-none-any.whl (9.3 MB)
[task 2025-01-27T12:13:28.084Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 MB 166.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:28.097Z] Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
[task 2025-01-27T12:13:28.111Z] Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
[task 2025-01-27T12:13:28.126Z] Downloading wcwidth-0.2.13-py2.py3-none-any.whl (34 kB)
[task 2025-01-27T12:13:28.142Z] Downloading zhon-2.0.2-py3-none-any.whl (83 kB)
[task 2025-01-27T12:13:28.158Z] Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
[task 2025-01-27T12:13:30.858Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 120.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:30.876Z] Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
[task 2025-01-27T12:13:30.959Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 173.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:30.975Z] Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
[task 2025-01-27T12:13:31.099Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 194.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:31.112Z] Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
[task 2025-01-27T12:13:31.138Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 23.9 MB/s eta 0:00:00
[task 2025-01-27T12:13:31.153Z] Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
[task 2025-01-27T12:13:36.012Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 82.0 MB/s eta 0:00:00
[task 2025-01-27T12:13:36.027Z] Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
[task 2025-01-27T12:13:36.919Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 136.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:36.932Z] Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
[task 2025-01-27T12:13:37.208Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 206.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:37.223Z] Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
[task 2025-01-27T12:13:37.970Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 166.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:37.985Z] Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
[task 2025-01-27T12:13:39.230Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 157.6 MB/s eta 0:00:00
[task 2025-01-27T12:13:39.246Z] Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)
[task 2025-01-27T12:13:40.230Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 MB 169.1 MB/s eta 0:00:00
[task 2025-01-27T12:13:40.243Z] Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
[task 2025-01-27T12:13:40.366Z] Downloading triton-2.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (167.9 MB)
[task 2025-01-27T12:13:41.468Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.9/167.9 MB 152.7 MB/s eta 0:00:00
[task 2025-01-27T12:13:41.482Z] Downloading nvidia_nvjitlink_cu12-12.8.61-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (39.2 MB)
[task 2025-01-27T12:13:41.712Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.2/39.2 MB 171.5 MB/s eta 0:00:00
[task 2025-01-27T12:13:49.033Z] Building wheels for collected packages: typo, opustrainer
[task 2025-01-27T12:13:49.034Z] Building wheel for typo (setup.py): started
[task 2025-01-27T12:13:49.229Z] Building wheel for typo (setup.py): finished with status 'done'
[task 2025-01-27T12:13:49.230Z] Created wheel for typo: filename=typo-0.1.5-py3-none-any.whl size=6878 sha256=d47cb595f47fc4caa00530b59db1c06e96268913660d445ab712f9b7e752f1db
[task 2025-01-27T12:13:49.230Z] Stored in directory: /builds/worker/.cache/pip/wheels/2e/2f/73/60e0ce42d1375a386b9171a37cd5536e173ad950a98e7dc6b1
[task 2025-01-27T12:13:49.237Z] Building wheel for opustrainer (pyproject.toml): started
[task 2025-01-27T12:13:49.479Z] Building wheel for opustrainer (pyproject.toml): finished with status 'done'
[task 2025-01-27T12:13:49.480Z] Created wheel for opustrainer: filename=opustrainer-0.3-py3-none-any.whl size=45114 sha256=d1960f7c346e8c1bdf6ddf8cbd9b54184eb7263df97740d99c93fd9a757e83ca
[task 2025-01-27T12:13:49.481Z] Stored in directory: /builds/worker/.cache/pip/wheels/09/01/1c/d82c1698dc71385b0bf6eae4f433e85a02f89f99eec2e21072
[task 2025-01-27T12:13:49.483Z] Successfully built typo opustrainer
[task 2025-01-27T12:13:50.084Z] Installing collected packages: wcwidth, typo, sentencepiece, prefixed, portalocker, opencc, mpmath, zhon, urllib3, typing-extensions, tqdm, threadpoolctl, tabulate, sympy, six, safetensors, ruamel-yaml-clib, regex, pyyaml, psutil, packaging, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, markupsafe, lxml, latexcodec, joblib, idna, fsspec, filelock, colorama, click, charset-normalizer, certifi, triton, scipy, sacremoses, sacrebleu, ruamel-yaml, requests, pybtex, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, hanzidentifier, blessed, scikit-learn, opustrainer, nvidia-cusolver-cu12, huggingface-hub, enlighten, torch, tokenizers, mtdata, transformers, simalign
[task 2025-01-27T12:14:50.109Z] Successfully installed blessed-1.20.0 certifi-2024.6.2 charset-normalizer-3.3.2 click-8.1.7 colorama-0.4.6 enlighten-1.10.1 filelock-3.15.4 fsspec-2024.6.1 hanzidentifier-1.2.0 huggingface-hub-0.23.4 idna-3.7 jinja2-3.1.4 joblib-1.4.2 latexcodec-3.0.0 lxml-5.3.0 markupsafe-2.1.5 mpmath-1.3.0 mtdata-0.4.1 networkx-3.3 numpy-1.26.4 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.8.61 nvidia-nvtx-cu12-12.1.105 opencc-1.1.9 opustrainer-0.3 packaging-24.1 portalocker-2.3.0 prefixed-0.7.1 psutil-6.0.0 pybtex-0.24.0 pyyaml-6.0.1 regex-2024.5.15 requests-2.31.0 ruamel-yaml-0.18.6 ruamel-yaml-clib-0.2.8 sacrebleu-2.4.2 sacremoses-0.1.1 safetensors-0.4.3 scikit-learn-1.5.0 scipy-1.14.0 sentencepiece-0.1.99 simalign-0.4 six-1.16.0 sympy-1.12.1 tabulate-0.9.0 threadpoolctl-3.5.0 tokenizers-0.19.1 torch-2.2.2 tqdm-4.66.4 transformers-4.42.3 triton-2.2.0 typing-extensions-4.12.2 typo-0.1.5 urllib3-2.2.2 wcwidth-0.2.13 zhon-2.0.2
[task 2025-01-27T12:14:50.871Z] [download-mono] Dataset: news-crawl_news.2008
[task 2025-01-27T12:14:50.871Z] [download-mono] Language: ru
[task 2025-01-27T12:14:50.871Z] [download-mono] Max Sentences: 500
[task 2025-01-27T12:14:50.871Z] [download-mono] Mininmum Fluency Threshold: 0.8
[task 2025-01-27T12:14:50.871Z] [download-mono] Artifacts: /builds/worker/artifacts
[task 2025-01-27T12:14:50.871Z] [download-mono] File Destination: /builds/worker/artifacts/news_2008.ru.zst
[task 2025-01-27T12:14:50.871Z] [download-mono] Downloading WMT newscrawl monolingual data
[task 2025-01-27T12:14:50.871Z] [download-mono] http://data.statmt.org/news-crawl/ru/news.2008.ru.shuffled.deduped.gz
[task 2025-01-27T12:14:50.871Z] [download-mono] URL: http://data.statmt.org/news-crawl/ru/news.2008.ru.shuffled.deduped.gz
[task 2025-01-27T12:14:52.718Z] [downloads] Download size: 2,312,968 bytes
[task 2025-01-27T12:14:53.037Z] [downloads] 5% downloaded (122880/2312968 bytes)
[task 2025-01-27T12:14:53.144Z] [downloads] 10% downloaded (237568/2312968 bytes)
[task 2025-01-27T12:14:53.247Z] [downloads] 15% downloaded (352256/2312968 bytes)
[task 2025-01-27T12:14:53.253Z] [downloads] 20% downloaded (466944/2312968 bytes)
[task 2025-01-27T12:14:53.259Z] [downloads] 25% downloaded (581632/2312968 bytes)
[task 2025-01-27T12:14:53.265Z] [downloads] 30% downloaded (696320/2312968 bytes)
[task 2025-01-27T12:14:53.359Z] [downloads] 35% downloaded (811008/2312968 bytes)
[task 2025-01-27T12:14:53.365Z] [downloads] 40% downloaded (925696/2312968 bytes)
[task 2025-01-27T12:14:53.371Z] [downloads] 45% downloaded (1048576/2312968 bytes)
[task 2025-01-27T12:14:53.377Z] [downloads] 50% downloaded (1163264/2312968 bytes)
[task 2025-01-27T12:14:53.383Z] [downloads] 55% downloaded (1277952/2312968 bytes)
[task 2025-01-27T12:14:53.461Z] [downloads] 60% downloaded (1392640/2312968 bytes)
[task 2025-01-27T12:14:53.467Z] [downloads] 65% downloaded (1507328/2312968 bytes)
[task 2025-01-27T12:14:53.474Z] [downloads] 70% downloaded (1622016/2312968 bytes)
[task 2025-01-27T12:14:53.480Z] [downloads] 75% downloaded (1736704/2312968 bytes)
[task 2025-01-27T12:14:53.486Z] [downloads] 80% downloaded (1851392/2312968 bytes)
[task 2025-01-27T12:14:53.492Z] [downloads] 85% downloaded (1966080/2312968 bytes)
[task 2025-01-27T12:14:53.499Z] [downloads] 90% downloaded (2088960/2312968 bytes)
[task 2025-01-27T12:14:53.505Z] [downloads] 95% downloaded (2203648/2312968 bytes)
[task 2025-01-27T12:14:53.513Z] [downloads] 100% downloaded - Download finished.
[taskcluster 2025-01-27 12:15:01.301Z] === Task Finished ===
[taskcluster 2025-01-27 12:15:01.711Z] Successful task run with exit code: 0 completed in 189.927 seconds
Loading