feat: translations integration tests #210
Merged
stage-taskcluster / translations-alignments-student-ru-en
succeeded
Jan 29, 2025 in 1h 58m 21s
Stage-TC (issue_comment)
alignments for student training corpus ru-en
Details
View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster
Task Status
Started: 2025-01-29T17:51:45.270Z
Resolved: 2025-01-29T17:54:24.216Z
Task Execution Time: 2 minutes, 38 seconds, 946 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0
Artifacts
- public/build/corpus.aln.zst
- public/build/corpus.tok-icu.en.zst
- public/build/corpus.tok-icu.ru.zst
- public/build/tmp/aln.fwd
- public/build/tmp/aln.fwd.aa
- public/build/tmp/aln.rev
- public/build/tmp/aln.rev.aa
- public/logs/live_backing.log
- public/logs/live.log
[taskcluster 2025-01-29T17:51:45.372Z] Worker Type (translations-1/b-linux-large-gcp-1tb-32-256-d2g) settings:
[taskcluster 2025-01-29T17:51:45.372Z] {
[taskcluster 2025-01-29T17:51:45.372Z] "config": {
[taskcluster 2025-01-29T17:51:45.372Z] "deploymentId": ""
[taskcluster 2025-01-29T17:51:45.372Z] },
[taskcluster 2025-01-29T17:51:45.372Z] "generic-worker": {
[taskcluster 2025-01-29T17:51:45.372Z] "config": {
[taskcluster 2025-01-29T17:51:45.372Z] "headlessTasks": true,
[taskcluster 2025-01-29T17:51:45.372Z] "runTasksAsCurrentUser": false
[taskcluster 2025-01-29T17:51:45.372Z] },
[taskcluster 2025-01-29T17:51:45.372Z] "engine": "multiuser",
[taskcluster 2025-01-29T17:51:45.372Z] "go-arch": "amd64",
[taskcluster 2025-01-29T17:51:45.372Z] "go-os": "linux",
[taskcluster 2025-01-29T17:51:45.372Z] "go-version": "go1.23.4",
[taskcluster 2025-01-29T17:51:45.372Z] "release": "https://github.com/taskcluster/taskcluster/releases/tag/v77.3.1",
[taskcluster 2025-01-29T17:51:45.372Z] "revision": "959a204190add062fe1217d14f2a0115ecd43fe8",
[taskcluster 2025-01-29T17:51:45.372Z] "source": "https://github.com/taskcluster/taskcluster/commits/959a204190add062fe1217d14f2a0115ecd43fe8",
[taskcluster 2025-01-29T17:51:45.372Z] "version": "77.3.1"
[taskcluster 2025-01-29T17:51:45.372Z] },
[taskcluster 2025-01-29T17:51:45.372Z] "image": "projects/taskcluster-imaging/global/images/gw-fxci-gcp-l1-2404-amd64-headless-googlecompute-2025-01-17",
...(202 lines hidden)...
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst resolved to 722788 bytes with sha256 3a20fe4e64f4727cad8e6216e169a1119020f22a6151e9cdb355ddab2403125f in 0.101s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst
Extracting /builds/worker/fetches/fast-align.tar.zst to /builds/worker/fetches
/builds/worker/fetches/fast-align.tar.zst extracted in 0.008s
Removing /builds/worker/fetches/fast-align.tar.zst
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/KQOb2v4STDiMyJbUPPgmmg/artifacts/public/build/corpus.priors resolved to 286222 bytes with sha256 70d87ddc51aff4f198af798d09e97d41666cf4df7bdb0aa65695113232c2f463 in 0.127s
Extracting /builds/worker/fetches/corpus.priors to /builds/worker/fetches
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V9BtfZroRBObdt4uXai6bw/artifacts/public/build/corpus.ru.zst resolved to 84107 bytes with sha256 2d5022b5c2414ecb6ead4f979fc6e9e1e93c0b27b25bf2dc9bf0263b54b5452c in 0.214s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V9BtfZroRBObdt4uXai6bw/artifacts/public/build/corpus.ru.zst
Extracting /builds/worker/fetches/corpus.ru.zst to /builds/worker/fetches
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V9BtfZroRBObdt4uXai6bw/artifacts/public/build/corpus.en.zst resolved to 19864 bytes with sha256 b79dadcbf4b30cdc1065de69126a4d6a3654fdfc74e2670fc791edbbcf9a0f1f in 0.232s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V9BtfZroRBObdt4uXai6bw/artifacts/public/build/corpus.en.zst
Extracting /builds/worker/fetches/corpus.en.zst to /builds/worker/fetches
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Z-GV1FPJQ0yv9xLiF1w4lg/artifacts/public/build/extract_lex.tar.zst resolved to 620576 bytes with sha256 9948dd20f0aff3239985abf0c5c1959ed60a78993e719ede8d87a0acee85eab5 in 0.264s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Z-GV1FPJQ0yv9xLiF1w4lg/artifacts/public/build/extract_lex.tar.zst
Extracting /builds/worker/fetches/extract_lex.tar.zst to /builds/worker/fetches
/builds/worker/fetches/extract_lex.tar.zst extracted in 0.007s
Removing /builds/worker/fetches/extract_lex.tar.zst
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V54HtUtPSou23YIBg1awvw/artifacts/public/build/marian.tar.zst resolved to 684172003 bytes with sha256 7839452e78344d844c731cc7ebb7775bd9d3ef0e3e68e80d6b3f920255bc5ef7 in 55.926s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V54HtUtPSou23YIBg1awvw/artifacts/public/build/marian.tar.zst
Extracting /builds/worker/fetches/marian.tar.zst to /builds/worker/fetches
/builds/worker/fetches/marian.tar.zst extracted in 5.245s
Removing /builds/worker/fetches/marian.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 61.250589272, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2025-01-29T17:52:52.553Z] finished fetching artifacts
[task 2025-01-29T17:52:52.553Z] executing ['bash', '-c', 'pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/alignments/requirements/alignments.txt && export BIN=$MOZ_FETCHES_DIR && export PATH=$PATH:$MOZ_FETCHES_DIR && export PYTHONPATH=$PYTHONPATH:$VCS_PATH && python3 $VCS_PATH/pipeline/alignments/align.py --corpus_src=$MOZ_FETCHES_DIR/corpus.ru.zst --corpus_trg=$MOZ_FETCHES_DIR/corpus.en.zst --output_path=$TASK_WORKDIR/artifacts/corpus.aln.zst --output_tokenized --tokenization=icu --priors_input_path=$MOZ_FETCHES_DIR/corpus.priors']
[task 2025-01-29T17:52:52.927Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2025-01-29T17:52:52.927Z] Defaulting to user installation because normal site-packages is not writeable
[task 2025-01-29T17:52:52.955Z] Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
[task 2025-01-29T17:52:53.119Z] Collecting pip
[task 2025-01-29T17:52:53.240Z] Downloading pip-25.0-py3-none-any.whl (1.8 MB)
[task 2025-01-29T17:52:53.342Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 18.7 MB/s eta 0:00:00
[task 2025-01-29T17:52:53.353Z] Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
[task 2025-01-29T17:52:53.622Z] Collecting setuptools
[task 2025-01-29T17:52:53.642Z] Downloading setuptools-75.8.0-py3-none-any.whl (1.2 MB)
[task 2025-01-29T17:52:53.648Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 427.1 MB/s eta 0:00:00
[task 2025-01-29T17:52:53.770Z] Installing collected packages: setuptools, pip
[task 2025-01-29T17:52:55.049Z] Successfully installed pip-25.0 setuptools-75.8.0
[task 2025-01-29T17:52:55.407Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2025-01-29T17:52:55.408Z] Defaulting to user installation because normal site-packages is not writeable
[task 2025-01-29T17:52:55.547Z] Collecting certifi==2024.7.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 7))
[task 2025-01-29T17:52:55.596Z] Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
[task 2025-01-29T17:52:55.686Z] Collecting charset-normalizer==3.3.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 9))
[task 2025-01-29T17:52:55.701Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
[task 2025-01-29T17:52:55.858Z] Collecting cython==3.0.9 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 11))
[task 2025-01-29T17:52:55.873Z] Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
[task 2025-01-29T17:52:55.890Z] Collecting eflomal==1.0.0b1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 13))
[task 2025-01-29T17:52:55.905Z] Downloading eflomal-1.0.0b1.tar.gz (102 kB)
[task 2025-01-29T17:52:55.931Z] Installing build dependencies: started
[task 2025-01-29T17:53:00.723Z] Installing build dependencies: finished with status 'done'
[task 2025-01-29T17:53:00.724Z] Getting requirements to build wheel: started
[task 2025-01-29T17:53:02.260Z] Getting requirements to build wheel: finished with status 'done'
[task 2025-01-29T17:53:02.262Z] Preparing metadata (pyproject.toml): started
[task 2025-01-29T17:53:02.839Z] Preparing metadata (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:53:02.860Z] Collecting idna==3.8 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 15))
[task 2025-01-29T17:53:02.876Z] Downloading idna-3.8-py3-none-any.whl.metadata (9.9 kB)
[task 2025-01-29T17:53:03.009Z] Collecting numpy==1.26.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 17))
[task 2025-01-29T17:53:03.024Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
[task 2025-01-29T17:53:03.046Z] Collecting opus-fast-mosestokenizer==0.0.8.5 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 19))
[task 2025-01-29T17:53:03.062Z] Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl.metadata (3.6 kB)
[task 2025-01-29T17:53:03.082Z] Collecting pyicu==2.8.1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 21))
[task 2025-01-29T17:53:03.097Z] Downloading PyICU-2.8.1.tar.gz (304 kB)
[task 2025-01-29T17:53:03.149Z] Installing build dependencies: started
[task 2025-01-29T17:53:04.881Z] Installing build dependencies: finished with status 'done'
[task 2025-01-29T17:53:04.882Z] Getting requirements to build wheel: started
[task 2025-01-29T17:53:05.080Z] Getting requirements to build wheel: finished with status 'done'
[task 2025-01-29T17:53:05.082Z] Preparing metadata (pyproject.toml): started
[task 2025-01-29T17:53:05.298Z] Preparing metadata (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:53:05.328Z] Collecting requests==2.31.0 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 23))
[task 2025-01-29T17:53:05.343Z] Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
[task 2025-01-29T17:53:05.378Z] Collecting tqdm==4.66.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 25))
[task 2025-01-29T17:53:05.395Z] Downloading tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
[task 2025-01-29T17:53:05.445Z] Collecting urllib3==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 27))
[task 2025-01-29T17:53:05.461Z] Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
[task 2025-01-29T17:53:05.464Z] Requirement already satisfied: zstandard==0.23.0 in /usr/local/lib/python3.10/dist-packages (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 29)) (0.23.0)
[task 2025-01-29T17:53:05.486Z] WARNING: The candidate selected for download or install is a yanked version: 'eflomal' candidate (version 1.0.0b1 at https://files.pythonhosted.org/packages/1d/cc/2cdf9c90a6bd8fd7323b47bf5a60fe738bfe8465844a2136b0543f5094e8/eflomal-1.0.0b1.tar.gz (from https://pypi.org/simple/eflomal/))
[task 2025-01-29T17:53:05.486Z] Reason for being yanked: <none given>
[task 2025-01-29T17:53:05.500Z] Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
[task 2025-01-29T17:53:05.515Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2025-01-29T17:53:05.531Z] Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
[task 2025-01-29T17:53:05.567Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 106.7 MB/s eta 0:00:00
[task 2025-01-29T17:53:05.581Z] Downloading idna-3.8-py3-none-any.whl (66 kB)
[task 2025-01-29T17:53:05.596Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2025-01-29T17:53:05.682Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 215.3 MB/s eta 0:00:00
[task 2025-01-29T17:53:05.698Z] Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl (837 kB)
[task 2025-01-29T17:53:05.701Z] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 837.2/837.2 kB 716.7 MB/s eta 0:00:00
[task 2025-01-29T17:53:05.715Z] Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2025-01-29T17:53:05.729Z] Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
[task 2025-01-29T17:53:05.744Z] Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
[task 2025-01-29T17:53:05.825Z] Building wheels for collected packages: eflomal, pyicu
[task 2025-01-29T17:53:05.826Z] Building wheel for eflomal (pyproject.toml): started
[task 2025-01-29T17:53:10.730Z] Building wheel for eflomal (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:53:10.732Z] Created wheel for eflomal: filename=eflomal-1.0.0b1-cp310-cp310-linux_x86_64.whl size=319816 sha256=590219817c6e9a30a333f80ad9699d7cb33bba46e7aee40becb57d7ece9b6788
[task 2025-01-29T17:53:10.732Z] Stored in directory: /tmp/pip-ephem-wheel-cache-34a3fq4n/wheels/b4/0c/55/4a2c6f00793a0d2fdaef4376b8cb3d9c6ae8df16366d6e31d1
[task 2025-01-29T17:53:10.736Z] Building wheel for pyicu (pyproject.toml): started
[task 2025-01-29T17:54:00.961Z] Building wheel for pyicu (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:54:00.968Z] Created wheel for pyicu: filename=PyICU-2.8.1-cp310-cp310-linux_x86_64.whl size=1747818 sha256=4a0a6a9a995cc03652628fe5e8c87a16e0f45d216594713e717f2ae0e5c2476f
[task 2025-01-29T17:54:00.968Z] Stored in directory: /tmp/pip-ephem-wheel-cache-34a3fq4n/wheels/6e/cb/d3/5d1dfd3a9e12d51f0a2bfb951374205f7eca2639a21b03ea12
[task 2025-01-29T17:54:00.970Z] Successfully built eflomal pyicu
[task 2025-01-29T17:54:01.010Z] Installing collected packages: pyicu, urllib3, tqdm, opus-fast-mosestokenizer, numpy, idna, cython, charset-normalizer, certifi, requests, eflomal
[task 2025-01-29T17:54:04.221Z] Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 cython-3.0.9 eflomal-1.0.0b1 idna-3.8 numpy-1.26.4 opus-fast-mosestokenizer-0.0.8.5 pyicu-2.8.1 requests-2.31.0 tqdm-4.66.4 urllib3-2.2.2
[task 2025-01-29T17:54:04.722Z] [alignments] Running with arguments: ['/builds/worker/checkouts/vcs/pipeline/alignments/align.py', '--corpus_src=/builds/worker/fetches/corpus.ru.zst', '--corpus_trg=/builds/worker/fetches/corpus.en.zst', '--output_path=/builds/worker/artifacts/corpus.aln.zst', '--output_tokenized', '--tokenization=icu', '--priors_input_path=/builds/worker/fetches/corpus.priors']
[task 2025-01-29T17:54:04.725Z] [alignments] Starting generating alignments.
[task 2025-01-29T17:54:04.725Z] [alignments] [decompress] From: /builds/worker/fetches/corpus.ru.zst
[task 2025-01-29T17:54:04.725Z] [alignments] [decompress] To: /builds/worker/fetches/corpus.ru
[task 2025-01-29T17:54:04.727Z]
[task 2025-01-29T17:54:04.727Z] etches/corpus.ru.zst : 0 MB...
[task 2025-01-29T17:54:04.727Z]
[task 2025-01-29T17:54:04.728Z] /builds/worker/fetches/corpus.ru.zst: 313899 bytes
[task 2025-01-29T17:54:04.728Z] [alignments] [decompress] Removed: /builds/worker/fetches/corpus.ru.zst
[task 2025-01-29T17:54:04.728Z] [alignments] [decompress] From: /builds/worker/fetches/corpus.en.zst
[task 2025-01-29T17:54:04.728Z] [alignments] [decompress] To: /builds/worker/fetches/corpus.en
[task 2025-01-29T17:54:04.730Z]
[task 2025-01-29T17:54:04.732Z] etches/corpus.en.zst : 0 MB...
[task 2025-01-29T17:54:04.732Z]
[task 2025-01-29T17:54:04.732Z] /builds/worker/fetches/corpus.en.zst: 1845773 bytes
[task 2025-01-29T17:54:04.733Z] [alignments] [decompress] Removed: /builds/worker/fetches/corpus.en.zst
[task 2025-01-29T17:54:04.733Z] [tokenizer] Tokenizing /builds/worker/fetches/corpus.ru with Moses tokenizer
[task 2025-01-29T17:54:04.776Z]
[task 2025-01-29T17:54:04.829Z] 0it [00:00, ?it/s]
[task 2025-01-29T17:54:04.829Z] 1472it [00:00, 27850.77it/s]
[task 2025-01-29T17:54:04.829Z] [tokenizer] Tokenizing /builds/worker/fetches/corpus.en with Moses tokenizer
[task 2025-01-29T17:54:04.876Z]
[task 2025-01-29T17:54:04.951Z] 0it [00:00, ?it/s]
[task 2025-01-29T17:54:04.951Z] 1472it [00:00, 19796.25it/s]
[task 2025-01-29T17:54:05.058Z] [alignments] Splitting corpus into parts
[task 2025-01-29T17:54:05.065Z] [alignments] Processing part aa
[task 2025-01-29T17:54:05.065Z] [alignments] Using provided priors: /builds/worker/fetches/corpus.priors
[task 2025-01-29T17:54:05.065Z] [alignments] Calculating alignments...
[task 2025-01-29T17:54:05.144Z] [eflomal] Prepared 1472 sentences for alignment
[task 2025-01-29T17:54:05.144Z] [eflomal] Reading lexical priors...
[task 2025-01-29T17:54:05.164Z] [eflomal] 152 (of 5365) pairs of lexical priors used
[task 2025-01-29T17:54:05.171Z] /builds/worker/.local/lib/python3.10/site-packages/eflomal/bin/eflomal -m 3 -s /tmp/tmps_6_1211 -t /tmp/tmpo29azyjd -n 3 -N 0.2 -1 32 -2 32 -3 130 -f /builds/worker/artifacts/tmp/aln.fwd.aa -r /builds/worker/artifacts/tmp/aln.rev.aa -p /tmp/tmpse10_j_l
[task 2025-01-29T17:54:05.190Z] Read texts (1472 sentences): 0.018 s
[task 2025-01-29T17:54:05.190Z] Vocabulary sizes are 6797 (source), 791 (target)
[task 2025-01-29T17:54:05.196Z] Created alignment structures: 0.004 s
[task 2025-01-29T17:54:05.197Z] Created alignment structures: 0.006 s
[task 2025-01-29T17:54:05.198Z] Randomized alignment: 0.002 s
[task 2025-01-29T17:54:05.198Z] Aligning with model 1 (32 iterations)
[task 2025-01-29T17:54:05.205Z] Randomized alignment: 0.007 s
[task 2025-01-29T17:54:05.205Z] Aligning with model 1 (32 iterations)
[task 2025-01-29T17:54:07.030Z] Done: 1.832 s
[task 2025-01-29T17:54:07.030Z] Aligning with model 2 (32 iterations)
[task 2025-01-29T17:54:07.306Z] Done: 2.101 s
[task 2025-01-29T17:54:07.306Z] Aligning with model 2 (32 iterations)
[task 2025-01-29T17:54:09.339Z] Done: 2.309 s
[task 2025-01-29T17:54:09.339Z] Aligning with model 3 (130 iterations)
[task 2025-01-29T17:54:09.810Z] Done: 2.504 s
[task 2025-01-29T17:54:09.810Z] Aligning with model 3 (130 iterations)
[task 2025-01-29T17:54:19.716Z] Done: 10.377 s
[task 2025-01-29T17:54:19.999Z] Final argmax iteration: 0.284 s
[task 2025-01-29T17:54:20.000Z] Writing alignments to /builds/worker/artifacts/tmp/aln.rev.aa for 1472 sentencess
[task 2025-01-29T17:54:21.425Z] Done: 11.615 s
[task 2025-01-29T17:54:21.722Z] Final argmax iteration: 0.296 s
[task 2025-01-29T17:54:21.722Z] Writing alignments to /builds/worker/artifacts/tmp/aln.fwd.aa for 1472 sentencess
[task 2025-01-29T17:54:21.740Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp/aln.fwd.aa']
[task 2025-01-29T17:54:21.743Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp/aln.rev.aa']
[task 2025-01-29T17:54:21.744Z] [alignments] Symmetrizing alignments...
[task 2025-01-29T17:54:21.949Z] [alignments] Saving tokenized corpus
[task 2025-01-29T17:54:21.952Z]
[task 2025-01-29T17:54:21.952Z]
[task 2025-01-29T17:54:21.954Z] Read : 0 / 0 MB ==> 25%
[task 2025-01-29T17:54:21.954Z]
[task 2025-01-29T17:54:21.954Z] /builds/worker/artifacts/corpus.tok-icu.ru : 23.61% (401988 => 94905 bytes, /builds/worker/artifacts/corpus.tok-icu.ru.zst)
[task 2025-01-29T17:54:21.957Z]
[task 2025-01-29T17:54:21.957Z]
[task 2025-01-29T17:54:21.960Z] Read : 0 / 2 MB ==> 0%
[task 2025-01-29T17:54:21.960Z]
[task 2025-01-29T17:54:21.960Z] /builds/worker/artifacts/corpus.tok-icu.en : 0.90% (2415594 => 21826 bytes, /builds/worker/artifacts/corpus.tok-icu.en.zst)
[task 2025-01-29T17:54:21.962Z] [alignments] Compressing final alignments
[task 2025-01-29T17:54:21.965Z]
[task 2025-01-29T17:54:21.965Z]
[task 2025-01-29T17:54:21.966Z] Read : 0 / 0 MB ==> 40%
[task 2025-01-29T17:54:21.966Z]
[task 2025-01-29T17:54:21.966Z] /builds/worker/artifacts/tmp/aln : 39.96% (220937 => 88277 bytes, /builds/worker/artifacts/tmp/aln.zst)
[task 2025-01-29T17:54:21.966Z] [alignments] Finished generating alignments.
[fetches 2025-01-29T17:54:22.005Z] removing /builds/worker/fetches
[fetches 2025-01-29T17:54:22.291Z] finished
+ exit_code=0
+ docker cp taskcontainer_OwBWuNoBT_S9Q0_lasIOhQ:/builds/worker/artifacts artifact0
+ docker rm taskcontainer_OwBWuNoBT_S9Q0_lasIOhQ
taskcontainer_OwBWuNoBT_S9Q0_lasIOhQ
+ exit 0
[taskcluster 2025-01-29T17:54:23.010Z] Exit Code: 0
[taskcluster 2025-01-29T17:54:23.010Z] User Time: 98.693ms
[taskcluster 2025-01-29T17:54:23.010Z] Kernel Time: 256.958ms
[taskcluster 2025-01-29T17:54:23.010Z] Wall Time: 2m34.232690854s
[taskcluster 2025-01-29T17:54:23.010Z] Result: SUCCEEDED
[taskcluster 2025-01-29T17:54:23.011Z] === Task Finished ===
[taskcluster 2025-01-29T17:54:23.011Z] Task Duration: 2m34.233037809s
[taskcluster 2025-01-29T17:54:23.210Z] Uploading artifact public/build/tmp/aln.fwd.aa from file /home/task_173817309997346/artifact0/tmp/aln.fwd.aa with content encoding "gzip", mime type "audio/x-pn-audibleaudio" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.278Z] Uploading artifact public/build/corpus.aln.zst from file /home/task_173817309997346/artifact0/corpus.aln.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.285Z] Uploading artifact public/build/corpus.tok-icu.ru.zst from file /home/task_173817309997346/artifact0/corpus.tok-icu.ru.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.291Z] Uploading artifact public/build/tmp/aln.rev from file /home/task_173817309997346/artifact0/tmp/aln.rev with content encoding "gzip", mime type "application/octet-stream" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.296Z] Uploading artifact public/build/tmp/aln.rev.aa from file /home/task_173817309997346/artifact0/tmp/aln.rev.aa with content encoding "gzip", mime type "audio/x-pn-audibleaudio" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.309Z] Uploading artifact public/build/tmp/aln.fwd from file /home/task_173817309997346/artifact0/tmp/aln.fwd with content encoding "gzip", mime type "application/octet-stream" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.330Z] Uploading artifact public/build/corpus.tok-icu.en.zst from file /home/task_173817309997346/artifact0/corpus.tok-icu.en.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:56:01.990Z
[taskcluster 2025-01-29T17:54:23.866Z] [mounts] Preserving cache: Moving "/home/task_173817309997346/cache0" to "caches/CNZruPCuSJqXMicaktBW5A"
[taskcluster 2025-01-29T17:54:24.006Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2025-02-28T15:56:01.990Z
Loading