Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: translations integration tests #210

Merged
merged 9 commits into from
Jan 29, 2025

remove `REV` from integration test task definitions

96fa36c
Select commit
Loading
Failed to load commit list.
Merged

feat: translations integration tests #210

remove `REV` from integration test task definitions
96fa36c
Select commit
Loading
Failed to load commit list.
stage-taskcluster / translations-alignments-backtranslated-ru-en succeeded Jan 29, 2025 in 1h 19m 6s

Stage-TC (issue_comment)

alignments for teacher's back-translated parallel training corpus ru-en

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2025-01-29T17:13:18.545Z
Resolved: 2025-01-29T17:15:05.657Z
Task Execution Time: 1 minute, 47 seconds, 112 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/build/mono.aln.zst
- public/build/mono.tok-icu.en.zst
- public/build/mono.tok-icu.ru.zst
- public/build/tmp/aln.fwd
- public/build/tmp/aln.fwd.aa
- public/build/tmp/aln.rev
- public/build/tmp/aln.rev.aa
- public/logs/live_backing.log
- public/logs/live.log


[taskcluster 2025-01-29T17:13:18.871Z] Worker Type (translations-1/b-linux-large-gcp-1tb-32-256-d2g) settings:
[taskcluster 2025-01-29T17:13:18.871Z]   {
[taskcluster 2025-01-29T17:13:18.871Z]     "config": {
[taskcluster 2025-01-29T17:13:18.871Z]       "deploymentId": ""
[taskcluster 2025-01-29T17:13:18.871Z]     },
[taskcluster 2025-01-29T17:13:18.871Z]     "generic-worker": {
[taskcluster 2025-01-29T17:13:18.871Z]       "config": {
[taskcluster 2025-01-29T17:13:18.871Z]         "headlessTasks": true,
[taskcluster 2025-01-29T17:13:18.871Z]         "runTasksAsCurrentUser": false
[taskcluster 2025-01-29T17:13:18.871Z]       },
[taskcluster 2025-01-29T17:13:18.871Z]       "engine": "multiuser",
[taskcluster 2025-01-29T17:13:18.871Z]       "go-arch": "amd64",
[taskcluster 2025-01-29T17:13:18.871Z]       "go-os": "linux",
[taskcluster 2025-01-29T17:13:18.871Z]       "go-version": "go1.23.4",
[taskcluster 2025-01-29T17:13:18.871Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v77.3.1",
[taskcluster 2025-01-29T17:13:18.871Z]       "revision": "959a204190add062fe1217d14f2a0115ecd43fe8",
[taskcluster 2025-01-29T17:13:18.871Z]       "source": "https://github.com/taskcluster/taskcluster/commits/959a204190add062fe1217d14f2a0115ecd43fe8",
[taskcluster 2025-01-29T17:13:18.871Z]       "version": "77.3.1"
[taskcluster 2025-01-29T17:13:18.871Z]     },
[taskcluster 2025-01-29T17:13:18.871Z]     "image": "projects/taskcluster-imaging/global/images/gw-fxci-gcp-l1-2404-amd64-headless-googlecompute-2025-01-17",

...(203 lines hidden)...

attempt 1/5attempt 1/5

attempt 1/5Downloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/fBo895glQsa76zkcoN68vg/artifacts/public/build/mono.ru.zst to /builds/worker/fetches/mono.ru.zst
Downloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/KQOb2v4STDiMyJbUPPgmmg/artifacts/public/build/corpus.priors to /builds/worker/fetches/corpus.priors
Downloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/bNabi1SkSn23aL4fhQXeaw/artifacts/public/build/mono.en.zst to /builds/worker/fetches/mono.en.zst

attempt 1/5
Downloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst to /builds/worker/fetches/fast-align.tar.zst
Downloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/bNabi1SkSn23aL4fhQXeaw/artifacts/public/build/mono.en.zstDownloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/fBo895glQsa76zkcoN68vg/artifacts/public/build/mono.ru.zst

Downloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/KQOb2v4STDiMyJbUPPgmmg/artifacts/public/build/corpus.priorsDownloading https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst

https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/bNabi1SkSn23aL4fhQXeaw/artifacts/public/build/mono.en.zst resolved to 16347 bytes with sha256 52ff99acd1a05e1e1a7463e6e1a4e08c05265943f20a8100c765a0b436013ed3 in 0.137s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/bNabi1SkSn23aL4fhQXeaw/artifacts/public/build/mono.en.zst
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst resolved to 722788 bytes with sha256 3a20fe4e64f4727cad8e6216e169a1119020f22a6151e9cdb355ddab2403125f in 0.143s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst
Extracting /builds/worker/fetches/fast-align.tar.zst to /builds/worker/fetches
/builds/worker/fetches/fast-align.tar.zst extracted in 0.008s
Removing /builds/worker/fetches/fast-align.tar.zst
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/fBo895glQsa76zkcoN68vg/artifacts/public/build/mono.ru.zst resolved to 3213 bytes with sha256 82a2c294b6dbb5a798cc119344ddf16f0b975046ef8794affe6559efdf54ad06 in 0.238s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/fBo895glQsa76zkcoN68vg/artifacts/public/build/mono.ru.zst
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/KQOb2v4STDiMyJbUPPgmmg/artifacts/public/build/corpus.priors resolved to 286222 bytes with sha256 70d87ddc51aff4f198af798d09e97d41666cf4df7bdb0aa65695113232c2f463 in 0.249s
Extracting /builds/worker/fetches/corpus.priors to /builds/worker/fetches
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.2538021379999975, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2025-01-29T17:13:59.881Z] finished fetching artifacts
[task 2025-01-29T17:13:59.881Z] executing ['bash', '-c', 'pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/alignments/requirements/alignments.txt && export BIN=$MOZ_FETCHES_DIR && export PATH=$PATH:$MOZ_FETCHES_DIR && export PYTHONPATH=$PYTHONPATH:$VCS_PATH && python3 $VCS_PATH/pipeline/alignments/align.py --corpus_src=$MOZ_FETCHES_DIR/mono.ru.zst --corpus_trg=$MOZ_FETCHES_DIR/mono.en.zst --output_path=$TASK_WORKDIR/artifacts/mono.aln.zst --output_tokenized --tokenization=icu --priors_input_path=$MOZ_FETCHES_DIR/corpus.priors']
[task 2025-01-29T17:14:00.180Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2025-01-29T17:14:00.181Z] Defaulting to user installation because normal site-packages is not writeable
[task 2025-01-29T17:14:00.202Z] Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
[task 2025-01-29T17:14:00.357Z] Collecting pip
[task 2025-01-29T17:14:00.454Z]   Downloading pip-25.0-py3-none-any.whl (1.8 MB)
[task 2025-01-29T17:14:00.662Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 8.9 MB/s eta 0:00:00
[task 2025-01-29T17:14:00.672Z] Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
[task 2025-01-29T17:14:00.934Z] Collecting setuptools
[task 2025-01-29T17:14:00.947Z]   Downloading setuptools-75.8.0-py3-none-any.whl (1.2 MB)
[task 2025-01-29T17:14:01.022Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 16.8 MB/s eta 0:00:00
[task 2025-01-29T17:14:01.129Z] Installing collected packages: setuptools, pip
[task 2025-01-29T17:14:02.362Z] Successfully installed pip-25.0 setuptools-75.8.0
[task 2025-01-29T17:14:02.701Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2025-01-29T17:14:02.702Z] Defaulting to user installation because normal site-packages is not writeable
[task 2025-01-29T17:14:02.837Z] Collecting certifi==2024.7.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 7))
[task 2025-01-29T17:14:02.876Z]   Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
[task 2025-01-29T17:14:02.958Z] Collecting charset-normalizer==3.3.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 9))
[task 2025-01-29T17:14:02.971Z]   Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
[task 2025-01-29T17:14:03.125Z] Collecting cython==3.0.9 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 11))
[task 2025-01-29T17:14:03.140Z]   Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
[task 2025-01-29T17:14:03.179Z] Collecting eflomal==1.0.0b1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 13))
[task 2025-01-29T17:14:03.193Z]   Downloading eflomal-1.0.0b1.tar.gz (102 kB)
[task 2025-01-29T17:14:03.217Z]   Installing build dependencies: started
[task 2025-01-29T17:14:07.962Z]   Installing build dependencies: finished with status 'done'
[task 2025-01-29T17:14:07.963Z]   Getting requirements to build wheel: started
[task 2025-01-29T17:14:09.078Z]   Getting requirements to build wheel: finished with status 'done'
[task 2025-01-29T17:14:09.080Z]   Preparing metadata (pyproject.toml): started
[task 2025-01-29T17:14:09.542Z]   Preparing metadata (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:14:09.561Z] Collecting idna==3.8 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 15))
[task 2025-01-29T17:14:09.575Z]   Downloading idna-3.8-py3-none-any.whl.metadata (9.9 kB)
[task 2025-01-29T17:14:09.693Z] Collecting numpy==1.26.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 17))
[task 2025-01-29T17:14:09.706Z]   Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
[task 2025-01-29T17:14:09.750Z] Collecting opus-fast-mosestokenizer==0.0.8.5 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 19))
[task 2025-01-29T17:14:09.765Z]   Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl.metadata (3.6 kB)
[task 2025-01-29T17:14:09.783Z] Collecting pyicu==2.8.1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 21))
[task 2025-01-29T17:14:09.797Z]   Downloading PyICU-2.8.1.tar.gz (304 kB)
[task 2025-01-29T17:14:09.845Z]   Installing build dependencies: started
[task 2025-01-29T17:14:11.441Z]   Installing build dependencies: finished with status 'done'
[task 2025-01-29T17:14:11.442Z]   Getting requirements to build wheel: started
[task 2025-01-29T17:14:11.607Z]   Getting requirements to build wheel: finished with status 'done'
[task 2025-01-29T17:14:11.608Z]   Preparing metadata (pyproject.toml): started
[task 2025-01-29T17:14:11.775Z]   Preparing metadata (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:14:11.804Z] Collecting requests==2.31.0 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 23))
[task 2025-01-29T17:14:11.818Z]   Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
[task 2025-01-29T17:14:11.851Z] Collecting tqdm==4.66.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 25))
[task 2025-01-29T17:14:11.867Z]   Downloading tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
[task 2025-01-29T17:14:11.916Z] Collecting urllib3==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 27))
[task 2025-01-29T17:14:11.929Z]   Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
[task 2025-01-29T17:14:11.931Z] Requirement already satisfied: zstandard==0.23.0 in /usr/local/lib/python3.10/dist-packages (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 29)) (0.23.0)
[task 2025-01-29T17:14:11.953Z] WARNING: The candidate selected for download or install is a yanked version: 'eflomal' candidate (version 1.0.0b1 at https://files.pythonhosted.org/packages/1d/cc/2cdf9c90a6bd8fd7323b47bf5a60fe738bfe8465844a2136b0543f5094e8/eflomal-1.0.0b1.tar.gz (from https://pypi.org/simple/eflomal/))
[task 2025-01-29T17:14:11.953Z] Reason for being yanked: <none given>
[task 2025-01-29T17:14:11.966Z] Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
[task 2025-01-29T17:14:11.982Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2025-01-29T17:14:11.997Z] Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
[task 2025-01-29T17:14:12.031Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 112.4 MB/s eta 0:00:00
[task 2025-01-29T17:14:12.044Z] Downloading idna-3.8-py3-none-any.whl (66 kB)
[task 2025-01-29T17:14:12.057Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2025-01-29T17:14:12.112Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 349.0 MB/s eta 0:00:00
[task 2025-01-29T17:14:12.126Z] Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl (837 kB)
[task 2025-01-29T17:14:12.131Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 837.2/837.2 kB 275.0 MB/s eta 0:00:00
[task 2025-01-29T17:14:12.144Z] Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2025-01-29T17:14:12.157Z] Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
[task 2025-01-29T17:14:12.170Z] Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
[task 2025-01-29T17:14:12.251Z] Building wheels for collected packages: eflomal, pyicu
[task 2025-01-29T17:14:12.252Z]   Building wheel for eflomal (pyproject.toml): started
[task 2025-01-29T17:14:16.428Z]   Building wheel for eflomal (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:14:16.429Z]   Created wheel for eflomal: filename=eflomal-1.0.0b1-cp310-cp310-linux_x86_64.whl size=319816 sha256=1667f5d80ab0d0236738fd20936a869229c321786e49252d684b8fd532b1a95e
[task 2025-01-29T17:14:16.429Z]   Stored in directory: /tmp/pip-ephem-wheel-cache-ygtvj6cz/wheels/b4/0c/55/4a2c6f00793a0d2fdaef4376b8cb3d9c6ae8df16366d6e31d1
[task 2025-01-29T17:14:16.432Z]   Building wheel for pyicu (pyproject.toml): started
[task 2025-01-29T17:14:58.855Z]   Building wheel for pyicu (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:14:58.860Z]   Created wheel for pyicu: filename=PyICU-2.8.1-cp310-cp310-linux_x86_64.whl size=1747820 sha256=61dd281fd6237dfe01b2549381a1968e78974d6ee8a2485013f56ceb7231832d
[task 2025-01-29T17:14:58.860Z]   Stored in directory: /tmp/pip-ephem-wheel-cache-ygtvj6cz/wheels/6e/cb/d3/5d1dfd3a9e12d51f0a2bfb951374205f7eca2639a21b03ea12
[task 2025-01-29T17:14:58.862Z] Successfully built eflomal pyicu
[task 2025-01-29T17:14:58.896Z] Installing collected packages: pyicu, urllib3, tqdm, opus-fast-mosestokenizer, numpy, idna, cython, charset-normalizer, certifi, requests, eflomal
[task 2025-01-29T17:15:01.334Z] Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 cython-3.0.9 eflomal-1.0.0b1 idna-3.8 numpy-1.26.4 opus-fast-mosestokenizer-0.0.8.5 pyicu-2.8.1 requests-2.31.0 tqdm-4.66.4 urllib3-2.2.2
[task 2025-01-29T17:15:01.682Z] [alignments] Running with arguments: ['/builds/worker/checkouts/vcs/pipeline/alignments/align.py', '--corpus_src=/builds/worker/fetches/mono.ru.zst', '--corpus_trg=/builds/worker/fetches/mono.en.zst', '--output_path=/builds/worker/artifacts/mono.aln.zst', '--output_tokenized', '--tokenization=icu', '--priors_input_path=/builds/worker/fetches/corpus.priors']
[task 2025-01-29T17:15:01.683Z] [alignments] Starting generating alignments.
[task 2025-01-29T17:15:01.684Z] [alignments] [decompress] From: /builds/worker/fetches/mono.ru.zst
[task 2025-01-29T17:15:01.684Z] [alignments] [decompress] To: /builds/worker/fetches/mono.ru
[task 2025-01-29T17:15:01.685Z] 
[task 2025-01-29T17:15:01.685Z] /fetches/mono.ru.zst : 0 MB...     
[task 2025-01-29T17:15:01.685Z]                                                                                
[task 2025-01-29T17:15:01.685Z] /builds/worker/fetches/mono.ru.zst: 51728 bytes 
[task 2025-01-29T17:15:01.686Z] [alignments] [decompress] Removed: /builds/worker/fetches/mono.ru.zst
[task 2025-01-29T17:15:01.686Z] [alignments] [decompress] From: /builds/worker/fetches/mono.en.zst
[task 2025-01-29T17:15:01.686Z] [alignments] [decompress] To: /builds/worker/fetches/mono.en
[task 2025-01-29T17:15:01.687Z] 
[task 2025-01-29T17:15:01.687Z] /fetches/mono.en.zst : 0 MB...     
[task 2025-01-29T17:15:01.687Z]                                                                                
[task 2025-01-29T17:15:01.687Z] /builds/worker/fetches/mono.en.zst: 38227 bytes 
[task 2025-01-29T17:15:01.687Z] [alignments] [decompress] Removed: /builds/worker/fetches/mono.en.zst
[task 2025-01-29T17:15:01.687Z] [tokenizer] Tokenizing /builds/worker/fetches/mono.ru with Moses tokenizer
[task 2025-01-29T17:15:01.716Z] 
[task 2025-01-29T17:15:01.738Z] 0it [00:00, ?it/s]
[task 2025-01-29T17:15:01.738Z] 493it [00:00, 22166.87it/s]
[task 2025-01-29T17:15:01.738Z] [tokenizer] Tokenizing /builds/worker/fetches/mono.en with Moses tokenizer
[task 2025-01-29T17:15:01.764Z] 
[task 2025-01-29T17:15:01.786Z] 0it [00:00, ?it/s]
[task 2025-01-29T17:15:01.786Z] 493it [00:00, 23152.45it/s]
[task 2025-01-29T17:15:01.862Z] [alignments] Splitting corpus into parts
[task 2025-01-29T17:15:01.870Z] [alignments] Processing part aa
[task 2025-01-29T17:15:01.870Z] [alignments] Using provided priors: /builds/worker/fetches/corpus.priors
[task 2025-01-29T17:15:01.870Z] [alignments] Calculating alignments...
[task 2025-01-29T17:15:01.884Z] [eflomal] Prepared 493 sentences for alignment
[task 2025-01-29T17:15:01.884Z] [eflomal] Reading lexical priors...
[task 2025-01-29T17:15:01.902Z] [eflomal] 46 (of 5365) pairs of lexical priors used
[task 2025-01-29T17:15:01.904Z] /builds/worker/.local/lib/python3.10/site-packages/eflomal/bin/eflomal -m 3 -s /tmp/tmpuc5simm7 -t /tmp/tmpzehfqjd1 -n 3 -N 0.2 -1 56 -2 56 -3 225 -f /builds/worker/artifacts/tmp/aln.fwd.aa -r /builds/worker/artifacts/tmp/aln.rev.aa -p /tmp/tmp6su6a3fv
[task 2025-01-29T17:15:01.907Z] Read texts (493 sentences): 0.002 s
[task 2025-01-29T17:15:01.907Z] Vocabulary sizes are 74 (source), 2242 (target)
[task 2025-01-29T17:15:01.909Z] Created alignment structures: 0.001 s
[task 2025-01-29T17:15:01.910Z] Created alignment structures: 0.003 s
[task 2025-01-29T17:15:01.911Z] Randomized alignment: 0.003 s
[task 2025-01-29T17:15:01.911Z] Aligning with model 1 (56 iterations)
[task 2025-01-29T17:15:01.914Z] Randomized alignment: 0.003 s
[task 2025-01-29T17:15:01.914Z] Aligning with model 1 (56 iterations)
[task 2025-01-29T17:15:02.130Z] Done: 0.216 s
[task 2025-01-29T17:15:02.130Z] Aligning with model 2 (56 iterations)
[task 2025-01-29T17:15:02.141Z] Done: 0.229 s
[task 2025-01-29T17:15:02.141Z] Aligning with model 2 (56 iterations)
[task 2025-01-29T17:15:02.384Z] Done: 0.254 s
[task 2025-01-29T17:15:02.384Z] Aligning with model 3 (225 iterations)
[task 2025-01-29T17:15:02.388Z] Done: 0.248 s
[task 2025-01-29T17:15:02.389Z] Aligning with model 3 (225 iterations)
[task 2025-01-29T17:15:03.443Z] Done: 1.054 s
[task 2025-01-29T17:15:03.459Z] Final argmax iteration: 0.016 s
[task 2025-01-29T17:15:03.459Z] Writing alignments to /builds/worker/artifacts/tmp/aln.fwd.aa for 493 sentencess
[task 2025-01-29T17:15:03.726Z] Done: 1.342 s
[task 2025-01-29T17:15:03.745Z] Final argmax iteration: 0.019 s
[task 2025-01-29T17:15:03.745Z] Writing alignments to /builds/worker/artifacts/tmp/aln.rev.aa for 493 sentencess
[task 2025-01-29T17:15:03.748Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp/aln.fwd.aa']
[task 2025-01-29T17:15:03.750Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp/aln.rev.aa']
[task 2025-01-29T17:15:03.751Z] [alignments] Symmetrizing alignments...
[task 2025-01-29T17:15:03.766Z] [alignments] Saving tokenized corpus
[task 2025-01-29T17:15:03.768Z] 
[task 2025-01-29T17:15:03.768Z]                                                                                
[task 2025-01-29T17:15:03.768Z] Read :  0 /  0 MB ==>  4%
[task 2025-01-29T17:15:03.768Z]                                                                                
[task 2025-01-29T17:15:03.768Z] /builds/worker/artifacts/mono.tok-icu.ru :  4.07%   ( 90763 =>   3691 bytes, /builds/worker/artifacts/mono.tok-icu.ru.zst) 
[task 2025-01-29T17:15:03.770Z] 
[task 2025-01-29T17:15:03.770Z]                                                                                
[task 2025-01-29T17:15:03.770Z] Read :  0 /  0 MB ==> 31%
[task 2025-01-29T17:15:03.770Z]                                                                                
[task 2025-01-29T17:15:03.770Z] /builds/worker/artifacts/mono.tok-icu.en : 30.71%   ( 61801 =>  18980 bytes, /builds/worker/artifacts/mono.tok-icu.en.zst) 
[task 2025-01-29T17:15:03.771Z] [alignments] Compressing final alignments
[task 2025-01-29T17:15:03.772Z] 
[task 2025-01-29T17:15:03.772Z]                                                                                
[task 2025-01-29T17:15:03.772Z] Read :  0 /  0 MB ==> 35%
[task 2025-01-29T17:15:03.772Z]                                                                                
[task 2025-01-29T17:15:03.772Z] /builds/worker/artifacts/tmp/aln : 34.75%   ( 25795 =>   8963 bytes, /builds/worker/artifacts/tmp/aln.zst) 
[task 2025-01-29T17:15:03.772Z] [alignments] Finished generating alignments.
[fetches 2025-01-29T17:15:03.794Z] removing /builds/worker/fetches
[fetches 2025-01-29T17:15:03.795Z] finished
+ exit_code=0
+ docker cp taskcontainer_KVC-vwPyTyOQ5bMgAC8Qjg:/builds/worker/artifacts artifact0
+ docker rm taskcontainer_KVC-vwPyTyOQ5bMgAC8Qjg
taskcontainer_KVC-vwPyTyOQ5bMgAC8Qjg
+ exit 0
[taskcluster 2025-01-29T17:15:04.709Z]    Exit Code: 0
[taskcluster 2025-01-29T17:15:04.709Z]    User Time: 96.329ms
[taskcluster 2025-01-29T17:15:04.709Z]  Kernel Time: 245.177ms
[taskcluster 2025-01-29T17:15:04.709Z]    Wall Time: 1m34.735286551s
[taskcluster 2025-01-29T17:15:04.709Z]       Result: SUCCEEDED
[taskcluster 2025-01-29T17:15:04.709Z] === Task Finished ===
[taskcluster 2025-01-29T17:15:04.709Z] Task Duration: 1m34.735685548s
[taskcluster 2025-01-29T17:15:04.944Z] Uploading artifact public/build/tmp/aln.rev.aa from file /home/task_173817079504036/artifact0/tmp/aln.rev.aa with content encoding "gzip", mime type "audio/x-pn-audibleaudio" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:04.950Z] Uploading artifact public/build/mono.tok-icu.ru.zst from file /home/task_173817079504036/artifact0/mono.tok-icu.ru.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:04.992Z] Uploading artifact public/build/tmp/aln.fwd from file /home/task_173817079504036/artifact0/tmp/aln.fwd with content encoding "gzip", mime type "application/octet-stream" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:04.993Z] Uploading artifact public/build/mono.tok-icu.en.zst from file /home/task_173817079504036/artifact0/mono.tok-icu.en.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:05.005Z] Uploading artifact public/build/tmp/aln.fwd.aa from file /home/task_173817079504036/artifact0/tmp/aln.fwd.aa with content encoding "gzip", mime type "audio/x-pn-audibleaudio" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:05.017Z] Uploading artifact public/build/tmp/aln.rev from file /home/task_173817079504036/artifact0/tmp/aln.rev with content encoding "gzip", mime type "application/octet-stream" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:05.044Z] Uploading artifact public/build/mono.aln.zst from file /home/task_173817079504036/artifact0/mono.aln.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:55:59.859Z
[taskcluster 2025-01-29T17:15:05.342Z] [mounts] Preserving cache: Moving "/home/task_173817079504036/cache0" to "caches/QPaaalXHTH2Ojt0mtimsHQ"
[taskcluster 2025-01-29T17:15:05.472Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2025-02-28T15:55:59.859Z