[taskcluster 2025-01-29T17:52:36.174Z] Worker Type (translations-1/b-linux-large-gcp-1tb-32-256-d2g) settings:
[taskcluster 2025-01-29T17:52:36.174Z]   {
[taskcluster 2025-01-29T17:52:36.174Z]     "config": {
[taskcluster 2025-01-29T17:52:36.174Z]       "deploymentId": ""
[taskcluster 2025-01-29T17:52:36.174Z]     },
[taskcluster 2025-01-29T17:52:36.174Z]     "generic-worker": {
[taskcluster 2025-01-29T17:52:36.174Z]       "config": {
[taskcluster 2025-01-29T17:52:36.174Z]         "headlessTasks": true,
[taskcluster 2025-01-29T17:52:36.174Z]         "runTasksAsCurrentUser": false
[taskcluster 2025-01-29T17:52:36.174Z]       },
[taskcluster 2025-01-29T17:52:36.174Z]       "engine": "multiuser",
[taskcluster 2025-01-29T17:52:36.174Z]       "go-arch": "amd64",
[taskcluster 2025-01-29T17:52:36.174Z]       "go-os": "linux",
[taskcluster 2025-01-29T17:52:36.174Z]       "go-version": "go1.23.4",
[taskcluster 2025-01-29T17:52:36.174Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v77.3.1",
[taskcluster 2025-01-29T17:52:36.174Z]       "revision": "959a204190add062fe1217d14f2a0115ecd43fe8",
[taskcluster 2025-01-29T17:52:36.174Z]       "source": "https://github.com/taskcluster/taskcluster/commits/959a204190add062fe1217d14f2a0115ecd43fe8",
[taskcluster 2025-01-29T17:52:36.174Z]       "version": "77.3.1"
[taskcluster 2025-01-29T17:52:36.174Z]     },
[taskcluster 2025-01-29T17:52:36.174Z]     "image": "projects/taskcluster-imaging/global/images/gw-fxci-gcp-l1-2404-amd64-headless-googlecompute-2025-01-17",

...(236 lines hidden)...

Extracting /builds/worker/fetches/corpus.en.zst to /builds/worker/fetches
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst resolved to 722788 bytes with sha256 3a20fe4e64f4727cad8e6216e169a1119020f22a6151e9cdb355ddab2403125f in 0.174s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/Ed5IpyGMSYWfriVbJ9GZkA/artifacts/public/build/fast-align.tar.zst
Extracting /builds/worker/fetches/fast-align.tar.zst to /builds/worker/fetches
/builds/worker/fetches/fast-align.tar.zst extracted in 0.007s
Removing /builds/worker/fetches/fast-align.tar.zst
https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V54HtUtPSou23YIBg1awvw/artifacts/public/build/marian.tar.zst resolved to 684172003 bytes with sha256 7839452e78344d844c731cc7ebb7775bd9d3ef0e3e68e80d6b3f920255bc5ef7 in 10.414s
Verified size of https://stage.taskcluster.nonprod.cloudops.mozgcp.net/api/queue/v1/task/V54HtUtPSou23YIBg1awvw/artifacts/public/build/marian.tar.zst
Extracting /builds/worker/fetches/marian.tar.zst to /builds/worker/fetches
/builds/worker/fetches/marian.tar.zst extracted in 5.112s
Removing /builds/worker/fetches/marian.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 15.609806061, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2025-01-29T17:53:49.106Z] finished fetching artifacts
[task 2025-01-29T17:53:49.106Z] executing ['bash', '-c', 'pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/alignments/requirements/alignments.txt && export BIN=$MOZ_FETCHES_DIR && export MARIAN=$MOZ_FETCHES_DIR && export PATH=$PATH:$MOZ_FETCHES_DIR && export PYTHONPATH=$PYTHONPATH:$VCS_PATH && $VCS_PATH/pipeline/alignments/generate-shortlist.sh $MOZ_FETCHES_DIR/corpus $MOZ_FETCHES_DIR/vocab.spm $TASK_WORKDIR/artifacts auto']
[task 2025-01-29T17:53:49.407Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2025-01-29T17:53:49.407Z] Defaulting to user installation because normal site-packages is not writeable
[task 2025-01-29T17:53:49.429Z] Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
[task 2025-01-29T17:53:49.584Z] Collecting pip
[task 2025-01-29T17:53:49.669Z]   Downloading pip-25.0-py3-none-any.whl (1.8 MB)
[task 2025-01-29T17:53:49.876Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 9.1 MB/s eta 0:00:00
[task 2025-01-29T17:53:49.886Z] Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
[task 2025-01-29T17:53:50.139Z] Collecting setuptools
[task 2025-01-29T17:53:50.152Z]   Downloading setuptools-75.8.0-py3-none-any.whl (1.2 MB)
[task 2025-01-29T17:53:50.224Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 17.6 MB/s eta 0:00:00
[task 2025-01-29T17:53:50.334Z] Installing collected packages: setuptools, pip
[task 2025-01-29T17:53:51.589Z] Successfully installed pip-25.0 setuptools-75.8.0
[task 2025-01-29T17:53:51.938Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2025-01-29T17:53:51.939Z] Defaulting to user installation because normal site-packages is not writeable
[task 2025-01-29T17:53:52.075Z] Collecting certifi==2024.7.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 7))
[task 2025-01-29T17:53:52.115Z]   Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
[task 2025-01-29T17:53:52.203Z] Collecting charset-normalizer==3.3.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 9))
[task 2025-01-29T17:53:52.217Z]   Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
[task 2025-01-29T17:53:52.374Z] Collecting cython==3.0.9 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 11))
[task 2025-01-29T17:53:52.408Z]   Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
[task 2025-01-29T17:53:52.426Z] Collecting eflomal==1.0.0b1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 13))
[task 2025-01-29T17:53:52.440Z]   Downloading eflomal-1.0.0b1.tar.gz (102 kB)
[task 2025-01-29T17:53:52.463Z]   Installing build dependencies: started
[task 2025-01-29T17:53:57.325Z]   Installing build dependencies: finished with status 'done'
[task 2025-01-29T17:53:57.326Z]   Getting requirements to build wheel: started
[task 2025-01-29T17:53:58.484Z]   Getting requirements to build wheel: finished with status 'done'
[task 2025-01-29T17:53:58.485Z]   Preparing metadata (pyproject.toml): started
[task 2025-01-29T17:53:58.960Z]   Preparing metadata (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:53:58.979Z] Collecting idna==3.8 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 15))
[task 2025-01-29T17:53:58.992Z]   Downloading idna-3.8-py3-none-any.whl.metadata (9.9 kB)
[task 2025-01-29T17:53:59.111Z] Collecting numpy==1.26.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 17))
[task 2025-01-29T17:53:59.126Z]   Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
[task 2025-01-29T17:53:59.147Z] Collecting opus-fast-mosestokenizer==0.0.8.5 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 19))
[task 2025-01-29T17:53:59.161Z]   Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl.metadata (3.6 kB)
[task 2025-01-29T17:53:59.178Z] Collecting pyicu==2.8.1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 21))
[task 2025-01-29T17:53:59.194Z]   Downloading PyICU-2.8.1.tar.gz (304 kB)
[task 2025-01-29T17:53:59.244Z]   Installing build dependencies: started
[task 2025-01-29T17:54:01.610Z]   Installing build dependencies: finished with status 'done'
[task 2025-01-29T17:54:01.611Z]   Getting requirements to build wheel: started
[task 2025-01-29T17:54:01.775Z]   Getting requirements to build wheel: finished with status 'done'
[task 2025-01-29T17:54:01.776Z]   Preparing metadata (pyproject.toml): started
[task 2025-01-29T17:54:01.945Z]   Preparing metadata (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:54:01.977Z] Collecting requests==2.31.0 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 23))
[task 2025-01-29T17:54:01.996Z]   Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
[task 2025-01-29T17:54:02.031Z] Collecting tqdm==4.66.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 25))
[task 2025-01-29T17:54:02.046Z]   Downloading tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
[task 2025-01-29T17:54:02.096Z] Collecting urllib3==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 27))
[task 2025-01-29T17:54:02.109Z]   Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
[task 2025-01-29T17:54:02.112Z] Requirement already satisfied: zstandard==0.23.0 in /usr/local/lib/python3.10/dist-packages (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 29)) (0.23.0)
[task 2025-01-29T17:54:02.130Z] WARNING: The candidate selected for download or install is a yanked version: 'eflomal' candidate (version 1.0.0b1 at https://files.pythonhosted.org/packages/1d/cc/2cdf9c90a6bd8fd7323b47bf5a60fe738bfe8465844a2136b0543f5094e8/eflomal-1.0.0b1.tar.gz (from https://pypi.org/simple/eflomal/))
[task 2025-01-29T17:54:02.130Z] Reason for being yanked: <none given>
[task 2025-01-29T17:54:02.144Z] Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
[task 2025-01-29T17:54:02.159Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2025-01-29T17:54:02.174Z] Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
[task 2025-01-29T17:54:02.208Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 112.5 MB/s eta 0:00:00
[task 2025-01-29T17:54:02.232Z] Downloading idna-3.8-py3-none-any.whl (66 kB)
[task 2025-01-29T17:54:02.245Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2025-01-29T17:54:02.324Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 234.6 MB/s eta 0:00:00
[task 2025-01-29T17:54:02.339Z] Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl (837 kB)
[task 2025-01-29T17:54:02.342Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 837.2/837.2 kB 694.2 MB/s eta 0:00:00
[task 2025-01-29T17:54:02.355Z] Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2025-01-29T17:54:02.369Z] Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
[task 2025-01-29T17:54:02.385Z] Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
[task 2025-01-29T17:54:02.463Z] Building wheels for collected packages: eflomal, pyicu
[task 2025-01-29T17:54:02.464Z]   Building wheel for eflomal (pyproject.toml): started
[task 2025-01-29T17:54:06.683Z]   Building wheel for eflomal (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:54:06.685Z]   Created wheel for eflomal: filename=eflomal-1.0.0b1-cp310-cp310-linux_x86_64.whl size=319815 sha256=37e357f99a347d0c586127f570b263da4a1be77383eee89bc55e4fcdd864d6d8
[task 2025-01-29T17:54:06.685Z]   Stored in directory: /tmp/pip-ephem-wheel-cache-x8g83_ou/wheels/b4/0c/55/4a2c6f00793a0d2fdaef4376b8cb3d9c6ae8df16366d6e31d1
[task 2025-01-29T17:54:06.687Z]   Building wheel for pyicu (pyproject.toml): started
[task 2025-01-29T17:54:50.471Z]   Building wheel for pyicu (pyproject.toml): finished with status 'done'
[task 2025-01-29T17:54:50.477Z]   Created wheel for pyicu: filename=PyICU-2.8.1-cp310-cp310-linux_x86_64.whl size=1747824 sha256=f198e85e87e2dd79f9bc4b04e277163ec6a8bc2c046ba0a7337336484360a0c7
[task 2025-01-29T17:54:50.477Z]   Stored in directory: /tmp/pip-ephem-wheel-cache-x8g83_ou/wheels/6e/cb/d3/5d1dfd3a9e12d51f0a2bfb951374205f7eca2639a21b03ea12
[task 2025-01-29T17:54:50.479Z] Successfully built eflomal pyicu
[task 2025-01-29T17:54:50.513Z] Installing collected packages: pyicu, urllib3, tqdm, opus-fast-mosestokenizer, numpy, idna, cython, charset-normalizer, certifi, requests, eflomal
[task 2025-01-29T17:54:52.991Z] Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 cython-3.0.9 eflomal-1.0.0b1 idna-3.8 numpy-1.26.4 opus-fast-mosestokenizer-0.0.8.5 pyicu-2.8.1 requests-2.31.0 tqdm-4.66.4 urllib3-2.2.2
[task 2025-01-29T17:54:53.294Z] + set -euo pipefail
[task 2025-01-29T17:54:53.294Z] + echo '###### Generating alignments and shortlist'
[task 2025-01-29T17:54:53.294Z] ###### Generating alignments and shortlist
[task 2025-01-29T17:54:53.294Z] + [[ -z /builds/worker/fetches ]]
[task 2025-01-29T17:54:53.294Z] + [[ -z /builds/worker/fetches ]]
[task 2025-01-29T17:54:53.294Z] + [[ -z ru ]]
[task 2025-01-29T17:54:53.294Z] + [[ -z en ]]
[task 2025-01-29T17:54:53.294Z] + corpus_prefix=/builds/worker/fetches/corpus
[task 2025-01-29T17:54:53.294Z] + vocab_path=/builds/worker/fetches/vocab.spm
[task 2025-01-29T17:54:53.294Z] + output_dir=/builds/worker/artifacts
[task 2025-01-29T17:54:53.294Z] + threads=auto
[task 2025-01-29T17:54:53.294Z] + '[' auto = auto ']'
[task 2025-01-29T17:54:53.294Z] ++ nproc
[task 2025-01-29T17:54:53.295Z] + threads=32
[task 2025-01-29T17:54:53.295Z] ++ dirname /builds/worker/checkouts/vcs/pipeline/alignments/generate-shortlist.sh
[task 2025-01-29T17:54:53.296Z] + cd /builds/worker/checkouts/vcs/pipeline/alignments
[task 2025-01-29T17:54:53.296Z] + mkdir -p /builds/worker/artifacts
[task 2025-01-29T17:54:53.297Z] + dir=/builds/worker/artifacts/tmp_shortlist
[task 2025-01-29T17:54:53.297Z] + mkdir -p /builds/worker/artifacts/tmp_shortlist
[task 2025-01-29T17:54:53.298Z] + corpus_src=/builds/worker/fetches/corpus.ru.zst
[task 2025-01-29T17:54:53.298Z] + corpus_trg=/builds/worker/fetches/corpus.en.zst
[task 2025-01-29T17:54:53.298Z] + echo '### Subword segmentation with SentencePiece'
[task 2025-01-29T17:54:53.298Z] ### Subword segmentation with SentencePiece
[task 2025-01-29T17:54:53.298Z] + zstdmt -dc /builds/worker/fetches/corpus.ru.zst
[task 2025-01-29T17:54:53.299Z] + parallel --no-notice --pipe -k -j 32 --block 50M /builds/worker/fetches/spm_encode --model /builds/worker/fetches/vocab.spm
[task 2025-01-29T17:54:53.625Z] + zstdmt -dc /builds/worker/fetches/corpus.en.zst
[task 2025-01-29T17:54:53.625Z] + parallel --no-notice --pipe -k -j 32 --block 50M /builds/worker/fetches/spm_encode --model /builds/worker/fetches/vocab.spm
[task 2025-01-29T17:54:53.915Z] + python3 align.py --corpus_src=/builds/worker/artifacts/tmp_shortlist/corpus.spm.ru --corpus_trg=/builds/worker/artifacts/tmp_shortlist/corpus.spm.en --output_path=/builds/worker/artifacts/corpus.aln
[task 2025-01-29T17:54:53.976Z] [alignments] Running with arguments: ['align.py', '--corpus_src=/builds/worker/artifacts/tmp_shortlist/corpus.spm.ru', '--corpus_trg=/builds/worker/artifacts/tmp_shortlist/corpus.spm.en', '--output_path=/builds/worker/artifacts/corpus.aln']
[task 2025-01-29T17:54:53.978Z] [alignments] Starting generating alignments.
[task 2025-01-29T17:54:54.053Z] [alignments] Splitting corpus into parts
[task 2025-01-29T17:54:54.063Z] [alignments] Processing part aa
[task 2025-01-29T17:54:54.063Z] [alignments] Calculating alignments...
[task 2025-01-29T17:54:54.137Z] [eflomal] Prepared 1472 sentences for alignment
[task 2025-01-29T17:54:54.137Z] /builds/worker/.local/lib/python3.10/site-packages/eflomal/bin/eflomal -m 3 -s /tmp/tmpkt4961_8 -t /tmp/tmp0al32p87 -n 3 -N 0.2 -1 32 -2 32 -3 130 -f /builds/worker/artifacts/tmp/aln.fwd.aa -r /builds/worker/artifacts/tmp/aln.rev.aa
[task 2025-01-29T17:54:54.155Z] Read texts (1472 sentences): 0.017 s
[task 2025-01-29T17:54:54.155Z] Vocabulary sizes are 490 (source), 54 (target)
[task 2025-01-29T17:54:54.157Z] Created alignment structures: 0.000 s
[task 2025-01-29T17:54:54.157Z] Created alignment structures: 0.000 s
[task 2025-01-29T17:54:54.161Z] Randomized alignment: 0.004 s
[task 2025-01-29T17:54:54.161Z] Aligning with model 1 (32 iterations)
[task 2025-01-29T17:54:54.162Z] Randomized alignment: 0.005 s
[task 2025-01-29T17:54:54.162Z] Aligning with model 1 (32 iterations)
[task 2025-01-29T17:54:57.288Z] Done: 3.127 s
[task 2025-01-29T17:54:57.288Z] Aligning with model 2 (32 iterations)
[task 2025-01-29T17:54:57.489Z] Done: 3.327 s
[task 2025-01-29T17:54:57.489Z] Aligning with model 2 (32 iterations)
[task 2025-01-29T17:55:00.485Z] Done: 3.197 s
[task 2025-01-29T17:55:00.485Z] Aligning with model 3 (130 iterations)
[task 2025-01-29T17:55:00.804Z] Done: 3.314 s
[task 2025-01-29T17:55:00.804Z] Aligning with model 3 (130 iterations)
[task 2025-01-29T17:55:14.782Z] Done: 14.297 s
[task 2025-01-29T17:55:15.173Z] Final argmax iteration: 0.391 s
[task 2025-01-29T17:55:15.173Z] Writing alignments to /builds/worker/artifacts/tmp/aln.rev.aa for 1472 sentencess
[task 2025-01-29T17:55:15.656Z] Done: 14.852 s
[task 2025-01-29T17:55:16.055Z] Final argmax iteration: 0.399 s
[task 2025-01-29T17:55:16.055Z] Writing alignments to /builds/worker/artifacts/tmp/aln.fwd.aa for 1472 sentencess
[task 2025-01-29T17:55:16.064Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp/aln.fwd.aa']
[task 2025-01-29T17:55:16.066Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp/aln.rev.aa']
[task 2025-01-29T17:55:16.067Z] [alignments] Symmetrizing alignments...
[task 2025-01-29T17:55:16.304Z] [alignments] Finished generating alignments.
[task 2025-01-29T17:55:16.324Z] + echo '### Creating shortlist'
[task 2025-01-29T17:55:16.324Z] ### Creating shortlist
[task 2025-01-29T17:55:16.324Z] + /builds/worker/fetches/extract_lex /builds/worker/artifacts/tmp_shortlist/corpus.spm.en /builds/worker/artifacts/tmp_shortlist/corpus.spm.ru /builds/worker/artifacts/corpus.aln /builds/worker/artifacts/tmp_shortlist/lex.s2t /builds/worker/artifacts/tmp_shortlist/lex.t2s
[task 2025-01-29T17:55:16.326Z] Starting...
[task 2025-01-29T17:55:16.382Z] 0 
[task 2025-01-29T17:55:16.382Z] Finished
[task 2025-01-29T17:55:16.382Z] + '[' -f /builds/worker/artifacts/tmp_shortlist/lex.s2t ']'
[task 2025-01-29T17:55:16.383Z] + zstdmt /builds/worker/artifacts/tmp_shortlist/lex.s2t
[task 2025-01-29T17:55:16.385Z] 
[task 2025-01-29T17:55:16.385Z]                                                                                
[task 2025-01-29T17:55:16.385Z] Read :  0 /  0 MB ==> 28%
[task 2025-01-29T17:55:16.385Z]                                                                                
[task 2025-01-29T17:55:16.385Z] /builds/worker/artifacts/tmp_shortlist/lex.s2t : 28.13%   ( 66246 =>  18634 bytes, /builds/worker/artifacts/tmp_shortlist/lex.s2t.zst) 
[task 2025-01-29T17:55:16.385Z] + rm /builds/worker/artifacts/tmp_shortlist/corpus.spm.en
[task 2025-01-29T17:55:16.386Z] + rm /builds/worker/artifacts/tmp_shortlist/corpus.spm.ru
[task 2025-01-29T17:55:16.387Z] + rm /builds/worker/artifacts/corpus.aln
[task 2025-01-29T17:55:16.388Z] + echo '### Shortlist pruning'
[task 2025-01-29T17:55:16.388Z] ### Shortlist pruning
[task 2025-01-29T17:55:16.388Z] + /builds/worker/fetches/spm_export_vocab --model=/builds/worker/fetches/vocab.spm --output=/builds/worker/artifacts/tmp_shortlist/vocab.txt
[task 2025-01-29T17:55:16.392Z] + zstdmt -dc /builds/worker/artifacts/tmp_shortlist/lex.s2t.zst
[task 2025-01-29T17:55:16.392Z] + grep -v NULL
[task 2025-01-29T17:55:16.392Z] + python3 prune_shortlist.py 100 /builds/worker/artifacts/tmp_shortlist/vocab.txt
[task 2025-01-29T17:55:16.392Z] + zstdmt
[task 2025-01-29T17:55:16.414Z] + echo '### Deleting tmp dir'
[task 2025-01-29T17:55:16.414Z] ### Deleting tmp dir
[task 2025-01-29T17:55:16.414Z] + rm -rf /builds/worker/artifacts/tmp_shortlist
[task 2025-01-29T17:55:16.415Z] + echo '###### Done: Generating alignments and shortlist'
[task 2025-01-29T17:55:16.415Z] ###### Done: Generating alignments and shortlist
[fetches 2025-01-29T17:55:16.415Z] removing /builds/worker/fetches
[fetches 2025-01-29T17:55:16.658Z] finished
+ exit_code=0
+ docker cp taskcontainer_YpPhxGrxSfy90dfHVMD4lw:/builds/worker/artifacts artifact0
+ docker rm taskcontainer_YpPhxGrxSfy90dfHVMD4lw
taskcontainer_YpPhxGrxSfy90dfHVMD4lw
+ exit 0
[taskcluster 2025-01-29T17:55:17.536Z]    Exit Code: 0
[taskcluster 2025-01-29T17:55:17.536Z]    User Time: 70.823ms
[taskcluster 2025-01-29T17:55:17.536Z]  Kernel Time: 232.5ms
[taskcluster 2025-01-29T17:55:17.536Z]    Wall Time: 2m13.270005437s
[taskcluster 2025-01-29T17:55:17.536Z]       Result: SUCCEEDED
[taskcluster 2025-01-29T17:55:17.536Z] === Task Finished ===
[taskcluster 2025-01-29T17:55:17.536Z] Task Duration: 2m13.270397328s
[taskcluster 2025-01-29T17:55:17.776Z] Uploading artifact public/build/lex.s2t.pruned.zst from file /home/task_173817315102974/artifact0/lex.s2t.pruned.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-02-28T15:56:01.992Z
[taskcluster 2025-01-29T17:55:17.818Z] Uploading artifact public/build/tmp/aln.rev.aa from file /home/task_173817315102974/artifact0/tmp/aln.rev.aa with content encoding "gzip", mime type "audio/x-pn-audibleaudio" and expiry 2025-02-28T15:56:01.992Z
[taskcluster 2025-01-29T17:55:17.834Z] Uploading artifact public/build/tmp/aln.rev from file /home/task_173817315102974/artifact0/tmp/aln.rev with content encoding "gzip", mime type "application/octet-stream" and expiry 2025-02-28T15:56:01.992Z
[taskcluster 2025-01-29T17:55:17.838Z] Uploading artifact public/build/tmp/aln.fwd.aa from file /home/task_173817315102974/artifact0/tmp/aln.fwd.aa with content encoding "gzip", mime type "audio/x-pn-audibleaudio" and expiry 2025-02-28T15:56:01.992Z
[taskcluster 2025-01-29T17:55:17.840Z] Uploading artifact public/build/tmp/aln.fwd from file /home/task_173817315102974/artifact0/tmp/aln.fwd with content encoding "gzip", mime type "application/octet-stream" and expiry 2025-02-28T15:56:01.992Z
[taskcluster 2025-01-29T17:55:18.317Z] [mounts] Preserving cache: Moving "/home/task_173817315102974/cache0" to "caches/BbHgJP_CR0epMIJaCMha8A"
[taskcluster 2025-01-29T17:55:18.449Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2025-02-28T15:56:01.992Z