bleu does not exist

[jiangsufirstlove]

FileNotFoundError: Couldn't find a module script at /data1/doc_data/ysh/OmniDocBench/bleu/bleu.py. Module 'bleu' doesn't exist on the Hugging Face Hub either.

[jiangsufirstlove]

Hello! Thank you for your contribution! I'd like to ask you what is the reason for this error reported when I'm doing a formula assessment and bleu is selected in the metrics

[ouyanglinke]

Please check the version of evaluate.

pip list | grep evaluate
# output: evaluate                  0.4.3

[jiangsufirstlove]

(omnidocbench) ysh@ubuntu:/data1/doc_data/ysh/OmniDocBench$ pip list | grep evaluate
evaluate 0.4.3
That's the same version I have.

[ouyanglinke]

Can you show me your end2end.yaml?

[jiangsufirstlove]

recogition_eval:
metrics:
- Edit_dist
- CDM
#- BLEU
dataset:
dataset_name: omnidocbench_single_module_dataset
ground_truth:
data_path: /data1/doc_data/ysh/OmniDocBench/scripts/eval_formulas8.jsonl
data_key: latex
category_filter: ['equation_isolated']
prediction:
data_key: pred_latex
category_type: formula
that is it

[jiangsufirstlove]

end2end.yaml
end2end_eval:
metrics:
text_block:
metric:
- Edit_dist
- BLEU
- METEOR
display_formula:
metric:
- Edit_dist
- CDM
table:
metric:
- TEDS
- Edit_dist
reading_order:
metric:
- Edit_dist
dataset:
dataset_name: end2end_dataset
ground_truth:
data_path: ./demo_data/omnidocbench_demo/OmniDocBench_demo.json
prediction:
data_path: ./demo_data/end2end
match_method: quick_match
# filter:
# language: english

[ouyanglinke]

Please check your proxy. It's HTTPS connection issue. Make sure you can connect HuggingFace.

[jiangsufirstlove]

OK，i will try，thank you very much！

…

---- Replied Message ---- | From | Linke ***@***.***> | | Date | 02/13/2025 16:15 | | To | opendatalab/OmniDocBench ***@***.***> | | Cc | jiangsufirstlove ***@***.***>, Author ***@***.***> | | Subject | Re: [opendatalab/OmniDocBench] bleu does not exist (Issue #26) | Please check your proxy. It's HTTPS connection issue. Make sure you can connect HuggingFace. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[xsank]

Please check your proxy. It's HTTPS connection issue. Make sure you can connect HuggingFace.

I could connect to the HF, but still with the same problem...

[ouyanglinke]

You can download BLEU from Huggingface.

Then change the code in

OmniDocBench/metrics/cal_metric.py

Line 100 in f41a4ec

bleu = evaluate.load("bleu", keep_in_memory=True, experiment_id=random.randint(1,1e8))

to

bleu = evaluate.load('/your/download/path/')

[jiangsufirstlove]

thank you very much！

…

---- Replied Message ---- | From | Linke ***@***.***> | | Date | 02/27/2025 22:08 | | To | opendatalab/OmniDocBench ***@***.***> | | Cc | jiangsufirstlove ***@***.***>, Author ***@***.***> | | Subject | Re: [opendatalab/OmniDocBench] bleu does not exist (Issue #26) | You can download BLEU from Huggingface. Then change the code in https://github.com/opendatalab/OmniDocBench/blob/f41a4ecb017844e9af67622ac4e18266b1352206/metrics/cal_metric.py#L100 to bleu=evaluate.load('/your/download/path/') — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> ouyanglinke left a comment (opendatalab/OmniDocBench#26) You can download BLEU from Huggingface. Then change the code in https://github.com/opendatalab/OmniDocBench/blob/f41a4ecb017844e9af67622ac4e18266b1352206/metrics/cal_metric.py#L100 to bleu=evaluate.load('/your/download/path/') — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>