Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero-shot Translation Prompt #2

Open
10zinten opened this issue Dec 26, 2024 · 1 comment
Open

Zero-shot Translation Prompt #2

10zinten opened this issue Dec 26, 2024 · 1 comment

Comments

@10zinten
Copy link
Contributor

10zinten commented Dec 26, 2024

Zero-shot Translation Prompt

This is our baseline for comparing other experiments.
LLM: claude sonet 3.5
Experiment Name: 01_zero_shot_translation

Hypothesis

None

Prompt

Translate the following Buddhist Tibetan passage into English: {tibetan_text} English:
## Core Instructions
   - Enclose final English translation in <t> tags

Evaluation Results

Corpus BLEU Score: 18.36
Corpus chrF++ Score: 44.11
Corpus TER Score: 100.55

Output:

use the experiment name to find the translation output from results.json

@10zinten 10zinten moved this from TO DO to TESTING in BO-EN MT Product Roadmap Dec 26, 2024
@10zinten 10zinten changed the title Zero-shot Translation Prompt | Claud Sonet 3.5 Zero-shot Translation Prompt Dec 26, 2024
@peterolin
Copy link

Would it make sense in this and other prompts to add a request for Claude to highlight any ambiguities, for instance by adding something like the following to all prompts?

"I would like you to list any points where the interpretation is ambiguous and an unfounded choice of rendering has been made, list these as items within and tag. Summarize with a number of such issues between tags. Additionaly rank this translation from 1-10 based on uncertainties in interpretation and potential incorrenctness in rendering."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: TESTING
Development

No branches or pull requests

2 participants