Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor fixes #291

Merged
merged 2 commits into from
Jul 28, 2024
Merged

minor fixes #291

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cd3e6a7
fixed typos and formatting
d-kleine Jul 28, 2024
a551189
Merge remote-tracking branch 'upstream/main'
d-kleine Jul 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions ch07/04_preference-tuning-with-dpo/create-preference-data-ollama.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,13 @@
" 2. We use the instruction-finetuned LLM to generate multiple responses and have LLMs rank them based on given preference criteria\n",
" 3. We use an LLM to generate preferred and dispreferred responses given certain preference criteria\n",
"- In this notebook, we consider approach 3\n",
"- This notebook uses a 70 billion parameter Llama 3.1-Instruct model through ollama to generate preference labels for an instruction dataset\n",
"- This notebook uses a 70 billion parameters Llama 3.1-Instruct model through ollama to generate preference labels for an instruction dataset\n",
"- The expected format of the instruction dataset is as follows:\n",
"\n",
"\n",
"### Input\n",
"\n",
"```python\n",
"```json\n",
"[\n",
" {\n",
" \"instruction\": \"What is the state capital of California?\",\n",
Expand All @@ -71,7 +71,7 @@
"\n",
"The output dataset will look as follows, where more polite responses are preferred (`'chosen'`), and more impolite responses are dispreferred (`'rejected'`):\n",
"\n",
"```python\n",
"```json\n",
"[\n",
" {\n",
" \"instruction\": \"What is the state capital of California?\",\n",
Expand All @@ -98,7 +98,7 @@
"]\n",
"```\n",
"\n",
"### Ouput\n",
"### Output\n",
"\n",
"\n",
"\n",
Expand Down Expand Up @@ -135,7 +135,7 @@
"id": "8bcdcb34-ac75-4f4f-9505-3ce0666c42d5",
"metadata": {},
"source": [
"## Installing Ollama and Downloading Llama 3"
"## Installing Ollama and Downloading Llama 3.1"
]
},
{
Expand Down Expand Up @@ -353,7 +353,7 @@
"source": [
"from pathlib import Path\n",
"\n",
"json_file = Path(\"..\") / \"01_main-chapter-code\" / \"instruction-data.json\"\n",
"json_file = Path(\"..\", \"01_main-chapter-code\", \"instruction-data.json\")\n",
"\n",
"with open(json_file, \"r\") as file:\n",
" json_data = json.load(file)\n",
Expand Down Expand Up @@ -498,7 +498,7 @@
"metadata": {},
"source": [
"- If we find that the generated responses above look reasonable, we can go to the next step and apply the prompt to the whole dataset\n",
"- Here, we add a `'chosen`' key for the preferred response and a `'rejected'` response for the dispreferred response"
"- Here, we add a `'chosen'` key for the preferred response and a `'rejected'` response for the dispreferred response"
]
},
{
Expand Down