Add dataset train-test split.

eth-nlped · Nov 20, 2023 · a109b8f · a109b8f
1 parent 0362d91
commit a109b8f
Show file tree

Hide file tree

Showing 7 changed files with 30,755 additions and 5,724 deletions.
diff --git a/README.md b/README.md
@@ -23,7 +23,9 @@ The dataset is released publicly.
 
 # Dataset
 The dataset is available in the data folder. It contains all 2861 conversations. 
-For full dataset see `data/mathdial.tsv` (tsv format) or `data/mathdial.jsonl` (jsonl format) and for a small sample see `data/example.jsonl`
+The dataset is split into train and test - see `data/train.csv` and `data/test.csv` (for csv format). 
+For jsonl format, see `data/train.jsonl` and `data/test.jsonl`. To see a small sample of the dataset, look at `data/example.jsonl`
+Please note that each row in the file consists of full conversations between a teacher and a student delimited with special `|EOM|` notation. 
 
 ## Data Structure
 - `qid` - unique identifier of the problem

diff --git a/data/mathdial.jsonl b/data/mathdial.jsonl
diff --git a/data/mathdial.tsv b/data/mathdial.tsv
diff --git a/data/test.csv b/data/test.csv
diff --git a/data/test.jsonl b/data/test.jsonl
diff --git a/data/train.csv b/data/train.csv
diff --git a/data/train.jsonl b/data/train.jsonl