Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why my training model dont have pkl file #23

Open
lyh512796310 opened this issue Mar 18, 2021 · 9 comments
Open

why my training model dont have pkl file #23

lyh512796310 opened this issue Mar 18, 2021 · 9 comments

Comments

@lyh512796310
Copy link

lyh512796310 commented Mar 18, 2021

i want to know that how to get the “dictionary.pkl” file
and how to generate this file

@songhe17
Copy link

Same problem here

@remyhuang
Copy link
Contributor

from collections import Counter
import pickle

all_elements= []
for midi_file in you_all_midi_files:
    events = model.extract_events(midi_file)
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

I’m not sure if you can run it directly, but it should be easy for you to modify.

@dhrumilp15
Copy link

dhrumilp15 commented May 21, 2021

You can use glob to grab your midi files! We can pull out extract_events from model.py to make the dictionary.

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

This grabs all files in the data folder with file extensions that start with .mid (This covers .mid and .midi files).

@ShacoLAY
Copy link

ShacoLAY commented Aug 7, 2021

Thank you for the amazing work on this project!

However, I have a problem for several days and would like to ask for help, I was able to generate my own music training dictionary according to the above code, but it does not work with the original REMI-tempo-checkpoint (I am directly replacing the previous dictionary with my own) and then the following error occurs:
屏幕截图 2021-08-08 153149

(5$UO2YM@02{_U1OB~HTK@9

In addition, can I use my own training data to generate my own checkpoint and evaluation my own music?
Do you have any insights about this problem?

Thanks again for your kind help!

@dedededefo
Copy link

感谢您在这个项目上所做的出色工作!

但是我这几天有问题想求助,我可以按照上面的代码生成自己的音乐训练字典,但是用原来的REMI-tempo-checkpoint不行(我直接用我自己的字典替换以前的字典),然后出现以下错误: 屏幕截图 2021-08-08 153149

(5$UO2YM@02{_U1OB~HTK@9

另外,我可以使用自己的训练数据来生成自己的检查点并评估自己的音乐吗? 你对这个问题有什么见解吗?

再次感谢您的帮助!

Hello, have you solved this problem? I have the same problem

@dedededefo
Copy link

您可以使用 glob 来获取您的 MIDI 文件!我们可以从中取出来extract_events制作model.py字典。

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

Uploading a7131adf7e2e0210f9f6fb7e4a1bf8e.png…

这将抓取文件data夹中文件扩展名以.mid(This cover .midand .midifiles) 开头的所有文件。

您可以使用 glob 来获取您的 MIDI 文件!我们可以从中取出来extract_events制作model.py字典。

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

这将抓取文件data夹中文件扩展名以.mid(This cover .midand .midifiles) 开头的所有文件。
image
Hello, why do I report an error when using my MIDI data to generate a dictionary? In the case of multiple MIDI, it will report an error, but one or two will be fine. What's the use of a dictionary

@dedededefo
Copy link

from  collections  import  Counter 
import  pickle

all_elements = []
用于 you_all_midi_files 中的 midi_file事件 = 模型extract_events ( midi_file )
    用于 事件 中的 事件element  =  '{}_{}'格式事件名称事件all_elements附加元素counts  =  Counter ( all_elements )
 event2word  = { c : i  for  i , c  in  enumerate ( counts.keys ( ) )}
 word2event = { i : c for i , c in enumerate ( counts.keys  ( ) )
 } pickle转储((event2wordword2event),打开'dictionary.pkl'    , 'wb' ))

我不确定你是否可以直接运行它,但它应该很容易修改。

Must a dictionary of the same size as the pre training checkpoint be generated

@dedededefo
Copy link

您可以使用 glob 来获取您的 MIDI 文件!我们可以从中取出来extract_events制作model.py字典。

from collections import Counter
import pickle
import glob
import utils
def extract_events(input_path, chord=False):
    note_items, tempo_items = utils.read_items(input_path)
    note_items = utils.quantize_items(note_items)
    max_time = note_items[-1].end
    if chord:
        chord_items = utils.extract_chords(note_items)
        items = chord_items + tempo_items + note_items
    else:
        items = tempo_items + note_items
    groups = utils.group_items(items, max_time)
    events = utils.item2event(groups)
    return events

all_elements= []
for midi_file in glob.glob("./data/*/*.mid*", recursive=True):
    events = extract_events(midi_file) # If you're analyzing chords, use `extract_events(midi_file, chord=True)`
    for event in events:
        element = '{}_{}'.format(event.name, event.value)
        all_elements.append(element)

counts = Counter(all_elements)
event2word = {c: i for i, c in enumerate(counts.keys())}
word2event = {i: c for i, c in enumerate(counts.keys())}
pickle.dump((event2word, word2event), open('dictionary.pkl', 'wb'))

这将抓取文件data夹中文件扩展名以.mid(This cover .midand .midifiles) 开头的所有文件。

Do you have to generate a dictionary of the same size as the pre training checkpoint to train your own data

@Hikari-Tsai
Copy link

Just for the record, this discuss only for train task from scratch. If you use finetune.py to train your model,just need copy the original dictionary.pkl from pretrain document.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants