how create a pre-trained domain specific Bert language model?

[puli83]

HI,

This is not a realy issue, but I did not find another way to contact you.

How you wrote in your Medium article, "In the near future, I will be publishing tutorials on how I further pre-trained BERT to create ESG-BERT" (here : https://medium.com/towards-data-science/nlp-meets-sustainable-investing-d0542b3c264b)

I cannot find this tutorial or a document that talk about it.

I have a very different domain but a similar task. I want to create a pre-trained Bert model specialised in specific feld (Phylosophy of science) and then, I'll fine-tune the model for text classification (related to phyosophical concept for the domain)

Can you give me some info about this tutorial to how create a pre-trained domain specific Bert language model?

you can write to me at the following address: [email protected]

I'll really appreciate your help, if you can. You made an excellent job.

Best regards,