Skip to content

Latest commit

 

History

History
43 lines (36 loc) · 1.64 KB

README.md

File metadata and controls

43 lines (36 loc) · 1.64 KB

ML Fill-in-the-Blanks

Google BERT Representative Image

ML Fill-in-the-Blanks is a natural language processing (NLP) model that is trained to predict the missing word in the sentence. This project uses pre-trained bert-base-uncased as the prediction model.

Learn more about BERT in this explainer. A visual guide is also available for access.

Details

  • Model - bert-base-uncased
  • Pre-trained task - MaskedLM

Usage

git clone [email protected]:mabreyes/ml-fill-in-the-blanks.git
cd ml-fill-in-the-blanks
pip install -r requirements.txt
python manage.py migrate
python manage.py collectstatic
python manage.py runserver

Visit the site in your local machine at 127.0.0.1:8000

Deploying to Heroku

  1. Setup Heroku CLI on your local machine. Read about it here.
  2. Create Heroku project on your working directory.
cd ml-fill-in-the-blanks
heroku create <my-project>
  1. Add files to staging and commit as usual.
git add .
git commit -m "Initial commit"
  1. Push to Heroku.
git push heroku <branch>

Site can now be accessed at https://<my-project>.herokuapp.com.

Notes

The attention visualization is done for layer 3 across all attention heads by taking their average. Read more about heads and what they mean on this article.