Bug Fixes

Fixed

• Bug fixed in Seq2Seq tasks.
• Bug fixed in NERModel where the classification report was missing in checkpoints.
• Bug fixed in ELECTRA.
• Bug fixed in Seq2Seq generic encoder-decoder model.
• Bug fixed in Seq2Seq tasks.
• Bug fixed in regression prediction.
• Bug fixed in loading multiclass classification models when num_labels aren't specified.

Custom Parameter Groups (Freezing layers)

Added

• Added support for custom parameter groups. docs

Fixed

• ELECTRA pretraining no longer replaces 10% of masked inputs with random tokens. @dev-sngwn

MobileBERT, Chinese text handling, and Transformers 3.0.1 compatibility

Added

• Added better support for Chinese text in Language Modeling. @taranais
• Added mobilebert for classification, NER, QA, and Seq2Seq. @flozi00

Fixed

• Fixed incorrect logic when using early_stopping_metric_minimize. @djstrong
• Fixed issue with cache directory being created even when no_cache is set. @henninglebbaeus

Changed

• Running loss is now shown next to the tqdm bar (with the tqdm bar description)
• Removed tokenizers and transformers version pins (added earlier to avoid compatibility issues)

interact_single() for Conv AI and bug fixes

Added

• Added interact_single() method to ConvAIModel. This accepts a message and conversation history (and an optional personality). @Amit80007

Fixed

• Fixed bug in multi modal classification @tekkon
• Fixed bug in Multi-Modal classification when using evaluate_during_training.

Changed

• Cleaned language_modeling_utils.py. @Pradhy729

Hyperparameter optimization and Dataclass args

Added

• All Simple Transformers models can now be used with W&B sweeps. (docs)
• eval_model() now logs to wandb in the following models (can be turned off with wandb_log=False)
  • ClassificationModel
  • NERModel
• Model args can now be specified through the relevant dataclass. (E.g. ClassificationArgs, T5Args, etc.) (docs)
• All model args are now saved and loaded automatically with Simple Transformers models.
• Multiprocessed decoding support added to Seq2Seq models
• Custom label support for Classification tasks (except multilabel). (docs)
• NER labels can be given as NERArgs.labels_list (persists through args saving)

Changed

• Changed NERModel.eval_model() to return usable model_outputs
• Improved tqdm descriptions in progress bars
• ConvAIModel arg no_sample renamed to do_sample for consistency

Resolving compatibility issues introduced in transformers v2.11.0

Fixed

• Fixed compatibility issues with breaking changes in transformers==2.11.0. @fcggamou

Lazy Loading Classification Datasets

Added

• Added Lazy Loading support for classification tasks (except multi-label). (Docs)

Longformer for Language Modeling

Added

• Added Longformer model support for Language Modeling.

XLM-RoBERTa support for Question Answering

Added

• Added XLM-RoBERTa support for question answering tasks.
• Added save_optimizer_and_scheduler (default 1) to global_args which controls whether optimizer and scheduler is saved along with the model. Disabling significantly reduces the disk space used by saved models.

Fixed

• Bug in XLM tokenizer when preprocessing QA datasets.
• QuestionAnsweringModel.predict(n_best_size=n) now correctly returns n answers per question (along with n probabilities).

BREAKING CHANGE

• QuestionAnsweringModel.predict() now returns two lists (a list of dicts with question ids mapped to answers and a list of dicts with question ids mapped to the answer probabilities). @alexucb

Fixed major issues with ELECTRA language model training from scratch

Fixed

• Fixed issues with training ELECTRA language models from scratch. @aced125 @Laksh1997
• Fixed bug in save_discriminator() method.

Changed

• The parallel process count is now limited to 61 by default on Windows systems. @leungi