Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

118 distilbert api #132

Merged
merged 33 commits into from
Aug 10, 2023
Merged

118 distilbert api #132

merged 33 commits into from
Aug 10, 2023

Conversation

yiwen-h
Copy link
Member

@yiwen-h yiwen-h commented Aug 9, 2023

Closes #118 by creating docker image which:

  • installs pxtextmining==0.5.4
  • mounts a data folder containing a json file
  • makes predictions using the json data
  • outputs the predictions into json format and saves them in the data folder

It has been implemented in Azure via https://github.com/tomjemmett/pxtextmining_docker_api

yiwen-h added 20 commits July 3, 2023 15:04
@yiwen-h
started basic framework for dockerizing sentiment endpoint
8556d2b
@yiwen-h
working docker container - cant use Alpine
243c69b
@yiwen-h
got docker container to mount data folder, accept filename as argument
3ad8e6b
@yiwen-h
json input file now deleted if NOT run locally
64fe780
@yiwen-h
Predictions now outputted as json file in data_out folder
86414f2
@yiwen-h
added label to dockerfile
b726388
@yiwen-h
added most tests for docker_run
6ab83a3
@yiwen-h
added larger json file - about 8000 comments
da605dc
@yiwen-h
wrote get_y_score function
e853659
@yiwen-h
prediction dfs now include probabilities as well
9f135e6
@yiwen-h
prediction dfs now include probabilities for sklearn multilabel
439dece
@yiwen-h
added macro roc auc score to model summary
78632fc
@yiwen-h
write_model_preds now uses probabilities from predict_multilabel df o…
db180e0 
…utput
@yiwen-h
added model_performance.additional_analysis which calculates confusio…
6eb5259 
…n matrix and auc_roc scores for labels
@yiwen-h
confusion matrix info and roc_auc_score now in model analysis
42b5ec6
@yiwen-h
Replaced macro roc_aoc score with average_precision_score in perf ana…
368f9d8 
…lysis
@yiwen-h
renamed "support" column to be more userfriendly
bfb6dca
@yiwen-h
fixed ruff complaining about == instead of isinstance in tests
40a3cfd
@yiwen-h
fixed ruff complaining about == instead of isinstance in test_factory…
f1256ed 
…_pipeline
@yiwen-h
some broken dependencies causing test to fail, trying to fix pyprojec…
301c8b7 
…t.toml
@yiwen-h yiwen-h merged commit 5894def into development Aug 10, 2023
3 checks passed
@yiwen-h yiwen-h deleted the 118_distilbert_API branch August 10, 2023 10:42
ChrisBeeley
ChrisBeeley reviewed Aug 11, 2023
View reviewed changes
Copy link
Member

@ChrisBeeley ChrisBeeley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ChrisBeeley ChrisBeeley ChrisBeeley left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yiwen-h @ChrisBeeley