Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup PR for existing Survey Assist RF PR #938 #995

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6471a96
add copy of code used in TRB paper
hlu109 Aug 12, 2022
9d6a1af
update user uuid lookup; add documentation note
hlu109 Dec 16, 2022
9bd9b18
Add additional logging to the calculation so that we can monitor the …
shankari Feb 15, 2023
1b9ece0
making `cluster_performance.ipynb`, `generate_figs_for_poster` and `…
humbleOldSage Aug 22, 2023
e7d2a14
Unified Interface for fit function
humbleOldSage Aug 26, 2023
59633e0
Fixing `models.py` to support `regenerate_classification_performance_…
humbleOldSage Aug 30, 2023
0adb5fe
[PARTIALLY TESTED] Single database read and Code Cleanuo
humbleOldSage Sep 14, 2023
e9abd51
[PARTIALLY TESTED] Survey Assist Using RF
humbleOldSage Oct 2, 2023
3820d87
[NOT TESTED]Predict implemented
humbleOldSage Oct 3, 2023
5b2572e
[NOT TESTED] Model storage and Model Testing included
humbleOldSage Oct 9, 2023
bf7f406
[TESTED]Forest Model Integration
humbleOldSage Nov 2, 2023
1d7be5a
Minor fixes
humbleOldSage Nov 2, 2023
b3d0db2
Delete Config file
humbleOldSage Nov 3, 2023
c514fe0
Merge remote-tracking branch 'e-mission-eval-private-data/move-models…
humbleOldSage Nov 3, 2023
3b038a9
removedfile
humbleOldSage Nov 3, 2023
94fc848
Update model.py
humbleOldSage Nov 3, 2023
87f109c
Merge branch 'master' into SurveyAssistUsingRandomForest
humbleOldSage Dec 7, 2023
33cdaab
[Tested, Will fail]Integrating RF model on server and more Unit test
humbleOldSage Dec 9, 2023
01fcb2a
minor fix
humbleOldSage Dec 9, 2023
f5fec64
Delete model.py
humbleOldSage Dec 12, 2023
585cc90
Update TestForestModel.py
humbleOldSage Dec 13, 2023
61bbe3f
Minor Fixes
humbleOldSage Dec 16, 2023
a32ce4f
[Tested]Adding Integration test
humbleOldSage Jan 2, 2024
052cb08
Improving test
humbleOldSage Jan 10, 2024
104dd9a
Integration Testing for forest model
humbleOldSage Feb 5, 2024
1b523ed
[Tested] Improvements for model integration
humbleOldSage Mar 15, 2024
35a1346
Forest Model related data additions
humbleOldSage Mar 21, 2024
19bb394
Update TestForestModelIntegration.py
humbleOldSage Mar 22, 2024
450094c
[TESTED] Updated ForestModelLoadAndSave.py
humbleOldSage Mar 22, 2024
ad968de
Fixing TestForestModelLoadandSave.py
humbleOldSage Mar 28, 2024
6d4cc3a
Import format changed + Whitespace / Newline fixes
Nov 18, 2024
419babb
Reverting changes; utility functions already present in clustering.py
Nov 18, 2024
e7f5d21
Cleaned up TestForestModelLoadandSave.py
Nov 21, 2024
6daf0b8
Using train / test data split + Added value-check tests + Reduced ins…
Nov 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update TestForestModel.py
Improving the test file by changing the way previpous predictions are stored.
humbleOldSage committed Dec 13, 2023
commit 585cc9038220732975894177a9a29672b1c3bffc
52 changes: 35 additions & 17 deletions emission/tests/modellingTests/TestForestModel.py
Original file line number Diff line number Diff line change
@@ -114,20 +114,38 @@ def testRandomForestRegression(self):

## predictions take the form like :
#
#{'labels': {'mode_confirm': 'ebike', 'replaced_mode': 'walk', 'purpose_confirm': 'dog-park'}, 'p': 1.0}
# we can store these predictions in a json and then for every run other than the first we
# can load the predictions and compare

try:
if os.path.exists(file_path) and os.path.getsize(file_path)>0:
with open(file_path, 'r') as f:
prev_predictions_list = json.load(f)
logging.debug()
self.assertEqual(prev_predictions_list,curr_predictions_list," previous predictions should match current predictions")
else:
with open(file_path,'w') as file:
json.dump(curr_predictions_list,file,indent=4)
logging.debug("Previous predicitons stored for future matching" )
except json.JSONDecodeError:
logging.debug("jsonDecodeErrorError")
return " decoding JSON."
#{'labels': {'mode_confirm': 'ebike', 'replaced_mode': 'walk', 'purpose_confirm': 'dog-park'}, 'p': 1.0}

#Below are two ways we can store prev. predictions list . Whichever way we finalise, I'll delete the other one.
#
#Method 1 : Run predictions for the first time and hardcode them into
#prev_prdictions_list. For every iteration, simply compare them
#
# for the current data that we read from json, the predictions we get is an empty list. If
# we use a different file with more data, this'll take the for as mentioned above
#
prev_predictions_list= [
(
[],
-1
)
]

self.assertEqual(prev_predictions_list,curr_predictions_list," previous predictions should match current predictions")


#Method 2 ( which was failing): Store these predictions into a json and read from
#that json
#
# try:
# if os.path.exists(file_path) and os.path.getsize(file_path)>0:
# with open(file_path, 'r') as f:
# prev_predictions_list = json.load(f)
# logging.debug()
# self.assertEqual(prev_predictions_list,curr_predictions_list," previous predictions should match current predictions")
# else:
# with open(file_path,'w') as file:
# json.dump(curr_predictions_list,file,indent=4)
# logging.debug("Previous predicitons stored for future matching" )
# except json.JSONDecodeError:
# logging.debug("jsonDecodeErrorError")