Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter app #104

Merged
merged 17 commits into from
Oct 8, 2024
Merged

Filter app #104

merged 17 commits into from
Oct 8, 2024

Conversation

danyoungday
Copy link
Collaborator

Redid app to use our new flow of selecting a context -> filtering prescriptors by desired outcome -> choosing a prescriptor -> play with the actions -> see the outcomes.

Additionally updated deployment to download a smaller, preprocessed dataset instead of the entire thing. We also save the demo models in the repo for simplicity.

@danyoungday danyoungday self-assigned this Oct 7, 2024
from app.components.references import ReferencesComponent
from app.components.sliders import SlidersComponent
from app.components.trivia import TriviaComponent
from app.utils import EvolutionHandler
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We get rid of all the old components but reuse some in our new components. The components now correspond to portions of the page rather than by functionality which makes things neater

intro_component = IntroComponent()
context_component = ContextComponent(app_df, evolution_handler)
filter_component = FilterComponent(evolution_handler)
dms_component = DMSComponent(app_df, evolution_handler)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We pass our app df and evolution handler to the components so that they have the ability to prescribe, look at context, etc.

in tons of carbon per hectare)
context_component.register_callbacks(app)
filter_component.register_callbacks(app)
dms_component.register_callbacks(app)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of all the long series of callbacks we now register all the callbacks at once with these simple functions. We could abstract these into a general Component class later and just call register_callbacks on them all.

dms_component.get_div(),
references_component.get_references_div()
]
)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Layout is now super clean. Just calls get_div from each of our components in the order they should display on the apge

window.dccFunctions = window.dccFunctions || {};
window.dccFunctions.percentSlider = function(value) {
return Math.round(value * 100);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that our sliders show as percents and not proportions (10% vs. 0.1)

Component containing map as well as dropdowns and input fields for picking a more specific context.
"""
def __init__(self, app_df: pd.DataFrame, handler: EvolutionHandler):
self.map_component = MapComponent(app_df)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We reuse the old map component in our new context component

def create_label_and_value(self, label: str, value: html.Div) -> html.Div:
"""
Standard dash function that pairs a label with any arbitrary value Div.
"""
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function lines up a label with a div so we reuse it for the Lat/dropdown, lon/dropdown, year/input

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trimmed a lot of the fat off and only kept what we use in the new app

Input("year-input", "value"),
Input("lat-dropdown", "value"),
Input("lon-dropdown", "value")
)
def update_context_chart(chart_type, year, lat, lon):
def update_context_chart(year, lat, lon):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use the pie charts anymore but may add them back in later

presc = pd.Series(sliders, index=constants.RECO_COLS)
# If we have no prescription just show the context chart
bad_sliders = sum(sliders) < -0.01 or sum(sliders) > 0.01
if all(slider == 0 for slider in sliders) or bad_sliders:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We add a prescribe + predict button instead of automatically updating the chart so we don't have to validate that the sliders sum to 100 on the fly

Output("pbar", "children"),
[Input({"type": "diff-slider", "index": ALL}, "value")]
)
def update_pbar(sliders) -> int:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use a progress bar to show how much land is used

class FilterComponent:
"""
Component with sliders/preset buttons to filter outcomes by as well as visualizing the selected prescriptors.
"""
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is almost the exact same as the climate change demo

"""
Plots the prescriptions for the selected candidates and grays the rest out, shifting them to the right.
Also adds the original context as the far leftmost bar.
"""
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic for our nice stacked bar chart

def select_preset(n_clicks, results_json):
"""
Selects a preset for the filter sliders from the 3 presets.
"""
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We make the presets just slice the outcomes into thirds and each third is our preset

@@ -1,4 +1,4 @@
dash==2.10.2
dash==2.15
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed to update dash to get functionality to have the sliders show as percentages

hf_hub_download(repo_id="projectresilience/land-use-app-data",
filename="app_data.csv",
local_dir="app/data",
repo_type="dataset")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our process data script no longer downloads the entire dataset then processes it. It simply downloads a preprocessed file I left in huggingface

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scaler for our predictor/prescriptors

def add_nonland(context_actions_df: pd.DataFrame) -> pd.DataFrame:
"""
Adds a nonland column to the context_actions_df
"""
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is modified to take a dataframe now instead of a series

cand_path = self.prescriptor_path / f"{cand_id}.pt"
candidate = Candidate(**candidate_params, cand_id=cand_id)
candidate.load_state_dict(torch.load(cand_path))
prescriptors[cand_id] = LandUsePrescriptor(candidate, encoder)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now the prescriptors are hard-coded and saved in the repo. In the future these should live in HuggingFace

@@ -8,7 +8,7 @@ dask==2023.6.1
datasets==2.16.1
flake8==7.1.0
Flask==2.2.5
geopandas==0.13.2
geopandas==0.14.4
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated geopandas because we were getting an error with the fiona package

@@ -20,7 +20,7 @@ pandas==1.5.3
plotly==5.14.1
protobuf==3.20.3
-e git+https://github.com/Project-Resilience/sdk.git@c0e5300e56f5fce2f6cf55c328c83e7055e541be#egg=prsdk
pylint==2.17.4
pylint==3.3.1
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated pylint as we were linting unnecessary files

Copy link
Member

@ofrancon ofrancon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm


if __name__ == '__main__':
app.run_server(host='0.0.0.0', debug=False, port=4057, use_reloader=False, threaded=False)
app.run_server(host='0.0.0.0', debug=False, port=4057, use_reloader=True, threaded=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to leave user_reloader to True?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://dash.plotly.com/devtools you're not changing the code in the deployed app, and it makes it slower to start, so I'll leave it to False

@danyoungday danyoungday merged commit 95f48ac into main Oct 8, 2024
1 check passed
@danyoungday danyoungday deleted the filter-app branch October 8, 2024 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants