Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strip leading zeros from CEMS emission_unit_id_epa #402

Merged
merged 3 commits into from
Dec 17, 2024

Conversation

grgmiller
Copy link
Collaborator

@grgmiller grgmiller commented Dec 14, 2024

Purpose

We had noticed that in 2023, the PJM power sector results file had a missing fuel category. I traced this to plant 50852, and it turns out the cause of the issue was that the fuel type backfill using the PSDC was failing because we were not stripping leading zeros from the unit ID when loading the table, so the merge was not working.

After digging into this more, I realized that pudl does not automatically strip leading zeros from emissions_unit_id_epa (see catalyst-cooperative/pudl#3992). This PR adds functionality to always strip these leading zeros whenever loading CEMS data

This PR also updates the sandbox notebook to use 2023 as the year, and also adds some starter functions for loading intermediate output data for exploration since these frequently need to be loaded.

Testing

Ran pipeline for 2023

Where to look

  • It's helpful to clarify where your new code lives if you moved files around or there could be confusion/

  • What files are most important?

Usage Example/Visuals

How the code can be used and/or images of any graphs, tables or other visuals (not always applicable).

Review estimate

How long will it take for reviewers and observers to understand this code change?

Future work

What issues were identified that are not being addressed in this PR but should be addressed in future work?

Checklist

  • Update the documentation to reflect changes made in this PR
  • Format all updated python files using black
  • Clear outputs from all notebooks modified
  • Add docstrings and type hints to any new functions created

@grgmiller grgmiller changed the base branch from main to development December 14, 2024 01:29
@grgmiller grgmiller marked this pull request as ready for review December 17, 2024 16:28
@grgmiller grgmiller changed the title Fix missing fuel s Fix missing fuels in PJM Dec 17, 2024
@grgmiller grgmiller requested a review from rouille December 17, 2024 16:29
Copy link
Collaborator

@rouille rouille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@grgmiller grgmiller changed the title Fix missing fuels in PJM Strip leading zeros from CEMS emission_unit_id_epa Dec 17, 2024
@grgmiller grgmiller merged commit 7d80e74 into development Dec 17, 2024
2 checks passed
@grgmiller grgmiller deleted the greg/missing_pjm branch December 17, 2024 21:10
@rouille rouille mentioned this pull request Dec 20, 2024
4 tasks
@grgmiller grgmiller mentioned this pull request Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants