Inconsistent leading zeros in emissions_unit_id_epa

[grgmiller]

Describe the bug

As has been noted in other issues, leading zeros can cause issues when crosswalking data (#964 and #2366).

It appears that EPA data suffers from the same issue with emissions_unit_id_epa, specifically across the hourly CEMS emissions data, and the core_epa__assn_eia_epacamd table.

Some examples:

Plant 50852, unit 002001:

• CAMD online database: 002001
• PSDC: 2001
• PUDL hourly emissions table: 2001

Plant 2446, unit 051B:

• CAMD online database: 051B
• PSDC: 051B
• PUDL hourly emissions table: 051B

It appears that EPA is inconsistent with how if it strips zeros across its data products: in some cases, it strips them, and in other cases, it leaves them in place. Even within the PSDC, there are some units that have leading zeros and some where this is stripped. This is leading to issues when trying to crosswalk both sources across time.

I'd propose that PUDL programmatically strip leading zeros from this field like you already do for boiler and generator IDs.

Bug Severity

How badly is this bug affecting you?

• Medium: With some effort, I can work around the bug.

Expected behavior

Consistent EPA unit IDs across datasets

Software Environment?

N/A

Additional context

This is causing some issues with crosswalking in OGE. Currently, we are manually stripping leading zeros once we import the data.

[e-belfer]

Right now we are calling the helper method remove_leading_zeros_from_numeric_strings on the emissions_unit_id_epa columns in core_epa__assn_eia_epacamd and in the transform method in pudl.transform.epacems. The problem is that the EPA unit IDs aren't always numeric, as the example provided above (051B) illustrates. We're only removing leading zeros when there are only numbers (e.g., 002001 becomes 2001), not when letters are included because we haven't seen cases where the alphanumeric values have inconsistent leading zeros. It's the same way that we process our generator and boiler IDs.

Thanks for catching this one and alerting us, @grgmiller ! Are there cases that you're seeing where alphanumeric IDs with leading zeros are causing matching problems?