FERC1 2022

[jdangerx]

This integrates the 2022 FERC1 data into PUDL!

Changes:

• Use ferc-xbrl-extractor==1.2.0, which loads all XBRL facts into the SQLite files instead of trying to grab facts from only one filing. This necessitates deduplicating the XBRL facts by the XBRL context (entity ID, period, utility type, etc.) and grabbing only the facts from the latest applicable filing.
  • in doing this, I added hypothesis and pandera to facilitate property-based testing - it's a little slow, so we should use sparingly, but it's a thing we can do now!
• Update a bunch of settings so we actually try to use the 2022 data
• Update retained_earnings - the DBF data includes "previous year" and "current year" data for some factoids, and XBRL only includes "current year." For compatibility across datasets we added the "previous year" data to the XBRL data, but were doing so in a way that assumed only one XBRL year.

Once #2947 is merged, we will also have the following changes:

• we assign 'report_year' for XBRL data by checking the actual XBRL date columns, instead of associating the XBRL ReportYear fact with a filing name and pulling report year from filing name.
  • this was an issue because we interpret "report year" as "the year the data describes" everywhere else, but because some contexts don't have ReportYears associated, and many filings include data for multiple years, we can't actually associate one filing to one year.
• validation test updates - see comments below for rationale.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b51d235) 88.5% compared to head (76e456e) 88.6%.

Additional details and impacted files

@@          Coverage Diff          @@
##             dev   #2948   +/-   ##
=====================================
  Coverage   88.5%   88.6%           
=====================================
  Files         91      91           
  Lines      10808   10854   +46     
=====================================
+ Hits        9572    9618   +46     
  Misses      1236    1236           

Files	Coverage Δ
src/pudl/helpers.py	86.5% <100.0%> (+0.1%)	⬆️
src/pudl/io_managers.py	90.2% <100.0%> (+1.5%)	⬆️
src/pudl/metadata/sources.py	100.0% <ø> (ø)
src/pudl/transform/ferc1.py	96.6% <100.0%> (+<0.1%)	⬆️
src/pudl/transform/params/ferc1.py	100.0% <ø> (ø)
src/pudl/validate.py	49.4% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.