Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #3005 and #3592: Hide null ORCID fields in contributor representation and correct flipped starting/ending balance for FERC1 liabilities #3969

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions migrations/versions/a56fa0739cd7_add_my_cool_table.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
"""Add my cool table

Revision ID: a56fa0739cd7
Revises: 450d100cd30b
Create Date: 2024-11-16 20:46:01.862062

"""
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = 'a56fa0739cd7'
down_revision = '450d100cd30b'
branch_labels = None
depends_on = None


def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
pass
# ### end Alembic commands ###


def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
pass
# ### end Alembic commands ###
10 changes: 9 additions & 1 deletion src/pudl/metadata/classes.py
Original file line number Diff line number Diff line change
Expand Up @@ -897,11 +897,19 @@ class Contributor(PudlMeta):
"work package leader",
] = "project member"
organization: String | None = None
orcid: String | None = None
orcid: String | None | None = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think maybe there's an extra | None in this attribute definition.


def __repr__(self):
"""Representation excluding None values."""
attrs = {k: v for k, v in self.__dict__.items() if v is not None}
attr_str = ", ".join(f"{k}={repr(v)}" for k, v in attrs.items())
return f"{self.__class__.__name__}({attr_str})"
Comment on lines +902 to +906
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the consequences of having omitting the attribute in all cases of None values will be. Under the datapackage spec it's allowed, but I could also see the existence of those fields being expected on the other end where this is being used in the pudl-archiver repo -- potentially with the None value being used to check for completeness of the fields we want to have.

I'm also not sure if __repr__() is the method being used to output the attributes of the class. This is an immutable Pydandic BaseModel, so getting the JSON representation is probably being done via Contributor.model_dump().

If you instantiate a Contributor and use model_dump() does it behave as expected?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this and found that redefining __repr__ doesn't affect the output of model_dump() and it seems like the right place for this to happen is probably over in the pudl-archiver repository since the serialization behavior we're trying to fix is specific to when we're creating a new datapackage.json for use on Zenodo.

I went ahead and tried to make a PR that does this


@staticmethod
def dict_from_id(x: str) -> dict:
"""Construct dictionary from PUDL identifier."""
# data = copy.deepcopy(CONTRIBUTORS[x])

return copy.deepcopy(CONTRIBUTORS[x])

@classmethod
Expand Down
4 changes: 2 additions & 2 deletions src/pudl/transform/params/ferc1.py
Original file line number Diff line number Diff line change
Expand Up @@ -4649,8 +4649,8 @@
"row_seq": "row_seq",
"row_prvlg": "row_prvlg",
"report_prd": "report_prd",
"beg_yr_bal": "ending_balance",
"end_yr_bal": "starting_balance",
"beg_yr_bal": "starting_balance",
"end_yr_bal": "ending_balance",
Comment on lines -4652 to +4653
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omg we literally just switched them 🤦🏼 Thank you for fixing!

"dr_acct_num": "account_detail",
"dr_amount": "decrease_in_other_regulatory_liabilities",
"credits": "increase_in_other_regulatory_liabilities",
Expand Down