Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map PHMSA Natural Gas Transmission Part M columns #3270

Merged
merged 8 commits into from
Jan 24, 2024
Merged

Map PHMSA Natural Gas Transmission Part M columns #3270

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c495624
phmsa part m 2010-present mapping
cmgosnell Jan 19, 2024
5316bac
add part m into the raw assets
cmgosnell Jan 19, 2024
183537b
Merge branch 'phmsa-transmission-k' into phmsa-transmission-m
cmgosnell Jan 19, 2024
b7f91bd
adding old years (oooops! fully missed those b4)
cmgosnell Jan 22, 2024
ab3459b
Merge branch 'main' into phmsa-transmission-m
cmgosnell Jan 23, 2024
d3a6f45
remove my shame.... i mean remove a new line in a csv i didn't mean t…
cmgosnell Jan 23, 2024
2f3f5be
Merge branch 'main' into phmsa-transmission-m
cmgosnell Jan 24, 2024
8cd1dab
fix column names!
cmgosnell Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/pudl/extract/phmsagas.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ def process_renamed(self, newdata: pd.DataFrame, page: str, **partition):
"raw_phmsagas__yearly_miles_of_transmission_pipe_by_nps",
"raw_phmsagas__yearly_inspections_and_assessments",
"raw_phmsagas__yearly_miles_of_transmission_pipe_by_smys",
"raw_phmsagas__yearly_failures_leaks_repairs",
)

phmsagas_raw_dfs = excel.raw_df_factory(Extractor, name="phmsagas")
Expand Down
189 changes: 189 additions & 0 deletions src/pudl/package_data/phmsagas/column_maps/yearly_failures_leaks_repairs.csv
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/file_map.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ yearly_miles_of_transmission_pipe_by_nps,annual_gas_transmission_gathering_1990.
yearly_miles_of_gathering_pipe_by_nps,annual_gas_transmission_gathering_1990.xlsx,annual_gas_transmission_gathering_1991.xlsx,annual_gas_transmission_gathering_1992.xlsx,annual_gas_transmission_gathering_1993.xlsx,annual_gas_transmission_gathering_1994.xlsx,annual_gas_transmission_gathering_1995.xlsx,annual_gas_transmission_gathering_1996.xlsx,annual_gas_transmission_gathering_1997.xlsx,annual_gas_transmission_gathering_1998.xlsx,annual_gas_transmission_gathering_1999.xlsx,annual_gas_transmission_gathering_2000.xlsx,annual_gas_transmission_gathering_2001.xlsx,annual_gas_transmission_gathering_2002.xlsx,annual_gas_transmission_gathering_2003.xlsx,annual_gas_transmission_gathering_2004.xlsx,annual_gas_transmission_gathering_2005.xlsx,annual_gas_transmission_gathering_2006.xlsx,annual_gas_transmission_gathering_2007.xlsx,annual_gas_transmission_gathering_2008.xlsx,annual_gas_transmission_gathering_2009.xlsx,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_inspections_and_assessments,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_failures_leaks_repairs,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/page_map.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ yearly_miles_of_transmission_pipe_by_nps,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_miles_of_gathering_pipe_by_nps,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,3,3,3,3,3,3,3,3,3,3,3,3
yearly_inspections_and_assessments,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,1,1,1,1,1,1,1,1,1,1,1,1,1
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,5,5,5,5,5,5,5,5,5,5,5,5,5
yearly_failures_leaks_repairs,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,7,7,7,7,7,7,7,7,7,7,7,7,7
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/page_part_map.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ yearly_miles_of_transmission_pipe_by_nps,gas_transmission_gathering
yearly_miles_of_gathering_pipe_by_nps,gas_transmission_gathering
yearly_inspections_and_assessments,gas_transmission_gathering
yearly_miles_of_transmission_pipe_by_smys,gas_transmission_gathering
yearly_failures_leaks_repairs,gas_transmission_gathering
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/skipfooter.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ yearly_miles_of_transmission_pipe_by_nps,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_miles_of_gathering_pipe_by_nps,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_inspections_and_assessments,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_failures_leaks_repairs,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,0,0,0,0
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/skiprows.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ yearly_miles_of_transmission_pipe_by_nps,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_miles_of_gathering_pipe_by_nps,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,2,2,2,2,2,2,2,2,2,2,2
yearly_inspections_and_assessments,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,2,2,2,2,2,2,2,2,2,2,2,2,2
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,2,2,2,2,2,2,2,2,2,2,2,2,2
yearly_failures_leaks_repairs,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,2,2,2,2,2,2,2,2,2,2,2,2,2