Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map PHMSA Natural Gas Transmission Part M columns #3270

Merged
merged 8 commits into from
Jan 24, 2024
Merged

Map PHMSA Natural Gas Transmission Part M columns #3270

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c495624
phmsa part m 2010-present mapping
cmgosnell Jan 19, 2024
5316bac
add part m into the raw assets
cmgosnell Jan 19, 2024
183537b
Merge branch 'phmsa-transmission-k' into phmsa-transmission-m
cmgosnell Jan 19, 2024
b7f91bd
adding old years (oooops! fully missed those b4)
cmgosnell Jan 22, 2024
ab3459b
Merge branch 'main' into phmsa-transmission-m
cmgosnell Jan 23, 2024
d3a6f45
remove my shame.... i mean remove a new line in a csv i didn't mean t…
cmgosnell Jan 23, 2024
2f3f5be
Merge branch 'main' into phmsa-transmission-m
cmgosnell Jan 24, 2024
8cd1dab
fix column names!
cmgosnell Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/pudl/extract/phmsagas.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ def process_renamed(self, newdata: pd.DataFrame, page: str, **partition):
"raw_phmsagas__yearly_gas_transmission_miles_by_pressure_test_range_and_internal_inspection",
"raw_phmsagas__yearly_transmission_preparer_certification",
"raw_phmsagas__yearly_miles_of_transmission_pipe_by_smys",
"raw_phmsagas__yearly_transmission_failures_leaks_repairs",
"raw_phmsagas__yearly_miles_of_pipe_by_decade_installed",
)

Expand Down
221 changes: 221 additions & 0 deletions src/pudl/package_data/phmsagas/column_maps/yearly_transmission_failures_leaks_repairs.csv
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/file_map.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ yearly_transmission_hca_miles_by_determination_method_and_risk_model,-1,-1,-1,-1
yearly_gas_transmission_miles_by_pressure_test_range_and_internal_inspection,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_transmission_preparer_certification,annual_gas_transmission_gathering_1990.xlsx,annual_gas_transmission_gathering_1991.xlsx,annual_gas_transmission_gathering_1992.xlsx,annual_gas_transmission_gathering_1993.xlsx,annual_gas_transmission_gathering_1994.xlsx,annual_gas_transmission_gathering_1995.xlsx,annual_gas_transmission_gathering_1996.xlsx,annual_gas_transmission_gathering_1997.xlsx,annual_gas_transmission_gathering_1998.xlsx,annual_gas_transmission_gathering_1999.xlsx,annual_gas_transmission_gathering_2000.xlsx,annual_gas_transmission_gathering_2001.xlsx,annual_gas_transmission_gathering_2002.xlsx,annual_gas_transmission_gathering_2003.xlsx,annual_gas_transmission_gathering_2004.xlsx,annual_gas_transmission_gathering_2005.xlsx,annual_gas_transmission_gathering_2006.xlsx,annual_gas_transmission_gathering_2007.xlsx,annual_gas_transmission_gathering_2008.xlsx,annual_gas_transmission_gathering_2009.xlsx,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_transmission_failures_leaks_repairs,annual_gas_transmission_gathering_1990.xlsx,annual_gas_transmission_gathering_1991.xlsx,annual_gas_transmission_gathering_1992.xlsx,annual_gas_transmission_gathering_1993.xlsx,annual_gas_transmission_gathering_1994.xlsx,annual_gas_transmission_gathering_1995.xlsx,annual_gas_transmission_gathering_1996.xlsx,annual_gas_transmission_gathering_1997.xlsx,annual_gas_transmission_gathering_1998.xlsx,annual_gas_transmission_gathering_1999.xlsx,annual_gas_transmission_gathering_2000.xlsx,annual_gas_transmission_gathering_2001.xlsx,annual_gas_transmission_gathering_2002.xlsx,annual_gas_transmission_gathering_2003.xlsx,annual_gas_transmission_gathering_2004.xlsx,annual_gas_transmission_gathering_2005.xlsx,annual_gas_transmission_gathering_2006.xlsx,annual_gas_transmission_gathering_2007.xlsx,annual_gas_transmission_gathering_2008.xlsx,annual_gas_transmission_gathering_2009.xlsx,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
yearly_miles_of_pipe_by_decade_installed,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,annual_gas_transmission_gathering_2001.xlsx,annual_gas_transmission_gathering_2002.xlsx,annual_gas_transmission_gathering_2003.xlsx,annual_gas_transmission_gathering_2004.xlsx,annual_gas_transmission_gathering_2005.xlsx,annual_gas_transmission_gathering_2006.xlsx,annual_gas_transmission_gathering_2007.xlsx,annual_gas_transmission_gathering_2008.xlsx,annual_gas_transmission_gathering_2009.xlsx,annual_gas_transmission_gathering_2010.xlsx,annual_gas_transmission_gathering_2011.xlsx,annual_gas_transmission_gathering_2012.xlsx,annual_gas_transmission_gathering_2013.xlsx,annual_gas_transmission_gathering_2014.xlsx,annual_gas_transmission_gathering_2015.xlsx,annual_gas_transmission_gathering_2016.xlsx,annual_gas_transmission_gathering_2017.xlsx,annual_gas_transmission_gathering_2018.xlsx,annual_gas_transmission_gathering_2019.xlsx,annual_gas_transmission_gathering_2020.xlsx,annual_gas_transmission_gathering_2021.xlsx,annual_gas_transmission_gathering_2022.xlsx
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/page_map.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ yearly_transmission_hca_miles_by_determination_method_and_risk_model,-1,-1,-1,-1
yearly_gas_transmission_miles_by_pressure_test_range_and_internal_inspection,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,11,11,11,11,11,11,11,11,11,11,11
yearly_transmission_preparer_certification,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,8,8,8,8,8,8,8,8,8,8,8,8
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,5,5,5,5,5,5,5,5,5,5,5,5,5
yearly_transmission_failures_leaks_repairs,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,7,7,7,7,7,7,7,7,7,7,7,7,7
yearly_miles_of_pipe_by_decade_installed,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,4,4,4,4,4,4,4,4,4,4,4,4,4
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/page_part_map.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ yearly_transmission_hca_miles_by_determination_method_and_risk_model,gas_transmi
yearly_gas_transmission_miles_by_pressure_test_range_and_internal_inspection,gas_transmission_gathering
yearly_transmission_preparer_certification,gas_transmission_gathering
yearly_miles_of_transmission_pipe_by_smys,gas_transmission_gathering
yearly_transmission_failures_leaks_repairs,gas_transmission_gathering
yearly_miles_of_pipe_by_decade_installed,gas_transmission_gathering
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/skipfooter.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ yearly_transmission_hca_miles_by_determination_method_and_risk_model,-1,-1,-1,-1
yearly_gas_transmission_miles_by_pressure_test_range_and_internal_inspection,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,0,0
yearly_transmission_preparer_certification,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_transmission_failures_leaks_repairs,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
yearly_miles_of_pipe_by_decade_installed,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
1 change: 1 addition & 0 deletions src/pudl/package_data/phmsagas/skiprows.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ yearly_transmission_hca_miles_by_determination_method_and_risk_model,-1,-1,-1,-1
yearly_gas_transmission_miles_by_pressure_test_range_and_internal_inspection,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,2,2,2,2,2,2,2,2,2,2,2
yearly_transmission_preparer_certification,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,2,2,2,2,2,2,2,2,2,2,2
yearly_miles_of_transmission_pipe_by_smys,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,2,2,2,2,2,2,2,2,2,2,2,2,2
yearly_transmission_failures_leaks_repairs,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,2,2,2,2,2,2,2,2,2,2,2
yearly_miles_of_pipe_by_decade_installed,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,0,0,0,0,0,0,0,0,2,2,2,2,2,2,2,2,2,2,2,2,2