Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create shelves files for pds4 #57

Draft
wants to merge 42 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2e94c7b
Add validation directory (moved from rms-webtools repo)
juzen2003 Dec 6, 2023
4be35d4
Merge remote-tracking branch 'origin/main'
juzen2003 Dec 16, 2023
5e5fe97
Merge remote-tracking branch 'origin/main'
juzen2003 Jan 3, 2024
fd70eb6
Merge remote-tracking branch 'origin/main'
juzen2003 Jan 25, 2024
9f9c733
Merge remote-tracking branch 'origin/main'
juzen2003 Feb 8, 2024
077c359
Merge remote-tracking branch 'origin/main'
juzen2003 Feb 12, 2024
291555e
Merge remote-tracking branch 'origin/main'
juzen2003 Apr 15, 2024
d195a79
Merge remote-tracking branch 'origin/main'
juzen2003 Apr 16, 2024
1f762cb
Merge remote-tracking branch 'origin/main'
juzen2003 Apr 23, 2024
92bd65c
Merge remote-tracking branch 'origin/main'
juzen2003 Apr 26, 2024
205b9df
Merge remote-tracking branch 'origin/main'
juzen2003 Apr 29, 2024
ab669f9
Add command line tool to show opus products output with the given
juzen2003 Aug 19, 2024
c7d84e5
Merge remote-tracking branch 'origin/main'
juzen2003 Sep 6, 2024
a075bbc
Merge remote-tracking branch 'origin/main'
juzen2003 Sep 9, 2024
428d727
Merge remote-tracking branch 'origin/main'
juzen2003 Sep 10, 2024
fd4c7b6
Merge remote-tracking branch 'origin/main' into create_shelves_files_…
juzen2003 Sep 11, 2024
756547c
Merge remote-tracking branch 'origin/main'
juzen2003 Sep 11, 2024
d2d3803
Merge remote-tracking branch 'origin/main'
juzen2003 Sep 12, 2024
a875899
Merge branch 'main' into create_shelves_files_for_pds4
juzen2003 Sep 12, 2024
4ffbc45
Create pds4 directory to store maintenance files for pds4.
juzen2003 Sep 18, 2024
4b9b154
Merge remote-tracking branch 'origin/main' into create_shelves_files_…
juzen2003 Sep 18, 2024
cd66002
Update pds4/pds4checksums.py so that checksum files can be created under
juzen2003 Sep 18, 2024
fe7c687
Merge remote-tracking branch 'origin/main' into create_shelves_files_…
juzen2003 Sep 19, 2024
ddbbb2a
Update pds4/pds4infoshelf.py to create info sheleves files under
juzen2003 Sep 19, 2024
69bfd7a
Removed the debug print statement in the pdsfile.py
juzen2003 Sep 19, 2024
cb08184
Merge remote-tracking branch 'origin/main' into create_shelves_files_…
juzen2003 Sep 21, 2024
bc74047
Create pds4/pds4indexshelf.py & pds4/pds4linkshelf.py
juzen2003 Sep 21, 2024
5509b5a
Update pds4linkshelf.py to make sure we can create files in
juzen2003 Sep 23, 2024
b0a7944
Remove the code that bypass _support & ring_modeles when creating
juzen2003 Sep 25, 2024
10b6743
Add an intelligence to link a file to its correspsonding label if the
juzen2003 Oct 10, 2024
c3f1572
Fixed the code indentation from line 112-428 in pds4linkshelf.py
juzen2003 Oct 10, 2024
0ef4578
Add the intelligence to ignore files when there is no label at the same
juzen2003 Oct 10, 2024
3f3d2da
Add the intelligence to identify files like errata.txt, or checksum
juzen2003 Oct 11, 2024
a7c725d
Separate BUNDLENAME* and BUNDLESET* regex, put them under PDS3 & PDS4
juzen2003 Oct 15, 2024
1890a8f
Merge remote-tracking branch 'origin/main' into create_shelves_files_…
juzen2003 Oct 15, 2024
e059710
Removed redundant BUNDLESET* and BUNDLENAME* regex for PDS3.
juzen2003 Oct 15, 2024
d83b146
Change is_volume_dir to is_bundle_dir in pds4checksums.py
juzen2003 Oct 17, 2024
1c06fe9
Update read_links function in pds4linkshelf.py to capture file names
juzen2003 Oct 18, 2024
0eea1fe
Remove the commented out code in the read_links function of
juzen2003 Oct 18, 2024
b2fca02
Update generate_links and improve its running time by:
juzen2003 Oct 21, 2024
8fa331d
Remove commented out logger.debug generate_links
juzen2003 Oct 21, 2024
39639c6
At line 156-158 of pds4linkshelf.py, skip the empty entry of a csv file
juzen2003 Oct 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions holdings_maintenance/pds3/pdschecksums.py
Original file line number Diff line number Diff line change
Expand Up @@ -614,7 +614,7 @@ def main():
'their MD5 checksums to the checksum file. ' +
'Checksums of pre-existing files are not checked.')

parser.add_argument('volume', nargs='+', type=str,
parser.add_argument('--volume', nargs='+', type=str,
help='The path to the root directory of a volume or ' +
'volume set. For a volume set, all the volume ' +
'directories inside it are handled in sequence. ' +
Expand Down Expand Up @@ -768,21 +768,21 @@ def main():

# Save logs in up to two places
if pdsf.volname:
logfiles = set([pdsf.log_path_for_volume('_md5',
logfiles = set([pdsf.log_path_for_bundle('_md5',
task=args.task,
dir='pdschecksums'),
pdsf.log_path_for_volume('_md5',
pdsf.log_path_for_bundle('_md5',
task=args.task,
dir='pdschecksums',
place='parallel')])
else:
logfiles = set([pdsf.log_path_for_volset('_md5',
task=args.task,
dir='pdschecksums'),
pdsf.log_path_for_volset('_md5',
task=args.task,
dir='pdschecksums',
place='parallel')])
logfiles = set([pdsf.log_path_for_bundleset('_md5',
task=args.task,
dir='pdschecksums'),
pdsf.log_path_for_bundleset('_md5',
task=args.task,
dir='pdschecksums',
place='parallel')])

# Create all the handlers for this level in the logger
local_handlers = []
Expand Down
2 changes: 1 addition & 1 deletion holdings_maintenance/pds3/pdsinfoshelf.py
Original file line number Diff line number Diff line change
Expand Up @@ -662,7 +662,7 @@ def main():
'than the shelf file, update the shelf file\'s ' +
'modification date.')

parser.add_argument('volume', nargs='+', type=str,
parser.add_argument('--volume', nargs='+', type=str,
help='The path to the root of the volume or volume ' +
'set. For a volume set, all the volume ' +
'directories inside it are handled in sequence.')
Expand Down
Loading
Loading