Skip to content

As a PDS operator, I want to upload DOI metadata for custom date ranges to SFTPΒ #505

Description

@jordanpadams

Checked for duplicates

Yes - I've already checked

πŸ§‘β€πŸ”¬ User Persona(s)

PDS Operator, System Administrator

πŸ’ͺ Motivation

...so that I can transfer backlogs of DOI metadata (e.g., the last 2 months) to the ADS SFTP server, instead of being limited to only the previous week's DOIs.

πŸ“– Additional Details

Currently, the DOI weekly roundup SFTP functionality is implemented as a standalone script (scripts/sftp_drop_weekly_roundup.py) that is hardcoded to upload DOI metadata for only the previous week (Monday-Sunday).

Current Limitations:

  • No CLI integration - must run script directly
  • Fixed to previous week only - cannot specify custom date ranges
  • Difficult to backfill historical data or handle missed uploads

Desired Enhancement:

  1. Integrate roundup functionality into the main pds-doi-cmd CLI as a new subcommand
  2. Add support for custom date ranges via --start-date and --end-date parameters
  3. Maintain backward compatibility - default behavior should still be "previous week" when no dates specified
  4. Deprecate the standalone script in favor of the new CLI command

Example Usage:

# Default behavior (previous week)
pds-doi-cmd roundup

# Custom date range (e.g., backfill last 2 months)
pds-doi-cmd roundup --start-date 2024-10-01 --end-date 2024-11-30

# Help
pds-doi-cmd roundup --help

This would provide a more flexible and maintainable solution for DOI metadata distribution to ADS.


For Internal Dev Team To Complete

Acceptance Criteria

Given a PDS operator needs to upload DOI metadata for a specific date range
When I perform pds-doi-cmd roundup --start-date YYYY-MM-DD --end-date YYYY-MM-DD
Then I expect the system to upload DOI metadata for that date range to the configured SFTP server

βš™οΈ Engineering Details

πŸŽ‰ I&T

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions