Skip to content

Data src s3 creds#2

Open
g-saransh wants to merge 3 commits into
mainfrom
data_src_s3_creds
Open

Data src s3 creds#2
g-saransh wants to merge 3 commits into
mainfrom
data_src_s3_creds

Conversation

@g-saransh
Copy link
Copy Markdown
Owner

@g-saransh g-saransh commented May 10, 2023

GitHub issue link:

Problem: The current commitTable mechanism assumes that the source data-files and catalog are both accessible via the same s3 credentials. That may not always be true.

Solution: Add a check for additional environment variables in the commitTable functions. If declared, the source s3 credentials are accordingly set. Currently this check if done only within commitTable action. However, we can also create a global function, if desired.

Testing:

  • Unit tests
  • Additional tests (add results below)

Documentation:

  • Documentation not needed
  • Updated README file
  • Documentation prepared (provide link below)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants