Skip to content

Feature/data 2025 reverse etl operator#44

Merged
siklosid merged 14 commits intomasterfrom
feature/data-2025_reverse_etl_operator
Jan 8, 2025
Merged

Feature/data 2025 reverse etl operator#44
siklosid merged 14 commits intomasterfrom
feature/data-2025_reverse_etl_operator

Conversation

@siklosid
Copy link
Member

@siklosid siklosid commented Jan 3, 2025

DATA-2025

The new reverse etl operator is inherited from the batch operator with automatically filling out some of the batch parameters, like job name, input and output task name, target storage type, etc.

  • Creating new reverse etl oprator
  • Adding dynamo and sns io types
  • Fixing some smaller issues that were breaking the cli commands
  • Added the possiblity that an inherited operator can overwrite attributes of the base operator. E.g in our case from required to optional.

@siklosid siklosid requested a review from a team as a code owner January 3, 2025 11:32
Copy link

@claudiazi claudiazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! left few small comments

…ng some arguments default value handled by the job itslef; Making some arguments required instead hardwiring default value
@pull-request-size pull-request-size bot added size/XL and removed size/L labels Jan 7, 2025
Copy link

@claudiazi claudiazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! thaaanks!

@siklosid siklosid merged commit 76c774d into master Jan 8, 2025
1 check passed
@siklosid siklosid deleted the feature/data-2025_reverse_etl_operator branch January 8, 2025 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants