This repository was archived by the owner on Jul 24, 2024. It is now read-only.

Decouple hook executor from the operator, add Cloud Build execution and adjust testing - #48

Open

dinigo wants to merge 79 commits into

gocardless:masterfrom

Commits on Oct 18, 2021

Decouple hook executor from the operator, add Cloud Build execution and adjust testing
dinigo
committed
0fb5be6
Add hook to run in google cloud build
dinigo
committed
e53b744
Fix base operator
dinigo
committed
b922f21
Test GCP Cloud Build hook
dinigo
committed
f2fb018
Fix cloudbuild logic
dinigo
committed
ad837ee
Clean unused imports and fix path to upload dbt sources in GCS
dinigo
committed
543d744
Fix the upload sources test
dinigo
committed
5ac7f2a
Add docstrings to all DbtBaseOperator subclases
dinigo
committed
53243c0
Add `env` param to DbtBaseOperator to pass env variables to runtime
dinigo
committed
644f594

Commits on Oct 22, 2021

Export the DbtCloudBuildHook from the hooks package
dinigo
committed
5cc650d
Check that google provider API version is the right for the implementation (5)
dinigo
committed
db5e121
Add project_dir flag to the command line
dinigo
committed
f55624a
Remove cloud build timeout and wait. Thats google provider api v6, not 5
dinigo
committed
ef26f88
If dbt_bin is provided None or empty just don't add it to the command
dinigo
committed
129aa93
Add `project_dir` option flag
dinigo
committed
289d563
Add the `use_color` flag (#43). If set adds the dbt flag
dinigo
committed
081222d
Template all the fields from the DbtBaseOperator (#46 and #30)
dinigo
committed
bff2d5c

Commits on Oct 25, 2021

Refactor each hook into it's own file
dinigo
committed
6a459e8
Remove upload functionality, provided by `LocalFilesystemToGCSOperator`
dinigo
committed
dcaa3bb
Adjust imports after refactor
dinigo
committed
f234900
Test Cloud Build DBT hook and remove test cloudbuild upload
dinigo
committed
c582899
Fix test params for generate_dbt_cli_command
dinigo
committed
efdb64d
Export DbtBaseOperator and DbtCloudBuildOperator from the .operators pkg
dinigo
committed
678989c
Add missing dbt flags, add optional config and Config type to enforce it
dinigo
committed
3ff8c4f

Commits on Oct 26, 2021

Remove dbt_bin from constructors and defer hook instantiation
dinigo
committed
457f916
Fix test for TestDbtCliHook checking return code different from 0
dinigo
committed
5c3159c
Bring back "dir" to fix back the API, add warnings
dinigo
committed
bec7fe5
DbtCleanOperator was missing from operator exports (__init__)
dinigo
committed
0c7587a
Fix typo (thanks @fenimore) and reorganize imports
dinigo
committed
ce732b2
Comply with linter (flake8)
dinigo
committed
8ea9958

Commits on Oct 27, 2021

Refactor, move google operator out of the dbt_operator
dinigo
committed
f1c84fe
Display Cloud Build logs in the Airflow logs
dinigo
committed
a3e1213
Get rid of __init__ files exporting classes from different places
dinigo
committed
7776c45
Fix TestDbtCloudBuildHook with the new config sent to Cloud Build
dinigo
committed
2f8b9e7
Remove unused DbtCloudBuildHook param `dbt_bin`
dinigo
committed
3c90bc5

Commits on Nov 2, 2021

Commits on Dec 7, 2021

Make abstract method implement a single LOC `pass`. thanks @fenimore
dinigo
committed
82ec4f4
Explain conditional import of TypedDict as suggested by @andrewrjones
dinigo
committed
b505d59
Reuse type in different conditionals, suggested by @johanna-ojeling-datatonic
dinigo
committed
239e05f
Make project_id optional in the DbtCloudBuildHook

dinigo
and
Johanna Öjeling
authored
0cc0205
Fix typo in pydoc

dinigo
and
Johanna Öjeling
authored
a1dbbc7
Make `env` dict for environment variables for google dbt hook optional

dinigo
and
Johanna Öjeling
authored
cb0566d
Make `service_account ` for google dbt hook optional

dinigo
and
Johanna Öjeling
authored
d4e5184
Shorten up conditional assignment of project_id

dinigo
and
Johanna Öjeling
authored
98c90f0
Make `env` dict for environment variables for base dbt hook optional

dinigo
and
Johanna Öjeling
authored
33af351
Shorten up conditional assignment of env dict

dinigo
and
Johanna Öjeling
authored
139bfe3
Make `env ` for base dbt hook optional

dinigo
and
Johanna Öjeling
authored
c6b3699
Add descriptions to each parametrized dbt test
dinigo
committed
face550
Add README.md code examples
dinigo
committed
6f7a507
Split check values in two for readability
dinigo
committed
1c41b36
Extract provider version check to a function and test it, also typos
dinigo
committed
52884ae
Fix conditionals comparing with None since None is `falsy` in python
dinigo
committed
df62625

Commits on Jan 13, 2022

Fix python files not being detected in setup by `find_packages`:
dinigo
committed
642f6dd
Bump version
dinigo
committed
614309b
Make operator font color white, easier to read against a mid-dark bg
dinigo
committed
23e16a6

Commits on Jan 14, 2022

Allow false flags and reorder dbt render
dinigo
committed
ce219a0
Share DbtBaseOperator props between subclasses
dinigo
committed
22ddab1
Ignore deprecated @apply_defaults, It's needed for backwards compat
dinigo
committed
de1fec4
Fix dbt config, some params for dbt and some for the specific command
dinigo
committed
0f6e313
Allow a custom image other than the official from fishtown
dinigo
committed
f2075f5
Add DbtCloudBuildOperator templated fields to the DbtBaseOperator ones
dinigo
committed
b279112
Bump dbt docker image for cloud build
dinigo
committed
957f76b
Cloud build hook can raise silently. Surround with a try/catch
dinigo
committed
a2a3346
Add default conn_id, dbt_version and dbt_image and simpler cloudbuild hook
dinigo
committed
4515199

Commits on Feb 1, 2022

Fix compoundd commands like 'dbt docs generate' not running in GCP
dinigo
committed
f42cdfe
Export artifacts (`target` folder) when running in cloud build:
dinigo
committed
0c1bcf4
Enhance type safety
dinigo
committed
697a2ba

Commits on Mar 8, 2022

Commits on Oct 24, 2022