permissions not applying in 21.9.0

[Marty08]

I'm running command line authorize of a view in V21.9.0 and the process runs successfully with no errors.

All sources have a green tick except for the view I'm trying to authorize which still has an X

permissions to upstream datasets are not applied even though all upstream sources have a tick in the command line.

I've reverted back to version 20.4.1 and the issue is not present and all permissions granted.

Below is the output:

target-project:shared_view.sample_data
└── team-project:authorised_views.sample_data (✓)
└── team-project:calculations.sample (✓)
├── team-project:calculations.calculated_sample (✓)
│ ├── team-project:sales.header (✓)
│ │ └── source-project:sales.header (✓)
│ ├── team-project:sales.body (✓)
│ │ └── source-project:sales.body (✓)
│ ├── team-project:sales.dept (✓)
│ │ └── source-project:sales.dept (✓)
│ └── team-project:sales.sales (✓)
│ └── source-project:sales.sales (✓)
├── team-project:customer.customer_data(✓) **** permission denied here****
│ └── source-project:customer.customer_data (✓)
├── team-project:sales.store (✓)
│ └── source-project:sales.store (✓)

When trying to query the data in target-project:shared_view.sample_data, the permission denied at
team-project:customer.customer_data in version 21.9.0

Works perfectly with no issues in 20.4.1

[christippett]

Thanks for raising this @Marty08, I'll see about deploying a test environment I can use to run some integration tests. I've probably done something silly somewhere.

Glad at least the previous version is working for you.

[christippett]

@Marty08 I've started laying the groundwork for proper integration tests (https://github.com/servian/bigquery-view-analyzer/tree/feature/integration-tests). Any test cases you can contribute from your experience working with authorized views would be appreciated mate. Just a few bullet points would be ideal!

[Marty08]

@christippett

Happy to provide examples, I'll refer to the view being authorised as the target view and anything inside the view or needing authorisation as upstream:

• Upstream view(s) is in same project.dataset of target view
• Upstream has same dataset.view_name, different project name
• Upstream view may appear more than once for authorisation
• Target view in same dataset as upstream
• Target view in same project as upstream, same view name but different dataset name
• Upstream view contains UDF
• Upstream view contains date suffix table's e.g test_data_20210901 (test_data_*)

The last two are useful but possibly out of scope for testing

[Marty08]

@christippett , more of a question than a request.
Have you looked into authorising tables that have column level security applied via data catalogue?

e.g pii data and security groups:
https://cloud.google.com/bigquery/docs/column-level-security-intro

[christippett]

@Marty08 I haven't. Looks interesting though - I'm not working directly with BigQuery much these days so thanks for bringing this to my attention. I'll add to one of my things to look into.

[christippett]

@TWinsnes / @polleyg over to you to prioritise development effort on this.