[META] Fix flaky tests

[reta]

What is the bug?
We have a number of flaky tests in the repo, this is a meta issue to address it

How can one reproduce the bug?
See please individual sub-issues below

What is the expected behavior?
No flaky tests

What is your host/environment?

• OS: any
• Version: any
• Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

[krisfreedain]

Catch All Triage - 1 2

@sam-herman @reta maintainers - can you take a look? thanks - you may want to add some additional labels to this repo as well