Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/codespell.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
afile
bu
te
tre
vas
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,8 @@ repos:
- id: ruff-format
files: birdhouse/components/jupyterhub/jupyterhub_custom/jupyterhub_custom/
args: [ --config=birdhouse/components/jupyterhub/jupyterhub_custom/ruff.toml ]
- repo: https://github.com/codespell-project/codespell
rev: v2.4.2
hooks:
- id: codespell
Comment thread
fmigneault marked this conversation as resolved.
args: [ '--ignore-words=.github/codespell.txt' ]
63 changes: 34 additions & 29 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,20 @@
[//]: # ( see: https://github.com/c4urself/bump2version/issues/99)
[//]: # ( - bump2version will not tag automatically, so it must be done manually after PR is merged and approved)
[//]: # ( This is to ensure that new tags are applied directly on merge-commit, an not a commit within the PR)
[//]: # ( see decission: https://github.com/bird-house/birdhouse-deploy/pull/161#discussion_r661746230)
[//]: # ( see decision: https://github.com/bird-house/birdhouse-deploy/pull/161#discussion_r661746230)

[//]: # (**DEFINE LATEST CHANGES UNDER BELOW 'Unreleased' SECTION - THEY WILL BE INTEGRATED IN NEXT RELEASE VERSION**)
[//]: # ( bump2version will take care to generate a new empty 'Unreleased' section after version bump)

[Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest)
------------------------------------------------------------------------------------------------------------------

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)
## Fixes

- Common spelling errors throughout the code base have been corrected via `codespell` pre-commit hook

The `codespell` hook scans for words that are suspected to be misspelled English words and suggests corrections
by way of pre-commit. A newline-separated list of words (`.github/codespell.txt`) is used to track ignored patterns.

[2.28.2](https://github.com/bird-house/birdhouse-deploy/tree/2.28.2) (2026-06-05)
------------------------------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -562,7 +567,7 @@
- The *Job Results* responses of `weaver` can return a lot of `Link` headers. This is done to provide job metadata
references and provenance traceability details, but also for actual results locations that can vary in quantity
depending on the actual process execution.
Therefore, the Ngnix `proxy_buffer_size` and `proxy_buffers` directives of the `proxy` service must be added with
Therefore, the Nginx `proxy_buffer_size` and `proxy_buffers` directives of the `proxy` service must be added with
sufficiently large values to avoid HTTP 502 errors when the response headers exceed the default buffer sizes.
The `WEAVER_PROXY_RESPONSE_BUFFER_SIZE` and `WEAVER_PROXY_RESPONSE_BUFFER_COUNT` variables are added to allow
further customization as needed by the server. Their defaults are reasonable values to meet minimal requirements
Expand Down Expand Up @@ -762,7 +767,7 @@
Note: this means that if no optional components require `optional-components/proxy-log-volume` as a dependency
then logs from the `proxy` container will only be written to stdout/stderr. This means that there is no need
for any additional custom log rotation handling since the logs are handled directly by docker. This means that
the `proxy` service itself no longer need to be dependant on the `scheduler` and `scheduler-job-logrotate-nginx`
the `proxy` service itself no longer need to be dependent on the `scheduler` and `scheduler-job-logrotate-nginx`
components.

Note: a previous discussion suggested that logs could be parsed directly from the stdout stream of the `proxy`
Expand Down Expand Up @@ -1300,7 +1305,7 @@

- STAC Browser: fix resolution against the pre-built prefix path to align with proxy configuration.
- Add `STAC_BROWSER_PATH_PREFIX` variable that makes the mapping more easily configurable.
However, changing its value requires building the corresponding docker wih the same prefix.
However, changing its value requires building the corresponding docker with the same prefix.
The change will not be effective "simply" by modifying the variable.
- Revert to `crim-ca/stac-browser:3.3.5` (rather than `crim-ca/stac-browser:3.4.0-dev`)
without need of the temporary patch (relates to https://github.com/radiantearth/stac-browser/pull/653).
Expand Down Expand Up @@ -1355,7 +1360,7 @@
This bug only affect external repos still using old variable names for
delayed eval.

- Lost new lines when new value is transfered to old value and vice-versa
- Lost new lines when new value is transferred to old value and vice-versa

Example: if `ENABLE_JUPYTERHUB_MULTI_NOTEBOOKS` (old var) is set in
`env.local`, the new matching var `JUPYTERHUB_ENABLE_MULTI_NOTEBOOKS` is
Expand All @@ -1367,7 +1372,7 @@
it will generate badly formatted code since the new lines are lost.

The reverse case: external repos still using old vars for template
expansion but in `env.local` the new var is used, the value transfered
expansion but in `env.local` the new var is used, the value transferred
from the new var to the old var is missing all the new lines and also
generate broken code.

Expand Down Expand Up @@ -1845,7 +1850,7 @@
- application data, user data, and log data for all components
- birdhouse logs
- docker container logs
- local environement file
- local environment file

Restoring data either involves restoring it to a named volume (determined by `BIRDHOUSE_BACKUP_VOLUME`) or in the case
of user data and application data, to overwrite the current data with the backup.
Expand Down Expand Up @@ -1993,7 +1998,7 @@
create additional jobs by adding them as custom components instead.

What about... ?
- just schedule these jobs for a non-existant day like February 31st?
- just schedule these jobs for a non-existent day like February 31st?
- Answer: This would technically work but is not obvious to the user. It is better to make this explicit.
- just set the schedule to the `'#'` string?
- Answer: This is a hack that would work based on the specific way that the docker-crontab image sets schedules.
Expand Down Expand Up @@ -2164,7 +2169,7 @@

- Fix bug where compose directory can't be found in `bin/birdhouse` script

The `COMPOSE_DIR` variable cannnot be discovered properly if:
The `COMPOSE_DIR` variable cannot be discovered properly if:

- the `bin/birdhouse` script is called with the `configs --print-config-command` options.
- the result of that call is `eval`ed in order to load the birdhouse configuration settings into
Expand Down Expand Up @@ -2402,7 +2407,7 @@
- new integration tests are written in the `tests/integration/` directory. More tests will be added in the
future!
- `conftest.py` scripts updated to bring the stack up/down in a consistent way for the integration tests.
- unit tests updated to accomodate new testing infrastructure as needed.
- unit tests updated to accommodate new testing infrastructure as needed.
- unit tests updated to test logging outputs better
- `birdhouse` interface script updated to support testing infrastructure (this should not change anything for
other end-users).
Expand Down Expand Up @@ -2436,7 +2441,7 @@

- Add resolver for http nginx configuration

Nginx requires a resolver to be explicity defined when using `proxy_pass` with a variable in the argument passed
Nginx requires a resolver to be explicitly defined when using `proxy_pass` with a variable in the argument passed
to `proxy_pass`. This resolver is defined explicitly for the https server block but not for the http server block.

This adds the explicit resolver for the http server block as well so that `proxy_pass` works when called using using
Expand Down Expand Up @@ -3130,7 +3135,7 @@

## Fixes

- Autodeploy broken due to instanciated left-over files in ./config/ dir
- Autodeploy broken due to instantiated left-over files in ./config/ dir

The `.gitignore` syntax was wrong. Regression from v2.0.0.

Expand Down Expand Up @@ -3660,7 +3665,7 @@ described for Weaver in
- Monitoring: allow access to magpie members of group `monitoring`

To allow accessing the various monitoring WebUI without having full blown
magpie admin priviledge to add and remove users.
magpie admin privileges to add and remove users.

Add existing users to this new `monitoring` group to allow them access to the
various monitoring WebUI. This way, we do not need to share the `admin` user
Expand Down Expand Up @@ -3799,7 +3804,7 @@ described for Weaver in

Before, test runs can only be started from inside the checkout, at some
"popular" locations inside the checkout. Now it can be started from
litterally anywhere.
literally anywhere.


[1.26.7](https://github.com/bird-house/birdhouse-deploy/tree/1.26.7) (2023-06-19)
Expand Down Expand Up @@ -3922,12 +3927,12 @@ described for Weaver in
- CanarieAPI: update to `0.7.1`.

- The server node now provides a generic ``server`` configuration for the current ``platform`` definition.
- Added multiple missing docuementation references for all the services included within `CanarieAPI` configurations.
- Added multiple missing documentation references for all the services included within `CanarieAPI` configurations.
- With new `CanarieAPI` version, a slightly improved UI with more service details are provided for the active server:

![image](https://user-images.githubusercontent.com/19194484/232822454-e39c0111-54dc-4f9b-adf6-5ea6e59d67e3.png)

- Add optional variables witht defaults to define reference Docker image version tags.
- Add optional variables with defaults to define reference Docker image version tags.

Following optional variables are defined by default. These are used as reference in the respective Docker compose
service definition of these components, as well as in their `CanarieAPI` configuration to retrieve the release time
Expand Down Expand Up @@ -4028,7 +4033,7 @@ described for Weaver in

## Fixes

- Fix disapearing Thredds docker image
- Fix disappearing Thredds docker image

The current docker image version for thredds (4.6.18) is no longer hosted in the
[Unidata docker repository](https://hub.docker.com/r/unidata/thredds-docker/tags).
Expand Down Expand Up @@ -4059,9 +4064,9 @@ described for Weaver in
## Fixes
- Enforce the load order of components defined in env.local

Extra components defined in the `EXTRA_CONF_DIRS` variables were being loaded before the dependant components
Extra components defined in the `EXTRA_CONF_DIRS` variables were being loaded before the dependent components
defined in the `COMPONENT_DEPENDENCIES` variables in each default.env file. This meant that if an extra component
was meant to override some setting defined in a dependant component, the setting would not be overridden by the
was meant to override some setting defined in a dependent component, the setting would not be overridden by the
extra component.

This change enforces the following load order rules:
Expand Down Expand Up @@ -4279,7 +4284,7 @@ described for Weaver in

- env.local.example: change default Geoserver admin user from 'admin' to 'admingeo'

This only impacts new deployment when `env.local.example` is instanciated
This only impacts new deployment when `env.local.example` is instantiated
to `env.local`.

This is to avoid confusion with the admin user of Magpie, which is also 'admin'.
Expand Down Expand Up @@ -4606,7 +4611,7 @@ described for Weaver in
`env.local` is the last file to be read. However those steps should not be
forgotten in any `default.env` for all components.

So the impact or burden is on the developpers to write their `default.env`
So the impact or burden is on the developers to write their `default.env`
file properly, not on the users that only modify the `env.local` file.

All `default.env` files header have been updated with notice about this new
Expand Down Expand Up @@ -5012,12 +5017,12 @@ described for Weaver in

All the major browsers and OS platform has previously added the new root
certificate "ISRG Root X1" ahead of time so the transition to the new
root certificate is seemless for all clients.
root certificate is seamless for all clients.

Python `requests` package bundle their own copy of known root
certificates and is late to add this new root cert "ISRG Root X1". Had
it automatically fallback to the OS copy of the root cert bundle, this
would have been seemless.
would have been seamless.

The fix is to force `requests` to use the OS copy of the root cert bundle.

Expand Down Expand Up @@ -5220,7 +5225,7 @@ described for Weaver in
for the moment so will have to be handle in another PR.

So for the moment we have not achieved full non-root user in cronjobs
launched by the `scheduler` compoment but the most important part, the part
launched by the `scheduler` component but the most important part, the part
that perform the actual job (rsync or execute custom command using an
external docker container) is running as non-root.

Expand Down Expand Up @@ -7806,13 +7811,13 @@ Deployed to https://medus.ouranos.ca/jupyter/ for acceptance testing.

[1.11.0](https://github.com/bird-house/birdhouse-deploy/tree/1.11.0) (2020-08-25)
------------------------------------------------------------------------------------------------------------------
- Improved plugable component architecture.
- Improved pluggable component architecture.

Before this PR, components needing default values, needing template variable substitution, needing to execute commands pre and post `docker-compose up` are hardcoding their needs directly to the "core" system, basically "leaking" their requirements out even when they are not activated (fixes https://github.com/bird-house/birdhouse-deploy/issues/62).

This PR provides true plugable architecture for the components so they can provide all their needs without having to modify the code of the "core" system.
This PR provides true pluggable architecture for the components so they can provide all their needs without having to modify the code of the "core" system.

All the components (monitoring, generic_bird, emu, testthredds) are modified to leverage the new plugable architecture, with additional customizations given it is cleaner/easier to have default configuration values.
All the components (monitoring, generic_bird, emu, testthredds) are modified to leverage the new pluggable architecture, with additional customizations given it is cleaner/easier to have default configuration values.

Given this PR both changes the architecture and modify many components at the same time, it is best to read each commit separately to easier understand which code change belongs to which "goal".

Expand Down Expand Up @@ -8481,7 +8486,7 @@ Deployed to https://medus.ouranos.ca/jupyter/ for acceptance testing.
ab56994 jupyter: limit memory of public user to 500 MB
90c1950 jupyter: prevent user from loading user-owned config at spawner server startup
e8f2fa3 jupyter: avoid terminating user running jobs on Hub update
3f97cc7 jupyter: get ready to prevent browser session re-use even if password changed
3f97cc7 jupyter: get ready to prevent browser session reuse even if password changed
e2ebcc3 jupyter: disable notebook terminal for security reasons

[1.8.0](https://github.com/bird-house/birdhouse-deploy/tree/1.8.0) (2020-02-03)
Expand Down
8 changes: 4 additions & 4 deletions birdhouse/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ Note about WPS request timeout

Default value ``PROXY_READ_TIMEOUT_VALUE`` in |birdhouse-default.env|_.

Overrideable in ``env.local`` file, as usual for all values in ``default.env`` file.
Overridable in ``env.local`` file, as usual for all values in ``default.env`` file.

.. |components-proxy-nginx.conf| replace:: ``components/proxy/nginx.conf``
.. _components-proxy-nginx.conf: ./components/proxy/nginx.conf
Expand Down Expand Up @@ -620,7 +620,7 @@ Release Procedure

* Review PR description if something needs to be added or updated after the PR
review process. The goal is for the PR description to capture all the
essential informations for someone else not participating in the PR review
essential information for someone else not participating in the PR review
process to understand it easily. This "someone else" might even be your
future self trying to understand what was going through your mind when you
opened this PR :)
Expand Down Expand Up @@ -852,7 +852,7 @@ The following section is meant for project maintainers which describes the requi
GitHub Workflow automations. Only administrators and users with repository secrets privilieges can use this information.

There are a handful of automations that require specialized secrets, configurations, and tokens.
Workflows thats specifically depend on the *Birdhouse-Helper-Bot* App require that the app have certain repositiory
Workflows that specifically depend on the *Birdhouse-Helper-Bot* App require that the app have certain repository
privileges, specifically:

* Read access to actions, issues, metadata, and organization secrets
Expand All @@ -871,7 +871,7 @@ Additionally, a set of secrets is also required in order to make use of the spec

* ``BIRDHOUSE_HELPER_BOT_ID`` : The ID number of the App. Available within the relevant entry under GitHub Apps.

* ``BIRDHOUSE_HELPER_BOT_KEY`` : The unqiue key of the App. If this needs to be changed, it must be regenerated.
* ``BIRDHOUSE_HELPER_BOT_KEY`` : The unique key of the App. If this needs to be changed, it must be regenerated.
This is only shown once in order to prevent misuse/distribution.

All the above prvilieges and keys must be visible/accessible to the repository that uses workflows that rely on
Expand Down
2 changes: 1 addition & 1 deletion birdhouse/components/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -640,7 +640,7 @@ The `Job Results <https://pavics-weaver.readthedocs.io/en/latest/processes.html#
can return a lot of ``Link`` headers. This is done to provide job metadata references and provenance traceability
details, but also for actual results locations that can vary in quantity depending on the actual process execution.

By default, the Ngnix `proxy_buffer_size <https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size>`_
By default, the Nginx `proxy_buffer_size <https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size>`_
and `proxy_buffers <https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers>`_ directives of
the |proxy|_ service are added to the |weaver|_ API endpoints with sufficiently large values to avoid HTTP 502 errors
when the response headers exceed the default buffer sizes.
Expand Down
2 changes: 1 addition & 1 deletion birdhouse/components/cowbird/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ export USER_WORKSPACES="user_workspaces"
# If this path is changed, users will have to restart their personal Jupyter
# server for the change to take effect.
# This file explains the workspace to the user when using Cowbird.
# Note that the variable can still be overriden by the `env.local` file.
# Note that the variable can still be overridden by the `env.local` file.
export JUPYTERHUB_COWBIRD_README="$COMPOSE_DIR/components/cowbird/user_workspace_readme.ipynb"
export JUPYTERHUB_README="${JUPYTERHUB_README:-${JUPYTERHUB_COWBIRD_README}}"

Expand Down
Loading