Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
8a64be5
update stac, weaver, magpie, twitcher, cowbird - security fixes for E…
fmigneault Dec 19, 2025
db7a954
Merge remote-tracking branch 'origin/master' into security-updates
fmigneault Jan 6, 2026
99b35ae
Merge branch 'master' into security-updates
fmigneault Jan 8, 2026
17b9e17
update weaver post-docker-compose-up to handle multiple magpie auth c…
fmigneault Jan 9, 2026
323f7bc
update components with extra urllib3 pin for security fix
fmigneault Jan 9, 2026
280dd71
Merge remote-tracking branch 'origin/master' into security-updates
fmigneault Jan 9, 2026
0d24071
Merge remote-tracking branch 'origin/security-updates' into security-…
fmigneault Jan 9, 2026
d1556f8
update weaver post-docker-compose-up with birdhouse log utility
fmigneault Jan 10, 2026
b5d102a
add '-n' option details to log functions
fmigneault Jan 13, 2026
1b0ef22
adjust newlines/prefix with log level destinations
fmigneault Jan 13, 2026
9542e71
fix issues with weaver logging
mishaschwartz Jan 14, 2026
215f145
revert unnecessary change
mishaschwartz Jan 14, 2026
0c20ed7
error message shouldn't be sent without a newline
mishaschwartz Jan 14, 2026
7122be7
review updates
mishaschwartz Jan 15, 2026
82e5bb8
more tests
mishaschwartz Jan 16, 2026
d8b4418
fix issues with weaver logging (#634)
fmigneault Jan 16, 2026
8aa18b1
Merge branch 'master' into security-updates
fmigneault Jan 16, 2026
b512a18
Merge remote-tracking branch 'origin/master' into security-updates
fmigneault Feb 12, 2026
54008c5
update stac 2.3.0
fmigneault Feb 12, 2026
49251ab
update weaver 6.8.2
fmigneault Feb 12, 2026
93efc73
fix reported 'stac-fastapi==6.2.1' within stac-app docker + add vars …
fmigneault Feb 12, 2026
819a5ad
update weaver 6.8.3 with more security/headers bugfix + add proxy buf…
fmigneault Feb 12, 2026
2deeaec
Merge remote-tracking branch 'origin/master' into security-updates
fmigneault Feb 17, 2026
9256d82
Bump version: 2.23.0 → 2.23.1 [skip ci]
fmigneault Feb 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tool.bumpversion]
current_version = "2.23.0"
current_version = "2.23.1"
commit = true
message = "Bump version: {current_version} → {new_version} [skip ci]"
tag = false
Expand Down
52 changes: 52 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,58 @@

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)

[2.23.1](https://github.com/bird-house/birdhouse-deploy/tree/2.23.1) (2026-02-17)
------------------------------------------------------------------------------------------------------------------

## Changes

- STAC API: Security update, minor OpenAPI version reporting fixes, and `stac-fastapi`/`starlette` compatibility fix
using version [2.3.0](https://github.com/crim-ca/stac-app/releases/tag/2.3.0)
(relates to [crim-ca/stac-app#65](https://github.com/crim-ca/stac-app/pull/65),
[crim-ca/stac-app#69](https://github.com/crim-ca/stac-app/pull/69) and
[crim-ca/stac-app#74](https://github.com/crim-ca/stac-app/pull/74)).

- Cowbird: Security update using version [2.6.0](https://github.com/Ouranosinc/cowbird/releases/tag/2.6.0)
(relates to [Ouranosinc/cowbird#98](https://github.com/Ouranosinc/cowbird/pull/98)).

- Magpie: Security update using version [4.3.1](https://github.com/Ouranosinc/Magpie/releases/tag/4.3.1)
(relates to [Ouranosinc/Magpie#640](https://github.com/Ouranosinc/Magpie/pull/640)
and [Ouranosinc/Magpie#642](https://github.com/Ouranosinc/Magpie/pull/642)).

- Twitcher: Security update using version [0.11.0](https://github.com/bird-house/twitcher/releases/tag/v0.11.0)
(relates to [bird-house/twitcher#143](https://github.com/bird-house/twitcher/pull/143),
[bird-house/twitcher#145](https://github.com/bird-house/twitcher/pull/145),
[bird-house/twitcher#146](https://github.com/bird-house/twitcher/pull/146) and
[bird-house/twitcher#148](https://github.com/bird-house/twitcher/pull/148)).

- Weaver: Security and dependency fix update using version [6.8.3](https://github.com/crim-ca/weaver/releases/tag/6.8.3)
(relates to [crim-ca/weaver#868](https://github.com/crim-ca/weaver/pull/868),
[crim-ca/weaver#869](https://github.com/crim-ca/weaver/pull/869),
[crim-ca/weaver#877](https://github.com/crim-ca/weaver/pull/877) and
[crim-ca/weaver#881](https://github.com/crim-ca/weaver/pull/881)).

- Weaver: Update `post-docker-compose-up` script.
- Handle multiple Magpie cookies in response.
This can happen depending on specific internal HTTP libraries versions of the services.
To retain backward/forward compatibility, all cookies returned from Magpie are chained in following `curl` commands.
- Use birdhouse `log` utility to report operations produced by the script rather than custom "echo level".

- Weaver: Job Result Proxy Buffers
- The *Job Results* responses of `weaver` can return a lot of `Link` headers. This is done to provide job metadata
references and provenance traceability details, but also for actual results locations that can vary in quantity
depending on the actual process execution.
Therefore, the Ngnix `proxy_buffer_size` and `proxy_buffers` directives of the `proxy` service must be added with
sufficiently large values to avoid HTTP 502 errors when the response headers exceed the default buffer sizes.
The `WEAVER_PROXY_RESPONSE_BUFFER_SIZE` and `WEAVER_PROXY_RESPONSE_BUFFER_COUNT` variables are added to allow
further customization as needed by the server. Their defaults are reasonable values to meet minimal requirements
by `weaver`'s metadata `Link` and a few result outputs.

- Birdhouse: Allow `log <LEVEL> -n ...` and `log <LEVEL> -p ...` to generate log outputs without newline/prefixes.

These options allow writing multiple log entries onto the same line for correct visual rendering of distinct `log`
calls separated to allow some intermediate logic. The `log` function invocations with these options respect the
log levels in order to make the messages consistent with enabled redirections and verbosity.

[2.23.0](https://github.com/bird-house/birdhouse-deploy/tree/2.23.0) (2026-02-13)
------------------------------------------------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ override BIRDHOUSE_MAKE_DIR := $(shell realpath -P $$(dirname $(BIRDHOUSE_MAKE_C
# Generic variables
override SHELL := bash
override APP_NAME := birdhouse-deploy
override APP_VERSION := 2.23.0
override APP_VERSION := 2.23.1

# utility to remove comments after value of an option variable
override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")
Expand Down
8 changes: 4 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ for a full-fledged production platform.
* - citation
- | |citation|

.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/2.23.0.svg
.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/2.23.1.svg
:alt: Commits since latest release
:target: https://github.com/bird-house/birdhouse-deploy/compare/2.23.0...master
:target: https://github.com/bird-house/birdhouse-deploy/compare/2.23.1...master

.. |latest-version| image:: https://img.shields.io/badge/tag-2.23.0-blue.svg?style=flat
.. |latest-version| image:: https://img.shields.io/badge/tag-2.23.1-blue.svg?style=flat
:alt: Latest Tag
:target: https://github.com/bird-house/birdhouse-deploy/tree/2.23.0
:target: https://github.com/bird-house/birdhouse-deploy/tree/2.23.1

.. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
:alt: ReadTheDocs Build Status (latest version)
Expand Down
2 changes: 1 addition & 1 deletion RELEASE.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.23.0 2026-02-13T00:28:48Z
2.23.1 2026-02-17T22:14:37Z
2 changes: 1 addition & 1 deletion bin/birdhouse
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ print_config_command() {
}

print_log_command() {
echo ". ${COMPOSE_DIR}/scripts/logging.include.sh"
echo "export __BIRDHOUSE_SUPPORTED_INTERFACE=True ; . ${COMPOSE_DIR}/scripts/logging.include.sh"
}

# Support multiple short flags together (ex: -abc instead of -a -b -c)
Expand Down
25 changes: 24 additions & 1 deletion birdhouse/components/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ service but this runs in docker containers and is specifically designed to inter
Birdhouse stack.

Available jobs
-------------
--------------

Scheduler jobs can be enabled by enabling optional components. Birdhouse comes with a variety of
these jobs in the ``optional-components`` directory. To enable any of these jobs, add the relevant
Expand Down Expand Up @@ -623,6 +623,29 @@ Customizing the Component
entirely, the ``WEAVER_ALT_PREFIX`` variable should be explicitly set to an empty value.


Managing Large Job Results
--------------------------

The `Job Results <https://pavics-weaver.readthedocs.io/en/latest/processes.html#job-results>`_ responses from `Weaver`
can return a lot of ``Link`` headers. This is done to provide job metadata references and provenance traceability
details, but also for actual results locations that can vary in quantity depending on the actual process execution.

By default, the Ngnix `proxy_buffer_size <https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size>`_
and `proxy_buffers <https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers>`_ directives of
the ``proxy`` service are added to the `Weaver` API endpoints with sufficiently large values to avoid HTTP 502 errors
when the response headers exceed the default buffer sizes.

If your processes happen to generate even larger results (e.g.: they return many NetCDF files from batch processing),
you may need to further increase these buffer sizes using
the ``WEAVER_PROXY_RESPONSE_BUFFER_SIZE`` and ``WEAVER_PROXY_RESPONSE_BUFFER_COUNT`` variables.

If your processes generate a *very large* number of results, you may also want to consider
alternate *content negotiation strategies* as described in
the `Job Results <https://pavics-weaver.readthedocs.io/en/latest/processes.html#job-results>`_
and `Process Execution Results <https://pavics-weaver.readthedocs.io/en/latest/processes.html#proc-exec-results>`_
sections of the `Weaver` documentation. Certain execution request parameters can be explicitly provided to limit
the number of returned headers and their representation in the responses.

.. _finch: https://github.com/bird-house/finch
.. _flyingpigeon: https://github.com/bird-house/flyingpigeon
.. _Weaver: https://github.com/crim-ca/weaver
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,8 @@ SERVICES = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '2.23.0',
'releaseTime': '2026-02-13T00:28:48Z',
'version': '2.23.1',
'releaseTime': '2026-02-17T22:14:37Z',
'institution': '${BIRDHOUSE_INSTITUTION}',
'researchSubject': '${BIRDHOUSE_SUBJECT}',
'supportEmail': '${BIRDHOUSE_SUPPORT_EMAIL}',
Expand Down Expand Up @@ -141,8 +141,8 @@ PLATFORMS = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '2.23.0',
'releaseTime': '2026-02-13T00:28:48Z',
'version': '2.23.1',
'releaseTime': '2026-02-17T22:14:37Z',
'institution': '${BIRDHOUSE_INSTITUTION}',
'researchSubject': '${BIRDHOUSE_SUBJECT}',
'supportEmail': '${BIRDHOUSE_SUPPORT_EMAIL}',
Expand Down
2 changes: 1 addition & 1 deletion birdhouse/components/cowbird/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ VARS="$VARS $EXTRA_VARS"
# Cowbird Configuration
# =====================

export COWBIRD_VERSION="2.5.2"
export COWBIRD_VERSION="2.6.0"
export COWBIRD_DOCKER=pavics/cowbird
export COWBIRD_IMAGE='${COWBIRD_DOCKER}:${COWBIRD_VERSION}'
export COWBIRD_IMAGE_API='${COWBIRD_IMAGE}-webservice'
Expand Down
2 changes: 1 addition & 1 deletion birdhouse/components/magpie/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# are applied and must be added to the list of DELAYED_EVAL.

# Tag version that will be used to update Magpie API, Magpie CLI, and matching Twitcher with Magpie Adapter
export MAGPIE_VERSION=4.2.0
export MAGPIE_VERSION=4.3.1
Comment thread
fmigneault marked this conversation as resolved.
export MAGPIE_IMAGE='pavics/magpie:${MAGPIE_VERSION}'
export MAGPIE_IMAGE_URI='registry.hub.docker.com/${MAGPIE_IMAGE}'

Expand Down
7 changes: 5 additions & 2 deletions birdhouse/components/stac/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ export STAC_PGPASSWORD='${BIRDHOUSE_POSTGRES_PASSWORD}'
# crim-ca/stac-app:1.0.0 uses STAC-fastapi version 3.0.3 with pgstac 0.6.10 (techically >=0.7,<0.8, but 0.6 works)
# crim-ca/stac-app:1.1.0 uses STAC-fastapi version 5.2.0 with pgstac 0.9.6 (techically >=0.8,<0.10)
# crim-ca/stac-app:2.0.1 uses STAC-fastapi version 6.0.0 with pgstac 0.9.6+ (techically >=0.8,<0.10)
export STAC_VERSION=6.0.0-crim-2.1.0
export STAC_IMAGE='ghcr.io/crim-ca/stac-app:2.1.0'
export STAC_FASTAPI_PACKAGE_VERSION="6.2.1" # informative only
export STAC_APP_DOCKER_VERSION="2.3.0" # actual image tag
export STAC_VERSION='${STAC_FASTAPI_PACKAGE_VERSION}-crim-${STAC_APP_DOCKER_VERSION}'
export STAC_IMAGE='ghcr.io/crim-ca/stac-app:${STAC_APP_DOCKER_VERSION}'
export STAC_IMAGE_URI='${STAC_IMAGE}'
export STAC_LICENSE_URL='https://raw.githubusercontent.com/crim-ca/stac-app/refs/heads/main/LICENSE'
export STAC_OPENAPI_SPEC_PATH='/api'
Expand Down Expand Up @@ -95,6 +97,7 @@ export DELAYED_EVAL="
STAC_POSTGRES_PASSWORD
STAC_PGUSER
STAC_PGPASSWORD
STAC_VERSION
STAC_IMAGE
STAC_IMAGE_URI
STAC_DB_TAGGED
Expand Down
2 changes: 1 addition & 1 deletion birdhouse/components/twitcher/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# This is because Twitcher must be built with the MagpieAdapter component.
# (https://github.com/Ouranosinc/Magpie/blob/master/Dockerfile.adapter)
# The following reference is only indicative for the 'service-config.json'.
export TWITCHER_VERSION=0.10.0
export TWITCHER_VERSION=0.11.1
export TWITCHER_RELEASE='${TWITCHER_VERSION}-magpie-${MAGPIE_VERSION}'
export TWITCHER_DOCKER=pavics/twitcher
export TWITCHER_IMAGE='${TWITCHER_DOCKER}:magpie-${MAGPIE_VERSION}'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@
proxy_set_header X-Forwarded-Proto $real_scheme;
proxy_set_header X-Forwarded-Host $http_host:$server_port;
proxy_buffering off;
# allow larger buffer size for endpoints that return more metadata in headers
# even if (body) buffering is disabled above, headers are still buffered by nginx since it must process them
# notably, job results can return varying amount of metadata and per-output link headers if requested this way
proxy_buffer_size ${WEAVER_PROXY_RESPONSE_BUFFER_SIZE};
proxy_buffers ${WEAVER_PROXY_RESPONSE_BUFFER_COUNT} ${WEAVER_PROXY_RESPONSE_BUFFER_SIZE};
}

# NOTE:
Expand Down
24 changes: 16 additions & 8 deletions birdhouse/components/weaver/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,19 @@ EXTRA_VARS='
$WEAVER_MONGODB_URL
$WEAVER_MANAGER_NAME
$WEAVER_WORKER_NAME
$WEAVER_MANAGER_LOG_LEVEL
$WEAVER_WORKER_LOG_LEVEL
$WEAVER_WPS_NAME
$WEAVER_WPS_OUTPUTS_DIR
$WEAVER_WPS_OUTPUTS_PATH
$WEAVER_WPS_PATH
$WEAVER_WPS_WORKDIR
$WEAVER_MANAGER_LOG_LEVEL
$WEAVER_WORKER_LOG_LEVEL
$WEAVER_WPS_PROVIDERS_MAX_TIME
$WEAVER_WPS_PROVIDERS_RETRY_COUNT
$WEAVER_WPS_PROVIDERS_RETRY_AFTER
$WEAVER_ALT_PREFIX_PROXY_LOCATION
$WEAVER_PROXY_RESPONSE_BUFFER_SIZE
$WEAVER_PROXY_RESPONSE_BUFFER_COUNT
'
# extend the original 'VARS' from 'birdhouse/birdhouse-compose.sh' to employ them for template substitution
# adding them to 'VARS', they will also be validated in case of override of 'default.env' using 'env.local'
Expand All @@ -55,7 +57,7 @@ OPTIONAL_VARS="
export WEAVER_CONFIG=HYBRID

# default release version that will be used to fetch docker images (API mananger & celery workers services)
export WEAVER_VERSION=6.6.2
export WEAVER_VERSION=6.8.3
export WEAVER_DOCKER=pavics/weaver
export WEAVER_IMAGE='${WEAVER_DOCKER}:${WEAVER_VERSION}'
export WEAVER_MANAGER_IMAGE='${WEAVER_IMAGE}-manager'
Expand All @@ -73,12 +75,19 @@ export WEAVER_MONGODB_VERSION=5.0
export WEAVER_MONGODB_HOST=weaver-mongodb
export WEAVER_MONGODB_PORT=27017
export WEAVER_MONGODB_URL='mongodb://${WEAVER_MONGODB_HOST}:${WEAVER_MONGODB_PORT}'
# Data persistence location on the host
export WEAVER_MONGODB_DATA_DIR='${BIRDHOUSE_DATA_PERSIST_ROOT}/mongodb_weaver_persist'

# real names of the weaver/worker services
# 'WEAVER_MANAGER_NAME' value will generate "<server-fqdn>/<name>" URI to access its API behind secured proxy
# 'WEAVER_MANAGER_NAME' and 'WEAVER_WORKER_NAME' are also employed to define name of the containers in docker-compose
export WEAVER_MANAGER_NAME=weaver
export WEAVER_WORKER_NAME=weaver-worker

# logging
export WEAVER_MANAGER_LOG_LEVEL=INFO
export WEAVER_WORKER_LOG_LEVEL=INFO

# below is used to define a separate Magpie service that can protect the WPS-1 endpoint of Weaver
# FIXME:
# remove when corresponding WPS-1/WPS-REST 'process' entries are managed under common service
Expand Down Expand Up @@ -116,9 +125,10 @@ export WEAVER_ALT_PREFIX_PROXY_LOCATION='
")
'

# logging
export WEAVER_MANAGER_LOG_LEVEL=INFO
export WEAVER_WORKER_LOG_LEVEL=INFO
# nginx proxy buffer settings for weaver API
# mainly to allow larger job results responses that can return multiple link headers to output files and metadata
export WEAVER_PROXY_RESPONSE_BUFFER_SIZE=32k
export WEAVER_PROXY_RESPONSE_BUFFER_COUNT=8

# control maximum timeout to abandon registration (duration in seconds, across whole procedure)
export WEAVER_WPS_PROVIDERS_MAX_TIME=120
Expand All @@ -127,8 +137,6 @@ export WEAVER_WPS_PROVIDERS_RETRY_COUNT=5
# control interval time between retries (duration in seconds, counts toward maximum timeout)
export WEAVER_WPS_PROVIDERS_RETRY_AFTER=5

export WEAVER_MONGODB_DATA_DIR='${BIRDHOUSE_DATA_PERSIST_ROOT}/mongodb_weaver_persist'

# If "True", Weaver providers that are no longer working (not responding when deployed) and are not named in
# WEAVER_WPS_PROVIDERS will be unregistered. This is useful when deploying Weaver with fewer providers than a previous
# deployment.
Expand Down
Loading