Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 60 additions & 3 deletions .github/workflows/functional-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,78 @@ on:
branches: [main, master]
pull_request:
branches: [main, master]
# EduIDE-deployment calls this after it has deployed an environment, so the
# suite runs against the rollout that just went green rather than against
# whatever the repository variable happens to point at.
workflow_call:
inputs:
environment:
type: string
required: true
description: >-
Landing page hostname to test, e.g.
e2e.eduide.student.k8s.aet.cit.tum.de. In EduIDE-deployment the
environment name and the landing page hostname are the same string,
so the caller passes its environment name straight through.
artemis_url:
type: string
required: false
default: ''
description: >-
Artemis instance to integrate against, as a full URL, for example
https://artemis.tum.de. Environments front different Artemis
instances, so it belongs to the caller. Empty falls back to the
ARTEMIS_URL repository variable.
secrets:
KEYCLOAK_USER:
required: true
KEYCLOAK_PWD:
required: true
# A called workflow can only read secrets it declares. The functional
# project never touches Artemis, so these stay optional and simply stay
# empty for a caller that does not have them.
ARTEMIS_USER:
required: false
ARTEMIS_PWD:
required: false
jobs:
test:
timeout-minutes: 60
runs-on: [self-hosted, e2e-test]
# Was `[self-hosted, e2e-test]`, which no registered runner answers to. The
# only self-hosted runners in this org are two ARC scale sets on the student
# cluster (`arc-buildkit-*-stud-amd64`), and an ARC scale-set runner is
# addressed by its scale set name, never by `self-hosted`. Every run since
# 2026-08-27 15:13 sat queued for that reason.
#
# The environments under test are reachable from the public internet - Let's
# Encrypt validates their certificates over HTTP-01 - so a hosted runner
# reaches them and needs no cluster access.
runs-on: ubuntu-latest
env:
KEYCLOAK_USER: ${{ secrets.KEYCLOAK_USER }}
KEYCLOAK_PWD: ${{ secrets.KEYCLOAK_PWD }}
LANDINGPAGE_URL: ${{ vars.LANDINGPAGE_URL }}
ARTEMIS_URL: ${{ vars.ARTEMIS_URL }}
# Called: test what the caller just deployed. Triggered directly: there is
# no input, so keep using the repository variable as before.
LANDINGPAGE_URL: ${{ inputs.environment && format('https://{0}', inputs.environment) || vars.LANDINGPAGE_URL }}
# Configurable, so a caller can point the suite at its own Artemis.
# Falls back to the repository variable when triggered directly.
ARTEMIS_URL: ${{ inputs.artemis_url || vars.ARTEMIS_URL }}
ARTEMIS_USER: ${{ secrets.ARTEMIS_USER }}
ARTEMIS_PWD: ${{ secrets.ARTEMIS_PWD }}
NUM_INSTANCES: 10

steps:
# In a called workflow the github context belongs to the CALLER, so a bare
# checkout would clone the calling repository and leave no test suite to
# run. job.workflow_repository / job.workflow_sha point at the repo and
# commit this workflow file itself came from, which is what we want there.
# inputs.environment is only set when we are called, so direct push and
# pull_request runs keep checking out github.repository at github.sha
# exactly as before.
- uses: actions/checkout@v4
with:
repository: ${{ inputs.environment && job.workflow_repository || github.repository }}
ref: ${{ inputs.environment && job.workflow_sha || github.sha }}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- uses: actions/setup-node@v4
with:
node-version: lts/*
Expand Down
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,39 @@ This repository provides E2E integration tests for the [Theia Cloud IDE](https:/
| artemis | Runs the **integration** test with Artemis, either local or deployed depending on the URLs set in the env file. | [![Playwright Tests](https://github.com/ls1intum/theia-scale-tests/actions/workflows/artemis-integration-tests.yml/badge.svg)](https://github.com/ls1intum/theia-scale-tests/actions/workflows/artemis-integration-tests.yml) |
| \*-setup | These are setup projects and not meant to be run on its own. Dependencies are already set. |

### Running the functional suite from another repository

`.github/workflows/functional-tests.yml` is also a reusable workflow, so a deployment pipeline can run the suite against the environment it has just rolled out. `EduIDE/EduIDE-deployment` uses it that way:

```yaml
e2e:
needs: deploy
uses: EduIDE/theia-scale-tests/.github/workflows/functional-tests.yml@main
with:
environment: e2e.eduide.student.k8s.aet.cit.tum.de
artemis_url: https://artemis.tum.de
secrets:
KEYCLOAK_USER: ${{ secrets.E2E_KEYCLOAK_USER }}
KEYCLOAK_PWD: ${{ secrets.E2E_KEYCLOAK_PWD }}
```

| Input | Required | |
|---|---|---|
| `environment` | yes | Landing page hostname. Becomes `LANDINGPAGE_URL` as `https://<environment>` |
| `artemis_url` | no | Artemis instance to integrate against, as a full URL. Environments front different Artemis instances, so it belongs to the caller. Empty falls back to the `ARTEMIS_URL` repository variable |

`KEYCLOAK_USER` and `KEYCLOAK_PWD` are required; `ARTEMIS_USER` and `ARTEMIS_PWD`
are optional and only needed by suites that reach Artemis.

The job runs on `ubuntu-latest`. It used to ask for `[self-hosted, e2e-test]`,
which no registered runner answers to, so every run queued indefinitely. The
environments under test are reachable from the public internet, so a hosted
runner is enough.

`environment` is the landing page hostname; the job turns it into `LANDINGPAGE_URL=https://<environment>`. In EduIDE-deployment the environment name under `environments/` and the landing page hostname are the same string, so the caller passes its environment name through unchanged. Pushes and pull requests on this repository send no input and keep reading `LANDINGPAGE_URL` from the repository variable.

`ARTEMIS_USER` and `ARTEMIS_PWD` are declared as optional secrets. The `functional` project never touches Artemis, so a caller that does not have them can leave them out.

## Development

- Single User Playwright Tests are run using a Test Account for Keycloak, to change the Test User, change the environment variables in GitHub Secrets \
Expand Down
Loading