Skip to content

BohdanObruch/testomat_tests

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

75 Commits
.github
.github
 
 
src
src
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
pyproject.toml
pyproject.toml
 
 
uv.lock
uv.lock
 
 

Repository files navigation

Testomat E2E Tests

End-to-end testing framework for Testomat with three layers: Playwright UI, Selenium UI, and API tests.

Project Structure

testomat_tests/
|-- src/                          # Source code
|   |-- api/                      # API client + models
|   |   |-- client.py
|   |   |-- logger.py
|   |   |-- utils.py
|   |   |-- controllers/          # API endpoint controllers
|   |   |   |-- analytics.py
|   |   |   |-- attachment.py
|   |   |   |-- comment.py
|   |   |   |-- jira_issue.py
|   |   |   |-- permission.py
|   |   |   |-- plan.py
|   |   |   |-- project.py
|   |   |   |-- run.py
|   |   |   |-- rungroup.py
|   |   |   |-- settings.py
|   |   |   |-- step.py
|   |   |   |-- suite.py
|   |   |   |-- template.py
|   |   |   |-- test.py
|   |   |   |-- testrun.py
|   |   |   |-- timeline.py
|   |   |   |-- user.py
|   |   |   `-- user_project.py
|   |   `-- models/               # API response/request models
|   |       |-- jira_issue.py
|   |       |-- jira_issues.py
|   |       |-- plans.py
|   |       |-- project.py
|   |       |-- responses.py
|   |       |-- run.py
|   |       |-- run_group.py
|   |       |-- step.py
|   |       |-- suite.py
|   |       |-- template.py
|   |       |-- test.py
|   |       `-- test_run.py
|   `-- web/                      # Web UI automation
|       |-- application.py        # Application facade (entry point)
|       |-- helpers/              # Helpers/utilities
|       |   `-- cookie_helper.py
|       |-- pages/                # Page Object Models (Playwright)
|       |   |-- home_page.py
|       |   |-- login_page.py
|       |   |-- new_projects_page.py
|       |   |-- project_page.py
|       |   `-- projects_page.py
|       |-- components/           # Reusable UI components
|       |   |-- add_test_menu.py
|       |   |-- navigation_tabs.py
|       |   |-- new_suite.py
|       |   |-- project_card.py
|       |   |-- projects_header.py
|       |   |-- side_bar.py
|       |   |-- suite.py
|       |   `-- test_modal.py
|       `-- selenium/             # Selenium framework
|           |-- application.py
|           |-- core/
|           |   |-- base_page.py
|           |   `-- waits.py
|           |-- components/
|           |   |-- add_test_menu.py
|           |   |-- navigation_tabs.py
|           |   |-- new_suite.py
|           |   |-- project_card.py
|           |   |-- projects_header.py
|           |   |-- side_bar.py
|           |   |-- suite.py
|           |   `-- test_modal.py
|           `-- pages/
|               |-- login_page.py
|               |-- login_page_v2.py
|               |-- new_projects_page.py
|               |-- project_page.py
|               `-- projects_page.py
|
|-- tests/                        # Test suite
|   |-- conftest.py               # Pytest plugins
|   |-- fixtures/                 # Shared pytest fixtures
|   |   |-- api.py
|   |   |-- config.py
|   |   |-- hooks.py
|   |   |-- playwright.py
|   |   |-- selenium.py
|   |   `-- settings.py
|   |-- first_test.py
|   |-- api/                      # API tests
|   |   |-- projects_test.py
|   |   |-- suite_and_test_creation_test.py
|   |   |-- test_analytics.py
|   |   |-- test_misc.py
|   |   |-- test_plans.py
|   |   |-- test_runs.py
|   |   |-- test_suites.py
|   |   `-- test_templates.py
|   `-- web/                      # Web UI tests
|       |-- selenium/
|       |   |-- simple_selenium_test.py
|       |   `-- enterprise_plan/
|       |       |-- project_creation_test.py
|       |       |-- projects_page_test.py
|       |       |-- switch_company_test.py
|       |       |-- test_suite.py
|       |       `-- test_delete_project.py
|       |-- cookies_test.py
|       |-- login_page_test.py
|       |-- enterprise_plan/
|       |   |-- project_creation_test.py
|       |   |-- projects_page_test.py
|       |   |-- switch_company_test.py
|       |   `-- test_create_suite.py
|       `-- free_plan/
|           `-- free_projects_test.py
|
|-- test-result/                  # Test execution artifacts
|   |-- allure-results/           # Raw Allure results
|   |-- screenshots/              # Screenshots on failure
|   |-- traces/                   # Playwright traces
|   |-- videos/                   # Video recordings (local runs only)
|   `-- .auth/                    # Auth state storage
|
|-- .env                          # Environment configuration
|-- pyproject.toml                # Project configuration
`-- uv.lock                       # Dependency lock file

Requirements

  • Python >= 3.14
  • uv package manager (recommended)

Installation

Using uv (Recommended)

# Install uv if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create virtual environment and install dependencies
uv sync

# Install Playwright browsers
uv run playwright install

Using pip

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -e .
playwright install

Configuration

Create a .env file in the project root with the following variables:

BASE_URL=https://testomat.io
BASE_APP_URL=https://app.testomat.io
EMAIL=your_email@example.com
PASSWORD=your_password
TESTOMAT_TOKEN=your_api_token

Running Tests

# Run all tests
uv run pytest

# Run smoke tests
uv run pytest -m smoke

# Run regression tests
uv run pytest -m regression

# Run web UI tests
uv run pytest -m web

# Run API tests
uv run pytest -m api

# Run Selenium tests
uv run pytest -m selenium

# Run tests in headed mode (Playwright + Selenium)
uv run pytest --headed

# Run specific test file
uv run pytest tests/web/login_page_test.py

# Run specific Selenium test file
uv run pytest tests/web/selenium/enterprise_plan/test_suite.py

# Override Playwright artifacts (defaults are set in pyproject.toml)
uv run pytest --tracing=on --screenshot=on --video=on

CI (GitHub Actions)

The project uses reusable GitHub Actions workflows for API/UI test execution and a parent workflow that publishes a unified Allure report.

Workflows

  • .github/workflows/all_tests_report.yml:

    • Triggers: pull_request to main, schedule (0 8 * * 1-5), workflow_dispatch
    • Runs Ruff checks on PRs, then calls reusable Playwright/API/Selenium workflows
    • Publishes unified Allure 3 report to GitHub Pages

  • .github/workflows/playwright_ui_tests.yml:

    • Matrix: smoke, regression
    • Command: uv run pytest tests/web -m "${suite} and not selenium" -n 3 --dist=loadscope
    • Uploads playwright-<suite>-allure-results and traces on failure

  • .github/workflows/api_tests.yml:

    • Matrix: smoke, regression
    • Commands:
      • smoke: uv run pytest tests/api -m "api and smoke" -n 3 --dist=loadscope
      • regression: uv run pytest tests/api -m "api and regression" -n 3 --dist=loadscope
    • Uploads api-<suite>-allure-results

  • .github/workflows/selenium_tests.yml:

    • Matrix: smoke, regression
    • Command: uv run pytest tests/web/selenium -m "selenium and ${suite}" -n 2 --dist=loadscope
    • Uploads selenium-<suite>-allure-results and selenium-<suite>-screenshots

  • .github/actions/setup/action.yml:

    • Shared setup used by all reusable workflows
    • Installs uv, Python 3.14, dependencies via uv sync --frozen
    • Optionally installs Playwright Chromium or Google Chrome based on action inputs

Important Trigger Note

Regular push does not start the main CI pipeline by default. Use one of the following:

  • open/update a PR to main
  • run All Tests And Unified Report manually (workflow_dispatch)
  • rely on scheduled weekday run

Allure Artifacts

Playwright fixture (tests/fixtures/playwright.py) attaches artifacts directly to Allure:

  • Trace: attached as .zip with application/vnd.allure.playwright-trace
  • Screenshot: attached as PNG
  • Video: attached as .webm

Selenium fixture (tests/fixtures/selenium.py) also attaches PNG screenshots to Allure, and saves the same files to test-result/screenshots.

Notes:

  • Screenshots for both Playwright and Selenium are controlled by --screenshot (on, only-on-failure, off).
  • Selenium screenshot files are always stored under test-result/screenshots when capture is enabled.
  • CI uploads Selenium screenshots from test-result/screenshots as a separate artifact.
  • Videos are created only for Playwright local runs.
  • On CI (CI=true/1/...), video recording is disabled, so video files/attachments are not produced.
  • Traces and screenshots still work according to your pytest options.

Test Markers

Marker Description
smoke Quick validation tests
regression Full test suite
web Web UI specific tests
api API tests using httpx
selenium Selenium WebDriver UI tests
slow Long-running tests

Architecture

Page Object Model (POM)

Pages live in src/web/pages and are composed with reusable components from src/web/components. Helper utilities are in src/web/helpers.

Fluent Interface Pattern

Methods return Self for method chaining:

app.login_page.open().is_loaded().login_user(email, password)

Application Facade

The Application class provides a unified entry point to all pages:

class Application:
    def __init__(self, page: Page):
        self.page = page
        self.home_page = HomePage(page)
        self.login_page = LoginPage(page)
        self.projects_page = ProjectsPage(page)
        # ...

API Client + Models

API client lives in src/api/client.py, with response models in src/api/models. API tests are under tests/api and use the api_client fixture from tests/fixtures/api.py (JWT is cached per session). Models are built with Pydantic.

Selenium Page Objects

Selenium pages are in src/web/selenium/pages with shared waits and base classes in src/web/selenium/core. Reusable UI components live in src/web/selenium/components, and the Selenium entry point is src/web/selenium/application.py. Selenium tests live in tests/web/selenium (including the enterprise_plan suite) and use the driver fixture from tests/fixtures/selenium.py.

Selenium Application Facade

SeleniumApplication provides a single entry point to Selenium pages and flows.

def test_create_project_via_selenium(selenium_app):
    project_name = "Acme QA"
    (
        selenium_app.new_projects_page.open()
        .is_loaded()
        .fill_project_title(project_name)
        .click_create()
        .is_loaded_empty_project()
        .empty_project_name_is(project_name)
        .close_read_me()
    )

Fixture Strategy

Fixture Scope Purpose
configs session Load environment variables
browser_instance session Reuse browser across tests
context function Fresh context per test
page function Fresh page per test
app function Application facade for fresh page
auth_state session Cached authenticated storage state
logged_context function Authenticated context per test
logged_app function Pre-authenticated page per test
free_project_app function Auth state with empty company_id
shared_page function Shared page for parametrized tests
api_client session Authenticated API client (JWT)
driver function Selenium WebDriver instance
selenium_app function Selenium app facade (logged in)
created_project_selenium function New project via Selenium app

Code Quality

The project uses Ruff for linting and formatting.

# Check for linting issues
ruff check .

# Fix linting issues automatically
ruff check --fix .

# Format code
ruff format .

Browser Configuration

Playwright defaults (configured in tests/fixtures/playwright.py):

  • Resolution: 1920x1080
  • Locale: uk-UA
  • Timezone: Europe/Kyiv
  • Permissions: geolocation
  • Headless: enabled by default
  • Video recording: retain on failure for local runs (default in pyproject.toml)
  • Video on CI: disabled
  • Screenshots: only on failure (default in pyproject.toml)
  • Tracing: retain on failure (default in pyproject.toml)

Selenium defaults (configured in tests/fixtures/selenium.py):

  • Chrome WebDriver with window size 1920x1080
  • Headless mode by default (--headless=new)
  • --headed switches to headed mode
  • Screenshots captured using the same pytest --screenshot option and written to test-result/screenshots

Dependencies

Runtime

  • playwright - Browser automation
  • pytest - Test framework
  • pytest-xdist - Parallel test execution
  • allure-pytest - Allure integration for pytest
  • pytest-playwright - Playwright pytest integration
  • faker - Test data generation
  • httpx - API client
  • pydantic - Data validation and API models
  • selenium - Selenium WebDriver
  • python-dotenv - Environment variable management

Development

  • datamodel-code-generator - Generate Pydantic models from API schema
  • ruff - Linter and formatter

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages