Add backend API, frontend types, SDK & CI updates - #7
Merged
BarryHenryJr merged 4 commits intoMar 13, 2026
Merged
Conversation
Large refactor and feature addition across the repo: introduces a structured backend API (core api, selectors, services, management commands, tasks) with tests and new Django settings, adds OpenAPI export and generated frontend types, and ships a small Python SDK with tests. CI workflow was overhauled: renamed/streamlined jobs (validate, security, codeql), upgraded runtimes, pnpm support, automated OpenAPI export/verification, and unified validation steps to run backend/frontend/SDK checks and tests. Makefile and README were updated to reflect the new developer workflow and commands. Removed some legacy files and simplified the project layout to improve developer experience and ensure type-safe frontend integration with the backend API.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a major refactor and feature addition to the TelemetryTaco project, focusing on a structured backend API, type-safe frontend integration, and a streamlined developer workflow. The changes enhance the project's maintainability, scalability, and developer experience, ensuring a robust foundation for future development. Highlights
Changelog
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This is an impressive and extensive refactoring that significantly improves the project's structure, maintainability, and developer experience. The introduction of a structured backend API with services and selectors, the overhaul of the frontend with React Query and generated types, and the robust new Python SDK are all excellent changes. The CI/CD and developer tooling updates also streamline the workflow considerably.
I've left a few minor suggestions for improvement, mostly related to documentation clarity and code simplification. Overall, this is a very high-quality pull request.
There was a problem hiding this comment.
Pull request overview
This PR refactors TelemetryTaco into a more structured backend API + typed frontend integration, adds a queue-backed Python SDK, and overhauls CI/dev workflows to validate backend/frontend/SDK and keep OpenAPI artifacts in sync.
Changes:
- Backend: split settings modules, introduce service/selector/task layers, add OpenAPI export + management commands, and add pytest coverage.
- Frontend: generate OpenAPI TypeScript types, migrate data fetching to React Query, add Vitest setup/tests, and refresh UI components/styles.
- Tooling: update shell scripts + pnpm commands, add SDK package skeleton/tests, and streamline CI jobs (validate/security/codeql).
Reviewed changes
Copilot reviewed 72 out of 76 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| start.sh | Adds POETRY_CACHE_DIR defaulting and passes it into Poetry commands. |
| seed.sh | Ensures dependency install + seed command uses Poetry cache dir. |
| restart-backend.sh | Ensures backend restart uses Poetry cache dir. |
| package.json | Adds unified validate/test scripts, OpenAPI generation, and sets Poetry cache dir in backend commands. |
| frontend/vite.config.ts | Adds Vitest configuration and splits vendor chunks (React Query). |
| frontend/tailwind.config.js | Switches primary/secondary colors to CSS variables for theme control. |
| frontend/src/vite-env.d.ts | Adds Vitest global type references for tests. |
| frontend/src/test/test-utils.tsx | Adds Testing Library render helper with React Query provider. |
| frontend/src/test/setup.ts | Adds jest-dom matchers for Vitest. |
| frontend/src/shared/ui/panel-message.tsx | Introduces reusable panel empty/error/loading message component. |
| frontend/src/shared/api/types.ts | Adds app-level types over generated OpenAPI types (EventRecord/InsightPoint/etc). |
| frontend/src/shared/api/generated.ts | Adds generated OpenAPI TypeScript definitions. |
| frontend/src/shared/api/client.ts | Adds API URL builder + typed fetch wrapper with structured errors. |
| frontend/src/main.tsx | Wraps app in AppProviders (React Query provider). |
| frontend/src/index.css | Updates theme tokens and base layout styling (dark scheme, radius, etc.). |
| frontend/src/hooks/useEventStream.ts | Removes legacy polling hook in favor of query layer. |
| frontend/src/features/insights/queries.ts | Adds React Query-based insights polling + error mapping. |
| frontend/src/features/insights/components/insight-line-chart.tsx | Adds insights chart visualization component. |
| frontend/src/features/insights/components/insight-chart-card.tsx | Adds insights card with loading/error/empty states and lazy chart import. |
| frontend/src/features/insights/components/insight-chart-card.test.tsx | Adds Vitest coverage for insights card states. |
| frontend/src/features/events/queries.ts | Adds React Query-based events polling + error mapping. |
| frontend/src/features/events/components/live-event-stream-card.tsx | Adds live event stream UI with expandable rows and deferred rendering. |
| frontend/src/features/events/components/live-event-stream-card.test.tsx | Adds Vitest coverage for live stream states + rendering. |
| frontend/src/components/LiveEventStream.tsx | Removes legacy event stream component. |
| frontend/src/components/InsightChart.tsx | Removes legacy insights chart component. |
| frontend/src/app/query-client.ts | Centralizes React Query client configuration. |
| frontend/src/app/providers.tsx | Adds app-wide providers wrapper for React Query. |
| frontend/src/app/App.tsx | Introduces new app shell layout and lazy-loads insights surface. |
| frontend/src/App.tsx | Re-exports new app entry from @/app/App. |
| frontend/package.json | Adds Vitest + OpenAPI type generation scripts and dependencies. |
| frontend/openapi.json | Adds committed OpenAPI schema output for type generation. |
| frontend/eslint.config.js | Adds test-file globals and adjusts rules for vitest/test environment. |
| backend/telemetry_taco/urls.py | Switches to centralized telemetry_taco.api API instance. |
| backend/telemetry_taco/settings/base.py | Introduces new base settings (env, redis/cache, celery, rate limits, logging). |
| backend/telemetry_taco/settings/development.py | Adds dev overrides (CORS defaults, dev rate limits). |
| backend/telemetry_taco/settings/test.py | Adds test overrides (sqlite/locmem cache, eager Celery, relaxed rate limits). |
| backend/telemetry_taco/settings/production.py | Adds production validation for secret key requirements. |
| backend/telemetry_taco/settings/init.py | Defaults telemetry_taco.settings to development settings module. |
| backend/telemetry_taco/settings.py | Removes legacy monolithic settings module. |
| backend/telemetry_taco/api.py | Centralizes NinjaAPI instance and attaches core router. |
| backend/requirements.txt | Removes legacy pip requirements file in favor of Poetry. |
| backend/pyproject.toml | Adds pytest tooling config and dependencies for pytest-django. |
| backend/poetry.lock | Updates lockfile for newly added dev deps and marker changes. |
| backend/core/tests/test_tasks.py | Adds test coverage for batch task idempotency via UUID. |
| backend/core/tests/test_api.py | Adds endpoint-level tests for capture/batch/events/insights/health/retention command. |
| backend/core/tests/init.py | Adds tests package marker for pytest discovery. |
| backend/core/tests.py | Removes unused default Django test stub. |
| backend/core/tasks/events.py | Adds batch processing task + purge task + timestamp parsing helpers. |
| backend/core/tasks/init.py | Exposes tasks from tasks package for imports/autodiscovery. |
| backend/core/tasks.py | Removes legacy single-event task module. |
| backend/core/services/ingestion.py | Adds ingestion normalization + enqueue logic for capture and batch endpoints. |
| backend/core/services/health.py | Adds readiness/liveness dependency checks (DB/cache). |
| backend/core/services/init.py | Exposes services API surface. |
| backend/core/selectors/events.py | Adds selectors for list events, insights aggregation, and retention purge. |
| backend/core/selectors/init.py | Exposes selectors API surface. |
| backend/core/management/commands/purge_expired_events.py | Adds retention purge management command. |
| backend/core/management/commands/export_openapi_schema.py | Adds management command to export OpenAPI schema JSON. |
| backend/core/api/schemas.py | Adds request/response schemas for capture/events/insights/health endpoints. |
| backend/core/api/events.py | Introduces Ninja router implementing capture/batch/events/insights/health endpoints. |
| backend/core/api/init.py | Exposes core API router. |
| backend/core/api.py | Removes legacy router + schema implementations (superseded by module structure). |
| backend/core/admin.py | Registers Event model in Django admin with useful list/search config. |
| backend/core/init.py | Exposes Celery app for Django/Celery integration. |
| backend/SETUP.md | Updates backend setup guide to new settings modules and workflow. |
| backend/.env.test | Adds test env defaults file. |
| backend/.env.example | Adds example env with new configuration knobs. |
| README.md | Updates repo docs to the new MVP-focused structure and workflow commands. |
| Makefile | Updates targets to use pnpm-based test/validate flows. |
| .github/workflows/cicd.yml | Overhauls CI into validate/security/codeql jobs with OpenAPI/type verification. |
| sdk/tests/test_client.py | Adds SDK tests for batching flush behavior and queue-full policy. |
| sdk/telemetry_taco/client.py | Adds queue-backed SDK client with batching, retries, and flush/close semantics. |
| sdk/telemetry_taco/init.py | Exposes SDK public API via package init. |
| sdk/telemetry_taco.py | Removes legacy single-file SDK implementation. |
| sdk/pyproject.toml | Adds PEP 621 project metadata for the SDK package. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dea6824a5b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Add stable cursor pagination and readiness status handling; refactor event ingestion and worker robustness. Key changes: - API: support `before=timestamp,id` cursor parsing and accept plain timestamps for backward compatibility; list_events now accepts string cursors and readiness endpoint can return 503 when dependencies are degraded. - Selectors: list_recent_events updated to handle timestamp+id cursors for stable pagination across equal timestamps. - Schemas/OpenAPI/frontend types: replace ModelSchema with explicit Pydantic/Ninja Schema (from_attributes), adjust EventResponseSchema fields (id, properties, timestamp required) and add 503 response in OpenAPI and generated TS types. - Tasks/Services: refactor event persistence (extract _build_event and _persist_events), remove client-side chunking when enqueuing events, ensure tasks return processed counts and log event_count accordingly, and expose a process_event_task that persists single events. - SDK: improve TelemetryTaco worker shutdown and draining behavior, defend against non-serializable batches when encoding payloads, and add a test ensuring a bad batch doesn't kill the worker. - Tests: add and update tests for stable pagination, invalid cursors, readiness 503 behavior, and process_event_task; adjust existing tests for cursor semantics. - Misc: update backend .env.example guidance for SECRET_KEY generation; add backend test sqlite DB; streamline package.json scripts to use poetry directly; add setuptools package find config in sdk pyproject.toml. These changes improve pagination stability, make background processing more resilient, clarify API contracts, and align generated client types with the server schemas.
Collapse the multiline Q(...) | Q(...) condition in backend/core/selectors/events.py into a single line for readability. This is a pure formatting change with no behavioral impact.
There was a problem hiding this comment.
Pull request overview
This PR introduces a more structured end-to-end telemetry stack: a refactored Django Ninja API with split settings and pytest coverage, OpenAPI export + generated frontend types, a new queued/batched Python SDK, and a consolidated CI workflow to validate backend/frontend/SDK together.
Changes:
- Backend: split settings modules, structured API/services/selectors/tasks, management commands, and expanded pytest coverage.
- Frontend: OpenAPI-driven generated types, React Query polling surfaces, Vitest setup, and UI refresh.
- Tooling/CI: unified validation scripts (pnpm), OpenAPI generation verification, and streamlined GitHub Actions jobs.
Reviewed changes
Copilot reviewed 72 out of 77 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| start.sh | Adds POETRY_CACHE_DIR usage for Poetry commands. |
| seed.sh | Adds POETRY_CACHE_DIR usage for Poetry install/run. |
| restart-backend.sh | Adds POETRY_CACHE_DIR for backend restart command. |
| package.json | Adds OpenAPI type-gen + backend/frontend/sdk test/validate scripts. |
| Makefile | Routes test/validate through pnpm scripts. |
| .github/workflows/cicd.yml | New Validate/Security/CodeQL job layout + OpenAPI/type checks. |
| README.md | Updates docs to reflect new architecture and workflow. |
| frontend/vite.config.ts | Adds Vitest config and new build chunk splitting. |
| frontend/tailwind.config.js | Switches primary/secondary colors to CSS variables. |
| frontend/src/vite-env.d.ts | Adds Vitest globals reference types. |
| frontend/src/test/test-utils.tsx | Adds React Query test wrapper utilities. |
| frontend/src/test/setup.ts | Adds jest-dom matchers for Vitest. |
| frontend/src/shared/ui/panel-message.tsx | Adds reusable empty/error panel component. |
| frontend/src/shared/api/client.ts | Adds typed fetch helper + API error wrapper. |
| frontend/src/shared/api/types.ts | Adds app-level types based on generated OpenAPI components. |
| frontend/src/shared/api/generated.ts | Adds generated OpenAPI TypeScript bindings. |
| frontend/src/main.tsx | Wraps app with providers. |
| frontend/src/index.css | Updates theme tokens and base styles. |
| frontend/src/hooks/useEventStream.ts | Removes legacy polling hook. |
| frontend/src/features/insights/queries.ts | Adds React Query insights polling query. |
| frontend/src/features/insights/components/insight-line-chart.tsx | Adds insights chart component. |
| frontend/src/features/insights/components/insight-chart-card.tsx | Adds insights card with loading/empty/error states + lazy chart. |
| frontend/src/features/insights/components/insight-chart-card.test.tsx | Adds Vitest coverage for insights card states. |
| frontend/src/features/events/queries.ts | Adds React Query events polling query. |
| frontend/src/features/events/components/live-event-stream-card.tsx | Adds new live event stream card component. |
| frontend/src/features/events/components/live-event-stream-card.test.tsx | Adds Vitest coverage for live event stream states. |
| frontend/src/components/LiveEventStream.tsx | Removes legacy live stream component. |
| frontend/src/components/InsightChart.tsx | Removes legacy insights component. |
| frontend/src/app/query-client.ts | Centralizes QueryClient creation defaults. |
| frontend/src/app/providers.tsx | Adds top-level app providers. |
| frontend/src/app/App.tsx | New app shell combining insights + live stream. |
| frontend/src/App.tsx | Re-exports app entry from new location. |
| frontend/package.json | Adds vitest/testing deps + OpenAPI type-gen script. |
| frontend/openapi.json | Adds committed OpenAPI schema snapshot used for type generation. |
| frontend/eslint.config.js | Updates ESLint for Vitest globals and disables no-undef. |
| backend/telemetry_taco/urls.py | Switches API instantiation to telemetry_taco.api. |
| backend/telemetry_taco/api.py | Adds centralized NinjaAPI instance/version. |
| backend/telemetry_taco/settings/base.py | Introduces shared base settings with env config. |
| backend/telemetry_taco/settings/development.py | Adds development-specific overrides. |
| backend/telemetry_taco/settings/test.py | Adds isolated test settings (sqlite/locmem/eager celery). |
| backend/telemetry_taco/settings/production.py | Adds production checks for SECRET_KEY correctness. |
| backend/telemetry_taco/settings/init.py | Defaults settings module to development. |
| backend/telemetry_taco/settings.py | Removes legacy single-file settings. |
| backend/requirements.txt | Removes legacy pip requirements file. |
| backend/pyproject.toml | Adds pytest/pytest-django config and deps. |
| backend/poetry.lock | Updates lockfile for new dev dependencies. |
| backend/core/api/events.py | Adds structured API routes (capture/batch/events/insights/health). |
| backend/core/api/schemas.py | Adds pydantic schemas for API surface. |
| backend/core/api/init.py | Exposes router via package init. |
| backend/core/api.py | Removes legacy API module. |
| backend/core/services/ingestion.py | Adds ingestion normalization + batch enqueue service. |
| backend/core/services/health.py | Adds liveness/readiness status helpers. |
| backend/core/services/init.py | Exposes service functions. |
| backend/core/selectors/events.py | Adds selectors for list/insights/retention purge. |
| backend/core/selectors/init.py | Exposes selector functions. |
| backend/core/tasks/events.py | Adds Celery tasks for batch/single processing and purge. |
| backend/core/tasks/init.py | Exposes task callables. |
| backend/core/tasks.py | Removes legacy single-event task module. |
| backend/core/management/commands/purge_expired_events.py | Adds retention purge command. |
| backend/core/management/commands/export_openapi_schema.py | Adds OpenAPI export command used by CI/frontend. |
| backend/core/tests/test_api.py | Adds pytest coverage for API behavior and edge cases. |
| backend/core/tests/test_tasks.py | Adds pytest coverage for Celery task behavior. |
| backend/core/tests/init.py | Adds tests package marker. |
| backend/core/tests.py | Removes legacy Django tests module stub. |
| backend/core/admin.py | Registers Event model in admin. |
| backend/core/init.py | Exposes celery app import hook. |
| backend/SETUP.md | Updates backend setup docs to match new workflow/settings. |
| backend/.env.example | Adds example env values for local dev. |
| backend/.env.test | Adds test env file (repo-provided). |
| sdk/telemetry_taco/client.py | Replaces SDK with queued, batched sender + retries. |
| sdk/telemetry_taco/init.py | Exposes SDK public API. |
| sdk/telemetry_taco.py | Removes legacy SDK implementation file. |
| sdk/pyproject.toml | Adds SDK packaging + pytest pythonpath config. |
| sdk/tests/test_client.py | Adds SDK tests for batching, queue policy, serialization failures. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Backend: make event timestamp parsing more robust — accept naive datetimes (make aware), accept date objects (convert to start-of-day aware datetime), require strings for ISO parsing, and raise on unsupported types; added tests for date timestamps and invalid timestamp objects. Frontend: memoize QueryClient with useState to avoid recreating it on each render. SDK: normalize and validate base_url, use UTC constant, add thread-safety for capture/close with a state lock and closing flag, implement dropping/replacing oldest queued event safely, improve error handling/logging when sending batches fails; added and updated tests to cover sentinel preservation, base_url normalization/validation, and failed request handling.
BarryHenryJr
deleted the
6-chore-refactor-refactor-and-clean-up-the-codebase
branch
March 13, 2026 19:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Large refactor and feature addition across the repo: introduces a structured backend API (core api, selectors, services, management commands, tasks) with tests and new Django settings, adds OpenAPI export and generated frontend types, and ships a small Python SDK with tests. CI workflow was overhauled: renamed/streamlined jobs (validate, security, codeql), upgraded runtimes, pnpm support, automated OpenAPI export/verification, and unified validation steps to run backend/frontend/SDK checks and tests. Makefile and README were updated to reflect the new developer workflow and commands. Removed some legacy files and simplified the project layout to improve developer experience and ensure type-safe frontend integration with the backend API.