Skip to content

Conversation

@nttg8100
Copy link
Member

@nttg8100 nttg8100 commented Jan 9, 2026

This pull request introduces several important improvements focused on backend security and API consistency, along with some infrastructure fixes and dependency management changes. The most significant updates enhance how temporary storage credentials are handled for jobs, ensuring they are generated securely within the worker rather than being passed through the broker. Additionally, API responses for list endpoints are standardized, and a new serializer is introduced for safer analysis updates.

Backend Security and Job Credential Handling:

  • Temporary storage credentials for jobs are now generated just-in-time within the worker (submit_job in tasks.py), preventing credentials from being exposed via Redis/Celery. Credentials are fetched from the database and injected into job parameters only at execution time, and credential files are cleaned up after job completion or failure. [1] [2] [3] [4] [5]
  • The token field in the Github model is now stored using EncryptedTextField for improved credential security.

API Consistency and Serialization:

  • All list API endpoints now use "items" instead of "item" for returned collections, and corresponding test cases are updated for consistency. [1] [2] [3] [4] [5] [6] [7] [8] [9]
  • Introduced AnalysisUpdateSerializer for validating analysis updates, enforcing required fields and improving error handling for missing or unauthorized updates. [1] [2] [3]

Infrastructure and Dependency Management:

  • Job status SERVER_ERROR is replaced with HPC_DISCONNECTED to better reflect failure scenarios.
  • Makefile targets for backend and test dependencies now ensure that ASGI query string parsing is patched and dependencies are installed consistently. [1] [2] [3]
  • Dockerfile is updated to patch the socketify ASGI query string parsing issue and ensure required system libraries are installed.

Miscellaneous Fixes:

  • The .gitignore entry for .pixi is corrected to avoid ignoring the configuration file.
  • Minor code cleanup and removal of debug print statements.
  • Docker Compose for production now runs Traefik in detached mode.

nttg8100 and others added 28 commits January 4, 2026 13:02
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: handle server disconnect

* fix: add github token to encrypt

* feat: remove credentails from params in celery

* feat: increase max_length token

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@nttg8100 nttg8100 requested a review from Copilot January 9, 2026 13:16
@nttg8100 nttg8100 self-assigned this Jan 9, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This release focuses on enhancing backend security by improving credential handling for job execution, standardizing API response formats across list endpoints, and implementing various infrastructure improvements. The key changes ensure that temporary storage credentials are generated just-in-time within workers rather than being passed through Redis/Celery, improving the overall security posture.

Key Changes:

  • Temporary storage credentials are now generated securely within workers at execution time, preventing exposure through Redis/Celery broker
  • All list API endpoints standardized to use "items" instead of "item" for consistency
  • Job status updated from SERVER_ERROR to HPC_DISCONNECTED for better semantic clarity

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
frontend/src/features/project/components/Projects.tsx Increased default page size from 5 to 10 items
frontend/src/features/credential/contexts/CredentialContext.tsx Added username check for GitHub credential updates
frontend/src/features/analysis/contexts/AnalysisContext.tsx Updated expected HTTP status code from 200 to 204 for analysis updates
docker-compose.yaml Added UTC timezone configuration for Redis service
backend/run_server.sh Removed unnecessary quotes around worker count argument
backend/requirements.txt Added cffi development dependency
backend/perf/install.sh Added bombardier installation and load testing, updated Python version requirement
backend/migrations/models/6_20260105224743_update.py Migration to change GitHub token field from VARCHAR to TEXT for encryption support
backend/migrations/models/5_20260105205918_update.py Migration updating job status comments from SERVER_ERROR to HPC_DISCONNECTED
backend/app/utils/executor/ssh.py Enhanced tunnel cleanup, added auth headers for Traefik, improved job monitoring logic
backend/app/settings.py Simplified BASE_API_HOST and IS_SSL configuration logic
backend/app/service_project/tests/test_project.py Updated tests to use "items" instead of "item" for list responses
backend/app/service_project/controller.py Changed list endpoint response key from "item" to "items"
backend/app/service_organization/tests/test_organization.py Updated tests to use "items" instead of "item" for list responses
backend/app/service_organization/controller.py Changed list endpoint response key from "item" to "items"
backend/app/service_job/tests/test_api_web_job.py Updated tests to use "items" instead of "item" for list responses
backend/app/service_job/tasks.py Implemented just-in-time credential generation within worker, added credential cleanup
backend/app/service_job/models.py Renamed SERVER_ERROR status to HPC_DISCONNECTED
backend/app/service_job/controller.py Modified job creation to pass storage ID instead of credentials, added credential validation
backend/app/service_credential/models/personal.py Changed GitHub token field to use EncryptedTextField
backend/app/service_analysis/tests/test_analysis.py Updated tests to use "items" instead of "item" for list responses
backend/app/service_analysis/serializers.py Added AnalysisUpdateSerializer for validating analysis updates
backend/app/service_analysis/controller.py Implemented AnalysisUpdateSerializer, added validation for unauthorized updates
backend/app/main.py Removed debug print statement
backend/Dockerfile Added socketify ASGI query string parsing fix, reorganized dependency installation
backend/.gitignore Corrected .pixi gitignore pattern to avoid ignoring configuration file
Makefile Added install-backend-deps target with socketify patch, updated test dependency targets
Comments suppressed due to low confidence (1)

backend/perf/install.sh:1

  • The REQUIRED_PIXI_PKGS variable includes python=3.12, but lines 550-552 duplicate the package installations without using this variable. Remove the duplicate installation commands on lines 550-552 to avoid redundancy and potential version conflicts.
# app.py

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nttg8100 nttg8100 merged commit 51394c7 into main Jan 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants