Feat/opentelemetry#7
Open
kudroma404 wants to merge 33 commits into
Open
Conversation
…ing dependencies)
…ion handling in pipelines
…ng for external model paths
…; removed one redundant cuda arch (10.0) (#4)
* exposed option for controlling face amount for decimation
Log lines for task received, parameters, generation timing, and task completion use the format: task_id: message. Co-authored-by: Cursor <cursoragent@cursor.com>
Loki and VictoriaMetrics labels use generator_id instead of worker_id. SERVICE_NAME is set to 404-generator-mesh-v1 for metrics and log streams. Co-authored-by: Cursor <cursoragent@cursor.com>
GENERATION_SYNTHETIC_FAILURE_RATE (default 0) triggers a random failure before GPU work so generation_error_count increments without wasting compute. Log startup when the rate is positive. Co-authored-by: Cursor <cursoragent@cursor.com>
Loki stream labels and VictoriaMetrics counter/histogram labels use generator_mesh_v1_id instead of generator_id. Update serve wiring and VictoriaMetricsManager call sites. Co-authored-by: Cursor <cursoragent@cursor.com>
Wire task_id into Loki stream labels via record extra; contextualize /generate and generation_block so executor-thread logs include it. Co-authored-by: Cursor <cursoragent@cursor.com>
…ontext Replace sys.stderr with a Loguru-backed writer for the duration of generation_block so that tqdm progress bars and any other stderr output from third-party libraries (trellis, o_voxel) are captured as structured DEBUG log lines. Because the redirect runs inside logger.contextualize(), the task_id label is automatically included and the lines are shipped to Loki alongside every other structured log from that task. Co-authored-by: Cursor <cursoragent@cursor.com>
The synthetic failure mechanism was used for Victoria metrics testing and is no longer needed. Remove _maybe_synthetic_generation_failure, its call site, the startup warning log, and the now-unused random import. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace unused generation_synthetic_failure_rate with test_run bool. When TEST_RUN=true the /generate endpoint skips real GPU work and rotates through HTTP 500 / 503 / 429 errors and a mock success on each successive request, recording metrics for each step. Mirrors the worker's existing MockGenerator pattern. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restore TEST_RUN as the mesh v1 error-injection switch and fail roughly half of real generation requests. Co-authored-by: Cursor <cursoragent@cursor.com>
Prefix xatlas and tqdm stderr progress lines with task_id so plain container logs can be correlated to requests. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… metric Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…/failure Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sting" This reverts commit 479cd2e.
A 1x1 degenerate image causes preprocess_image to produce an empty bounding box, crashing with a numpy zero-size reduction error deep in the pipeline. Fail early with a clear message instead. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.