Implement Evaluator Versions - #402

Closed

dphuang2 wants to merge 45 commits into

mainfrom

dhuang/dxe-478-implement-evaluator-versions

Commits on Jan 7, 2026

Refactor Fireworks client integration
Dylan Huang
committed
0a00bcd
remove launch.json
Dylan Huang
committed
d465a89
Add .vscode/launch.json to .gitignore
Dylan Huang
committed
348bb58
Enhance environment variable loading in auth module
Dylan Huang
committed
acaa901
Add evaluator version creation in evaluation module
Dylan Huang
committed
4b71ddb

Commits on Jan 8, 2026

test
Dylan Huang
committed
3dbcd59
REVERT this later
Dylan Huang
committed
532e071
Merge branch 'main' into dhuang/dxe-478-implement-evaluator-versions
Dylan Huang
committed
5e7a5fa

Commits on Jan 9, 2026

fix mock tests
Dylan Huang
committed
060d72c
Add error handling for evaluator creation in evaluation module
Dylan Huang
committed
bc31c9f
Support EP_REMOTE_API_KEY
Dylan Huang
committed
ea08062
Merge branch 'main' into dhuang/dxe-478-implement-evaluator-versions
Dylan Huang
committed
f246087

Commits on Jan 12, 2026

include launch.json.backup
Dylan Huang
committed
6b53ac1
rename to .example and add docker run extra arg
Dylan Huang
committed
ec0c8ca
use ignore-docker by default
Dylan Huang
committed
fc036f5

Commits on Jan 13, 2026

delete backup
Dylan Huang
committed
4566584
ignore-docker by default in dev
Dylan Huang
committed
f103b69
Refactor evaluator function calls to use Fireworks directly for method signature introspection, avoiding unnecessary API requests during help invocations.
Dylan Huang
committed
9c3e417
use in-flight SDK version
Dylan Huang
committed
ea673f4
Enhance evaluator handling by returning version ID on creation and updating polling functions to target specific evaluator versions. Refactor related CLI commands and tests to accommodate these cha…
Dylan Huang
committed
26fbc2d
update
Dylan Huang
committed
4702307
use published a22 of fireworks-ai
Dylan Huang
committed
9d1bc74
uv lock
Dylan Huang
committed
3314bec

Commits on Jan 14, 2026

Refactor dotenv handling in auth module and integrate environment variable loading into local test command. Introduced functions to find and retrieve values from .env files, enhancing configuration…
Dylan Huang
committed
66f191a
add create rft launch configuration
Dylan Huang
committed
165afe1
Refactor dotenv handling in auth module and integrate environment variable loading into local test command. Introduced functions to find and retrieve values from .env files, enhancing configuration…
Dylan Huang
committed
838c7a5
Merge branch 'pass-dot-env-to-docker-container' into dhuang/dxe-478-implement-evaluator-versions
Dylan Huang
committed
71599e6
actually not necessary for local test since local-test mounts the workspace so it should include the .env file
Dylan Huang
committed
0144c9f
increase sql retries
Dylan Huang
committed
c8774a6
Refactor dotenv loading to use explicit paths in CLI and API modules
Dylan Huang
committed
2076f0a
Merge branch 'main' into dhuang/dxe-478-implement-evaluator-versions
Dylan Huang
committed
8acdc35
Refactor dotenv loading to use explicit paths in CLI and API modules
Dylan Huang
committed
432a649
Merge branch 'ensure-explicit-dotenv' into dhuang/dxe-478-implement-evaluator-versions
Dylan Huang
committed
ab04086
"ep create evj"
Dylan Huang
committed
3c2db59
use SDK for Dataset API calls
Dylan Huang
committed
17eb18f

Commits on Jan 15, 2026

Implement evaluator upload and status polling in create commands
Dylan Huang
committed
1fd66f7
Add secret management for uploads in CLI
Dylan Huang
committed
fc4f913
handle existing secrets with caution
Dylan Huang
committed
2f88428
Integrate secrets upload handling in CLI commands
Dylan Huang
committed
c6a8c51
Remove unused `_to_pyargs_nodeid` function from `upload.py` to enhance code clarity and maintainability.
Dylan Huang
committed
a2165fb
increase sql retries
Dylan Huang
committed
1445d75
Refactor secret loading in CLI to use python-dotenv
Dylan Huang
committed
7969a6e

Commits on Jan 16, 2026

make connection more robust
Dylan Huang
committed
d4a445b
Merge branch 'increase-sql-retries' into dhuang/dxe-478-implement-evaluator-versions
Dylan Huang
committed
b3adfee
passes
Dylan Huang
committed
37f4856