Skip to content

fix: Respect cc_model_filenames in Python backend auto-fill#489

Open
nightflight-dk wants to merge 12 commits into
triton-inference-server:mainfrom
nightflight-dk:cc_model_filename-respected-by-py-backend
Open

fix: Respect cc_model_filenames in Python backend auto-fill#489
nightflight-dk wants to merge 12 commits into
triton-inference-server:mainfrom
nightflight-dk:cc_model_filename-respected-by-py-backend

Conversation

@nightflight-dk
Copy link
Copy Markdown
Contributor

fix: Respect cc_model_filenames in Python backend auto-fill

Problem

When backend is set to "python" and default_model_filename is left empty, AutoCompleteBackendFields unconditionally sets default_model_filename to "model.py". This ignores any cc_model_filenames mappings the user may have configured to use a custom model filename (e.g., custom_model.py per device).

This causes the Python backend stub to always look for model.py, even when cc_model_filenames explicitly maps to a different file.

Fix

In model_config_utils.cc, the auto-fill of default_model_filename now checks that cc_model_filenames is also empty before defaulting to "model.py":

if (config->default_model_filename().empty() &&
    config->cc_model_filenames().empty()) {
  config->set_default_model_filename(kPythonFilename);
}

nightflight-dk and others added 8 commits February 10, 2026 22:38
Co-authored-by: Yingge He <157551214+yinggeh@users.noreply.github.com>
ManagedIdentityCredentialOptions does not have a ClientId field.
Use the constructor overload that accepts clientId as a string
parameter instead. Also consolidate identity includes to the
top-level azure/identity.hpp header and remove WIP comment.
@nightflight-dk nightflight-dk force-pushed the cc_model_filename-respected-by-py-backend branch from cfae5e6 to 5f7ac3b Compare April 12, 2026 01:37
@nightflight-dk
Copy link
Copy Markdown
Contributor Author

nightflight-dk commented Apr 12, 2026

@whoisj this helps provide cc dependent implementation part of the same repo

@whoisj
Copy link
Copy Markdown
Contributor

whoisj commented Apr 21, 2026

LGTM. I'll run it through our internal CI lab.

@nightflight-dk, could you please address the pre-commit issues? Thanks.

CI Pipeline ID: 49121709

you can always run pre-commit locally via

pre-commit run --files <file you've changed>

Comment thread src/test/model_config_utils_test.cc
@nightflight-dk nightflight-dk force-pushed the cc_model_filename-respected-by-py-backend branch from 5f7ac3b to 31c39bb Compare April 21, 2026 23:28
@nightflight-dk nightflight-dk force-pushed the cc_model_filename-respected-by-py-backend branch from 31c39bb to 0fc4a6d Compare April 22, 2026 01:44
@nightflight-dk nightflight-dk requested a review from yinggeh April 22, 2026 01:50
Copy link
Copy Markdown
Contributor

@whoisj whoisj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @yinggeh are you satisfied with the requested test case?

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Python backend auto-fill so default_model_filename is only defaulted to model.py when both default_model_filename and cc_model_filenames are empty, preserving user-provided compute-capability filename mappings.

Changes:

  • Update AutoCompleteBackendFields to skip defaulting default_model_filename when cc_model_filenames is populated.
  • Add a dedicated unit test binary covering Python backend defaulting / non-defaulting behavior.
  • Ignore /build_test output directory.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/model_config_utils.cc Adjusts Python backend auto-fill condition to respect cc_model_filenames.
src/test/model_config_utils_test.cc Adds unit tests validating the new auto-fill behavior and preservation rules.
src/test/CMakeLists.txt Adds model_config_utils_test target and links required dependencies.
.gitignore Ignores build_test directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to +57
std::string cmd = "rm -rf " + root_path_;
(void)system(cmd.c_str());
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TempModelDir::~TempModelDir uses system("rm -rf ...") for cleanup. This is non-portable (breaks on Windows), can behave incorrectly if the path contains spaces/shell metacharacters, and system may not be declared here (no include), causing a compile failure in C++ builds. Prefer using the existing filesystem helper (e.g., triton::core::DeletePath(root_path_)), and optionally assert/expect the returned Status in teardown/cleanup.

Suggested change
std::string cmd = "rm -rf " + root_path_;
(void)system(cmd.c_str());
auto status = tc::DeletePath(root_path_);
EXPECT_TRUE(status.IsOk()) << status.AsString();

Copilot uses AI. Check for mistakes.
Comment on lines +64 to +69
std::string version_dir = tc::JoinPath({root_path_, version});
mkdir(version_dir.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
if (!filename.empty()) {
std::ofstream f(tc::JoinPath({version_dir, filename}));
f << "# placeholder";
}
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test setup ignores failures from mkdir(...) and from writing the placeholder file (stream open/write errors). If either step fails, the test may pass/fail for the wrong reason. Please check the return value of mkdir (and consider handling EEXIST), and validate the std::ofstream state (or use triton::core::MakeDirectory / WriteTextFile which return a Status).

Copilot uses AI. Check for mistakes.
Comment on lines +116 to +118
EXPECT_EQ(config.default_model_filename(), "")
<< "default_model_filename should remain empty when cc_model_filenames "
"is set";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, which python file backend should load if not "gpu"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

5 participants