Skip to content

Document evalhub server command and server_config_file config key from eval-hub-sdk PR #149 #81

Description

@coderabbitai

Summary

Document the new evalhub server CLI command group and related server configuration management introduced in eval-hub/eval-hub-sdk#149.

Background

PR eval-hub/eval-hub-sdk#149 adds:

  • An evalhub server command group for managing the eval-hub-server lifecycle:
    • evalhub server run — run the server in the foreground.
    • evalhub server start — start the server as a background daemon (writes a PID file and log file, waits for the health endpoint before reporting success).
    • evalhub server stop — stop the background daemon (graceful shutdown with a timeout, then force-kill).
    • evalhub server status — report whether the server is running and healthy, including URL and log path when available.
  • Server configuration management through the existing evalhub config workflow using a new server_config_file key:
    • evalhub config set server_config_file <path> — validates the YAML file and stores a profile-scoped copy under ~/.config/evalhub/server/<profile>/config.yaml.
    • evalhub config get server_config_file / show — displays the stored value.
    • evalhub config unset server_config_file — removes the stored config file and cleans up the profile directory.
  • A new --unfold flag on evalhub config get to print the contents of file-based configuration values (currently only applicable to server_config_file).
  • Support for TLS-enabled servers (health checks over HTTPS, https:// URLs reported in start/status output).

Requested documentation updates

  • Add a new "Server" section to the CLI reference documenting evalhub server run|start|stop|status, including example invocations (foreground vs background) and description of PID/log file locations.
  • Document the server_config_file config key: how to set/get/unset it, and the YAML validation requirements (must exist, be valid YAML, and be a mapping).
  • Document the --unfold flag on evalhub config get, its mutual exclusivity with --unmask, and that it only applies to file-based config keys.
  • Include examples covering: starting the server in the foreground, starting/stopping in the background, checking status, handling TLS, and removing the server config setting.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions