Skip to content

Clarify bundled model fallback and veo-3 default in CLI #1

Description

@AM-young-fun

Background

During staging validation of the new browser login flow, flatkey login succeeded against staging-console.flatkey.ai, but a subsequent video generation failed with Invalid token. While debugging, flatkey models --type video and CLI defaults were misleading because veo-3 appears even when the remote model registry is unavailable, empty, or queried against the wrong origin.

Evidence

Current CLI behavior:

  • src/models.js includes bundled fallback video models: veo-3, veo-3-fast, and seedance2.
  • handleModels returns remote models when available, otherwise silently returns getBundledModels(...).
  • planVideoRequest defaults model to veo-3.

This means a user can see veo-3 in available models even when it came from local fallback, not the active Flatkey environment. In staging/prod origin mixups, that makes it look like veo-3 is actually available for the current account/router.

Root cause / hypothesis

The CLI intentionally has a local bundled model fallback, but it does not make the fallback obvious enough. Combined with the video default of veo-3, staging validation can accidentally test a model that is not actually returned by /v1/available_models for the target environment.

Proposed scope

  • Make flatkey models distinguish remote results from bundled fallback clearly in human output.
  • In JSON output, keep or strengthen source: "remote" | "bundled" and avoid hiding request failures.
  • Consider warning when falling back to bundled models after /v1/available_models fails.
  • Consider changing the default video model from veo-3 to the currently supported/default product model, or require --model when the model registry cannot be reached.
  • Add regression tests for remote empty/error responses and human output clarity.

Acceptance criteria

  • A staging user can tell whether veo-3 came from the live model registry or local bundled fallback.
  • flatkey models --type video --json remains machine-readable and includes source information.
  • Video generation defaults do not imply that a model is remotely available when registry lookup failed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions