Skip to content

Bring traditional-local to feature parity for built-in model comparisons#22

Merged
sharpninja merged 2 commits intomainfrom
copilot/build-traditional-model-features
Mar 21, 2026
Merged

Bring traditional-local to feature parity for built-in model comparisons#22
sharpninja merged 2 commits intomainfrom
copilot/build-traditional-model-features

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 21, 2026

The built-in comparison path was asymmetric: bitnet-b1.58-sharp supported checkpoint round-tripping and runtime inspection surfaces that traditional-local did not. This change closes those gaps so both built-in models can be compared through the same repository workflows.

  • Checkpoint parity

    • Add TraditionalLocalCheckpoint with JSON save/load support and round-trip validation.
    • Persist the traditional model’s learned state: token embeddings, output weights, output bias, vocabulary, context window, response settings, and seed.
    • Enable the same “train → save → reload → compare response” workflow already available to the BitNet model.
  • Inspection parity

    • Make TraditionalLocalHostedAgentModel implement IInspectableHostedAgentModel.
    • Expose GetTernaryWeightStats() from TraditionalLocalModel so the CLI can inspect both built-in models instead of treating traditional-local as unsupported.
    • Generalize inspection output from BitNet-specific wording to model-neutral Weight sign distribution.
  • Training report parity

    • Stop returning placeholder 0/0/0 weight counts from traditional-model training.
    • Populate TrainingReport with real negative/zero/positive counts derived from the trained traditional model state.
  • Comparison coverage

    • Add focused tests for:
      • traditional-model checkpoint round-trip preservation
      • traditional-model inspectable weight stats
      • non-placeholder weight distribution in training reports
      • checkpoint round-trip coverage for both built-in models in benchmark-oriented execution tests
  • Docs

    • Update usage/visualization docs to reflect that both built-in models now expose the same CLI inspection surface.
using var model = HostedAgentModelFactory.Create("traditional-local", VerbosityLevel.Quiet);

var stats = ((IInspectableHostedAgentModel)model).GetTernaryWeightStats();
// stats now reports real negative / zero / positive counts

TraditionalLocalCheckpoint.Save(((TraditionalLocalHostedAgentModel)model).Model, checkpointPath);
var reloaded = TraditionalLocalCheckpoint.Load(checkpointPath, VerbosityLevel.Quiet);

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI self-assigned this Mar 21, 2026
Co-authored-by: sharpninja <16146732+sharpninja@users.noreply.github.com>
Agent-Logs-Url: https://github.com/sharpninja/BitNet-b1.58-Sharp/sessions/0123abc0-11dc-4c81-a8ba-d7c1cc9de437
Copilot AI changed the title [WIP] Ensure traditional model has same features as bitnet model Bring traditional-local to feature parity for built-in model comparisons Mar 21, 2026
Copilot AI requested a review from sharpninja March 21, 2026 07:26
@sharpninja sharpninja marked this pull request as ready for review March 21, 2026 07:30
Copilot AI review requested due to automatic review settings March 21, 2026 07:30
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR brings the traditional-local model up to parity with the built-in BitNet model for repo workflows (benchmark comparisons, checkpoint round-tripping, and CLI inspection), so both models can be compared through the same training/visualization surfaces.

Changes:

  • Add JSON checkpoint save/load + round-trip validation for TraditionalLocalModel.
  • Expose traditional-local weight sign distribution via IInspectableHostedAgentModel and propagate real counts into TrainingReport.
  • Update CLI/UI strings + docs to use model-neutral “Weight sign distribution”, and add tests covering round-trips and inspection parity.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/BitNetSharp.Tests/TraditionalLocalCheckpointTests.cs Adds a checkpoint save/load round-trip test for TraditionalLocalModel.
tests/BitNetSharp.Tests/HostedAgentBenchmarksExecutionTests.cs Adds cross-model checkpoint round-trip execution coverage for both built-ins.
tests/BitNetSharp.Tests/BitNetModelTests.cs Verifies non-placeholder weight sign distribution and inspectable stats for traditional-local.
src/BitNetSharp.Core/TraditionalLocalModel.cs Implements weight sign stats + exports/imports of learned state; populates TrainingReport with real counts.
src/BitNetSharp.Core/TraditionalLocalCheckpoint.cs Introduces JSON checkpoint serialization + load and validation helpers for traditional-local.
src/BitNetSharp.Core/BitNetVisualizer.cs Generalizes histogram title to “Weight sign distribution”.
src/BitNetSharp.App/TraditionalLocalHostedAgentModel.cs Implements IInspectableHostedAgentModel for traditional-local.
src/BitNetSharp.App/Program.cs Generalizes CLI inspection messaging and histogram title for inspectable models.
docs/usage.md Updates usage text to reflect inspection parity across built-in models.
docs/training-and-visualization.md Updates visualization docs to reflect model-neutral signed weight histogram output.

@sharpninja sharpninja merged commit 8d38bb2 into main Mar 21, 2026
6 checks passed
@sharpninja sharpninja deleted the copilot/build-traditional-model-features branch March 21, 2026 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants