mistral small#3

Open

sumo43 wants to merge 29 commits intodev-updatedfrom

Commits on Jul 21, 2025

add new attn
Artem Yatsenko
committed

Commits on Jul 23, 2025

fix imports
Artem Yatsenko
committed

Commits on Jul 26, 2025

jul 24 changes
Artem Yatsenko
committed
works but ooms?
sumo43
committed
temp remove vision encoder
sumo43
committed
final
sumo43
committed

Commits on Jul 29, 2025

Merge branch 'dev-updated' into mistral-small
teknium1
authored

Commits on Jul 31, 2025

add mistral conversion
Artem Yatsenko
committed
fix checkpoint loading, add finetuning script mirroring qwen
Artem Yatsenko
committed
cleanup of mistral3 code
Artem Yatsenko
committed

Commits on Aug 1, 2025

pt 1 multimodal sample packing preprocessing func
Artem Yatsenko
committed

Commits on Aug 4, 2025

add packing with images
Artem Yatsenko
committed

Commits on Aug 5, 2025

hf preprocess instead of mistral
Artem Yatsenko
committed

Commits on Aug 14, 2025

add back vision encoder, make change to preprocess (we now only keep image filepaths)
Artem Yatsenko
committed
add multimodal packed ds, update to conversion script
Artem Yatsenko
committed
add instructions, set better default configs
Artem Yatsenko
committed
limit for testing preproc multimodal
Artem Yatsenko
committed
update readme
Artem Yatsenko
committed
add interleaved packed ds
Artem Yatsenko
committed

Commits on Aug 15, 2025

add interleaved text-image and textonly preprocess script & functionality
Artem Yatsenko
committed

Commits on Aug 18, 2025

bugfix: limit keyword in multimodal data prprocess script
Artem Yatsenko
committed

Commits on Aug 20, 2025

add conversion script back for mistral
Artem Yatsenko
committed

Commits on Aug 21, 2025

fix freqs_cis bug. add more configs
Artem Yatsenko
committed
update gitignore
Artem Yatsenko
committed
small changes to scripts
Artem Yatsenko
committed
remove junk and update gitignore
Artem Yatsenko
committed
fix VLM embedding with TP
Artem Yatsenko
committed

Commits on Aug 25, 2025

temp fix for vision encoder loading in TP context
Artem Yatsenko
committed

Commits on Aug 27, 2025

nvidia VLM dataset support
Artem Yatsenko
committed