Skip to content

Latest commit

 

History

7 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

Usage

Shortest command

Run python extractor.py [family].

Deep extraction

Run python extractor.py [family] --deep.

Change output directory

By default all extracted families will be saved to a folder named "extracted" by default unless overridden by -o/--output.

Run python extractor.py [family] -o [output_dir]

Dont copy licenses

Use --no-licenses to skip copying licenses.

It's not recommended to skip copying licenses if you plan to open-source or publish your model in the future.

Overwrite existing outputs

This is useful for if you modified files and cannot revert to the originals.

WARINING overwriting directories is IRREVERSIBLE.

Use --force if you AFTER backing up any files you wish to keep.


Argument descriptions

Argument Description Default
family The model family to extract from the installed transformers package. Examples: llama, qwen2, bert. None
-o, --output Sets the output directory where the extracted package will be created. A folder named after the selected model family is created inside this directory. ./extracted
--deep Also extracts other referenced model families that the selected model depends on. Useful when a model implementation imports code from another model family. Disabled by default
--include-convert Includes weight conversion scripts (convert_*_to_hf.py) if they exist for the selected model family. Disabled by default
--no-licenses Skips generating LICENSES.md. License information and per-file copyright headers are collected during extraction.
--list [FILTER] Lists available model families and exits. An optional filter can be provided to show only matching families. No filter by default
--force Overwrites the existing extracted model family folder if it already exists. Disabled by default
-h, --help Displays the help message and available command-line options. No default state

About

Quickly extract a family from the huggingface transformers python library for easily applying your own edits!

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages