diversify-text

This package helps you generate stylistically diverse paraphrases of your own texts using huggingface transformer models locally.

pip install diversify-text

Full documentation

Table of contents

• Usage
  • Single text
  • Control number of styles
  • Pick styles from the bank
  • Bring your own style examples
  • Repeats
  • Prompting method
  • Zero-shot method
  • Caching
  • Using the class directly
  • List of texts
  • Creating a custom method
• Install
• Contributing
  • Development setup
  • Running tests
  • Working with uv
  • Building docs locally
• Citation

Usage

For file inputs (CSV, TSV, TXT), output options, and punctuation splitting, see the full usage guide.

Single text

from diversify_text import diversify

results = diversify("The experiment was conducted in a controlled lab setting.")

[{
    "original": "The experiment was conducted in a controlled lab setting.",
    "paraphrases": [
        {"style": "informal", "text": "the experiment was in a controlled lab setting so it didnt suck..."},
        {"style": "obama", "text": "Well it was a controlled lab setting that the experiment was conducted in."},
        {"style": "question", "text": "Did you know that the experiment was conducted in a controlled lab setting? It was a re-test."},
        {"style": "formal", "text": "I heard the experiment was conducted in a controlled lab setting."},
        {"style": "song_lyrics", "text": "I mean, this experiment was conducted in a controlled lab setting, so that was a good thing."},
    ]
}]

Control number of styles

results = diversify("Some text.", n=3)

[{"original": "Some text.", "paraphrases": [
    {"style": "informal", "text": "..."},
    {"style": "obama", "text": "..."},
    {"style": "question", "text": "..."},
]}]

n is the number of distinct styles (default 5), drawn from the built-in style bank in order — one paraphrase per style. Requesting more styles than the bank contains raises an error; you never silently get the same style twice.

Pick styles from the bank

Select specific built-in styles with styles, by name and/or by (0-based) bank index:

results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    styles=["recipe", "personal_blog"],
)

# indices work too — handy for trying things without knowing the names
results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    styles=[0, 7, "recipe"],
)

Unknown names and out-of-range indices raise an error listing what is available. Note that indices follow bank order, which may change between releases as the bank is curated — names are the stable way to pin a style.

Bring your own style examples

Pass style_texts to define target styles with your own texts. A flat list is one style; a list of lists is several styles; a dict maps style names to example sets:

# one style, defined by its example texts
results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    style_texts=[
        "We found something really interesting — check this out!",
        "You won't believe how well this worked!",
    ],
)

# several styles, named
results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    style_texts={
        "academic": [
            "The results demonstrate a statistically significant effect.",
            "Participants were randomly assigned to one of two conditions.",
        ],
        "enthusiastic": [
            "We found something really interesting — check this out!",
            "You won't believe how well this worked!",
        ],
    },
)

[{
    "original": "The experiment was conducted in a controlled lab setting.",
    "paraphrases": [
        {"style": "academic", "text": "The experiment was carried out under controlled laboratory conditions."},
        {"style": "enthusiastic", "text": "Guess what — we ran the whole experiment in a controlled lab, how cool is that!"},
    ]
}]

styles and style_texts can be combined in one call (bank styles come first in the output). n cannot be combined with either — the number of styles is already determined, so passing n raises an error.

Repeats

repeats controls how many paraphrases are generated per style (default 1). With more than one repeat, the output interleaves the styles:

results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    styles=["recipe", "personal_blog"],
    repeats=2,
)

# styles interleave: recipe, personal_blog, recipe, personal_blog
[{
    "original": "The experiment was conducted in a controlled lab setting.",
    "paraphrases": [
        {"style": "recipe", "text": "..."},
        {"style": "personal_blog", "text": "..."},
        {"style": "recipe", "text": "..."},
        {"style": "personal_blog", "text": "..."},
    ]
}]

Prompting method

The default style transfer method is TinyStyler. Alternatively, use the prompting method, which generates paraphrases via a causal language model (default: SmolLM3-3B) with the style examples inserted into a few-shot style transfer prompt:

results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    method="prompting",
    style_texts={
        "academic": [
            "The results demonstrate a statistically significant effect.",
            "Participants were randomly assigned to one of two conditions.",
        ],
    },
)

Only prompts that take style example texts are supported — every method receives the same input and produces the same output.

Zero-shot method

The zero_shot method defines styles by rewrite instructions instead of example texts. It has its own style bank of instruction styles:

results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    method="zero_shot",
    styles=["formal", "caps"],
)

With this method, style_texts are instructions — exactly one per style. An instruction can place the input text itself with [DOCUMENT SEGMENT]; otherwise the text is appended at the end:

results = diversify(
    "The experiment was conducted in a controlled lab setting.",
    method="zero_shot",
    style_texts={"pirate": ["Rewrite the text as an old-timey pirate would say it."]},
)

Caching

The diversify() function automatically caches loaded models between calls. The generation model and the semantic filter are cached independently, so toggling semantic_filter does not reload the generation model and vice versa. Call clear_cache() to drop cached models and allow memory to be reclaimed when possible:

from diversify_text import clear_cache

clear_cache()

Using the class directly

You can also instantiate a Diversifier yourself for full control over the model lifecycle:

from diversify_text import Diversifier

div = Diversifier(device="cuda", method="tinystyler")

batch_1 = div.diversify(texts_1, styles=["recipe", "personal_blog"])
batch_2 = div.diversify(texts_2, style_texts=my_examples)

List of texts

results = diversify([
    "The experiment was conducted in a controlled lab setting.",
    "She graduated from MIT in 2019.",
])

[
    {"original": "The experiment ...", "paraphrases": [{"style": "informal", "text": "..."}, ...]},
    {"original": "She graduated ...", "paraphrases": [{"style": "informal", "text": "..."}, ...]},
]

Creating a custom method

from diversify_text import Diversifier
from diversify_text.method import DiversificationMethod


class MyMethod(DiversificationMethod):
    name = "my_method"

    def generate(self, texts, style_dict, *, max_new_tokens, temperature, top_p, **kwargs):
        # style_dict maps each target style name to its example texts,
        # e.g. {"recipe": ["Cut a peeled brown onion...", ...]}.
        # It is resolved by the core from the caller's `styles` / `style_texts`.
        return [[f"{text} :: {name}" for name in style_dict] for text in texts]


results = Diversifier(method=MyMethod()).diversify("Hello", styles=["recipe", "personal_blog"])

[{"original": "Hello", "paraphrases": [
    {"style": "recipe", "text": "Hello :: recipe"},
    {"style": "personal_blog", "text": "Hello :: personal_blog"},
]}]

A method returns list[list[str]] — for each input text, one generated string per style in style_dict order. The core attaches the style labels to the output and runs the method once per repeat, so custom methods stay simple and stateless.

Install

pip install diversify-text

Requires Python 3.10+.

Contributing

Development setup

Note

You must have uv installed. Full installation guide: https://docs.astral.sh/uv/getting-started/installation/

git clone https://github.com/AnnaWegmann/diversify_text.git
cd diversify_text
uv sync --group dev
source .venv/bin/activate

Running tests

# Run all tests
pytest

# Run a specific test file
pytest tests/test_core.py

# Run a specific test class or method
pytest tests/test_core.py::TestDiversifier
pytest tests/test_core.py::TestDiversifier::test_single_text_returns_one_result

Tests are also individually runnable via PyCharm's built-in test runner (right-click any test class or method).

Working with uv

Adding packages with uv add

To add packages to your project, always use uv add rather than uv pip install. This ensures that your dependencies are properly managed and recorded in your pyproject.toml.

uv add <package-name>

Adding packages to the dev group

If you need to add a package specifically for your development environment:

uv add --group dev <package-name>

Switching between dev and standard mode

After you are done with testing and want to go back to standard mode, you can remove the dev-only packages:

uv sync --no-group dev

This will disable all additional groups and just load your main project dependencies.

Best practice: run uv lock -U

Whenever you upgrade, downgrade, or change versions of packages, it's good practice to run:

uv lock -U

This updates your lock file to ensure all versions are consistent and everything is in sync.

Building docs locally

uv sync --group docs
sphinx-build -b html docs docs/_build/html
open docs/_build/html/index.html

Citation

If you use diversify in your research, we are happy about a citation (placeholder currently).

@inproceedings{wegmann2026diversify,
    title = {diversify_text: An Amazing Library for Text Diversification},
    author = {Wegmann, Anna and Others},
    url={https://github.com/AnnaWegmann/diversify_text},
    year = {2026},
}