v1.2.0

[chonknick]

✨ Highlights

• New SemanticChunker with advance peak detection, skip_window support, group embedding support and better thresholding! The new semantic chunker requires much less tuning for consistently sized chunks.

# Import the chunker
from chonkie import SemanticChunker

# Initialize the chunker
chunker = SemanticChunker(chunk_size=512)

# define the text
text = ...

# CHONK!
chunk = chunker(text)

• Support for MongoDB via MongoDBHandshake!

from chonkie import MongoDBHandshake, SemanticChunker

# Initialize the handshake
handshake = MongoDBHandshake(uri="mongodb://localhost:8080", db_name="my_documents")

# Create some chunks
chunker = SemanticChunker()
chunks = chunker.chunk("Chonkie loves to chonk your texts!")

# Write chunks to MongoDB
handshake.write(chunks)

• Support for Pinecone via PineconeHandshake!

from chonkie import PineconeHandshake, SemanticChunker

# Initialize the handshake
handshake = PineconeHandshake(api_key=...)

# Create some chunks
chunker = SemanticChunker()
chunks = chunker.chunk("Chonkie loves to chonk your texts!")

# Write chunks to Pinecone
handshake.write(chunks)

• Support for Weaviate via WeaviateHandshake!

from chonkie import WeaviateHandshake, SemanticChunker

# Initialize the handshake
handshake = WeaviateHandshake(url=url, api_key=...)

# Create some chunks
chunker = SemanticChunker()
chunks = chunker.chunk("Chonkie loves to chonk your texts!")

# Write chunks to Weaviate
handshake.write(chunks)

What's Changed

• Fix: Update README — Absolute path for logo by @chonk-lain in Fix: Update README — Absolute path for logo #235
• Update README.md — Add recent integrations to the list by @chonknick in Update README.md — Add recent integrations to the list #237
• docs: fix url by @superpung in docs: fix url #240
• Support weaviate by @xucailiang in Support weaviate #204
• Fix: Remove failing weaviate cloud tests from CI/CD by @chonknick in Fix: Remove failing weaviate cloud tests from CI/CD #246
• Fix: Add _truncate to the OpenAIEmbddings (Bug: SDPM Chunker attempts to embed massive chunks & hits openAI error #239) by @chonknick in Fix: Add _truncate to the OpenAIEmbddings (#239) #247
• Feat: Add PineconeHandshake by @chonk-lain in Feat: Add PineconeHandshake #243
• docs: updated handshakes overview and move changelog to Python SDK by @sahibpreetsingh12 in docs: updated handshakes overview and move changelog to Python SDK #255
• Feat: Add MongoDBHandshake by @chonk-lain in Feat: Add MongoDBHandshake #253
• Feat: Add better SemanticChunker with peak finding, window similarities and more! by @chonknick in Feat: Add better SemanticChunker with peak finding, window similarities and more! #251
• Migrate the cookbook to chonkie-inc/cookbook by @chonknick in Migrate the cookbook to chonkie-inc/cookbook #258
• Docs: update mongodb docs by @chonk-lain in Docs: update mongodb docs #259

New Contributors

• @chonk-lain made their first contribution in Fix: Update README — Absolute path for logo #235
• @superpung made their first contribution in docs: fix url #240
• @xucailiang made their first contribution in Support weaviate #204
• @sahibpreetsingh12 made their first contribution in docs: updated handshakes overview and move changelog to Python SDK #255

Full Changelog: v1.1.2...v1.2.0

---

This discussion was created from the release v1.2.0.