Skip to content

[v2.0] Phase 3: Python wrapper with graceful fallback #25

Description

@fsecada01

Implement TextSpitter/splitters.py — thin Python classes that use the Rust backend when available and fall back to pure Python transparently.

Wiki: Phase 3 detail
Branch: feature/rust-backend

Tasks

  • 3.1 Implement _RUST_AVAILABLE import guard (try: from text_spitter_rust import ...; except ImportError)
  • 3.2 Implement CharacterTextSplitter Python class with split_text / split_texts routing to Rust or Python backend
  • 3.3 Implement _split_text_pythonfully working pure-Python fallback (not a stub); must produce identical output to Rust implementation
  • 3.4 Expose use_rust=False parameter for forcing pure-Python mode (debugging + parity testing)

Acceptance criteria

  • CharacterTextSplitter(use_rust=True).split_text(text) == CharacterTextSplitter(use_rust=False).split_text(text) for all inputs
  • If text_spitter_rust is not installed, the class works silently via Python fallback

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustRust implementation worktrackingParent tracking issue with sub-tasksv2.0TextSpitter v2.0 Rust backend

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions