TOON: Token-Oriented Object Notation

TOON (Token-Oriented Object Notation) is a lightweight, human-readable data serialization format designed to be token-efficient for Large Language Models (LLMs) while remaining easy for humans to read and write.

It serves as a concise alternative to JSON, removing syntactic noise (like excessive quotes, braces, and brackets) to reduce token usage and improve clarity.

🚀 Why TOON?

1. Token Efficiency for AI/LLMs

JSON is verbose. For LLMs (like GPT-4, Claude, Gemini), every character counts. TOON reduces the token footprint by eliminating structural overhead, which can lead to:

• Lower Costs: Fewer tokens processed means lower API bills.
• Larger Context: Fit more data into the model's context window.
• Faster Generation: Less syntax for the model to generate.

2. Human Readability

TOON looks like a clean configuration file or a summary report. It uses significant whitespace and minimal punctuation, making it ideal for:

• Logs and debug output.
• Configuration files.
• Data summaries for dashboards.

📦 Installation

Add this line to your application's Gemfile:

gem 'toonify'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install toonify

💻 Usage

The toonify gem provides a simple API to convert between JSON and TOON.

Basic Conversion

require 'toonify'

# Input JSON
json_data = '{"name": "Alice", "role": "Engineer", "active": true}'

# Encode JSON -> TOON
toon_output = Toonify::Toon.encode(json_data)
puts toon_output
# Output:
# name: Alice
# role: Engineer
# active: true

# Decode TOON -> JSON
json_output = Toonify::Toon.decode(toon_output)
puts json_output
# Output: {"name":"Alice","role":"Engineer","active":true}

Handling Complex Data

TOON shines with nested structures and arrays. It automatically detects tabular data and formats it concisely.

Error Handling

The converter is strict about input types to ensure reliability.

begin
  Toonify::Toon.encode('invalid json')
rescue ArgumentError => e
  puts e.message # => "Invalid JSON input"
end

🔍 Format Specification

TOON uses a few simple rules:

• Key-Value: key: value
• Nested Objects: Indented blocks (YAML-style).
• Primitive Arrays: key[count]: val1,val2,val3
• Object Arrays (Tabular): key[count]{headers}: followed by CSV-like rows.

🤝 Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ran010/toonify.

📄 License

The gem is available as open source under the terms of the MIT License.