feat: Implement Ghost Reader (v1.2.0)

CHANGELOG.md

@@ -5,6 +5,14 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.0] - 2026-01-11
+
+### Added
+- **Ghost Reader**: A zero-copy, zero-allocation `ref struct` view over binary data for ultra-low latency access.
+- **Convenience Methods**: `ComputeSize()` and `ToBytes()` extension methods for generated Ghost types.
+- **Documentation**: New `docs/GHOST_READER.md` dedicated guide.
+- **Reliability**: Robust handling for nullable strings and buffer overflows in `RappWriter`.
+
 ## [1.1.0] - 2026-01-07
 
 ### Added

README.md

@@ -252,6 +252,22 @@ Rapp is **100% compatible with Native AOT** (Ahead-of-Time) compilation, making
 
 ## 🎯 Advanced Features
 
+### 👻 The Ghost Reader (Zero-Copy View)
+
+Rapp includes a "Ghost Reader" feature for ultra-low latency scenarios (trading, gaming, IoT). Instead of deserializing objects on the heap, it creates a **Zero-Copy, Zero-Allocation** `ref struct` view over your raw binary data.
+
+```csharp
+// 1. Define
+[RappGhost]
+public class Tick { public double Price; public long Time; }
+
+// 2. Read (Zero Allocation)
+var ghost = new TickGhost(buffer);
+if (ghost.Price > 100.0) { ... } // instant access
+```
+
+[👉 Read the full Ghost Reader Documentation](docs/GHOST_READER.md)
+
 ### Integration with Existing Code
 
 Rapp works with your existing `HybridCache` setup - no code changes required:

docs/GHOST_READER.md

@@ -0,0 +1,100 @@
+# The Rapp Ghost Reader 👻
+
+The **Ghost Reader** is a high-performance, Zero-Copy view over your raw data. It allows you to access properties of your objects directly from a binary buffer without deserializing the entire object or allocating a single byte of memory on the heap.
+
+This pattern is ideal for high-throughput scenarios (gaming, trading, IoT) where garbage collection (GC) pauses are unacceptable.
+
+## 🚀 Key Benefits
+
+| Feature | Traditional (JSON/Protobuf) | Rapp Ghost Reader |
+| :--- | :--- | :--- |
+| **Allocation** | Creates new objects on Heap | **Zero Allocation** (Stack only) |
+| **Parsing** | Parses full object on read | **Lazy / On-Demand** (O(1) access) |
+| **GC Pressure** | High (creates garbage) | **None** (Invisible to GC) |
+| **Speed** | Slow (Reflection + Allocations) | **Instant** (Pointer arithmetic) |
+
+## 📦 How to Use
+
+### 1. Opt-in with `[RappGhost]`
+
+Mark your class or struct with the attribute. Rapp supports primitives (`int`, `bool`, `double`, etc.) and `string` (variable length).
+
+```csharp
+using Rapp;
+
+[RappGhost]
+public class Player
+{
+    public int Id { get; set; }
+    public string Name { get; set; }
+    public bool IsActive { get; set; }
+    public double Health { get; set; }
+}
+```
+
+### 2. Write Data (Serialization)
+
+Use the generated `.WriteTo()` extension method to serialize your object to a span of bytes.
+
+```csharp
+var player = new Player { Id = 1, Name = "Viking", IsActive = true, Health = 100.0 };
+Span<byte> buffer = stackalloc byte[128]; // Stack allocate for speed!
+
+int bytesWritten = player.WriteTo(buffer);
+```
+
+### 3. Read Data (The Ghost)
+
+Wrap the buffer with the generated `*Ghost` struct. This looks and feels like your object, but it's just a view over the bytes.
+
+```csharp
+// Create the view (Zero allocation)
+var ghost = new PlayerGhost(buffer.Slice(0, bytesWritten));
+
+// READ properties instantly
+if (ghost.IsActive) // Reads byte at offset X
+{
+    Console.WriteLine($"Player {ghost.Id} has {ghost.Health} health.");
+}
+
+// READ Strings
+// Returns ReadOnlySpan<byte> to avoid allocation
+ReadOnlySpan<byte> nameBytes = ghost.Name; 
+
+// Or convert to string if you really need it (allocates)
+Console.WriteLine(ghost.NameString); 
+```
+
+### 🎁 Convenience Methods
+
+If you prefer ease of use over raw stack allocation, Rapp generates helper methods for you:
+
+```csharp
+var player = new Player { Name = "Convenient" };
+
+// 1. Compute Exact Size
+int size = player.ComputeSize();
+
+// 2. Direct to Byte Array (Allocates, but easier)
+byte[] data = player.ToBytes();
+```
+
+## 🧠 How It Works (The "Head/Tail" Layout)
+
+Rapp uses a split binary layout to guarantee O(1) random access for fixed fields while supporting variable-length data (strings).
+
+```text
+[ ID (4b) ] [ Health (8b) ] [ Name Pointer (2b) ] ... [ Name Length (2b) ] [ "Viking" (bytes) ]
+<---------------- HEAD (Fixed) ----------------->     <------------ TAIL (Variable) ---------->
+```
+
+1.  **Head Region**: Contains all fixed-size data (`int`, `double`, `bool`) and *pointers* (offsets) to variable data.
+    *   This allows us to know exactly where `Id` is (Offset 0) without parsing anything else.
+2.  **Tail Region**: Contains the variable-length data (strings).
+    *   The "Pointer" in the Head tells the Ghost Reader exactly where to jump in the generic buffer to find the string.
+
+## ⚠️ Constraints & Best Practices
+
+1.  **Max String Size**: The current implementation uses `ushort` for pointers and lengths, limiting individual string lengths and the total buffer size to **64KB**. This is optimized for network packets (UDP/TCP).
+2.  **Buffer Safety**: The Ghost Reader assumes the underlying buffer is valid for the lifetime of the struct. Since it is a `ref struct`, the compiler enforces that it cannot escape the stack frame of the buffer, providing memory safety.
+3.  **Schema Evolution**: The Ghost Reader generates code based on *current* class definitions. If you change the class, you must recompile. It does not currently support backward/forward compatibility (schema versioning) like the main Rapp serializer.

src/Rapp.Gen/RappGhostGenerator.cs

@@ -0,0 +1,257 @@
+using System;
+using System.Text;
+using System.Linq;
+using System.Collections.Generic;
+using Microsoft.CodeAnalysis;
+using Microsoft.CodeAnalysis.CSharp.Syntax;
+using Microsoft.CodeAnalysis.Text;
+
+namespace Rapp.Gen
+{
+    [Generator]
+    public class RappGhostGenerator : IIncrementalGenerator
+    {
+        public void Initialize(IncrementalGeneratorInitializationContext context)
+        {
+            var p = context.SyntaxProvider.CreateSyntaxProvider(
+                predicate: (s, _) => IsTarget(s),
+                transform: (ctx, _) => GetTarget(ctx))
+                .Where(m => m != null);
+
+            context.RegisterSourceOutput(p, Execute);
+        }
+
+        private static bool IsTarget(SyntaxNode node)
+        {
+            return node is ClassDeclarationSyntax c && c.AttributeLists.Count > 0;
+        }
+
+        private static ClassDeclarationSyntax? GetTarget(GeneratorSyntaxContext context)
+        {
+            var classDeclaration = (ClassDeclarationSyntax)context.Node;
+
+            foreach (var attributeList in classDeclaration.AttributeLists)
+            {
+                foreach (var attribute in attributeList.Attributes)
+                {
+                    if (context.SemanticModel.GetSymbolInfo(attribute).Symbol is IMethodSymbol attributeSymbol &&
+                        attributeSymbol.ContainingType.ToDisplayString() == "Rapp.RappGhostAttribute")
+                    {
+                        return classDeclaration;
+                    }
+                }
+            }
+            return null;
+        }
+
+        private static void Execute(SourceProductionContext context, ClassDeclarationSyntax? classDecl)
+        {
+            if (classDecl is null) return;
+
+            var className = classDecl.Identifier.Text;
+            var ghostName = $"{className}Ghost";
+            var sb = new StringBuilder();
+            sb.AppendLine("// <auto-generated/>");
+
+            string ns = GetNamespace(classDecl);
+
+            sb.AppendLine("using System;");
+            sb.AppendLine("using System.Buffers.Binary;");
+            sb.AppendLine("using System.Runtime.InteropServices;");
+            sb.AppendLine("using Rapp;"); // For RappWriter
+            sb.AppendLine();
+            sb.AppendLine($"namespace {ns}");
+            sb.AppendLine("{");
+
+            // --- GHOST STRUCT ---
+            sb.AppendLine($"    public ref struct {ghostName}");
+            sb.AppendLine("    {");
+            sb.AppendLine("        private readonly ReadOnlySpan<byte> _buffer;");
+            sb.AppendLine();
+            sb.AppendLine($"        public {ghostName}(ReadOnlySpan<byte> buffer)");
+            sb.AppendLine("        {");
+            sb.AppendLine("            _buffer = buffer;");
+            sb.AppendLine("        }");
+            sb.AppendLine();
+
+            int currentOffset = 0;
+            var properties = new List<(string Name, string Type, int Offset, bool IsVar)>();
+
+            foreach (var member in classDecl.Members)
+            {
+                if (member is PropertyDeclarationSyntax prop)
+                {
+                    string propName = prop.Identifier.Text;
+                    string propType = prop.Type.ToString(); // "string", "string?", "int", etc.
+
+                    // Normalize type for lookup
+                    string normalizedType = propType.TrimEnd('?');
+
+                    int size = GetSize(normalizedType);
+                    bool isString = normalizedType == "string";
+
+                    properties.Add((propName, propType, currentOffset, isString));
+
+                    // Increment offset for HEAD calculation
+                    currentOffset += size;
+                }
+            }
+
+            // Generate Getters
+            foreach (var p in properties)
+            {
+                string normalizedType = p.Type.TrimEnd('?');
+                string? getter = GenerateGetter(normalizedType, p.Offset);
+
+                if (p.IsVar) // string or string? -> ReadOnlySpan<byte>
+                {
+                    // Special handling for string property in Ghost
+                    sb.AppendLine($"        public ReadOnlySpan<byte> {p.Name}");
+                    sb.AppendLine(getter);
+
+                    // Helper for string
+                    sb.AppendLine($"        public string {p.Name}String => System.Text.Encoding.UTF8.GetString({p.Name});");
+                }
+                else if (getter != null)
+                {
+                    sb.AppendLine($"        public {normalizedType} {p.Name} => {getter};");
+                }
+            }
+
+            sb.AppendLine("    }"); // End struct
+
+            // --- WRITER EXTENSIONS ---
+            sb.AppendLine();
+            sb.AppendLine($"    public static class {className}RappExtensions");
+            sb.AppendLine("    {");
+            sb.AppendLine($"        public static int WriteTo(this {className} value, Span<byte> buffer)");
+            sb.AppendLine("        {");
+            sb.AppendLine($"            var writer = new RappWriter(buffer, headSize: {currentOffset});");
+
+            foreach (var p in properties)
+            {
+                string normalizedType = p.Type.TrimEnd('?');
+                GenerateWriteCall(sb, p.Name, normalizedType);
+            }
+
+            sb.AppendLine("            return writer.Length;");
+            sb.AppendLine("        }");
+            sb.AppendLine();
+
+            // --- ComputeSize ---
+            sb.AppendLine($"        public static int ComputeSize(this {className} value)");
+            sb.AppendLine("        {");
+            sb.AppendLine($"            int size = {currentOffset}; // Head Size");
+            foreach (var p in properties)
+            {
+                 // Normalized type is needed for logic check, though GetByteCount handles string?
+                if (p.IsVar)
+                {
+                    sb.AppendLine($"            if (value.{p.Name} != null)");
+                    sb.AppendLine($"            {{");
+                    sb.AppendLine($"                size += 2 + System.Text.Encoding.UTF8.GetByteCount(value.{p.Name});");
+                    sb.AppendLine($"            }}");
+                }
+            }
+            sb.AppendLine("            return size;");
+            sb.AppendLine("        }");
+            sb.AppendLine();
+
+            // --- ToBytes ---
+            sb.AppendLine($"        public static byte[] ToBytes(this {className} value)");
+            sb.AppendLine("        {");
+            sb.AppendLine("            int size = value.ComputeSize();");
+            sb.AppendLine("            byte[] buffer = new byte[size];");
+            sb.AppendLine("            value.WriteTo(buffer);");
+            sb.AppendLine("            return buffer;");
+            sb.AppendLine("        }");
+
+            sb.AppendLine("    }");
+
+            sb.AppendLine("}"); // End namespace
+
+            context.AddSource($"{ghostName}.g.cs", SourceText.From(sb.ToString(), Encoding.UTF8));
+        }
+
+        private static int GetSize(string type)
+        {
+            return type switch
+            {
+                "int" => 4,
+                "uint" => 4,
+                "long" => 8,
+                "ulong" => 8,
+                "short" => 2,
+                "ushort" => 2,
+                "byte" => 1,
+                "bool" => 1,
+                "double" => 8,
+                "float" => 4,
+                "string" => 2, // Pointer size (ushort)
+                _ => 0,
+            };
+        }
+
+        private static string? GenerateGetter(string type, int offset)
+        {
+            if (type == "string")
+            {
+                return $@"
+        {{
+            get
+            {{
+                ushort ptr = BinaryPrimitives.ReadUInt16LittleEndian(_buffer.Slice({offset}));
+                if (ptr == 0) return ReadOnlySpan<byte>.Empty;
+                var tail = _buffer.Slice(ptr);
+                ushort length = BinaryPrimitives.ReadUInt16LittleEndian(tail);
+                return tail.Slice(2, length);
+            }}
+        }}";
+            }
+            // ... (rest of implementation)
+
+            return type switch
+            {
+                "int" => $"BinaryPrimitives.ReadInt32LittleEndian(_buffer.Slice({offset}))",
+                "uint" => $"BinaryPrimitives.ReadUInt32LittleEndian(_buffer.Slice({offset}))",
+                "long" => $"BinaryPrimitives.ReadInt64LittleEndian(_buffer.Slice({offset}))",
+                "ulong" => $"BinaryPrimitives.ReadUInt64LittleEndian(_buffer.Slice({offset}))",
+                "short" => $"BinaryPrimitives.ReadInt16LittleEndian(_buffer.Slice({offset}))",
+                "ushort" => $"BinaryPrimitives.ReadUInt16LittleEndian(_buffer.Slice({offset}))",
+                "bool" => $"_buffer[{offset}] != 0",
+                "byte" => $"_buffer[{offset}]",
+                "double" => $"BitConverter.Int64BitsToDouble(BinaryPrimitives.ReadInt64LittleEndian(_buffer.Slice({offset})))", 
+                _ => null
+            };
+        }
+
+        private static void GenerateWriteCall(StringBuilder sb, string name, string type)
+        {
+            string call = type switch
+            {
+                "int" => $"writer.WriteInt32(value.{name});",
+                "long" => $"writer.WriteInt64(value.{name});",
+                "short" => $"writer.WriteInt16(value.{name});",
+                "byte" => $"writer.WriteByte(value.{name});",
+                "bool" => $"writer.WriteBool(value.{name});",
+                "double" => $"writer.WriteDouble(value.{name});", 
+                "float" => $"writer.WriteSingle(value.{name});",
+                "string" => $"writer.WriteString(value.{name});",
+                _ => $"// Unsupported type: {type}"
+            };
+            sb.AppendLine($"            {call}");
+        }
+
+        private static string GetNamespace(SyntaxNode node)
+        {
+            var curr = node.Parent;
+            while (curr != null)
+            {
+                if (curr is NamespaceDeclarationSyntax ns) return ns.Name.ToString();
+                if (curr is FileScopedNamespaceDeclarationSyntax fns) return fns.Name.ToString();
+                curr = curr.Parent;
+            }
+            return "Rapp.Generated";
+        }
+    }
+}