Skip to content

[00115] Remove Small and Medium benchmark categories #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rorychatt merged 1 commit into from
Apr 12, 2026
Merged

[00115] Remove Small and Medium benchmark categories #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions BENCHMARKS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ Raw JSON string in, parsed patch out — the real-world scenario.

| Category | NativeJsonDiff | JsonDiffPatch.Net | SystemTextJson.JsonDiffPatch | JsonPatch.Net |
|----------|---------------|-------------------|------------------------------|---------------|
| Small | **2.90 us / 488 B** | 2.82 us / 15.2 KB (0.97x) | 2.68 us / 5.9 KB (0.92x) | 4.91 us / 6.7 KB (1.69x) |
| Medium | **39.21 us / 2.7 KB** | 26.15 us / 100.6 KB (0.67x) | 26.44 us / 53.6 KB (0.67x) | 20.25 us / 62.7 KB (0.52x) |
| Large | **487.54 us / 26.5 KB** | 501.18 us / 1.1 MB (1.03x) | 286.07 us / 678.4 KB (0.59x) | 218.51 us / 854.1 KB (0.45x) |
| DeepNested | **4.27 us / 680 B** | 6.43 us / 36.1 KB (1.51x) | 5.58 us / 15.9 KB (1.31x) | 13.18 us / 15.7 KB (3.09x) |
| ArrayHeavy | **164.73 us / 114.7 KB** | 217.00 us / 795.9 KB (1.32x) | 1,976.87 us / 2.8 MB (12.00x) | 105.99 us / 339.1 KB (0.64x) |
Expand All @@ -24,8 +22,6 @@ Competitors get pre-parsed DOM objects; NativeJsonDiff still takes raw bytes.

| Category | NativeJsonDiff | JsonDiffPatch.Net | SystemTextJson.JsonDiffPatch | JsonPatch.Net |
|----------|---------------|-------------------|------------------------------|---------------|
| Small | **2.57 us / 488 B** | 476 ns / 1.5 KB (0.19x) | 1.23 us / 1.0 KB (0.48x) | 780 ns / 1.9 KB (0.30x) |
| Medium | **35.25 us / 2.7 KB** | 4.43 us / 9.1 KB (0.13x) | 13.15 us / 7.5 KB (0.37x) | 7.62 us / 16.6 KB (0.22x) |
| Large | **463.25 us / 26.5 KB** | 60.38 us / 86.0 KB (0.13x) | 153.13 us / 144.4 KB (0.33x) | 96.31 us / 320.1 KB (0.21x) |
| DeepNested | **4.30 us / 680 B** | 1.98 us / 9.6 KB (0.46x) | 2.28 us / 4.7 KB (0.53x) | 1.53 us / 5.0 KB (0.35x) |
| ArrayHeavy | **169.33 us / 114.7 KB** | 89.47 us / 378.6 KB (0.53x) | 2,016.13 us / 2.6 MB (11.91x) | 45.63 us / 145.9 KB (0.27x) |
Expand All @@ -40,8 +36,6 @@ Apply a pre-computed patch to a document.

| Category | NativeJsonDiff | JsonDiffPatch.Net | SystemTextJson.JsonDiffPatch | JsonPatch.Net |
|----------|---------------|-------------------|------------------------------|---------------|
| Small | **2.63 us / 1.0 KB** | 1.63 us / 7.8 KB (0.71x) | 341 ns / 1.9 KB (0.15x) | 382 ns / 2.2 KB (0.17x) |
| Medium | **19.14 us / 8.4 KB** | 15.44 us / 70.7 KB (0.81x) | 3.52 us / 19.1 KB (0.18x) | 10.07 us / 20.2 KB (0.53x) |
| Large | **256.75 us / 191.2 KB** | 141.11 us / 692.0 KB (0.55x) | 35.48 us / 190.5 KB (0.14x) | 456.89 us / 200.1 KB (1.78x) |
| DeepNested | **3.90 us / 1.8 KB** | 12.19 us / 65.9 KB (3.13x) | 881 ns / 4.3 KB (0.23x) | 1.42 us / 5.7 KB (0.36x) |
| ArrayHeavy | **218.24 us / 31.8 KB** | 175.55 us / 753.5 KB (0.80x) | 6.67 us / 28.2 KB (0.03x) | 62.74 us / 185.8 KB (0.29x) |
Expand Down
6 changes: 1 addition & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,12 @@ Most .NET JSON diff libraries run entirely in managed code — parsing JSON tree

## Performance

NativeJsonDiff is benchmarked against every major .NET JSON diff library using [BenchmarkDotNet](https://benchmarkdotnet.org/) across 9 payload categories (Small, Medium, Large, DeepNested, ArrayHeavy, ApiResponse, ConfigFile, LargeStateTree, VeryLargeStateTree — up to 2MB JSON trees).
NativeJsonDiff is benchmarked against every major .NET JSON diff library using [BenchmarkDotNet](https://benchmarkdotnet.org/) across 8 payload categories (Large, DeepNested, ArrayHeavy, ApiResponse, ConfigFile, LargeStateTree, VeryLargeStateTree, WidgetTree — up to 2MB JSON trees).

### End-to-End Diff (raw JSON in, patch out)

| Category | NativeJsonDiff | JsonDiffPatch.Net | SystemTextJson.JsonDiffPatch | JsonPatch.Net |
|----------|---------------|-------------------|------------------------------|---------------|
| Small | **2.90 us / 488 B** | 2.82 us / 15.2 KB | 2.68 us / 5.9 KB | 4.91 us / 6.7 KB |
| Medium | **39.21 us / 2.7 KB** | 26.15 us / 100.6 KB | 26.44 us / 53.6 KB | 20.25 us / 62.7 KB |
| Large | **487.54 us / 26.5 KB** | 501.18 us / 1.1 MB | 286.07 us / 678.4 KB | 218.51 us / 854.1 KB |
| LargeStateTree | **717.72 us / 40.5 KB** | 1,766.34 us / 4.3 MB (2.5x) | 861.40 us / 1.7 MB (1.2x) | 852.54 us / 2.0 MB (1.2x) |
| VeryLargeStateTree | **18.39 ms / 121.8 KB** | 136.29 ms / 83.9 MB (7.4x) | 44.98 ms / 31.8 MB (2.4x) | 53.44 ms / 61.9 MB (2.9x) |
Expand All @@ -38,8 +36,6 @@ NativeJsonDiff is benchmarked against every major .NET JSON diff library using [

| Category | NativeJsonDiff | JsonDiffPatch.Net | SystemTextJson.JsonDiffPatch | JsonPatch.Net |
|----------|---------------|-------------------|------------------------------|---------------|
| Small | **2.63 us / 1.0 KB** | 1.63 us / 7.8 KB | 341 ns / 1.9 KB | 382 ns / 2.2 KB |
| Medium | **19.14 us / 8.4 KB** | 15.44 us / 70.7 KB | 3.52 us / 19.1 KB | 10.07 us / 20.2 KB |
| Large | **256.75 us / 191.2 KB** | 141.11 us / 692.0 KB | 35.48 us / 190.5 KB | 456.89 us / 200.1 KB |
| LargeStateTree | **488.27 us / 263.0 KB** | 1,862.22 us / 4.6 MB (3.8x) | 30.36 us / 140.6 KB | 191.85 us / 177.2 KB |
| VeryLargeStateTree | **20.17 ms / 6.9 MB** | 291.13 ms / 197.4 MB (16.3x) | 1.66 ms / 4.7 MB | 311.73 us / 507.3 KB |
Expand Down
2 changes: 1 addition & 1 deletion src/Ivy.NativeJsonDiff.Benchmarks/Benchmarks/EndToEndDiffBenchmarks.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ namespace Ivy.NativeJsonDiff.Benchmarks.Benchmarks;
[Config(typeof(CiBenchmarkConfig))]
public class EndToEndDiffBenchmarks
{
[Params("Small", "Medium", "Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree")]
[Params("Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree")]
public string Category { get; set; } = null!;

private byte[] _oldBytes = null!;
Expand Down
2 changes: 1 addition & 1 deletion src/Ivy.NativeJsonDiff.Benchmarks/Benchmarks/JsonApplyBenchmarks.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ namespace Ivy.NativeJsonDiff.Benchmarks.Benchmarks;
[Config(typeof(CiBenchmarkConfig))]
public class JsonApplyBenchmarks
{
[Params("Small", "Medium", "Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree")]
[Params("Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree")]
public string Category { get; set; } = null!;

private byte[] _oldBytes = null!;
Expand Down
2 changes: 1 addition & 1 deletion src/Ivy.NativeJsonDiff.Benchmarks/Benchmarks/PreParsedDiffBenchmarks.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ namespace Ivy.NativeJsonDiff.Benchmarks.Benchmarks;
[Config(typeof(CiBenchmarkConfig))]
public class PreParsedDiffBenchmarks
{
[Params("Small", "Medium", "Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree")]
[Params("Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree")]
public string Category { get; set; } = null!;

private byte[] _oldBytes = null!;
Expand Down
30 changes: 1 addition & 29 deletions src/Ivy.NativeJsonDiff.Benchmarks/TestData/JsonPayloadGenerator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,10 @@ namespace Ivy.NativeJsonDiff.Benchmarks.TestData;
public static class JsonPayloadGenerator
{
public static readonly string[] AllCategories =
["Small", "Medium", "Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree"];
["Large", "DeepNested", "ArrayHeavy", "ApiResponse", "ConfigFile", "LargeStateTree", "VeryLargeStateTree", "WidgetTree"];

public static (byte[] Old, byte[] New) Generate(string category) => category switch
{
"Small" => GenerateSmall(),
"Medium" => GenerateMedium(),
"Large" => GenerateLarge(),
"DeepNested" => GenerateDeepNested(),
"ArrayHeavy" => GenerateArrayHeavy(),
Expand All @@ -23,32 +21,6 @@ public static class JsonPayloadGenerator
_ => throw new ArgumentException($"Unknown category: {category}")
};

private static (byte[] Old, byte[] New) GenerateSmall()
{
var old = new Dictionary<string, object>();
var @new = new Dictionary<string, object>();
for (int i = 0; i < 10; i++)
{
old[$"field{i}"] = $"value{i}";
@new[$"field{i}"] = i == 5 ? "changed" : $"value{i}";
}

return (Serialize(old), Serialize(@new));
}

private static (byte[] Old, byte[] New) GenerateMedium()
{
var old = new Dictionary<string, object>();
var @new = new Dictionary<string, object>();
for (int i = 0; i < 100; i++)
{
old[$"field{i}"] = $"value{i}";
@new[$"field{i}"] = i % 10 == 0 ? "changed" : $"value{i}";
}

return (Serialize(old), Serialize(@new));
}

private static (byte[] Old, byte[] New) GenerateLarge()
{
var old = new Dictionary<string, object>();
Expand Down
5 changes: 1 addition & 4 deletions src/Ivy.NativeJsonDiff.Tests/JsonPayloadGeneratorTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ namespace Ivy.NativeJsonDiff.Tests;
public class JsonPayloadGeneratorTests
{
[Theory]
[InlineData("Small")]
[InlineData("Medium")]
[InlineData("Large")]
[InlineData("DeepNested")]
[InlineData("ArrayHeavy")]
Expand Down Expand Up @@ -83,13 +81,12 @@ public void Generate_OldAndNew_AreDifferent()
[Fact]
public void AllCategories_ContainsExpectedEntries()
{
Assert.Contains("Small", JsonPayloadGenerator.AllCategories);
Assert.Contains("ApiResponse", JsonPayloadGenerator.AllCategories);
Assert.Contains("ConfigFile", JsonPayloadGenerator.AllCategories);
Assert.Contains("LargeStateTree", JsonPayloadGenerator.AllCategories);
Assert.Contains("VeryLargeStateTree", JsonPayloadGenerator.AllCategories);
Assert.Contains("WidgetTree", JsonPayloadGenerator.AllCategories);
Assert.Equal(10, JsonPayloadGenerator.AllCategories.Length);
Assert.Equal(8, JsonPayloadGenerator.AllCategories.Length);
}

[Fact]
Expand Down
Loading