Nullable in serialization corrupts JSON

[NickKhalow]

Segment.Serialization.JsonObject – Critical Serialization Bug

Describe the Bug

There is a critical JSON serialization defect in Segment.Serialization.JsonObject.
The serializer produces invalid JSON when serializing Nullable<T> fields, resulting in malformed fragments such as:

"is_empty_scene": ,

instead of:

"is_empty_scene": null

This breaks JSON format and causes Segment’s ingestion pipeline to silently drop events.

Our custom Rust implementation revealed the issue by producing correct JSON, making the discrepancy visible.

---

To Reproduce

1. Download the reproduction project:
   https://github.com/NickKhalow/JsonBugReproSegment
2. Run the test suite.
3. Observe malformed JSON output produced by JsonObject when handling nullable fields.

Example of the broken output:

{"scene_hash": null,"old_parcel": "(NaN, NaN)","is_empty_scene": ,"new_parcel": "(1, -1)"}

Notice "is_empty_scene": , - the value is missing entirely.

---

Expected Behavior

JsonObject should correctly serialize Nullable<T> values:

• Nullable
• Nullable
• Nullable

Expected valid JSON:

"is_empty_scene": true

or:

"is_empty_scene": null

but never:

"is_empty_scene": ,

This violates the JSON specification and corrupts the payload.

---

Platform

• Library: Segment.Serialization.JsonObject
• Version: In production where the critical bug was found - 2.4.2 , repro project with the latest version at the moment - 2.5.3
• Platform: .NET and Unity
• Integrations: None (reproduces in isolation)

---

Additional Context

The bug is due to incorrect handling of Nullable<T> inside JsonObjectConverter.
When a nullable value is null, the serializer emits nothing, leaving a dangling comma and creating invalid JSON.

The silently ignore malformed payloads, this issue resulted in consistent unnoticed loss of analytics events.

[wenxi-zeng]

hi @NickKhalow thanks for reporting this issue! we'll take a look and release a fix asap.

[NickKhalow]

@wenxi-zeng I fixed the bug, please apply asap segmentio/Serialization.NET#18

[wenxi-zeng]

hi @NickKhalow, thanks for the pr. we took a look at the problem, the root cause is our serialization library only supports serialization of primitive types. anything other than that is considered custom types and you'd have to do the conversion by your own. since nullable types are widely used, we will add supports to it asap, but in the meanwhile please follow the approaches below to get unblocked.

Approach 1: extension methods

    public static class JsonElementExtensions
    {
        public static JsonElement ToJsonElement(this bool? value)
        {
            return value.HasValue ? (JsonElement)value.Value : JsonNull.Instance;
        }

        public static JsonElement ToJsonElement(this int? value)
        {
            return value.HasValue ? (JsonElement)value.Value : JsonNull.Instance;
        }

        public static JsonElement ToJsonElement(this long? value)
        {
            return value.HasValue ? (JsonElement)value.Value : JsonNull.Instance;
        }

        public static JsonElement ToJsonElement(this short? value)
        {
            return value.HasValue ? (JsonElement)value.Value : JsonNull.Instance;
        }

        public static JsonElement ToJsonElement(this double? value)
        {
            return value.HasValue ? (JsonElement)value.Value : JsonNull.Instance;
        }

        public static JsonElement ToJsonElement(this float? value)
        {
            return value.HasValue ? (JsonElement)value.Value : JsonNull.Instance;
        }
    }

then use it as follow:

public void JsonValidityCrash()
        {
            DataModel data = new DataModel
            {
                old_parcel = "RedRock",
                new_parcel = string.Empty,
                scene_hash = Guid.NewGuid().ToString(),
                is_empty_scene = null
            };

            JsonObject json = new JsonObject
            {
                { "old_parcel", data.old_parcel },
                { "new_parcel", data.new_parcel },
                { "scene_hash", data.scene_hash },
                { "is_empty_scene", data.is_empty_scene.ToJsonElement() }
            };

            string stringified = JsonUtility.ToJson(json);
            DataModel jsonObject = JsonUtility.FromJson<DataModel>(stringified);
            // ...
        }

Approach 2: implicit operator

        [Serializable]
        public struct DataModel
        {
            public string old_parcel;
            public string new_parcel;
            public string scene_hash;

            public bool? is_empty_scene;

            public static implicit operator JsonElement(DataModel data)
            {
                return new JsonObject
                {
                    { "old_parcel", data.old_parcel },
                    { "new_parcel", data.new_parcel },
                    { "scene_hash", data.scene_hash },
                    { "is_empty_scene", data.is_empty_scene.HasValue ? (JsonElement)data.is_empty_scene.Value : JsonNull.Instance }
                };
            }
        }

and then you can use it:

        public void JsonValidityCrash()
        {
            DataModel data = new DataModel
            {
                old_parcel = "RedRock",
                new_parcel = string.Empty,
                scene_hash = Guid.NewGuid().ToString(),
                is_empty_scene = null
            };

            JsonObject json = (JsonObject)data;
            // ...
        }

Approach 3: custom serializer

#if NETSTANDARD2_0
        [JsonConverter(typeof(DataModelSystemTextJsonConverter))]
#else
        [JsonConverter(typeof(DataModelNewtonsoftConverter))]
#endif
        [Serializable]
        public struct DataModel
        {
            public string old_parcel;
            public string new_parcel;
            public string scene_hash;

            public bool? is_empty_scene;
        }

#if NETSTANDARD2_0
        public class DataModelSystemTextJsonConverter : JsonConverter<DataModel>
        {
            public override void Write(Utf8JsonWriter writer, DataModel value, JsonSerializerOptions options)
            {
                writer.WriteStartObject();
                writer.WriteString("old_parcel", value.old_parcel);
                writer.WriteString("new_parcel", value.new_parcel);
                writer.WriteString("scene_hash", value.scene_hash);
                
                if (value.is_empty_scene.HasValue)
                    writer.WriteBoolean("is_empty_scene", value.is_empty_scene.Value);
                else
                    writer.WriteNull("is_empty_scene");
                    
                writer.WriteEndObject();
            }

            public override DataModel Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
            {
                if (reader.TokenType != JsonTokenType.StartObject)
                    throw new JsonException("Expected start of object");

                var dataModel = new DataModel();

                while (reader.Read())
                {
                    if (reader.TokenType == JsonTokenType.EndObject)
                        break;

                    if (reader.TokenType == JsonTokenType.PropertyName)
                    {
                        string propertyName = reader.GetString();
                        reader.Read();

                        switch (propertyName)
                        {
                            case "old_parcel":
                                dataModel.old_parcel = reader.GetString();
                                break;
                            case "new_parcel":
                                dataModel.new_parcel = reader.GetString();
                                break;
                            case "scene_hash":
                                dataModel.scene_hash = reader.GetString();
                                break;
                            case "is_empty_scene":
                                if (reader.TokenType == JsonTokenType.Null)
                                    dataModel.is_empty_scene = null;
                                else
                                    dataModel.is_empty_scene = reader.GetBoolean();
                                break;
                        }
                    }
                }

                return dataModel;
            }
        }
#else
        public class DataModelNewtonsoftConverter : JsonConverter<DataModel>
        {
            public override void WriteJson(JsonWriter writer, DataModel value, JsonSerializer serializer)
            {
                writer.WriteStartObject();
                writer.WritePropertyName("old_parcel");
                writer.WriteValue(value.old_parcel);
                writer.WritePropertyName("new_parcel");
                writer.WriteValue(value.new_parcel);
                writer.WritePropertyName("scene_hash");
                writer.WriteValue(value.scene_hash);
                writer.WritePropertyName("is_empty_scene");
                writer.WriteValue(value.is_empty_scene);
                writer.WriteEndObject();
            }

            public override DataModel ReadJson(JsonReader reader, Type objectType, DataModel existingValue, bool hasExistingValue, JsonSerializer serializer)
            {
                JObject jsonObject = JObject.Load(reader);
                var dataModel = new DataModel();

                dataModel.old_parcel = jsonObject["old_parcel"]?.Value<string>();
                dataModel.new_parcel = jsonObject["new_parcel"]?.Value<string>();
                dataModel.scene_hash = jsonObject["scene_hash"]?.Value<string>();
                dataModel.is_empty_scene = jsonObject["is_empty_scene"]?.Value<bool?>();

                return dataModel;
            }
        }
#endif

and use it as follow:

public void JsonValidityCrash()
        {
            DataModel data = new DataModel
            {
                old_parcel = "RedRock",
                new_parcel = string.Empty,
                scene_hash = Guid.NewGuid().ToString(),
                is_empty_scene = null
            };

            string stringified = JsonUtility.ToJson(data);
            DataModel jsonObject = JsonUtility.FromJson<DataModel>(stringified);
            // ...
        }

[wenxi-zeng]

hi @NickKhalow, 2.6.0 now supports nullable primitives. please give it a try