Clarification on null and empty string handling for decimal and string types

[k-nkmt]

While writing type-aware conversion and validation code for the rows element, I noticed a couple of points that I would like to clarify.

For decimal values, when the value is missing, examples such as adlbh use empty string.
In the specification, missing values are defined as follows:

Missing values are represented by null. Empty strings are represented by "".

However, the handling of decimal values is not entirely clear.
In my opinion, null would be more appropriate for a missing decimal value, but is it intended that empty strings are also acceptable?

While checking the implementation examples related to this, I also noticed that empty string are generally used for missing string values.
This may reflect the SAS convention where an empty string represents a missing value for character variables. However, in many modern systems, null and empty string are clearly distinguished, and treating empty strings simply as missing values could potentially cause issues.

Possible approaches might include:

• Explicitly define that empty string and null are not distinguished and should be treated equivalently.
• Add a column attribute to specify which values should be treated as missing (e.g., missing="").
• Add a column attribute indicating that empty strings should be preserved separately from missing values (e.g., keep_empty=true).

If there is an intended way to handle this, it would be very helpful to know.

[DmitryMK]

Hi, this is my personal opinion and not a team statement.

The specification does not instruct how NULL or empty string should be utilized for the reason that it is not a standard goal to define it. As you mentioned languages other than SAS differentiate NULL and empty string values and the goal of the standard to support it.

If you are generating data from a source which differentiates NULL and empty string, then it is logical to keep it as is. If you are submitting from source which does not do it, it is up to you how to store it, but you might want to explain it for the receiving side (e.g. all missing numeric values which were represented as decimal are shown as empty strings).
And then it is the receiving side choice how to handle them.

We might end up with a guideline how to deal with it and probably specify that if you are converting from SAS, suggested option is to use empty string (or NULL if it is a choice), but this guideline should be outside of the standard itself.

[k-nkmt]

Thank you for the reply.
Since this is a somewhat subtle point, I would like to clarify the point I was trying to raise.

I agree that how systems or tools internally interpret null and empty string values is outside the scope of the standard.
The point I was referring to is how missing values are expected to be represented in dataset-json itself.
The specification currently states:

Missing values are represented by null. Empty strings are represented by "".

Because these two statements appear together, this wording seems to imply that missing values should be represented as null, while "" represents an actual empty string value.

However, in the current examples, empty strings also appear to be used for missing string values. Both the examples and the current primary use case appear to be as a replacement for XPT. In that context, empty strings are effectively equivalent to missing values, so this may be why it has not caused issues so far.

Personally, I would prefer the definition to be explicit, since dataset-json is intended to support a wide range of data exchange scenarios. However, if the current behavior is considered sufficient in practice, I am also fine with leaving it as is and closing this issue.

[DmitryMK]

I see your point. There will be a team meeting tomorrow and we will try to discuss this question.