Feat: add Clapp et al. (1978)

[IsNeron]

No description provided.

[mentaljam]

The current implementation.lookup approach solves Clapp 1978, but I think it puts the abstraction at the wrong level and will make the schema unnecessarily restrictive.

The important distinction is that a lookup is not a kind of whole-function implementation. It is a typed mapping that may be used as one step inside an otherwise ordinary formula implementation.

For example, a future PTF may take a texture class and bulk density. With the current design, lookup implementations require exactly one matching input and fully determine all outputs, so that case does not compose. What we want instead is roughly:

texture_class -> lookup -> texture_parameters --+
                                                +-> formula -> outputs
bulk_density -----------------------------------+

I suggest modeling three independent concepts:

1. an enum type for the categorical input;
2. a record type for the lookup result;
3. a lookup definition that maps the enum type to the record type.

The existing spec-level $defs mechanism looks like the right foundation for this rather than introducing a parallel type system. Today $defs is limited to parameters/output definitions and local #/$defs/... references; it could be extended to contain enum, record, and lookup definitions.

A source-local definition could look conceptually like this:

$defs:
  Clapp1978TextureClass:
    type: enum
    description: Texture classes used by this source.
    values:
      - name: sand
        label: Sand
      - name: loamy_sand
        label: Loamy sand
      # ...

  Clapp1978Parameters:
    type: record
    name: Clapp1978Parameters
    fields:
      - name: b
        symbol: b
        unit: "1"
        domain: value > 0
        description: ...
      - name: saturation_suction
        symbol: psi_s
        unit: cm
        domain: value > 0
        description: ...
      # ...

  Clapp1978ParametersByTexture:
    type: lookup
    input:
      $ref: "#/$defs/Clapp1978TextureClass"
    output:
      $ref: "#/$defs/Clapp1978Parameters"
    values:
      - key: sand
        value:
          b: 4.05
          saturation_suction: 3.50
          # ...
      - key: loamy_sand
        value:
          b: 4.38
          saturation_suction: 1.78
          # ...

The lookup rows should refer to semantic enum members (sand, loamy_sand, ...) rather than introducing stable ptfkit numeric codes. Those codes are artificial here and are not part of the scientific model or a cross-target ABI contract.

Then the function can reference the declared types directly:

inputs:
  - name: soil_texture
    type:
      $ref: "#/$defs/Clapp1978TextureClass"
    description: Soil texture class.

outputs:
  $ref: "#/$defs/Clapp1978Parameters"

implementation:
  variables:
    - name: parameters
      lookup:
        table:
          $ref: "#/$defs/Clapp1978ParametersByTexture"
        key: soil_texture

The same record definition is therefore used for both the lookup result and the public PTF output; there is no need to duplicate its fields. More generally, lookup becomes another way of defining an implementation variable, alongside expr, and the implementation remains composable with arbitrary additional inputs and formulas.

This probably means the semantic IR should preserve the record-valued lookup instead of immediately lowering a table row into N independent scalar Expr::Lookup values. We will then need typed variables and field access (parameters.b, parameters.saturation_suction, etc.), but that is a much more faithful representation of the source data and gives us a real minimal type system rather than another special case.

Reusable domain types should also be separable from a particular APA/source spec. For example, if ptfkit eventually has a canonical USDA texture classification that is genuinely shared by multiple publications, it could live in something like specs/types/soil.yaml and be referenced from source specs. Publication-specific classifications should remain local $defs; Clapp 1978 should not be forced to use a global USDA enum if the classes represented by the paper are not actually identical to that global classification.

For code generation, enums should map to the natural language construct of each independent target. We do not have cross-target bindings or a shared ABI requirement, so representation attributes and stable numeric discriminants should not leak into the canonical spec just for codegen convenience:

• Rust: ordinary enum (no #[repr(u8)] unless some future API independently requires it);
• C++: ordinary scoped enum class (no fixed underlying type unless independently required);
• C: ordinary typedef enum, with prefixed enumerators because C enumerators are unscoped.

For the lookup itself, Rust can then be generated very naturally as a conversion between the enum and record. If the mapping is exhaustive, impl From<Texture> for Parameters is appropriate; if the lookup intentionally covers only part of the enum, TryFrom<Texture> for Parameters is the natural form. A partial mapping is then represented as a typed failure, rather than encoding missing values as a schema-level default: nan. Individual public targets can decide how to surface that failure (for example ValueError in Python or NaNs where an existing numeric API specifically requires that behavior).

So I would rework this PR around the following model:

• remove implementation.lookup as an alternative whole-function implementation;
• introduce enum definitions;
• make record definitions independently reusable;
• introduce lookup definitions as typed Enum -> Record mappings;
• allow implementation variables to be populated from a lookup;
• allow inputs/outputs/lookups to reference the same $defs types;
• add support for external reusable type definitions where the domain concept genuinely spans multiple source specs;
• remove stable ptfkit texture codes from this feature unless a particular public API has an independent reason to expose numeric codes.

I think this gives us a substantially more general schema while making the Clapp implementation itself simpler and more semantically accurate.

[mentaljam]

A couple of follow-up points on the enum design, especially for the Python target.

For Python, I do not think the wrapper should accept arbitrary arrays of enum objects and convert them to numeric ufunc inputs on every PTF call. That would make reusable categorical arrays unnecessarily expensive: the same array would be re-encoded every time it is passed to another function.

Instead, I suggest two public representations for an enum input:

scalar: EnumValue
array:  EnumArray[EnumValue]

EnumArray should be a reusable typed wrapper around the already encoded numeric NumPy array used by the generated ufunc. Encoding then happens once, explicitly, when the array is constructed:

textures = TextureClass.array([
    TextureClass.SAND,
    TextureClass.LOAMY_SAND,
    TextureClass.CLAY,
])

foo(soil_texture=textures)
bar(soil_texture=textures, bulk_density=density)
baz(soil_texture=textures)

Internally, textures can own something like a uint32 ndarray of target-private ordinal values. Every wrapper call then extracts that ndarray without copying and passes it directly to the CPython/NumPy ufunc. The integer representation is purely an implementation detail of the Python target: it is not present in the canonical schema, is not documented as a public code, and does not need to match Rust/C/C++.

A generic infrastructure type is preferable to one generated array class per enum, e.g. conceptually:

class EnumArray(Generic[E]):
    _enum_type: type[E]
    _codes: NDArray[np.uint32]

with each generated enum exposing a convenient constructor such as TextureClass.array(...). The wrapper can verify that an EnumArray belongs to the expected enum type before forwarding its internal ndarray. Slicing should ideally preserve the typed wrapper and share the underlying data where NumPy can do so.

For scalar enum inputs, encoding inside the wrapper is fine because it is O(1). So a generated PTF signature should be conceptually closer to:

soil_texture: TextureClass | EnumArray[TextureClass]

rather than accepting str | int | ArrayLike and repeatedly interpreting arbitrary arrays.

This also fits naturally with shared enum definitions: if several PTFs use the same domain enum, they also use the same Python enum type and the same reusable EnumArray[ThatEnum].

On textual enum values: we decided not to implement parsing or serialization in this PR. However, the schema should still preserve the canonical textual value defined by the source classification/standard rather than inventing a normalized spelling. For USDA, for example, the canonical value is "loamy sand", not "loamy_sand" and not a presentation-only "Loamy sand".

So enum members should distinguish their schema identifier from their canonical domain value, for example:

values:
  - name: sand
    value: "sand"
  - name: loamy_sand
    value: "loamy sand"
  - name: sandy_loam
    value: "sandy loam"

name is the stable schema/member identifier used by lookup references and as the source for target-specific identifiers. value is the exact canonical textual representation from the domain definition. A lookup should reference the member identity:

- key: loamy_sand
  value: ...

rather than embedding either numeric codes or doing string matching on "loamy sand".

For Python, I would generate an ordinary Enum with the canonical string stored in .value, for example:

class TextureClass(Enum):
    SAND = "sand"
    LOAMY_SAND = "loamy sand"

I would not use StrEnum yet. StrEnum makes enum members behave as strings (isinstance(member, str), equality with strings, etc.), which effectively introduces string parsing/serialization semantics into the public API before we have decided to support them. Ordinary Enum keeps the API typed while preserving the exact canonical string so that serialization can be added later without changing the schema or enum data.

In short, the Python part of the redesign should be:

• generated ordinary Enum values containing the canonical domain strings;
• no public numeric enum codes;
• no string parsing/serialization functionality in this PR;
• scalar enums encoded privately in the wrapper;
• reusable typed EnumArray[E] objects encoded once and passed to ufuncs without repeated conversion;
• numeric ndarray ordinals remain strictly private to the Python target.

[mentaljam]

One small addition to the enum schema proposal: enum members should also allow an optional description field.

For example:

values:
  - name: sand
    value: "sand"
    description: Coarse-textured soil class dominated by sand-sized particles.
  - name: loamy_sand
    value: "loamy sand"
    description: USDA loamy sand textural class.

description is not part of the enum identity or runtime value. It is documentation metadata and should be propagated into generated API documentation/doc comments for the corresponding enum member where the target language supports that naturally.

Conceptually:

pub enum TextureClass {
    /// Coarse-textured soil class dominated by sand-sized particles.
    Sand,

    /// USDA loamy sand textural class.
    LoamySand,
}

and similarly for C++/C generated documentation and Python docs.

[mentaljam]

The enum/API part is now much closer to the intended design, but the core lookup model is still not where we discussed it should be.

The main remaining issue is that lookup is still modeled as an alternative whole-function implementation:

implementation:
  lookup:
    input: soil_texture
    rows: ...

and the compiler still enforces that a lookup implementation has exactly one matching function input. This keeps the original limitation: a future PTF cannot take an enum input plus additional numeric inputs, obtain a record from a lookup, and then continue with ordinary formula variables.

The lookup should instead be an independently declared typed mapping, and implementation variables should be able to obtain their value from such a mapping.

For Clapp, the shape should be closer to:

$defs:
  UsdaTextureClass:
    type: enum
    values:
      ...

  Clapp1978Parameters:
    type: record
    name: Clapp1978Parameters
    fields:
      ...

  Clapp1978ParametersByTexture:
    type: lookup
    input:
      $ref: "#/$defs/UsdaTextureClass"
    output:
      $ref: "#/$defs/Clapp1978Parameters"
    values:
      - key: sand
        value:
          b: 4.05
          saturation_suction: 3.50
          ...

Then the function can use the same record definition as its public output and as the lookup result:

outputs:
  $ref: "#/$defs/Clapp1978Parameters"

implementation:
  variables:
    - name: parameters
      lookup:
        table:
          $ref: "#/$defs/Clapp1978ParametersByTexture"
        key: soil_texture

This is the important part of the redesign: enum, record, and lookup are separate concepts, and lookup becomes one way to populate a typed implementation variable rather than a special implementation mode for the entire function.

The current semantic lowering still flattens the lookup into one scalar Expr::Lookup per output field. That is why generated Rust currently emits a separate match soil_texture for every field. The IR should preserve the record-valued lookup, so Clapp can conceptually become one conversion from UsdaTextureClass to Clapp1978Parameters (and more complex PTFs can then use fields such as parameters.b in later expressions).

Related to this, inputs are currently still modeled as the old scalar Parameter plus an optional enum reference. Since enums are now real types, it would be cleaner for the input declaration itself to reference its type explicitly rather than treating enum-ness as an extra property of a numeric parameter.

The Python changes are otherwise in the right direction: ordinary Enum values preserve the canonical domain strings, public numeric codes are gone, scalar enum values are encoded privately, and reusable enum arrays avoid repeated encoding.

I would keep EnumArray deliberately minimal. It does not need to imitate a NumPy array or expose indexing/slicing/iteration semantics. Its responsibility is simply to encode an Iterable[Enum] once, store the real numeric NumPy array internally, remember the enum type, and let generated wrappers retrieve that array after a type check and pass it directly to the CPython/NumPy ufunc. The current __getitem__, __iter__, shape, size, etc. add API surface that is not required for this purpose and can be removed unless there is another concrete use case for them.

So the remaining changes I would focus on are:

• make record definitions independently reusable via $defs;
• add lookup definitions to $defs as typed Enum -> Record mappings;
• allow implementation variables to be populated from a lookup;
• preserve record-valued lookups in the semantic IR instead of flattening them into scalar lookup expressions;
• use the same record definition for both the lookup result and the PTF output;
• make enum inputs properly typed rather than scalar parameters with an attached enum reference;
• keep EnumArray as a small one-time encoder/typed holder for the underlying NumPy integer array.