Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 17 additions & 9 deletions crates/geolibre/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,7 @@ fn field_kind(name: &str) -> Option<ParamKind> {
}
}

/// Builds the form and staging metadata together, including legacy Composite Bands inputs.
fn spec_of(m: &Value) -> Option<(ToolSpec, Meta)> {
let id = m.get("id")?.as_str()?;
let name = m.get("display_name").and_then(Value::as_str).filter(|n| !n.is_empty()).unwrap_or(id);
Expand All @@ -232,13 +233,21 @@ fn spec_of(m: &Value) -> Option<(ToolSpec, Meta)> {
for p in m.get("params").and_then(Value::as_array).into_iter().flatten() {
let Some(pname) = p.get("name").and_then(Value::as_str) else { continue };
let schema = p.get("schema");
let data = Data::of(p.get("data_kind").and_then(Value::as_str).unwrap_or(""));
// GeoLibre 1.8.0/1.8.1 describe this raster list as a string option (#1684).
// Correct both the form and staging; leave explicit metadata from newer runners alone.
let composite_inputs = id == "composite_bands"
&& pname == "inputs"
&& p.get("io_role").is_none()
&& p.get("data_kind").and_then(Value::as_str) == Some("string")
&& schema.and_then(|s| s.get("kind")).and_then(Value::as_str) == Some("string");
let data = if composite_inputs { Data::Raster } else { Data::of(p.get("data_kind").and_then(Value::as_str).unwrap_or("")) };
let role = match p.get("io_role").and_then(Value::as_str) {
None if composite_inputs => Role::Input,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (confidence: medium): this makes legacy composite_bands.inputs an Input parameter, so it now goes through prepare_with → names (lib.rs ~589–591). names only reads Value::String, so an array value such as ["dem", "embedded"] is dropped with no notice. value_of passes arrays through, and rustgis_analysis::layer_list documents arrays as a supported layer-list format. Nothing on the geolibre path (params_for only normalizes built-in tools) turns an array into ;-joined text. The result is either a runner error for a missing required input (the wrong message) or a silent run with no inputs. Other many-input GeoLibre tools already have this gap; this PR routes Composite Bands into it.

Suggested fix: make names accept arrays too, e.g.

let names = |v: &Value| -> Vec<String> {
    let items: Vec<&str> = match v {
        Value::String(s) => s.split(';').collect(),
        Value::Array(a) => a.iter().filter_map(Value::as_str).collect(),
        _ => Vec::new(),
    };
    items.into_iter().map(str::trim).filter(|n| !n.is_empty()).map(str::to_string).collect()
};

Non-string array items would still be skipped silently. If you want them refused, return a Result and use ? at the call in prepare_with. A regression test beside composite_bands_legacy_manifest_selects_and_stages_rasters_in_order would pin it: {"inputs": ["embedded", "dem"]} should stage in the same order as "embedded;dem".

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced and fixed in f5be4c6a. Preparation now uses the shared layer-list parser for CRS selection and file staging. The regressions cover strings and arrays in both orders, plus rejection of malformed arrays. GeoLibre 1.8.1 output-band checks, full CI, wasm and hooks passed.

Some("input") => Role::Input,
Some("output") => Role::Output,
_ => Role::Option,
};
let many = schema.and_then(|s| s.get("cardinality")).and_then(Value::as_str).is_some_and(|c| c != "single");
let many = composite_inputs || schema.and_then(|s| s.get("cardinality")).and_then(Value::as_str).is_some_and(|c| c != "single");
let optional = !p.get("required").and_then(Value::as_bool).unwrap_or(false);
let default = defaults.and_then(|d| d.get(pname)).filter(|v| optional && !v.is_null()).cloned();
meta.push(MetaParam { name: pname.to_string(), role, data, default });
Expand Down Expand Up @@ -574,21 +583,20 @@ pub fn prepare(spec: &ToolSpec, inputs: &dyn Inputs, params: &Value) -> Result<J
prepare_with(spec, &meta, inputs, params)
}

/// Validates input name lists before choosing a CRS or staging files, preserving their order.
fn prepare_with(spec: &ToolSpec, meta: &Meta, inputs: &dyn Inputs, params: &Value) -> Result<Job, String> {
let id = spec.id.strip_prefix(PREFIX).ok_or("not a GeoLibre tool")?;
let value_of = |name: &str| params.get(name).filter(|v| !v.is_null() && v.as_str() != Some(""));
let names = |v: &Value| -> Vec<String> {
v.as_str().map(|s| s.split(';').map(str::trim).filter(|n| !n.is_empty()).map(str::to_string).collect()).unwrap_or_default()
};
let names = rustgis_analysis::layer_list;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Behaviour change beyond Composite Bands (confidence: high that it changes; medium that it matters). names now calls rustgis_analysis::layer_list for every Input-role list in every GeoLibre tool, not just legacy composite_bands.inputs. Three effects:

  • Array values are now accepted for all such tools. Previously they were silently dropped, so this fixes a silent loss, but it is still a change.
  • Non-string, non-array values (for example a number) now error. Previously they were skipped.
  • An explicit empty array [] (or a whitespace-only string) for an optional input now fails with "at least one input layer is required". Previously it was skipped. Only API callers can send []: the form sends ;-joined text and clears to "", which value_of still filters.

The PR description says "other tools retain their current behavior", which is not accurate. Please either correct that sentence or add a test that pins the other-tool behaviour (an optional many-input given [] and given an array). I did not find a crash path.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected the PR description. The metadata fallback is specific to Composite Bands; the shared staging parser now accepts arrays and explicitly rejects empty or malformed lists for GeoLibre inputs. Optional values can still be omitted or cleared with null or an empty string. This avoids the previous silent drops. No source change was needed.


// Rasters set the coordinate system; otherwise the vector data's UTM zone.
let mut raster_crs = Vec::new();
let mut vector_extents = Vec::new();
for p in meta.params.iter().filter(|p| p.role == Role::Input) {
for n in value_of(&p.name).map(names).unwrap_or_default() {
if let Some(r) = inputs.raster(&n) {
for n in value_of(&p.name).map(names).transpose()?.unwrap_or_default() {
if let Some(r) = inputs.raster(n) {
raster_crs.push(r.raster.crs);
} else if let Some(ds) = inputs.dataset(&n) {
} else if let Some(ds) = inputs.dataset(n) {
vector_extents.push(ds.extent());
}
}
Expand All @@ -610,7 +618,7 @@ fn prepare_with(spec: &ToolSpec, meta: &Meta, inputs: &dyn Inputs, params: &Valu
match p.role {
Role::Input => {
let Some(v) = value_of(&p.name) else { continue };
let files = names(v).iter().map(|n| stage(inputs, n, p.data, &tf, &mut job)).collect::<Result<Vec<_>, _>>()?;
let files = names(v)?.into_iter().map(|n| stage(inputs, n, p.data, &tf, &mut job)).collect::<Result<Vec<_>, _>>()?;
if !files.is_empty() {
job.args.push((p.name.clone(), Arg::Files(files)));
}
Expand Down
67 changes: 67 additions & 0 deletions crates/geolibre/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,73 @@ fn inputs_of_several_layers_are_layer_lists() {
);
}

#[test]
fn composite_bands_legacy_manifest_selects_and_stages_rasters_in_order() {
// The input/output subset of `geolibre manifests` in 1.8.0 and 1.8.1 (#1684).
// Unlike the generic list fixture above, the real input has no role or cardinality.
let manifest = json!({"id": "composite_bands", "params": [
{"name": "inputs", "data_kind": "string", "required": true, "schema": {"kind": "string"}},
{"name": "output", "io_role": "output", "data_kind": "raster", "schema": {"kind": "output"}}
]});
let (spec, meta) = spec_of(&manifest).unwrap();
assert_eq!(spec.params[0].kind, ParamKind::RasterLayers);
let mut f = Fixture::new();
f.dem.bands[0].data.fill(14.3);
// An in-memory layer and an embedded file must both be staged, in the selected order.
for (names, values) in [
(json!("dem;embedded"), [14.3, 0.0]),
(json!("embedded;dem"), [0.0, 14.3]),
(json!(["dem", "embedded"]), [14.3, 0.0]),
(json!(["embedded", "dem"]), [0.0, 14.3]),
] {
let params = rustgis_analysis::resolve_params(&spec, &json!({"inputs": names})).unwrap();
let job = prepare_with(&spec, &meta, &f, &params).unwrap();
assert_eq!(job.crs, f.dem.crs);
assert_eq!(job.files.len(), 2);
for ((name, bytes), value) in job.files.iter().zip(values) {
let raster = rustgis_data::raster::read_bytes(name, bytes).unwrap();
assert_eq!(raster.bands[0].data[0], value);
}
let staged = job.files.iter().map(|(name, _)| format!("/work/{name}")).collect::<Vec<_>>().join(";");
assert_eq!(job.argv("/work"), [format!("--inputs={staged}"), "--output=/work/output.tif".into()]);
}
// Refuse malformed arrays rather than silently dropping an entry or the entire input.
for invalid in [json!(["dem", 7]), json!([])] {
let error = prepare_with(&spec, &meta, &f, &json!({"inputs": invalid})).unwrap_err();
assert!(error.contains("input layer"), "{error}");
}
// Other tools' string options must not become raster inputs just because of their name.
let mut unrelated = manifest.clone();
unrelated["id"] = json!("other_tool");
assert_eq!(spec_of(&unrelated).unwrap().0.params[0].kind, ParamKind::Text);
// A newer runner's explicit metadata keeps its usual meaning.
let mut explicit = manifest;
explicit["params"][0] = json!({"name":"inputs", "io_role":"input", "data_kind":"vector", "schema":{"cardinality":"many"}});
assert_eq!(spec_of(&explicit).unwrap().0.params[0].kind, ParamKind::VectorLayers);
}

#[test]
fn composite_bands_runner_preserves_selected_band_order() {
let Some(mut f) = setup() else { return };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test coverage (confidence: high). setup() returns None when no GeoLibre runner is installed, and this test then returns early after an eprintln!. In a CI job without the runner it passes without checking anything. The unit test composite_bands_legacy_manifest_selects_and_stages_rasters_in_order does cover the ordering without a runner, so the regression itself is guarded. The runner test is only the end-to-end check the PR description cites. Please say in the PR which CI job runs it with a runner, or note that it is local-only, so the claim is not read as CI coverage.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified the Testing section: the GeoLibre 1.8.1 output-band check was run locally. It follows the existing optional-runner convention; the staging and input-validation regression runs without a runner in CI.

f.dem.bands[0].data.fill(14.3);
for (names, values) in [
(json!("dem;embedded"), [14.3, 0.0]),
(json!("embedded;dem"), [0.0, 14.3]),
(json!(["dem", "embedded"]), [14.3, 0.0]),
(json!(["embedded", "dem"]), [0.0, 14.3]),
] {
let result = rustgis_analysis::run("geolibre:composite_bands", &f, &json!({"inputs": names})).unwrap();
let Output::Layers(layers) = result.output else { panic!("layers") };
let Some(OutputLayer::Raster { bytes, file_name, .. }) = layers.first() else { panic!("a raster: {layers:?}") };
let raster = rustgis_data::raster::read_bytes(file_name, bytes).unwrap();
assert_eq!(raster.bands.len(), 2);
assert_eq!(raster.crs, f.dem.crs);
for (band, value) in raster.bands.iter().zip(values) {
assert!((band.data[0] - value).abs() < 1e-5, "{} != {value}", band.data[0]);
}
}
}

#[test]
fn field_suggestions_keep_categorical_values_and_plural_lists() {
let manifest = json!({"id": "field_forms", "params": [
Expand Down
Loading