Repository navigation
fix(geolibre): restore Composite Bands raster selection #1694
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -221,6 +221,7 @@ fn field_kind(name: &str) -> Option<ParamKind> { | |
| } | ||
| } | ||
|
|
||
| /// Builds the form and staging metadata together, including legacy Composite Bands inputs. | ||
| fn spec_of(m: &Value) -> Option<(ToolSpec, Meta)> { | ||
| let id = m.get("id")?.as_str()?; | ||
| let name = m.get("display_name").and_then(Value::as_str).filter(|n| !n.is_empty()).unwrap_or(id); | ||
|
|
@@ -232,13 +233,21 @@ fn spec_of(m: &Value) -> Option<(ToolSpec, Meta)> { | |
| for p in m.get("params").and_then(Value::as_array).into_iter().flatten() { | ||
| let Some(pname) = p.get("name").and_then(Value::as_str) else { continue }; | ||
| let schema = p.get("schema"); | ||
| let data = Data::of(p.get("data_kind").and_then(Value::as_str).unwrap_or("")); | ||
| // GeoLibre 1.8.0/1.8.1 describe this raster list as a string option (#1684). | ||
| // Correct both the form and staging; leave explicit metadata from newer runners alone. | ||
| let composite_inputs = id == "composite_bands" | ||
| && pname == "inputs" | ||
| && p.get("io_role").is_none() | ||
| && p.get("data_kind").and_then(Value::as_str) == Some("string") | ||
| && schema.and_then(|s| s.get("kind")).and_then(Value::as_str) == Some("string"); | ||
| let data = if composite_inputs { Data::Raster } else { Data::of(p.get("data_kind").and_then(Value::as_str).unwrap_or("")) }; | ||
| let role = match p.get("io_role").and_then(Value::as_str) { | ||
| None if composite_inputs => Role::Input, | ||
| Some("input") => Role::Input, | ||
| Some("output") => Role::Output, | ||
| _ => Role::Option, | ||
| }; | ||
| let many = schema.and_then(|s| s.get("cardinality")).and_then(Value::as_str).is_some_and(|c| c != "single"); | ||
| let many = composite_inputs || schema.and_then(|s| s.get("cardinality")).and_then(Value::as_str).is_some_and(|c| c != "single"); | ||
| let optional = !p.get("required").and_then(Value::as_bool).unwrap_or(false); | ||
| let default = defaults.and_then(|d| d.get(pname)).filter(|v| optional && !v.is_null()).cloned(); | ||
| meta.push(MetaParam { name: pname.to_string(), role, data, default }); | ||
|
|
@@ -574,21 +583,20 @@ pub fn prepare(spec: &ToolSpec, inputs: &dyn Inputs, params: &Value) -> Result<J | |
| prepare_with(spec, &meta, inputs, params) | ||
| } | ||
|
|
||
| /// Validates input name lists before choosing a CRS or staging files, preserving their order. | ||
| fn prepare_with(spec: &ToolSpec, meta: &Meta, inputs: &dyn Inputs, params: &Value) -> Result<Job, String> { | ||
| let id = spec.id.strip_prefix(PREFIX).ok_or("not a GeoLibre tool")?; | ||
| let value_of = |name: &str| params.get(name).filter(|v| !v.is_null() && v.as_str() != Some("")); | ||
| let names = |v: &Value| -> Vec<String> { | ||
| v.as_str().map(|s| s.split(';').map(str::trim).filter(|n| !n.is_empty()).map(str::to_string).collect()).unwrap_or_default() | ||
| }; | ||
| let names = rustgis_analysis::layer_list; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Behaviour change beyond Composite Bands (confidence: high that it changes; medium that it matters).
The PR description says "other tools retain their current behavior", which is not accurate. Please either correct that sentence or add a test that pins the other-tool behaviour (an optional many-input given
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Corrected the PR description. The metadata fallback is specific to Composite Bands; the shared staging parser now accepts arrays and explicitly rejects empty or malformed lists for GeoLibre inputs. Optional values can still be omitted or cleared with null or an empty string. This avoids the previous silent drops. No source change was needed. |
||
|
|
||
| // Rasters set the coordinate system; otherwise the vector data's UTM zone. | ||
| let mut raster_crs = Vec::new(); | ||
| let mut vector_extents = Vec::new(); | ||
| for p in meta.params.iter().filter(|p| p.role == Role::Input) { | ||
| for n in value_of(&p.name).map(names).unwrap_or_default() { | ||
| if let Some(r) = inputs.raster(&n) { | ||
| for n in value_of(&p.name).map(names).transpose()?.unwrap_or_default() { | ||
| if let Some(r) = inputs.raster(n) { | ||
| raster_crs.push(r.raster.crs); | ||
| } else if let Some(ds) = inputs.dataset(&n) { | ||
| } else if let Some(ds) = inputs.dataset(n) { | ||
| vector_extents.push(ds.extent()); | ||
| } | ||
| } | ||
|
|
@@ -610,7 +618,7 @@ fn prepare_with(spec: &ToolSpec, meta: &Meta, inputs: &dyn Inputs, params: &Valu | |
| match p.role { | ||
| Role::Input => { | ||
| let Some(v) = value_of(&p.name) else { continue }; | ||
| let files = names(v).iter().map(|n| stage(inputs, n, p.data, &tf, &mut job)).collect::<Result<Vec<_>, _>>()?; | ||
| let files = names(v)?.into_iter().map(|n| stage(inputs, n, p.data, &tf, &mut job)).collect::<Result<Vec<_>, _>>()?; | ||
| if !files.is_empty() { | ||
| job.args.push((p.name.clone(), Arg::Files(files))); | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -133,6 +133,73 @@ fn inputs_of_several_layers_are_layer_lists() { | |
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn composite_bands_legacy_manifest_selects_and_stages_rasters_in_order() { | ||
| // The input/output subset of `geolibre manifests` in 1.8.0 and 1.8.1 (#1684). | ||
| // Unlike the generic list fixture above, the real input has no role or cardinality. | ||
| let manifest = json!({"id": "composite_bands", "params": [ | ||
| {"name": "inputs", "data_kind": "string", "required": true, "schema": {"kind": "string"}}, | ||
| {"name": "output", "io_role": "output", "data_kind": "raster", "schema": {"kind": "output"}} | ||
| ]}); | ||
| let (spec, meta) = spec_of(&manifest).unwrap(); | ||
| assert_eq!(spec.params[0].kind, ParamKind::RasterLayers); | ||
| let mut f = Fixture::new(); | ||
| f.dem.bands[0].data.fill(14.3); | ||
| // An in-memory layer and an embedded file must both be staged, in the selected order. | ||
| for (names, values) in [ | ||
| (json!("dem;embedded"), [14.3, 0.0]), | ||
| (json!("embedded;dem"), [0.0, 14.3]), | ||
| (json!(["dem", "embedded"]), [14.3, 0.0]), | ||
| (json!(["embedded", "dem"]), [0.0, 14.3]), | ||
| ] { | ||
| let params = rustgis_analysis::resolve_params(&spec, &json!({"inputs": names})).unwrap(); | ||
| let job = prepare_with(&spec, &meta, &f, ¶ms).unwrap(); | ||
| assert_eq!(job.crs, f.dem.crs); | ||
| assert_eq!(job.files.len(), 2); | ||
| for ((name, bytes), value) in job.files.iter().zip(values) { | ||
| let raster = rustgis_data::raster::read_bytes(name, bytes).unwrap(); | ||
| assert_eq!(raster.bands[0].data[0], value); | ||
| } | ||
| let staged = job.files.iter().map(|(name, _)| format!("/work/{name}")).collect::<Vec<_>>().join(";"); | ||
| assert_eq!(job.argv("/work"), [format!("--inputs={staged}"), "--output=/work/output.tif".into()]); | ||
| } | ||
| // Refuse malformed arrays rather than silently dropping an entry or the entire input. | ||
| for invalid in [json!(["dem", 7]), json!([])] { | ||
| let error = prepare_with(&spec, &meta, &f, &json!({"inputs": invalid})).unwrap_err(); | ||
| assert!(error.contains("input layer"), "{error}"); | ||
| } | ||
| // Other tools' string options must not become raster inputs just because of their name. | ||
| let mut unrelated = manifest.clone(); | ||
| unrelated["id"] = json!("other_tool"); | ||
| assert_eq!(spec_of(&unrelated).unwrap().0.params[0].kind, ParamKind::Text); | ||
| // A newer runner's explicit metadata keeps its usual meaning. | ||
| let mut explicit = manifest; | ||
| explicit["params"][0] = json!({"name":"inputs", "io_role":"input", "data_kind":"vector", "schema":{"cardinality":"many"}}); | ||
| assert_eq!(spec_of(&explicit).unwrap().0.params[0].kind, ParamKind::VectorLayers); | ||
| } | ||
|
|
||
| #[test] | ||
| fn composite_bands_runner_preserves_selected_band_order() { | ||
| let Some(mut f) = setup() else { return }; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Test coverage (confidence: high).
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Clarified the Testing section: the GeoLibre 1.8.1 output-band check was run locally. It follows the existing optional-runner convention; the staging and input-validation regression runs without a runner in CI. |
||
| f.dem.bands[0].data.fill(14.3); | ||
| for (names, values) in [ | ||
| (json!("dem;embedded"), [14.3, 0.0]), | ||
| (json!("embedded;dem"), [0.0, 14.3]), | ||
| (json!(["dem", "embedded"]), [14.3, 0.0]), | ||
| (json!(["embedded", "dem"]), [0.0, 14.3]), | ||
| ] { | ||
| let result = rustgis_analysis::run("geolibre:composite_bands", &f, &json!({"inputs": names})).unwrap(); | ||
| let Output::Layers(layers) = result.output else { panic!("layers") }; | ||
| let Some(OutputLayer::Raster { bytes, file_name, .. }) = layers.first() else { panic!("a raster: {layers:?}") }; | ||
| let raster = rustgis_data::raster::read_bytes(file_name, bytes).unwrap(); | ||
| assert_eq!(raster.bands.len(), 2); | ||
| assert_eq!(raster.crs, f.dem.crs); | ||
| for (band, value) in raster.bands.iter().zip(values) { | ||
| assert!((band.data[0] - value).abs() < 1e-5, "{} != {value}", band.data[0]); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| #[test] | ||
| fn field_suggestions_keep_categorical_values_and_plural_lists() { | ||
| let manifest = json!({"id": "field_forms", "params": [ | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug (confidence: medium): this makes legacy
composite_bands.inputsanInputparameter, so it now goes throughprepare_with→names(lib.rs ~589–591).namesonly readsValue::String, so an array value such as["dem", "embedded"]is dropped with no notice.value_ofpasses arrays through, andrustgis_analysis::layer_listdocuments arrays as a supported layer-list format. Nothing on the geolibre path (params_foronly normalizes built-in tools) turns an array into;-joined text. The result is either a runner error for a missing required input (the wrong message) or a silent run with no inputs. Other many-input GeoLibre tools already have this gap; this PR routes Composite Bands into it.Suggested fix: make
namesaccept arrays too, e.g.Non-string array items would still be skipped silently. If you want them refused, return a
Resultand use?at the call inprepare_with. A regression test besidecomposite_bands_legacy_manifest_selects_and_stages_rasters_in_orderwould pin it:{"inputs": ["embedded", "dem"]}should stage in the same order as"embedded;dem".There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reproduced and fixed in f5be4c6a. Preparation now uses the shared layer-list parser for CRS selection and file staging. The regressions cover strings and arrays in both orders, plus rejection of malformed arrays. GeoLibre 1.8.1 output-band checks, full CI, wasm and hooks passed.