Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions tools/3dtrees_metadata/.shed.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: 3dtrees_metadata
owner: bgruening
description: Extract administrative and ecoregion metadata for 3Dtrees collections.
homepage_url: https://github.com/3dTrees-earth/3dtrees_metadata
long_description: |
Extracts administrative and ecoregion context for a 3Dtrees collection summary
by computing the collection centroid from collection_summary.json and
intersecting it with selected GADM and WWF Terrestrial Ecoregions v2.0 vector
datasets.
remote_repository_url: https://github.com/3dTrees-earth/galaxytools/tree/3dtrees_metadata_v1_0_0/tools/3dtrees_metadata
type: unrestricted
categories:
- "Geo Science"
106 changes: 106 additions & 0 deletions tools/3dtrees_metadata/3dtrees_metadata.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
<tool id="3dtrees_metadata" name="3Dtrees: Metadata" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="24.2">
<description>Extract administrative and ecoregion metadata for a point-cloud collection</description>
<macros>
<token name="@TOOL_VERSION@">1.0.0</token>
<token name="@VERSION_SUFFIX@">0</token>
</macros>
<requirements>
<container type="docker">ghcr.io/3dtrees-earth/3dtrees_metadata:@TOOL_VERSION@</container>
</requirements>
<command detect_errors="exit_code"><![CDATA[
ln -s '$collection_summary' collection_summary.json &&
python -u /src/run.py
--collection-summary collection_summary.json
--metadata-layers '$metadata_layers'
--output-file additional_metadata.json
2>&1
]]></command>
<inputs>
<param name="collection_summary" type="data" format="json" label="Collection summary" help="collection_summary.json from the 3Dtrees: LAS/LAZ Standardization collection mode."/>
<param argument="--metadata-layers" type="select" multiple="true" display="checkboxes" min="1" label="Metadata layers to extract" help="Choose at least one metadata layer. Both are selected by default.">
<option value="gadm" selected="true">GADM administrative boundaries</option>
<option value="ecoregion" selected="true">WWF Terrestrial Ecoregions v2.0</option>
</param>
</inputs>
<outputs>
<data name="additional_metadata" format="json" label="additional_metadata" from_work_dir="additional_metadata.json"/>
</outputs>
<tests>
<test expect_num_outputs="1">
<param name="collection_summary" value="collection_summary.json" ftype="json"/>
<param name="metadata_layers" value="gadm,ecoregion"/>
<output name="additional_metadata" ftype="json">
<assert_contents>
<has_text text='"matched": true'/>
<has_text text='"selected_level": 1'/>
<has_text text='"name": "Baden-Wuerttemberg"'/>
<has_text text='"ecoregion_name": "Black Forest"'/>
<has_text text='"biome_name": "Temperate Broadleaf &amp; Mixed Forests"'/>
<has_text text='"is_forest_biome": true'/>
</assert_contents>
</output>
</test>
<test expect_num_outputs="1">
<param name="collection_summary" value="collection_summary.json" ftype="json"/>
<param name="metadata_layers" value="gadm"/>
<output name="additional_metadata" ftype="json">
<assert_contents>
<has_text text='"admin"'/>
<has_text text='"selected_level": 1'/>
<has_text text='"name": "Baden-Wuerttemberg"'/>
<not_has_text text='"ecoregion"'/>
</assert_contents>
</output>
</test>
<test expect_num_outputs="1">
<param name="collection_summary" value="collection_summary.json" ftype="json"/>
<param name="metadata_layers" value="ecoregion"/>
<output name="additional_metadata" ftype="json">
<assert_contents>
<has_text text='"ecoregion_name": "Black Forest"'/>
<has_text text='"is_forest_biome": true'/>
<not_has_text text='"admin"'/>
</assert_contents>
</output>
</test>
</tests>
<help format="markdown"><![CDATA[
**What it does**

This tool reads a 3Dtrees `collection_summary.json`, computes the WGS84 centroid
from `collection.multipolygon_wkt`, and intersects that point with the selected
reference layers. GADM provides administrative context; WWF Terrestrial
Ecoregions v2.0 data provides ecoregion, realm, and biome context.

-----

**Input**

- **Collection summary**: `collection_summary.json` produced by the 3Dtrees
standardization tool in collection mode.
- **Metadata layers to extract**: one or more internal reference layers. The tool
reads the selected reference datasets from the Docker image.

-----

**Output**

The tool writes `additional_metadata.json` with:

- GADM source details and checked layers
- Collection centroid longitude/latitude in EPSG:4326
- Whether the centroid matched a GADM feature
- Selected administrative level and hierarchy
- Raw matched GADM fields for downstream metadata enrichment
- Optional WWF ecoregion name/id, realm, biome code/name, and forest-biome flag
]]></help>
<creator>
<person name="Kilian Gerberding" email="kilian.gerberding@geosense.uni-freiburg.de" url="https://orcid.org/0009-0002-5001-2571"/>
<organization name="3Dtrees-Team, University of Freiburg" url="https://github.com/3dTrees-earth"/>
</creator>
<citations>
<citation type="bibtex">
@misc{3dtrees_metadata, title = {3Dtrees: Metadata}, author = {3Dtrees Project}, year = {2026}}
</citation>
</citations>
</tool>
8 changes: 8 additions & 0 deletions tools/3dtrees_metadata/test-data/collection_summary.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"collection": {
"multipolygon_wkt": "MULTIPOLYGON(((7.84 47.98, 7.86 47.98, 7.86 48.00, 7.84 48.00, 7.84 47.98)))"
},
"n_tiles": 2,
"homogeneous_crs": true,
"homogeneous_attribute_names": true
}
35 changes: 35 additions & 0 deletions tools/3dtrees_metadata/test-data/gadm_fixture.geojson
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"type": "FeatureCollection",
"name": "gadm_fixture",
"crs": {
"type": "name",
"properties": {
"name": "urn:ogc:def:crs:OGC:1.3:CRS84"
}
},
"features": [
{
"type": "Feature",
"properties": {
"GID_0": "DEU",
"COUNTRY": "Germany",
"GID_1": "DEU.1_1",
"NAME_1": "Baden-Wuerttemberg",
"TYPE_1": "Land",
"ENGTYPE_1": "State"
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[7.70, 47.80],
[8.00, 47.80],
[8.00, 48.20],
[7.70, 48.20],
[7.70, 47.80]
]
]
}
}
]
}
34 changes: 34 additions & 0 deletions tools/3dtrees_metadata/test-data/wwf_ecoregions_fixture.geojson
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"type": "FeatureCollection",
"name": "wwf_ecoregions_fixture",
"crs": {
"type": "name",
"properties": {
"name": "urn:ogc:def:crs:OGC:1.3:CRS84"
}
},
"features": [
{
"type": "Feature",
"properties": {
"ECO_NAME": "Black Forest",
"ECO_ID": "PA0414",
"REALM": "PA",
"BIOME": 4,
"BIOME_DESC": "Temperate Broadleaf & Mixed Forests"
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[7.70, 47.80],
[8.00, 47.80],
[8.00, 48.20],
[7.70, 48.20],
[7.70, 47.80]
]
]
}
}
]
}
12 changes: 9 additions & 3 deletions tools/3dtrees_smart_tile/smart_tile.xml
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
<tool id="3dtrees_smart_tile" name="3DTrees: SmartTile" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="24.2">
<description>Create COPC tiles, subsample outputs, filter border overlap, and remap prediction dimensions</description>
<macros>
<token name="@TOOL_VERSION@">2.0.0</token>
<token name="@TOOL_VERSION@">2.0.1</token>
<token name="@VERSION_SUFFIX@">0</token>
<xml name="produce_merged_file_param">
<param argument="--produce-merged-file" type="boolean" truevalue="True" falsevalue="False" checked="true" label="Also write one merged output (LAZ)" help="If enabled (default), produce a single merged LAZ in addition to the per-file outputs."/>
<param name="produce_merged_file" type="select" label="Also write one merged output (LAZ)" help="If enabled (default), produce a single merged LAZ in addition to the per-file outputs.">
<option value="False">No</option>
<option value="True" selected="true">Yes</option>
</param>
</xml>
<xml name="merge_enrichment_conditional">
<conditional name="merge_enrichment_mode">
Expand Down Expand Up @@ -59,6 +62,7 @@
--resolution-1 '$operation.resolution_1'
--resolution-2 '$operation.resolution_2'
--output-copc-res1 $operation.output_copc_res1
--output-copc-res2 $operation.output_copc_res2
--dimension-reduction $operation.dimension_reduction
--chunk-size '$operation.chunk_size'
--num-spatial-chunks \${GALAXY_SLOTS:-4}
Expand Down Expand Up @@ -229,6 +233,7 @@
<param argument="--resolution-1" type="float" min="0.001" max="1.0" value="0.01" label="Subsample resolution 1 (m)" help="First subsampling resolution, in meters."/>
<param argument="--resolution-2" type="float" min="0.001" max="1.0" value="0.1" label="Subsample resolution 2 (m)" help="Second subsampling resolution, in meters."/>
<param argument="--output-copc-res1" type="boolean" truevalue="True" falsevalue="False" checked="true" label="Write resolution 1 as COPC (.copc.laz)" help="If enabled (default), the first subsampled output collection is written as COPC LAZ (`*.copc.laz`). If disabled, it is written as standard LAZ (`*.laz`). This does not affect the internal COPC-normalized source cache."/>
<param argument="--output-copc-res2" type="boolean" truevalue="True" falsevalue="False" checked="false" label="Write resolution 2 as COPC (.copc.laz)" help="If enabled, the second subsampled output collection is written as COPC LAZ (`*.copc.laz`). If disabled (default), it is written as standard LAZ (`*.laz`). This matches the new SmartTile res2 COPC output option."/>
<param argument="--dimension-reduction" type="boolean" truevalue="True" falsevalue="False" checked="true" label="Reduce dimensions in subsampled outputs" help="If enabled (default), subsampled outputs are written with a minimal LAS schema (smaller files). The initial source-to-COPC normalization step still preserves all input dimensions."/>
<param argument="--chunk-size" type="integer" min="100000" max="200000000" value="20000000" label="Chunk size (points)" help="Performance knob. Controls chunked reads used during COPC normalization and later spatial slicing steps."/>
</when>
Expand Down Expand Up @@ -363,7 +368,7 @@
<output_collection name="output_subsampled_res2" type="list" count="1"/>
<output name="output_tiling_preview">
<assert_contents>
<has_image_center_of_mass center_of_mass="1813, 1768" eps="100"/>
<has_image_center_of_mass center_of_mass="1974, 1785" eps="100"/>
</assert_contents>
</output>
<output name="output_tile_bounds_json">
Expand Down Expand Up @@ -449,6 +454,7 @@
- It then creates overlapping tiles and writes two subsampled output collections.
- `Reduce dimensions in subsampled outputs` only affects the subsampled outputs, not the internal COPC cache.
- `Write resolution 1 as COPC (.copc.laz)` only affects the first subsampled output collection (default: enabled).
- `Write resolution 2 as COPC (.copc.laz)` only affects the second subsampled output collection (default: disabled).

**Filter**

Expand Down
54 changes: 53 additions & 1 deletion tools/3dtrees_standardization/standard.xml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<tool id="3dtrees_standardization" name="3DTrees: LAS/LAZ Standardization" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="24.2">
<description>Standardize LAS/LAZ files or validate collections for consistency</description>
<macros>
<token name="@TOOL_VERSION@">1.1.0</token>
<token name="@TOOL_VERSION@">1.2.0</token>
<token name="@VERSION_SUFFIX@">0</token>
</macros>
<requirements>
Expand All @@ -27,6 +27,12 @@
--dataset-path ./input_files/
--output-dir ./
--method collection
#if $mode.approx_lon is not None:
--approx-lon '$mode.approx_lon'
#end if
#if $mode.approx_lat is not None:
--approx-lat '$mode.approx_lat'
#end if
#end if
]]></command>

Expand All @@ -49,6 +55,12 @@
<when value="collection">
<param name="input_collection" type="data" format="laz" multiple="true" label="Point Cloud Collection"
help="Multiple LAZ/LAS files to validate for consistency (CRS homogeneity, attribute consistency, overlaps)"/>
<param argument="--approx-lon" type="float" optional="true" min="-180" max="180"
label="Approximate longitude"
help="Optional WGS84 longitude that overrides the calculated collection centroid when paired with approximate latitude."/>
<param argument="--approx-lat" type="float" optional="true" min="-90" max="90"
label="Approximate latitude"
help="Optional WGS84 latitude that overrides the calculated collection centroid when paired with approximate longitude."/>
</when>
</conditional>
</inputs>
Expand Down Expand Up @@ -90,16 +102,55 @@
</assert_contents>
</output>
</test>
<test expect_num_outputs="1">
<conditional name="mode">
<param name="task" value="collection"/>
<param name="input_collection" value="mikro_no_crs.laz,mikro2_no_crs.laz" ftype="laz"/>
<param name="approx_lon" value="7.85"/>
<param name="approx_lat" value="47.99"/>
</conditional>
<output name="collection_summary" ftype="json">
<assert_contents>
<has_json_property_with_value property="n_tiles" value="2"/>
<has_json_property_with_value property="homogeneous_crs" value="true"/>
<has_json_property_with_value property="homogeneous_attribute_names" value="true"/>
<has_json_property_with_value property="longitude" value="7.85"/>
<has_json_property_with_value property="latitude" value="47.99"/>
<has_json_property_with_text property="crs" text="EPSG:4326"/>
<has_json_property_with_text property="source" text="approximate_input"/>
<has_json_property_with_value property="available" value="true"/>
<has_json_property_with_text property="multipolygon_crs" text="LOCAL_OR_UNKNOWN"/>
<has_json_property_with_text property="multipolygon_source" text="source"/>
<has_json_property_with_value property="multipolygon_available" value="true"/>
<has_json_property_with_value property="multipolygon_wgs84_available" value="false"/>
<has_json_property_with_text property="concave_hull_crs" text="LOCAL_OR_UNKNOWN"/>
<has_json_property_with_text property="concave_hull_source" text="source"/>
</assert_contents>
</output>
</test>
<test expect_num_outputs="1">
<conditional name="mode">
<param name="task" value="collection"/>
<param name="input_collection" value="mikro.laz,mikro2.laz" ftype="laz"/>
<param name="approx_lon" value="7.86"/>
<param name="approx_lat" value="48.01"/>
</conditional>
<output name="collection_summary" ftype="json">
<assert_contents>
<has_json_property_with_value property="n_tiles" value="2"/>
<has_json_property_with_value property="homogeneous_crs" value="true"/>
<has_json_property_with_value property="homogeneous_attribute_names" value="true"/>
<has_json_property_with_value property="longitude" value="7.86"/>
<has_json_property_with_value property="latitude" value="48.01"/>
<has_json_property_with_text property="crs" text="EPSG:4326"/>
<has_json_property_with_text property="source" text="approximate_input"/>
<has_json_property_with_value property="available" value="true"/>
<has_json_property_with_text property="multipolygon_crs" text="EPSG:4326"/>
<has_json_property_with_text property="multipolygon_source" text="wgs84_transform"/>
<has_json_property_with_value property="multipolygon_available" value="true"/>
<has_json_property_with_value property="multipolygon_wgs84_available" value="true"/>
<has_json_property_with_text property="concave_hull_crs" text="EPSG:4326"/>
<has_json_property_with_text property="concave_hull_source" text="wgs84_transform"/>
</assert_contents>
</output>
</test>
Expand Down Expand Up @@ -134,6 +185,7 @@ Validates consistency of multiple LAS/LAZ files:
- **Tile Geometry**: Detects overlaps and gaps between tiles
- **Global Statistics**: Computes aggregated statistics across all files
- **Removeable Attributes**: Identifies attributes that are all-NA or constant-zero
- **Collection Centroid**: Computes a WGS84 centroid from valid collection geometry, or uses optional approximate longitude/latitude as a trusted override when provided

**Outputs (collection mode):**
- ``collection_summary.json`` containing collection-level checks and flags, per-file metadata (CRS, attributes, point counts, convex hulls), and suggested removeable attributes for standardization
Expand Down
Binary file not shown.
Binary file not shown.
Loading