Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions guides/admin-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ urn: IMF.RES:WEO(6.0.0) # SDMX urn of the dataset
citation: # citation for the dataset
url: https://data.imf.org/en/datasets/IMF.RES:WEO # URL to the dataset web page
provider: IMF.RES # data provider
description: &weo_description > # detailed description of the dataset
description: > # detailed description of the dataset
The World Economic Outlook (WEO) database contains selected macroeconomic
data series from the statistical appendix of the World Economic Outlook
report, which presents the IMF staff's analysis and projections of economic
Expand Down Expand Up @@ -149,8 +149,6 @@ indicatorDimensionsRequiredForQuery: # indicator dimension that required to be f
indexer: # indexer configuration for the dataset
indicator:
unpack: true # whether to unpack the indicator dimensions (e.g. in WEO packed indicators are used, therefore unpack=true)
use_code_list_description: false # whether to use code list description for indexing
description: *weo_description # description to use for indexing
```

Once the dataset is created, you can edit its details by clicking the "Edit" button in the datasets section. This is
Expand Down
7 changes: 4 additions & 3 deletions learning/administration/02-dataset-assessment.md
Original file line number Diff line number Diff line change
Expand Up @@ -447,9 +447,10 @@ Your channel already has **IMF WEO** (World Economic Outlook) onboarded. A stake
- WEO has forecasts that ANEA doesn't
- Overlap exists for headline GDP figures, but ANEA provides more detailed decomposition

**Recommendation:** Onboard ANEA. Document the overlap in both dataset descriptions so the agent can select the right
dataset based on query specificity. Use clear `indexer.description` values: WEO for "broad macroeconomic indicators and
forecasts", ANEA for "detailed national accounts components and breakdowns".
**Recommendation:** Onboard ANEA. Document the overlap in both dataset descriptions
so the agent can select the right dataset based on query specificity. For example,
WEO: "provides broad macroeconomic indicators and forecasts",
ANEA: "provides detailed national accounts components and breakdowns".

</details>

Expand Down
50 changes: 16 additions & 34 deletions learning/administration/03b-indicator-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@

- The concept of required vs. optional indicator dimensions
- The difference between packed and unpacked indicators and the `unpack` indexer setting
- The `useCodeListDescription` indexer setting
- Concrete examples from IMF, Eurostat, ECB, BIS, and other agencies
- How to apply these concepts in practice through exercises

Expand All @@ -16,27 +15,30 @@

Once you've identified all INDICATOR dimensions, decide which are **required**:

- **Required** (`isRequired: true`) — The user's query must specify a filter for at least one
required indicator dimension. Queries without any required indicator dimension filter are rejected.
- **Optional** (no `isRequired`, or `isRequired: false`) — The dimension is optional;
queries can proceed without filtering on it.
- **Required** (`isRequired: true`) — Filtering by this dimension alone is enough to trigger
a query against this dataset.
- **Optional** (no `isRequired`, or `isRequired: false`) — The dimension provides supplementary
context, not sufficient on its own to trigger a query.

A dataset query is **only executed** when the user's query filters on at least one of the dataset's
required dimensions. Queries that don't match any required dimension are skipped.

### The Decision Question

> *"If the user doesn't specify this dimension, can the system still return a meaningful answer?"*
> *"If the user's query specifies a filter for this dimension, should it be enough to trigger a query against this dataset?"*
>
> **Yes** → optional. **No** → required.
> **Yes** → required. **No** → optional.

### Required — Without It, the Query Is Meaningless
### Required — Filtering by This Dimension Produces a Sensible Query

- WEO `INDICATOR` — *"What is [something] for Germany?"* has no meaning without specifying what indicator. Required.
- CPI `INDEX_TYPE` + `COICOP_1999` — without specifying CPI vs. HICP **and** a product category, *"What is inflation?"*
is too vague. Both required.
- WEO: `INDICATOR` — *"What is [something] for Germany?"* filters on `INDICATOR`, and that alone is enough to query WEO. Required.
- CPI: `INDEX_TYPE` + `COICOP_1999` — filtering by either CPI vs. HICP or a product category is enough
to trigger a query against this dataset. Both required.

### Optional — Without It, the System Returns Useful (but Less Specific) Data
### Optional — This Dimension Is Supplementary, Not Sufficient on Its Own

- CPI `TYPE_OF_TRANSFORMATION` — if not specified, the system returns "Index" by default. The answer is still
meaningful, just with a default transformation. Optional.
- CPI `TYPE_OF_TRANSFORMATION` — filtering only by transformation type
should not be enough to trigger a query against this dataset. Optional.
- ECB BSI `ADJUSTMENT` — seasonal adjustment is a refinement, not essential for returning meaningful data. Optional.

### Rules
Expand Down Expand Up @@ -169,24 +171,6 @@ Compare with WEO's `"Gross domestic product, constant prices, Percent change"`

---

## The `useCodeListDescription` Setting

You'll see `useCodeListDescription` in indexer configurations alongside `unpack`:

- **What it does:** When set to `true`, the indexer includes code list item *descriptions* (not just names) in the search index. This gives the semantic and keyword search more text to match against.
- **When to use it:** Currently set to `true` in all IMF dataset configurations. Follow this pattern for new datasets when the provider includes meaningful descriptions in their code lists.
- **Distinct from `indexer.description`:** The `indexer.description` field describes the *entire dataset* (used for dataset selection). `useCodeListDescription` controls whether *individual code list item descriptions* are included in the indicator search index.

```yaml
indexer:
description: "Dataset-level description used for dataset selection" # indexer.description
indicator:
unpack: true
useCodeListDescription: true # includes code list item descriptions in the search index
```

---

## Concrete Examples

### IMF WEO — Simple Structure
Expand Down Expand Up @@ -534,7 +518,6 @@ values, not just the dimension ID.
- Packed indicators combine multiple concepts in comma-separated values → set `unpack: true`
- Unpacked indicators have single-concept values across multiple dimensions → set `unpack: false`
- Commas in natural English descriptions (e.g., "exchange rate, period average") do not indicate packing
- Use `useCodeListDescription: true` when the provider includes meaningful code list descriptions

---

Expand Down Expand Up @@ -618,7 +601,6 @@ Compare with the ECB BSI exercise: BSI's values like "Loans" and "Outstanding am
indexer:
indicator:
unpack: true
useCodeListDescription: true
```

</details>
Expand Down
Loading
Loading