Skip to content

Add BLN3 measures to fdm database#606

Merged
SvenVw merged 15 commits into
developmentfrom
FDM573
May 12, 2026
Merged

Add BLN3 measures to fdm database#606
SvenVw merged 15 commits into
developmentfrom
FDM573

Conversation

@SvenVw
Copy link
Copy Markdown
Collaborator

@SvenVw SvenVw commented May 6, 2026

Summary by CodeRabbit

  • New Features

    • BLN3 measures catalogue and full measure management (add, list, update, remove) per farm
    • Per‑farm enable/disable for measure catalogues; existing farms auto‑enabled for BLN; new farms enable BLN on creation
  • Tests

    • Extensive unit and integration tests for measures, catalogue sync, hashing, validation, and lifecycle
  • Chores

    • Optional NMI_API_KEY env var and sync support to import measures from NMI API

Closes #573

SvenVw added 5 commits May 6, 2026 13:32
…tMeasuresCatalogue(catalogueName, nmiApiKey)` as a dispatcher (mirroring `getFertilizersCatalogue`), with BLN3 implemented in `measures/catalogues/bln.ts`. Adding future catalogues (e.g. ANLb) only requires a new file and extending the `CatalogueMeasureName` union. Also exports `hashMeasure` and the `CatalogueMeasure`, `CatalogueMeasureItem`, `CatalogueMeasureName` types using pandex naming conventions (`m_id`, `m_source`, `m_name`, etc.).
…sures_catalogue`, `measures`, and `measure_adopting` follow the action-asset model. Exports `addMeasure`, `getMeasure`, `getMeasures`, `getMeasuresForFarm`, `getMeasuresFromCatalogue`, `updateMeasure`, `removeMeasure`, `syncMeasuresCatalogueArray`, `enableMeasureCatalogue`, `disableMeasureCatalogue`, `isMeasureCatalogueEnabled`, `getEnabledMeasureCatalogues`, and the `Measure` / `MeasureCatalogue` types. `syncCatalogues` now accepts an optional `nmiApiKey` to populate the measures catalogue. All existing farms have the `bln` catalogue enabled by default via migration.
@SvenVw SvenVw self-assigned this May 6, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 6, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: f36a6a2a-1d77-4953-99de-81e6e4e60c6b

📥 Commits

Reviewing files that changed from the base of the PR and between b3fe199 and 9df5dbf.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (2)
  • fdm-core/src/measure.test.ts
  • fdm-core/src/measure.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • fdm-core/src/measure.test.ts
  • fdm-core/src/measure.ts

📝 Walkthrough

Walkthrough

The PR implements BLN3 measures: adds DB schema and migration, Drizzle types, full measure CRUD and catalogue sync APIs, fdm-data BLN fetch + hashing, tests, app wiring to enable BLN on farm create, and NMI_API_KEY configuration.

Changes

Measures Feature Implementation

Layer / File(s) Summary
Schema & Database
fdm-core/src/db/schema.ts, fdm-core/src/db/migrations/0028_spotty_greymalkin.sql, fdm-core/src/db/migrations/meta/_journal.json
Adds measures_catalogue, measures, measure_adopting, measure_catalogue_enabling tables, indexes, FKs, and seeds BLN enabling for existing farms.
Drizzle Schema Exports
fdm-core/src/db/schema.ts
Exports TypeSelect/TypeInsert aliases for new tables (measuresCatalogue, measures, measureAdopting, measureCatalogueEnabling).
Public Types & Exports
fdm-core/src/measure.types.ts, fdm-core/src/index.ts, fdm-data/src/index.ts
Adds Measure, MeasureCatalogue types and re-exports measure APIs plus getMeasuresCatalogue / hashMeasure from data.
Core CRUD & Helpers
fdm-core/src/measure.ts
Implements addMeasure, getMeasure, getMeasures, getMeasuresForFarm, getMeasuresFromCatalogue, updateMeasure, removeMeasure, and timeframe helper with permission checks and transactions.
Catalogue Management & Sync
fdm-core/src/catalogues.ts
Adds enableMeasureCatalogue, disableMeasureCatalogue, isMeasureCatalogueEnabled, getEnabledMeasureCatalogues, syncMeasuresCatalogueArray, internal syncMeasuresCatalogue, and updates syncCatalogues to accept { nmiApiKey? }.
fdm-data: BLN & Hashing
fdm-data/src/measures/catalogues/bln.ts, fdm-data/src/measures/hash.ts, fdm-data/src/measures/d.ts, fdm-data/src/measures/index.ts
Implements BLN3 fetch (Bearer token, timeout, response validation, transform), deterministic hashMeasure, catalogue types, and dispatcher getMeasuresCatalogue.
Cascades & Cleanup
fdm-core/src/farm.ts, fdm-core/src/field.ts
Removes measure enabling/adopting/measures rows when farms or fields are deleted to maintain referential integrity.
Tests
fdm-core/src/catalogues.test.ts, fdm-core/src/measure.test.ts, fdm-data/src/measures/hash.test.ts
New and extended tests for catalogue sync, hashing, measure CRUD, and cascade behaviors.
App Integration & Config
fdm-app/app/routes/farm.create._index.tsx, fdm-app/app/lib/fdm-migrate.server.js, docker-compose.yml, fdm-app/.env.example
Farm creation enables BLN measure catalogue; migration reads NMI_API_KEY and passes to sync; docker-compose and .env.example include NMI_API_KEY.
Changesets
.changeset/*.md
Adds/updates changeset entries: fdm-app (patch), fdm-core (minor), fdm-data (minor).

Sequence Diagram

sequenceDiagram
  participant User
  participant App as fdm-app
  participant Core as fdm-core
  participant Data as fdm-data
  participant API as NMI API
  participant DB as Database

  User->>App: Create farm
  App->>Core: enableMeasureCatalogue(fdm, principal, farm,"bln")
  Core->>DB: insert measure_catalogue_enabling

  Migrate->>Core: syncCatalogues(fdm, {nmiApiKey})
  Core->>Data: getMeasuresCatalogue("bln", nmiApiKey)
  Data->>API: HTTP fetch BLN3 (Bearer token)
  API-->>Data: JSON measures
  Data->>Data: hashMeasure items
  Data-->>Core: CatalogueMeasure[]
  Core->>DB: upsert measures_catalogue
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • nmi-agro/fdm#95: Related catalogue-enabling and sync infrastructure extended for measures.
  • nmi-agro/fdm#186: Related cascade deletion changes touching field/farm cleanup logic.
  • nmi-agro/fdm#156: Related app-level farm creation wiring and backend setup patterns.

Suggested labels

fdm-core, fdm-data, fdm-app, enhancement, branch:development

Suggested reviewers

  • BoraIneviNMI
  • gerardhros

"🐰 New measures bloom beneath the sun,
BLN's catalogue now joins the run.
Farms wake, keys pass, hashes align,
Fields and records tidy in line.
Hop, sync, test — the harvest is fine!"

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'Add BLN3 measures to fdm database' accurately describes the main objective of the changeset, which is to implement BLN3 measures support by adding database tables, APIs, and catalogue synchronization.
Linked Issues check ✅ Passed The PR implements all major requirements from issue #573: adds three new DB tables (measures_catalogue, measures, measure_applying) with proper schemas, implements CRUD operations (addMeasure, getMeasure, getMeasures, getMeasuresForFarm, getMeasuresFromCatalogue, updateMeasure, removeMeasure), adds fdm-data measures module with catalogue sync, extends syncCatalogues with nmiApiKey support, and exports all functions/types from index.ts.
Out of Scope Changes check ✅ Passed All changes are scoped to implementing BLN3 measures support: database schema/migrations, measure CRUD operations, catalogue synchronization, type definitions, and related configuration (environment variables, docker-compose setup). No unrelated changes detected outside the stated objectives.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch FDM573

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@sentry
Copy link
Copy Markdown

sentry Bot commented May 6, 2026

Codecov Report

❌ Patch coverage is 95.23810% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
fdm-core/src/db/schema.ts 69.23% 4 Missing ⚠️
fdm-core/src/measure.ts 96.42% 4 Missing ⚠️
fdm-core/src/catalogues.ts 97.72% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@SvenVw SvenVw marked this pull request as ready for review May 6, 2026 13:08
@SvenVw SvenVw changed the title FDM573 @coderabbitai May 6, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

🧹 Nitpick comments (1)
fdm-data/src/measures/hash.test.ts (1)

35-41: ⚡ Quick win

Add an immutability test for hashMeasure input.

After fixing the mutation bug, add a regression test asserting the input object is unchanged after hashing (especially hash).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fdm-data/src/measures/hash.test.ts` around lines 35 - 41, Add a regression
test in hash.test.ts that verifies hashMeasure does not mutate its input: call
hashMeasure with an object whose hash property is null or set to a sentinel,
capture a deep-cloned copy (or the original), then after awaiting hashMeasure
assert the original object's properties (especially the hash field) are
unchanged and equal to the pre-call copy; reference the existing test case that
uses baseMeasure and the hashMeasure function so you extend that block to
include these immutability assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@fdm-core/src/db/migrations/0028_spotty_greymalkin.sql`:
- Around line 39-43: The two new foreign keys that reference farms and fields
are created with ON DELETE NO ACTION, which will block deletions once
measure_catalogue_enabling rows are inserted; update the ALTER TABLE statements
that add constraints measure_adopting_b_id_fields_b_id_fk (on table
measure_adopting column b_id -> fdm.fields.b_id) and
measure_catalogue_enabling_b_id_farm_farms_b_id_farm_fk (on table
measure_catalogue_enabling column b_id_farm -> fdm.farms.b_id_farm) to use ON
DELETE CASCADE (keep ON UPDATE NO ACTION) so deleting a farm or field will
cascade and remove dependent rows instead of failing.

In `@fdm-core/src/db/schema.ts`:
- Around line 973-978: The foreign-key references for b_id and b_id_measure
currently default to NO ACTION; update the Drizzle schema so these references
include an ON DELETE CASCADE behavior (e.g., add onDelete: 'CASCADE' to the
references call for the b_id reference to fields.b_id and for the b_id_measure
reference to measures.b_id_measure), and make the same change for the other
occurrence around lines 994-996 to mirror the SQL migration and allow cascading
deletes of fields/measures/farms.

In `@fdm-core/src/field.ts`:
- Around line 659-676: The code deletes rows from schema.measures for every
measure linked to the removed field (measuresToDelete) but does not ensure those
measures are not still referenced by other rows in schema.measureAdopting;
update the logic in the same transaction (where the current deletes occur) to
only delete measures that have no remaining adopting rows: after deleting from
schema.measureAdopting for the target b_id, compute the set of candidate
measureIds (from measuresToDelete) and issue a delete on schema.measures
restricted to those ids and where NOT EXISTS (or equivalent) a row in
schema.measureAdopting with the same b_id_measure — i.e., only remove measures
with zero remaining references. Use the existing symbols (measuresToDelete,
measureIds, tx, schema.measureAdopting, schema.measures) so the change is
localized.

In `@fdm-core/src/measure.ts`:
- Around line 33-40: Validate and reject inverted date ranges on both write
paths: in addMeasure ensure that when m_end is provided it is not earlier than
m_start (compare m_end.getTime() >= m_start.getTime()) and throw or return a
rejected error if it is; apply the same check to the update measure routine in
this file (the update function covering lines ~342-378) so any update that sets
m_end earlier than m_start is rejected before persisting.
- Around line 33-61: The addMeasure function currently only checks field write
permission and FKs, so callers can still add measures from a catalogue that was
disabled for the farm; before inserting into schema.measures and
schema.measureAdopting you must verify the catalogue is enabled for this b_id
and m_id (the same logic used by disableMeasureCatalogue). Add a check in
addMeasure (e.g., call or inline the enablement query used by
disableMeasureCatalogue) that queries the measures_catalogue/enablement table
for the (b_id, m_id) pair and throw/return an error if the catalogue is
disabled, then proceed with creating b_id_measure and the transaction only when
enabled.

In `@fdm-data/src/measures/catalogues/bln.ts`:
- Around line 25-27: The fetch call in bln.ts that requests
"https://api.nmi-agro.nl/maatwerk/bln3/measures" has no timeout and can block
startup; update the call in the function that performs the catalogue sync (the
code path invoked by syncCatalogues) to use AbortSignal.timeout(...) to create a
signal (e.g. const signal = AbortSignal.timeout(TIMEOUT_MS)) and pass that
signal into fetch options, and ensure you handle the abort by catching the
thrown DOMException/AbortError and logging or throwing a clear timeout error
instead of letting the process hang; keep the existing Authorization header when
adding the signal.
- Around line 33-34: Guard access to json.data before calling .map: after const
json = await res.json(), check that json && Array.isArray(json.data) and if not
either throw a descriptive Error (including response status/URL/context) or
return an empty array; update the code around the json.data.map(...) expression
(the mapping that converts BLN3ApiMeasure items) so it only runs when json.data
is an array and otherwise provides a clear error message referencing the
response and json shape.

In `@fdm-data/src/measures/hash.ts`:
- Around line 4-27: hashMeasure currently mutates the incoming
CatalogueMeasureItem by setting measure.hash = null; instead make hashing pure
by creating a shallow copy (e.g. copy = { ...measure }) and set copy.hash = null
before filtering/sorting, then use that copy in the Object.entries/filter and
when building sortedMeasure; update references in the function (measure → copy)
so the original caller object is not mutated while preserving the same
filtering, sorting, and h32ToString(JSON.stringify(...)) behavior.

---

Nitpick comments:
In `@fdm-data/src/measures/hash.test.ts`:
- Around line 35-41: Add a regression test in hash.test.ts that verifies
hashMeasure does not mutate its input: call hashMeasure with an object whose
hash property is null or set to a sentinel, capture a deep-cloned copy (or the
original), then after awaiting hashMeasure assert the original object's
properties (especially the hash field) are unchanged and equal to the pre-call
copy; reference the existing test case that uses baseMeasure and the hashMeasure
function so you extend that block to include these immutability assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 2636e93c-073a-45ab-91c8-4dae6afa8418

📥 Commits

Reviewing files that changed from the base of the PR and between 0078974 and fe0bdcb.

📒 Files selected for processing (25)
  • .changeset/few-boxes-work.md
  • .changeset/hip-oranges-nail.md
  • .changeset/social-news-like.md
  • docker-compose.yml
  • fdm-app/.env.example
  • fdm-app/app/lib/fdm-migrate.server.js
  • fdm-app/app/routes/farm.create._index.tsx
  • fdm-core/src/catalogues.test.ts
  • fdm-core/src/catalogues.ts
  • fdm-core/src/db/migrations/0028_spotty_greymalkin.sql
  • fdm-core/src/db/migrations/meta/0028_snapshot.json
  • fdm-core/src/db/migrations/meta/_journal.json
  • fdm-core/src/db/schema.ts
  • fdm-core/src/farm.ts
  • fdm-core/src/field.ts
  • fdm-core/src/index.ts
  • fdm-core/src/measure.test.ts
  • fdm-core/src/measure.ts
  • fdm-core/src/measure.types.ts
  • fdm-data/src/index.ts
  • fdm-data/src/measures/catalogues/bln.ts
  • fdm-data/src/measures/d.ts
  • fdm-data/src/measures/hash.test.ts
  • fdm-data/src/measures/hash.ts
  • fdm-data/src/measures/index.ts

Comment thread fdm-core/src/db/migrations/0028_spotty_greymalkin.sql Outdated
Comment thread fdm-core/src/db/schema.ts
Comment thread fdm-core/src/field.ts
Comment thread fdm-core/src/measure.ts
Comment thread fdm-core/src/measure.ts
Comment thread fdm-data/src/measures/catalogues/bln.ts Outdated
Comment thread fdm-data/src/measures/catalogues/bln.ts Outdated
Comment thread fdm-data/src/measures/hash.ts
@SvenVw SvenVw changed the title @coderabbitai Add BLN3 measures to fdm database May 6, 2026
@SvenVw SvenVw requested a review from gerardhros May 7, 2026 08:28
Comment thread fdm-core/src/db/migrations/0028_spotty_greymalkin.sql
Comment thread fdm-core/src/db/schema.ts
Comment thread fdm-core/src/catalogues.ts
Comment thread fdm-core/src/measure.ts
await fdm.transaction(async (tx) => {
await tx
.delete(schema.measureAdopting)
.where(eq(schema.measureAdopting.b_id_measure, b_id_measure))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens if same measure is applied twice in a year with different time windows. is this where criteria then sufficient?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good one, at 9df5dbf I added check for add and update measure to check whether the measure overlaps with same measure in the timeframe

Copy link
Copy Markdown
Collaborator

@gerardhros gerardhros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. See a few minor questions.

@SvenVw SvenVw merged commit a67df6a into development May 12, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add BLN3 measures tables to database schema

2 participants