PLU 593: afterFind and Excel multiple lookup#1671
Open
kevinkim-ogp wants to merge 12 commits into
Open
Conversation
…nputs (#1580) ### TL;DR Add `maxRows` property to multirow components to limit the number of rows that can be added. ### What changed? - Added `maxRows` field to the `ActionSubstepArgument` GraphQL schema - Extended `MultiRowProps` and related interfaces to include optional `maxRows` property - Implemented logic to conditionally show the "Add Row" button based on the `maxRows` limit - Updated GraphQL query to fetch the `maxRows` value - Added type definitions for `maxRows` in multirow field interfaces ### How to test? - [ ] Update any action that uses `multirow` or `multirow-multicol` input with the `maxRows` argument - [ ] Verify that you can add rows up to the specified limit - [ ] Verify that the "Add Row" button disappears when the maximum number of rows is reached - [ ] Verify that if `maxRows` is 1, you should not be able to add rows on init - [ ] Verify that current `multirow` and `multirow-multicol` inputs still work as intended - [ ] Can still add rows - [ ] Can still delete rows
### TL;DR Adds support for zero-downtime step parameter migrations via a new `transformStepParameters` hook on app definitions. ### What changed? A new optional `transformStepParameters` function has been added to the `IApp` interface. When defined, this function is automatically invoked in the `$afterFind` lifecycle hook on the `Step` model, transforming stored parameters into the current expected format before they are returned to the frontend or used during execution. ### Why make this change? When parameter formats change for an action, existing steps stored in the database become stale. Previously, this required a database migration to update all affected rows. By applying transformations at read time via `$afterFind`, legacy parameter formats can be seamlessly upgraded on-the-fly, enabling zero-downtime migrations without touching the database. ### Tests _These are just sanity checks that adding this hook does not break Pipe functionality._ - [ ] Verify that existing Step parameters are not modified (no `transformStepParameters` specified for any app yet) - [ ] Verify that executing 'Check step' still works - [ ] Veify that published Pipes still execute and work properly
feat: add version column to steps feat: add version to step in IGlobalVariable feat: add version to step model fix: add version to step
…tep parameters (#1578) ### TL;DR - Adds a parameter transformation layer for M365 Excel actions to migrate old `lookupColumn`/`lookupValue` fields to a new `filters` array format. - When both old and new formats are present, the `filters` array takes precedence and the old `lookupColumn`/`lookupValue` fields are stripped. - A `TRANSFORMABLE_ACTIONS` constant was added to `constants.ts` to define which actions are subject to this transformation. ### Tests _Nothing_ _to_ _test_ _here,_ _the_ _transformation_ _layer_ _does_ _not_ _actually_ _run_ _yet,_ _this_ _PR_ _just_ _creates_ _the_ _function._
) ### TL;DR - Adds a `getLatestVersion` function to fetch the latest version for the step - Refactored `createVersionedStepTransformer` to return both`transformStepParameters` and `getLatestStepVersion` as a coupled pair. This ensures that we expose both functions in the app. ### Tests - [ ] Verify that function makes sense as no transformer has been added yet. _Sanity_ _check_ - [ ] Verify that you can still create steps - [ ] Verify that you can check step - [ ] Verify that you can modify step
### TL;DR Steps now have a `version` assigned at creation time based on the app's `stepTransformer`. ### What changed? When creating a step, the mutation now looks up the app's `stepTransformer` (if one exists) and calls `getLatestStepVersion` with the step key to determine the version to store. If no `appKey` is provided, no `stepTransformer` exists on the app, or no step key is given, the version defaults to `1`. ### How to test? _To actually test, create a stepTransformer for any app_ - [ ] Verify that old steps are modified to the new parameters - [ ] Verify that newly created steps use the latest version number _Sanity_ _check_ - [ ] Verify that you can still create steps - [ ] Verify that you can check step - [ ] Verify that you can modify step
### TL;DR Step parameters are now transformed and versioned on save when an app defines a `stepTransformer`. ### What changed? When `updateStep` is called, it now checks if the app associated with the step defines a `stepTransformer`. If one exists, the step's parameters are passed through `transformStepParameters` using the **DB-stored version** (not the version sent from the frontend), and the step is saved with the latest version returned by `getLatestStepVersion`. If no `stepTransformer` is defined, behavior is unchanged and no `version` field is written to the patch. ### How to test? _How_ _to_ _simulate_ _an_ _old_ _version:_ - Load your app without a `stepTransformer`, open the editor to that action step - Add a `stepTransformer` to your app - Restart the app (make sure the frontend does not reload) - Save the step or check step - [ ] Verify that the step is saved with the latest version and transformed parameters _Sanity_ _check_ - [ ] Verify that updating steps without transformation still works - [ ] Verify that checking steps without transformation still works
#1581) ### What changed? - Expose the step's `key` field in the global variable context available to app actions and triggers. - Prepares for subsequent PRs that will need to access the key via `$.step.key` ### Tests - [ ] Verify that existing actions still work
#1582) ### TL;DR Replaces `lookupColumn` and `lookupValue` fields in M365 Excel lookup actions with `filters` multirow-multicol field, while maintaining backward compatibility via parameter transformation. ### What changed? The `get-table-row`, `get-table-rows`, and `update-table-row` actions now use a single `filters` field of type `multirow-multicol` instead of two separate `lookupColumn` (dropdown) and `lookupValue` (string) fields. The subfield definitions are shared via a new `LOOKUP_CONDITIONS_SUBFIELDS` constant. `transformStepParameters` function is applied at runtime to convert existing step parameters from the old format into the new `filters` array structure before validation, ensuring existing workflows continue to function without data migration. This transformer is also registered at the app level via `transformStepParameters`. The `filters` field is currently capped at `maxRows: 1` to restrict to a single filter condition in this phase. ### Tests _Create_ _a_ _new_ _Get_ _table_ _row_ _action_ - [ ] Verify that you see the new UI - [ ] Verify that the action is saved to the DB with the `filters` key - [ ] Verify that the search works _Create_ _a_ _new_ _Get_ _multiple_ _table_ _rows_ _action_ - [ ] Verify that you see the new UI - [ ] Verify that the action is saved to the DB with the `filters` key - [ ] Verify that the search works _Create_ _a_ _new_ _Update_ _row_ _action_ - [ ] Verify that you see the new UI - [ ] Verify that the action is saved to the DB with the `filters` key - [ ] Verify that the update works _Backward_ _compatibility_ _(create_ _Get_ _table_ _row,_ _Get_ _multiple_ _table_ _rows,_ _and_ _Update_ _table_ _row_ _actions_ _on_ _develop-v2_) first. - [ ] Verify that Get table row searches based on the correct column and value - [ ] Verify that Get table row UI shows the multirow-multicol - [ ] Verify that Get multiple table rows searches based on the correct column and value - [ ] Verify that Get multiple table rows UI shows the multirow-multicol - [ ] Verify that Update table row updates based on the correct column and value
### TL;DR
Update M365 Excel "Get Table Row", "Get Table Rows", and "Update Table Row" actions to support up to 3 lookup filters.
### What changed?
- The `maxRows` limit on the lookup conditions field for "Get Table Row", "Get Table Rows", and "Update Table Row" actions has been increased from 1 to 3, allowing users to specify up to 3 column/value filter conditions.
- `getTableRowImpl` and the "Get Table Rows" action now accept a `filters` array instead of a single `lookupColumn`/`lookupValue` pair. Rows are matched only when **all** filters match (AND logic).
- Column index resolution and row matching logic have been updated to iterate over all provided filters.
- Integration tests for "Get Table Rows" have been updated to use the `filters` array parameter and expanded to cover multi-filter scenarios (all match, partial match, empty string values).
- A new unit test file for `getTableRowImpl` covers single filter, multiple filters, and edge cases (empty rows, empty string values, non-zero header row index, case-sensitive matching).
### How to test?
- [ ] Verify that you can add up to 3 lookup conditions
- [ ] Get table row
- [ ] Get multiple table rows
- [ ] Update table row
- [ ] Verify that the lookup conditions are applied correctly
- [ ] Get table row should only return 1 row
- [ ] Get multiple table rows
- [ ] Update table row
…s validation (#1584) ### TL;DR Extracts the maximum lookup conditions limit (3) into a shared constant and enforces it at the schema validation level. ### What changed? Introduces `MAX_LOOKUP_CONDITIONS` constant and is now used in place of the hardcoded value `3` across the `get-table-row` and `get-table-rows` actions. The `baseLookupParametersSchema` now enforces this limit via a `.max()` validation with a user-facing error message: _"You can only add up to 3 lookup conditions."_ ### How to test? - [ ] Verify that you cannot create more than 3 lookup conditons (will need to call the GraphQL mutation directly) - [ ] Verify that you can still create the action with 1-3 condition
…1640) ### TL;DR Extracted step version resolution logic into a shared `getStepVersion` helper. ### What changed? A new `getStepVersion(appKey, key)` helper was introduced that returns the latest step version from an app's `stepTransformer`, falling back to `1` if the app, key, or transformer is absent. This helper is now used consistently across `create-step`, `createTemplatedFlow`,`duplicate-branch`, `duplicate-flow`, and `create-flow-with-steps` mutations, replacing the previously inline and inconsistent version resolution logic. ### How to test? - [ ] Duplicate step: should use transformed parameters and latest version number - [ ] Duplicate branch: should use transformed parameters and latest version number - [ ] Create step: should use the `getStepVersion` helper can create steps correctly - [ ] Template: should still create templates normally (no template with Excel at the moment, but placing it there first) - [ ] Run the existing test suite to verify no regressions in the affected mutations.
kevinkim-ogp
changed the title
kevinkim-ogp
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
What changed?
AFTER-FIND — Step versioning & parameter transformation infrastructure
Introduces a mechanism to migrate stored step parameters on-the-fly at read time, avoiding the need for bulk database migrations when an action's parameter schema changes.
createVersionedStepTransformer— factory function that returns a coupled (transformStepParameters,getLatestStepVersion) pair, ensuring apps expose both functions together.getStepVersionhelper: shared utility used across create-step, update-step, createTemplatedFlow, duplicate-branch, duplicate-flow, and create-flow-with-steps mutations to consistently resolve and persist the latest version on write.EXCEL-MULTI-LOOKUP — M365 Excel multi-condition lookup
Upgrades the M365 Excel Get Table Row, Get Multiple Table Rows, and Update Table Row actions to support up to 3 AND-combined filter conditions.
lookupColumn/lookupValue— the three lookup actions now use a singlemultirow-multicolfilters field. AtransformStepParametersfunction handles backward compatibility, migrating existing steps from the old two-field format to the new filters array at read time — no data migration required.IGlobalVariable:$.step.keyis now available in the global variable context, enabling key-aware transformers.MAX_LOOKUP_CONDITIONS = 3constant — enforced at the schema validation layer with a user-facing error messagegetTableRowImpland "Get Table Rows" now apply AND logic across all provided filters.Tests
Step versioning
M365 Excel multi-lookup
Deploy notes
Run migration: packages/backend/src/db/migrations/20260430114420_add_steps_version.ts