Skip to content

Feature/1681 reverse dependency lookup for deploy#2547

Open
yuliialikhyt wants to merge 11 commits intodevelopfrom
feature/1681-reverse-dependency-lookup-for-deploy
Open

Feature/1681 reverse dependency lookup for deploy#2547
yuliialikhyt wants to merge 11 commits intodevelopfrom
feature/1681-reverse-dependency-lookup-for-deploy

Conversation

@yuliialikhyt
Copy link
Collaborator

@yuliialikhyt yuliialikhyt commented Mar 14, 2026

PR details

Introduced refresh option for assets. Added a function to lookup emails that are referencing the block.

Can be run after deploy or separately. Can refresh triggeredSends linked to an email, or if the user runs the function for a code block (asset-other or asset-block), then the function will lookup emails that are using the code block. The function refreshes emails on the business unit it's been invoked on.

Examples:

mcdev refresh SFMC/YL asset "34b29f13-3bb7-4365-963d-02f2c3d1629b"
mcdev refresh SFMC/_ParentBU_ asset "TEST_HTML_SHARED"    
mcdev d SFMC/YL asset "34b29f13-3bb7-4365-963d-02f2c3d1629b" --fromRetrieve --refresh

What changes did you make? (Give an overview)

Further details (optional)

Tested on a couple of different code blocks (htmlblock, buttonblock, freeformblock and other), on a Parent BU and on a child BU. Tested refresh of a shared block.

Retrieve of published journeys was not necessary, the existing functionality of _refreshTriggeredSend function already validates triggeredSend status. Also, journeys with a status Finishing are not accepting new contacts, but could be still sending emails.

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • test scripts updated
  • Wiki updated (if applicable)

@yuliialikhyt yuliialikhyt changed the base branch from main to develop March 14, 2026 10:26
@yuliialikhyt yuliialikhyt added the c/asset COMPONENT label Mar 14, 2026
@github-actions
Copy link

github-actions bot commented Mar 14, 2026

Coverage Report

Commit:0e10944
Base: develop@f69499e

Type Base This PR
Total Statements Coverage  71.09%  71.28%  (+0.19%)
Total Branches Coverage  70.75%  70.82%  (+0.07%)
Total Functions Coverage  83.81%  83.97%  (+0.16%)
Total Lines Coverage  71.09%  71.28%  (+0.19%)
Details (changed files):
File Statements Branches Functions Lines
lib/metadataTypes/Asset.js  83.62%  73.86%  94.23%  83.62%

@yuliialikhyt yuliialikhyt marked this pull request as ready for review March 22, 2026 11:00
@JoernBerkefeld JoernBerkefeld requested a review from Copilot March 24, 2026 23:46
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the existing --refresh deploy behavior so that refreshing triggered send definitions also works when deploying/updating Content Builder blocks (and other non-email assets) by reverse-looking-up emails that reference those blocks.

Changes:

  • Replaces the post-deploy refresh hook to call a new Asset.refresh() entrypoint instead of refreshing only when the deployed asset is an email.
  • Adds reverse dependency lookup for blocks via an Asset query (MUSTCONTAIN) to find emails referencing the updated block, then refreshes related TriggeredSendDefinitions.
  • Updates/extends the test suite and mock REST/SOAP fixtures to cover the new refresh behavior and request patterns.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
lib/metadataTypes/Asset.js Implements Asset.refresh() and _findEmailsUsingBlock() and wires refresh into postDeployTasks()
@types/lib/metadataTypes/Asset.d.ts Updates typings to include new helper and refresh() return type
@types/lib/metadataTypes/Asset.d.ts.map Regenerated source map for updated typings
test/resourceFactory.js Extends request-to-fixture mapping to support MUSTCONTAIN query operator in POST body
test/type.asset.test.js Adds test case for deploying a block with refresh: true and validates request count
test/type.triggeredSend.test.js Extends expected refreshed triggered send keys and request count
test/type.journey.test.js Updates expected journey count and request count due to additional mocked interaction
test/resources/9999999/triggeredSendDefinition/retrieve-TriggeredSendStatusINdummy,Active-response.xml Adds an additional TriggeredSendDefinition result to mock “Active” retrieval
test/resources/9999999/triggeredSendDefinition/retrieve-CustomerKey=testExistingRefresh_triggeredSend-response.xml New SOAP fixture for retrieving a specific TriggeredSendDefinition by CustomerKey
test/resources/9999999/interaction/v1/interactions/key_testExisting_email_block_refresh/get-response.json New REST fixture for a specific published interaction containing the triggered send key
test/resources/9999999/interaction/v1/interactions/get-response-status=Published.json Extends published interaction listing to include the new interaction
test/resources/9999999/asset/v1/content/assets/query/post-response-contentMUSTCONTAINtestExisting_block_refresh.json New REST fixture for reverse lookup of emails referencing a block key
test/resources/9999999/asset/v1/content/assets/query/post-response-assetType.idIN219,220,221,222,223,224,225,226,227,228,230,232,240,241,242,243,244,245.json Extends existing asset query fixture to include the new email asset
test/resources/9999999/asset/v1/content/assets/334322/patch-response.json New REST fixture for patching the updated block asset
test/mockRoot/deploy/testInstance/testBU/asset/block/testExisting_block_refresh.asset-block-meta.json Adds mock deploy metadata for the test block asset
test/mockRoot/deploy/testInstance/testBU/asset/block/testExisting_block_refresh.asset-block-meta.html Adds mock deploy HTML content for the test block asset

toPublish.push(...Object.values(await this._findEmailsUsingBlock(codeBlockKeys)));
}
Util.logger.info(` - Found ${toPublish.length} assets to publish`);
return toPublish.length ? await this._refreshTriggeredSend(toPublish) : [];
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refresh() passes toPublish (an array) into _refreshTriggeredSend(), but _refreshTriggeredSend() expects a MetadataTypeMap keyed by customerKey and iterates Object.keys(metadata) to access items. With an array, keys become "0", "1", ... which is brittle and can break if callers ever pass a non-array iterable. Consider normalizing toPublish into an object keyed by customerKey (and ideally de-dupe) before calling _refreshTriggeredSend(), or update _refreshTriggeredSend() to explicitly accept an array and iterate items directly.

Suggested change
return toPublish.length ? await this._refreshTriggeredSend(toPublish) : [];
const toPublishMap =
toPublish.length > 0
? toPublish.reduce(
/**
* @param {MetadataTypeMap} acc
* @param {MetadataTypeItem} item
* @returns {MetadataTypeMap}
*/
(acc, item) => {
if (item && item.customerKey) {
acc[item.customerKey] = item;
}
return acc;
},
/** @type {MetadataTypeMap} */ ({})
)
: /** @type {MetadataTypeMap} */ ({});
return Object.keys(toPublishMap).length
? await this._refreshTriggeredSend(toPublishMap)
: [];

Copilot uses AI. Check for mistakes.
Comment on lines +3071 to +3082
static async refresh(keyArr) {
const codeBlockKeys = [];
const toPublish = [];
Util.logger.info(' - Caching dependent Metadata: asset');
const assets = await this.retrieveForCache(undefined, null, undefined, false);
cache.mergeMetadata('asset', assets.metadata);
Util.logger.info(' - Caching dependent Metadata: shared-asset');
const sharedAssets = await this.retrieveForCache(undefined, null, undefined, true);
cache.mergeMetadata('asset', sharedAssets.metadata);
for (const key of keyArr) {
const item = cache.getByKey(this.definition.type, key);
if (!item) {
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refresh() declares keyArr as optional in the JSDoc/typings, but the implementation does for (const key of keyArr) which will throw if keyArr is undefined (e.g., if a caller invokes refresh without keys). Either make the parameter required everywhere (update docs/types) or add a default/guard (treat missing keys as an empty list or throw a clear error).

Copilot uses AI. Check for mistakes.
Comment on lines +922 to +928
payload.query = {
property: 'id',
simpleOperator: 'greaterThan',
value: items.at(-1).id,
};
lastPage = 0;
moreResults = true;
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In _findEmailsUsingBlock(), the pagination fallback for "all shards failed" sets value: items.at(-1).id. If the first response triggers this condition and returns no items, items.at(-1) is undefined and this will throw, aborting the refresh. Add a guard (e.g., only switch to the greaterThan strategy once at least one item has been collected, otherwise log/return).

Suggested change
payload.query = {
property: 'id',
simpleOperator: 'greaterThan',
value: items.at(-1).id,
};
lastPage = 0;
moreResults = true;
if (!items.length) {
// cannot apply greaterThan pagination without at least one item
Util.logger.warn(
'Asset._findEmailsUsingBlock: received "all shards failed" response without any items; stopping pagination.'
);
moreResults = false;
} else {
payload.query = {
property: 'id',
simpleOperator: 'greaterThan',
value: items.at(-1).id,
};
lastPage = 0;
moreResults = true;
}

Copilot uses AI. Check for mistakes.
}
} while (moreResults);
} catch (ex) {
Util.logger.error(`An error occured while attempting to query assets. ${ex.message}`);
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in log message: "An error occured" should be "An error occurred".

Suggested change
Util.logger.error(`An error occured while attempting to query assets. ${ex.message}`);
Util.logger.error(`An error occurred while attempting to query assets. ${ex.message}`);

Copilot uses AI. Check for mistakes.
*
* @private
* @param {string[]} keys metadata keys
* @returns {Promise.<object[]>} - array of assets
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSDoc for _findEmailsUsingBlock() says it returns an array (Promise.<object[]>), but the implementation returns an object map (via Object.fromEntries(...)). Please update the JSDoc/typings to match the actual return type (or change the implementation to return an array).

Suggested change
* @returns {Promise.<object[]>} - array of assets
* @returns {Promise.<AssetMap>} - map of assets keyed by their keyField

Copilot uses AI. Check for mistakes.
@JoernBerkefeld
Copy link
Contributor

two things that caught my attention:

  1. so far, we run refresh on journeys and on triggered send. which, technically, your solution aims to do as well. however, you are now asking to run refresh with the asset key as reference in the command which shall then find relevant journeys. something similar was only possible via deploy --refresh up until now if im not mistaken. you are not refreshing the asset but the journey. maybe this is nothing bad but still worth thinking about.

  2. if i understood your logic correctly it finds emails that use the content block in question (not sure how exactly thats done tbh though). but what it doesnt do is recursively going through content blocks up to emails - or is it?
    if i ran mcdev refresh cred/bu -m asset:myChangedBlock, would it find email "fooBar" if fooBar loads block X which loads block Y which loads myChangedBlock: fooBar > X > Y > myChangedBlock

@JoernBerkefeld JoernBerkefeld added this to the 9.1.0 milestone Mar 25, 2026
@yuliialikhyt
Copy link
Collaborator Author

I've created a refresh function specifically for the Asset type. Journey and TriggeredSend already have similar functionality, no changes were necessary.

Initially, I considered retrieving published journeys as part of this, but I reverted that approach. I realized that the existing TriggeredSend refresh logic already checks whether a TriggeredSend linked to an email activity is Active. So, the process looks up email keys based on code block keys and then runs refresh on the active TriggeredSends associated with those emails.

The logic I've added:

Refresh command on asset:

  1. Cache assets and shared assets (since refresh only receives keys, nothing is cached initially; shared assets are mainly needed for code blocks)
  2. Iterate over the provided keys (these can be a mix of email keys and code block keys)
  3. Look up each key in the cached assets and determine whether it is an email or a code block
  4. If it's an email - add it to the refresh queue
  5. If it's a code block - store it separately and look up emails referencing it
  6. Run the existing logic to refresh TriggeredSends for those emails

Deploy with refresh:

Deploy as usual
Run refresh as part of postDeployTasks - the rest of the process is the same as above

To find emails referencing a content block, I used the POST /asset/v1/content/assets/query endpoint with a condition on the content property. It correctly resolves references even if the block is included via ContentBlockById or ContentBlockByName. The user can pass content block customerKeys as usual. But I see now that it returns some false positives, like when the key also happens to be a part of the word.

I'll refine the condition to something more specific, and add recursion to handle embedded blocks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c/asset COMPONENT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] reverse dependency lookup for deploy --refresh [TASK] investigate minimum scope / permissions required to run mcdev

3 participants