diff --git a/CONTEXT.md b/CONTEXT.md index 114269c..688065f 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -50,6 +50,7 @@ Use these terms consistently in code, docs, tests, and architecture discussions. - **Worktree**: An isolated branch checkout, usually under `.worktrees/`, optionally with its own local runtime state. - **Preflight**: A cached check that validates the API surfaces and credentials needed before longer portal or resource workflows. - **Inventory**: Structured discovery of portal state, such as sites, pages, structures, and templates. +- **Page evidence**: Normalized searchable references found while inspecting a Page, such as Fragment, Widget, Portlet, Structure, Template, Journal article, or Display Page article evidence. - **Portal resource**: A Liferay content artifact managed as a stable CLI workflow: structures, templates, ADTs, and fragments. - **Resource workflow**: A file-based `ldev resource` operation for reading, exporting, importing, or syncing portal resources. - **Resource migration**: A structured workflow for changing journal structures while preserving or cleaning up existing content. @@ -82,12 +83,12 @@ ldev resource migration-init --site /global --structure BASIC Every command should fit one phase of the operational loop: -| Phase | Purpose | Representative commands | -| --- | --- | --- | -| Understand | Resolve project, runtime, and portal state. | `ldev context`, `ldev status`, `ldev portal inventory ...` | -| Diagnose | Localize a failure. | `ldev doctor`, `ldev logs diagnose`, `ldev osgi diag ` | -| Fix | Apply the smallest safe local change. | `ldev deploy module`, `ldev resource import-* --check-only`, then a deliberate mutation | -| Verify | Prove the result with fresh evidence. | `ldev portal check`, `ldev portal inventory ... --json`, `ldev resource structure/template/adt`, `ldev logs diagnose --since 5m` | +| Phase | Purpose | Representative commands | +| ---------- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | +| Understand | Resolve project, runtime, and portal state. | `ldev context`, `ldev status`, `ldev portal inventory ...` | +| Diagnose | Localize a failure. | `ldev doctor`, `ldev logs diagnose`, `ldev osgi diag ` | +| Fix | Apply the smallest safe local change. | `ldev deploy module`, `ldev resource import-* --check-only`, then a deliberate mutation | +| Verify | Prove the result with fresh evidence. | `ldev portal check`, `ldev portal inventory ... --json`, `ldev resource structure/template/adt`, `ldev logs diagnose --since 5m` | Resource verification must be read-after-write. Do not treat log output as sufficient proof that a portal resource changed correctly. diff --git a/docs/agentic/index.md b/docs/agentic/index.md index e6e3335..abb0665 100644 --- a/docs/agentic/index.md +++ b/docs/agentic/index.md @@ -122,11 +122,18 @@ ldev portal inventory sites --json ldev portal inventory pages --site /global --json ldev portal inventory page --url /home --json ldev portal inventory structures --site /global --with-templates --json +ldev portal inventory where-used --type structure --key BASIC --site /global --json ``` For structure/template work, `inventory structures --with-templates` is the right first call. +For impact analysis, use `inventory where-used` after the resource key is known. +It gives agents a task-shaped answer to “which Pages use this fragment, widget, +Structure, Template, or ADT?” before a mutation is proposed. + +Prefer the scoped form with `--site` whenever the Site is already known. + ## Decision route The agent layer follows this rule: diff --git a/docs/commands/discovery.md b/docs/commands/discovery.md index 8400b60..3cef8e0 100644 --- a/docs/commands/discovery.md +++ b/docs/commands/discovery.md @@ -146,6 +146,43 @@ List web content templates for a site. ldev portal inventory templates --site /global --json ``` +## `ldev portal inventory where-used` + +Reverse lookup for portal resources. Use it when you already know the fragment, +widget, Structure, Template, or ADT key and need to answer the practical +question: which Pages use it? + +```bash +ldev portal inventory where-used --type fragment --key card-hero --site /guest +ldev portal inventory where-used --type widget --key com_liferay_journal_content_web_portlet_JournalContentPortlet --site /guest +ldev portal inventory where-used --type structure --key BASIC --site /facultat-farmacia-alimentacio +ldev portal inventory where-used --type adt --key UB_ADT_STUDIES_SEARCH --site /global +ldev portal inventory where-used --type template --key NEWS_TEMPLATE --site /global --include-private --json +``` + +Use `where-used` after discovery has identified the resource you care about. +It scans candidate Pages, extracts normalized Page evidence, and returns only +the Pages whose evidence matches the requested resource. + +Prefer `--site` whenever you already know the owning Site. Without it, +`where-used` scans every accessible Site and can take much longer. + +Options: + +- `--type ` — resource type to trace +- `--key ` — resource key to look up; repeatable +- `--site ` — limit the scan to one Site instead of all accessible Sites +- `--widget-type ` — required when an ADT key is ambiguous across widget types +- `--class-name ` — optional ADT disambiguator for the owning class +- `--include-private` — include private layouts in addition to public Pages +- `--max-depth ` — recursion depth for page hierarchy scans +- `--concurrency ` — concurrent page inspections +- `--page-size ` — page size for candidate collection APIs + +This is especially useful before changing a shared portal resource: it gives you +read-before-write impact analysis without opening the UI or guessing where a +resource might be referenced. + ## `ldev portal audit` Minimal runtime audit of accessible site metadata and API reachability. Defaults to JSON. diff --git a/docs/core-concepts/discovery.md b/docs/core-concepts/discovery.md index 0e1110e..3d4ed23 100644 --- a/docs/core-concepts/discovery.md +++ b/docs/core-concepts/discovery.md @@ -29,6 +29,8 @@ ldev portal inventory sites --json ldev portal inventory pages --site /global --json ldev portal inventory page --url /home --json ldev portal inventory structures --site /global --with-templates --json +ldev portal inventory templates --site /global --json +ldev portal inventory where-used --type structure --key BASIC --site /global --json ``` Each one consolidates information that the Headless API only returns in @@ -42,6 +44,17 @@ fragments. Highlights: - `inventory structures --with-templates` — structures enriched with their associated templates in one call, the right starting point for structure/template work +- `inventory templates` — template inventory when you already know the site and + need template-specific identifiers +- `inventory where-used` — impact analysis for a known fragment, widget, + structure, template, or ADT before proposing a mutation + +For structure/template incidents, prefer `inventory structures --with-templates` +as the first step: it returns both in one call, so you can route directly to +the matching `resource export-*` or `resource import-*` command. + +When possible, add `--site` so the scan stays scoped to one Site instead of all +accessible Sites. ## Preflight: fail fast before long flows diff --git a/docs/workflows/explore-portal.md b/docs/workflows/explore-portal.md index a640b67..432333c 100644 --- a/docs/workflows/explore-portal.md +++ b/docs/workflows/explore-portal.md @@ -19,6 +19,7 @@ Use it when: - you want a fast inventory of sites and pages - you need structured output for automation - an agent needs context before changing anything +- you need to know which Pages use a shared portal resource before changing it ## Start with sites @@ -126,12 +127,33 @@ The benefit is twofold: - an AI agent gets in one tool call what would otherwise need several, with matching JSON shape across MCP and CLI +## Reverse lookup from a resource + +Once you know the resource key, `where-used` gives you the part that the UI is +usually bad at: impact analysis across Pages. + +```bash +ldev portal inventory where-used --type fragment --key card-hero --site /guest --json +ldev portal inventory where-used --type structure --key BASIC --site /guest --json +ldev portal inventory where-used --type adt --key UB_ADT_STUDIES_SEARCH --site /global --json +``` + +Prefer the scoped form with `--site` unless you really need a cross-site scan. + +Use it for questions like: + +- which Pages contain this Fragment +- which Pages render Journal content through this widget +- which Pages depend on this Structure or Template +- which Pages are tied to this ADT before I edit it + ## Typical discovery flow ```bash ldev portal inventory sites --json ldev portal inventory pages --site /global --json ldev portal inventory page --url /home --json +ldev portal inventory where-used --type structure --key BASIC --site /global --json ``` End with the exact page, site and route context you need before you change diff --git a/src/commands/liferay/inventory.command.ts b/src/commands/liferay/inventory.command.ts index feb4840..0806d81 100644 --- a/src/commands/liferay/inventory.command.ts +++ b/src/commands/liferay/inventory.command.ts @@ -33,6 +33,11 @@ import { formatLiferayInventoryTemplates, runLiferayInventoryTemplates, } from '../../features/liferay/inventory/liferay-inventory-templates.js'; +import { + formatLiferayInventoryWhereUsed, + runLiferayInventoryWhereUsed, + type WhereUsedResourceType, +} from '../../features/liferay/inventory/liferay-inventory-where-used.js'; import {formatLiferayPreflight, runLiferayPreflight} from '../../features/liferay/liferay-preflight.js'; function collect(value: string, previous: string[]): string[] { @@ -77,6 +82,22 @@ type InventoryPreflightCommandOptions = { forceRefresh?: boolean; }; +type InventoryWhereUsedCommandOptions = { + type: string; + key: string[]; + site: string[]; + excludeSite: string[]; + widgetType?: string; + className?: string; + includePrivate?: boolean; + siteLimit?: string; + siteOrder?: string; + plan?: boolean; + maxDepth: string; + concurrency: string; + pageSize: string; +}; + export function createInventoryCommands(parent: Command): void { const inventory = new Command('inventory') .helpGroup('Discovery:') @@ -88,11 +109,12 @@ export function createInventoryCommands(parent: Command): void { Use these commands to discover IDs, URLs and keys before running export or import workflows. Commands: - sites List accessible sites - pages List site pages - page Inspect one page - structures List journal structures - templates List web content templates + sites List accessible sites + pages List site pages + page Inspect one page + structures List journal structures + templates List web content templates + where-used Reverse lookup: pages that contain a fragment/widget/structure/template/adt `, ); @@ -313,6 +335,95 @@ Notes: ), ); + addOutputFormatOption( + inventory + .command('where-used') + .description('Reverse-lookup: list every page that contains a given fragment, widget, structure, template or ADT') + .requiredOption('--type ', 'Resource type: fragment | widget | portlet | structure | template | adt') + .option( + '--key ', + 'Resource key to look up (repeat for OR-search across multiple keys)', + collect, + [] as string[], + ) + .option( + '--site ', + 'Limit lookup to one or more sites (repeatable; defaults to scanning all accessible sites)', + collect, + [] as string[], + ) + .option( + '--exclude-site ', + 'Exclude a site when scanning all accessible sites (repeatable)', + collect, + [] as string[], + ) + .option('--widget-type ', 'ADT widget type filter used only when --type adt') + .option('--class-name ', 'ADT class name filter used only when --type adt') + .option('--include-private', 'Also scan private layouts') + .option('--site-limit ', 'Maximum number of sites to scan when --site is not provided') + .option( + '--site-order ', + 'Site prioritization: site | name | content (content is most useful for template|structure lookups)', + 'site', + ) + .option('--plan', 'Show the selected site scan plan and exit without inspecting pages') + .option('--max-depth ', 'Maximum page tree recursion depth', '12') + .option('--concurrency ', 'Parallel page fetches per site', '4') + .option('--page-size ', 'Headless page size for site listings', '200') + .addHelpText( + 'after', + ` +Examples: + ldev portal inventory where-used --type fragment --key card-hero --site /guest + ldev portal inventory where-used --type fragment --key card-hero --site /guest --site /global + ldev portal inventory where-used --type widget --key com_liferay_journal_content_web_portlet_JournalContentPortlet --site /guest + ldev portal inventory where-used --type structure --key BASIC --site /facultat-farmacia-alimentacio + ldev portal inventory where-used --type adt --key UB_ADT_STUDIES_SEARCH --site /global + ldev portal inventory where-used --type template --key NEWS_TEMPLATE --site /global --include-private --json + ldev portal inventory where-used --type template --key UB_TPL_DESTACATS_MULTIMEDIA --site-order content --site-limit 10 --plan + ldev portal inventory where-used --type template --key UB_TPL_DESTACATS_MULTIMEDIA --site-order content --site-limit 10 --exclude-site /global + +Notes: + - Prefer one or more --site values when you already know the owning sites; scanning all accessible sites can take much longer. + - Without --site, use --site-order content plus --site-limit to prioritize the largest content sites first for template|structure lookups. + - For fragment|widget|portlet|adt, keep the default site order unless you have a specific reason to rank by content volume. + - The lookup walks the same data exposed by 'inventory page' so any reference visible there can be matched. + - --key may be repeated to OR-match several keys in a single pass. + - For widget/portlet lookups both the widgetName and the full portletId are matched. + - For ADT lookups the key is resolved through the ADT catalog first, then matched by widget displayStyle on pages. + - --plan resolves and prints the site scan order without fetching page inventories. + - Pages that fail to load (e.g. permission errors) are reported under failedPages without aborting the run. +`, + ), + ).action( + createFormattedAction( + async (context, options: InventoryWhereUsedCommandOptions) => { + const parsedMaxDepth = Number.parseInt(options.maxDepth, 10); + const parsedConcurrency = Number.parseInt(options.concurrency, 10); + const parsedPageSize = Number.parseInt(options.pageSize, 10); + const parsedSiteLimit = options.siteLimit !== undefined ? Number.parseInt(options.siteLimit, 10) : undefined; + + return runLiferayInventoryWhereUsed(context.config, { + type: options.type as WhereUsedResourceType, + keys: options.key, + ...(options.site.length > 0 ? {sites: options.site} : {}), + excludeSites: options.excludeSite, + widgetType: options.widgetType, + className: options.className, + includePrivate: Boolean(options.includePrivate), + ...(parsedSiteLimit !== undefined ? {siteLimit: parsedSiteLimit} : {}), + ...(options.siteOrder ? {siteOrder: options.siteOrder} : {}), + plan: Boolean(options.plan), + maxDepth: Number.isFinite(parsedMaxDepth) ? parsedMaxDepth : 12, + concurrency: Number.isFinite(parsedConcurrency) ? parsedConcurrency : 4, + pageSize: Number.isFinite(parsedPageSize) ? parsedPageSize : 200, + }); + }, + {text: formatLiferayInventoryWhereUsed}, + ), + ); + addOutputFormatOption( inventory .command('preflight') diff --git a/src/features/dashboard/client/components/activity.jsx b/src/features/dashboard/client/components/activity.jsx index 688d0fd..cbbb3bf 100644 --- a/src/features/dashboard/client/components/activity.jsx +++ b/src/features/dashboard/client/components/activity.jsx @@ -88,51 +88,55 @@ export function Activity({ {hiddenCount ? 'All visible activity is hidden right now.' : 'Long-running actions will stream here.'} ) : ( - tasks.map((task) => { - const collapsedTask = taskCollapsed(task); - const lastEntry = task.logs?.[task.logs.length - 1] ?? null; - const leaving = leavingTaskIds.includes(task.id); - return ( -
-
-
-
{task.label}
-
- {taskTime(task.startedAt)} - {task.endedAt ? ` - ${taskTime(task.endedAt)}` : ''} +
+ {tasks.map((task) => { + const collapsedTask = taskCollapsed(task); + const lastEntry = task.logs?.[task.logs.length - 1] ?? null; + const leaving = leavingTaskIds.includes(task.id); + return ( +
+
+
+
+
{task.label}
+ + {task.status === 'succeeded' ? 'done' : task.status} + +
+
+ {taskTime(task.startedAt)} + {task.endedAt ? ` - ${taskTime(task.endedAt)}` : ''} +
+ {collapsedTask && lastEntry?.message ?
{lastEntry.message}
: null}
- {collapsedTask && lastEntry?.message ?
{lastEntry.message}
: null} -
-
- - {task.status === 'running' ? ( - - ) : null} - {!isActiveTask(task) ? ( - - ) : null} - - {task.status === 'succeeded' ? 'done' : task.status} - -
-
-
- {(task.logs || []).map((entry) => ( -
- {taskTime(entry.timestamp)} - {entry.message} + {task.status === 'running' ? ( + + ) : null} + {!isActiveTask(task) ? ( + + ) : null}
- ))} -
-
- ); - }) +
+
+ {(task.logs || []).map((entry) => ( +
+ {taskTime(entry.timestamp)} + {entry.message} +
+ ))} +
+ + ); + })} +
)}
diff --git a/src/features/dashboard/client/components/maintenance.jsx b/src/features/dashboard/client/components/maintenance.jsx index a39ae4f..5194ad8 100644 --- a/src/features/dashboard/client/components/maintenance.jsx +++ b/src/features/dashboard/client/components/maintenance.jsx @@ -1,7 +1,14 @@ import {h} from 'preact'; +import {useState} from 'preact/hooks'; export function Maintenance({maintenance, onApply, onPreview, onSetDays}) { const protectedWorktrees = Array.isArray(maintenance.protected) ? maintenance.protected : []; + const [expanded, setExpanded] = useState(false); + + const handlePreview = () => { + setExpanded(true); + onPreview(maintenance.days); + }; return (
@@ -10,45 +17,52 @@ export function Maintenance({maintenance, onApply, onPreview, onSetDays}) {
Maintenance preview
Find stale worktrees before applying cleanup.
- -
- onSetDays(Number.parseInt(event.currentTarget.value, 10) || 7)} /> - -
- {maintenance.loading ? ( -
Loading maintenance preview...
- ) : maintenance.error ? ( -
Error: {maintenance.error}
- ) : maintenance.candidates.length || protectedWorktrees.length ? ( -
- {maintenance.candidates.length ? ( + {expanded ? ( +
+
+ onSetDays(Number.parseInt(event.currentTarget.value, 10) || 7)} /> + + +
+ {maintenance.loading ? ( +
Loading maintenance preview...
+ ) : maintenance.error ? ( +
Error: {maintenance.error}
+ ) : maintenance.candidates.length || protectedWorktrees.length ? (
-
- Apply GC removes the listed worktree directories and local runtime data. Local branches are kept. -
-
- {maintenance.candidates.map((candidate) => ( - - {candidate} - - ))} -
-
- ) : null} - {protectedWorktrees.length ? ( -
- Protected from GC: {protectedWorktrees.join(', ')} have uncommitted, staged, or untracked changes. + {maintenance.candidates.length ? ( +
+
+ Apply GC removes the listed worktree directories and local runtime data. Local branches are kept. +
+
+ {maintenance.candidates.map((candidate) => ( + + {candidate} + + ))} +
+
+ ) : null} + {protectedWorktrees.length ? ( +
+ Protected from GC: {protectedWorktrees.join(', ')} have uncommitted, staged, or untracked changes. +
+ ) : null}
- ) : null} + ) : ( +
No stale worktrees found.
+ )}
- ) : ( -
No stale worktrees found.
- )} + ) : null}
); } diff --git a/src/features/dashboard/client/lib/dashboard-session.js b/src/features/dashboard/client/lib/dashboard-session.js index 336cff3..6215042 100644 --- a/src/features/dashboard/client/lib/dashboard-session.js +++ b/src/features/dashboard/client/lib/dashboard-session.js @@ -12,7 +12,7 @@ export function useDashboardSession(options = {}) { const [countdown, setCountdown] = useState(20); const [data, setData] = useState(null); const [error, setError] = useState(''); - const [maintenance, setMaintenance] = useState({days: 7, candidates: [], protected: [], loading: false, error: null}); + const [maintenance, setMaintenance] = useState({days: 30, candidates: [], protected: [], loading: false, error: null}); const [searchQuery, setSearchQueryState] = useState(prefs.searchQuery || ''); const [tasks, setTasks] = useState([]); const [toast, setToast] = useState(''); diff --git a/src/features/dashboard/client/styles.css b/src/features/dashboard/client/styles.css index bea1260..3c759a3 100644 --- a/src/features/dashboard/client/styles.css +++ b/src/features/dashboard/client/styles.css @@ -21,9 +21,12 @@ main{padding:16px} .toolbar-search input::placeholder{color:var(--text2)} .toolbar-meta{font-size:12px;color:var(--text2)} .maintenance{background:var(--bg2);border:1px solid var(--border);border-radius:10px;padding:12px;display:flex;flex-direction:column;gap:8px} -.maintenance-header{display:flex;flex-wrap:wrap;align-items:center;gap:8px} +.maintenance-header{display:flex;flex-wrap:wrap;align-items:center;justify-content:space-between;gap:8px} .maintenance-title{font-size:13px;font-weight:700} .maintenance-sub{font-size:11px;color:var(--text2)} +.maintenance-link{background:none;border:none;color:var(--blue);cursor:pointer;font-size:12px;font-weight:600;padding:0} +.maintenance-link:hover{text-decoration:underline} +.maintenance-panel{display:flex;flex-direction:column;gap:8px} .maintenance-controls{display:flex;flex-wrap:wrap;align-items:center;gap:8px} .maintenance-controls input{width:84px;background:var(--bg3);border:1px solid var(--border);color:var(--text);padding:6px 8px;border-radius:7px;font-size:12px} .maintenance-results{display:flex;flex-direction:column;gap:8px} @@ -181,16 +184,21 @@ button.action:disabled{opacity:.45;cursor:not-allowed} .activity-actions{display:flex;align-items:center;gap:6px;flex-wrap:wrap;justify-content:flex-end} .activity-secondary,.activity-toggle{background:none;border:1px solid var(--border);color:var(--text2);padding:4px 8px;border-radius:999px;cursor:pointer;font-size:11px;font-weight:600} .activity-secondary:hover,.activity-toggle:hover{color:var(--text);border-color:var(--blue-border)} -.activity-body{max-height:min(20vh,240px);overflow-x:auto;overflow-y:hidden;padding:10px 12px 12px;display:grid;grid-auto-flow:column;grid-auto-columns:minmax(360px,32vw);gap:10px;scrollbar-width:thin} -.task-card{border:1px solid var(--border);border-radius:12px;background:rgba(13,17,23,.78);overflow:hidden;transform-origin:top center;transition:opacity .18s ease,transform .18s ease,border-color .18s ease,box-shadow .18s ease;height:100%;min-height:0;display:flex;flex-direction:column} +.activity-body{height:20vh;overflow-x:hidden;overflow-y:auto;padding:6px 14px 12px;scrollbar-width:thin} +.activity-list{display:flex;flex-direction:column} +.task-card{position:relative;overflow:hidden;transform-origin:top center;transition:opacity .18s ease,transform .18s ease,background-color .18s ease;height:auto;min-height:0;display:flex;flex-direction:column;padding-left:16px} +.task-card + .task-card{border-top:1px solid rgba(48,54,61,.58)} +.task-card::before{content:'';position:absolute;left:0;top:12px;bottom:12px;width:2px;border-radius:999px;background:rgba(139,148,158,.38)} .task-card.is-leaving{opacity:0;transform:translateY(-8px) scale(.985);pointer-events:none} .task-card.is-collapsed .task-head{border-bottom:none} .task-card.is-collapsed .task-log{grid-template-rows:0fr;opacity:0;padding-top:0;padding-bottom:0} -.task-card.running{border-color:rgba(88,166,255,.35);box-shadow:0 0 0 1px rgba(88,166,255,.08) inset} -.task-card.succeeded{border-color:var(--green-border)} -.task-card.failed{border-color:rgba(248,81,73,.35)} -.task-head{display:flex;align-items:flex-start;justify-content:space-between;gap:10px;padding:10px 11px;border-bottom:1px solid rgba(48,54,61,.7);min-height:72px} +.task-card.running::before{background:rgba(88,166,255,.9)} +.task-card.succeeded::before{background:rgba(63,185,80,.9)} +.task-card.failed::before{background:rgba(248,81,73,.9)} +.task-card.canceling::before{background:rgba(210,153,34,.9)} +.task-head{display:flex;align-items:flex-start;justify-content:space-between;gap:10px;padding:12px 0 8px;min-height:0} .task-head-copy{min-width:0;flex:1} +.task-title-row{display:flex;align-items:flex-start;justify-content:space-between;gap:8px} .task-head-actions{display:flex;align-items:center;gap:6px;flex-shrink:0;flex-wrap:wrap;justify-content:flex-end} .task-toggle,.task-dismiss{background:rgba(139,148,158,.08);border:1px solid rgba(139,148,158,.22);color:var(--text2);padding:4px 7px;border-radius:999px;cursor:pointer;font-size:10px;font-weight:700;text-transform:uppercase} .task-toggle:hover,.task-dismiss:hover{color:var(--text);border-color:var(--blue-border)} @@ -205,13 +213,13 @@ button.action:disabled{opacity:.45;cursor:not-allowed} .task-status.failed{color:var(--red);border-color:var(--red-border);background:var(--red-bg)} .task-status.canceling{color:var(--yellow);border-color:var(--yellow-border);background:rgba(210,153,34,.1)} .task-status.canceled{color:var(--text2);border-color:var(--border);background:rgba(139,148,158,.1)} -.task-log{display:grid;grid-template-rows:1fr;opacity:1;padding:10px 11px;overflow:auto;flex:1;min-height:0;transition:grid-template-rows .2s ease,opacity .16s ease,padding .2s ease} +.task-log{display:grid;grid-template-rows:1fr;opacity:1;padding:0 0 12px;overflow:auto;flex:1;min-height:0;transition:grid-template-rows .2s ease,opacity .16s ease,padding .2s ease;max-height:160px} .task-log > *{min-height:0} .task-log > .task-line:last-child{padding-bottom:0} .task-line{display:grid;grid-template-columns:46px 1fr;gap:8px;font-family:'Cascadia Code','Fira Code','JetBrains Mono',monospace;font-size:11px;line-height:1.5;color:#c9d1d9} .task-line.error .task-msg{color:#ffb3ad} .task-time{color:var(--text2)} -.task-empty{padding:28px 16px;color:var(--text2);font-size:12px;text-align:center;grid-column:1 / -1} +.task-empty{padding:28px 16px;color:var(--text2);font-size:12px;text-align:center} /* Create worktree modal */ .create-form{display:flex;flex-direction:column;gap:12px} @@ -241,7 +249,11 @@ button.action:disabled{opacity:.45;cursor:not-allowed} @media (max-width: 720px){ .layout{gap:12px} .activity{position:static} - .activity-body{max-height:none;grid-auto-columns:minmax(280px,84vw);padding:8px} + .activity-body{height:20vh;padding:6px 10px 10px} + .task-card{padding-left:12px} + .task-head{flex-direction:column;padding:10px 0 8px} + .task-title-row{align-items:center} + .task-head-actions{justify-content:flex-start} header{flex-wrap:wrap} .cwd{max-width:none;width:100%;order:3} } diff --git a/src/features/env/env-start.ts b/src/features/env/env-start.ts index 5f2a17e..2cd8477 100644 --- a/src/features/env/env-start.ts +++ b/src/features/env/env-start.ts @@ -13,6 +13,7 @@ import {ensureActivationKeyPrepared} from './env-activation-key.js'; import {EnvErrors} from './errors/env-error-factory.js'; import {waitForServiceHealthy, waitForPortalReady} from './env-health.js'; import {buildComposeEnv, ensureDoclibVolume, resolveEnvContext, seedBuildDockerConfigs} from './env-files.js'; +import {runEnvStop} from './env-stop.js'; export type EnvStartResult = { ok: true; @@ -53,19 +54,37 @@ export async function runEnvStart( await ensureDoclibVolume(context, {processEnv: options?.processEnv}); const composeEnv = buildComposeEnv(context, {baseEnv: options?.processEnv}); const signal = options?.signal; + const rollbackStartedEnvironment = async () => { + await runEnvStop(config, { + processEnv: options?.processEnv, + }); + }; - if (options?.printer) { - await withProgress(options.printer, 'Starting Docker services', async () => { + try { + if (options?.printer) { + await withProgress(options.printer, 'Starting Docker services', async () => { + await runDockerComposeOrThrow(context.dockerDir, ['up', '-d'], { + env: composeEnv, + signal, + }); + }); + } else { await runDockerComposeOrThrow(context.dockerDir, ['up', '-d'], { env: composeEnv, signal, }); - }); - } else { - await runDockerComposeOrThrow(context.dockerDir, ['up', '-d'], { - env: composeEnv, - signal, - }); + } + } catch (error) { + if (signal?.aborted) { + await rollbackStartedEnvironment(); + } + + throw error; + } + + if (signal?.aborted) { + await rollbackStartedEnvironment(); + throw new Error('Environment start was canceled.'); } if (waitForHealth) { diff --git a/src/features/liferay/content/liferay-content-journal-shared.ts b/src/features/liferay/content/liferay-content-journal-shared.ts index 218b6e9..175247a 100644 --- a/src/features/liferay/content/liferay-content-journal-shared.ts +++ b/src/features/liferay/content/liferay-content-journal-shared.ts @@ -10,6 +10,9 @@ import {normalizeLocalizedName} from '../portal/site-resolution.js'; export type JsonwsJournalArticleRow = { resourcePrimKey?: string; articleId?: string; + urlTitle?: string; + urlTitleCurrentValue?: string; + friendlyURL?: string; folderId?: string; groupId?: string; DDMStructureId?: string; diff --git a/src/features/liferay/inventory/liferay-inventory-display-page-url.ts b/src/features/liferay/inventory/liferay-inventory-display-page-url.ts new file mode 100644 index 0000000..0377b53 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-display-page-url.ts @@ -0,0 +1,44 @@ +import {buildPageUrl} from '../page-layout/liferay-layout-shared.js'; + +export function extractDisplayPageUrlTitle(friendlyUrl: string): string | null { + const candidate = friendlyUrl.startsWith('/') ? friendlyUrl.slice(1) : friendlyUrl; + if (!candidate.startsWith('w/') || candidate.length <= 2) { + return null; + } + + return decodeDisplayPageUrlTitle(candidate.slice(2)); +} + +export function buildDisplayPageFriendlyUrl(urlTitleOrPath: string | undefined): string | null { + const normalizedUrlTitle = normalizeDisplayPageUrlTitle(urlTitleOrPath); + if (!normalizedUrlTitle) { + return null; + } + + return `/w/${normalizedUrlTitle}`; +} + +export function buildDisplayPageUrl(siteFriendlyUrl: string, friendlyUrlPath: string | undefined): string | null { + const friendlyUrl = buildDisplayPageFriendlyUrl(friendlyUrlPath); + if (!friendlyUrl) { + return null; + } + + return buildPageUrl(siteFriendlyUrl, friendlyUrl, false); +} + +function normalizeDisplayPageUrlTitle(urlTitleOrPath: string | undefined): string | null { + const urlTitle = String(urlTitleOrPath ?? '') + .trim() + .replace(/^\/+/, ''); + + return urlTitle || null; +} + +function decodeDisplayPageUrlTitle(urlTitle: string): string { + try { + return decodeURIComponent(urlTitle); + } catch { + return urlTitle; + } +} diff --git a/src/features/liferay/inventory/liferay-inventory-evidence-contract.ts b/src/features/liferay/inventory/liferay-inventory-evidence-contract.ts new file mode 100644 index 0000000..0d8fa5f --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-evidence-contract.ts @@ -0,0 +1,86 @@ +import {z} from 'zod'; + +export type PageEvidenceContext = { + articleId?: string; + articleTitle?: string; + contentStructureId?: number; + contentStructureName?: string; +}; + +export const pageEvidenceResourceTypes = [ + 'fragment', + 'widget', + 'portlet', + 'structure', + 'template', + 'adt', + 'journalArticle', +] as const; + +export const pageEvidenceKinds = [ + 'fragmentEntry', + 'widgetEntry', + 'widgetAdt', + 'portlet', + 'journalArticle', + 'journalArticleStructure', + 'journalArticleTemplate', + 'fragmentMappedStructure', + 'fragmentMappedTemplate', + 'contentStructure', + 'displayPageArticle', +] as const; + +export const whereUsedResourceTypes = ['fragment', 'widget', 'portlet', 'structure', 'template', 'adt'] as const; + +export const whereUsedMatchKinds = [ + 'fragmentEntry', + 'widgetEntry', + 'widgetAdt', + 'portlet', + 'journalArticleStructure', + 'journalArticleTemplate', + 'fragmentMappedStructure', + 'fragmentMappedTemplate', + 'contentStructure', + 'displayPageArticle', +] as const; + +export const pageEvidenceSourceValues = [ + 'fragmentEntryLink', + 'portletLayout', + 'journalArticle', + 'renderedHtmlJournalContent', + 'contentStructure', + 'displayPageArticle', +] as const; + +export type PageEvidenceResourceTypeValue = (typeof pageEvidenceResourceTypes)[number]; +export type PageEvidenceKindValue = (typeof pageEvidenceKinds)[number]; +export type WhereUsedResourceTypeValue = (typeof whereUsedResourceTypes)[number]; +export type WhereUsedMatchKindValue = (typeof whereUsedMatchKinds)[number]; +export type PageEvidenceSourceValue = (typeof pageEvidenceSourceValues)[number]; + +export const pageEvidenceResourceTypeSchema = z.enum(pageEvidenceResourceTypes); +export const pageEvidenceKindSchema = z.enum(pageEvidenceKinds); +export const whereUsedResourceTypeSchema = z.enum(whereUsedResourceTypes); +export const whereUsedMatchKindSchema = z.enum(whereUsedMatchKinds); +export const pageEvidenceSourceSchema = z.enum(pageEvidenceSourceValues); + +export const pageEvidenceContextSchema = z + .object({ + articleId: z.string().optional(), + articleTitle: z.string().optional(), + contentStructureId: z.number().optional(), + contentStructureName: z.string().optional(), + }) + .optional(); + +export const pageEvidenceSchema = z.object({ + resourceType: pageEvidenceResourceTypeSchema, + key: z.string(), + kind: pageEvidenceKindSchema, + detail: z.string(), + source: pageEvidenceSourceSchema, + context: pageEvidenceContextSchema, +}); diff --git a/src/features/liferay/inventory/liferay-inventory-journal-article-resolver.ts b/src/features/liferay/inventory/liferay-inventory-journal-article-resolver.ts new file mode 100644 index 0000000..4643f38 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-journal-article-resolver.ts @@ -0,0 +1,54 @@ +import type {StructuredContent, JournalArticlePayload} from './liferay-inventory-page-assemble.js'; +import type {ArticleRef} from './liferay-inventory-page-fetch-article.js'; +import { + fetchLatestJournalArticle, + fetchStructuredContentById, + fetchStructuredContentByUuid, +} from './liferay-inventory-page-fetch-article.js'; +import {firstString} from './liferay-inventory-page-assemble.js'; +import type {LiferayGateway} from '../liferay-gateway.js'; + +export type ResolvedJournalArticleReference = { + article: JournalArticlePayload | null; + structuredContent: StructuredContent | null; + resolvedArticleId: string; +}; + +export async function resolveJournalArticleReference( + gateway: LiferayGateway, + ref: ArticleRef, + options?: { + article?: JournalArticlePayload | null; + structuredContent?: StructuredContent | null; + }, +): Promise { + let structuredContent = options?.structuredContent ?? null; + if (!structuredContent && ref.structuredContentId && ref.structuredContentId > 0) { + structuredContent = await fetchStructuredContentById(gateway, ref.structuredContentId); + } + + const resolvedArticleId = ref.articleId || structuredContent?.key || ''; + const article = + options?.article ?? + (resolvedArticleId ? await fetchLatestJournalArticle(gateway, ref.groupId, resolvedArticleId) : null); + + if (!structuredContent) { + const uuid = firstString(article?.uuid); + if (uuid) { + structuredContent = await fetchStructuredContentByUuid(gateway, ref.groupId, uuid); + } + } + + if (!structuredContent) { + const structuredContentId = Number(article?.id ?? article?.resourcePrimKey ?? -1); + if (structuredContentId > 0) { + structuredContent = await fetchStructuredContentById(gateway, structuredContentId); + } + } + + return { + article, + structuredContent, + resolvedArticleId, + }; +} diff --git a/src/features/liferay/inventory/liferay-inventory-page-assemble.ts b/src/features/liferay/inventory/liferay-inventory-page-assemble.ts index 413c982..612beec 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-assemble.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-assemble.ts @@ -1,6 +1,7 @@ import {isRecord, type JsonRecord} from '../../../core/utils/json.js'; import {firstNonBlank, firstString as firstStringUtil, normalizeScalarString} from '../../../core/utils/text.js'; import type {HeadlessPageElementPayload} from '../page-layout/liferay-site-page-shared.js'; +import {extractFragmentFieldResources, type FragmentEditableField} from './liferay-inventory-page-fragment-fields.js'; export type StructuredContent = { id?: number; @@ -20,6 +21,7 @@ export type ContentFieldSummary = { }; export type JournalArticleSummary = { + discoverySource?: 'journalArticle' | 'renderedHtmlJournalContent'; groupId?: number; siteFriendlyUrl?: string; siteName?: string; @@ -67,11 +69,6 @@ export type ContentStructureSummary = { exportPath?: string; }; -export type FragmentEditableField = { - id: string; - value: string; -}; - type ContentField = JsonRecord & { name?: unknown; label?: unknown; @@ -98,6 +95,8 @@ export type PageFragmentEntry = { portletId?: string; configuration?: Record; editableFields?: FragmentEditableField[]; + mappedTemplateKeys?: string[]; + mappedStructureKeys?: string[]; contentSummary?: string; title?: string; heroText?: string; @@ -111,7 +110,7 @@ export type PageFragmentEntry = { export function collectPageElements( pageElement: HeadlessPageElementPayload | null, - fragmentEntryLinks: FragmentEntryLink[], + fragmentEntryLinks: FragmentEntryLink[] = [], locale: string | null = null, ): PageFragmentEntry[] { const result: PageFragmentEntry[] = []; @@ -121,6 +120,7 @@ export function collectPageElements( if (entry.type !== 'widget' || !entry.widgetName) { continue; } + const widgetName = entry.widgetName; const match = fragmentEntryLinks.find((item) => (firstStringUtil(item.portletId) ?? '').includes(widgetName)); if (match) { @@ -150,12 +150,18 @@ function collectPageElementsRecursive( const definition = asRecord(element.definition); const key = firstStringUtil(asRecord(definition.fragment).key) ?? ''; if (key) { - const editableFields = extractFragmentEditableFields(definition.fragmentFields, locale); + const fragmentFields = extractFragmentFieldResources(definition.fragmentFields, locale); result.push({ type: 'fragment', fragmentKey: key, configuration: recordToStringMap(asRecord(definition.fragmentConfig)), - ...(editableFields.length > 0 ? {editableFields} : {}), + ...(fragmentFields.editableFields.length > 0 ? {editableFields: fragmentFields.editableFields} : {}), + ...(fragmentFields.mappedTemplateKeys.length > 0 + ? {mappedTemplateKeys: fragmentFields.mappedTemplateKeys} + : {}), + ...(fragmentFields.mappedStructureKeys.length > 0 + ? {mappedStructureKeys: fragmentFields.mappedStructureKeys} + : {}), ...(elementName ? {elementName} : {}), ...(cssClasses && cssClasses.length > 0 ? {cssClasses} : {}), ...(customCSS ? {customCSS} : {}), @@ -253,72 +259,6 @@ function shouldIncludeContentFieldLabelInPath(label: string, name: string): bool return !name.trim().toLowerCase().endsWith('fieldset'); } -function extractFragmentEditableFields(fragmentFields: unknown, locale: string | null = null): FragmentEditableField[] { - if (!Array.isArray(fragmentFields)) { - return []; - } - const result: FragmentEditableField[] = []; - for (const field of fragmentFields) { - const f = asRecord(field); - const id = firstStringUtil(f.id) ?? ''; - if (!id) { - continue; - } - const value = asRecord(f.value); - const text = asRecord(value.text); - const i18n = asRecord(text.value_i18n); - // Prefer the matched locale, then ca_ES, then es_ES, then any available - // TODO: consider improving locale matching logic if needed in the future - // Not hardcoded locales - const textValue = firstNonBlank( - firstStringUtil(locale ? i18n[locale] : undefined), - firstStringUtil(i18n['ca_ES']), - firstStringUtil(i18n['es_ES']), - firstStringUtil(Object.values(i18n)), - firstStringUtil(text.value), - ); - if (textValue) { - result.push({id, value: textValue.replace(/\s+/g, ' ')}); - continue; - } - // Image or document fields - const image = asRecord(value.image); - const fragmentImage = asRecord(value.fragmentImage); - const fragmentImageTitle = asRecord(fragmentImage.title); - const fragmentImageDescription = asRecord(fragmentImage.description); - const fragmentImageUrl = asRecord(fragmentImage.url); - const fragmentImageUrlI18n = asRecord(fragmentImageUrl.value_i18n); - const imageValue = firstNonBlank( - firstStringUtil(image.title), - firstStringUtil(image.description), - firstStringUtil(image.url), - firstStringUtil(image.contentURL), - firstStringUtil(image.src), - firstStringUtil(image.fileEntryId), - firstStringUtil(image.classPK), - firstStringUtil(fragmentImageTitle.value), - firstStringUtil(fragmentImageDescription.value), - firstNonBlank( - firstStringUtil(locale ? fragmentImageUrlI18n[locale] : undefined), - firstStringUtil(fragmentImageUrlI18n['ca_ES']), - firstStringUtil(fragmentImageUrlI18n['es_ES']), - firstStringUtil(Object.values(fragmentImageUrlI18n)), - firstStringUtil(fragmentImageUrl.value), - ), - ); - if (imageValue) { - result.push({id, value: imageValue}); - continue; - } - const document = asRecord(value.document); - const documentValue = firstNonBlank(firstStringUtil(document.title), firstStringUtil(document.url)); - if (documentValue) { - result.push({id, value: documentValue}); - } - } - return result; -} - export function asRecord(value: unknown): JsonRecord { return isRecord(value) ? value : {}; } diff --git a/src/features/liferay/inventory/liferay-inventory-page-evidence-detail.ts b/src/features/liferay/inventory/liferay-inventory-page-evidence-detail.ts new file mode 100644 index 0000000..e81dcfd --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-page-evidence-detail.ts @@ -0,0 +1,70 @@ +import type {ContentStructureSummary, JournalArticleSummary} from './liferay-inventory-page-assemble.js'; +import type {PagePortletSummary} from './liferay-inventory-page.js'; +import type {PageEvidenceContext} from './liferay-inventory-page-evidence.js'; + +export type JournalArticleEvidenceDescriptor = { + where: string; + context: PageEvidenceContext; +}; + +export function describeJournalArticleEvidence( + article: JournalArticleSummary, + structures: ContentStructureSummary[], +): JournalArticleEvidenceDescriptor { + const structure = structures.find( + (candidate) => + (article.contentStructureId && candidate.contentStructureId === article.contentStructureId) || + (article.ddmStructureKey && candidate.key === article.ddmStructureKey) || + (article.ddmStructureKey && candidate.name === article.ddmStructureKey), + ); + + return { + where: buildJournalArticleWhere(article), + context: { + ...(article.articleId ? {articleId: article.articleId} : {}), + ...(article.title ? {articleTitle: article.title} : {}), + ...(article.contentStructureId ? {contentStructureId: article.contentStructureId} : {}), + ...(structure?.name ? {contentStructureName: structure.name} : {}), + }, + }; +} + +export function buildFragmentDetail(fragmentKey: string, elementName: string | undefined, index: number): string { + return [`fragmentKey=${fragmentKey}`, elementName ? `elementName=${elementName}` : null, `index=${index}`] + .filter((value): value is string => value !== null) + .join(' '); +} + +export function buildWidgetDetail( + widgetName: string | undefined, + portletId: string | undefined, + elementName: string | undefined, + index: number, +): string { + return [ + widgetName ? `widgetName=${widgetName}` : null, + portletId ? `portletId=${portletId}` : null, + elementName ? `elementName=${elementName}` : null, + `index=${index}`, + ] + .filter((value): value is string => value !== null) + .join(' '); +} + +export function buildPortletDetail(portlet: PagePortletSummary): string { + return `column=${portlet.columnId} position=${portlet.position} portletId=${portlet.portletId}`; +} + +export function buildJournalArticleStructureDetail(descriptor: JournalArticleEvidenceDescriptor): string { + return [ + descriptor.where, + descriptor.context.contentStructureId ? `contentStructureId=${descriptor.context.contentStructureId}` : null, + descriptor.context.contentStructureName ? `contentStructureName=${descriptor.context.contentStructureName}` : null, + ] + .filter((value): value is string => value !== null) + .join(' '); +} + +function buildJournalArticleWhere(article: JournalArticleSummary): string { + return `articleId=${article.articleId} title=${article.title}`; +} diff --git a/src/features/liferay/inventory/liferay-inventory-page-evidence.ts b/src/features/liferay/inventory/liferay-inventory-page-evidence.ts new file mode 100644 index 0000000..e84e5d8 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-page-evidence.ts @@ -0,0 +1,304 @@ +import type { + PageEvidenceContext as PageEvidenceContextContract, + PageEvidenceKindValue, + PageEvidenceResourceTypeValue, + PageEvidenceSourceValue, +} from './liferay-inventory-evidence-contract.js'; +import type { + ContentStructureSummary, + JournalArticleSummary, + PageFragmentEntry, +} from './liferay-inventory-page-assemble.js'; +import type {LiferayInventoryPageResult, PagePortletSummary} from './liferay-inventory-page.js'; +import { + buildFragmentDetail, + buildJournalArticleStructureDetail, + buildPortletDetail, + buildWidgetDetail, + describeJournalArticleEvidence, +} from './liferay-inventory-page-evidence-detail.js'; + +export type PageEvidenceResourceType = PageEvidenceResourceTypeValue; +export type PageEvidenceKind = PageEvidenceKindValue; +export type PageEvidenceContext = PageEvidenceContextContract; + +export type PageEvidence = { + resourceType: PageEvidenceResourceType; + key: string; + kind: PageEvidenceKind; + detail: string; + source: PageEvidenceSourceValue; + context?: PageEvidenceContext; +}; + +type PageEvidenceInput = { + resourceType: PageEvidenceResourceType; + key: string; + kind: PageEvidenceKind; + detail: string; + source: PageEvidence['source']; + context?: PageEvidenceContext; +}; + +export function extractPageEvidence(page: LiferayInventoryPageResult): PageEvidence[] { + if (page.pageType === 'siteRoot') { + return []; + } + + if (page.evidence) { + return page.evidence; + } + + if (page.pageType === 'displayPage') { + return buildDisplayPageEvidence({ + article: page.article, + journalArticles: page.journalArticles, + contentStructures: page.contentStructures, + }); + } + + return buildRegularPageEvidence({ + fragmentEntryLinks: page.fragmentEntryLinks, + portlets: page.portlets, + journalArticles: page.journalArticles, + contentStructures: page.contentStructures, + }); +} + +export function buildRegularPageEvidence(input: { + fragmentEntryLinks?: PageFragmentEntry[]; + portlets?: PagePortletSummary[]; + journalArticles?: JournalArticleSummary[]; + contentStructures?: ContentStructureSummary[]; +}): PageEvidence[] { + return [ + ...buildFragmentEvidence(input.fragmentEntryLinks ?? []), + ...buildPortletEvidence(input.portlets ?? []), + ...buildJournalArticleEvidence(input.journalArticles ?? [], input.contentStructures ?? []), + ...buildContentStructureEvidence(input.contentStructures ?? []), + ]; +} + +export function buildDisplayPageEvidence(input: { + article: {key: string; contentStructureId: number}; + journalArticles?: JournalArticleSummary[]; + contentStructures?: ContentStructureSummary[]; +}): PageEvidence[] { + return [ + ...buildJournalArticleEvidence(input.journalArticles ?? [], input.contentStructures ?? []), + ...buildContentStructureEvidence(input.contentStructures ?? []), + createPageEvidence({ + resourceType: 'structure', + key: String(input.article.contentStructureId), + kind: 'displayPageArticle', + detail: `displayPage articleKey=${input.article.key} contentStructureId=${input.article.contentStructureId}`, + source: 'displayPageArticle', + context: {articleId: input.article.key, contentStructureId: input.article.contentStructureId}, + }), + ]; +} + +function buildFragmentEvidence(entries: PageFragmentEntry[]): PageEvidence[] { + const evidence: PageEvidence[] = []; + + entries.forEach((entry, index) => { + if (entry.type === 'fragment' && entry.fragmentKey) { + const detail = buildFragmentDetail(entry.fragmentKey, entry.elementName, index); + + evidence.push( + createPageEvidence({ + resourceType: 'fragment', + key: entry.fragmentKey, + kind: 'fragmentEntry', + detail, + source: 'fragmentEntryLink', + }), + ); + + for (const templateKey of entry.mappedTemplateKeys ?? []) { + evidence.push( + createPageEvidence({ + resourceType: 'template', + key: templateKey, + kind: 'fragmentMappedTemplate', + detail, + source: 'fragmentEntryLink', + }), + ); + } + + for (const structureKey of entry.mappedStructureKeys ?? []) { + evidence.push( + createPageEvidence({ + resourceType: 'structure', + key: structureKey, + kind: 'fragmentMappedStructure', + detail, + source: 'fragmentEntryLink', + }), + ); + } + return; + } + + if (entry.type === 'widget') { + const detail = buildWidgetDetail(entry.widgetName, entry.portletId, entry.elementName, index); + const candidates = [entry.widgetName, entry.portletId].filter(isNonEmptyString); + for (const candidate of candidates) { + evidence.push( + createPageEvidence({ + resourceType: 'widget', + key: candidate, + kind: 'widgetEntry', + detail, + source: 'fragmentEntryLink', + }), + ); + } + + appendAdtEvidenceFromConfiguration(evidence, entry.configuration, detail, 'fragmentEntryLink'); + } + }); + + return evidence; +} + +function buildPortletEvidence(portlets: PagePortletSummary[]): PageEvidence[] { + const evidence: PageEvidence[] = []; + + for (const portlet of portlets) { + const detail = buildPortletDetail(portlet); + const candidates = [portlet.portletId, portlet.portletName].filter(isNonEmptyString); + for (const candidate of candidates) { + evidence.push( + createPageEvidence({ + resourceType: 'portlet', + key: candidate, + kind: 'portlet', + detail, + source: 'portletLayout', + }), + ); + } + + appendAdtEvidenceFromConfiguration(evidence, portlet.configuration, detail, 'portletLayout'); + } + + return evidence; +} + +function buildJournalArticleEvidence( + articles: JournalArticleSummary[], + structures: ContentStructureSummary[], +): PageEvidence[] { + const evidence: PageEvidence[] = []; + + for (const article of articles) { + const descriptor = describeJournalArticleEvidence(article, structures); + const source = + article.discoverySource === 'renderedHtmlJournalContent' ? 'renderedHtmlJournalContent' : 'journalArticle'; + + if (article.articleId) { + evidence.push( + createPageEvidence({ + resourceType: 'journalArticle', + key: article.articleId, + kind: 'journalArticle', + detail: descriptor.where, + source, + context: descriptor.context, + }), + ); + } + + if (article.ddmStructureKey) { + evidence.push( + createPageEvidence({ + resourceType: 'structure', + key: article.ddmStructureKey, + kind: 'journalArticleStructure', + detail: buildJournalArticleStructureDetail(descriptor), + source, + context: descriptor.context, + }), + ); + } + + const templateCandidates = [ + article.ddmTemplateKey, + article.widgetDefaultTemplate, + article.widgetHeadlessDefaultTemplate, + ...(article.displayPageDdmTemplates ?? []), + ].filter(isNonEmptyString); + for (const templateKey of templateCandidates) { + evidence.push( + createPageEvidence({ + resourceType: 'template', + key: templateKey, + kind: 'journalArticleTemplate', + detail: descriptor.where, + source, + context: descriptor.context, + }), + ); + } + } + + return evidence; +} + +function buildContentStructureEvidence(structures: ContentStructureSummary[]): PageEvidence[] { + const evidence: PageEvidence[] = []; + + for (const structure of structures) { + const candidates = [structure.key, String(structure.contentStructureId)].filter(isNonEmptyString); + for (const key of candidates) { + evidence.push( + createPageEvidence({ + resourceType: 'structure', + key, + kind: 'contentStructure', + detail: `contentStructureId=${structure.contentStructureId} name=${structure.name}`, + source: 'contentStructure', + context: { + contentStructureId: structure.contentStructureId, + contentStructureName: structure.name, + }, + }), + ); + } + } + + return evidence; +} + +function appendAdtEvidenceFromConfiguration( + evidence: PageEvidence[], + configuration: Record | undefined, + detail: string, + source: PageEvidence['source'], +): void { + const rawDisplayStyle = configuration?.displayStyle; + const displayStyle = typeof rawDisplayStyle === 'string' ? rawDisplayStyle.trim() : undefined; + if (!displayStyle || !displayStyle.startsWith('ddmTemplate_')) { + return; + } + + evidence.push( + createPageEvidence({ + resourceType: 'adt', + key: displayStyle, + kind: 'widgetAdt', + detail: `${detail} displayStyle=${displayStyle}`, + source, + }), + ); +} + +function createPageEvidence(input: PageEvidenceInput): PageEvidence { + return input.context ? {...input} : {...input, context: undefined}; +} + +function isNonEmptyString(value: unknown): value is string { + return typeof value === 'string' && value.length > 0; +} diff --git a/src/features/liferay/inventory/liferay-inventory-page-fetch-article.ts b/src/features/liferay/inventory/liferay-inventory-page-fetch-article.ts index ca6e98f..f5e0c78 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-fetch-article.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-fetch-article.ts @@ -9,7 +9,7 @@ import { } from './liferay-inventory-page-assemble.js'; import {safeGatewayGet} from './liferay-inventory-page-fetch-http.js'; -export type ArticleRef = {articleId: string; groupId: number; ddmTemplateKey?: string}; +export type ArticleRef = {articleId: string; groupId: number; ddmTemplateKey?: string; structuredContentId?: number}; export async function resolveDisplayPageArticle( gateway: LiferayGateway, diff --git a/src/features/liferay/inventory/liferay-inventory-page-fetch-fragments.ts b/src/features/liferay/inventory/liferay-inventory-page-fetch-fragments.ts index 1ef96bd..3ae9f9c 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-fetch-fragments.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-fetch-fragments.ts @@ -17,11 +17,13 @@ export async function tryFetchFragmentEntryLinks( if (plid <= 0) { return []; } + const response = await safeGatewayGet( gateway, `/api/jsonws/fragment.fragmententrylink/get-fragment-entry-links?groupId=${groupId}&plid=${plid}`, 'fetch-fragment-entry-links', ); + return response.ok && Array.isArray(response.data) ? response.data : []; } diff --git a/src/features/liferay/inventory/liferay-inventory-page-fetch-journal.ts b/src/features/liferay/inventory/liferay-inventory-page-fetch-journal.ts index be47e9f..49b9b5b 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-fetch-journal.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-fetch-journal.ts @@ -26,14 +26,10 @@ import {listDdmTemplates, resolveResourceSite} from '../portal/template-queries. import {matchesDdmTemplate} from '../liferay-identifiers.js'; import {resolveSiteToken} from '../portal/site-token.js'; import {tryResolveArtifactSiteDir} from '../portal/artifact-paths.js'; -import { - type ArticleRef, - fetchContentStructureById, - fetchLatestJournalArticle, - fetchStructuredContentById, - fetchStructuredContentByUuid, -} from './liferay-inventory-page-fetch-article.js'; +import {type ArticleRef, fetchContentStructureById} from './liferay-inventory-page-fetch-article.js'; +import {resolveJournalArticleReference} from './liferay-inventory-journal-article-resolver.js'; import {safeGatewayGet} from './liferay-inventory-page-fetch-http.js'; +import type {HeadlessPageElementPayload} from '../page-layout/liferay-site-page-shared.js'; type TemplateInfo = { widgetTemplateCandidates: string[]; @@ -47,9 +43,10 @@ export async function collectLayoutJournalArticles( config: AppConfig, apiClient: HttpApiClient, defaultGroupId: number, - fragmentEntryLinks: FragmentEntryLink[], + pageElement?: HeadlessPageElementPayload | null, + fragmentEntryLinks: FragmentEntryLink[] = [], ): Promise { - const refs = extractArticleRefs(fragmentEntryLinks, defaultGroupId); + const refs = extractArticleRefs(defaultGroupId, pageElement, fragmentEntryLinks); const result: JournalArticleSummary[] = []; for (const ref of refs.values()) { @@ -73,7 +70,10 @@ export async function buildJournalArticleSummary( includeHeadlessInventoryFields?: boolean; }, ): Promise { - const article = options?.article ?? (await fetchLatestJournalArticle(gateway, ref.groupId, ref.articleId)); + const {article, structuredContent, resolvedArticleId} = await resolveJournalArticleReference(gateway, ref, { + article: options?.article, + structuredContent: options?.structuredContent, + }); const articleSite = (await safeFetchGroupInfo(config, ref.groupId, {apiClient, gateway})) ?? (options?.fallbackSite @@ -88,7 +88,7 @@ export async function buildJournalArticleSummary( groupId: ref.groupId, ...(articleSite?.friendlyUrl ? {siteFriendlyUrl: articleSite.friendlyUrl} : {}), ...(articleSite?.name ? {siteName: articleSite.name} : {}), - articleId: ref.articleId, + articleId: resolvedArticleId, title: firstString(article?.titleCurrentValue) ?? firstString(article?.title) ?? options?.fallbackTitle ?? ref.articleId, ddmStructureKey: firstString(article?.ddmStructureKey) ?? '', @@ -96,19 +96,6 @@ export async function buildJournalArticleSummary( ...(options?.fallbackContentStructureId ? {contentStructureId: Number(options.fallbackContentStructureId)} : {}), }; - let structuredContent = options?.structuredContent ?? null; - const uuid = firstString(article?.uuid); - if (!structuredContent && uuid) { - structuredContent = await fetchStructuredContentByUuid(gateway, ref.groupId, uuid); - } - - if (!structuredContent) { - const structuredContentId = Number(article?.id ?? article?.resourcePrimKey ?? -1); - if (structuredContentId > 0) { - structuredContent = await fetchStructuredContentById(gateway, structuredContentId); - } - } - if (structuredContent) { if (options?.includeHeadlessInventoryFields) { enrichJournalArticleWithStructuredContent(summary, structuredContent, ddmTemplateKey); @@ -227,14 +214,18 @@ export async function collectLayoutContentStructures( return result; } -function extractArticleRefs(fragmentEntryLinks: FragmentEntryLink[], defaultGroupId: number): Map { +function extractArticleRefs( + defaultGroupId: number, + pageElement?: HeadlessPageElementPayload | null, + fragmentEntryLinks: FragmentEntryLink[] = [], +): Map { const refs = new Map(); - for (const link of fragmentEntryLinks) { const editableValues = firstString(link.editableValues) ?? ''; if (!editableValues || editableValues === '{}') { continue; } + try { collectArticleRefsFromValue(JSON.parse(editableValues), refs, defaultGroupId); } catch { @@ -242,6 +233,8 @@ function extractArticleRefs(fragmentEntryLinks: FragmentEntryLink[], defaultGrou } } + collectArticleRefsFromValue(pageElement, refs, defaultGroupId); + return refs; } @@ -262,6 +255,19 @@ function collectArticleRefsFromValue(value: unknown, refs: Map, + defaultGroupId: number, + fieldKey?: string, +): void { + if (Object.keys(itemReference).length === 0) { + return; + } + + const contextSource = firstString(itemReference.contextSource); + if (contextSource === 'DisplayPageItem') { + return; + } + + const className = firstString(itemReference.className) ?? firstString(itemReference.itemClassName) ?? ''; + if (className && !className.includes('JournalArticle') && !className.includes('StructuredContent')) { + return; + } + + const articleId = + firstString(itemReference.articleId) ?? firstString(itemReference.key) ?? firstString(itemReference.itemKey); + const structuredContentId = Number( + firstString(itemReference.classPK) ?? + firstString(itemReference.classPk) ?? + firstString(itemReference.id) ?? + firstString(itemReference.itemId) ?? + Number.NaN, + ); + + if (!articleId && (!Number.isFinite(structuredContentId) || structuredContentId <= 0)) { + return; + } + + const groupId = + Number(firstString(itemReference.groupId) ?? firstString(itemReference.siteId) ?? defaultGroupId) || defaultGroupId; + const ddmTemplateKey = extractDdmTemplateKey(fieldKey ?? firstString(itemReference.fieldKey)); + const key = articleId + ? buildArticleRefKey(articleId, groupId) + : `structuredContent:${groupId}:${structuredContentId}`; + upsertArticleRef(refs, key, { + articleId: articleId ?? '', + groupId, + ...(ddmTemplateKey ? {ddmTemplateKey} : {}), + ...(Number.isFinite(structuredContentId) && structuredContentId > 0 ? {structuredContentId} : {}), + }); +} + +function buildArticleRefKey(articleId: string, groupId: number): string { + return `${groupId}:${articleId}`; +} + +function upsertArticleRef(refs: Map, key: string, nextRef: ArticleRef): void { + const previousRef = refs.get(key); + if (!previousRef) { + refs.set(key, nextRef); + return; + } + + refs.set(key, { + articleId: nextRef.articleId || previousRef.articleId, + groupId: nextRef.groupId, + ...(previousRef.ddmTemplateKey || nextRef.ddmTemplateKey + ? {ddmTemplateKey: previousRef.ddmTemplateKey ?? nextRef.ddmTemplateKey} + : {}), + ...(previousRef.structuredContentId || nextRef.structuredContentId + ? {structuredContentId: previousRef.structuredContentId ?? nextRef.structuredContentId} + : {}), + }); +} + +function extractDdmTemplateKey(fieldKey: string | undefined): string | undefined { + const trimmed = fieldKey?.trim(); + if (!trimmed?.startsWith('ddmTemplate_')) { + return undefined; + } + return trimmed.slice('ddmTemplate_'.length).trim() || undefined; +} + async function resolveStructureSiteByKey( gateway: LiferayGateway, config: AppConfig, diff --git a/src/features/liferay/inventory/liferay-inventory-page-fetch.ts b/src/features/liferay/inventory/liferay-inventory-page-fetch.ts index 6bc0dc5..0679e16 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-fetch.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-fetch.ts @@ -18,6 +18,11 @@ import { type PageFragmentEntry, } from './liferay-inventory-page-assemble.js'; import {KNOWN_LOCALES} from './liferay-inventory-page-url.js'; +import { + buildDisplayPageEvidence, + buildRegularPageEvidence, + type PageEvidence, +} from './liferay-inventory-page-evidence.js'; import type { LiferayInventoryPageResult, PagePortletSummary, @@ -25,6 +30,7 @@ import type { } from './liferay-inventory-page.js'; import type {HeadlessSitePagePayload} from '../page-layout/liferay-site-page-shared.js'; import {classNameIdLookupCache} from '../lookup-cache.js'; +import {buildDisplayPageFriendlyUrl, buildDisplayPageUrl} from './liferay-inventory-display-page-url.js'; import {resolveDisplayPageArticle, resolveStructuredContentData} from './liferay-inventory-page-fetch-article.js'; import {safeGatewayGet} from './liferay-inventory-page-fetch-http.js'; import {fetchComponentPageData} from './liferay-inventory-page-fetch-components.js'; @@ -114,8 +120,10 @@ export async function fetchDisplayPageInventory( siteName: site.name, siteFriendlyUrl: site.friendlyUrlPath, groupId: site.id, - url: buildPageUrl(site.friendlyUrlPath, `/w/${urlTitle}`, false), - friendlyUrl: `/w/${urlTitle}`, + url: + buildDisplayPageUrl(site.friendlyUrlPath, urlTitle) ?? + buildPageUrl(site.friendlyUrlPath, `/w/${urlTitle}`, false), + friendlyUrl: buildDisplayPageFriendlyUrl(urlTitle) ?? `/w/${urlTitle}`, article: { id: article.id ?? -1, key: article.key ?? '', @@ -124,6 +132,11 @@ export async function fetchDisplayPageInventory( contentStructureId: Number(article.contentStructureId ?? -1), }, ...(articleAdminUrls ? {adminUrls: articleAdminUrls} : {}), + evidence: buildDisplayPageEvidence({ + article: {key: article.key ?? '', contentStructureId: Number(article.contentStructureId ?? -1)}, + journalArticles: [journalArticle], + contentStructures, + }), journalArticles: [journalArticle], contentStructures, }; @@ -158,6 +171,7 @@ export async function fetchRegularPageInventory( let widgets: Array<{widgetName: string; portletId?: string; configuration?: Record}> = []; let journalArticles: JournalArticleSummary[] = []; let contentStructures: ContentStructureSummary[] = []; + let inheritedEvidence: PageEvidence[] = []; if (componentInspectionSupported) { const { @@ -177,8 +191,40 @@ export async function fetchRegularPageInventory( ...(entry.portletId ? {portletId: entry.portletId} : {}), ...(entry.configuration ? {configuration: entry.configuration} : {}), })); - journalArticles = await collectLayoutJournalArticles(gateway, config, apiClient, site.id, rawFragmentLinks); + journalArticles = await collectLayoutJournalArticles( + gateway, + config, + apiClient, + site.id, + pageElement, + rawFragmentLinks, + ); contentStructures = await collectLayoutContentStructures(gateway, config, apiClient, journalArticles); + inheritedEvidence = await collectMasterLayoutEvidence( + config, + gateway, + apiClient, + site.id, + site.friendlyUrlPath, + privateLayout, + Number(layout.masterLayoutPlid ?? 0), + matchedLocale ?? undefined, + ); + } + + if (['content', 'portlet'].includes(String(layout.type ?? '').toLowerCase())) { + const renderedJournalArticles = await collectRenderedJournalContentArticles( + config, + gateway, + apiClient, + site.id, + pageUrl, + ); + + if (renderedJournalArticles.length > 0) { + journalArticles = mergeJournalArticles(journalArticles, renderedJournalArticles); + contentStructures = await collectLayoutContentStructures(gateway, config, apiClient, journalArticles); + } } function buildRegularPageSummary( @@ -323,6 +369,10 @@ export async function fetchRegularPageInventory( layoutDetails, configurationTabs, componentInspectionSupported, + evidence: [ + ...buildRegularPageEvidence({fragmentEntryLinks, portlets, journalArticles, contentStructures}), + ...inheritedEvidence, + ], portlets, fragmentEntryLinks, widgets, @@ -336,6 +386,175 @@ export async function fetchRegularPageInventory( }; } +async function collectMasterLayoutEvidence( + config: AppConfig, + gateway: LiferayGateway, + apiClient: HttpApiClient, + siteId: number, + siteFriendlyUrl: string, + privateLayout: boolean, + masterLayoutPlid: number, + localeHint?: string, +): Promise { + if (masterLayoutPlid <= 0) { + return []; + } + + const masterLayout = await findLayoutByPlidRecursive(gateway, siteId, privateLayout, 0, masterLayoutPlid); + if (!masterLayout) { + return []; + } + + const masterFriendlyUrl = String(masterLayout.friendlyURL ?? '').trim(); + if (masterFriendlyUrl === '') { + return []; + } + + const {pageElement, rawFragmentLinks} = await fetchComponentPageData( + gateway, + siteId, + masterFriendlyUrl, + Number(masterLayout.plid ?? -1), + ); + const masterFragmentEntryLinks = collectPageElements(pageElement, rawFragmentLinks, localeHint ?? null); + const masterJournalArticles = await collectLayoutJournalArticles( + gateway, + config, + apiClient, + siteId, + pageElement, + rawFragmentLinks, + ); + const masterContentStructures = await collectLayoutContentStructures( + gateway, + config, + apiClient, + masterJournalArticles, + ); + + await enrichFragmentEntryExportPaths(config, gateway, siteFriendlyUrl, masterFragmentEntryLinks, apiClient); + + return buildRegularPageEvidence({ + fragmentEntryLinks: masterFragmentEntryLinks, + journalArticles: masterJournalArticles, + contentStructures: masterContentStructures, + }); +} + +async function collectRenderedJournalContentArticles( + config: AppConfig, + gateway: LiferayGateway, + apiClient: HttpApiClient, + siteId: number, + pageUrl: string, +): Promise { + const html = await fetchRenderedPageHtml(config, apiClient, pageUrl); + if (html === '') { + return []; + } + + const refs = extractRenderedJournalArticleRefs(html); + const results: JournalArticleSummary[] = []; + + for (const ref of refs) { + results.push({ + ...(await buildJournalArticleSummary( + gateway, + config, + apiClient, + {articleId: ref.articleId, groupId: siteId}, + { + fallbackTitle: ref.title ?? ref.articleId, + includeHeadlessInventoryFields: true, + }, + )), + discoverySource: 'renderedHtmlJournalContent', + }); + } + + return results; +} + +async function fetchRenderedPageHtml(config: AppConfig, apiClient: HttpApiClient, pageUrl: string): Promise { + try { + const response = await apiClient.get(config.liferay.url, pageUrl, { + headers: {Accept: 'text/html,application/xhtml+xml'}, + timeoutSeconds: config.liferay.timeoutSeconds, + }); + return response.ok ? response.body : ''; + } catch { + return ''; + } +} + +function extractRenderedJournalArticleRefs(html: string): Array<{articleId: string; title?: string}> { + const refs = new Map(); + const tagPattern = /]*class=["'][^"']*\bjournal-content-article\b[^"']*["'][^>]*>/gi; + + for (const match of html.matchAll(tagPattern)) { + const tag = match[0]; + const articleId = extractHtmlAttribute(tag, 'data-analytics-asset-id')?.trim(); + if (!articleId) { + continue; + } + + const title = extractHtmlAttribute(tag, 'data-analytics-asset-title'); + refs.set(articleId, { + articleId, + ...(title ? {title: decodeRenderedHtmlEntities(title)} : {}), + }); + } + + return [...refs.values()]; +} + +function extractHtmlAttribute(tag: string, attribute: string): string | undefined { + const pattern = new RegExp(`${attribute}=["']([^"']*)["']`, 'i'); + const match = tag.match(pattern); + return match?.[1]; +} + +function decodeRenderedHtmlEntities(value: string): string { + const entityMap: Record = { + ' ': ' ', + '&': '&', + '<': '<', + '>': '>', + '"': '"', + ''': "'", + 'ú': 'ú', + }; + + return value + .replace(/&(nbsp|amp|lt|gt|quot|#39|uacute);/g, (entity) => entityMap[entity] ?? entity) + .replace(/\s+/g, ' ') + .trim(); +} + +function mergeJournalArticles( + existing: JournalArticleSummary[], + discovered: JournalArticleSummary[], +): JournalArticleSummary[] { + const merged = new Map(); + + for (const article of existing) { + merged.set(buildJournalArticleIdentity(article), article); + } + + for (const article of discovered) { + const key = buildJournalArticleIdentity(article); + if (!merged.has(key)) { + merged.set(key, article); + } + } + + return [...merged.values()]; +} + +function buildJournalArticleIdentity(article: JournalArticleSummary): string { + return `${article.groupId ?? -1}:${article.articleId}`; +} + function resolveRegularPageUiType(layoutType: string | undefined): string { const normalized = String(layoutType ?? '') .trim() diff --git a/src/features/liferay/inventory/liferay-inventory-page-fragment-fields.ts b/src/features/liferay/inventory/liferay-inventory-page-fragment-fields.ts new file mode 100644 index 0000000..3029e67 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-page-fragment-fields.ts @@ -0,0 +1,140 @@ +import {isRecord, type JsonRecord} from '../../../core/utils/json.js'; +import {firstNonBlank, firstString as firstStringUtil} from '../../../core/utils/text.js'; + +export type FragmentEditableField = { + id: string; + value: string; +}; + +export type FragmentFieldResources = { + editableFields: FragmentEditableField[]; + mappedTemplateKeys: string[]; + mappedStructureKeys: string[]; +}; + +export function extractFragmentFieldResources( + fragmentFields: unknown, + locale: string | null = null, +): FragmentFieldResources { + const mappedResources = extractFragmentMappedResources(fragmentFields); + return { + editableFields: extractFragmentEditableFields(fragmentFields, locale), + mappedTemplateKeys: mappedResources.templateKeys, + mappedStructureKeys: mappedResources.structureKeys, + }; +} + +function extractFragmentEditableFields(fragmentFields: unknown, locale: string | null): FragmentEditableField[] { + if (!Array.isArray(fragmentFields)) { + return []; + } + const result: FragmentEditableField[] = []; + for (const field of fragmentFields) { + const f = asRecord(field); + const id = firstStringUtil(f.id) ?? ''; + if (!id) { + continue; + } + const value = asRecord(f.value); + const textValue = resolveFragmentTextValue(value, locale); + if (textValue) { + result.push({id, value: textValue.replace(/\s+/g, ' ')}); + continue; + } + + const imageValue = resolveFragmentImageValue(value, locale); + if (imageValue) { + result.push({id, value: imageValue}); + continue; + } + + const document = asRecord(value.document); + const documentValue = firstNonBlank(firstStringUtil(document.title), firstStringUtil(document.url)); + if (documentValue) { + result.push({id, value: documentValue}); + } + } + return result; +} + +function resolveFragmentTextValue(value: JsonRecord, locale: string | null): string { + const text = asRecord(value.text); + const i18n = asRecord(text.value_i18n); + return firstNonBlank( + firstStringUtil(locale ? i18n[locale] : undefined), + firstStringUtil(i18n['ca_ES']), + firstStringUtil(i18n['es_ES']), + firstStringUtil(Object.values(i18n)), + firstStringUtil(text.value), + ); +} + +function resolveFragmentImageValue(value: JsonRecord, locale: string | null): string { + const image = asRecord(value.image); + const fragmentImage = asRecord(value.fragmentImage); + const fragmentImageTitle = asRecord(fragmentImage.title); + const fragmentImageDescription = asRecord(fragmentImage.description); + const fragmentImageUrl = asRecord(fragmentImage.url); + const fragmentImageUrlI18n = asRecord(fragmentImageUrl.value_i18n); + return firstNonBlank( + firstStringUtil(image.title), + firstStringUtil(image.description), + firstStringUtil(image.url), + firstStringUtil(image.contentURL), + firstStringUtil(image.src), + firstStringUtil(image.fileEntryId), + firstStringUtil(image.classPK), + firstStringUtil(fragmentImageTitle.value), + firstStringUtil(fragmentImageDescription.value), + firstNonBlank( + firstStringUtil(locale ? fragmentImageUrlI18n[locale] : undefined), + firstStringUtil(fragmentImageUrlI18n['ca_ES']), + firstStringUtil(fragmentImageUrlI18n['es_ES']), + firstStringUtil(Object.values(fragmentImageUrlI18n)), + firstStringUtil(fragmentImageUrl.value), + ), + ); +} + +function extractFragmentMappedResources(fragmentFields: unknown): {templateKeys: string[]; structureKeys: string[]} { + const templateKeys = new Set(); + const structureKeys = new Set(); + collectMappedResourceKeys(fragmentFields, templateKeys, structureKeys); + return {templateKeys: [...templateKeys], structureKeys: [...structureKeys]}; +} + +function collectMappedResourceKeys(value: unknown, templateKeys: Set, structureKeys: Set): void { + if (Array.isArray(value)) { + for (const item of value) { + collectMappedResourceKeys(item, templateKeys, structureKeys); + } + return; + } + + const record = asRecord(value); + if (Object.keys(record).length === 0) { + return; + } + + const fieldKey = firstStringUtil(record.fieldKey)?.trim(); + if (fieldKey?.startsWith('ddmTemplate_')) { + const templateKey = fieldKey.slice('ddmTemplate_'.length).trim(); + if (templateKey) { + templateKeys.add(templateKey); + } + } + if (fieldKey?.startsWith('ddmStructure_')) { + const structureKey = fieldKey.slice('ddmStructure_'.length).trim(); + if (structureKey) { + structureKeys.add(structureKey); + } + } + + for (const nestedValue of Object.values(record)) { + collectMappedResourceKeys(nestedValue, templateKeys, structureKeys); + } +} + +function asRecord(value: unknown): JsonRecord { + return isRecord(value) ? value : {}; +} diff --git a/src/features/liferay/inventory/liferay-inventory-page-json-schema.ts b/src/features/liferay/inventory/liferay-inventory-page-json-schema.ts index 2b0ef46..1325220 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-json-schema.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-json-schema.ts @@ -1,5 +1,7 @@ import {z} from 'zod'; +import {pageEvidenceSchema} from './liferay-inventory-evidence-contract.js'; + const siteRootJsonSchema = z.object({ page: z.object({ type: z.literal('siteRoot'), @@ -71,6 +73,8 @@ const regularPageJsonSchema = z.object({ fragmentExportPath: z.string().optional(), configuration: z.record(z.string(), z.string()).optional(), contentSummary: z.string().optional(), + mappedTemplateKeys: z.array(z.string()).optional(), + mappedStructureKeys: z.array(z.string()).optional(), }), ) .optional(), @@ -102,6 +106,7 @@ const regularPageJsonSchema = z.object({ z.object({ articleId: z.string(), title: z.string(), + discoverySource: z.enum(['journalArticle', 'renderedHtmlJournalContent']).optional(), groupId: z.number().optional(), siteId: z.number().optional(), siteFriendlyUrl: z.string().optional(), @@ -118,6 +123,7 @@ const regularPageJsonSchema = z.object({ }), ) .optional(), + evidence: z.array(pageEvidenceSchema).optional(), capabilities: z.object({componentInspectionSupported: z.boolean()}).optional(), full: z .object({ @@ -158,6 +164,7 @@ const displayPageJsonSchema = z.object({ title: z.string(), friendlyUrlPath: z.string(), contentStructureId: z.number(), + discoverySource: z.enum(['journalArticle', 'renderedHtmlJournalContent']).optional(), groupId: z.number().optional(), siteId: z.number().optional(), siteFriendlyUrl: z.string().optional(), @@ -202,6 +209,7 @@ const displayPageJsonSchema = z.object({ neverExpire: z.boolean().optional(), }) .optional(), + evidence: z.array(pageEvidenceSchema).optional(), full: z .object({ articleDetails: z diff --git a/src/features/liferay/inventory/liferay-inventory-page-schema.ts b/src/features/liferay/inventory/liferay-inventory-page-schema.ts index d1cc2ba..9d5d726 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-schema.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-schema.ts @@ -1,5 +1,7 @@ import {z} from 'zod'; +import {pageEvidenceSchema} from './liferay-inventory-evidence-contract.js'; + const contentFieldSummarySchema = z.object({ path: z.string(), label: z.string(), @@ -9,6 +11,7 @@ const contentFieldSummarySchema = z.object({ }); const journalArticleSummarySchema = z.object({ + discoverySource: z.enum(['journalArticle', 'renderedHtmlJournalContent']).optional(), groupId: z.number().optional(), siteFriendlyUrl: z.string().optional(), siteName: z.string().optional(), @@ -65,6 +68,8 @@ const pageFragmentEntrySchema = z.object({ portletId: z.string().optional(), configuration: z.record(z.string(), z.string()).optional(), editableFields: z.array(z.object({id: z.string(), value: z.string()})).optional(), + mappedTemplateKeys: z.array(z.string()).optional(), + mappedStructureKeys: z.array(z.string()).optional(), contentSummary: z.string().optional(), title: z.string().optional(), heroText: z.string().optional(), @@ -113,6 +118,7 @@ const displayPageResultSchema = z.object({ translate: z.string(), }) .optional(), + evidence: z.array(pageEvidenceSchema).optional(), journalArticles: z.array(journalArticleSummarySchema).optional(), contentStructures: z.array(contentStructureSummarySchema).optional(), }); @@ -176,6 +182,7 @@ const regularPageResultSchema = z.object({ }) .optional(), componentInspectionSupported: z.boolean().optional(), + evidence: z.array(pageEvidenceSchema).optional(), portlets: z .array( z.object({ diff --git a/src/features/liferay/inventory/liferay-inventory-page-url.ts b/src/features/liferay/inventory/liferay-inventory-page-url.ts index b9cf29e..81083c8 100644 --- a/src/features/liferay/inventory/liferay-inventory-page-url.ts +++ b/src/features/liferay/inventory/liferay-inventory-page-url.ts @@ -1,4 +1,5 @@ import {LiferayErrors} from '../errors/index.js'; +import {extractDisplayPageUrlTitle} from './liferay-inventory-display-page-url.js'; export type InventoryPageOptions = { url?: string; @@ -174,11 +175,3 @@ function normalizeLocale(locale: string): string { } return LOCALE_MAP[locale] ?? locale; } - -function extractDisplayPageUrlTitle(friendlyUrl: string): string | null { - const candidate = friendlyUrl.startsWith('/') ? friendlyUrl.slice(1) : friendlyUrl; - if (!candidate.startsWith('w/') || candidate.length <= 2) { - return null; - } - return candidate.slice(2); -} diff --git a/src/features/liferay/inventory/liferay-inventory-page.ts b/src/features/liferay/inventory/liferay-inventory-page.ts index 1edfc65..e582052 100644 --- a/src/features/liferay/inventory/liferay-inventory-page.ts +++ b/src/features/liferay/inventory/liferay-inventory-page.ts @@ -3,6 +3,7 @@ import type {OAuthTokenClient} from '../../../core/http/auth.js'; import type {HttpApiClient} from '../../../core/http/client.js'; import {createLiferayApiClient} from '../../../core/http/client.js'; import {LiferayErrors} from '../errors/index.js'; +import type {LiferayGateway} from '../liferay-gateway.js'; import {createInventoryGateway} from './liferay-inventory-shared.js'; import {resolveSite} from '../portal/site-resolution.js'; import { @@ -30,6 +31,7 @@ import type { JournalArticleSummary, PageFragmentEntry, } from './liferay-inventory-page-assemble.js'; +import type {PageEvidence} from './liferay-inventory-page-evidence.js'; import type {HeadlessSitePagePayload} from '../page-layout/liferay-site-page-shared.js'; export {resolveInventoryPageRequest}; @@ -39,6 +41,7 @@ export type {LiferayInventoryPageJsonResult} from './liferay-inventory-page-json type InventoryPageDependencies = { apiClient?: HttpApiClient; tokenClient?: OAuthTokenClient; + gateway?: LiferayGateway; }; export type InventoryPageConfigurationGeneral = { @@ -163,6 +166,7 @@ export type LiferayInventoryPageResult = edit: string; translate: string; }; + evidence?: PageEvidence[]; journalArticles?: JournalArticleSummary[]; contentStructures?: ContentStructureSummary[]; } @@ -209,6 +213,7 @@ export type LiferayInventoryPageResult = configurationTabs?: InventoryPageConfigurationTabs; configurationRaw?: InventoryPageConfigurationRaw; componentInspectionSupported?: boolean; + evidence?: PageEvidence[]; portlets?: PagePortletSummary[]; fragmentEntryLinks?: PageFragmentEntry[]; widgets?: Array<{widgetName: string; portletId?: string; configuration?: Record}>; @@ -356,6 +361,12 @@ export function projectLiferayInventoryPageJson( ...(entry.fragmentExportPath ? {fragmentExportPath: entry.fragmentExportPath} : {}), ...(entry.configuration ? {configuration: entry.configuration} : {}), ...(entry.contentSummary ? {contentSummary: entry.contentSummary} : {}), + ...(entry.mappedTemplateKeys && entry.mappedTemplateKeys.length > 0 + ? {mappedTemplateKeys: entry.mappedTemplateKeys} + : {}), + ...(entry.mappedStructureKeys && entry.mappedStructureKeys.length > 0 + ? {mappedStructureKeys: entry.mappedStructureKeys} + : {}), })); const widgets = (result.fragmentEntryLinks ?? []) @@ -397,6 +408,7 @@ export function projectLiferayInventoryPageJson( ...(result.journalArticles && result.journalArticles.length > 0 ? {contentRefs: result.journalArticles.map(projectJournalArticleRef)} : {}), + ...(result.evidence && result.evidence.length > 0 ? {evidence: result.evidence} : {}), ...(fragments.length > 0 || widgets.length > 0 || portlets.length > 0 ? { components: { @@ -472,6 +484,7 @@ function projectDisplayPageJson( title: result.article.title, friendlyUrlPath: result.article.friendlyUrlPath, contentStructureId: result.article.contentStructureId, + ...(articleDetails?.discoverySource ? {discoverySource: articleDetails.discoverySource} : {}), ...(articleDetails?.groupId ? {groupId: articleDetails.groupId} : {}), ...(articleDetails?.siteId ? {siteId: articleDetails.siteId} : {}), ...(articleDetails?.siteFriendlyUrl ? {siteFriendlyUrl: articleDetails.siteFriendlyUrl} : {}), @@ -494,6 +507,7 @@ function projectDisplayPageJson( ...(rendering ? {rendering} : {}), ...(taxonomy ? {taxonomy} : {}), ...(lifecycle ? {lifecycle} : {}), + ...(result.evidence && result.evidence.length > 0 ? {evidence: result.evidence} : {}), }; if (!options?.full) { @@ -550,6 +564,7 @@ function projectJournalArticleRef(article: JournalArticleSummary) { return { articleId: article.articleId, title: article.title, + ...(article.discoverySource ? {discoverySource: article.discoverySource} : {}), ...(article.groupId ? {groupId: article.groupId} : {}), ...(article.siteId ? {siteId: article.siteId} : {}), ...(article.siteFriendlyUrl ? {siteFriendlyUrl: article.siteFriendlyUrl} : {}), diff --git a/src/features/liferay/inventory/liferay-inventory-sites.ts b/src/features/liferay/inventory/liferay-inventory-sites.ts index 904574f..20c16b5 100644 --- a/src/features/liferay/inventory/liferay-inventory-sites.ts +++ b/src/features/liferay/inventory/liferay-inventory-sites.ts @@ -128,7 +128,7 @@ function normalizeSite(row: HeadlessSite): LiferayInventorySite { }; } -function buildPagesCommand(siteFriendlyUrl: string): string { +export function buildPagesCommand(siteFriendlyUrl: string): string { return `inventory pages --site ${siteFriendlyUrl}`; } diff --git a/src/features/liferay/inventory/liferay-inventory-url.ts b/src/features/liferay/inventory/liferay-inventory-url.ts new file mode 100644 index 0000000..7bce209 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-url.ts @@ -0,0 +1,16 @@ +import {buildDisplayPageUrl as buildDisplayPageUrlInternal} from './liferay-inventory-display-page-url.js'; + +export function buildPortalAbsoluteUrl(baseUrl: string | undefined, pathOrUrl: string): string | undefined { + if (!baseUrl) { + return undefined; + } + try { + return new URL(pathOrUrl, baseUrl).toString(); + } catch { + return undefined; + } +} + +export function buildDisplayPageUrl(siteFriendlyUrl: string, friendlyUrlPath: string | undefined): string | null { + return buildDisplayPageUrlInternal(siteFriendlyUrl, friendlyUrlPath); +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-display-pages.ts b/src/features/liferay/inventory/liferay-inventory-where-used-display-pages.ts new file mode 100644 index 0000000..1f7912a --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-display-pages.ts @@ -0,0 +1,216 @@ +import type {AppConfig} from '../../../core/config/load-config.js'; +import {mapConcurrent} from '../../../core/concurrency.js'; +import {isCliError} from '../../../core/errors.js'; +import {createLiferayApiClient} from '../../../core/http/client.js'; +import type {OAuthTokenClient} from '../../../core/http/auth.js'; +import type {HttpApiClient} from '../../../core/http/client.js'; +import { + fetchJournalArticleRowsInFolder, + fetchJournalFoldersByParent, + type JsonwsJournalArticleRow, +} from '../content/liferay-content-journal-shared.js'; +import type {LiferayGateway} from '../liferay-gateway.js'; +import {LookupCache} from '../lookup-cache.js'; +import {createInventoryGateway, fetchPagedItems} from './liferay-inventory-shared.js'; +import {buildDisplayPageUrl} from './liferay-inventory-url.js'; +import type {LiferayInventorySite} from './liferay-inventory-sites.js'; + +type StructuredContentListItem = { + friendlyUrlPath?: string; +}; + +export type DisplayPageCandidate = { + fullUrl: string; + origin: 'headlessStructuredContent' | 'jsonwsJournal'; +}; + +export type DisplayPageSource = { + origin: DisplayPageCandidate['origin']; + collect: ( + config: AppConfig, + site: LiferayInventorySite, + options: WhereUsedDisplayPageScanOptions, + ) => Promise; +}; + +export type WhereUsedDisplayPageScanOptions = { + concurrency: number; + pageSize: number; + dependencies: { + apiClient?: HttpApiClient; + tokenClient?: OAuthTokenClient; + }; +}; + +type DisplayPageSourceCollectionResult = + | {kind: 'collected'; candidates: DisplayPageCandidate[]} + | {kind: 'unsupported'}; + +const DISPLAY_PAGE_SOURCES: DisplayPageSource[] = [ + {origin: 'headlessStructuredContent', collect: collectHeadlessStructuredContentDisplayPages}, + {origin: 'jsonwsJournal', collect: collectJsonwsJournalDisplayPages}, +]; + +const unsupportedDisplayPageSourceCache = new LookupCache({ttlMs: 3_600_000}); + +export async function collectDisplayPageCandidates( + config: AppConfig, + site: LiferayInventorySite, + options: WhereUsedDisplayPageScanOptions, +): Promise { + return collectDisplayPageCandidatesFromSources(config, site, options, DISPLAY_PAGE_SOURCES); +} + +export async function collectDisplayPageCandidatesFromSources( + config: AppConfig, + site: LiferayInventorySite, + options: WhereUsedDisplayPageScanOptions, + sources: DisplayPageSource[], +): Promise { + const candidates: DisplayPageCandidate[] = []; + for (const source of sources) { + if (isUnsupportedDisplayPageSourceCached(config, site, source.origin)) { + continue; + } + + const result = await collectDisplayPageCandidatesFromSource(config, site, options, source); + if (result.kind === 'unsupported') { + cacheUnsupportedDisplayPageSource(config, site, source.origin); + continue; + } + + candidates.push(...result.candidates); + } + return dedupeDisplayPageCandidates(candidates); +} + +async function collectDisplayPageCandidatesFromSource( + config: AppConfig, + site: LiferayInventorySite, + options: WhereUsedDisplayPageScanOptions, + source: DisplayPageSource, +): Promise { + try { + return { + kind: 'collected', + candidates: await source.collect(config, site, options), + }; + } catch (error) { + if (isUnsupportedDisplayPageScanError(source.origin, error)) { + return {kind: 'unsupported'}; + } + throw error; + } +} + +async function collectHeadlessStructuredContentDisplayPages( + config: AppConfig, + site: LiferayInventorySite, + options: WhereUsedDisplayPageScanOptions, +): Promise { + const structuredContents = await fetchPagedItems( + config, + `/o/headless-delivery/v1.0/sites/${site.groupId}/structured-contents`, + options.pageSize, + options.dependencies, + ); + + return structuredContents + .map((item) => buildDisplayPageUrl(site.siteFriendlyUrl, item.friendlyUrlPath)) + .filter((fullUrl): fullUrl is string => fullUrl !== null) + .map((fullUrl) => ({fullUrl, origin: 'headlessStructuredContent'})); +} + +async function collectJsonwsJournalDisplayPages( + config: AppConfig, + site: LiferayInventorySite, + options: WhereUsedDisplayPageScanOptions, +): Promise { + const apiClient = options.dependencies.apiClient ?? createLiferayApiClient(); + const gateway = createInventoryGateway(config, apiClient, options.dependencies); + const folderIds = await collectJournalFolderIds(gateway, site.groupId, 0, new Set([0])); + const pages = await mapConcurrent(folderIds, Math.max(1, Math.min(options.concurrency, 4)), async (folderId) => { + const rows = await fetchJournalArticleRowsInFolder(gateway, site.groupId, folderId); + return rows + .filter((row) => row.status === undefined || Number(row.status) === 0) + .map((row) => buildDisplayPageUrl(site.siteFriendlyUrl, resolveJournalArticleUrlTitle(row))) + .filter((fullUrl): fullUrl is string => fullUrl !== null) + .map((fullUrl) => ({fullUrl, origin: 'jsonwsJournal' as const})); + }); + + return pages.flat(); +} + +async function collectJournalFolderIds( + gateway: LiferayGateway, + groupId: number, + parentFolderId: number, + seen: Set, +): Promise { + const folders = await fetchJournalFoldersByParent(gateway, groupId, parentFolderId); + const childIds: number[] = []; + + for (const folder of folders) { + if (seen.has(folder.folderId)) { + continue; + } + seen.add(folder.folderId); + childIds.push(folder.folderId); + childIds.push(...(await collectJournalFolderIds(gateway, groupId, folder.folderId, seen))); + } + + return parentFolderId === 0 ? [0, ...childIds] : childIds; +} + +function dedupeDisplayPageCandidates(candidates: DisplayPageCandidate[]): DisplayPageCandidate[] { + const unique = new Map(); + for (const candidate of candidates) { + if (!unique.has(candidate.fullUrl)) { + unique.set(candidate.fullUrl, candidate); + } + } + return [...unique.values()]; +} + +function resolveJournalArticleUrlTitle(row: JsonwsJournalArticleRow): string | undefined { + return row.urlTitle ?? row.urlTitleCurrentValue ?? row.friendlyURL; +} + +function buildDisplayPageSourceCacheKey( + config: AppConfig, + site: LiferayInventorySite, + origin: DisplayPageCandidate['origin'], +): string { + return `${config.liferay.url}|${site.groupId}|${origin}`; +} + +function isUnsupportedDisplayPageSourceCached( + config: AppConfig, + site: LiferayInventorySite, + origin: DisplayPageCandidate['origin'], +): boolean { + return unsupportedDisplayPageSourceCache.get(buildDisplayPageSourceCacheKey(config, site, origin)) ?? false; +} + +function cacheUnsupportedDisplayPageSource( + config: AppConfig, + site: LiferayInventorySite, + origin: DisplayPageCandidate['origin'], +): void { + unsupportedDisplayPageSourceCache.set(buildDisplayPageSourceCacheKey(config, site, origin), true); +} + +export function resetDisplayPageSourceSupportCache(): void { + unsupportedDisplayPageSourceCache.clear(); +} + +function isUnsupportedDisplayPageScanError(origin: DisplayPageCandidate['origin'], error: unknown): boolean { + if (!isCliError(error)) return false; + if (error.code !== 'LIFERAY_INVENTORY_ERROR' && error.code !== 'LIFERAY_GATEWAY_ERROR') { + return false; + } + + // A 404 from headless structured contents means the API surface is not available. + // Permission failures should still surface so the caller can diagnose them. + return origin === 'headlessStructuredContent' && error.message.includes('status=404'); +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-errors.ts b/src/features/liferay/inventory/liferay-inventory-where-used-errors.ts new file mode 100644 index 0000000..eb2c00a --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-errors.ts @@ -0,0 +1,21 @@ +import {CliError} from '../../../core/errors.js'; + +export type WhereUsedCandidateLike = { + fullUrl: string; + origin?: 'layout' | 'headlessStructuredContent' | 'jsonwsJournal'; +}; + +export function extractErrorMessage(error: unknown): string { + if (error instanceof CliError) return error.message; + if (error instanceof Error) return error.message; + return String(error); +} + +export function isSkippableWhereUsedCandidateError(candidate: WhereUsedCandidateLike, error: unknown): boolean { + if (candidate.origin !== 'jsonwsJournal') { + return false; + } + + const message = extractErrorMessage(error); + return message.includes('No structured content found with friendlyUrlPath='); +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-format.ts b/src/features/liferay/inventory/liferay-inventory-where-used-format.ts new file mode 100644 index 0000000..e6aa8a3 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-format.ts @@ -0,0 +1,108 @@ +import type {WhereUsedPlanResult, WhereUsedResult, WhereUsedRunResult} from './liferay-inventory-where-used.js'; + +export function formatLiferayInventoryWhereUsed(result: WhereUsedRunResult): string { + if (result.inventoryType === 'whereUsedPlan') { + return formatWhereUsedPlan(result); + } + + const siteOrder = result.scope.siteOrder; + const siteLimit = result.scope.siteLimit ?? 'all'; + const excludedSites = result.scope.excludedSites; + const skippedSites = result.skippedSites ?? []; + + const lines: string[] = [ + 'WHERE USED', + `resourceType=${result.query.type}`, + `resourceKeys=${result.query.keys.join(',')}`, + `sites=${result.summary.totalSites}`, + `scannedPages=${result.summary.totalScannedPages}`, + `matchedPages=${result.summary.totalMatchedPages}`, + `totalMatches=${result.summary.totalMatches}`, + `failedPages=${result.summary.totalFailedPages}`, + `includePrivate=${result.scope.includePrivate}`, + `concurrency=${result.scope.concurrency}`, + `siteOrder=${siteOrder}`, + `siteLimit=${siteLimit}`, + `excludedSites=${excludedSites.join(',') || '-'}`, + `skippedSites=${skippedSites.length}`, + ]; + + if (result.summary.totalMatchedPages === 0) { + lines.push(''); + lines.push('No pages matched the requested resource.'); + return lines.join('\n'); + } + + for (const site of result.sites) { + if (site.matchedPages.length === 0 && site.failedPages === 0) continue; + lines.push(''); + lines.push( + `site=${site.siteFriendlyUrl} name=${site.siteName} groupId=${site.groupId} scanned=${site.scannedPages} matched=${site.matchedPages.length}`, + ); + for (const page of site.matchedPages) { + const pageUrl = page.viewUrl ?? page.fullUrl; + lines.push( + ` - [${page.pageType}] ${page.pageName} ${pageUrl}${page.privateLayout ? ' (private)' : ''}${page.hidden ? ' (hidden)' : ''}`, + ); + for (const match of page.matches) { + lines.push(` * ${match.label}: ${match.detail}${formatWhereUsedSourceSuffix(match.source)}`); + } + if (page.editUrl) { + lines.push(` editUrl=${page.editUrl}`); + } + } + if (site.failedPages > 0) { + lines.push(` ! ${site.failedPages} page(s) failed to load`); + } + } + + if (skippedSites.length > 0) { + lines.push(''); + lines.push(`Skipped ranking sites: ${skippedSites.length}`); + for (const site of skippedSites.slice(0, 5)) { + lines.push(` - site=${site.siteFriendlyUrl} groupId=${site.groupId} reason=${site.reason}`); + } + } + + return lines.join('\n'); +} + +function formatWhereUsedPlan(result: WhereUsedPlanResult): string { + const lines: string[] = [ + 'WHERE USED PLAN', + `resourceType=${result.query.type}`, + `resourceKeys=${result.query.keys.join(',')}`, + `totalSites=${result.summary.totalSites}`, + `selectedSites=${result.summary.selectedSites}`, + `excludedSites=${result.summary.excludedSites}`, + `skippedSites=${result.summary.skippedSites}`, + `includePrivate=${result.scope.includePrivate}`, + `concurrency=${result.scope.concurrency}`, + `siteOrder=${result.scope.siteOrder}`, + `siteLimit=${result.scope.siteLimit ?? 'all'}`, + ]; + + for (const site of result.sites) { + lines.push( + `${site.rank}. site=${site.siteFriendlyUrl} name=${site.siteName} groupId=${site.groupId}` + + `${site.structuredContents !== undefined ? ` structuredContents=${site.structuredContents}` : ''}` + + ` selectionReason=${site.selectionReason}`, + ); + } + + if (result.skippedSites && result.skippedSites.length > 0) { + lines.push(''); + lines.push(`Skipped sites: ${result.skippedSites.length}`); + for (const site of result.skippedSites.slice(0, 5)) { + lines.push(` - site=${site.siteFriendlyUrl} groupId=${site.groupId} reason=${site.reason}`); + } + } + + return lines.join('\n'); +} + +function formatWhereUsedSourceSuffix( + source: WhereUsedResult['sites'][number]['matchedPages'][number]['matches'][number]['source'], +): string { + return source === 'renderedHtmlJournalContent' ? ' [source=static Journal Content rendered in HTML]' : ''; +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-match.ts b/src/features/liferay/inventory/liferay-inventory-where-used-match.ts new file mode 100644 index 0000000..378bdab --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-match.ts @@ -0,0 +1,95 @@ +import {extractPageEvidence, type PageEvidence, type PageEvidenceKind} from './liferay-inventory-page-evidence.js'; +import type {WhereUsedResourceTypeValue} from './liferay-inventory-evidence-contract.js'; +import type {LiferayInventoryPageResult} from './liferay-inventory-page.js'; +import {normalizeWhereUsedEvidence} from './liferay-inventory-where-used-normalize.js'; + +export type WhereUsedResourceType = WhereUsedResourceTypeValue; + +export type WhereUsedQuery = { + type: WhereUsedResourceType; + keys: string[]; +}; + +export type WhereUsedMatchKind = Exclude; + +export type WhereUsedMatch = { + resourceType: WhereUsedResourceType; + matchedKey: string; + matchKind: WhereUsedMatchKind; + label: string; + detail: string; + source: PageEvidence['source']; +}; + +export function matchEvidenceAgainstResource(evidence: PageEvidence[], query: WhereUsedQuery): WhereUsedMatch[] { + const keys = new Set(query.keys.map((key) => normalizeWhereUsedKey(key, query.type))); + const seen = new Set(); + const matchedEvidence = normalizeWhereUsedEvidence( + evidence + .filter((item) => isEvidenceForResourceType(item, query.type)) + .filter((item) => item.kind !== 'journalArticle') + .filter((item) => keys.has(normalizeWhereUsedKey(item.key, query.type))), + query.type, + ); + + return matchedEvidence.flatMap((item) => { + const match: WhereUsedMatch = { + resourceType: query.type, + matchedKey: item.key, + matchKind: item.kind as WhereUsedMatchKind, + label: labelForMatchKind(item.kind as WhereUsedMatchKind, item.source), + detail: item.detail, + source: item.source, + }; + const identity = `${match.resourceType}\u0000${match.matchedKey}\u0000${match.matchKind}\u0000${match.detail}\u0000${match.source}`; + if (seen.has(identity)) { + return []; + } + seen.add(identity); + return [match]; + }); +} + +export function matchPageAgainstResource(page: LiferayInventoryPageResult, query: WhereUsedQuery): WhereUsedMatch[] { + return matchEvidenceAgainstResource(extractPageEvidence(page), query); +} + +function normalizeWhereUsedKey(key: string, type: WhereUsedResourceType): string { + return type === 'fragment' ? key.toLowerCase() : key; +} + +function isEvidenceForResourceType(evidence: PageEvidence, type: WhereUsedResourceType): boolean { + if (type === 'widget' || type === 'portlet') { + return evidence.resourceType === 'widget' || evidence.resourceType === 'portlet'; + } + return evidence.resourceType === type; +} + +function labelForMatchKind(kind: WhereUsedMatchKind, source: PageEvidence['source']): string { + switch (kind) { + case 'fragmentEntry': + return 'Fragment on page'; + case 'widgetEntry': + return 'Widget on page'; + case 'widgetAdt': + return 'Widget ADT'; + case 'portlet': + return 'Portlet on layout'; + case 'journalArticleStructure': + return source === 'renderedHtmlJournalContent' + ? 'Journal article structure (static Journal Content rendered in HTML)' + : 'Journal article structure'; + case 'journalArticleTemplate': + return source === 'renderedHtmlJournalContent' + ? 'Journal article template (static Journal Content rendered in HTML)' + : 'Journal article template'; + case 'fragmentMappedStructure': + return 'Fragment mapped structure'; + case 'fragmentMappedTemplate': + return 'Fragment mapped template'; + case 'contentStructure': + return 'Content structure'; + case 'displayPageArticle': + return 'Display page article'; + } +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-normalize.ts b/src/features/liferay/inventory/liferay-inventory-where-used-normalize.ts new file mode 100644 index 0000000..b677b77 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-normalize.ts @@ -0,0 +1,32 @@ +import type {PageEvidence} from './liferay-inventory-page-evidence.js'; + +export function normalizeWhereUsedEvidence( + evidence: PageEvidence[], + queryType: 'fragment' | 'widget' | 'portlet' | 'structure' | 'template' | 'adt', +): PageEvidence[] { + if (queryType !== 'structure') { + return evidence; + } + + return evidence.filter((item) => !isRedundantStructureEvidence(item, evidence)); +} + +function isRedundantStructureEvidence(evidence: PageEvidence, matchedEvidence: PageEvidence[]): boolean { + if (evidence.kind !== 'contentStructure') { + return false; + } + + const evidenceStructureId = evidence.context?.contentStructureId ?? parseNumericKey(evidence.key); + + return matchedEvidence.some( + (candidate) => + candidate.kind === 'journalArticleStructure' && + (candidate.key === evidence.key || + (evidenceStructureId !== undefined && candidate.context?.contentStructureId === evidenceStructureId)), + ); +} + +function parseNumericKey(key: string): number | undefined { + const value = Number(key); + return Number.isFinite(value) ? value : undefined; +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-page-candidates.ts b/src/features/liferay/inventory/liferay-inventory-where-used-page-candidates.ts new file mode 100644 index 0000000..2d88cf2 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-page-candidates.ts @@ -0,0 +1,114 @@ +import type {AppConfig} from '../../../core/config/load-config.js'; +import type {OAuthTokenClient} from '../../../core/http/auth.js'; +import type {HttpApiClient} from '../../../core/http/client.js'; +import {isCliError} from '../../../core/errors.js'; +import {runLiferayInventoryPages, type LiferayInventoryPagesNode} from './liferay-inventory-pages.js'; +import type {LiferayInventorySite} from './liferay-inventory-sites.js'; +import {collectDisplayPageCandidates} from './liferay-inventory-where-used-display-pages.js'; +import type {WhereUsedQuery} from './liferay-inventory-where-used-match.js'; +import {flattenPages, type FlatPage} from './liferay-inventory-where-used-pages.js'; + +export type WhereUsedPageCandidateOrigin = 'layout' | 'headlessStructuredContent' | 'jsonwsJournal'; + +export type WhereUsedPageCandidate = FlatPage & { + origin: WhereUsedPageCandidateOrigin; +}; + +export type WhereUsedPageCandidateContext = { + layoutScopes: boolean[]; + maxDepth: number; + concurrency: number; + pageSize: number; + dependencies: { + apiClient?: HttpApiClient; + tokenClient?: OAuthTokenClient; + }; +}; + +export async function collectWhereUsedPageCandidates( + config: AppConfig, + site: LiferayInventorySite, + query: WhereUsedQuery, + context: WhereUsedPageCandidateContext, +): Promise { + return dedupePageCandidates([ + ...(await collectLayoutPageCandidates(config, site, context)), + ...(await collectStructuredContentDisplayPageCandidates(config, site, query, context)), + ]); +} + +async function collectLayoutPageCandidates( + config: AppConfig, + site: LiferayInventorySite, + context: WhereUsedPageCandidateContext, +): Promise { + const candidates: WhereUsedPageCandidate[] = []; + + for (const privateLayout of context.layoutScopes) { + let pages: LiferayInventoryPagesNode[]; + try { + const pagesResult = await runLiferayInventoryPages( + config, + {site: site.siteFriendlyUrl, privateLayout, maxDepth: context.maxDepth}, + context.dependencies, + ); + pages = pagesResult.pages; + } catch (error) { + if (isSkippablePageSourceError(error)) continue; + throw error; + } + + candidates.push(...flattenPages(pages, privateLayout).map((page) => ({...page, origin: 'layout' as const}))); + } + + return candidates; +} + +async function collectStructuredContentDisplayPageCandidates( + config: AppConfig, + site: LiferayInventorySite, + query: WhereUsedQuery, + context: WhereUsedPageCandidateContext, +): Promise { + if (query.type !== 'structure' && query.type !== 'template') { + return []; + } + + const displayPages = await collectDisplayPageCandidates(config, site, { + concurrency: context.concurrency, + pageSize: context.pageSize, + dependencies: context.dependencies, + }); + + return displayPages.map((candidate) => ({ + fullUrl: candidate.fullUrl, + friendlyUrl: candidate.fullUrl, + name: candidate.fullUrl, + layoutId: -1, + plid: -1, + hidden: false, + privateLayout: false, + origin: candidate.origin, + })); +} + +function dedupePageCandidates(candidates: WhereUsedPageCandidate[]): WhereUsedPageCandidate[] { + const unique = new Map(); + + for (const candidate of candidates) { + const key = `${candidate.privateLayout ? 'private' : 'public'}:${candidate.fullUrl}`; + if (!unique.has(key)) { + unique.set(key, candidate); + } + } + + return [...unique.values()]; +} + +function isSkippablePageSourceError(error: unknown): boolean { + if (!isCliError(error)) return false; + if (error.code !== 'LIFERAY_INVENTORY_ERROR' && error.code !== 'LIFERAY_GATEWAY_ERROR') { + return false; + } + return error.message.includes('status=403') || error.message.includes('status=404'); +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-pages.ts b/src/features/liferay/inventory/liferay-inventory-where-used-pages.ts new file mode 100644 index 0000000..6fcef86 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-pages.ts @@ -0,0 +1,122 @@ +import type {LiferayInventoryPageResult} from './liferay-inventory-page.js'; +import type {LiferayInventoryPagesNode} from './liferay-inventory-pages.js'; +import {buildPortalAbsoluteUrl} from './liferay-inventory-url.js'; +import type {WhereUsedMatch} from './liferay-inventory-where-used-match.js'; + +export type FlatPage = { + fullUrl: string; + friendlyUrl: string; + name: string; + layoutId: number; + plid: number; + hidden: boolean; + privateLayout: boolean; +}; + +export type WhereUsedPageMatch = { + pageType: 'regularPage' | 'displayPage'; + pageName: string; + friendlyUrl: string; + fullUrl: string; + viewUrl?: string; + layoutId?: number; + plid?: number; + privateLayout: boolean; + hidden?: boolean; + editUrl?: string; + matches: WhereUsedMatch[]; +}; + +export function flattenPages(pages: LiferayInventoryPagesNode[], privateLayout: boolean): FlatPage[] { + const result: FlatPage[] = []; + const visit = (node: LiferayInventoryPagesNode): void => { + result.push({ + fullUrl: node.fullUrl, + friendlyUrl: node.friendlyUrl, + name: node.name, + layoutId: node.layoutId, + plid: node.plid, + hidden: node.hidden, + privateLayout, + }); + for (const child of node.children) { + visit(child); + } + }; + for (const node of pages) visit(node); + return result; +} + +export function buildPageMatch( + page: LiferayInventoryPageResult, + entry: FlatPage, + matches: WhereUsedMatch[], + portalBaseUrl?: string, +): WhereUsedPageMatch { + if (page.pageType === 'displayPage') { + const hasRenderableView = hasDisplayPageRendering(page); + + return { + pageType: 'displayPage', + pageName: page.article.title, + friendlyUrl: page.friendlyUrl, + fullUrl: page.url, + ...(hasRenderableView ? {viewUrl: buildPortalAbsoluteUrl(portalBaseUrl, page.url)} : {}), + privateLayout: entry.privateLayout, + ...(page.adminUrls ? {editUrl: page.adminUrls.edit} : {}), + matches, + }; + } + + if (page.pageType === 'regularPage') { + return { + pageType: 'regularPage', + pageName: page.pageName, + friendlyUrl: page.friendlyUrl, + fullUrl: page.url, + viewUrl: buildPortalAbsoluteUrl(portalBaseUrl, page.url), + layoutId: page.layout.layoutId, + plid: page.layout.plid, + hidden: page.layout.hidden, + privateLayout: page.privateLayout, + editUrl: page.adminUrls.edit, + matches, + }; + } + + return { + pageType: 'regularPage', + pageName: entry.name, + friendlyUrl: entry.friendlyUrl, + fullUrl: entry.fullUrl, + viewUrl: buildPortalAbsoluteUrl(portalBaseUrl, entry.fullUrl), + layoutId: entry.layoutId, + plid: entry.plid, + hidden: entry.hidden, + privateLayout: entry.privateLayout, + matches, + }; +} + +function hasDisplayPageRendering(page: Extract): boolean { + return ( + page.journalArticles?.some((article) => { + const renderedDisplayTemplate = article.renderedContents + ?.map((item) => item as Record) + .some( + (candidate) => + candidate.markedAsDefault === true && + typeof candidate.contentTemplateName === 'string' && + typeof candidate.renderedContentURL === 'string' && + candidate.renderedContentURL.includes('/rendered-content-by-display-page/'), + ); + + return Boolean( + article.displayPageDefaultTemplate || + article.displayPageTemplateCandidates?.length || + article.displayPageDdmTemplates?.length || + renderedDisplayTemplate, + ); + }) ?? false + ); +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-query-resolver.ts b/src/features/liferay/inventory/liferay-inventory-where-used-query-resolver.ts new file mode 100644 index 0000000..f83b045 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-query-resolver.ts @@ -0,0 +1,270 @@ +import type {AppConfig} from '../../../core/config/load-config.js'; +import type {OAuthTokenClient} from '../../../core/http/auth.js'; +import type {HttpApiClient} from '../../../core/http/client.js'; +import {CliError} from '../../../core/errors.js'; +import {matchesAdtRow, matchesDdmTemplate} from '../liferay-identifiers.js'; +import {buildSiteChain} from '../portal/site-resolution.js'; +import {listDdmTemplates, resolveResourceSite, type DdmTemplatePayload} from '../portal/template-queries.js'; +import {runLiferayResourceListAdts, type LiferayResourceAdtRow} from '../resource/liferay-resource-list-adts.js'; +import { + runLiferayResourceListFragments, + type LiferayResourceFragmentRow, +} from '../resource/liferay-resource-list-fragments.js'; +import {runLiferayInventoryTemplates} from './liferay-inventory-templates.js'; +import {runLiferayInventorySitesIncludingGlobal} from './liferay-inventory-sites.js'; +import type {WhereUsedQuery} from './liferay-inventory-where-used-match.js'; + +type WhereUsedResolverOptions = { + sites?: string[]; + widgetType?: string; + className?: string; +}; + +type WhereUsedResolverDependencies = { + apiClient?: HttpApiClient; + tokenClient?: OAuthTokenClient; +}; + +type WhereUsedAdtReference = { + displayStyle: string; + templateKey: string; +}; + +type WhereUsedAdtRowReference = Pick; +type WhereUsedFragmentRowReference = Pick; +type WhereUsedTemplateRowReference = Pick< + DdmTemplatePayload, + 'templateId' | 'templateKey' | 'externalReferenceCode' | 'nameCurrentValue' | 'name' +>; +type WhereUsedSearchSite = {siteId: number; siteFriendlyUrl: string; siteName: string}; + +export async function resolveWhereUsedQuery( + config: AppConfig, + query: WhereUsedQuery, + options: WhereUsedResolverOptions, + dependencies: WhereUsedResolverDependencies, +): Promise { + if (query.type === 'adt') { + const resolvedKeys: string[] = []; + + for (const key of query.keys) { + const rows = await collectWhereUsedAdtRows(config, key, options, dependencies); + resolvedKeys.push(...collectWhereUsedAdtKeys(rows, key)); + } + + return { + type: query.type, + keys: Array.from(new Set(resolvedKeys)), + }; + } + + if (query.type === 'fragment') { + const rows = await collectWhereUsedFragmentRows(config, options, dependencies); + return resolveKeysFromCatalog(query, rows, collectWhereUsedFragmentKeys); + } + + if (query.type === 'template') { + const rows = await collectWhereUsedTemplateRows(config, options, dependencies); + return resolveKeysFromCatalog(query, rows, collectWhereUsedTemplateKeys); + } + + return query; +} + +export function buildWhereUsedAdtKeys(adt: WhereUsedAdtReference): string[] { + const keys = new Set(); + + const displayStyle = adt.displayStyle.trim(); + if (displayStyle !== '') { + keys.add(displayStyle); + } + + const templateKey = adt.templateKey.trim(); + if (templateKey !== '') { + keys.add(templateKey.startsWith('ddmTemplate_') ? templateKey : `ddmTemplate_${templateKey}`); + } + + return [...keys]; +} + +export function collectWhereUsedAdtKeys(rows: WhereUsedAdtRowReference[], identifier: string): string[] { + const keys = new Set(); + + for (const row of rows) { + if (!matchesAdtRow(row, identifier)) { + continue; + } + + for (const key of buildWhereUsedAdtKeys({ + displayStyle: `ddmTemplate_${String(row.templateId).trim()}`, + templateKey: row.templateKey, + })) { + keys.add(key); + } + } + + if (keys.size === 0) { + throw new CliError(`ADT not found: ${identifier}`, {code: 'LIFERAY_RESOURCE_ERROR'}); + } + + return [...keys]; +} + +export function collectWhereUsedFragmentKeys(rows: WhereUsedFragmentRowReference[], identifier: string): string[] { + const normalizedIdentifier = identifier.trim().toLowerCase(); + const keys = new Set(); + + for (const row of rows) { + const fragmentKey = row.fragmentKey.trim(); + const fragmentName = row.fragmentName.trim(); + + if (fragmentKey.toLowerCase() !== normalizedIdentifier && fragmentName.toLowerCase() !== normalizedIdentifier) { + continue; + } + + if (fragmentKey !== '') { + keys.add(fragmentKey); + } + } + + return keys.size > 0 ? [...keys] : [identifier]; +} + +export function collectWhereUsedTemplateKeys(rows: WhereUsedTemplateRowReference[], identifier: string): string[] { + const keys = new Set(); + + for (const row of rows) { + if (!matchesDdmTemplate(row, identifier)) { + continue; + } + + const templateKey = String(row.templateKey ?? '').trim(); + if (templateKey !== '') { + keys.add(templateKey); + } + } + + return keys.size > 0 ? [...keys] : [identifier]; +} + +async function resolveSearchSites( + config: AppConfig, + options: Pick, + dependencies: WhereUsedResolverDependencies, +): Promise { + if (options.sites?.length) { + return collectExplicitWhereUsedSites(config, options.sites, dependencies); + } + + return (await runLiferayInventorySitesIncludingGlobal(config, undefined, dependencies)).map((site) => ({ + siteId: site.groupId, + siteFriendlyUrl: site.siteFriendlyUrl, + siteName: site.name, + })); +} + +function resolveKeysFromCatalog( + query: WhereUsedQuery, + rows: T[], + resolveKeys: (rows: T[], key: string) => string[], +): WhereUsedQuery { + const resolvedKeys = query.keys.flatMap((key) => resolveKeys(rows, key)); + return {type: query.type, keys: Array.from(new Set(resolvedKeys))}; +} + +async function collectWhereUsedAdtRows( + config: AppConfig, + identifier: string, + options: Pick, + dependencies: WhereUsedResolverDependencies, +): Promise { + const searchSites = await resolveSearchSites(config, options, dependencies); + const rows: LiferayResourceAdtRow[] = []; + + for (const site of searchSites) { + const siteRows = await runLiferayResourceListAdts( + config, + { + site: site.siteFriendlyUrl, + widgetType: options.widgetType, + className: options.className, + }, + dependencies, + ); + rows.push(...siteRows.filter((row) => matchesAdtRow(row, identifier))); + } + + return rows; +} + +async function collectWhereUsedFragmentRows( + config: AppConfig, + options: Pick, + dependencies: WhereUsedResolverDependencies, +): Promise { + const searchSites = await resolveSearchSites(config, options, dependencies); + const rows: LiferayResourceFragmentRow[] = []; + + for (const site of searchSites) { + rows.push(...(await runLiferayResourceListFragments(config, {site: site.siteFriendlyUrl}, dependencies))); + } + + return rows; +} + +async function collectWhereUsedTemplateRows( + config: AppConfig, + options: Pick, + dependencies: WhereUsedResolverDependencies, +): Promise { + const searchSites = await resolveSearchSites(config, options, dependencies); + const rows: WhereUsedTemplateRowReference[] = []; + + for (const site of searchSites) { + const resolvedSite = await resolveResourceSite(config, site.siteFriendlyUrl, dependencies); + const ddmRows = await listDdmTemplates(config, resolvedSite, dependencies, { + includeCompanyFallback: site.siteFriendlyUrl === '/global', + }); + + if (ddmRows.length > 0) { + rows.push(...ddmRows); + continue; + } + + const inventoryRows = await runLiferayInventoryTemplates( + config, + {site: resolvedSite.friendlyUrlPath}, + dependencies, + ); + rows.push( + ...inventoryRows.map((row) => ({ + templateId: row.id, + templateKey: row.externalReferenceCode || row.id, + externalReferenceCode: row.externalReferenceCode, + nameCurrentValue: row.name, + name: row.name, + })), + ); + } + + return rows; +} + +async function collectExplicitWhereUsedSites( + config: AppConfig, + sites: string[], + dependencies: WhereUsedResolverDependencies, +): Promise { + const uniqueSites = new Map(); + + for (const site of sites) { + const siteChain = await buildSiteChain(config, site, dependencies); + for (const entry of siteChain) { + if (!uniqueSites.has(entry.siteFriendlyUrl)) { + uniqueSites.set(entry.siteFriendlyUrl, entry); + } + } + } + + return [...uniqueSites.values()]; +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-schema.ts b/src/features/liferay/inventory/liferay-inventory-where-used-schema.ts new file mode 100644 index 0000000..2ae97bb --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-schema.ts @@ -0,0 +1,121 @@ +import {z} from 'zod'; + +import { + pageEvidenceSourceSchema, + whereUsedMatchKindSchema, + whereUsedResourceTypeSchema, +} from './liferay-inventory-evidence-contract.js'; + +const whereUsedSiteOrderSchema = z.enum(['site', 'name', 'content']); + +const whereUsedMatchSchema = z.object({ + resourceType: whereUsedResourceTypeSchema, + matchedKey: z.string(), + matchKind: whereUsedMatchKindSchema, + label: z.string(), + detail: z.string(), + source: pageEvidenceSourceSchema, +}); + +const whereUsedPageMatchSchema = z.object({ + pageType: z.enum(['regularPage', 'displayPage']), + pageName: z.string(), + friendlyUrl: z.string(), + fullUrl: z.string(), + viewUrl: z.string().optional(), + layoutId: z.number().optional(), + plid: z.number().optional(), + privateLayout: z.boolean(), + hidden: z.boolean().optional(), + editUrl: z.string().optional(), + matches: z.array(whereUsedMatchSchema), +}); + +const whereUsedSiteResultSchema = z.object({ + siteFriendlyUrl: z.string(), + siteName: z.string(), + groupId: z.coerce.number(), + scannedPages: z.number(), + failedPages: z.number(), + matchedPages: z.array(whereUsedPageMatchSchema), + errors: z.array(z.object({fullUrl: z.string(), reason: z.string()})).optional(), +}); + +const whereUsedSkippedSiteSchema = z.object({ + siteFriendlyUrl: z.string(), + groupId: z.coerce.number(), + reason: z.string(), +}); + +export const whereUsedResultSchema = z.object({ + inventoryType: z.literal('whereUsed'), + query: z.object({ + type: whereUsedResourceTypeSchema, + keys: z.array(z.string()), + }), + scope: z.object({ + sites: z.array(z.string()), + includePrivate: z.boolean(), + concurrency: z.number(), + maxDepth: z.number(), + siteOrder: whereUsedSiteOrderSchema.default('site'), + siteLimit: z.number().optional(), + excludedSites: z.array(z.string()).default([]), + plan: z.literal(false).default(false), + }), + summary: z.object({ + totalSites: z.number(), + totalScannedPages: z.number(), + totalMatchedPages: z.number(), + totalMatches: z.number(), + totalFailedPages: z.number(), + }), + sites: z.array(whereUsedSiteResultSchema), + skippedSites: z.array(whereUsedSkippedSiteSchema).optional(), +}); + +export const whereUsedPlanResultSchema = z.object({ + inventoryType: z.literal('whereUsedPlan'), + query: z.object({ + type: whereUsedResourceTypeSchema, + keys: z.array(z.string()), + }), + scope: z.object({ + sites: z.array(z.string()), + includePrivate: z.boolean(), + concurrency: z.number(), + maxDepth: z.number(), + siteOrder: whereUsedSiteOrderSchema, + siteLimit: z.number().optional(), + excludedSites: z.array(z.string()), + plan: z.literal(true), + }), + summary: z.object({ + totalSites: z.number(), + selectedSites: z.number(), + excludedSites: z.number(), + skippedSites: z.number(), + }), + sites: z.array( + z.object({ + rank: z.number(), + siteFriendlyUrl: z.string(), + siteName: z.string(), + groupId: z.coerce.number(), + structuredContents: z.number().optional(), + selectionReason: z.enum(['explicitSite', 'siteOrder', 'contentOrder']), + }), + ), + skippedSites: z.array(whereUsedSkippedSiteSchema).optional(), +}); + +export type WhereUsedResultContract = z.infer; +export type WhereUsedPlanResultContract = z.infer; + +export function validateWhereUsedResult(result: unknown): WhereUsedResultContract { + return whereUsedResultSchema.parse(result); +} + +export function validateWhereUsedPlanResult(result: unknown): WhereUsedPlanResultContract { + return whereUsedPlanResultSchema.parse(result); +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-site-selection.ts b/src/features/liferay/inventory/liferay-inventory-where-used-site-selection.ts new file mode 100644 index 0000000..c1e4c0a --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-site-selection.ts @@ -0,0 +1,116 @@ +import type {ContentStatsSite} from '../content/liferay-content-stats.js'; +import {buildPagesCommand, type LiferayInventorySite} from './liferay-inventory-sites.js'; +import type {WhereUsedSiteOrder} from './liferay-inventory-where-used-validation.js'; + +export type WhereUsedPlanSite = { + rank: number; + siteFriendlyUrl: string; + siteName: string; + groupId: number; + structuredContents?: number; + selectionReason: 'explicitSite' | 'siteOrder' | 'contentOrder'; +}; + +export type WhereUsedSiteSelectionInput = { + sites: LiferayInventorySite[]; + explicitSites?: string[]; + siteOrder: WhereUsedSiteOrder; + siteLimit?: number; + excludedSites: string[]; + contentStatsSites?: ContentStatsSite[]; + contentStatsSkippedSites?: Array<{groupId: number; siteFriendlyUrl: string; reason: string}>; +}; + +export type WhereUsedSiteSelection = { + selectedSites: LiferayInventorySite[]; + planSites: WhereUsedPlanSite[]; + totalSites: number; + excludedCount: number; + skippedSites: Array<{siteFriendlyUrl: string; groupId: number; reason: string}>; +}; + +export function selectWhereUsedSites(input: WhereUsedSiteSelectionInput): WhereUsedSiteSelection { + if (input.explicitSites && input.explicitSites.length > 0) { + const explicitSites = input.explicitSites.map((site) => site.trim()).filter((site) => site !== ''); + const selectedSites = explicitSites.map((explicitSite) => { + return ( + input.sites.find( + (site) => + site.siteFriendlyUrl === explicitSite || + site.siteFriendlyUrl === `/${explicitSite}` || + String(site.groupId) === explicitSite, + ) ?? { + groupId: -1, + siteFriendlyUrl: explicitSite.startsWith('/') ? explicitSite : `/${explicitSite}`, + name: explicitSite, + pagesCommand: buildPagesCommand(explicitSite), + } + ); + }); + + const uniqueSelectedSites = selectedSites.filter( + (site, index, allSites) => + allSites.findIndex((candidate) => candidate.siteFriendlyUrl === site.siteFriendlyUrl) === index, + ); + + return { + selectedSites: uniqueSelectedSites, + planSites: uniqueSelectedSites.map((site, index) => ({ + rank: index + 1, + siteFriendlyUrl: site.siteFriendlyUrl, + siteName: site.name, + groupId: site.groupId, + selectionReason: 'explicitSite', + })), + totalSites: input.sites.length, + excludedCount: 0, + skippedSites: input.contentStatsSkippedSites ?? [], + }; + } + + const excludedSites = new Set(input.excludedSites); + const filteredSites = input.sites.filter((site) => !excludedSites.has(site.siteFriendlyUrl)); + const structuredContentsBySite = new Map( + (input.contentStatsSites ?? []).map((site) => [site.siteFriendlyUrl, site.structuredContents]), + ); + + const orderedSites = filteredSites.slice().sort((left, right) => { + if (input.siteOrder === 'content') { + const leftCount = structuredContentsBySite.get(left.siteFriendlyUrl); + const rightCount = structuredContentsBySite.get(right.siteFriendlyUrl); + + if (leftCount !== undefined && rightCount !== undefined && leftCount !== rightCount) { + return rightCount - leftCount; + } + if (leftCount !== undefined && rightCount === undefined) return -1; + if (leftCount === undefined && rightCount !== undefined) return 1; + return left.siteFriendlyUrl.localeCompare(right.siteFriendlyUrl); + } + + if (input.siteOrder === 'name') { + const byName = left.name.localeCompare(right.name); + return byName !== 0 ? byName : left.siteFriendlyUrl.localeCompare(right.siteFriendlyUrl); + } + + return left.siteFriendlyUrl.localeCompare(right.siteFriendlyUrl); + }); + + const limitedSites = input.siteLimit !== undefined ? orderedSites.slice(0, input.siteLimit) : orderedSites; + + return { + selectedSites: limitedSites, + planSites: limitedSites.map((site, index) => ({ + rank: index + 1, + siteFriendlyUrl: site.siteFriendlyUrl, + siteName: site.name, + groupId: site.groupId, + ...(structuredContentsBySite.has(site.siteFriendlyUrl) + ? {structuredContents: structuredContentsBySite.get(site.siteFriendlyUrl)} + : {}), + selectionReason: input.siteOrder === 'content' ? 'contentOrder' : 'siteOrder', + })), + totalSites: input.sites.length, + excludedCount: input.sites.length - filteredSites.length, + skippedSites: input.contentStatsSkippedSites ?? [], + }; +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used-validation.ts b/src/features/liferay/inventory/liferay-inventory-where-used-validation.ts new file mode 100644 index 0000000..c725f18 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used-validation.ts @@ -0,0 +1,67 @@ +import {CliError} from '../../../core/errors.js'; +import {normalizeFriendlyUrl} from '../portal/site-resolution.js'; +import {whereUsedResourceTypes} from './liferay-inventory-evidence-contract.js'; +import type {WhereUsedQuery, WhereUsedResourceType} from './liferay-inventory-where-used-match.js'; + +export const whereUsedSiteOrderValues = ['site', 'name', 'content'] as const; +export type WhereUsedSiteOrder = (typeof whereUsedSiteOrderValues)[number]; + +export type ValidatedWhereUsedScopeOptions = { + siteOrder: WhereUsedSiteOrder; + siteLimit?: number; + excludedSites: string[]; + plan: boolean; +}; + +const VALID_RESOURCE_TYPES: WhereUsedResourceType[] = [...whereUsedResourceTypes]; +const VALID_SITE_ORDERS: WhereUsedSiteOrder[] = [...whereUsedSiteOrderValues]; + +export function validateWhereUsedQuery(options: {type: WhereUsedResourceType; keys: string[]}): WhereUsedQuery { + if (!VALID_RESOURCE_TYPES.includes(options.type)) { + throw new CliError(`--type must be one of: ${VALID_RESOURCE_TYPES.join(', ')}.`, {code: 'LIFERAY_INVENTORY_ERROR'}); + } + + const cleanedKeys = options.keys + .map((key) => (typeof key === 'string' ? key.trim() : '')) + .filter((key) => key.length > 0); + + if (cleanedKeys.length === 0) { + throw new CliError('Provide at least one --key value to look up.', { + code: 'LIFERAY_INVENTORY_ERROR', + }); + } + + return {type: options.type, keys: Array.from(new Set(cleanedKeys))}; +} + +export function validateWhereUsedScopeOptions(options: { + siteOrder?: string; + siteLimit?: number; + excludeSites?: string[]; + plan?: boolean; +}): ValidatedWhereUsedScopeOptions { + const siteOrder = (options.siteOrder ?? 'site').trim() as WhereUsedSiteOrder; + if (!VALID_SITE_ORDERS.includes(siteOrder)) { + throw new CliError(`--site-order must be one of: ${VALID_SITE_ORDERS.join(', ')}.`, { + code: 'LIFERAY_INVENTORY_ERROR', + }); + } + + const siteLimit = options.siteLimit; + if (siteLimit !== undefined && (!Number.isInteger(siteLimit) || siteLimit <= 0)) { + throw new CliError('--site-limit must be a positive integer.', {code: 'LIFERAY_INVENTORY_ERROR'}); + } + + const excludedSites = Array.from( + new Set( + (options.excludeSites ?? []).map((site) => normalizeFriendlyUrl(site.trim())).filter((site) => site.length > 0), + ), + ); + + return { + siteOrder, + ...(siteLimit !== undefined ? {siteLimit} : {}), + excludedSites, + plan: Boolean(options.plan), + }; +} diff --git a/src/features/liferay/inventory/liferay-inventory-where-used.ts b/src/features/liferay/inventory/liferay-inventory-where-used.ts new file mode 100644 index 0000000..0cf6df7 --- /dev/null +++ b/src/features/liferay/inventory/liferay-inventory-where-used.ts @@ -0,0 +1,519 @@ +import type {AppConfig} from '../../../core/config/load-config.js'; +import type {OAuthTokenClient} from '../../../core/http/auth.js'; +import type {HttpApiClient} from '../../../core/http/client.js'; +import {createLiferayApiClient} from '../../../core/http/client.js'; +import {mapConcurrent} from '../../../core/concurrency.js'; +import {CliError} from '../../../core/errors.js'; +import {runContentStats, type ContentStatsSite} from '../content/liferay-content-stats.js'; +import {normalizeFriendlyUrl} from '../portal/site-resolution.js'; +import {whereUsedResourceTypes} from './liferay-inventory-evidence-contract.js'; +import {extractPageEvidence} from './liferay-inventory-page-evidence.js'; +import {resolveInventoryPageRequest, runLiferayInventoryPage} from './liferay-inventory-page.js'; +import {createInventoryGateway} from './liferay-inventory-shared.js'; +import { + buildPagesCommand, + runLiferayInventorySitesIncludingGlobal, + type LiferayInventorySite, +} from './liferay-inventory-sites.js'; +import {resolveWhereUsedQuery as resolveWhereUsedPortalResourceQuery} from './liferay-inventory-where-used-query-resolver.js'; +import { + matchEvidenceAgainstResource, + type WhereUsedQuery, + type WhereUsedResourceType, +} from './liferay-inventory-where-used-match.js'; +import {validateWhereUsedPlanResult, validateWhereUsedResult} from './liferay-inventory-where-used-schema.js'; +import {buildPageMatch, type WhereUsedPageMatch} from './liferay-inventory-where-used-pages.js'; +import {collectWhereUsedPageCandidates} from './liferay-inventory-where-used-page-candidates.js'; + +export {matchEvidenceAgainstResource, matchPageAgainstResource} from './liferay-inventory-where-used-match.js'; +export {formatLiferayInventoryWhereUsed} from './liferay-inventory-where-used-format.js'; +export {validateWhereUsedPlanResult, validateWhereUsedResult} from './liferay-inventory-where-used-schema.js'; +export { + buildWhereUsedAdtKeys, + collectWhereUsedAdtKeys, + collectWhereUsedFragmentKeys, + collectWhereUsedTemplateKeys, +} from './liferay-inventory-where-used-query-resolver.js'; +export type { + WhereUsedMatch, + WhereUsedMatchKind, + WhereUsedQuery, + WhereUsedResourceType, +} from './liferay-inventory-where-used-match.js'; +export type {WhereUsedPageMatch} from './liferay-inventory-where-used-pages.js'; + +export type WhereUsedOptions = { + type: WhereUsedResourceType; + keys: string[]; + sites?: string[]; + excludeSites?: string[]; + widgetType?: string; + className?: string; + includePrivate?: boolean; + siteLimit?: number; + siteOrder?: string; + plan?: boolean; + maxDepth?: number; + concurrency?: number; + pageSize?: number; +}; + +export type WhereUsedDependencies = { + apiClient?: HttpApiClient; + tokenClient?: OAuthTokenClient; +}; + +export const whereUsedSiteOrderValues = ['site', 'name', 'content'] as const; +export type WhereUsedSiteOrder = (typeof whereUsedSiteOrderValues)[number]; + +export type ValidatedWhereUsedScopeOptions = { + siteOrder: WhereUsedSiteOrder; + siteLimit?: number; + excludedSites: string[]; + plan: boolean; +}; + +export type WhereUsedCandidateLike = { + fullUrl: string; + origin?: 'layout' | 'headlessStructuredContent' | 'jsonwsJournal'; +}; + +export type WhereUsedSiteResult = { + siteFriendlyUrl: string; + siteName: string; + groupId: number; + scannedPages: number; + failedPages: number; + matchedPages: WhereUsedPageMatch[]; + errors?: Array<{fullUrl: string; reason: string}>; +}; + +export type WhereUsedResult = { + inventoryType: 'whereUsed'; + query: WhereUsedQuery; + scope: { + sites: string[]; + includePrivate: boolean; + concurrency: number; + maxDepth: number; + siteOrder: WhereUsedSiteOrder; + siteLimit?: number; + excludedSites: string[]; + plan: false; + }; + summary: { + totalSites: number; + totalScannedPages: number; + totalMatchedPages: number; + totalMatches: number; + totalFailedPages: number; + }; + sites: WhereUsedSiteResult[]; + skippedSites?: Array<{siteFriendlyUrl: string; groupId: number; reason: string}>; +}; + +export type WhereUsedPlanSite = { + rank: number; + siteFriendlyUrl: string; + siteName: string; + groupId: number; + structuredContents?: number; + selectionReason: 'explicitSite' | 'siteOrder' | 'contentOrder'; +}; + +export type WhereUsedPlanResult = { + inventoryType: 'whereUsedPlan'; + query: WhereUsedQuery; + scope: { + sites: string[]; + includePrivate: boolean; + concurrency: number; + maxDepth: number; + siteOrder: WhereUsedSiteOrder; + siteLimit?: number; + excludedSites: string[]; + plan: true; + }; + summary: { + totalSites: number; + selectedSites: number; + excludedSites: number; + skippedSites: number; + }; + sites: WhereUsedPlanSite[]; + skippedSites?: Array<{siteFriendlyUrl: string; groupId: number; reason: string}>; +}; + +export type WhereUsedRunResult = WhereUsedResult | WhereUsedPlanResult; + +export type WhereUsedSiteSelectionInput = { + sites: LiferayInventorySite[]; + explicitSites?: string[]; + siteOrder: WhereUsedSiteOrder; + siteLimit?: number; + excludedSites: string[]; + contentStatsSites?: ContentStatsSite[]; + contentStatsSkippedSites?: Array<{groupId: number; siteFriendlyUrl: string; reason: string}>; +}; + +export type WhereUsedSiteSelection = { + selectedSites: LiferayInventorySite[]; + planSites: WhereUsedPlanSite[]; + totalSites: number; + excludedCount: number; + skippedSites: Array<{siteFriendlyUrl: string; groupId: number; reason: string}>; +}; + +const VALID_RESOURCE_TYPES: WhereUsedResourceType[] = [...whereUsedResourceTypes]; +const VALID_SITE_ORDERS: WhereUsedSiteOrder[] = [...whereUsedSiteOrderValues]; + +export function validateWhereUsedQuery(options: Pick): WhereUsedQuery { + if (!VALID_RESOURCE_TYPES.includes(options.type)) { + throw new CliError(`--type must be one of: ${VALID_RESOURCE_TYPES.join(', ')}.`, {code: 'LIFERAY_INVENTORY_ERROR'}); + } + + const cleanedKeys = options.keys + .map((key) => (typeof key === 'string' ? key.trim() : '')) + .filter((key) => key.length > 0); + + if (cleanedKeys.length === 0) { + throw new CliError('Provide at least one --key value to look up.', { + code: 'LIFERAY_INVENTORY_ERROR', + }); + } + + return {type: options.type, keys: Array.from(new Set(cleanedKeys))}; +} + +export function validateWhereUsedScopeOptions( + options: Pick, +): ValidatedWhereUsedScopeOptions { + const siteOrder = (options.siteOrder ?? 'site').trim() as WhereUsedSiteOrder; + if (!VALID_SITE_ORDERS.includes(siteOrder)) { + throw new CliError(`--site-order must be one of: ${VALID_SITE_ORDERS.join(', ')}.`, { + code: 'LIFERAY_INVENTORY_ERROR', + }); + } + + const siteLimit = options.siteLimit; + if (siteLimit !== undefined && (!Number.isInteger(siteLimit) || siteLimit <= 0)) { + throw new CliError('--site-limit must be a positive integer.', {code: 'LIFERAY_INVENTORY_ERROR'}); + } + + const excludedSites = Array.from( + new Set( + (options.excludeSites ?? []).map((site) => normalizeFriendlyUrl(site.trim())).filter((site) => site.length > 0), + ), + ); + + return { + siteOrder, + ...(siteLimit !== undefined ? {siteLimit} : {}), + excludedSites, + plan: Boolean(options.plan), + }; +} + +export async function runLiferayInventoryWhereUsed( + config: AppConfig, + options: WhereUsedOptions, + dependencies?: WhereUsedDependencies, +): Promise { + const baseQuery = validateWhereUsedQuery(options); + const scopeOptions = validateWhereUsedScopeOptions(options); + const apiClient = dependencies?.apiClient ?? createLiferayApiClient(); + const gateway = createInventoryGateway(config, apiClient, { + apiClient, + tokenClient: dependencies?.tokenClient, + }); + const sharedDependencies = {apiClient, tokenClient: dependencies?.tokenClient, gateway}; + const query = await resolveWhereUsedPortalResourceQuery(config, baseQuery, options, sharedDependencies); + const concurrency = Math.max(1, options.concurrency ?? 4); + const maxDepth = Math.max(0, options.maxDepth ?? 12); + const includePrivate = Boolean(options.includePrivate); + + const resolvedScope = await resolveTargetSites( + config, + options.sites, + options.pageSize, + scopeOptions, + sharedDependencies, + ); + const layoutScopes: boolean[] = includePrivate ? [false, true] : [false]; + + if (scopeOptions.plan) { + return validateWhereUsedPlanResult({ + inventoryType: 'whereUsedPlan', + query, + scope: { + sites: resolvedScope.selectedSites.map((site) => site.siteFriendlyUrl), + includePrivate, + concurrency, + maxDepth, + siteOrder: scopeOptions.siteOrder, + ...(scopeOptions.siteLimit !== undefined ? {siteLimit: scopeOptions.siteLimit} : {}), + excludedSites: scopeOptions.excludedSites, + plan: true, + }, + summary: { + totalSites: resolvedScope.totalSites, + selectedSites: resolvedScope.selectedSites.length, + excludedSites: resolvedScope.excludedCount, + skippedSites: resolvedScope.skippedSites.length, + }, + sites: resolvedScope.planSites, + ...(resolvedScope.skippedSites.length > 0 ? {skippedSites: resolvedScope.skippedSites} : {}), + }) as WhereUsedPlanResult; + } + + const siteResults: WhereUsedSiteResult[] = []; + for (const site of resolvedScope.selectedSites) { + siteResults.push( + await scanSite(config, site, query, { + layoutScopes, + concurrency, + maxDepth, + pageSize: options.pageSize ?? 200, + sharedDependencies, + }), + ); + } + + return validateWhereUsedResult({ + inventoryType: 'whereUsed', + query, + scope: { + sites: resolvedScope.selectedSites.map((site) => site.siteFriendlyUrl), + includePrivate, + concurrency, + maxDepth, + siteOrder: scopeOptions.siteOrder, + ...(scopeOptions.siteLimit !== undefined ? {siteLimit: scopeOptions.siteLimit} : {}), + excludedSites: scopeOptions.excludedSites, + plan: false, + }, + summary: summarize(siteResults), + sites: siteResults, + ...(resolvedScope.skippedSites.length > 0 ? {skippedSites: resolvedScope.skippedSites} : {}), + }) as WhereUsedResult; +} + +type ScanContext = { + layoutScopes: boolean[]; + concurrency: number; + maxDepth: number; + pageSize: number; + sharedDependencies: WhereUsedDependencies; +}; + +async function scanSite( + config: AppConfig, + site: LiferayInventorySite, + query: WhereUsedQuery, + context: ScanContext, +): Promise { + const result: WhereUsedSiteResult = { + siteFriendlyUrl: site.siteFriendlyUrl, + siteName: site.name, + groupId: Number(site.groupId), + scannedPages: 0, + failedPages: 0, + matchedPages: [], + }; + const errors: Array<{fullUrl: string; reason: string}> = []; + const candidates = await collectWhereUsedPageCandidates(config, site, query, { + layoutScopes: context.layoutScopes, + concurrency: context.concurrency, + maxDepth: context.maxDepth, + pageSize: context.pageSize, + dependencies: context.sharedDependencies, + }); + result.scannedPages = candidates.length; + + const pageResults = await mapConcurrent(candidates, context.concurrency, async (candidate) => { + try { + const page = await runLiferayInventoryPage( + config, + resolveInventoryPageRequest({url: candidate.fullUrl}), + context.sharedDependencies, + ); + const matches = matchEvidenceAgainstResource(extractPageEvidence(page), query); + if (matches.length === 0) return null; + return buildPageMatch(page, candidate, matches, config.liferay.url); + } catch (error) { + if (isSkippableWhereUsedCandidateError(candidate, error)) { + return null; + } + errors.push({fullUrl: candidate.fullUrl, reason: extractErrorMessage(error)}); + return 'failed' as const; + } + }); + + for (const item of pageResults) { + if (item === null) continue; + if (item === 'failed') { + result.failedPages += 1; + continue; + } + result.matchedPages.push(item); + } + + if (errors.length > 0) { + result.errors = errors; + } + return result; +} + +function summarize(siteResults: WhereUsedSiteResult[]): WhereUsedResult['summary'] { + return { + totalSites: siteResults.length, + totalScannedPages: siteResults.reduce((acc, site) => acc + site.scannedPages, 0), + totalMatchedPages: siteResults.reduce((acc, site) => acc + site.matchedPages.length, 0), + totalMatches: siteResults.reduce( + (acc, site) => acc + site.matchedPages.reduce((sum, page) => sum + page.matches.length, 0), + 0, + ), + totalFailedPages: siteResults.reduce((acc, site) => acc + site.failedPages, 0), + }; +} + +async function resolveTargetSites( + config: AppConfig, + siteOptions: string[] | undefined, + pageSize: number | undefined, + scopeOptions: ValidatedWhereUsedScopeOptions, + dependencies: WhereUsedDependencies, +): Promise { + const sites = await runLiferayInventorySitesIncludingGlobal(config, {pageSize: pageSize ?? 200}, dependencies); + + let contentStatsSites: ContentStatsSite[] | undefined; + let contentStatsSkippedSites: Array<{groupId: number; siteFriendlyUrl: string; reason: string}> | undefined; + + if ((!siteOptions || siteOptions.length === 0) && scopeOptions.siteOrder === 'content' && sites.length > 0) { + const contentStats = await runContentStats( + config, + { + limit: sites.length, + excludeSites: scopeOptions.excludedSites, + sortBy: 'content', + }, + dependencies, + ); + + if (contentStats.mode === 'sites') { + contentStatsSites = contentStats.sites; + contentStatsSkippedSites = contentStats.skippedSites; + } + } + + return selectWhereUsedSites({ + sites, + ...(siteOptions && siteOptions.length > 0 ? {explicitSites: siteOptions} : {}), + siteOrder: scopeOptions.siteOrder, + ...(scopeOptions.siteLimit !== undefined ? {siteLimit: scopeOptions.siteLimit} : {}), + excludedSites: scopeOptions.excludedSites, + ...(contentStatsSites ? {contentStatsSites} : {}), + ...(contentStatsSkippedSites ? {contentStatsSkippedSites} : {}), + }); +} + +export function selectWhereUsedSites(input: WhereUsedSiteSelectionInput): WhereUsedSiteSelection { + if (input.explicitSites && input.explicitSites.length > 0) { + const explicitSites = input.explicitSites.map((site) => site.trim()).filter((site) => site !== ''); + const selectedSites = explicitSites.map((explicitSite) => { + return ( + input.sites.find( + (site) => + site.siteFriendlyUrl === explicitSite || + site.siteFriendlyUrl === `/${explicitSite}` || + String(site.groupId) === explicitSite, + ) ?? { + groupId: -1, + siteFriendlyUrl: explicitSite.startsWith('/') ? explicitSite : `/${explicitSite}`, + name: explicitSite, + pagesCommand: buildPagesCommand(explicitSite), + } + ); + }); + + const uniqueSelectedSites = selectedSites.filter( + (site, index, allSites) => + allSites.findIndex((candidate) => candidate.siteFriendlyUrl === site.siteFriendlyUrl) === index, + ); + + return { + selectedSites: uniqueSelectedSites, + planSites: uniqueSelectedSites.map((site, index) => ({ + rank: index + 1, + siteFriendlyUrl: site.siteFriendlyUrl, + siteName: site.name, + groupId: site.groupId, + selectionReason: 'explicitSite', + })), + totalSites: input.sites.length, + excludedCount: 0, + skippedSites: input.contentStatsSkippedSites ?? [], + }; + } + + const excludedSites = new Set(input.excludedSites); + const filteredSites = input.sites.filter((site) => !excludedSites.has(site.siteFriendlyUrl)); + const structuredContentsBySite = new Map( + (input.contentStatsSites ?? []).map((site) => [site.siteFriendlyUrl, site.structuredContents]), + ); + + const orderedSites = filteredSites.slice().sort((left, right) => { + if (input.siteOrder === 'content') { + const leftCount = structuredContentsBySite.get(left.siteFriendlyUrl); + const rightCount = structuredContentsBySite.get(right.siteFriendlyUrl); + + if (leftCount !== undefined && rightCount !== undefined && leftCount !== rightCount) { + return rightCount - leftCount; + } + if (leftCount !== undefined && rightCount === undefined) return -1; + if (leftCount === undefined && rightCount !== undefined) return 1; + return left.siteFriendlyUrl.localeCompare(right.siteFriendlyUrl); + } + + if (input.siteOrder === 'name') { + const byName = left.name.localeCompare(right.name); + return byName !== 0 ? byName : left.siteFriendlyUrl.localeCompare(right.siteFriendlyUrl); + } + + return left.siteFriendlyUrl.localeCompare(right.siteFriendlyUrl); + }); + + const limitedSites = input.siteLimit !== undefined ? orderedSites.slice(0, input.siteLimit) : orderedSites; + + return { + selectedSites: limitedSites, + planSites: limitedSites.map((site, index) => ({ + rank: index + 1, + siteFriendlyUrl: site.siteFriendlyUrl, + siteName: site.name, + groupId: site.groupId, + ...(structuredContentsBySite.has(site.siteFriendlyUrl) + ? {structuredContents: structuredContentsBySite.get(site.siteFriendlyUrl)} + : {}), + selectionReason: input.siteOrder === 'content' ? 'contentOrder' : 'siteOrder', + })), + totalSites: input.sites.length, + excludedCount: input.sites.length - filteredSites.length, + skippedSites: input.contentStatsSkippedSites ?? [], + }; +} + +function extractErrorMessage(error: unknown): string { + if (error instanceof CliError) return error.message; + if (error instanceof Error) return error.message; + return String(error); +} + +export function isSkippableWhereUsedCandidateError(candidate: WhereUsedCandidateLike, error: unknown): boolean { + if (candidate.origin !== 'jsonwsJournal') { + return false; + } + + const message = extractErrorMessage(error); + return message.includes('No structured content found with friendlyUrlPath='); +} diff --git a/src/features/liferay/liferay-gateway.ts b/src/features/liferay/liferay-gateway.ts index c7ca8cb..2bd1e38 100644 --- a/src/features/liferay/liferay-gateway.ts +++ b/src/features/liferay/liferay-gateway.ts @@ -1,4 +1,5 @@ import type {AppConfig} from '../../core/config/load-config.js'; +import {CliError} from '../../core/errors.js'; import type {OAuthTokenClient} from '../../core/http/auth.js'; import {createOAuthTokenClient} from '../../core/http/auth.js'; import type {HttpRequestOptions, HttpResponse, HttpApiClient} from '../../core/http/client.js'; @@ -101,7 +102,7 @@ export class LiferayGateway { }); }); - const success = expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + const success = expectGatewayJsonSuccess(response, label, path); return (success.data ?? null) as T; } @@ -114,7 +115,7 @@ export class LiferayGateway { this.apiClient.postJson(this.config.liferay.url, path, payload, buildAuthOptions(this.config, accessToken)), ); - const success = expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + const success = expectGatewayJsonSuccess(response, label, path); return (success.data ?? null) as T; } @@ -127,7 +128,7 @@ export class LiferayGateway { this.apiClient.postForm(this.config.liferay.url, path, form, buildAuthOptions(this.config, accessToken)), ); - const success = expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + const success = expectGatewayJsonSuccess(response, label, path); return (success.data ?? null) as T; } @@ -140,7 +141,7 @@ export class LiferayGateway { this.apiClient.postMultipart(this.config.liferay.url, path, form, buildAuthOptions(this.config, accessToken)), ); - const success = expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + const success = expectGatewayJsonSuccess(response, label, path); return (success.data ?? null) as T; } @@ -161,7 +162,7 @@ export class LiferayGateway { }); }); - const success = expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + const success = expectGatewayJsonSuccess(response, label, path); return (success.data ?? null) as T; } @@ -195,7 +196,7 @@ export class LiferayGateway { this.apiClient.delete(this.config.liferay.url, path, buildAuthOptions(this.config, accessToken)), ); - const success = expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + const success = expectGatewayJsonSuccess(response, label, path); return (success.data ?? null) as T; } @@ -223,3 +224,14 @@ export function createLiferayGateway( ): LiferayGateway { return new LiferayGateway(config, apiClient ?? createLiferayApiClient(), tokenClient ?? createOAuthTokenClient()); } + +function expectGatewayJsonSuccess(response: HttpResponse, label: string, path: string): HttpResponse { + try { + return expectJsonSuccess(response, label, 'LIFERAY_GATEWAY_ERROR'); + } catch (error) { + if (error instanceof CliError && error.code === 'LIFERAY_GATEWAY_ERROR' && !error.message.includes(' path=')) { + error.message = `${error.message} path=${path}`; + } + throw error; + } +} diff --git a/src/features/mcp-server/mcp-server-tools.ts b/src/features/mcp-server/mcp-server-tools.ts index 168e7ac..2b94d41 100644 --- a/src/features/mcp-server/mcp-server-tools.ts +++ b/src/features/mcp-server/mcp-server-tools.ts @@ -7,6 +7,7 @@ import * as sitesTool from './tools/tool-liferay-inventory-sites.js'; import * as structuresTool from './tools/tool-liferay-inventory-structures.js'; import * as pagesTool from './tools/tool-liferay-inventory-pages.js'; import * as pageTool from './tools/tool-liferay-inventory-page.js'; +import * as whereUsedTool from './tools/tool-liferay-inventory-where-used.js'; import * as checkTool from './tools/tool-liferay-check.js'; import * as doctorTool from './tools/tool-liferay-doctor.js'; import * as templatesTool from './tools/tool-liferay-inventory-templates.js'; @@ -33,6 +34,7 @@ export const ALL_TOOLS: McpToolModule[] = [ structuresTool, pagesTool, pageTool, + whereUsedTool, doctorTool, templatesTool, deployStatusTool, diff --git a/src/features/mcp-server/tools/tool-liferay-inventory-where-used.ts b/src/features/mcp-server/tools/tool-liferay-inventory-where-used.ts new file mode 100644 index 0000000..be4465f --- /dev/null +++ b/src/features/mcp-server/tools/tool-liferay-inventory-where-used.ts @@ -0,0 +1,67 @@ +import {z} from 'zod'; +import type {AppConfig} from '../../../core/config/schema.js'; +import { + runLiferayInventoryWhereUsed, + type WhereUsedResourceType, +} from '../../liferay/inventory/liferay-inventory-where-used.js'; +import {runJsonTool} from './tool-result.js'; + +export const TOOL_NAME = 'liferay_inventory_where_used'; + +export const inputSchema = { + type: z + .enum(['fragment', 'widget', 'portlet', 'structure', 'template', 'adt']) + .describe('Portal resource type to reverse-lookup'), + keys: z.array(z.string()).min(1).describe('One or more resource keys to OR-match'), + sites: z.array(z.string()).optional().describe('Limit lookup to one or more sites'), + excludeSites: z.array(z.string()).optional().describe('Exclude sites when scanning all accessible sites'), + widgetType: z.string().optional().describe('ADT widget type filter used only when type=adt'), + className: z.string().optional().describe('ADT class name filter used only when type=adt'), + includePrivate: z.boolean().optional().describe('Also scan private layouts'), + siteLimit: z.number().optional().describe('Maximum number of sites to scan when sites is not provided'), + siteOrder: z.enum(['site', 'name', 'content']).optional().describe('Site prioritization: site | name | content'), + plan: z.boolean().optional().describe('Return the selected site scan plan without inspecting pages'), + maxDepth: z.number().optional().describe('Maximum page tree recursion depth'), + concurrency: z.number().optional().describe('Parallel page fetches per site'), + pageSize: z.number().optional().describe('Headless page size for site listings'), +}; + +export const description = + 'Reverse-lookup Pages that contain a given Fragment, Widget, Portlet, Structure, Template, or ADT.'; + +export async function handleTool( + input: { + type: WhereUsedResourceType; + keys: string[]; + sites?: string[]; + excludeSites?: string[]; + widgetType?: string; + className?: string; + includePrivate?: boolean; + siteLimit?: number; + siteOrder?: 'site' | 'name' | 'content'; + plan?: boolean; + maxDepth?: number; + concurrency?: number; + pageSize?: number; + }, + config: AppConfig, +) { + return runJsonTool(() => + runLiferayInventoryWhereUsed(config, { + type: input.type, + keys: input.keys, + ...(input.sites ? {sites: input.sites} : {}), + ...(input.excludeSites ? {excludeSites: input.excludeSites} : {}), + ...(input.widgetType ? {widgetType: input.widgetType} : {}), + ...(input.className ? {className: input.className} : {}), + ...(input.includePrivate !== undefined ? {includePrivate: input.includePrivate} : {}), + ...(input.siteLimit !== undefined ? {siteLimit: input.siteLimit} : {}), + ...(input.siteOrder ? {siteOrder: input.siteOrder} : {}), + ...(input.plan !== undefined ? {plan: input.plan} : {}), + ...(input.maxDepth !== undefined ? {maxDepth: input.maxDepth} : {}), + ...(input.concurrency !== undefined ? {concurrency: input.concurrency} : {}), + ...(input.pageSize !== undefined ? {pageSize: input.pageSize} : {}), + }), + ); +} diff --git a/src/testing/fake-docker.ts b/src/testing/fake-docker.ts index 893b017..8c7dc9a 100644 --- a/src/testing/fake-docker.ts +++ b/src/testing/fake-docker.ts @@ -82,6 +82,10 @@ function fail(message) { process.exit(1); } +function sleep(ms) { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + function decodeEscapes(text) { return text .replace(/\\\\n/g, '\\n') @@ -108,6 +112,9 @@ if ( args[0] === 'compose' && ['pull', 'up', 'stop', 'down', 'restart', 'logs', 'rm'].includes(args[1] ?? '') ) { + if (args[1] === 'up' && process.env.FAKE_DOCKER_DELAY_COMPOSE_UP_MS) { + await sleep(Number(process.env.FAKE_DOCKER_DELAY_COMPOSE_UP_MS)); + } if (args[1] === 'logs' && process.env.FAKE_DOCKER_LOGS_OUTPUT) { print(decodeEscapes(process.env.FAKE_DOCKER_LOGS_OUTPUT)); } diff --git a/templates/ai/skills/developing-liferay/SKILL.md b/templates/ai/skills/developing-liferay/SKILL.md index 7a560d7..001511f 100644 --- a/templates/ai/skills/developing-liferay/SKILL.md +++ b/templates/ai/skills/developing-liferay/SKILL.md @@ -5,8 +5,7 @@ description: 'Guides implementation changes in Liferay projects that run with ld # Developing Liferay -Use this skill after the affected surface is clear. For issue-scale work, the -outer gate owner should be `runtime-change-workflow`. +Use this skill after the affected surface is clear. For issue-scale work, the outer gate owner should be `runtime-change-workflow`. ## Bootstrap @@ -22,20 +21,15 @@ Inspect: - `context.commands.*` - `doctor.readiness.*` -If these fields are missing, stop and report that the installed `ldev` AI assets -are out of sync with the CLI. +If these fields are missing, stop and report that the installed `ldev` AI assets are out of sync with the CLI. ## Discover Before Editing -If the task mentions a site, page, URL, structure, template, ADT, or fragment, -resolve it with the portal discovery contract in -[../../docs/PORTAL_DISCOVERY.md](../../docs/PORTAL_DISCOVERY.md) before code -search or edits. +If the task mentions a site, page, URL, structure, template, ADT, or fragment, resolve it with the portal discovery contract in [../../docs/PORTAL_DISCOVERY.md](../../docs/PORTAL_DISCOVERY.md) before code search or edits. -Use local `ldev` MCP tools for read-only inventory when visible. Use the CLI for -file exports/imports and file-backed resource mutations. For structured content -or site page mutations that do not yet have a dedicated `ldev` command, prefer -OAuth-backed Headless APIs plus read-back proof. +Preferred discovery commands live in `PORTAL_DISCOVERY.md`; use `inventory where-used` when the key is already known and impact must be scoped before editing. + +Use local `ldev` MCP tools for read-only inventory when visible. Use the CLI for file exports/imports and file-backed resource mutations. For structured content or site page mutations that do not yet have a dedicated `ldev` command, prefer OAuth-backed Headless APIs plus read-back proof. ## Choose The Implementation Path @@ -64,9 +58,11 @@ If more than one surface changed, apply and verify each surface separately. ## Resource Boundary -For structures, templates, ADTs, and fragments, do not use deploy commands. -Switch to `portal-resource-workflow`, which owns source-of-truth resolution, -export/import, import-vs-migration, read-after-write, and browser validation. +For structures, templates, ADTs, and fragments, do not use deploy commands. Switch to `portal-resource-workflow`, which owns source-of-truth resolution, export/import, import-vs-migration, read-after-write, and browser validation. + +If you already know the resource key and need impact analysis before editing it, use `ldev portal inventory where-used`. It is the preferred discovery step for “what Pages will I affect if I change this Structure, Template, Fragment, widget, or ADT?” + +Prefer `--site` by default so discovery stays fast and scoped to the Site you are already working on. For production promotion notes for runtime-backed resources, use `references/runtime-resource-production-handoff.md`. diff --git a/templates/ai/skills/liferay-expert/SKILL.md b/templates/ai/skills/liferay-expert/SKILL.md index 7c2d7d0..24a1fa0 100644 --- a/templates/ai/skills/liferay-expert/SKILL.md +++ b/templates/ai/skills/liferay-expert/SKILL.md @@ -5,8 +5,7 @@ description: 'Routes technical Liferay work to the right ldev specialist workflo # Liferay Expert -This is the domain router for reusable `ldev` Liferay workflows. It should -classify quickly and then hand off; deep playbooks live in specialist skills. +This is the domain router for reusable `ldev` Liferay workflows. Classify quickly and hand off; deep playbooks live in specialist skills. ## Bootstrap @@ -21,15 +20,11 @@ Use `bootstrap.context` to route: - `context.liferay.auth.oauth2.*.status` for configured credentials. - `context.paths.resources.*` for local resource directories. -If required fields are missing, stop and report that the installed `ldev` AI -assets are out of sync with the CLI. +If required fields are missing, stop and report that the installed `ldev` AI assets are out of sync with the CLI. ## Resolve Runtime Context -If the task mentions a site, page, URL, structure, template, ADT, or fragment, -resolve it with the portal discovery contract in -[../../docs/PORTAL_DISCOVERY.md](../../docs/PORTAL_DISCOVERY.md) before -searching or editing. +If the task mentions a site, page, URL, structure, template, ADT, or fragment, resolve it with the portal discovery contract in [../../docs/PORTAL_DISCOVERY.md](../../docs/PORTAL_DISCOVERY.md) before searching or editing. ## Routing @@ -41,24 +36,43 @@ searching or editing. - Journal structure change with data movement or compatibility risk -> `migrating-journal-structures` - Browser reproduction or visual proof -> `automating-browser-tests` -For deeper routing examples, read `references/routing.md`. For Display Page -Templates, Navigation Menus, multi-site ownership, and content volume checks, -read `references/site-objects.md`. +For deeper routing examples, read `references/routing.md`. For Display Page Templates, Navigation Menus, multi-site ownership, and content volume checks, read `references/site-objects.md`. ## Command Boundaries - `ldev context --json`: offline repo/config facts. - `ldev status --json`: Docker/runtime state. -- `ldev doctor --json`: active checks and readiness; add `--runtime`, - `--portal`, or `--osgi` when that surface matters. +- `ldev doctor --json`: active checks and readiness; add `--runtime`, `--portal`, or `--osgi` when that surface matters. +- `ldev portal inventory ... --json`: resolve site, page, structure, template, ADT, and where-used context before edits. Do not substitute these commands for each other in plans or handoffs. +Use `inventory structures --with-templates` for structure/template discovery, `inventory page --url --json --full` only when routing needs expanded page details, and `inventory where-used` when the task starts from a known key and needs impact analysis. Prefer `--site` unless a cross-site answer is required. + +MCP equivalents when visible: `liferay_inventory_page`, `liferay_inventory_structures`, `liferay_inventory_templates`. + +## AI asset maintenance + +When skills or agent context files are out of date, run `ldev ai status --target --json` first, then `ldev ai update --target ` or `ldev ai update --target --skill `. + +## OAuth2 prerequisite + +Most portal and resource commands require OAuth2 credentials. If `context.liferay.auth.oauth2.clientId.status` is not `"present"`, set up credentials first: + +```bash +ldev start +ldev oauth install --write-env +``` + +`--write-env` persists the credentials to `.liferay-cli.local.yml`. If the admin account is in password-reset state, unblock it first: + +```bash +ldev oauth admin-unblock +``` + ## Guardrails - Use `ldev` as the official interface. -- Prefer local `ldev` MCP tools for read-only discovery when visible; fall back - to CLI with `--json`. +- Prefer local `ldev` MCP tools for read-only discovery when visible; fall back to CLI with `--json`. - Do not invent portal mutations when an `ldev resource ...` workflow exists. -- Keep the smallest specialist skill active; do not carry every Liferay skill - into the same task unless routing proves it is needed. +- Keep the smallest specialist skill active; do not carry every Liferay skill into the same task unless routing proves it is needed. diff --git a/templates/ai/workspace-rules/ldev-portal-discovery.md b/templates/ai/workspace-rules/ldev-portal-discovery.md index 9d3771b..ef5cbd6 100644 --- a/templates/ai/workspace-rules/ldev-portal-discovery.md +++ b/templates/ai/workspace-rules/ldev-portal-discovery.md @@ -16,12 +16,14 @@ Recommended sequence: 1. `ldev portal inventory sites --json` 2. `ldev portal inventory pages --site /my-site --json` 3. `ldev portal inventory page --url /web/my-site/home --json` +4. `ldev portal inventory where-used --type structure --key --site /my-site --json` when the task asks where a resource is used Why: - task-shaped output - stable JSON contract - better page/context enrichment than low-level API assembly +- direct reverse lookup for portal resources without UI searching The default output is minimal and suitable for most discovery tasks. Use `--full` when you need raw data not present by default: @@ -35,6 +37,13 @@ ldev portal inventory page --url /web/my-site/home --json --full - For **regular pages**: `full.configurationRaw` (full `sitePageMetadata` + `pageDefinition`), `full.components.fragments` (with `editableFields` and `heroText`). +Use `where-used` when the task starts from a known resource key instead of a +known URL. It is the preferred route for questions like “which Pages use this +Structure, Template, ADT, widget, or Fragment?” + +Default to the scoped form with `--site`. A global scan across all accessible +Sites is slower and should be reserved for tasks that explicitly need it. + For the full workflow, route to vendor skills such as: - `liferay-expert` diff --git a/tests/integration/env.integration.test.ts b/tests/integration/env.integration.test.ts index e6cf702..1615afb 100644 --- a/tests/integration/env.integration.test.ts +++ b/tests/integration/env.integration.test.ts @@ -1,13 +1,14 @@ import fs from 'fs-extra'; import path from 'node:path'; -import {describe, expect, test} from 'vitest'; +import {describe, expect, test, vi} from 'vitest'; import {loadConfig} from '../../src/core/config/load-config.js'; import {runEnvInit} from '../../src/features/env/env-init.js'; import {runEnvRecreate} from '../../src/features/env/env-recreate.js'; import {runEnvRestore} from '../../src/features/env/env-restore.js'; import {runEnvSetup} from '../../src/features/env/env-setup.js'; +import {runEnvStart} from '../../src/features/env/env-start.js'; import {runProcess} from '../../src/core/platform/process.js'; import {createFakeDockerBin, readFakeDockerCalls} from '../../src/testing/fake-docker.js'; import {parseTestJson} from '../../src/testing/cli-test-helpers.js'; @@ -99,6 +100,38 @@ describe('env integration', () => { ); }, 45000); + test('env start abort rolls back docker compose when canceled during compose up', async () => { + const repoRoot = await createEnvRepoFixture(); + const fakeBinDir = await createFakeDockerBin(); + const processEnv = { + ...process.env, + PATH: `${fakeBinDir}:${process.env.PATH ?? ''}`, + FAKE_DOCKER_DELAY_COMPOSE_UP_MS: '250', + }; + const config = loadConfig({cwd: repoRoot, env: process.env}); + const controller = new AbortController(); + + const startPromise = runEnvStart(config, { + wait: false, + processEnv, + signal: controller.signal, + }); + + await vi.waitFor( + async () => { + const calls = await readFakeDockerCalls(fakeBinDir); + expect(calls).toContain('compose up -d'); + }, + {timeout: 5_000}, + ); + controller.abort(); + + await expect(startPromise).rejects.toThrow(); + + const calls = await readFakeDockerCalls(fakeBinDir); + expect(calls).toEqual(expect.arrayContaining(['compose up -d', 'compose stop', 'compose down'])); + }, 30000); + test('env start respects DOCLIB_PATH and does not remount doclib to the default bind path', async () => { const repoRoot = await createEnvRepoFixture(); const fakeBinDir = await createFakeDockerBin(); diff --git a/tests/unit/liferay-gateway.test.ts b/tests/unit/liferay-gateway.test.ts index 19344fb..7c36d5e 100644 --- a/tests/unit/liferay-gateway.test.ts +++ b/tests/unit/liferay-gateway.test.ts @@ -139,6 +139,20 @@ describe('LiferayGateway', () => { await expect(gateway.getJson('/api/test', 'fetch')).rejects.toThrow(/status=503/); }); + test('includes request path in error message', async () => { + const apiClient = createMockApiClient(); + const tokenClient = createMockTokenClient(); + const response = mockHttpResponse(false, 404, null); + + vi.mocked(apiClient.get).mockResolvedValue(response); + + const gateway = new LiferayGateway(mockConfig, apiClient, tokenClient); + + await expect( + gateway.getJson('/o/headless-delivery/v1.0/sites/20121/structured-contents', 'fetch'), + ).rejects.toThrow(/path=\/o\/headless-delivery\/v1\.0\/sites\/20121\/structured-contents/); + }); + test('returns null data as null', async () => { const apiClient = createMockApiClient(); const tokenClient = createMockTokenClient(); diff --git a/tests/unit/liferay-inventory-journal-article-resolver.test.ts b/tests/unit/liferay-inventory-journal-article-resolver.test.ts new file mode 100644 index 0000000..e96d5d1 --- /dev/null +++ b/tests/unit/liferay-inventory-journal-article-resolver.test.ts @@ -0,0 +1,77 @@ +import {beforeEach, describe, expect, test, vi} from 'vitest'; + +import {resolveJournalArticleReference} from '../../src/features/liferay/inventory/liferay-inventory-journal-article-resolver.js'; +import { + fetchLatestJournalArticle, + fetchStructuredContentById, + fetchStructuredContentByUuid, +} from '../../src/features/liferay/inventory/liferay-inventory-page-fetch-article.js'; + +vi.mock('../../src/features/liferay/inventory/liferay-inventory-page-fetch-article.js', () => ({ + fetchLatestJournalArticle: vi.fn(), + fetchStructuredContentById: vi.fn(), + fetchStructuredContentByUuid: vi.fn(), +})); + +describe('resolveJournalArticleReference', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + test('returns provided article and structured content without fetching', async () => { + const article = {articleId: 'ART-1', uuid: 'uuid-1'}; + const structuredContent = {id: 101, key: 'ART-1', contentStructureId: 301}; + + const result = await resolveJournalArticleReference( + {} as never, + {articleId: 'ART-1', groupId: 20121}, + {article, structuredContent}, + ); + + expect(result).toEqual({article, structuredContent, resolvedArticleId: 'ART-1'}); + expect(fetchStructuredContentById).not.toHaveBeenCalled(); + expect(fetchLatestJournalArticle).not.toHaveBeenCalled(); + expect(fetchStructuredContentByUuid).not.toHaveBeenCalled(); + }); + + test('uses structured content key as resolved article id when ref has no articleId', async () => { + vi.mocked(fetchStructuredContentById).mockResolvedValue({id: 101, key: 'ART-1', contentStructureId: 301} as never); + vi.mocked(fetchLatestJournalArticle).mockResolvedValue({articleId: 'ART-1', uuid: 'uuid-1'} as never); + + const result = await resolveJournalArticleReference({} as never, { + articleId: '', + groupId: 20121, + structuredContentId: 101, + }); + + expect(result.resolvedArticleId).toBe('ART-1'); + expect(fetchStructuredContentById).toHaveBeenCalledWith(expect.anything(), 101); + expect(fetchLatestJournalArticle).toHaveBeenCalledWith(expect.anything(), 20121, 'ART-1'); + }); + + test('resolves structured content by article uuid before falling back to article id', async () => { + vi.mocked(fetchLatestJournalArticle).mockResolvedValue({articleId: 'ART-1', uuid: 'uuid-1', id: 999} as never); + vi.mocked(fetchStructuredContentByUuid).mockResolvedValue({ + id: 101, + key: 'ART-1', + contentStructureId: 301, + } as never); + + const result = await resolveJournalArticleReference({} as never, {articleId: 'ART-1', groupId: 20121}); + + expect(result.structuredContent).toEqual({id: 101, key: 'ART-1', contentStructureId: 301}); + expect(fetchStructuredContentByUuid).toHaveBeenCalledWith(expect.anything(), 20121, 'uuid-1'); + expect(fetchStructuredContentById).not.toHaveBeenCalledWith(expect.anything(), 999); + }); + + test('falls back to article numeric id when uuid does not resolve structured content', async () => { + vi.mocked(fetchLatestJournalArticle).mockResolvedValue({articleId: 'ART-1', uuid: 'uuid-1', id: 999} as never); + vi.mocked(fetchStructuredContentByUuid).mockResolvedValue(null); + vi.mocked(fetchStructuredContentById).mockResolvedValue({id: 999, key: 'ART-1', contentStructureId: 301} as never); + + const result = await resolveJournalArticleReference({} as never, {articleId: 'ART-1', groupId: 20121}); + + expect(result.structuredContent).toEqual({id: 999, key: 'ART-1', contentStructureId: 301}); + expect(fetchStructuredContentById).toHaveBeenCalledWith(expect.anything(), 999); + }); +}); diff --git a/tests/unit/liferay-inventory-page-fetch-journal.test.ts b/tests/unit/liferay-inventory-page-fetch-journal.test.ts new file mode 100644 index 0000000..03332dc --- /dev/null +++ b/tests/unit/liferay-inventory-page-fetch-journal.test.ts @@ -0,0 +1,111 @@ +import {beforeEach, describe, expect, test, vi} from 'vitest'; + +const fetchGroupInfoMock = vi.fn(); +const resolveJournalArticleReferenceMock = vi.fn(); + +vi.mock('../../src/features/liferay/portal/site-resolution.js', () => ({ + buildSiteChain: vi.fn(), + fetchGroupInfo: fetchGroupInfoMock, +})); + +vi.mock('../../src/features/liferay/inventory/liferay-inventory-journal-article-resolver.js', () => ({ + resolveJournalArticleReference: resolveJournalArticleReferenceMock, +})); + +const {collectLayoutJournalArticles} = + await import('../../src/features/liferay/inventory/liferay-inventory-page-fetch-journal.js'); + +describe('collectLayoutJournalArticles', () => { + beforeEach(() => { + vi.clearAllMocks(); + fetchGroupInfoMock.mockImplementation((_gateway: unknown, groupId: number) => ({ + friendlyUrl: `/site-${groupId}`, + name: `Site ${groupId}`, + parentGroupId: 0, + })); + resolveJournalArticleReferenceMock.mockImplementation((_gateway: unknown, ref: {groupId: number}) => ({ + article: { + articleId: `ART-${ref.groupId}`, + titleCurrentValue: `Article ${ref.groupId}`, + }, + structuredContent: null, + resolvedArticleId: `ART-${ref.groupId}`, + })); + }); + + test('keeps articles from different groups when portlet preferences reuse the same article id', async () => { + const pageElement = { + pageElements: [ + { + portletPreferencesMap: { + articleId: ['SHARED-ARTICLE'], + groupId: ['101'], + }, + }, + { + portletPreferencesMap: { + articleId: ['SHARED-ARTICLE'], + groupId: ['202'], + }, + }, + ], + }; + + const result = await collectLayoutJournalArticles( + {} as never, + {liferay: {url: 'http://localhost:8080'}} as never, + {} as never, + 999, + pageElement as never, + ); + + expect(resolveJournalArticleReferenceMock).toHaveBeenCalledTimes(2); + expect(result).toHaveLength(2); + expect(result.map((item) => item.groupId)).toEqual([101, 202]); + expect(result.map((item) => item.articleId)).toEqual(['ART-101', 'ART-202']); + }); + + test('merges complementary article ref fields for the same article and group', async () => { + fetchGroupInfoMock.mockRejectedValueOnce(new Error('skip site enrichment')); + + const pageElement = { + pageElements: [ + { + portletPreferencesMap: { + articleId: ['SHARED-ARTICLE'], + groupId: ['101'], + ddmTemplateKey: ['NEWS_TEMPLATE'], + }, + }, + { + itemReference: { + className: 'com.liferay.journal.model.JournalArticle', + articleId: 'SHARED-ARTICLE', + groupId: '101', + classPK: '555', + }, + }, + ], + }; + + await collectLayoutJournalArticles( + {} as never, + {liferay: {url: 'http://localhost:8080'}} as never, + {} as never, + 999, + pageElement as never, + ); + + expect(resolveJournalArticleReferenceMock).toHaveBeenCalledTimes(1); + expect(resolveJournalArticleReferenceMock).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ + articleId: 'SHARED-ARTICLE', + groupId: 101, + ddmTemplateKey: 'NEWS_TEMPLATE', + structuredContentId: 555, + }), + {article: undefined, structuredContent: undefined}, + ); + }); +}); diff --git a/tests/unit/liferay-inventory-page.test.ts b/tests/unit/liferay-inventory-page.test.ts index 702cab9..af55a5c 100644 --- a/tests/unit/liferay-inventory-page.test.ts +++ b/tests/unit/liferay-inventory-page.test.ts @@ -8,6 +8,7 @@ import { resolveInventoryPageRequest, runLiferayInventoryPage, } from '../../src/features/liferay/inventory/liferay-inventory-page.js'; +import {matchPageAgainstResource} from '../../src/features/liferay/inventory/liferay-inventory-where-used.js'; import { isRegularPageRequest, isSiteRootRequest, @@ -28,10 +29,6 @@ function pageDefinitionResp(pageElements: unknown[] = []) { return new Response(JSON.stringify({pageDefinition: {pageElement: {type: 'Root', pageElements}}}), {status: 200}); } -function fragmentEntryLinksResp(items: unknown[] = []) { - return new Response(JSON.stringify(items), {status: 200}); -} - function classNameIdResp(classNameId = 20006) { return new Response(JSON.stringify({classNameId}), {status: 200}); } @@ -188,6 +185,15 @@ describe('liferay inventory page', () => { }); }); + test('decodes percent-encoded display page url titles', () => { + expect(resolveInventoryPageRequest({url: '/web/ub/w/%C3%88xit-de-les-seleccions'})).toMatchObject({ + kind: 'webContentDisplayPage', + site: 'ub', + friendlyUrl: '/w/%C3%88xit-de-les-seleccions', + urlTitle: 'Èxit-de-les-seleccions', + }); + }); + test('ignores absolute URL origin and keeps configured portal URL', async () => { vi.spyOn(globalThis, 'fetch').mockRejectedValue(new Error('network disabled for test')); @@ -257,10 +263,6 @@ describe('liferay inventory page', () => { return pageDefinitionResp(); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -338,58 +340,6 @@ describe('liferay inventory page', () => { ); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp([ - { - portletId: 'com_liferay_journal_content_web_portlet_JournalContentPortlet_INSTANCE_abc', - editableValues: JSON.stringify({ - journal_content: { - portletPreferencesMap: { - articleId: ['ART-001'], - groupId: ['20121'], - ddmTemplateKey: ['TPL-1'], - }, - }, - }), - }, - ]); - } - - if (url.includes('/journal.journalarticle/get-latest-article')) { - return new Response( - JSON.stringify({ - id: 41001, - articleId: 'ART-001', - titleCurrentValue: 'Home article', - ddmStructureKey: 'BASIC', - }), - {status: 200}, - ); - } - - if (url.endsWith('/o/headless-delivery/v1.0/structured-contents/41001')) { - return new Response( - JSON.stringify({ - id: 41001, - contentStructureId: 301, - priority: 0, - contentFields: [ - { - label: 'Headline', - name: 'headline', - dataType: 'string', - contentFieldValue: {data: 'Hello'}, - }, - ], - }), - {status: 200}, - ); - } - - if (url.endsWith('/o/headless-delivery/v1.0/content-structures/301')) { - return new Response(JSON.stringify({id: 301, name: 'Basic Web Content'}), {status: 200}); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -447,37 +397,379 @@ describe('liferay inventory page', () => { { type: 'widget', widgetName: 'com_liferay_journal_content_web_portlet_JournalContentPortlet', - portletId: 'com_liferay_journal_content_web_portlet_JournalContentPortlet_INSTANCE_abc', }, ]); - expect(result.journalArticles).toEqual([ + expect(result.journalArticles).toEqual([]); + expect(result.contentStructures).toEqual([]); + expect(formatLiferayInventoryPage(result)).toContain('REGULAR PAGE'); + }); + + test('accepts widget ADT evidence in validated page results', () => { + const result = { + pageType: 'regularPage', + pageSubtype: 'content', + pageUiType: 'Content Page', + siteName: 'UB', + siteFriendlyUrl: '/ub', + groupId: 20121, + url: '/web/ub/rss', + friendlyUrl: '/rss', + pageName: 'RSS', + privateLayout: false, + layout: { + layoutId: 11, + plid: 1011, + friendlyUrl: '/rss', + type: 'content', + hidden: false, + }, + layoutDetails: {}, + adminUrls: { + view: '', + edit: '', + configureGeneral: '', + configureDesign: '', + configureSeo: '', + configureOpenGraph: '', + configureCustomMetaTags: '', + translate: '', + }, + evidence: [ + { + resourceType: 'adt', + key: 'ddmTemplate_40801', + kind: 'widgetAdt', + detail: 'widgetName=asset-publisher index=0 displayStyle=ddmTemplate_40801', + source: 'fragmentEntryLink', + }, + ], + }; + + expect(() => validateLiferayInventoryPageResultV2(result)).not.toThrow(); + }); + + test('includes static journal-content template evidence from rendered widget-page HTML', async () => { + const apiClient = createLiferayApiClient({ + fetchImpl: createTestFetchImpl((url) => { + if (url === 'http://localhost:8080/web/guest/home') { + return new Response( + '
' + + '
' + + '
' + + '
' + + '
', + {status: 200}, + ); + } + + if (url.includes('/by-friendly-url-path/guest')) { + return siteResp(20121, '/guest', 'Guest'); + } + + if (url.includes('/by-friendly-url-path/global')) { + return siteResp(20122, '/global', 'Global'); + } + + if (url.includes('/api/jsonws/group/get-group?groupId=20121')) { + return groupResp(10157, '/guest', 'Guest'); + } + + if (url.includes('parentLayoutId=0')) { + return layoutsResp([ + { + layoutId: 11, + plid: 1011, + type: 'portlet', + nameCurrentValue: 'Home', + friendlyURL: '/home', + hidden: false, + typeSettings: + 'layout-template-id=home\ncolumn-top=com_liferay_asset_publisher_web_portlet_AssetPublisherPortlet_INSTANCE_top\n', + }, + ]); + } + + if (url.includes('parentLayoutId=11')) { + return layoutsResp([]); + } + + if ( + url.includes( + '/api/jsonws/journal.journalarticle/get-latest-article?groupId=20121&articleId=ART-HEADER&status=0', + ) + ) { + return new Response( + JSON.stringify({ + id: 41001, + resourcePrimKey: 41001, + articleId: 'ART-HEADER', + titleCurrentValue: 'No editar - Menú superior', + ddmStructureKey: 'HEADER_MENU', + ddmTemplateKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + contentStructureId: 301, + }), + {status: 200}, + ); + } + + if (url.endsWith('/o/headless-delivery/v1.0/structured-contents/41001')) { + return new Response( + JSON.stringify({ + id: 41001, + key: 'ART-HEADER', + title: 'No editar - Menú superior', + contentStructureId: 301, + contentFields: [], + }), + {status: 200}, + ); + } + + if ( + url.includes( + '/o/data-engine/v2.0/sites/20121/data-definitions/by-content-type/journal/by-data-definition-key/HEADER_MENU', + ) + ) { + return new Response('{"id":301,"dataDefinitionKey":"HEADER_MENU","name":{"en_US":"Header menu"}}', { + status: 200, + }); + } + + if (url.endsWith('/o/headless-delivery/v1.0/content-structures/301')) { + return new Response('{"id":301,"dataDefinitionKey":"HEADER_MENU","name":"Header menu"}', {status: 200}); + } + + if ( + url.includes( + '/api/jsonws/classname/fetch-class-name?value=com.liferay.dynamic.data.mapping.model.DDMStructure', + ) + ) { + return new Response('{"classNameId":1001}', {status: 200}); + } + + if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.journal.model.JournalArticle')) { + return new Response('{"classNameId":1002}', {status: 200}); + } + + if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { + return classNameIdResp(); + } + + if ( + url.includes( + '/api/jsonws/ddm.ddmtemplate/get-templates?companyId=10157&groupId=20121&classNameId=1001&resourceClassNameId=1002&status=0', + ) + ) { + return new Response( + '[{"templateId":"40801","templateKey":"UB_TPL_ENLACES_MENU_SUPERIOR","nameCurrentValue":"Menu superior","classPK":301}]', + {status: 200}, + ); + } + + throw new Error(`Unexpected URL ${url}`); + }), + }); + + const result = await runLiferayInventoryPage( + CONFIG, + {url: '/web/guest/home'}, + {apiClient, tokenClient: TOKEN_CLIENT}, + ); + + expect(() => validateLiferayInventoryPageResultV2(result)).not.toThrow(); + if (result.pageType !== 'regularPage') { + throw new Error('Expected regular page'); + } + + expect(result.journalArticles).toMatchObject([ + { + articleId: 'ART-HEADER', + title: 'No editar - Menú superior', + ddmTemplateKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + }, + ]); + expect(matchPageAgainstResource(result, {type: 'template', keys: ['UB_TPL_ENLACES_MENU_SUPERIOR']})).toEqual([ { - groupId: 20121, - articleId: 'ART-001', - title: 'Home article', - ddmStructureKey: 'BASIC', - ddmTemplateKey: 'TPL-1', - contentStructureId: 301, - contentFields: [ - { - path: 'Headline', - label: 'Headline', - name: 'headline', - type: 'string', - value: 'Hello', - }, - ], + resourceType: 'template', + matchedKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + matchKind: 'journalArticleTemplate', + label: 'Journal article template (static Journal Content rendered in HTML)', + detail: 'articleId=ART-HEADER title=No editar - Menú superior', + source: 'renderedHtmlJournalContent', + }, + ]); + }); + + test('includes static journal-content template evidence from rendered content-page HTML', async () => { + const apiClient = createLiferayApiClient({ + fetchImpl: createTestFetchImpl((url) => { + if (url === 'http://localhost:8080/web/guest/home') { + return new Response( + '
' + + '
' + + '
' + + '
' + + '
', + {status: 200}, + ); + } + + if (url.includes('/by-friendly-url-path/guest')) { + return siteResp(20121, '/guest', 'Guest'); + } + + if (url.includes('/by-friendly-url-path/global')) { + return siteResp(20122, '/global', 'Global'); + } + + if (url.includes('/api/jsonws/group/get-group?groupId=20121')) { + return groupResp(10157, '/guest', 'Guest'); + } + + if (url.includes('parentLayoutId=0')) { + return layoutsResp([ + { + layoutId: 11, + plid: 1011, + type: 'content', + nameCurrentValue: 'Home', + friendlyURL: '/home', + hidden: false, + }, + ]); + } + + if (url.includes('parentLayoutId=11')) { + return layoutsResp([]); + } + + if (url.includes('/o/headless-delivery/v1.0/sites/20121/site-pages/home?fields=pageDefinition')) { + return new Response( + JSON.stringify({ + pageDefinition: { + pageElement: { + type: 'Root', + pageElements: [ + { + type: 'Fragment', + definition: { + fragment: {key: 'banner'}, + }, + }, + ], + }, + }, + }), + {status: 200}, + ); + } + + if ( + url.includes( + '/api/jsonws/journal.journalarticle/get-latest-article?groupId=20121&articleId=ART-HEADER&status=0', + ) + ) { + return new Response( + JSON.stringify({ + id: 41001, + resourcePrimKey: 41001, + articleId: 'ART-HEADER', + titleCurrentValue: 'No editar - Menú superior', + ddmStructureKey: 'HEADER_MENU', + ddmTemplateKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + contentStructureId: 301, + }), + {status: 200}, + ); + } + + if (url.endsWith('/o/headless-delivery/v1.0/structured-contents/41001')) { + return new Response( + JSON.stringify({ + id: 41001, + key: 'ART-HEADER', + title: 'No editar - Menú superior', + contentStructureId: 301, + contentFields: [], + }), + {status: 200}, + ); + } + + if ( + url.includes( + '/o/data-engine/v2.0/sites/20121/data-definitions/by-content-type/journal/by-data-definition-key/HEADER_MENU', + ) + ) { + return new Response('{"id":301,"dataDefinitionKey":"HEADER_MENU","name":{"en_US":"Header menu"}}', { + status: 200, + }); + } + + if (url.endsWith('/o/headless-delivery/v1.0/content-structures/301')) { + return new Response('{"id":301,"dataDefinitionKey":"HEADER_MENU","name":"Header menu"}', {status: 200}); + } + + if ( + url.includes( + '/api/jsonws/classname/fetch-class-name?value=com.liferay.dynamic.data.mapping.model.DDMStructure', + ) + ) { + return new Response('{"classNameId":1001}', {status: 200}); + } + + if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.journal.model.JournalArticle')) { + return new Response('{"classNameId":1002}', {status: 200}); + } + + if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { + return classNameIdResp(); + } + + if ( + url.includes( + '/api/jsonws/ddm.ddmtemplate/get-templates?companyId=10157&groupId=20121&classNameId=1001&resourceClassNameId=1002&status=0', + ) + ) { + return new Response( + '[{"templateId":"40801","templateKey":"UB_TPL_ENLACES_MENU_SUPERIOR","nameCurrentValue":"Menu superior","classPK":301}]', + {status: 200}, + ); + } + + throw new Error(`Unexpected URL ${url}`); + }), + }); + + const result = await runLiferayInventoryPage( + CONFIG, + {url: '/web/guest/home'}, + {apiClient, tokenClient: TOKEN_CLIENT}, + ); + + expect(() => validateLiferayInventoryPageResultV2(result)).not.toThrow(); + if (result.pageType !== 'regularPage') { + throw new Error('Expected regular page'); + } + + expect(result.journalArticles).toMatchObject([ + { + articleId: 'ART-HEADER', + title: 'No editar - Menú superior', + ddmTemplateKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + discoverySource: 'renderedHtmlJournalContent', }, ]); - expect(result.contentStructures).toEqual([ + expect(matchPageAgainstResource(result, {type: 'template', keys: ['UB_TPL_ENLACES_MENU_SUPERIOR']})).toEqual([ { - contentStructureId: 301, - key: 'BASIC', - name: 'Basic Web Content', + resourceType: 'template', + matchedKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + matchKind: 'journalArticleTemplate', + label: 'Journal article template (static Journal Content rendered in HTML)', + detail: 'articleId=ART-HEADER title=No editar - Menú superior', + source: 'renderedHtmlJournalContent', }, ]); - expect(formatLiferayInventoryPage(result)).toContain('REGULAR PAGE'); - expect(formatLiferayInventoryPage(result)).toContain('contentField Headline=Hello'); }); test('skips local fragment export path enrichment outside a repo', async () => { @@ -518,10 +810,6 @@ describe('liferay inventory page', () => { ); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -654,10 +942,6 @@ describe('liferay inventory page', () => { return pageDefinitionResp(); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -716,10 +1000,6 @@ describe('liferay inventory page', () => { return pageDefinitionResp(); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -785,10 +1065,6 @@ describe('liferay inventory page', () => { return pageDefinitionResp(); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -846,10 +1122,6 @@ describe('liferay inventory page', () => { return pageDefinitionResp(); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - if (url.includes('/api/jsonws/classname/fetch-class-name?value=com.liferay.portal.kernel.model.Layout')) { return classNameIdResp(); } @@ -916,6 +1188,19 @@ describe('liferay inventory page', () => { }, }, }, + { + id: 'mapped-title', + value: { + html: { + mapping: { + fieldKey: 'ddmTemplate_NEWS_TEMPLATE_DETAIL', + itemReference: { + contextSource: 'DisplayPageItem', + }, + }, + }, + }, + }, ], }, }, @@ -927,10 +1212,6 @@ describe('liferay inventory page', () => { ); } - if (url.includes('/fragment.fragmententrylink/get-fragment-entry-links')) { - return fragmentEntryLinksResp(); - } - throw new Error(`Unexpected URL ${url}`); }), }); @@ -954,6 +1235,7 @@ describe('liferay inventory page', () => { {id: 'image', value: 'Demo image'}, {id: 'intro-paragraph', value: 'Intro'}, ], + mappedTemplateKeys: ['NEWS_TEMPLATE_DETAIL'], }, ]); expect(formatLiferayInventoryPage(result)).toContain('[image] Demo image'); diff --git a/tests/unit/liferay-inventory-where-used.test.ts b/tests/unit/liferay-inventory-where-used.test.ts new file mode 100644 index 0000000..56ac5fb --- /dev/null +++ b/tests/unit/liferay-inventory-where-used.test.ts @@ -0,0 +1,1225 @@ +import {beforeEach, describe, expect, test, vi} from 'vitest'; + +import {CliError} from '../../src/core/errors.js'; +import type {LiferayInventoryPageResult} from '../../src/features/liferay/inventory/liferay-inventory-page.js'; +import { + collectDisplayPageCandidatesFromSources, + resetDisplayPageSourceSupportCache, + type DisplayPageSource, +} from '../../src/features/liferay/inventory/liferay-inventory-where-used-display-pages.js'; +import {buildPageMatch} from '../../src/features/liferay/inventory/liferay-inventory-where-used-pages.js'; +import {collectWhereUsedPageCandidates} from '../../src/features/liferay/inventory/liferay-inventory-where-used-page-candidates.js'; +import { + buildWhereUsedAdtKeys, + collectWhereUsedFragmentKeys, + collectWhereUsedTemplateKeys, + collectWhereUsedAdtKeys, + formatLiferayInventoryWhereUsed, + isSkippableWhereUsedCandidateError, + matchPageAgainstResource, + selectWhereUsedSites, + validateWhereUsedPlanResult, + validateWhereUsedResult, + validateWhereUsedQuery, + validateWhereUsedScopeOptions, + type WhereUsedResult, +} from '../../src/features/liferay/inventory/liferay-inventory-where-used.js'; +import {buildPortalAbsoluteUrl} from '../../src/features/liferay/inventory/liferay-inventory-url.js'; + +const REGULAR_PAGE_BASE: Extract = { + pageType: 'regularPage', + pageSubtype: 'content', + pageUiType: 'Content Page', + siteName: 'Guest', + siteFriendlyUrl: '/guest', + groupId: 20121, + url: '/web/guest/home', + friendlyUrl: '/home', + pageName: 'Home', + privateLayout: false, + layout: {layoutId: 11, plid: 1011, friendlyUrl: '/home', type: 'content', hidden: false}, + layoutDetails: {}, + adminUrls: { + view: '', + edit: '', + configureGeneral: '', + configureDesign: '', + configureSeo: '', + configureOpenGraph: '', + configureCustomMetaTags: '', + translate: '', + }, +}; + +describe('validateWhereUsedQuery', () => { + test('rejects unknown resource type', () => { + expect(() => validateWhereUsedQuery({type: 'unknown' as never, keys: ['x']})).toThrow(/--type/); + }); + + test('rejects empty keys', () => { + expect(() => validateWhereUsedQuery({type: 'fragment', keys: []})).toThrow(/--key/); + expect(() => validateWhereUsedQuery({type: 'fragment', keys: [' ']})).toThrow(/--key/); + }); + + test('deduplicates and trims keys', () => { + expect(validateWhereUsedQuery({type: 'fragment', keys: [' card ', 'card', 'hero']})).toEqual({ + type: 'fragment', + keys: ['card', 'hero'], + }); + }); +}); + +describe('validateWhereUsedScopeOptions', () => { + test('rejects invalid site-order', () => { + expect(() => validateWhereUsedScopeOptions({siteOrder: 'slowest' as never})).toThrow(/--site-order/); + }); + + test('rejects invalid site-limit', () => { + expect(() => validateWhereUsedScopeOptions({siteLimit: 0})).toThrow(/--site-limit/); + }); + + test('normalizes and deduplicates excluded sites', () => { + expect( + validateWhereUsedScopeOptions({excludeSites: ['global', '/global', ' /ub '], siteOrder: 'content', plan: true}), + ).toEqual({ + siteOrder: 'content', + excludedSites: ['/global', '/ub'], + plan: true, + }); + }); +}); + +describe('selectWhereUsedSites', () => { + const sites = [ + {groupId: 3, siteFriendlyUrl: '/global', name: 'Global', pagesCommand: ''}, + {groupId: 2, siteFriendlyUrl: '/ub', name: 'Universitat de Barcelona', pagesCommand: ''}, + {groupId: 4, siteFriendlyUrl: '/labweb', name: 'LabWeb', pagesCommand: ''}, + ]; + + test('orders by structured content volume and applies site-limit', () => { + const selection = selectWhereUsedSites({ + sites, + siteOrder: 'content', + siteLimit: 2, + excludedSites: [], + contentStatsSites: [ + { + groupId: 4, + siteFriendlyUrl: '/labweb', + name: 'LabWeb', + rootFolderCount: 1, + folderCount: 10, + structuredContents: 900, + topFolders: [], + }, + { + groupId: 2, + siteFriendlyUrl: '/ub', + name: 'Universitat de Barcelona', + rootFolderCount: 1, + folderCount: 10, + structuredContents: 700, + topFolders: [], + }, + ], + }); + + expect(selection.selectedSites.map((site) => site.siteFriendlyUrl)).toEqual(['/labweb', '/ub']); + expect(selection.planSites).toEqual([ + expect.objectContaining({ + rank: 1, + siteFriendlyUrl: '/labweb', + structuredContents: 900, + selectionReason: 'contentOrder', + }), + expect.objectContaining({ + rank: 2, + siteFriendlyUrl: '/ub', + structuredContents: 700, + selectionReason: 'contentOrder', + }), + ]); + }); + + test('filters excluded sites before ordering', () => { + const selection = selectWhereUsedSites({ + sites, + siteOrder: 'site', + excludedSites: ['/global'], + }); + + expect(selection.selectedSites.map((site) => site.siteFriendlyUrl)).toEqual(['/labweb', '/ub']); + expect(selection.excludedCount).toBe(1); + }); + + test('uses explicit site even if not present in the accessible site list', () => { + const selection = selectWhereUsedSites({ + sites, + explicitSites: ['/missing'], + siteOrder: 'content', + siteLimit: 1, + excludedSites: ['/global'], + }); + + expect(selection.selectedSites).toEqual([ + expect.objectContaining({siteFriendlyUrl: '/missing', groupId: -1, name: '/missing'}), + ]); + expect(selection.planSites[0]).toMatchObject({selectionReason: 'explicitSite', rank: 1}); + }); + + test('keeps multiple explicit sites in the requested order', () => { + const selection = selectWhereUsedSites({ + sites, + explicitSites: ['/ub', '/missing', '/labweb'], + siteOrder: 'content', + siteLimit: 1, + excludedSites: ['/global'], + }); + + expect(selection.selectedSites).toEqual([ + expect.objectContaining({siteFriendlyUrl: '/ub', groupId: 2}), + expect.objectContaining({siteFriendlyUrl: '/missing', groupId: -1, name: '/missing'}), + expect.objectContaining({siteFriendlyUrl: '/labweb', groupId: 4}), + ]); + expect(selection.planSites).toEqual([ + expect.objectContaining({rank: 1, siteFriendlyUrl: '/ub', selectionReason: 'explicitSite'}), + expect.objectContaining({rank: 2, siteFriendlyUrl: '/missing', selectionReason: 'explicitSite'}), + expect.objectContaining({rank: 3, siteFriendlyUrl: '/labweb', selectionReason: 'explicitSite'}), + ]); + }); +}); + +describe('buildWhereUsedAdtKeys', () => { + test('includes both templateId-based and templateKey-based displayStyle values', () => { + expect(buildWhereUsedAdtKeys({displayStyle: 'ddmTemplate_2919390', templateKey: '19690812'})).toEqual([ + 'ddmTemplate_2919390', + 'ddmTemplate_19690812', + ]); + }); + + test('deduplicates when templateKey already matches the displayStyle suffix', () => { + expect(buildWhereUsedAdtKeys({displayStyle: 'ddmTemplate_19690812', templateKey: '19690812'})).toEqual([ + 'ddmTemplate_19690812', + ]); + }); +}); + +describe('collectWhereUsedAdtKeys', () => { + test('includes keys from every matching ADT row instead of stopping at the first visible-name match', () => { + const keys = collectWhereUsedAdtKeys( + [ + { + templateId: 2919390, + templateKey: 'UB_ADT_CUSTOM_FILTER_DATERANGEPICKER', + displayName: 'UB_ADT_CUSTOM_FILTER_DATERANGEPICKER_perEsborrar', + adtName: 'UB_ADT_CUSTOM_FILTER_DATERANGEPICKER_perEsborrar', + }, + { + templateId: 19690813, + templateKey: '19690812', + displayName: 'UB_ADT_CUSTOM_FILTER_DATERANGEPICKER', + adtName: 'UB_ADT_CUSTOM_FILTER_DATERANGEPICKER', + }, + ], + 'UB_ADT_CUSTOM_FILTER_DATERANGEPICKER', + ); + + expect(keys).toEqual( + expect.arrayContaining([ + 'ddmTemplate_2919390', + 'ddmTemplate_UB_ADT_CUSTOM_FILTER_DATERANGEPICKER', + 'ddmTemplate_19690813', + 'ddmTemplate_19690812', + ]), + ); + expect(keys).toHaveLength(4); + }); +}); + +describe('collectWhereUsedFragmentKeys', () => { + test('resolves a fragment name to its fragment key', () => { + expect( + collectWhereUsedFragmentKeys( + [ + { + fragmentKey: 'ub-frg-navigation', + fragmentName: 'UB_FRG_1rN_INDEX', + }, + ], + 'UB_FRG_1rN_INDEX', + ), + ).toEqual(['ub-frg-navigation']); + }); + + test('keeps the original identifier when no fragment name or key matches', () => { + expect( + collectWhereUsedFragmentKeys( + [ + { + fragmentKey: 'ub-frg-navigation', + fragmentName: 'UB_FRG_1rN_INDEX', + }, + ], + 'missing-fragment', + ), + ).toEqual(['missing-fragment']); + }); +}); + +describe('collectWhereUsedTemplateKeys', () => { + test('resolves a template display name to the templateKey used in page evidence', () => { + expect( + collectWhereUsedTemplateKeys( + [ + { + templateId: '2919390', + templateKey: 'NEWS_CARD_TEMPLATE', + externalReferenceCode: 'news-card-template-erc', + nameCurrentValue: 'News Card Template', + name: 'News Card Template', + }, + ], + 'News Card Template', + ), + ).toEqual(['NEWS_CARD_TEMPLATE']); + }); + + test('resolves a template externalReferenceCode to the templateKey used in page evidence', () => { + expect( + collectWhereUsedTemplateKeys( + [ + { + templateId: '2919390', + templateKey: 'NEWS_CARD_TEMPLATE', + externalReferenceCode: 'news-card-template-erc', + nameCurrentValue: 'News Card Template', + name: 'News Card Template', + }, + ], + 'news-card-template-erc', + ), + ).toEqual(['NEWS_CARD_TEMPLATE']); + }); + + test('keeps the original identifier when no template alias matches', () => { + expect( + collectWhereUsedTemplateKeys( + [ + { + templateId: '2919390', + templateKey: 'NEWS_CARD_TEMPLATE', + externalReferenceCode: 'news-card-template-erc', + nameCurrentValue: 'News Card Template', + name: 'News Card Template', + }, + ], + 'missing-template', + ), + ).toEqual(['missing-template']); + }); +}); + +describe('matchPageAgainstResource - fragments', () => { + test('matches fragment by fragmentKey on regular page', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [ + {type: 'fragment', fragmentKey: 'banner', elementName: 'main-banner'}, + {type: 'fragment', fragmentKey: 'card-hero'}, + {type: 'widget', widgetName: 'com_liferay_journal_content_web_portlet_JournalContentPortlet'}, + ], + }; + + const matches = matchPageAgainstResource(page, {type: 'fragment', keys: ['card-hero']}); + expect(matches).toHaveLength(1); + expect(matches[0]).toMatchObject({ + resourceType: 'fragment', + matchedKey: 'card-hero', + matchKind: 'fragmentEntry', + }); + expect(matches[0].detail).toContain('index=1'); + }); + + test('matches fragment keys case-insensitively for exported fragment slugs', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [{type: 'fragment', fragmentKey: 'ub_frg_text_enriquit'}], + }; + + expect(matchPageAgainstResource(page, {type: 'fragment', keys: ['UB_FRG_TEXT_ENRIQUIT']})).toEqual([ + { + resourceType: 'fragment', + matchedKey: 'ub_frg_text_enriquit', + matchKind: 'fragmentEntry', + label: 'Fragment on page', + detail: 'fragmentKey=ub_frg_text_enriquit index=0', + source: 'fragmentEntryLink', + }, + ]); + }); + + test('returns empty when fragment is not present', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [{type: 'fragment', fragmentKey: 'banner'}], + }; + expect(matchPageAgainstResource(page, {type: 'fragment', keys: ['missing']})).toHaveLength(0); + }); + + test('OR-matches across multiple keys in a single pass', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [ + {type: 'fragment', fragmentKey: 'banner'}, + {type: 'fragment', fragmentKey: 'card-hero'}, + ], + }; + expect(matchPageAgainstResource(page, {type: 'fragment', keys: ['banner', 'card-hero']})).toHaveLength(2); + }); +}); + +describe('matchPageAgainstResource - widgets and portlets', () => { + test('matches widget by widgetName or portletId in fragmentEntryLinks', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [ + { + type: 'widget', + widgetName: 'com_liferay_journal_content_web_portlet_JournalContentPortlet', + portletId: 'com_liferay_journal_content_web_portlet_JournalContentPortlet_INSTANCE_abc', + }, + ], + }; + + expect( + matchPageAgainstResource(page, { + type: 'widget', + keys: ['com_liferay_journal_content_web_portlet_JournalContentPortlet'], + }), + ).toHaveLength(1); + + expect( + matchPageAgainstResource(page, { + type: 'portlet', + keys: ['com_liferay_journal_content_web_portlet_JournalContentPortlet_INSTANCE_abc'], + }), + ).toHaveLength(1); + }); + + test('matches portlets table on widget pages', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + pageSubtype: 'portlet', + pageUiType: 'Widget Page', + portlets: [ + { + columnId: 'column-1', + position: 0, + portletId: 'com_liferay_journal_content_web_portlet_JournalContentPortlet', + portletName: 'Journal Content', + }, + ], + }; + + const matches = matchPageAgainstResource(page, { + type: 'widget', + keys: ['com_liferay_journal_content_web_portlet_JournalContentPortlet'], + }); + expect(matches).toHaveLength(1); + expect(matches[0].matchKind).toBe('portlet'); + }); +}); + +describe('matchPageAgainstResource - structures and templates', () => { + test('matches normalized page evidence without reading page inspection details', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + evidence: [ + { + resourceType: 'template', + key: 'UB_TPL_NOVEDAD_NOTA_PRENSA_DETALLE', + kind: 'journalArticleTemplate', + detail: 'articleId=ART-1 title=Article', + source: 'journalArticle', + }, + ], + }; + + expect(matchPageAgainstResource(page, {type: 'template', keys: ['UB_TPL_NOVEDAD_NOTA_PRENSA_DETALLE']})).toEqual([ + { + resourceType: 'template', + matchedKey: 'UB_TPL_NOVEDAD_NOTA_PRENSA_DETALLE', + matchKind: 'journalArticleTemplate', + label: 'Journal article template', + detail: 'articleId=ART-1 title=Article', + source: 'journalArticle', + }, + ]); + }); + + test('labels rendered HTML journal content template matches explicitly', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + evidence: [ + { + resourceType: 'template', + key: 'UB_TPL_ENLACES_MENU_SUPERIOR', + kind: 'journalArticleTemplate', + detail: 'articleId=ART-1 title=Article', + source: 'renderedHtmlJournalContent', + }, + ], + }; + + expect(matchPageAgainstResource(page, {type: 'template', keys: ['UB_TPL_ENLACES_MENU_SUPERIOR']})).toEqual([ + { + resourceType: 'template', + matchedKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + matchKind: 'journalArticleTemplate', + label: 'Journal article template (static Journal Content rendered in HTML)', + detail: 'articleId=ART-1 title=Article', + source: 'renderedHtmlJournalContent', + }, + ]); + }); + + test('formats where-used plans and validates the dedicated plan contract', () => { + const result = validateWhereUsedPlanResult({ + inventoryType: 'whereUsedPlan', + query: {type: 'template', keys: ['UB_TPL_DESTACATS_MULTIMEDIA']}, + scope: { + sites: ['/labweb', '/ub'], + includePrivate: false, + concurrency: 4, + maxDepth: 12, + siteOrder: 'content', + siteLimit: 2, + excludedSites: ['/global'], + plan: true, + }, + summary: { + totalSites: 3, + selectedSites: 2, + excludedSites: 1, + skippedSites: 0, + }, + sites: [ + { + rank: 1, + siteFriendlyUrl: '/labweb', + siteName: 'LabWeb', + groupId: 4, + structuredContents: 900, + selectionReason: 'contentOrder', + }, + ], + }); + + expect(formatLiferayInventoryWhereUsed(result)).toContain('WHERE USED PLAN'); + expect(formatLiferayInventoryWhereUsed(result)).toContain('siteOrder=content'); + expect(formatLiferayInventoryWhereUsed(result)).toContain('structuredContents=900'); + }); + + test('includes skipped ranking sites in real where-used results and formatter output', () => { + const result = validateWhereUsedResult({ + inventoryType: 'whereUsed', + query: {type: 'template', keys: ['UB_TPL_DESTACATS_MULTIMEDIA']}, + scope: { + sites: ['/labweb', '/ub'], + includePrivate: false, + concurrency: 4, + maxDepth: 12, + siteOrder: 'content', + siteLimit: 2, + excludedSites: ['/global'], + plan: false, + }, + summary: { + totalSites: 2, + totalScannedPages: 30, + totalMatchedPages: 1, + totalMatches: 1, + totalFailedPages: 0, + }, + sites: [ + { + siteFriendlyUrl: '/labweb', + siteName: 'LabWeb', + groupId: 4, + scannedPages: 30, + failedPages: 0, + matchedPages: [ + { + pageType: 'displayPage', + pageName: 'Video', + friendlyUrl: '/w/video', + fullUrl: '/web/labweb/w/video', + privateLayout: false, + matches: [ + { + resourceType: 'template', + matchedKey: 'UB_TPL_DESTACATS_MULTIMEDIA', + matchKind: 'journalArticleTemplate', + label: 'Journal article template', + detail: 'articleId=123 title=Video', + source: 'journalArticle', + }, + ], + }, + ], + }, + ], + skippedSites: [ + { + siteFriendlyUrl: '/departaments', + groupId: 22, + reason: 'content stats failed', + }, + ], + }); + + const formatted = formatLiferayInventoryWhereUsed(result); + expect(result.skippedSites).toEqual([ + expect.objectContaining({siteFriendlyUrl: '/departaments', groupId: 22, reason: 'content stats failed'}), + ]); + expect(formatted).toContain('skippedSites=1'); + expect(formatted).toContain('Skipped ranking sites: 1'); + expect(formatted).toContain('site=/departaments groupId=22 reason=content stats failed'); + }); + + test('matches structure via journal article ddmStructureKey', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + journalArticles: [ + { + articleId: 'ART-1', + title: 'Home', + ddmStructureKey: 'BASIC', + ddmTemplateKey: 'DEFAULT', + contentStructureId: 301, + }, + ], + contentStructures: [{contentStructureId: 301, key: 'BASIC', name: 'Basic'}], + }; + + const matches = matchPageAgainstResource(page, {type: 'structure', keys: ['BASIC']}); + + expect(matches).toHaveLength(1); + expect(matches[0]).toMatchObject({ + matchKind: 'journalArticleStructure', + detail: 'articleId=ART-1 title=Home contentStructureId=301 contentStructureName=Basic', + }); + }); + + test('suppresses redundant contentStructure matches when querying by contentStructureId', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + journalArticles: [ + { + articleId: 'ART-1', + title: 'Home', + ddmStructureKey: 'BASIC', + contentStructureId: 301, + }, + ], + contentStructures: [{contentStructureId: 301, key: 'BASIC', name: 'Basic'}], + }; + + const matches = matchPageAgainstResource(page, {type: 'structure', keys: ['301']}); + + expect(matches).toEqual([ + { + resourceType: 'structure', + matchedKey: '301', + matchKind: 'contentStructure', + label: 'Content structure', + detail: 'contentStructureId=301 name=Basic', + source: 'contentStructure', + }, + ]); + }); + + test('suppresses redundant contentStructure matches when query includes key and contentStructureId', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + journalArticles: [ + { + articleId: 'ART-1', + title: 'Home', + ddmStructureKey: 'BASIC', + contentStructureId: 301, + }, + ], + contentStructures: [{contentStructureId: 301, key: 'BASIC', name: 'Basic'}], + }; + + const matches = matchPageAgainstResource(page, {type: 'structure', keys: ['BASIC', '301']}); + + expect(matches).toEqual([ + { + resourceType: 'structure', + matchedKey: 'BASIC', + matchKind: 'journalArticleStructure', + label: 'Journal article structure', + detail: 'articleId=ART-1 title=Home contentStructureId=301 contentStructureName=Basic', + source: 'journalArticle', + }, + ]); + }); + + test('matches template via ddmTemplateKey and widgetDefaultTemplate', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + journalArticles: [ + { + articleId: 'ART-1', + title: 'Home', + ddmStructureKey: 'BASIC', + ddmTemplateKey: 'CARD', + widgetDefaultTemplate: 'WIDGET-CARD', + }, + ], + }; + + expect(matchPageAgainstResource(page, {type: 'template', keys: ['CARD']})).toHaveLength(1); + expect(matchPageAgainstResource(page, {type: 'template', keys: ['WIDGET-CARD']})).toHaveLength(1); + expect(matchPageAgainstResource(page, {type: 'template', keys: ['nope']})).toHaveLength(0); + }); + + test('matches template via display page DDM template references', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + journalArticles: [ + { + articleId: 'ART-1', + title: 'Home', + ddmStructureKey: 'BASIC', + displayPageDdmTemplates: ['DETAIL-TEMPLATE'], + }, + ], + }; + + expect(matchPageAgainstResource(page, {type: 'template', keys: ['DETAIL-TEMPLATE']})).toHaveLength(1); + }); + + test('matches template via fragment mapped template keys', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [ + { + type: 'fragment', + fragmentKey: 'ub_frg_title', + mappedTemplateKeys: ['UB_TPL_NOVEDAD_NOTA_PRENSA_DETALLE'], + }, + ], + }; + + const matches = matchPageAgainstResource(page, { + type: 'template', + keys: ['UB_TPL_NOVEDAD_NOTA_PRENSA_DETALLE'], + }); + expect(matches).toHaveLength(1); + expect(matches[0].matchKind).toBe('fragmentMappedTemplate'); + }); + + test('matches adt via widget displayStyle configuration', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + fragmentEntryLinks: [ + { + type: 'widget', + widgetName: 'com_liferay_asset_publisher_web_portlet_AssetPublisherPortlet', + portletId: 'com_liferay_asset_publisher_web_portlet_AssetPublisherPortlet_INSTANCE_abcd', + configuration: {displayStyle: 'ddmTemplate_40801'}, + }, + ], + }; + + expect(matchPageAgainstResource(page, {type: 'adt', keys: ['ddmTemplate_40801']})).toEqual([ + { + resourceType: 'adt', + matchedKey: 'ddmTemplate_40801', + matchKind: 'widgetAdt', + label: 'Widget ADT', + detail: + 'widgetName=com_liferay_asset_publisher_web_portlet_AssetPublisherPortlet portletId=com_liferay_asset_publisher_web_portlet_AssetPublisherPortlet_INSTANCE_abcd index=0 displayStyle=ddmTemplate_40801', + source: 'fragmentEntryLink', + }, + ]); + }); + + test('ignores widget template candidates in where-used template matches', () => { + const page: LiferayInventoryPageResult = { + ...REGULAR_PAGE_BASE, + journalArticles: [ + { + articleId: '33112379', + title: 'Quan els gats esdevenen una amenaça per a la biodiversitat', + ddmStructureKey: 'UB_STR_OPINION_EXPERTO', + ddmTemplateKey: 'UB_TPL_OPINION_EXPERTO_ITEM', + widgetDefaultTemplate: 'UB_TPL_OPINION_EXPERTO_ITEM', + widgetTemplateCandidates: ['UB_TPL_OPINION_EXPERTO_ITEM'], + }, + ], + }; + + expect(matchPageAgainstResource(page, {type: 'template', keys: ['UB_TPL_OPINION_EXPERTO_ITEM']})).toEqual([ + { + resourceType: 'template', + matchedKey: 'UB_TPL_OPINION_EXPERTO_ITEM', + matchKind: 'journalArticleTemplate', + label: 'Journal article template', + detail: 'articleId=33112379 title=Quan els gats esdevenen una amenaça per a la biodiversitat', + source: 'journalArticle', + }, + ]); + }); + + test('matches structure on a display page via article.contentStructureId', () => { + const page: LiferayInventoryPageResult = { + pageType: 'displayPage', + pageSubtype: 'journalArticle', + contentItemType: 'WebContent', + siteName: 'Guest', + siteFriendlyUrl: '/guest', + groupId: 20121, + url: '/web/guest/w/article', + friendlyUrl: '/article', + article: {id: 99, key: 'ART-1', title: 'Article', friendlyUrlPath: '/article', contentStructureId: 301}, + journalArticles: [{articleId: 'ART-1', title: 'Article', ddmStructureKey: 'BASIC'}], + }; + + expect(matchPageAgainstResource(page, {type: 'structure', keys: ['BASIC']})).toHaveLength(1); + expect(matchPageAgainstResource(page, {type: 'structure', keys: ['301']})).toHaveLength(1); + }); +}); + +describe('matchPageAgainstResource - siteRoot pages', () => { + test('returns empty for siteRoot pages', () => { + const page: LiferayInventoryPageResult = { + pageType: 'siteRoot', + siteName: 'Guest', + siteFriendlyUrl: '/guest', + groupId: 20121, + url: '/web/guest', + pages: [], + }; + expect(matchPageAgainstResource(page, {type: 'fragment', keys: ['banner']})).toHaveLength(0); + }); +}); + +describe('buildPageMatch', () => { + test('omits display page viewUrl when there is no evidence of display-page rendering', () => { + const page: LiferayInventoryPageResult = { + pageType: 'displayPage', + pageSubtype: 'journalArticle', + contentItemType: 'WebContent', + siteName: 'UB', + siteFriendlyUrl: '/ub', + groupId: 2685349, + url: '/web/ub/w/xarxes-internacionals', + friendlyUrl: '/w/xarxes-internacionals', + article: { + id: 7109595, + key: '7109595', + title: 'Xarxes internacionals', + friendlyUrlPath: 'xarxes-internacionals', + contentStructureId: 2810759, + }, + adminUrls: { + edit: 'http://localhost:8080/group/ub/edit-article', + translate: 'http://localhost:8080/group/ub/translate-article', + }, + journalArticles: [ + { + articleId: '7109595', + title: 'Xarxes internacionals', + ddmStructureKey: 'UB_STR_LISTA_ENLACES', + }, + ], + contentStructures: [{contentStructureId: 2810759, name: 'UB_STR_LISTA_ENLACES'}], + }; + + const match = buildPageMatch( + page, + { + fullUrl: '/web/ub/w/xarxes-internacionals', + friendlyUrl: '/web/ub/w/xarxes-internacionals', + name: '/web/ub/w/xarxes-internacionals', + layoutId: -1, + plid: -1, + hidden: false, + privateLayout: false, + }, + [ + { + resourceType: 'structure', + matchedKey: 'UB_STR_LISTA_ENLACES', + matchKind: 'journalArticleStructure', + label: 'Journal article structure', + detail: 'articleId=7109595 title=Xarxes internacionals', + source: 'journalArticle', + }, + ], + 'http://localhost:8080', + ); + + expect(match.pageType).toBe('displayPage'); + expect(match).not.toHaveProperty('viewUrl'); + expect(match.fullUrl).toBe('/web/ub/w/xarxes-internacionals'); + expect(match.editUrl).toBe('http://localhost:8080/group/ub/edit-article'); + }); +}); + +describe('formatLiferayInventoryWhereUsed', () => { + test('reports zero matches with a friendly message', () => { + const result: WhereUsedResult = { + inventoryType: 'whereUsed', + query: {type: 'fragment', keys: ['banner']}, + scope: { + sites: ['/guest'], + includePrivate: false, + concurrency: 4, + maxDepth: 12, + siteOrder: 'site', + excludedSites: [], + plan: false, + }, + summary: { + totalSites: 1, + totalScannedPages: 5, + totalMatchedPages: 0, + totalMatches: 0, + totalFailedPages: 0, + }, + sites: [ + { + siteFriendlyUrl: '/guest', + siteName: 'Guest', + groupId: 20121, + scannedPages: 5, + failedPages: 0, + matchedPages: [], + }, + ], + }; + + const text = formatLiferayInventoryWhereUsed(result); + expect(text).toContain('WHERE USED'); + expect(text).toContain('resourceType=fragment'); + expect(text).toContain('No pages matched'); + }); + + test('lists matched pages with match details', () => { + const result: WhereUsedResult = { + inventoryType: 'whereUsed', + query: {type: 'fragment', keys: ['banner']}, + scope: { + sites: ['/guest'], + includePrivate: false, + concurrency: 4, + maxDepth: 12, + siteOrder: 'site', + excludedSites: [], + plan: false, + }, + summary: { + totalSites: 1, + totalScannedPages: 1, + totalMatchedPages: 1, + totalMatches: 1, + totalFailedPages: 0, + }, + sites: [ + { + siteFriendlyUrl: '/guest', + siteName: 'Guest', + groupId: 20121, + scannedPages: 1, + failedPages: 0, + matchedPages: [ + { + pageType: 'regularPage', + pageName: 'Home', + friendlyUrl: '/home', + fullUrl: '/web/guest/home', + viewUrl: 'http://localhost:8080/web/guest/home', + layoutId: 11, + plid: 1011, + hidden: false, + privateLayout: false, + editUrl: 'http://localhost:8080/web/guest/home?p_l_mode=edit', + matches: [ + { + resourceType: 'fragment', + matchedKey: 'banner', + matchKind: 'fragmentEntry', + label: 'Fragment on page', + detail: 'fragmentKey=banner index=0', + source: 'fragmentEntryLink', + }, + ], + }, + ], + }, + ], + }; + + const text = formatLiferayInventoryWhereUsed(result); + expect(text).toContain('site=/guest'); + expect(text).toContain('Home'); + expect(text).toContain('Home http://localhost:8080/web/guest/home'); + expect(text).toContain('Fragment on page: fragmentKey=banner'); + expect(text).toContain('editUrl=http://localhost:8080/web/guest/home'); + }); + + test('marks rendered HTML journal content provenance in formatted output', () => { + const result: WhereUsedResult = { + inventoryType: 'whereUsed', + query: {type: 'template', keys: ['UB_TPL_ENLACES_MENU_SUPERIOR']}, + scope: { + sites: ['/ub'], + includePrivate: false, + concurrency: 4, + maxDepth: 12, + siteOrder: 'site', + excludedSites: [], + plan: false, + }, + summary: { + totalSites: 1, + totalScannedPages: 1, + totalMatchedPages: 1, + totalMatches: 1, + totalFailedPages: 0, + }, + sites: [ + { + siteFriendlyUrl: '/ub', + siteName: 'UB', + groupId: 2685349, + scannedPages: 1, + failedPages: 0, + matchedPages: [ + { + pageType: 'regularPage', + pageName: 'Inici', + friendlyUrl: '/inici', + fullUrl: '/web/ub/inici', + viewUrl: 'http://localhost:8080/web/ub/inici', + layoutId: 1, + plid: 76, + hidden: false, + privateLayout: false, + matches: [ + { + resourceType: 'template', + matchedKey: 'UB_TPL_ENLACES_MENU_SUPERIOR', + matchKind: 'journalArticleTemplate', + label: 'Journal article template (static Journal Content rendered in HTML)', + detail: 'articleId=2686429 title=No editar - Menú superior', + source: 'renderedHtmlJournalContent', + }, + ], + }, + ], + }, + ], + }; + + const text = formatLiferayInventoryWhereUsed(result); + expect(text).toContain( + 'Journal article template (static Journal Content rendered in HTML): articleId=2686429 title=No editar - Menú superior [source=static Journal Content rendered in HTML]', + ); + }); +}); + +describe('validateWhereUsedResult', () => { + test('coerces numeric portal groupId values returned as strings', () => { + const result = validateWhereUsedResult({ + inventoryType: 'whereUsed', + query: {type: 'template', keys: ['TPL']}, + scope: {sites: ['/actualitat'], includePrivate: false, concurrency: 4, maxDepth: 12}, + summary: { + totalSites: 1, + totalScannedPages: 0, + totalMatchedPages: 0, + totalMatches: 0, + totalFailedPages: 0, + }, + sites: [ + { + siteFriendlyUrl: '/actualitat', + siteName: 'Actualitat', + groupId: '2710030', + scannedPages: 0, + failedPages: 0, + matchedPages: [], + }, + ], + }); + + expect(result.sites[0].groupId).toBe(2710030); + }); + + test('accepts adt query and match kind in the result schema', () => { + const result = validateWhereUsedResult({ + inventoryType: 'whereUsed', + query: {type: 'adt', keys: ['ddmTemplate_40801']}, + scope: {sites: ['/global'], includePrivate: false, concurrency: 4, maxDepth: 12}, + summary: { + totalSites: 1, + totalScannedPages: 1, + totalMatchedPages: 1, + totalMatches: 1, + totalFailedPages: 0, + }, + sites: [ + { + siteFriendlyUrl: '/global', + siteName: 'Global', + groupId: 20121, + scannedPages: 1, + failedPages: 0, + matchedPages: [ + { + pageType: 'regularPage', + pageName: 'Search', + friendlyUrl: '/search', + fullUrl: '/web/global/search', + privateLayout: false, + matches: [ + { + resourceType: 'adt', + matchedKey: 'ddmTemplate_40801', + matchKind: 'widgetAdt', + label: 'Widget ADT', + detail: 'widgetName=asset-publisher index=0 displayStyle=ddmTemplate_40801', + source: 'fragmentEntryLink', + }, + ], + }, + ], + }, + ], + }); + + expect(result.query.type).toBe('adt'); + expect(result.sites[0].matchedPages[0].matches[0].matchKind).toBe('widgetAdt'); + }); +}); + +describe('where-used display page sources', () => { + beforeEach(() => { + resetDisplayPageSourceSupportCache(); + }); + + test('continues with later sources when one source has a skippable portal error', async () => { + const sources: DisplayPageSource[] = [ + { + origin: 'headlessStructuredContent', + collect: () => + Promise.reject(new CliError('structured contents failed with status=404', {code: 'LIFERAY_GATEWAY_ERROR'})), + }, + { + origin: 'jsonwsJournal', + collect: () => + Promise.resolve([ + {fullUrl: '/web/actualitat/w/la-universitat-del-futur', origin: 'jsonwsJournal'}, + {fullUrl: '/web/actualitat/w/la-universitat-del-futur', origin: 'jsonwsJournal'}, + ]), + }, + ]; + + const candidates = await collectDisplayPageCandidatesFromSources( + {liferay: {url: 'http://localhost:8080'}} as never, + {groupId: 2710030, siteFriendlyUrl: '/actualitat', name: 'Actualitat', pagesCommand: ''}, + {concurrency: 4, pageSize: 200, dependencies: {}}, + sources, + ); + + expect(candidates).toEqual([{fullUrl: '/web/actualitat/w/la-universitat-del-futur', origin: 'jsonwsJournal'}]); + }); + + test('stops retrying a display page source after it returns a skippable portal error', async () => { + const collectHeadless = vi.fn(() => + Promise.reject(new CliError('structured contents failed with status=404', {code: 'LIFERAY_GATEWAY_ERROR'})), + ); + const collectJsonws = vi.fn(() => + Promise.resolve([{fullUrl: '/web/actualitat/w/article', origin: 'jsonwsJournal'}]), + ); + + const sources: DisplayPageSource[] = [ + {origin: 'headlessStructuredContent', collect: collectHeadless}, + {origin: 'jsonwsJournal', collect: collectJsonws}, + ]; + + const config = {liferay: {url: 'http://localhost:8080'}} as never; + const site = {groupId: 2710030, siteFriendlyUrl: '/actualitat', name: 'Actualitat', pagesCommand: ''}; + const options = {concurrency: 4, pageSize: 200, dependencies: {}}; + + await collectDisplayPageCandidatesFromSources(config, site, options, sources); + await collectDisplayPageCandidatesFromSources(config, site, options, sources); + + expect(collectHeadless).toHaveBeenCalledTimes(1); + expect(collectJsonws).toHaveBeenCalledTimes(2); + }); + + test('does not hide headless permission errors as unsupported sources', async () => { + const sources: DisplayPageSource[] = [ + { + origin: 'headlessStructuredContent', + collect: () => + Promise.reject(new CliError('structured contents failed with status=403', {code: 'LIFERAY_GATEWAY_ERROR'})), + }, + ]; + + await expect( + collectDisplayPageCandidatesFromSources( + {liferay: {url: 'http://localhost:8080'}} as never, + {groupId: 2710030, siteFriendlyUrl: '/actualitat', name: 'Actualitat', pagesCommand: ''}, + {concurrency: 4, pageSize: 200, dependencies: {}}, + sources, + ), + ).rejects.toThrow(/status=403/); + }); +}); + +describe('where-used page candidates', () => { + test('skips display page candidates for resource types that cannot match them', async () => { + const candidates = await collectWhereUsedPageCandidates( + {liferay: {url: 'http://localhost:8080'}} as never, + {groupId: 2710030, siteFriendlyUrl: '/actualitat', name: 'Actualitat', pagesCommand: ''}, + {type: 'fragment', keys: ['ub_frg_title']}, + {layoutScopes: [], concurrency: 4, maxDepth: 12, pageSize: 200, dependencies: {}}, + ); + + expect(candidates).toEqual([]); + }); + + test('skips synthetic jsonws display pages when no structured content can be resolved', () => { + expect( + isSkippableWhereUsedCandidateError( + {fullUrl: '/web/ub/w/article', origin: 'jsonwsJournal'}, + new Error( + 'No structured content found with friendlyUrlPath=article. Verify the article URL title and site visibility, or confirm JSONWS/headless permissions for this OAuth client.', + ), + ), + ).toBe(true); + + expect( + isSkippableWhereUsedCandidateError( + {fullUrl: '/web/ub/w/article', origin: 'headlessStructuredContent'}, + new Error('No structured content found with friendlyUrlPath=article.'), + ), + ).toBe(false); + }); +}); + +describe('buildPortalAbsoluteUrl', () => { + test('normalizes relative portal paths against configured base URL', () => { + expect(buildPortalAbsoluteUrl('http://localhost:8080', '/web/actualitat/w/article')).toBe( + 'http://localhost:8080/web/actualitat/w/article', + ); + }); +}); diff --git a/tests/unit/mcp-server-tools.test.ts b/tests/unit/mcp-server-tools.test.ts index dadd607..9fac9b3 100644 --- a/tests/unit/mcp-server-tools.test.ts +++ b/tests/unit/mcp-server-tools.test.ts @@ -17,6 +17,7 @@ describe('mcp server tools', () => { 'liferay_inventory_structures', 'liferay_inventory_pages', 'liferay_inventory_page', + 'liferay_inventory_where_used', 'liferay_doctor', 'liferay_inventory_templates', 'liferay_deploy_status', diff --git a/tests/unit/tool-liferay-inventory-where-used.test.ts b/tests/unit/tool-liferay-inventory-where-used.test.ts new file mode 100644 index 0000000..149bbb3 --- /dev/null +++ b/tests/unit/tool-liferay-inventory-where-used.test.ts @@ -0,0 +1,97 @@ +import {beforeEach, describe, expect, test, vi} from 'vitest'; + +import type {AppConfig} from '../../src/core/config/schema.js'; +import {runLiferayInventoryWhereUsed} from '../../src/features/liferay/inventory/liferay-inventory-where-used.js'; + +vi.mock('../../src/features/liferay/inventory/liferay-inventory-where-used.js', () => ({ + runLiferayInventoryWhereUsed: vi.fn(), +})); + +const {handleTool} = await import('../../src/features/mcp-server/tools/tool-liferay-inventory-where-used.js'); + +describe('liferay_inventory_where_used MCP tool', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + test('forwards where-used options and returns structured JSON content', async () => { + vi.mocked(runLiferayInventoryWhereUsed).mockResolvedValue({ + inventoryType: 'whereUsedPlan', + query: {type: 'template', keys: ['UB_TPL_DESTACATS_MULTIMEDIA']}, + scope: { + sites: ['/labweb'], + includePrivate: false, + concurrency: 4, + maxDepth: 12, + siteOrder: 'content', + siteLimit: 10, + excludedSites: ['/global'], + plan: true, + }, + summary: { + totalSites: 3, + selectedSites: 1, + excludedSites: 1, + skippedSites: 0, + }, + sites: [ + { + rank: 1, + siteFriendlyUrl: '/labweb', + siteName: 'LabWeb', + groupId: 4, + structuredContents: 900, + selectionReason: 'contentOrder', + }, + ], + } as never); + + const result = await handleTool( + { + type: 'template', + keys: ['UB_TPL_DESTACATS_MULTIMEDIA'], + sites: ['/labweb'], + excludeSites: ['/global'], + includePrivate: false, + siteLimit: 10, + siteOrder: 'content', + plan: true, + maxDepth: 12, + concurrency: 4, + pageSize: 200, + }, + {} as AppConfig, + ); + + expect(runLiferayInventoryWhereUsed).toHaveBeenCalledWith(expect.anything(), { + type: 'template', + keys: ['UB_TPL_DESTACATS_MULTIMEDIA'], + sites: ['/labweb'], + excludeSites: ['/global'], + includePrivate: false, + siteLimit: 10, + siteOrder: 'content', + plan: true, + maxDepth: 12, + concurrency: 4, + pageSize: 200, + }); + expect(result.structuredContent).toEqual(expect.objectContaining({inventoryType: 'whereUsedPlan'})); + expect(result.content[0]).toEqual(expect.objectContaining({type: 'text'})); + }); + + test('returns MCP error content when where-used fails', async () => { + vi.mocked(runLiferayInventoryWhereUsed).mockRejectedValue(new Error('portal unavailable')); + + const result = await handleTool( + { + type: 'fragment', + keys: ['banner'], + }, + {} as AppConfig, + ); + + expect(result.isError).toBe(true); + expect(result.content).toEqual([{type: 'text', text: 'portal unavailable'}]); + }); +});