Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ propagation in its PR.

## Durable work state

- OpenSpec change `clarify-operator-actions` is planning-complete and validated. Planning PRs 319 and 320 are merged.
- Beads parent `infra-8tt.93` remains in progress. Implementation child `infra-8tt.93.1` is open and owns the ten-task checklist in `openspec/changes/clarify-operator-actions/tasks.md`.
- No runtime implementation or release child exists yet. Create the release child only after offline implementation acceptance.
- OpenSpec change `clarify-operator-actions` is implemented and archived at `openspec/changes/archive/2026-10-27-clarify-operator-actions/`. All ten tasks have offline acceptance proof.
- Beads implementation `infra-8tt.93.1` is complete. Release child `infra-8tt.93.2` owns publication, exact-candidate lab qualification and stable promotion.
- Parent `infra-8tt.93` remains in progress until release acceptance. No publication or production GO was granted by implementation.

## Working mode: fast-track

Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,18 @@ Set the required credentials and target details in `.env`. The [full configurati

Secrets belong in the environment or a private `.env` file with restricted permissions. Do not place them in Compose committed to Git. Use `docker compose config --quiet` for validation; resolved output can expose secrets.

## Operational results

Required user actions send one consolidated notification per completed run,
including simulation, until current evidence resolves them. Clean success sends
no heartbeat. Private logs contain the affected scope and safe next action.
Notification delivery does not change financial processing or exit codes.

`Cycle result` separates success, success with user action required, and failure.
Per-account imports and cash are separate evidence. `request reported successful`
for cash does not claim independent balance readback or release activity holds.
Unavailable evidence is shown explicitly. See [configuration](docs/configuration.md).

## Corporate-action safety

The Flex query must select **Corporate Actions**, including when it is empty.
Expand Down
24 changes: 24 additions & 0 deletions docs/activity-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,3 +188,27 @@ Offline journal inspection can prepare a guarded reverse payload for an unchange
confirmed row only. Reversal requires separate authority and never runs automatically.
See [maintenance](cleanup.md#activity-review-journals) and
[release qualification](releasing.md#activity-review-qualification).


### User action required and container path

The private log names the configured ACTIVITY_REVIEW_FILE container path, section,
proposal key, held event and reason. A proposal count is not a held-event count.
Resolve the host path from your Compose volume mapping. For example,
`./review:/state` maps `/state/activity-review.yaml` to
`./review/activity-review.yaml`, relative to the Compose project directory.
The connector does not discover host mounts.

For an eligible `normal run` proposal, compare current and proposed values and
execution path. Edit only `approve` between runs. Only lowercase literal `yes`
approves that exact eligible proposal. Never approve proposals in bulk. Simulation
executes no financial write. Already approved work waiting for the qualified limit
remains automatic work; do not approve again or force reruns.

For missing proof, invalid/stale choices or conflicts, inspect the reason and
restore current source and destination proof. `yes` cannot make blocked work
executable. Historical maintenance needs the documented preparation and separate
operation authorization; normal sync cannot execute it. For an uncertain request,
inspect the preserved journal and exact destination readback. Do not resend,
delete or edit the journal to release a hold. False matches need the documented
manual resolution and retain separate cleanup authority.
121 changes: 102 additions & 19 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,10 +257,9 @@ Set `DRY_RUN=1` explicitly for the preview. The full pipeline runs (fetch, conve
Set a valid IANA `TZ`, such as `Europe/Paris`, even for one-off Docker runs.
Missing or invalid `TZ` forces every account into `DRY_RUN` before processing,
with one WARNING explaining the reason. No activities, cash or platform are
written. With explicit false `DRY_RUN`, the final notification includes the
reason when Apprise is configured, even if all accounts succeed. Successful
simulations selected by default, blank or truthy `DRY_RUN` send no notification,
including expiry-only reminders and previews. Failed simulations send one
written. The final notification includes the reason when Apprise is configured,
even in default, blank or truthy simulation. Required token renewal also notifies
in simulation. A deliberate clean simulation alone sends no notification. Failed simulations send one
configured Failure notification. After review, set `DRY_RUN=false` and retain
valid `TZ` to permit writes.

Expand Down Expand Up @@ -306,7 +305,8 @@ withholding; signed withholding and refund rows and their total remain separate
evidence. Coherence requires exact quantity, gross and fee agreement; a complete
representation with a discrepancy is not coherent.
The diagnostic adds no reads, writes or files. Findings do not change
import selection, cash updates, exit codes or notifications. Live runs do not
import selection, cash updates or exit codes. Required user checks join the
consolidated notification. Live runs do not
run this comparison. Use the one-off command below without `CRON`.

`absent` means no canonical destination comment was found. It does not establish
Expand All @@ -316,8 +316,8 @@ conflicting security/date context and missing/conflicting accruals stay uncertai
Missing destination profile ISIN is unavailable evidence and permits amount
comparison; a nonempty conflicting ISIN blocks identity. Details name the
active, activity-ID, data-source, symbol, local-date and ISIN checks with
expected and observed values. INFO counts failed checks and shows anomaly
blocks with securities, source and destination UTC/local dates, activity IDs,
expected and observed values. INFO counts failed checks. WARNING shows actionable
anomaly blocks with securities, source and destination UTC/local dates, activity IDs,
quantity, gross, fee and currency. An `absent` group suppressed by deduplication
shows the skip path that import selection takes: `skipped by historical comment`
when the account has an old `dividend#<symbol>#<date>` comment (active or not),
Expand All @@ -333,13 +333,13 @@ verify older history. No category authorizes a repair.

Use a one-off container with no `CRON` environment variable. The command reads
only the report generated by its configured Flex query; Docker does not extend
the broker's report window. INFO includes anomaly and uncertainty details, IDs
the broker's report window. WARNING includes actionable anomaly and uncertainty details, IDs
and source references; DEBUG also includes coherent comparisons. Keep Docker
logs private and capture them before removing a container if you need evidence.
The diagnostic creates no persistent report and log retention depends on Docker's
logging configuration. Financial state remains unchanged, but authentication and
IBKR report generation still contact the configured services. Technical read
failures still fail the run; diagnostic findings do not send notifications.
failures still fail the run; actionable diagnostic findings join one notification.

```bash
docker run --rm --network your-ghostfolio-network -e DRY_RUN=1 -e LOG_LEVEL=DEBUG \
Expand All @@ -351,7 +351,7 @@ docker run --rm --network your-ghostfolio-network -e DRY_RUN=1 -e LOG_LEVEL=DEBU
ghcr.io/flowcool/ghostfolio-ibkr-sync:latest
```

The dividend comparison ends with one line, `[DRY RUN] Dividend report: N to check, N informational, N noise`, followed by the detail of each item to check. Findings are written in plain English: the reason, the IBKR and Ghostfolio values side by side, one action line and the statement that nothing was written. Coherent and outside-window findings appear at `DEBUG` only. Differences of less than half a minor unit are noise; a difference of exactly one minor unit with equal quantity is named "minimal difference, cause unknown"; a legacy-comment row on the same security and day is named "probable duplicate". Each item to check that has a candidate carries a "Proposed match": the IBKR dividend and the candidate Ghostfolio rows, each labelled Keep, Duplicate or Differs, with a "strong" or "weak" confidence and the row to act on; with no candidate it reads "no confident match". A proposal never writes, repairs or changes the counts, exit code or notifications.
The dividend comparison ends with one line, `[DRY RUN] Dividend report: N to check, N informational, N noise`, followed by the detail of each item to check. Findings are written in plain English: the reason, the IBKR and Ghostfolio values side by side, one action line and the statement that nothing was written. Coherent and outside-window findings appear at `DEBUG` only. Differences of less than half a minor unit are noise; a difference of exactly one minor unit with equal quantity is named "minimal difference, cause unknown"; a legacy-comment row on the same security and day is named "probable duplicate". Each item to check that has a candidate carries a "Proposed match": the IBKR dividend and the candidate Ghostfolio rows, each labelled Keep, Duplicate or Differs, with a "strong" or "weak" confidence and the row to act on; with no candidate it reads "no confident match". A proposal never writes, repairs or changes the counts or exit code. Formatting does not add a second notification.

Look for lines starting with `[DRY RUN]`; when there is nothing to import the log simply says `No new activities to import`. Replace `your-ghostfolio-network` with the Docker network shared with Ghostfolio. Run it again after upgrading to a new version: comparing the log with the previous version's shows exactly what the new version changes.

Expand Down Expand Up @@ -382,12 +382,12 @@ No ENV change is needed. This is connector policy based on IBKR's
[Version 3 error table](https://www.ibkrguides.com/clientportal/performanceandstatements/flex3error.htm),
not a guarantee that IBKR recovers.

## Failure notifications (Apprise)
## Notifications (Apprise)

Optional. When `APPRISE_URLS` is set, a run that **completes with exit 1** sends one
notification through [Apprise](https://github.com/caronc/apprise/wiki), which supports
ntfy, Gotify, Telegram, Discord, Slack, email and many more. A completed successful run
in explicit live mode also sends a reminder when its legacy `GHOST_TOKEN` is about to expire (see
in any simulation or live mode also sends a reminder when its legacy `GHOST_TOKEN` is about to expire (see
[Token expiry](troubleshooting.md#token-expiry)); the reminder keeps exit 0. Give it a JSON list (up to 10
URLs, each up to 2048 characters; put one URL per entry, as an entry holding several
space- or comma-separated URLs counts each one toward the limit of 10):
Expand All @@ -411,7 +411,7 @@ of days remaining (`0` means expired). If the run also fails, the reminder is in
in the same failure notification. At most one notification is sent per completed run;
there is no daily state, so multiple runs in one day can repeat the reminder.

With explicit false `DRY_RUN`, missing or invalid IANA `TZ` also sends a final
Missing or invalid IANA `TZ` in any mode also sends a final
notification naming the forced-preview reason, even on success. It shares the single summary with any
failures or expiry reminder. Notification delivery does not change exit status.

Expand All @@ -420,12 +420,11 @@ including default, blank or truthy `DRY_RUN` and invalid-value rejection. Failur
with an expiry reminder or forced preview uses the same summary, never a second
dispatch. Delivery does not change exit status.

**When nothing is sent:** successful default/blank/truthy simulations, including
expiry-only reminders and previews; successful live runs without an expiry
reminder or forced preview; runs with only other warnings (for example unmapped
ISINs); runs stopped by a signal; and when `APPRISE_URLS` is blank or `[]`. If
`APPRISE_URLS` or `APPRISE_TIMEOUT` is invalid, the run logs
`APPRISE_URLS or APPRISE_TIMEOUT is invalid: failure notifications disabled` and syncs as usual.
**When nothing is sent:** clean successful runs without a required action or an
eligible corporate-action notice; deliberate simulation alone; interrupted runs;
and disabled or invalid Apprise settings. Mapping, review, diagnostic checks and
other explicit required actions repeat once per completed run until resolved.
See [User action required](#user-action-required) and [Notification result](#notification-result).

**Delivery:** Apprise runs in a separate short-lived worker process with only network
settings in its environment (`PATH`, locale, CA bundle and proxy variables; never the IBKR or
Expand Down Expand Up @@ -505,3 +504,87 @@ without a compatibility launcher, so update any script or note that calls it.
- **Active position context** - drafts and excluded activities/accounts retain their import identifiers for deduplication, but cannot contribute holdings, manual matches or dividend proximity evidence. Holdings and manual reconciliation use active YAHOO profiles; other sources cannot back YAHOO sells. Excluded target accounts are refused. Future timestamps are deferred with exit 1 until a later run; this conservative instant check avoids server-local draft assignment even when import responses omit tags. Accepted imports use the actual server profile symbol; a canonicalized symbol can require an explicit mapping before sells under the original ticker are allowed.
- **Batch guard limitations** - the gate checks net quantity, not chronological balances. Buy and sell phases are separate requests, so successful buys can remain if sells fail. A dry-run previews candidates without verifying server acceptance.
- **No overlap protection** - avoid concurrent sync runs targeting the same Ghostfolio account; deduplication is based on a snapshot read at startup.


### User action required

Required actions send one consolidated reminder on every completed run, including
simulation. Repeated observations of one event and reason count once. Reminders
continue while current evidence requires action. Clean success sends no heartbeat.
Review approval does not suppress a reminder when missing proof still blocks work.


### Notification result

Each completed run reports `not required`, `disabled`, `invalid configuration`,
`sent`, `failed`, or `timeout, delivery unconfirmed`. Configure APPRISE_URLS and
APPRISE_TIMEOUT when a required notification is disabled or invalid. `sent`
means the transport reported success; it does not prove device receipt. A timeout
can still deliver later. The connector makes one attempt and does not retry.
Delivery failure does not change the financial result or process exit code.


Mapping and trade reminders use `mapping_unverified`, `mapping_conflict`,
`trade_manual`, `trade_position` or `trade_invalid`. Private logs identify the
source scope. Verify the intended security in Ghostfolio or correct MAPPING_FILE.
For manual matches, compare source and destination before any reference repair.
Current valid source fields and verified destination ownership prove resolution.
Mapping checks still run for already imported trades in the acquired window.
Normal CASH/OPT exclusions, absent positions and recovered network retries do not
create required-action reminders by themselves.


Dividend actions use `dividend_rejected` for missing/conflicting cash, quantity,
payment association or withholding evidence. `dividend_reversal` identifies an
orphan reversal. `dividend_signed_tax` identifies signed zero-net tax, orphan tax
or a withholding refund. `dividend_check` identifies a discrepant proximity skip.
Inspect preserved Flex and Ghostfolio evidence. Never invent payment or quantity,
and never resend an uncertain transaction. Complete consistent current evidence,
verified destination agreement or documented manual resolution proves resolution.
Ordinary zero-net payments without signed tax remain exclusions. Future dates
remain deferred work. A representation limit alone does not request correction.


Simulation reports dividend items to check at WARNING. Concrete comparisons
that require intervention join the same `dividend_check` reminder. Inspect the
named activity and both sides of the comparison. Current agreement or documented
manual resolution proves resolution. Information, accepted rounding and noise
alone send no reminder. Named representation or coverage limits remain visible
as evidence limitations without requesting an impossible correction. No diagnostic
changes financial selection, grants cleanup authority or writes a correction.


Corporate-action holds and concrete historical discrepancies repeat in the same
summary as other actions. Private conclusions name scope and the configured state
path. Restore the stated evidence; acknowledgment and state reset do not release
holds. Nonblocking findings without a user action keep first/change/resumption
notifications. Automatic evidence maintenance remains a retry and is reported as
such; it does not request approval. Token renewal and invalid or missing TZ also
notify in simulation. Correct the named setting; simulation does not resolve it.


### Cycle result

Completed cycles report `success`, `success with user action required`, or
`failure`. Failure takes priority. Exit codes keep their existing meaning.
Accepted imports or simulated proposals, duplicate skips, unique review-held
events, other holds and deferred automatic work appear separately. Decision
counts can overlap. Unknown evidence reads `unavailable`, never an invented zero.
Cash is reported per account as `simulated`, `request reported successful`,
`not updated` or `uncertain/failed`. Request success is not independent balance
readback. Cash success does not mean held activities were imported.

For example, `success with user action required` can report zero accepted imports,
review-held events and `cash=request reported successful`. The notification result
appears beside the cycle result. Automatic deferral alone remains `success`.


### Application log levels and streams

DEBUG and INFO use stdout. WARNING and above use stderr. Each application record
has one timestamp and one level. The scheduler passes application output directly
and keeps its lifecycle messages separate. A transport stream is not a severity.
For a clean scheduled run, application INFO stays INFO without scheduler wrapping.
`LOG_LEVEL` still filters application records. LAB_EVIDENCE JSON remains at DEBUG.
Startup shows `ghostfolio-ibkr-sync build <APP_VERSION>`. This identifies the build;
a stable tag can reuse an RC digest and retain the qualified RC build identity.
Loading
Loading