Fix/composer issues

[jbpenrath]

Purpose

Remove a bug that prevent to use backspace in an empty block.
TypeCellOS/BlockNote#2610

Improve email exporter to support table elements. We do not add
blocknote tool to add explicitly table but we allow user to
copy/paste it and render it properly.

Summary by CodeRabbit

• New Features

  • Tiered blob storage with optional S3 offload, encryption, rotation, and scheduled offload/GC tasks; upload reservation flow and object-storage support.
  • Thread assignment/mention workflows with assign/unassign APIs and serializer/permission enhancements.
  • Full table → email HTML exporter and exported editor schema.

• Bug Fixes

  • Scoped composer styles to avoid affecting detached editor portals.
  • Upload/download size limits now return 413 when exceeded.

• Tests

  • Large expansion of unit/integration tests across storage, blob GC, thread events, exporter, composer, and RFC5322 composition.

• Documentation

  • Tiered storage and permissions architecture docs added/expanded.

• Chores

  • BlockNote deps upgraded to v0.49.0; Python/tooling/runtime bumps.

[coderabbitai]

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

Adds BlockNote table rendering, shared test factories, and scoped composer CSS; implements tiered blob storage with encryption, offload, GC, reservation, and reconciliation; introduces thread assignment/mention services, new ThreadEvent types, serializers, viewset behavior, and migrations; rewrites RFC5322 composer to stdlib email; plus many tests, docs, CI, and runtime bumps.

Changes

Frontend: BlockNote tables, tests, CSS, and minor wiring

Layer / File(s)	Summary
Dependency and schema export src/frontend/package.json, src/frontend/src/features/forms/components/message-composer/index.tsx	Bump @blocknote packages to 0.49.0; export BLOCKNOTE_SCHEMA from message-composer module.
CSS scoping fix src/frontend/src/features/blocknote/blocknote-view-field/_index.scss	Move .composer-field-input rules under .composer-field to avoid styling the BlockNote portal appended to document.body.
Shared block factories src/frontend/src/features/blocknote/__tests__/block-factories.ts	Add permissive inline/block factory helpers including tableCell() and table() for tests.
Email exporter: table rendering src/frontend/src/features/blocknote/email-exporter/index.tsx	Add case 'table' in renderBlock, normalizeTableCell, tableCellStyle, and renderTable to render BlockNote tables to email-safe HTML (<table>, optional <colgroup>, th/td, colspan/rowspan, inline styles).
Email exporter tests src/frontend/src/features/blocknote/email-exporter/index.test.tsx	Replace local helpers with shared factories; add extensive table tests, inline template-variable tests, and nested-combination coverage; remove prior “table not rendered” expectation.
Markdown & utils tests src/frontend/src/features/blocknote/markdown-exporter.test.ts, src/frontend/src/features/blocknote/utils.test.ts	Add regression tests for markdown exporter and resolveTemplateVariables.
Message form mailbox filtering src/frontend/src/features/forms/components/message-form/index.tsx	Filter available mailboxes for reply/forward by selected thread accesses and preserve draft parent fallback.

Backend: Tiered blob storage, GC, reservations, offload tasks, and DB model/migration changes

Layer / File(s)	Summary
Settings, docs, env, CI wiring src/backend/messages/settings.py, docs/tiered-storage.md, docs/env.md, env.d/development/backend.defaults, Makefile, .github/workflows/messages.yml	Add message-blobs S3 storage entry and new MESSAGES_BLOBS_* config; document tiered storage; update dev defaults and CI/make targets to create buckets.
Models and migrations (schema-first) src/backend/core/models.py, src/backend/core/migrations/0027_*.py, 0028_*.py, 0029_*.py	Introduce encryption_key_id, storage_location, nullable raw_content, MailboxBlob reservation model, change multiple FKs to PROTECT, add Attachment.message FK and backfill/drop M2M via migrations.
Tiered storage service & tasks src/backend/core/services/tiered_storage.py, src/backend/core/services/tiered_storage_tasks.py	Implement TieredStorageService with AEAD encrypt/decrypt, storage key computation, upload/download, rotate, re_store, and delete-if-orphaned; add offload_blobs_task and offload_one_blob.
Blob manager, GC and reservation logic src/backend/core/models.py (BlobManager/Blob changes), src/backend/core/services/blob_gc.py	Hash-first BlobManager.create_blob, user_can_access & is_referenced, Blob.get_content() routing to TieredStorage; schedule_for_gc, upload_and_reserve_blob, release_upload, and gc_orphan_blobs_task with Redis candidate set and advisory-lock deletion.
Management commands: verify/re_store src/backend/core/management/commands/verify_blobs.py, re_store_blobs.py	Add verify_blobs (db↔storage audit, optional hash verification) and re_store_blobs (rotation/restore with dry-run/limit).
Tests and CI for tiered storage src/backend/core/tests/services/test_tiered_storage.py, src/backend/core/tests/tasks/test_tiered_storage_tasks.py, src/backend/core/tests/commands/* and many other test updates	Comprehensive unit, DB, and E2E tests for encryption, offload, GC, verify/re_store commands, and Celery tasks; test fixtures to ensure S3 buckets and Redis are available.
Signals & scheduling integration src/backend/core/signals.py, src/backend/core/templates/admin/core/blob/change_form.html	Signals now enqueue blobs for GC on post-delete; admin blob download UI added.
Celery beat schedule src/backend/messages/celery_app.py	Schedule offload and GC tasks hourly.
Docker/runtime/tooling src/backend/Dockerfile, bin/update_app_cacert.sh, bin/scalingo_postfrontend, src/backend/pyproject.toml	Bump Python runtime to 3.14.4, update uv digest, adjust cert copy paths, update pyproject python requirement and pytest marker for redis.

Backend: Thread assignment/mention feature, permissions, serializers, viewsets, admin and cleanup wiring

Layer / File(s)	Summary
Enums and model-level schemas src/backend/core/enums.py, src/backend/core/models.py	Add ASSIGN/UNASSIGN ThreadEvent types, MAILBOX_ROLES_CAN_BE_ASSIGNED, thread-event JSON schemas, ThreadEvent.validate_data, and UserEvent partial-unique constraint for ASSIGN.
Service layer for assignments/mentions src/backend/core/services/thread_events.py	Implement assign_users, unassign_users, sync_im_mentions, undo-window absorption, and cleanup helpers for mailbox/thread access changes.
Serializers & OpenAPI src/backend/core/api/serializers.py, src/backend/core/api/openapi.json	Add OpenAPI-only serializers for thread-event data variants, ThreadMentionableUserSerializer (can_post_comments), extend MailboxSerializer/ThreadSerializer with count_assigned, is_shared, and assigned_users, and update OpenAPI to include new event types and schemas.
Viewsets: create/update behavior and permissions src/backend/core/api/viewsets/thread_event.py, thread_access.py, mailbox_access.py, thread.py, thread_user.py, blob.py	ThreadEvent create uses transactional locking and delegates ASSIGN/UNASSIGN to service layer (204 when absorbed); perform_update is atomic and re-syncs mentions; ThreadAccess/MailboxAccess viewsets run transactional perform_update/perform_destroy to trigger cleanup; ThreadViewSet supports has_assigned_to_me/has_unassigned filters and prefetches assigned user events; Blob upload enforces size cap and uses reservation flow; download authorization uses Blob.objects.user_can_access.
Admin changes & cleanup routing src/backend/core/admin.py	Superuser-only admin gate; route inline save/delete through cleanup helpers that invoke thread-events cleanup under transactions; Blob admin download endpoint.
Factories & tests src/backend/core/factories.py, many src/backend/core/tests/*	Factories updated to use reservation-aware blob creation and to sync ThreadEvent side-effects; large test additions/updates validate assignment/unassign, serializer validation, API behavior, query-count regression tests, and many adjustments to reflect GC/reservation semantics.
Migrations & DB constraints src/backend/core/migrations/0026_*.py	Add conditional unique constraint for UserEvent assign (user, thread) uniqueness.

Backend: RFC5322 composer rewrite, outbound flow changes, and tests

Layer / File(s)	Summary
Composer rewrite src/backend/core/mda/rfc5322/composer.py, README.md	Replace Flanker-based composer with stdlib email-based strict composer: header sanitization, Message-ID validation/normalization, robust multipart assembly, CTE policy, and compose_email returning bytes.
Outbound flow & signing src/backend/core/mda/outbound.py, src/backend/core/mda/autoreply.py	Split compose-and-store into compose_and_sign_mime (returns signed bytes) and transactional blob persist; autoreply now composes & signs then creates message blob. Draft/outbound paths adjusted to sign outside long transactions and schedule GC for replaced blobs.
Composer tests & fuzzing src/backend/core/tests/mda/test_rfc5322_composer.py, test_rfc5322_composer_fuzz.py	Massive expansion of RFC/security/MIME-structure tests and Hypothesis-based fuzz tests to validate invariants, header sanitization, quoting, boundaries, and parseability.
Adjusted tests various mda/outbound tests	Update to use factories, unfold quoted-printable soft breaks in assertions, and reflect new sign-and-store behavior in mocks.

Miscellaneous, infra, docs, and test harness updates

Layer / File(s)	Summary
Makefile & scripts Makefile, bin/pytest	Replace import-bucket target with create-buckets (adds message-blobs), add fuzz-back-intensive, and forward FUZZ_EXAMPLES into test container.
Docs docs/permissions.md, docs/architecture.md, docs/env.md	Document thread/user events, permission principles, tiered storage design, and new environment variables.
Checks & utils src/backend/core/checks.py, src/backend/core/utils.py	Add Django system checks for compression/encryption/redis lifecycle and validate_json_schema helper; make JSON parsing error raise ImproperlyConfigured without leaking secrets.
Many tests dozens under src/backend/core/tests/...	Update/extend many test modules to reflect new blob reservation/GC/offload behavior, serializer/viewset changes, added fields (is_shared, count_assigned, can_post_comments, assigned_users), and enable Redis-backed fixtures where required.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~240 minutes

Possibly related PRs

• suitenumerique/messages#645: Highly related — implements thread assignment/assignee features and overlaps on services, models, serializers, and OpenAPI.
• suitenumerique/messages#639: Highly related — earlier work on assignment/ThreadEvent types and related API/schema changes.
• suitenumerique/messages#577: Related to frontend email exporter and BlockNote rendering changes.

Suggested reviewers

• sylvinus
• sdemagny

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/frontend/src/features/blocknote/blocknote-view-field/_index.scss`:
- Around line 57-60: The SCSS block has stylelint errors caused by stray empty
lines before declarations/comments; remove the blank line(s) immediately
preceding the declarations such as --min-height, --toolbar-height, border-radius
and border-width and the other affected blocks around the later scoped block
(the region that contains the same CSS custom properties at lines ~116-124),
ensuring no empty line before any declaration or comment and consistent single
newline between rule blocks, then re-run stylelint to confirm the file
(blocknote-view-field/_index.scss) is lint-clean.

In `@src/frontend/src/features/blocknote/email-exporter/index.test.tsx`:
- Around line 750-905: Add a regression test in
src/frontend/src/features/blocknote/email-exporter/index.test.tsx that ensures
exportBlocks handles malformed table data without throwing (e.g., a row that's
null, a cell that's null, or a row object missing a cells array); create a test
case using the existing table/tableCell helpers to build a table-like block
where one row is null and another row is { /* missing cells */ } cast to
AnyBlock, call exportBlocks([...]) and assert it does not throw and does not
emit a <table> (or returns null/no table HTML). Locate the test near the other
'table' describe block and reference exportBlocks, table, tableCell to validate
fail-soft behavior.

In `@src/frontend/src/features/blocknote/email-exporter/index.tsx`:
- Around line 436-444: normalizeTableCell and the table row rendering assume
cells and rows are arrays/objects, causing exceptions for null/undefined or
legacy shapes; update normalizeTableCell to defensively handle
non-array/non-object inputs by returning { content: [], props: {} } for
null/undefined or primitive values, treat Array inputs as content, and ensure
props defaults to {}; likewise, in the table row rendering (the code that
iterates over row.cells) guard against rows that are null or where row.cells is
not an array by skipping those rows or coercing row.cells to an empty array
before mapping, and ensure each cell is passed through normalizeTableCell so
malformed cells yield safe defaults.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 21c4ee7e-0e48-42d5-851b-9f49defbb05f

📥 Commits

Reviewing files that changed from the base of the PR and between 43e368a and 265b73e.

⛔ Files ignored due to path filters (1)

• src/frontend/package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (4)

• src/frontend/package.json
• src/frontend/src/features/blocknote/blocknote-view-field/_index.scss
• src/frontend/src/features/blocknote/email-exporter/index.test.tsx
• src/frontend/src/features/blocknote/email-exporter/index.tsx

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/frontend/src/features/forms/components/message-form/index.tsx (1)

369-390: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Restore draft sync after sender transfer.

In the non-new flow, this deletes the current draft and creates a replacement, but the redirect is gone and local draft state is not updated. The form stays bound to the deleted draft id, so the next save/send can target a message that no longer exists.

Minimal fix

-            // router.replace(`/mailbox/${mailboxId}/thread/${threadId}?has_draft=1`);
+            await router.replace(`/mailbox/${mailboxId}/thread/${threadId}?has_draft=1`);

If you want to keep the user on the same route, update `draft`/thread state from `response.data` before returning instead.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/frontend/src/features/forms/components/message-form/index.tsx` around
lines 369 - 390, handleChangeSender currently deletes the old draft, creates a
replacement via draftCreateMutation, but in the non-"new" path never updates
local state or navigates, leaving the form bound to a deleted id; to fix, after
receiving response from draftCreateMutation update the local draft state (call
setDraft(response.data as Message) and any thread id fields) and then either
perform the redirect
(router.replace(`/mailbox/${data.senderId}/thread/${response.data.thread_id}?has_draft=1`))
or ensure form bindings use the new draft id so subsequent save/send targets the
new message; adjust code around deleteMessageMutation, draftCreateMutation,
setDraft and router.replace to keep UI and route in sync.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/frontend/src/features/forms/components/message-form/index.tsx`:
- Around line 102-113: The reply prefill logic uses selectedMailbox but
defaultSenderId can differ after filtering (availableMailboxes), causing the
chosen sender to be treated as external; compute the actual reply sender email
(e.g., replySenderEmail derived from defaultSenderId by looking up
availableMailboxes.find(m => m.id === defaultSenderId)?.email or from
draft.sender.email) and then replace uses of selectedMailbox?.email in the
toRecipients/ccRecipients filters and in the "reply to self" check with that
replySenderEmail so the filtered recipients and self-reply branch align with the
filtered sender selection (refer to availableMailboxes, defaultSenderId,
selectedMailbox, draft, toRecipients, ccRecipients).

---

Outside diff comments:
In `@src/frontend/src/features/forms/components/message-form/index.tsx`:
- Around line 369-390: handleChangeSender currently deletes the old draft,
creates a replacement via draftCreateMutation, but in the non-"new" path never
updates local state or navigates, leaving the form bound to a deleted id; to
fix, after receiving response from draftCreateMutation update the local draft
state (call setDraft(response.data as Message) and any thread id fields) and
then either perform the redirect
(router.replace(`/mailbox/${data.senderId}/thread/${response.data.thread_id}?has_draft=1`))
or ensure form bindings use the new draft id so subsequent save/send targets the
new message; adjust code around deleteMessageMutation, draftCreateMutation,
setDraft and router.replace to keep UI and route in sync.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 245b3f4d-9d52-422e-8d60-059988fc63f8

📥 Commits

Reviewing files that changed from the base of the PR and between 265b73e and 4e25906.

⛔ Files ignored due to path filters (1)

• src/frontend/package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (9)

• src/frontend/package.json
• src/frontend/src/features/blocknote/__tests__/block-factories.ts
• src/frontend/src/features/blocknote/blocknote-view-field/_index.scss
• src/frontend/src/features/blocknote/email-exporter/index.test.tsx
• src/frontend/src/features/blocknote/email-exporter/index.tsx
• src/frontend/src/features/blocknote/markdown-exporter.test.ts
• src/frontend/src/features/blocknote/utils.test.ts
• src/frontend/src/features/forms/components/message-composer/index.tsx
• src/frontend/src/features/forms/components/message-form/index.tsx

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/frontend/src/features/forms/components/message-form/index.tsx`:
- Around line 102-108: Detect when availableMailboxes is empty (e.g., compute a
hasNoEditAccess flag alongside the existing useMemo for availableMailboxes) and
render a user-facing banner before the form when hasNoEditAccess is true;
specifically, after computing availableMailboxes (which currently filters by
ThreadAccessRoleChoices.editor using selectedThread, mailboxes and mode), add
logic to check availableMailboxes.length === 0 and display an error/info Banner
telling the user they lack permission to reply/forward from any of their
mailboxes for this thread so the sender dropdown isn’t simply empty and
confusing.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c870d003-14e4-4427-9d89-a08a385ceafb

📥 Commits

Reviewing files that changed from the base of the PR and between 4e25906 and 91a507e.

📒 Files selected for processing (7)

• src/frontend/src/features/blocknote/__tests__/block-factories.ts
• src/frontend/src/features/blocknote/email-exporter/index.test.tsx
• src/frontend/src/features/blocknote/email-exporter/index.tsx
• src/frontend/src/features/blocknote/markdown-exporter.test.ts
• src/frontend/src/features/blocknote/utils.test.ts
• src/frontend/src/features/forms/components/message-composer/index.tsx
• src/frontend/src/features/forms/components/message-form/index.tsx