Skip to content

HTML notebook outputs such as pandas DataFrames are missing from the served page and appear only after JavaScript runs #216

Description

@mmcky

Found while re-measuring #206 for the #147 gate. It is filed without a parent: readers with JavaScript do see the tables, so no lecture page visibly breaks at cutover.

Problem

On a MyST lecture page, a notebook output is left out of the HTML the site serves when its data holds anything besides plain text or a PNG, GIF, JPEG or BMP image. Pandas DataFrames are the common case. The output's container is served empty, and the table is built in the reader's browser after the page's JavaScript loads. The Sphinx lecture sites serve the same tables inside the page itself.

Measured on 2026-09-11 on the /pandas page of lecture-python-programming's jb2 preview (theme v2.7.0), using Chromium at 1440px:

Sphinx: https://python-programming.quantecon.org/pandas.html MyST: jb2 preview /pandas
DataFrame outputs 33 tables 34 outputs with text/html + text/plain in the page data
table.dataframe in the served HTML (JavaScript off) 33 0
table.dataframe with JavaScript on 33 33, about 0.5 s after DOMContentLoaded

Who notices:

  • No JavaScript run: readers without JavaScript, and any tool that reads the served HTML without running scripts, such as crawlers that don't execute JavaScript, link previews, archivers and reader views.
  • Everyone on first load: the tables appear after the text around them, which shifts the page.

Why it happens

  • Most outputs skip server rendering. @myst-theme/jupyter 1.3.0 renders an output directly only when it is a stream or error, or when every one of its MIME types is text/plain, PNG, GIF, JPEG or BMP (DIRECT_OUTPUT_TYPES and DIRECT_MIME_TYPES, dist/output.js lines 16-23; isOutputSafe, lines 24-39). Every other output goes to JupyterOutput (line 74).
  • JupyterOutput only runs in the browser. It builds the DOM with thebe-core's rendermime registry inside useEffect (dist/jupyter.js lines 9-35). Upstream calls this interim: "FUTURE: we'll be rendering AST outputs directly in future" (dist/output.js line 60).
  • The theme adds nothing here. Its only output override wraps stderr streams and hands every other output to upstream (app/renderers.tsx L79-L101).
  • The QuantEcon mystmd fork converts HTML outputs to AST, but only for PDF. reduceOutputs runs text/html outputs through htmlTransform (transforms/outputs.ts L247-L317). It runs only when simplifyFigures is set (process/mdast.ts L543-L547), and only the tex and typst builds set it; the site build does not.

Options

Option Where What Trade-offs
A. Server-render static HTML outputs This theme's output override For an output whose richest type is text/html with no scripts, render a sanitised copy of that HTML on the server and keep it through hydration. Interactive outputs (scripts, plotly, widgets) stay with JupyterOutput Ships with the theme alone, and works on stock mystmd too. Needs an HTML sanitiser in the bundle, a rule for what counts as static, and handling for the <style scoped> blocks pandas emits
B. Convert HTML outputs to AST at build time QuantEcon/mystmd, site build Reuse reduceOutputs' htmlTransform path, so static text/html outputs reach the page as ordinary content that myst-to-react renders on the server DataFrames become ordinary tables (theme-styled and searchable). Conversion is lossy for complex HTML, and it is fork-only until upstreamed, so stock-mystmd builds keep today's behaviour
C. Render outputs from AST upstream jupyter-book/myst-theme The direction upstream's own comment points to The best long-term home; no timeline

Acceptance criteria

  • On a lecture-python-programming build, /pandas shows its DataFrame tables with JavaScript disabled.
  • With JavaScript enabled, each table renders once, with no duplicate after hydration, and interactive outputs such as plotly figures and widgets still work.
  • No unsanitised author HTML reaches the server-rendered page.
  • A visual-test fixture stores a text/html DataFrame output, and a test asserts the table is present in the served HTML.

Context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementImprovement to existing material

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions