The Dickens Notes project

The Dickens Notes project allows users to explore and interpret the ‘working notes’ Charles Dickens kept for his novels. Dickens composed and published all of his fifteen novels in monthly or weekly installments, and for many novels he used these working notes–-one sheet of paper for each installment–-to consider character combinations and plot developments; to record titles, events, and key phrases; and to document decisions made or deferred to subsequent numbers.

Dickens Notes pairs accurate transcriptions of the working notes with scholarly introductions and annotations that illuminate their significance for understanding the dynamics of Dickens’s creative process and Victorian serial form.

Adding or editing text content

In Dickens Notes, most pages are written in markdown, in .md or .mdx files found in src/pages. Each page is one file, and the URL follows the file path. To edit the General Introduction at /introduction/general in the browser, you'd edit the file src/pages/introduction/general.mdx.

If you aren't familiar with markdown, Github's docs on writing in markdown are a great start.

The easiest way to add a new page or modify text content is through Github's online editor. If you're on the repo page in the browser, type . to open the codebase in a lightweight version of VS Code in the browser called github.dev. You can find instructions on how to edit files and save changes here.

The general process will be this:

1. Open the repository in github.dev
2. Create a new branch for your work, or switch to the branch you've been working on.
3. Make changes.
4. Save your changes through Source Control per the docs.
5. In Github, create PR from your working branch.
6. Merge the PR when you're ready.

Viewing Specific Annotation in Mirador

You can view a specific annotation by populating the canvas and annotationid property. The annotationid should match the id field in the annotation. The canvas should direct mirador to the correct canvas (image) that the annotation is on. For example: https://tubular-narwhal-84b42b.netlify.app/notes/bleak-house/mirador/?canvas=https://dickensnotes.github.io/dickens-annotations/canvas/img/derivatives/iiif/bleakhousetranscriptions/BHWN04.json&annotationid=00547443-abdb-4f41-bb6b-c3a0cf5076cf.json

Running the DDNP locally

Dickens Notes is built on Astro, uses TailwindCSS for styling, and several React libraries, including Mirador.

It also leverages pnpm to manage JS dependencies.

To run locally:

1. Ensure you have Node and pnpm installed.
2. Ensure you have Python 3 and uv installed (required for search index building).
3. Clone the repository.
4. Run pnpm install at the root of the repository to install JavaScript dependencies.
5. Run uv sync to install Python dependencies (BeautifulSoup, requests, PyYAML).
6. Run pnpm run dev at the root of the repository to run the application.

Commands

All commands are run from the root of the project, from a terminal:

Command	Action
pnpm install	Installs JavaScript dependencies
uv sync	Installs Python dependencies (for search indexing)
pnpm run dev	Starts local dev server at localhost:4321
pnpm run build:index	Builds search index from annotations, notes, and pages
pnpm run build	Builds search index + production site to ./dist/
pnpm run preview	Preview your build locally, before deploying
pnpm test	Runs test suite (build + search functionality tests)
pnpm test:watch	Runs tests in watch mode (re-runs on file changes)
pnpm test:ui	Opens interactive test UI in browser

Feel free to check the Astro documentation or jump into their Discord server for help with Astro.

Search Index

Dickens Notes uses MiniSearch to index and search across three types of content:

• Working Notes (65 text files from 4 novels)
• Annotations (fetched from remote IIIF API)
• Site Content (MDX pages)

The search index is built automatically during pnpm run build, or can be built manually with pnpm run build:index. The build process:

1. Fetches annotations from the remote API using Python
2. Indexes working notes text files
3. Indexes site content pages
4. Generates public/assets/javascript/search-data.json (~4 MB)

Note: The search index builder requires Python 3 and uv. Install Python dependencies with uv sync.

Testing

The project includes an automated test suite built with Vitest to ensure search functionality works correctly and prevent regressions.

Test Coverage

Build Pipeline Tests (tests/build-index.test.js)

• Validates search index structure and content
• Verifies document counts (1121 total: 1041 annotations, 65 working notes, 15 site content)
• Checks correct breakdown by novel and content type
• Validates URL structure for all result types

Search Functionality Tests (tests/search.test.js)

• Tests search initialization and document retrieval
• Validates query execution with known terms
• Checks result structure and relevance sorting
• Tests prefix and fuzzy matching features

Running Tests

pnpm test          # Run all tests once
pnpm test:watch    # Run tests in watch mode (auto-reruns on changes)
pnpm test:ui       # Open interactive test UI in browser

All tests must pass before merging changes to the search functionality.

Global/window shim

Due to Vite, we have to shim window for Mirador to work. We're currently doing this by adding a snippet to the HTML of any page with Mirador.

<script>window.global = window;</script>