feat: tressiter for colored codeblocks - #625
Draft
eszlamczyk wants to merge 15 commits into
Draft
Conversation
6 tasks
eszlamczyk
force-pushed
the
feat/51/code-block-highlighter-treesiter-vendor
branch
from
August 3, 2026 13:10
55c0b2a to
f7a8103
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 25 changed files in this pull request and generated no new comments.
Suppressed comments (2)
packages/react-native-enriched-markdown/plugin/src/withIosCodeHighlight.ts:18
- The early return for the default case (
enabled && !hasLanguages) means the plugin won’t remove previously injectedENRICHED_MARKDOWN_*lines if a project switches back to defaults after customizing. That can leave stale Podfile ENV overrides in place and make the app build behave differently than the Expo config.
Running the mod unconditionally (but injecting nothing for the default case) keeps the Podfile consistent with config changes.
const hasLanguages = Array.isArray(languages) && languages.length > 0;
// Default (enabled, curated set) needs no Podfile ENV lines.
if (enabled && !hasLanguages) {
return config;
}
packages/react-native-enriched-markdown/plugin/src/withAndroidCodeHighlight.ts:34
- Even when the config returns to the default (
enabled && !hasLanguages), it’s still useful to run the mod so any previously-addedenrichedMarkdown.*properties get removed. With the current early return, switching from a customized config back to defaults leaves stale gradle.properties entries behind.
Consider always dropping existing keys, and only re-adding properties when the config is non-default.
// Default (enabled, curated set) needs no gradle property.
if (enabled && !hasLanguages) {
return config;
}
return withGradleProperties(config, (gradleConfig) => {
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What/Why?
Second part of #51 — closes #51.
Part one landed the code-block segmentation/rendering seam. This PR wires the
actual native syntax highlighter behind it: fenced code blocks are now
highlighted on iOS and Android via a vendored tree-sitter
runtime plus a curated set of grammars, foreground-only (it only recolors
tokens, never changes text metrics, so a block's measured height always matches
its drawn height).
Highlighting is on by default with a curated language set, and can be trimmed or
fully disabled per platform to control binary size.
The vendoring problem it solves
Tree-sitter grammars ship as machine-generated
parser.cparse tables — the fullsupported set is ~177 MB / ~1.1M LOC of generated C. Committing that would
bloat every clone, dominate PR diffs, and swamp GitHub's language stats. It also
has no business living in git: it is 100% reproducible from the pinned grammar
packages. The same is true of the tree-sitter runtime (~21k lines) and the
codegen'd registry — all machine-generated, all reproducible from pins.
So none of the vendored C is committed. The entire
vendor/tree is gitignoredand reproduced on demand from
vendor/grammar-versions.json:vendor/grammars/— the ~177 MB ofparser.c/scanner.ctables, copiedfrom the grammar devDependencies (version-pinned in the lockfile).
vendor/tree-sitter/— the ~21k-line runtime, fetched and sha256-verifiedfrom the pinned GitHub release tarball (
runtime.tarball+runtime.sha256inthe manifest). No native build, no compiler — just
lib/src+lib/include.vendor/generated/— the default registry, codegen'd from the vendoredgrammars by
gen-registry.mjs.Restore is idempotent (
.stampfingerprints per runtime pin and per grammar set),wired into
prepare(soyarn installself-heals) andprepack(so the publishedtarball ships the full set — consumers install it prebaked and never fetch).
Net effect: the branch reads as ~2k lines of genuinely reviewable diff — all
the vendored/generated C is out of git entirely — while a
git clone+yarn install(or annpm installof the published tarball) still ends up withthe complete set.
New scripts / infra
vendor/vendor-grammars.mjs— one idempotent restore command. Fetches +sha256-verifies the runtime from the pinned tarball, copies each grammar's
minimal source set (only
parser.c,scanner.cwhen present,tree_sitter/*.h,highlights.scm,LICENSE) from the grammar devDependencies, and regeneratesthe default registry. Separate
.stampfingerprints (runtime ref+sha; pinnedgrammar versions) make repeated
yarn installs a no-op — no re-fetch, norewrite of 177 MB.
--forcerebuilds regardless;--only a,bscopes to specificgrammars;
--runtime-src <lib>vendors the runtime from a local checkout offlineinstead of the network.
vendor/gen-registry.mjs— offline, build-time codegen. Emitsgenerated_queries.h+generated_registry.cppfor exactly the selectedlanguage subset, so the link step is valid no matter how many grammars are
vendored. Invoked by the Android CMake configure step, the iOS podspec, and by
vendor-grammars.mjsto (re)generate the default set. Resolves; inherits:directives (e.g. cpp inherits c) by inlining parent highlights.vendor/grammar-versions.json— single pin manifest: the tree-sitterruntime ref (
v0.26.3, ABI 13-15) + tarball URL + sha256, and every grammar'spackage/version/scanner flags plus which form the curated default set.
scripts/prepare-npm-publish.sh—prepackrestores the whole vendor tree,copies
core/cppinto the RN package, and shipsgen-registry.mjs+grammar-versions.jsonalongside so a consumer can compile a custom languageset;
postpackrestores the symlink and cleans up.preparehook now runsvendor-grammars:installbefore the bob build, so afresh
yarn installself-heals the entire gitignored vendor tree.Consumers can pick their language set (and shrink the binary) via
ENRICHED_MARKDOWN_CODE_HIGHLIGHT_LANGUAGES(Podfile ENV) /enrichedMarkdown.codeHighlightLanguages(gradle.properties) / the Expo configplugin - see
docs/CODE_HIGHLIGHT.md.Testing
Quick summary of what was verified:
yarn installregenerates the gitignoredvendor/grammars/from thedevDependencies; a second install is a
.stampno-op.gen-registry.mjsproduces a registry for both the default set and an arbitrary--only/--languagessubset; the link step stays valid.default languages, and respects
codeBlock.syntaxColors.and drop the unselected grammars from the binary.
yarn workspace react-native-enriched-markdown packproduces a tarball thatcontains the full vendored grammar set (prepack restore), while the git
working tree does not — verified by installing the tarball into a scratch RN app
and comparing sizes.
PR Checklist