Phase 1: Foundation & MVP ✅ COMPLETE
→ Lexer, Parser, basic interpreter, InMemoryFs
→ 15 core commands
→ Build pipeline (MoonBit → JS → npm)
Phase 2: Shell Feature Completeness 🔧 CORE COMPLETE
→ Full variable expansion, arrays, functions
→ Control flow (if/for/while/case)
→ Redirections and pipes
→ 20 additional commands implemented
→ Compatibility backlog still open (see test snapshot)
Phase 3: Text Processing Powerhouse 🔧 CORE COMPLETE
→ grep, sed, awk (full implementations)
→ jq (via bobzhang/moonjq community package)
→ diff, comm, base64, md5sum, sha256sum, gzip, tar
→ All 87 target commands implemented
→ Spec compatibility hardening still in progress
Phase 4: Production Hardening 🔧 IN PROGRESS
→ Comparison test suite: 523/523 (100%)
→ Security test suite: 27 files (fuzzing, prototype-pollution, sandbox)
→ Network: curl, html-to-markdown
→ Custom command bridge
→ Remaining: AgentFS adapter, npm publish
→ Release/minified npm packaging paths landed (`vp run build`, `vp run build:publish`)
→ OverlayFs/MountableFs ⏸️ superseded by AgentFS
Phase 5: Multi-Platform Expansion 🔧 PARTIALLY COMPLETE
→ Data processors: yq, xan, csvlook (done)
→ Compression: gzip/gunzip/zcat real DEFLATE via gmlewis/gzip (done)
→ tar: MBTAR1 self-contained format (bobzhang/tar unsuitable)
→ VM bridges: python3, sqlite3 (done)
→ Browser website demo + browser build path landed
→ Remaining: WASM target, published browser packaging hardening, standalone REPL
Phase 6: just-bash 3 API Compatibility 🔧 IN PROGRESS
→ Surface parity matrix, command-name helpers, ByteString helpers, ExecOptions compatibility
→ Deprecated top-level execution limit aliases, processInfo special variables, and trace callback types landed
→ TypeScript compile-only consumer check added for public facade exports
→ InMemoryFs public async runtime API landed and is accepted by Bash({ fs })
→ MountableFs routing, mount management, and cross-filesystem copy/move landed
→ ReadWriteFs direct root-confined Node filesystem access landed
→ OverlayFs copy-on-write host-directory access landed
→ Sandbox runCommand, Command, file helpers, custom fs, and overlayRoot landed
→ Transform/parser facade for simple commands, pipelines, basic compound AST, and recursive command collection landed
→ Node ESM, CommonJS, and browser subpath package exports landed
→ Basic TypeScript js-exec facade, upstream CLI option/error semantics, CommonJS/module-mode fs/path shims, and upstream node stub landed
→ Inline-tools executor companion and executor package subpath landed
→ Executor setup custom sources, approval gates, and elicitation gates landed
→ Basic javascript.invokeTool tools proxy wiring landed
→ Remaining: full QuickJS/js-exec runtime, remaining Node-compatible module shims and full ESM loader support, GraphQL/OpenAPI/MCP executor SDK discovery, SDK-native elicitation, full upstream AST coverage
Current comparison test pass rate: 523/523 (100%)
Command coverage: 87/87 (100%)
Spec/security hardening snapshot (2026-04-20): comparison suite is green at 523/523, security attacks remain 0 failed, and gzip uses real DEFLATE. Bash/grep/jq spec compatibility hardening remains in progress.
✅ COMPLETE: core milestone is delivered and production-usable; minor or non-blocking backlog may still be listed as unchecked items.🔧 CORE COMPLETE: core implementation is landed, but compatibility hardening/backlog remains.⏸️ REPLACED: original task is superseded by an approved architecture decision.- Pass/fail truth source:
docs/TEST_STATUS_2026-02-19.md,docs/TEST_FAILURE_MATRIX_2026-02-19.md,docs/TEST_FIX_PLAN_2026-02-19.md.
Goal: Parse and execute simple bash scripts in-memory. Establish the full build pipeline from MoonBit source to npm package.
- Initialize MoonBit project (
moon.mod.json) - Configure JS backend (
moon.pkg.jsonwith"targets": {"js": {...}}) - Set up TypeScript wrapper project (
package.json,tsconfig.json) - Configure build pipeline (
moon build --target js --release→vp pack; optional publish minify viavp run build:publish) - Set up test infrastructure (MoonBit tests + TS integration tests)
- CI/CD pipeline (GitHub Actions)
- Define all AST node types as MoonBit
enum/struct - Implement
to_string()/Showfor debug printing - Implement
to_json()for AST serialization (debugging/transform plugins)
- Token type definition
- Basic word tokenization
- Quoted string handling (single, double, ANSI-C)
- Escape sequence handling
- Operator tokenization (
|,&&,||,;,&) - Redirection tokenization (
<,>,>>,<<,<<<, etc.) - Reserved word recognition (
if,then,else,fi,for,while, etc.) - Comment stripping (
#) - Heredoc content collection
- Parser limit enforcement (MAX_INPUT_SIZE, MAX_TOKENS)
- Comprehensive lexer tests
- Recursive descent parser structure
- Script (statement list) parsing
- Simple command parsing (words + redirections)
- Pipeline parsing (
cmd1 | cmd2) - List parsing (
&&,||,;,&) - Depth limit enforcement
- Parser error reporting with position info
- Parser tests
-
HashMap-based file storage - Path normalization (
.,.., multiple slashes) - Null byte validation
-
read_file,write_file,append_file -
exists,stat -
mkdir(with recursive) -
rm(with recursive + force) -
cp(with recursive) -
readdir -
symlink,readlink(with loop detection) -
chmod - Default layout creation (
/home/user,/bin,/tmp) - Filesystem tests
- Execution context (env, cwd, stdin/stdout/stderr)
- Simple command execution
- Pipeline execution (stdout chaining)
- Execution limit tracking (command count)
- Exit code handling (
$?) - Basic variable expansion (
$VAR,${VAR}) - Command substitution (
$(cmd)) - Word splitting on IFS
| Command | Priority | Status |
|---|---|---|
echo |
P0 | ✅ |
cat |
P0 | ✅ |
cd |
P0 | ✅ |
pwd |
P0 | ✅ |
ls |
P0 | ✅ |
mkdir |
P0 | ✅ |
rm |
P0 | ✅ |
cp |
P0 | ✅ |
mv |
P0 | ✅ |
touch |
P0 | ✅ |
head |
P0 | ✅ |
tail |
P0 | ✅ |
wc |
P0 | ✅ |
true |
P0 | ✅ |
false |
P0 | ✅ |
-
Bashclass withexec()method -
ExecResult/BashExecResulttypes -
BashOptionsconfiguration -
InitialFilessupport - Bridge setup (global registration)
- Type definitions (
.d.ts)
-
moon build --target jsintegration - Vite+
packbundling (ESM + types) - npm package structure
- Verify drop-in compatibility with just-bash API
- Create a command classification sheet (direct library / stdlib composition / state-machine / FFI).
- For every Phase 1 command, record the primary MoonBit API(s) and fallback path.
- Implement
cat,head,tail,wcusing shared line-stream helpers (string+array) rather than per-command parsing code. - Implement
ls,cd,pwd,mkdir,touch,rm,cp,mvon top of a single VFS trait and common path-normalization utility. - Keep
echo,true,falseas builtin fast-path commands with zero allocator-heavy logic. - Add tests that assert algorithm reuse behavior (for example: one shared sorter/path normalizer used by multiple commands).
- Add a merge gate: no new hand-rolled algorithm for a command if an approved package/stdlib path exists.
Goal: Support the full range of Bash control flow, expansions, and shell builtins.
-
if / elif / else / fi -
for var in words; do ... done - C-style
for ((i=0; i<10; i++)); do ... done -
while condition; do ... done -
until condition; do ... done -
case word in pattern) ... ;; esac - Case terminators:
;;,;&,;;& - Subshell
( commands ) - Group
{ commands; } - Loop control:
break,continue(with depth)
- Assignment (
VAR=value) - Local variables (
local VAR=value) - Export (
export VAR=value) - Readonly (
readonly VAR=value) - Unset (
unset VAR) - Indexed arrays (
arr=(a b c),${arr[0]},${arr[@]}) - Associative arrays (
declare -A map,${map[key]}) - Special variables (
$?,$#,$@,$*,$0,$$,$!,$RANDOM,$LINENO) -
declarewith attributes (-i,-l,-u,-n,-r,-a,-A) - Namerefs (
declare -n ref=var)
- Brace expansion (
{a,b,c},{1..10},{1..10..2}) - Tilde expansion (
~,~user) - Full parameter expansion (all
${VAR...}forms) - Arithmetic expansion (
$(( ))) - Process substitution (
<(cmd),>(cmd)) - Quote removal
- Glob/pathname expansion (
*,?,[...]) - Extended globbing (
?(pat),*(pat),+(pat),@(pat),!(pat)) - Globstar (
**)
- Integer arithmetic (
+,-,*,/,%) - Comparison operators (
<,>,<=,>=,==,!=) - Logical operators (
&&,||,!) - Bitwise operators (
&,|,^,~,<<,>>) - Ternary (
cond ? a : b) - Assignment operators (
=,+=,-=,*=,/=, etc.) - Pre/post increment/decrement (
++,--) - Parenthesized grouping
- Int64 semantics (matching bash 64-bit wrap-around)
-
test/[command -
[[ ]]extended test - File tests (
-f,-d,-e,-r,-w,-x,-s,-L, etc.) - String tests (
-z,-n,=,!=,<,>) - Numeric tests (
-eq,-ne,-lt,-le,-gt,-ge) - Pattern matching in
[[ ]](==,!=with globs) - Regex matching (
=~) withBASH_REMATCH - Logical operators (
-a,-o,!,&&,||) - Variable test (
-v)
- Input:
<,<<,<<<,<> - Output:
>,>>,>| - Stderr:
2>,2>>,&>,&>> - Duplication:
>&,<& - FD variables:
{var}>file - Multiple redirections per command
-
/dev/null,/dev/stdin,/dev/stdout,/dev/stderr
- Function definition (
function name { }andname() { }) - Function invocation
- Positional parameters (
$1,$2, ...,$@,$*,$#) -
shift - Local variable scoping
-
returnwith exit code - Recursive call depth tracking
-
set(options + positional params,set -o/+o) -
shopt(shell options) -
alias/unalias -
read(with-r,-p,-a,-d,-t,-n) -
mapfile/readarray -
printf(full format string support) -
source/. -
pushd/popd/dirs -
trap(limited - signal handling simulation) -
type/command -
hash -
enable -
eval -
exec(redirection context + command dispatch) -
getopts -
let -
bash/sh(sub-script execution)
| Command | Status | Notes |
|---|---|---|
env |
✅ | Print environment |
export |
✅ | Set env vars |
printenv |
✅ | Print env vars |
basename |
✅ | Path manipulation |
dirname |
✅ | Path manipulation |
which |
✅ | Command lookup |
seq |
✅ | Number sequences |
date |
✅ | Date formatting (FFI timer) |
sleep |
✅ | Delay (via FFI timer bridge) |
expr |
✅ | Expression evaluation (Pratt parser) |
tee |
✅ | Stdin to file + stdout |
sort |
✅ | Sort lines |
uniq |
✅ | Deduplicate lines |
grep |
✅ | Pattern matching (BRE/ERE/fixed) |
cut |
✅ | Field extraction |
tr |
✅ | Character translation |
rev |
✅ | Reverse lines |
tac |
✅ | Reverse file |
paste |
✅ | Merge lines |
nl |
✅ | Number lines |
- Implement
env,printenv,export,alias,unaliason a unified session-stateHashMapmodel. - Implement
basename,dirnameas pure path-string transforms shared by parser expansions and command layer. - Implement
sort,uniq,cut,tr,rev,paste,nlwith composable iterator/text helpers, not command-specific ad hoc loops. - Implement
seqon top of the math/eval helper module with strict numeric bounds. - Implement
datewith adapter; keep output format compatibility tests in comparison suite. - Implement
sleepvia FFI timer bridge only; enforce cancellation + timeout behavior in security tests. - Before Phase 3, run a reuse audit documenting which commands still require custom engines (
awkand parser-related paths).
Goal: Implement the complex text processing commands that make the sandbox useful for real data pipeline work.
- BRE (Basic Regular Expressions)
- ERE (Extended Regular Expressions) -
-E - Fixed string matching -
-F - All flags (
-i,-v,-c,-n,-l,-L,-w,-x,-q) - Context lines (
-A,-B,-C) - Recursive search (
-r,-R) - Multiple patterns (
-e) - Pattern file (
-f) - Integration with
@regexplibrary -
egrep,fgrep,rgaliases
- Address types (line number,
$,/regex/, range, step, negation!) - Substitute command (
s/pattern/replacement/flags) withg,p,i, occurrence count - Delete (
d), Print (p), Append (a), Insert (i), Change (c) - Transliterate (
y) - Read/Write file (
r,w) - Branch (
b), Test (t,T) - Hold space operations (
h,H,g,G,x) - Multiline (
N,P,D) - In-place editing simulation (
-i) - Multiple expressions (
-e) - Script file (
-f) - Label resolution and branch execution
- Quit (
q,Q), List (l), Line number (=) - BRE/ERE support, empty-regex reuse
- Iteration limit enforcement
- AWK lexer and parser
- Pattern-action rules (BEGIN, END,
/regex/, expression) - Field splitting (
$0,$1, ...,$NF) - Built-in variables (
NR,NF,FNR,FS,RS,OFS,ORS,SUBSEP,FILENAME,ENVIRON, etc.) - String functions (
length,substr,index,split,gsub,sub,match,sprintf,printf,tolower,toupper) - Math functions (
sin,cos,atan2,sqrt,int,log,exp,rand,srand) - User-defined functions (with array parameter pass-by-reference)
- Arrays (associative,
for-inloops,inoperator,delete) - Control flow (
if/else,for,while,do-while,for-in,break,continue,next,exit,return) - Getline (bare, from file, from command pipe)
- I/O:
print,printf,close(),system() - Print/printf redirection to files (
>,>>) - Ternary expressions, string concatenation
-
OFMTformatting - Iteration limit enforcement
- Prototype-pollution hardening (for-in, function params, getline vars)
Migrated from handwritten evaluator to bobzhang/moonjq (MoonBit creator's package, commit dbc5247). Full jq language support provided by the community package, including:
- Identity, field access, array/object indexing, slicing
- Pipes, comma, object/array construction
- Conditionals, comparison, logical operators
- String interpolation
- Try-catch, alternative (
//), variable binding (as $var) - Reduce, foreach, recursive descent (
..) - Path expressions, format strings
- All core built-in functions
- Iteration limit enforcement (via wrapper)
-
diff(unified format, viamoonbit-community/piediff) -
cmp(byte-level file comparison) -
comm(compare sorted files) -
join(join on common field) -
column(columnate) -
fold(wrap lines) -
expand/unexpand(tabs ↔ spaces) -
od(octal dump) -
strings(find printable strings) -
xargs(build commands from stdin) -
split(split files)
-
md5sum(viagmlewis/md5) -
sha1sum(viagmlewis/sha1) -
sha256sum(viashu-kitamura/sha256) -
base64(encode/decode, viagmlewis/base64)
-
gzip(viagmlewis/gzip+gmlewis/flate) -
gunzip(viagmlewis/gzip) -
zcat(viagmlewis/gzip) -
tar(viabobzhang/tar, pure-memory byte stream)
-
find(with-name,-type,-path,-exec,-maxdepth,-perm, etc.) -
du(disk usage, with-h,-s,-d) -
stat(file status) -
file(file type detection) -
tree(directory tree display) -
ln(symbolic links) -
readlink(resolve symlinks) -
rmdir(remove empty directories) -
chmod(standalone command)
-
hostname -
whoami -
time(command timing) -
timeout(run with time limit) -
history(command history) -
help(help text) -
clear(no-op in sandbox)
Goal: Production-ready release with comprehensive testing, security hardening, and advanced filesystem support.
架构决策(2026-02-19): AI agent 主场景下,OverlayFs 的"宿主磁盘读层 + 内存写层"设计 被 AgentFS(Turso,SQLite-backed VFS)完整替代。AgentFS 天然提供 COW(
fs_whiteout表)、 持久化、可审计、可快照能力,且已有 just-bash 一等集成。详见docs/AGENTFS_ANALYSIS.md。若未来需支持本地开发工具场景(直接读宿主项目目录、不预装进 SQLite),可重新激活此计划。
-
FFI-backed disk read layer→ AgentFS SQLite 替代 -
Memory write layer→ AgentFS 写回 SQLite -
Deleted file tracking→ AgentFSfs_whiteout表 -
Path security validation→ AgentFS 内部处理 -
Size limits on disk reads→ AgentFS 内部处理 - AgentFS adapter in TypeScript wrapper layer (NEW)
架构决策(2026-02-19): AgentFS 单个 SQLite 即完整命名空间,无需多后端路由。 MountableFs 的多挂载点设计在 AgentFS 模式下不再必要。详见
docs/AGENTFS_ANALYSIS.md。
-
Multi-mount point routing→ AgentFS 单一命名空间 -
Mount/unmount API→ 不再需要 -
Path normalization across mounts→ 不再需要
-
curlcommand implementation (viaglobalThis.fetchFFI) -
html-to-markdowncommand - URL prefix allowlist enforcement
- HTTP method restriction
- Redirect following with validation
- Timeout and response size limits
- Prototype-pollution hardening (AWK for-in, function params, getline vars, builtins)
- Execution limit enforcement (commands, loops, call depth, string size)
- Pipefail semantics
- JS global patching (Function, eval, etc.)
- Audit mode
- Violation callbacks
- Configurable exclusions
- Plugin registration API
- AST visitor infrastructure
- Built-in plugins (CommandCollector, Tee)
- Custom command bridge (
__moon_bash_custom__via FFI) - User-provided command handlers (async, via TS wrapper)
- Lazy command loading
- Command filtering (
commandsoption)
- Comparison test framework (record + replay, 26 fixture files)
- Test fixtures against real bash output: 523/523 (100%)
- Spec suites integrated: bash / awk / sed / grep / jq
- Bash spec corpus imported and runnable (from Oils project)
- AWK spec corpus imported and runnable
- Spec compatibility fully green (as of 2026-02-19: grep
54failed, jq170failed, bash spec chunked run683+failed confirmed) - Security fuzz testing (grammar-based, flag-driven, malformed, coverage-boost generators)
- Prototype-pollution test suite (6 files, comprehensive coverage)
- Sandbox escape tests (command security, injection, dynamic execution, information disclosure)
- Resource limit tests (DoS, memory, output size, pipeline limits)
- Security suites fully green (as of 2026-02-19:
tests/security/attackshas1failing case; fuzzing has2failing suites) - Agent workflow tests: 13 real-world scenarios
- OOM-safe batched test execution (
vp run test:safe) - Edge case coverage (Unicode, binary, huge files)
- Performance benchmarks vs just-bash
- API reference documentation
- Migration guide from just-bash
- Performance comparison benchmarks
- npm publish (initial release)
- mooncakes.io publish
Goal: Extend MoonBash beyond the npm ecosystem.
-
moon build --target wasmconfiguration - WASI interface layer
- Python bindings (
wasmtime/wasmer) - Rust bindings (
wasmtimecrate) - PyPI publish (
pip install moon-bash)
- Browser-specific build path (
vp run build:website) - Static website demo (
examples/website/) replicating thejustbash.devfull-screen terminal style - MoonBit frontend mount package (
src/website/) for browser UI bootstrapping - Browser wrapper entry scaffold (
src/wrapper/browser.ts) - IndexedDB-backed persistent VFS
- xterm.js integration example
- Web Worker support (off-main-thread execution)
- CSP-compatible (no eval, no dynamic imports)
Current status note (2026-04-18): the browser demo is real and locally runnable today, but the npm package export surface is still primarily Node-oriented. The website build bundles directly from source plus compiled MoonBit JS, which is sufficient to demonstrate browser usability and embedability.
Update (2026-04-18): the browser website runtime has since been refactored toward MoonBit. The demo still uses a thin JS bootstrap, but DOM construction, browser state, async command execution, history/completion, and autoplay verification now live in src/website/*.mbt.
- Browser demo line history
- Browser demo tab completion
- General-purpose REPL abstraction
- Prompt customization (
PS1) - CLI binary (
npx moon-bash)
Current status note (2026-04-18): the website demo already includes a minimal interactive shell loop in the browser, but a reusable standalone REPL for package consumers or CLI use is still pending.
-
yq(YAML processor, viamoonbit-community/yaml) -
xan(CSV processor, viaxunyoyo/NyaCSV) -
csvlook(CSV display, viaxunyoyo/NyaCSV) -
rg(ripgrep-compatible search, mapped to grep-E) -
python3integration (optional, via Pyodide FFI bridge) -
sqlite3integration (optional, via sql.js FFI bridge) -
yqextended: XML/TOML support
Shell features not yet implemented, roughly ordered by impact:
| Feature | Category | Notes |
|---|---|---|
Associative arrays (declare -A) |
Variable system | -A attribute parsed but full data structure pending |
shopt (shell options) |
Builtin | e.g. nullglob, extglob, globstar |
trap (signal handling) |
Builtin | Would be simulated in sandbox |
pushd / popd / dirs |
Builtin | Directory stack |
Process substitution (<(cmd)) |
Expansion | Requires /dev/fd emulation |
Extended globbing (?(pat), etc.) |
Expansion | Requires shopt -s extglob |
Globstar (**) |
Expansion | Requires shopt -s globstar |
FD variables ({var}>file) |
Redirection | Bash 4.1+ feature |
hash / enable |
Builtin | Command hash table management |
grep -f / sed -f |
Commands | Pattern/script file loading |
| CI/CD pipeline | Infra | GitHub Actions |
AST to_json() |
Debug | AST serialization |
All binary/codec work is pure MoonBit (zero JS runtime dependencies). Community packages are compile-time only, fully inlined via DCE.
| Package | Used By | Purpose |
|---|---|---|
bobzhang/moonjq |
jq |
Full jq language interpreter |
bobzhang/tar |
tar |
Pure-memory tar archiver |
moonbit-community/piediff |
diff, cmp |
Myers + Patience diff algorithms |
moonbit-community/yaml |
yq |
YAML parser/emitter |
gmlewis/gzip + gmlewis/flate |
gzip, gunzip, zcat |
Pure DEFLATE compression |
gmlewis/base64 |
base64 |
Base64 encode/decode |
gmlewis/md5 |
md5sum |
MD5 hash |
gmlewis/sha1 |
sha1sum |
SHA-1 hash |
shu-kitamura/sha256 |
sha256sum |
SHA-256 hash |
xunyoyo/NyaCSV |
xan, csvlook |
CSV parsing |
moonbitlang/regexp |
grep, sed, awk |
Regular expressions |
justjavac/glob |
VFS glob | Glob pattern matching |
| Metric | Target | Current |
|---|---|---|
| API surface compatibility | 100% drop-in for just-bash@3.0.1 |
🔧 root/browser exports, type/package checks, and executor custom-source slices landed; runtime gaps remain |
| Bundle size (gzip) | <100 KB | TBD |
| Cold start time | <5 ms | TBD |
| Command surface coverage | just-bash@3.0.1 default + optional command groups |
🔧 helper lists aligned; basic JS facade landed |
| Bash behavior accuracy | >95% (comparison tests) | ✅ 100% (523/523) |
| Spec suite integration | Oils bash + awk + sed + grep + jq | ✅ integrated |
| Spec compatibility pass status (2026-02-19) | Full green | 🔧 grep 54 failed, jq 170 failed, bash chunked run 683+ failed confirmed |
| Security test files | Comprehensive | ✅ 27 files integrated |
| Security pass status (2026-02-19) | Full green | 🔧 1 failing attack case + 2 failing fuzz suites |
| Agent workflow tests | Real-world scenarios | ✅ 13 scenarios |
| ReDoS vulnerability | 0 (VM-based regex) | ✅ 0 |
| Zero-day filesystem escapes | 0 (architectural guarantee) | ✅ 0 |
| FFI boundaries | Minimal (4 system primitives) | ✅ 4 (fetch, timer, VM, custom) |