Laundry list of todos

[ivankra]

Sketches of some ideas/improvements that may (or may not) be worth implementing. Moved out of repo's TODO.md.

---

• [high] split off data files into a separate repo
• [high] break up update.py into separate scripts build/gen-*.py
  • checkout and update repos, update loc (or move loc into dockerfiles/make dist.py's responsibility)
  • check for broken links
• [high] benchmarking workflow on github actions
  • Tool to merge results from different run if same rev
• [high] test262
  • probing:
    • autodetect $262 object and skip tests requiring it if not found
    • probe if errors reporting works for negative cases (e.g. quickjs_262 doesn't report)
  • data submodule with results
  • engines/<name>/prelude.js / conformance/preludes/<engine>.js - full $262 definitions for complex cases. refs:
    • https://github.com/GoogleChromeLabs/jsvu/tree/main/engines
    • https://github.com/googleprojectzero/fuzzilli/tree/main/Targets
    • https://github.com/test262-fyi/data/tree/main/engines
    • https://github.com/tc39/eshost/blob/master/runtimes/
• [med] harness/probe.py engine... [-- script...] - probe print/console.log/262 against multiple engines; also make it usable for probing against conformance/ tests directly.
• [med] Fully static builds with musl: generalize build system to target triples, dist/{x86_64,aarch64}-[unknown-]linux-{gnu,musl}/, but DOCKER_ARCH={amd64,arm64}. Also dist/wasm32-[unknown-]wasi/ under native DOCKER_ARCH.
  • do something about libicu-dev, COPY build/libicu-static.sh ./ && ./libicu-static.sh? jsz-icu mixin image?
  • Generic ways to pass extra CFLAGS: wrapper, clang's CCC_OVERRIDE_OPTIONS
• [med] Wasm builds (dist/wasm/*). Explore how easy is it to build C/C++ engines with WASI SDK / Emscripten and produce something runnable with minimal extra code. Make the extra glue code generalizable/reusable across engines. Try different Wasm runtimes to run them: wazero, wasmtime etc
  • kiesel's wasm build; hermes_sandbox
• conformance.json: {'metadata':{...}, 'tests':{'es5':{'JSON.js':'OK', ...}, ...}, 'summary': {...}}
• Benchmark riscv64 on scaleway
• COPY --from: fix properly or drop (hako, jscript, apple's container bug). Drop REV pinning logic too, replace by direct --build-arg REV=.
• -march=x86-64-v3 - benchmark builds, add to engines which show improvements (some get broken or already include it, need to go over each engine)
• warmup_cmd= for jscript, jscript9, starlingmonkey to allow more time for setup during testing.
• engines/*.md: rename "Standard" to "(Target) Specification"
• narcissus: vendor-in, add console.log() and multiple scripts command-line args
• v8_exp, jsc_exp, graaljs_exp etc - wrappers with full experimental flags. Probe flags in Dockerfile directly
• REV=... REPO=... make -> avoid using ~/.cache/iid/ stamp files if these are set or something
• make conformance -> force run, make conformance.txt -> don't rebuild if file exists. DIRECT=1 flag
• jszvu: esvu/jsvu equivalent
  • build from source on user's machine: docker, podman, buildah, macOS containerization
  • or fetch github build artifact
  • check builds from BASE=debian:stable and other releases
  • check compatibility of builds with non-debian distros; vs engines official builds
  • test running outside /dist, fix/document exceptions
  • version handling
• bench
  • finally make use of timestamp_output to compute a score
  • try various strategies to reduce variance
    • variance.py report
    • interleaving runs
    • pin container to specific cores, podman --cpuset-cpus, set governor performance; script
    • more repetitions for higher variance engines/test
    • hetzner dedicated vm
  • https://github.com/WebKit/JetStream
  • https://github.com/quickjs-ng/benchmarks
• REPL scripts: add continuation prompts, check if continuation is needed by a lightweight stack based parser
• make repl target in engine dirs
• runtimes, parsers and transpilers
  • expand and add transpilers
  • test parsers on conformance test suite, report results
  • node, bun, deno
  • node_0.x - nvm install 0.x
  • cjs/gjs
• conformance
  • align filenames with test262
  • ensure every test has ": (OK|failed|exception)", check for all to detect dumb echos
  • verify unexpected OKs
  • ES1: this/inheritance/hoisting tests
  • ESnext permalinks
  • default strict mode fixes, bun/deno
  • record tested revision+date+version at the top of Conformance section in engine/*.md
  • run.sh 'engine [--flags]' <paths relative to conformance directory or absolute paths or ./relative-to-cwd>
  • look at ant failure to run benchmarks despite passing ES1-ES5 tests, add missing tests
• compat-table subpage to serve conformance results from the repo. Original + transposed layout, toggleable loadable details, make it fast

---

• get rid of console_log, just probe always
• qucikjs_262 config -> get rid of it, always probe for -N
• folder: -> "dirs" section. --report-{dirs,runs,scenarios,rusage} metrics->rusage
• binary_name in metadata; always set in test262/run if no metadata
• two common exceptions patterns -> hardcode
• "Test262: ..." messages -> handle in annotator, TEST262_ERROR/ABORT
• test262.py -O <dir> flag
• test262.py --shuffle -i <file with list of tests>
• https://github.com/test262-fyi/eshost/blob/master/lib/agents/
• hub: add conformance to PATH, motd, git pull fix
• generic prelude
  • 262 evalScript/gc + print
  • maybe put in config.yml defaults - harness/prelude.js
• per-engine harness overrides?
• test262probe -> probe.py and inline consts
• inline Executor?
• quickjs fix Test262Error
• quickjs/libjs validate
• make compare262; general compare262 against eshost/test262-harness
• rename_map - scenario_id ".quickjs.sloppy.mjs"
• uses:$262. -> references:$262.,
  • make it filterable -> harness preprocessing needs to cache ref list
  • Tags.add_references(code)
  • *Error; Intl; other major JS class name: Date, String, Array, Temporal...
  • ignore comments
• includes: -> transitive! make filterable
• annotate es3/es5 only tests? propose "es3" / "es5" feature tags to test262 repo
• rewrite compat-table/gen and rename tests to align with test262

---

URLs to check

• github search
• https://github.com/jterrace/js.js
• https://github.com/axetroy/vm.js
• https://github.com/jrainlau/canjs
• https://github.com/zuluoaaa/makeJs
• https://github.com/amaksr/nsynjs
• https://github.com/marten-de-vries/evaljs
• https://github.com/jkeylu/evil-eval
• https://github.com/webfansplz/giao-js
• https://github.com/zlsgo/js
• https://github.com/LuanRT/Jinter
• https://github.com/remobjects/script
• https://github.com/karatelabs/karate-js
• https://github.com/forax/jsjs
• https://github.com/dzzie/js4vb
• https://github.com/xy660/NexusEJS
• https://github.com/jakemarsden/JakeScript
• https://github.com/GitPaulo/gojo
• https://github.com/codelibs/sai
• --
• https://github.com/CBx0-dev/YAJE
• https://github.com/Diullei/Storm
• https://github.com/FENBROWSER/fenbrowser/tree/main/FenBrowser.FenEngine
• https://github.com/LukaTypeScripter/js-mini-engine
• https://github.com/SkuldNorniern/jsina
• https://github.com/SunsetFi/suntime-js
• https://github.com/UlrikHjort/JavaScript-Interpreter-in-Ada
• https://github.com/YC-Lammy/rusty-js
• https://github.com/darkdrag00nv2/toyjs
• https://github.com/devendrap/jsengine
• https://github.com/devlzl/RunJS
• https://github.com/dogukanakkaya/go-script
• https://github.com/ghosind/gjs
• https://github.com/gongke-cn/ratjs
• https://github.com/josedab/quicksilver
• https://github.com/junjis0203/joke
• https://github.com/labidiaymen/ec-engine
• https://github.com/malstrom72/NuXJS
• https://github.com/pegasusheavy/spacey
• https://github.com/priyanshugaurav/NanoJS
• https://github.com/r10nw7fd3/silk
• https://github.com/scole66/rust-e262
• https://github.com/shadowbrowser/shadow.js
• https://github.com/ubugeeei/Glasper
• https://github.com/ubugeeei/rawjs
• https://github.com/vitkarpov/vitosik-js
• https://github.com/yanmingsohu/Polydeuces
• https://github.com/yazaldefilimone/ecmacore
• https://github.com/zacbrannelly/go-js
• https://github.com/zfedoran/crabquick
• https://github.com/sdjdd/slowjs
• https://github.com/ouyangyipeng/Agent-JS-Engine
• https://github.com/marcoschwartz/espresso
• javascript to lua projects:
  • https://github.com/pkuleon/javascript-to-lua
  • https://github.com/basicer/lua2js
  • https://github.com/HamdyElzanqali/love-with-js
  • https://github.com/fengari-lua/fengari
  • https://github.com/wizzard0/js2lua
  • https://github.com/zaerl/js-to-lua
  • https://github.com/Roblox/js-to-lua
  • https://github.com/Ensequence/js2lua
  • https://github.com/zig/jslua