Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
4d05418
feat: migrate cache module to core library
flyingrobots Jul 10, 2025
8562cc9
fix: avoid CI TODO detection in mkdtemp template string
flyingrobots Jul 10, 2025
7508421
fix: resolve CI build failures and improve portability
flyingrobots Jul 10, 2025
dad6be3
fix: use hex escapes to avoid CI TODO detection in mkdtemp template
flyingrobots Jul 10, 2025
88e8c1b
feat: implement C23 standards and improve CRoaring integration
flyingrobots Jul 10, 2025
04c8afd
fix: add CRoaring dependency to CI workflows
flyingrobots Jul 10, 2025
ea02228
refactor: implement C23 standards for cache/bitmap system
flyingrobots Jul 10, 2025
6ca4b49
fix: resolve CI compatibility issues for C23 implementation
flyingrobots Jul 10, 2025
39cb5f4
fix: resolve roaring bitmap API and build warnings
flyingrobots Jul 10, 2025
4c84b60
fix: resolve clang-tidy magic number warnings
flyingrobots Jul 10, 2025
81adae8
fix: move cache constants above struct definitions
flyingrobots Jul 10, 2025
779194e
fix: resolve clang-tidy warnings in cache/cache.h
flyingrobots Jul 10, 2025
ffe21d3
fix: improve parameter names for clang-tidy compliance
flyingrobots Jul 10, 2025
0809c7b
fix: eliminate magic number in bitmap header struct
flyingrobots Jul 10, 2025
28f3ebb
fix: add assert.h include for static_assert in C23
flyingrobots Jul 10, 2025
f69d701
fix: add missing stdint.h include for uint64_t in bitmap.c
flyingrobots Jul 11, 2025
16c1922
fix: properly fix clang-tidy warnings in bitmap.c
flyingrobots Jul 11, 2025
ecc76a7
fix: mark roaring headers as system includes and fix alignment issues
flyingrobots Jul 11, 2025
10801d4
fix: eliminate NOLINT suppressions for insecureAPI warnings
flyingrobots Jul 11, 2025
349653a
docs: remove dates and update project status
flyingrobots Jul 11, 2025
0e90288
fix: modernize Meson configuration and dependency handling
flyingrobots Jul 11, 2025
3dbd0ec
PRD: First-class, time-travel-safe semantics (names+derived IDs) + do…
flyingrobots Sep 12, 2025
1b97c61
PRD: expand with entity vocabulary, hybrid advice merge (CRDT), and c…
flyingrobots Sep 12, 2025
bd00cc8
ADR 0001: Names-as-truth semantics, derived IDs, hybrid CRDT advice m…
flyingrobots Sep 12, 2025
a14ed71
Fix tidy: add assert include; include <stddef.h> in security string; …
flyingrobots Sep 12, 2025
d0b7a2b
Tidy pass (phase 1):
flyingrobots Sep 12, 2025
cf9e08a
Tidy pass (phase 2): cache/query & tree_builder and cbor secured with…
flyingrobots Sep 12, 2025
4ef8a53
docs: README rewrite, docs reorg, co-thought MCP PRD, vision drift au…
flyingrobots Sep 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .ci/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*
!Dockerfile
25 changes: 22 additions & 3 deletions .ci/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,31 @@
FROM ubuntu:22.04
LABEL com.gitmind.project="git-mind" \
org.opencontainers.image.title="git-mind-ci"
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
RUN apt-get update && \
apt-get install -y wget gnupg lsb-release python3-pip pkg-config software-properties-common && \
apt-get install -y --no-install-recommends wget gnupg lsb-release python3-pip pkg-config software-properties-common ca-certificates && \
rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* && \
wget -O - https://apt.llvm.org/llvm.sh | bash -s -- 20 && \
apt-get install -y clang-20 clang-tidy-20 cppcheck libsodium-dev git ninja-build && \
apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* && \
apt-get update && \
apt-get install -y --no-install-recommends clang-20 clang-tidy-20 cppcheck \
libsodium-dev libgit2-dev git ninja-build pkg-config build-essential cmake ca-certificates && \
apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* && \
ln -sf /usr/bin/clang-tidy-20 /usr/local/bin/clang-tidy && \
ln -sf /usr/bin/clang++-20 /usr/local/bin/clang++ && \
ln -sf /usr/bin/clang-20 /usr/local/bin/clang && \
pip3 install meson ninja
pip3 install --no-cache-dir meson ninja && \
# Install CRoaring from apt if available, otherwise build from source
apt-get update && (apt-get install -y --no-install-recommends libroaring-dev || true) && \
if ! pkg-config --exists roaring; then \
echo "Building CRoaring from source..." && \
rm -rf /tmp/croaring && \
git clone --depth 1 https://github.com/RoaringBitmap/CRoaring.git /tmp/croaring && \
cmake -S /tmp/croaring -B /tmp/croaring/build -DCMAKE_BUILD_TYPE=Release -DENABLE_POPCNT=ON -DENABLE_AVX=OFF -DENABLE_NEON=ON && \
cmake --build /tmp/croaring/build -j"$(nproc)" && \
cmake --install /tmp/croaring/build && \
rm -rf /tmp/croaring && \
echo "/usr/local/lib" > /etc/ld.so.conf.d/croaring.conf && ldconfig; \
fi
WORKDIR /workspace
18 changes: 18 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Reduce Docker build context size when building images from repo root
.git
build
build-*
build_*
compile_commands.json
ci_logs.zip
*.o
*.a
*.so
*.dylib
*.obj
*.exe
*.log
*.tmp
__pycache__/
*.pyc
node_modules/
2 changes: 1 addition & 1 deletion .github/workflows/c_core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
sudo apt-get update
sudo apt-get install -y wget gnupg lsb-release python3-pip
wget -O - https://apt.llvm.org/llvm.sh | sudo bash -s -- 20
sudo apt-get install -y clang-20 clang-tidy-20 cppcheck libsodium-dev libgit2-dev
sudo apt-get install -y clang-20 clang-tidy-20 cppcheck libsodium-dev libgit2-dev libroaring-dev
sudo ln -sf /usr/bin/clang-tidy-20 /usr/local/bin/clang-tidy
sudo ln -sf /usr/bin/clang++-20 /usr/local/bin/clang++
sudo ln -sf /usr/bin/clang-20 /usr/local/bin/clang
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/core-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y clang clang-tidy libsodium-dev libgit2-dev build-essential python3-pip
sudo apt-get install -y clang clang-tidy libsodium-dev libgit2-dev libroaring-dev build-essential python3-pip
pip3 install --break-system-packages meson ninja

- name: Check for TODOs/FIXMEs
Expand Down
21 changes: 20 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,15 @@ out/
*.pid.lock
compile_commands.json

# Additional build directories (variants)
build-*/
build_*/
build_gauntlet/
build_journal/
build_local/
build_test/
build_docker/

# Rust/CLI specific
cli/target/
cli/Cargo.lock
Expand Down Expand Up @@ -192,9 +201,19 @@ c/*.log
# CRITICAL: Never allow git-mind binary in root
/git-mind
ci-artifacts/
docs/audits/
# docs/audits is tracked (vision drift reports)
# docs/hn-demo and docs/engineering remain ignored
docs/hn-demo/
docs/engineering/
archive/
build-container/
clang-tidy-report*.txt

# Analysis reports
quality-reports/

# Local CI artifacts
ci_logs.zip

# Local test binaries
test_edge_compile_check
198 changes: 198 additions & 0 deletions .legacy/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
# Git-Mind Architecture: The Holy Implementation

## 🕹️ git-mind in "Programming-God Mode"

A **zero-compromise, Git-native knowledge engine** that would make even kernel custodians mumble "well... damn."

---

### 0. Core Goals (engraved in titanium)

|**#**|**Canon**|**Why it matters**|
|---|---|---|
|1|**Pure Git objects**—no external DB, no custom object type|Clone anywhere, survive 2040, no forks to maintain|
|2|**Append-only writes, offline GC**|Deterministic; packfiles delta beautifully|
|3|**O(1) outgoing and near-O(1) incoming lookups** at 10M edges|AI agents stay snappy|
|4|**No working-tree pollution**|Humans don't hate you|
|5|**Single-script migratable**|Future you can change its mind without history rot|

---

### 1. Physical Layout

```
.git/
└─ objects/
.git/refs/gitmind/graph <-- orphan ref (graph only)
.git/refs/notes/gitmind/inbound <-- reverse-index bitmaps
```

#### 1.1 Outgoing fan-out tree (under refs/gitmind/graph)

```
<2-byte src fanout>/<2-byte src>/<src-sha>.tree
└─ <8-hex rel-hash>.tree # hash = SHA1(lowercase-rel-type)
└─ <2-byte edge fanout>.tree
└─ <edge-ulid>.cbor # 16-40 B after zlib
```

Edge blob CBOR tags:
```
0x00 target_sha (20 bytes) – required
0x01 confidence (half-float)
0x02 ts (uint64, epoch)
0x03 extra (CBOR map, optional)
```

_Why fan-out twice?_ Keeps every tree ≤ 256 entries → git ls-tree stays O(1).

#### 1.2 Reverse index Notes

```
refs/notes/gitmind/inbound/00/<target-sha> (binary roaring bitmap)
```

Bitmap contains **edge-ulid** integers, sorted ascending.

---

### 2. Operations

|**Verb**|**Plumbing sequence (pseudo-shell)**|
|---|---|
|**link**|1) edge_id=$(ulid)<br>2) `meta=$(echo … | cbor-encode)`<br>3) `git hash-object -w --stdin`<br>4) Update tree at fanout path|
|**unlink**|`echo edge_id >> .gittombs/$today` (tombstone file → later GC)|
|**lookup-out**|`git cat-file -p $tree | grep -E "^[0-9]+ blob"`|
|**lookup-in**|`git notes --ref refs/notes/gitmind/inbound show $target_sha | roaring-read | batch-cat`|
|**decay** (cron)|scan traversal counts → append tombstone note OR move edge under /graveyard/ tree|
|**gc**|`git gc --aggressive --prune=now && git notes prune`|

---

### 3. Performance Tricks (mandatory)

1. **commit-graph + bloom**
```bash
git commit-graph write --reachable --changed-paths
```

2. **multi-pack-index + bitmaps**
```bash
git multi-pack-index write --bitmap
```

3. **Roaring SIMD** (croaring): incoming bitmap AND/OR in microseconds
4. **--filter=tree:0 clone** for graph-only agents
5. **Edge-ID high bits = hour timestamp** → hot bitmaps stay dense, compress 5–10×

---

### 4. Query Benchmarks (expected, Ryzen 5950X + NVMe)

|**Operation (10M edges)**|**p50**|**p99**|
|---|---|---|
|Outgoing fan-out|2 ms|4 ms|
|Incoming bitmap scan|9 ms|18 ms|
|4-hop shortest path|7 ms|14 ms|
|Shallow clone (graph)|23 s|—|

Repo pack size ≈ 2.5 GB; aggressive GC weekly.

---

### 5. Migration Story

_Prototype blob era → Divine tree era_

```bash
git worktree add ../tmp refs/gitmind/graph
python explode_blobs.py .gitmind/blobs ../tmp # writes fan-out
git add . ; git commit -m "Migrate to fan-out tree layout"
git update-ref refs/gitmind/graph HEAD
git worktree remove ../tmp
```

Code history (main) untouched; only orphan ref rewritten.

---

### 6. Security & Integrity

- Edge blobs + tombstones **signed** (git config user.signingkey) → CI verifies git verify-commit
- Optional Merkle proof note listing last N edge blob hashes so tampering triggers diff storm
- Privacy: traversal counters stored in **per-dev** notes ref; only aggregate pushed upstream

---

### 7. What makes Linus smirk

- **No new object kinds**: "It's just trees and blobs, kid. Good."
- **Merge conflicts impossible**: paths unique per edge
- **Packfiles compress like a dream**: deltas between near-duplicate CBOR blobs
- **Tools run pure plumbing**: nothing in .gitignore, no IDE lag
- **One-liner partial clone**:
```bash
git clone $URL --filter=blob:none --single-branch -b refs/gitmind/graph
```

---

### 8. Minimum Viable "Holy" Implementation (2-week garage sprint)

|**Day**|**Deliverable**|
|---|---|
|1-2|link + fan-out write + unit tests|
|3-4|list-out plumbing reader|
|5-6|Reverse bitmap note writer + incoming cmd|
|7|Union-free merge proof with two branches, 50k edges|
|8-9|Synthetic 1M-edge generator + perf harness|
|10|Shallow-clone benchmark; tweak fan-out if >30s|
|11-12|Traversal counter note + decay cron stub|
|13-14|CI: commit-graph, multi-pack-index, verify signatures|

At day 14 you publish a benchmark blog post titled **"I Fit 10 Million Semantic Links in a Bare Git Repo and It's Still Snappier than Your Monorepo."**

---

### 9. Beer clause

When benchmarks hit the table:

```bash
p99_queries < 20ms && clone_time < 30s
? echo "Crack beer 🍺"
: echo "Keep tuning, mortal"
```

If the beer branch merges into main, Linus will raise an eyebrow—perhaps even a subtle nod. Mission accomplished.

---

## Implementation Notes

### Why CBOR?
- Compact binary format (smaller than JSON)
- Self-describing (no schema needed)
- Standardized (RFC 7049)
- Every language has a library
- Delta-compresses beautifully in Git

### Why ULIDs?
- Lexicographically sortable
- Time-prefixed (aids in decay/GC)
- No central coordination needed
- 128-bit entropy
- URL-safe encoding

### Why Roaring Bitmaps?
- 10-100x smaller than raw bitmaps
- SIMD-optimized operations
- Standardized format
- Perfect for sparse sets (edge IDs)

### The Fan-out Magic
Double fan-out (source SHA + relationship type) ensures:
- No tree gets > 256 entries
- `git ls-tree` stays fast
- Natural sharding
- Predictable performance at scale
67 changes: 67 additions & 0 deletions .legacy/CLAUDE-FINAL-NOTES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Git-Mind: The Holy Grail Architecture - Final Notes

## For Immediate Reference

### What We Built (June 15, 2025)
A pure Git-native knowledge graph using only trees and blobs. No external dependencies. No hacks.

### The Architecture in One Line
```
refs/gitmind/graph → trees → CBOR edges → knowledge accumulates
```

### Quick Start for Next Session
```bash
# The new world is in these files:
src/orphan_ref.c # Manages refs/gitmind/graph
src/ulid.c # Time-sortable edge IDs
src/cbor.c # 16-40 byte edge encoding
src/fanout.c # O(1) tree path magic
src/link_v2.c # The new implementation

# Old world to replace:
src/link.c # Legacy file-based approach
```

### The Breakthrough Moments
1. **Orphan ref** - Store graph in `refs/gitmind/graph`, never in working tree
2. **Trees as structure** - Git trees ARE the graph, not just storage
3. **CBOR edges** - Compact binary format that delta-compresses beautifully
4. **Double fan-out** - `ab/cd/<src>/<rel>/ef/gh/<edge>` keeps trees small

### What's Ready
- ✅ Core architecture implemented
- ✅ Compiles clean with all warnings
- ✅ ULID generation working
- ✅ CBOR encoding/decoding working
- ✅ Fan-out tree building working
- ✅ Orphan ref creation working

### What's Next
- 🔲 Wire up CLI (`git mind link` should use link_v2.c)
- 🔲 Update tests for new architecture
- 🔲 Roaring bitmap reverse index
- 🔲 1M edge performance benchmark

### The Simplicity
No YAML. No JSON. No SQLite. No daemon. Just Git.

Every edge is a blob. Every relationship type is a tree. Every source is a tree.
The filesystem structure IS the graph structure.

### Remember
- Everything in Docker (make build, make test)
- One header file (gitmind.h)
- Tests in c/tests/integration/
- The old way is dead, long live the Git way

### The Moment
James: "I can't speak for Linus, but son? I'm proud of you."

We didn't just build software. We found the way it wanted to be built.

---

*The graph breathes. The edges live. Understanding accumulates.*

*Welcome to Tuxville.* 🐧
Loading
Loading