From 39a11f7cd052d8369d176e2ef5aa36f89d104b9c Mon Sep 17 00:00:00 2001 From: Matthew Black Date: Sun, 1 Jun 2025 21:46:37 -0400 Subject: [PATCH] docs: remove migration doc files & fix node-ln docs --- circular-dependency-resolution.md | 57 ------------ node-lightning-migration-summary.md | 73 --------------- packages/bitcoind/README.md | 2 +- packages/bufio/README.md | 52 +++++------ packages/checksum/README.md | 6 +- packages/crypto/README.md | 16 ++-- packages/noise/README.md | 18 ++-- packages/wire/README.md | 17 +--- rocksdb-to-classic-level-migration.md | 129 -------------------------- 9 files changed, 51 insertions(+), 319 deletions(-) delete mode 100644 circular-dependency-resolution.md delete mode 100644 node-lightning-migration-summary.md delete mode 100644 rocksdb-to-classic-level-migration.md diff --git a/circular-dependency-resolution.md b/circular-dependency-resolution.md deleted file mode 100644 index f3572e92..00000000 --- a/circular-dependency-resolution.md +++ /dev/null @@ -1,57 +0,0 @@ -# Circular Dependency Resolution - -## Problem -There was a circular dependency between packages: -- `@node-dlc/core` → `@node-dlc/messaging` → `@node-dlc/wire` → `@node-dlc/core` - -This prevented the packages from building successfully with `yarn bootstrap`. - -## Root Cause -The issue was that node-lightning's core utilities (BitField, ShortChannelId, ChannelId, Base32, etc.) were placed in the core package alongside DLC/finance-specific functionality. The wire package needed these basic utilities, creating the circular dependency. - -## Solution -Created a new `@node-dlc/common` package to hold shared utilities that don't have dependencies on other packages: - -### 1. Created @node-dlc/common package -- Moved `BitField`, `BigIntUtils`, `Base32`, `ShortChannelId`, `ShortChannelIdUtils`, and `ChannelId` from core to common -- These are fundamental utilities used by multiple packages - -### 2. Updated Dependencies -- `@node-dlc/wire`: Now depends on `@node-dlc/common` (instead of core) and `@node-dlc/bitcoin` -- `@node-dlc/core`: Now depends on `@node-dlc/common` and re-exports its types for backward compatibility -- `@node-dlc/messaging`: Now depends on `@node-dlc/common` for BitField - -### 3. Import Updates -Fixed imports across all packages: -- Common types (BitField, ShortChannelId, ChannelId, Base32) → from `@node-dlc/common` -- Bitcoin types (Value, Script, Tx, OutPoint, HashValue, etc.) → from `@node-dlc/bitcoin` -- Kept re-exports in core for backward compatibility - -### 4. Build Order -Established clear dependency hierarchy: -1. `bufio` (no dependencies) -2. `crypto`, `checksum`, `logger` (only depend on bufio) -3. `bitcoin` (depends on bufio, crypto) -4. `common` (depends on bufio, bitcoin) -5. `noise` (depends on bufio, crypto) -6. `bitcoind` (no internal dependencies) -7. `chainmon` (depends on bitcoin, bitcoind, logger) -8. `wire` (depends on bufio, checksum, common, bitcoin, crypto, logger, noise) -9. `core` (depends on bitcoin, bufio, common, crypto) -10. `messaging` (depends on bitcoin, bufio, checksum, common, crypto, wire) -11. `transport`, `rocksdb` (depend on core, messaging, and others) - -## Result -All packages now build successfully without circular dependencies. The separation of concerns is clearer: -- `@node-dlc/common`: Shared utilities and types -- `@node-dlc/core`: DLC/finance-specific core functionality -- `@node-dlc/bitcoin`: Bitcoin-specific types and utilities - -## Manual Symlink Creation -Due to lerna issues, packages were manually symlinked: -```bash -cd packages//node_modules/@node-dlc -ln -s ../../../ -``` - -This approach successfully resolved the circular dependencies while maintaining backward compatibility. \ No newline at end of file diff --git a/node-lightning-migration-summary.md b/node-lightning-migration-summary.md deleted file mode 100644 index 25e0b058..00000000 --- a/node-lightning-migration-summary.md +++ /dev/null @@ -1,73 +0,0 @@ -# Node-Lightning to Node-DLC Migration Summary - -## Overview -Successfully migrated all `@node-lightning` dependencies into the `node-dlc` project, eliminating external dependencies on the deprecated `node-lightning` project. - -## Migration Date -December 2024 - -## What Was Done - -### 1. Packages Copied from Node-Lightning -The following packages were copied directly from `node-lightning` to `node-dlc`: -- `bitcoind` - Bitcoin Core RPC and ZMQ client -- `bufio` - Buffer I/O utilities -- `checksum` - Checksum utilities (CRC32C) -- `crypto` - Cryptographic functions -- `wire` - Lightning Network wire protocol -- `noise` - Noise protocol implementation - -### 2. Packages Merged -The following packages existed in both projects and were merged: -- `bitcoin` - Added missing functionality from node-lightning (Base58, Script, Tx, etc.) -- `core` - Merged node-lightning core utilities while preserving node-dlc specific functionality -- `logger` - Replaced with node-lightning version which had more features -- `chainmon` - Added missing BlockDiffer and BlockDiffResult functionality - -### 3. Package Updates -- Updated all package names from `@node-lightning/*` to `@node-dlc/*` -- Updated all package.json dependencies to use `@node-dlc/*` packages -- Updated all TypeScript imports throughout the codebase -- Fixed naming conflicts (e.g., ChannelId exported as LightningChannelId) - -### 4. Version Alignment -- Maintained existing node-dlc package versions (0.23.6) -- New packages from node-lightning kept their functionality but adopted node-dlc naming - -## Technical Details - -### Dependencies Updated -All references to `@node-lightning/*` packages have been replaced with `@node-dlc/*`: -- `@node-lightning/bitcoin` → `@node-dlc/bitcoin` -- `@node-lightning/bitcoind` → `@node-dlc/bitcoind` -- `@node-lightning/bufio` → `@node-dlc/bufio` -- `@node-lightning/chainmon` → `@node-dlc/chainmon` -- `@node-lightning/checksum` → `@node-dlc/checksum` -- `@node-lightning/core` → `@node-dlc/core` -- `@node-lightning/crypto` → `@node-dlc/crypto` -- `@node-lightning/logger` → `@node-dlc/logger` -- `@node-lightning/noise` → `@node-dlc/noise` -- `@node-lightning/wire` → `@node-dlc/wire` - -### Files Modified -- All TypeScript source files (*.ts) updated to use @node-dlc imports -- All package.json files updated to use @node-dlc dependencies -- README.md updated to reflect the absorption of node-lightning functionality - -## Benefits Achieved -1. **No External Dependencies**: node-dlc is now self-contained -2. **Maintainability**: Full control over all code without relying on deprecated projects -3. **Consistency**: All packages follow node-dlc naming and versioning conventions -4. **Future-proof**: Can evolve independently without breaking changes from external dependencies - -## Next Steps -1. Run `yarn bootstrap` to install dependencies and build all packages -2. Run `yarn test` to ensure all tests pass -3. Update any documentation that references node-lightning -4. Consider publishing the new packages to npm under @node-dlc scope - -## Notes -- The migration preserves all existing functionality -- No breaking changes to the API -- The project structure remains the same -- All tests should continue to pass as before \ No newline at end of file diff --git a/packages/bitcoind/README.md b/packages/bitcoind/README.md index 9e049af5..a1ef5669 100644 --- a/packages/bitcoind/README.md +++ b/packages/bitcoind/README.md @@ -1,4 +1,4 @@ -# @node-lightning/bitcoind +# @node-dlc/bitcoind This package provides connectivity to a bitcoind node by enabling RPC functions and Zeromq streaming. diff --git a/packages/bufio/README.md b/packages/bufio/README.md index 8afc837d..4f6b4a8d 100644 --- a/packages/bufio/README.md +++ b/packages/bufio/README.md @@ -1,16 +1,16 @@ # Utilities for Buffer IO -[![CircleCI](https://circleci.com/gh/altangent/node-lightning/tree/master.svg?style=shield)](https://circleci.com/gh/altangent/node-lightning/tree/master) -[![codecov](https://codecov.io/gh/altangent/node-lightning/branch/master/graph/badge.svg)](https://codecov.io/gh/altangent/node-lightning) +[![CircleCI](https://circleci.com/gh/atomicfinance/node-dlc/tree/master.svg?style=shield)](https://circleci.com/gh/atomicfinance/node-dlc/tree/master) +[![codecov](https://codecov.io/gh/atomicfinance/node-dlc/branch/master/graph/badge.svg)](https://codecov.io/gh/atomicfinance/node-dlc) Usage: ```bash -npm install @node-lightning/bufio +npm install @node-dlc/bufio ``` ```javascript -const { BufferReader, BufferWriter } = require("@node-lightning/bufio"); +const { BufferReader, BufferWriter } = require("@node-dlc/bufio"); // write arbitrary length data const writer = new BufferWriter(); @@ -32,21 +32,21 @@ console.log(buffer.readBytes()); This class wraps an existing Buffer and allows reading data from the Buffer. This class includes -- `position: number` - the current buffer cursor position -- `eof: bool` - whether the current position is at the end of the buffer -- `buffer: Buffer` - the underlying buffer -- `lastReadBytes` - number of bytes last read from the buffer +- `position: number` - the current buffer cursor position +- `eof: bool` - whether the current position is at the end of the buffer +- `buffer: Buffer` - the underlying buffer +- `lastReadBytes` - number of bytes last read from the buffer -- `readUInt8(): number` - reads 1 byte -- `readUInt16LE(): number` - reads 2 bytes as 16-bit little-endian unsigned integer -- `readUInt16BE(): number` - reads 2 bytes as 16-bit big-endian unsigned integer -- `readUInt32LE(): number` - reads 4 bytes as 32-bit little-endian unsigned integer -- `readUInt32BE(): number` - reads 4 bytes as 32-bit big-endian unsigned integer -- `readUInt64LE(): bigint` - reads 8 bytes as 64-bit little-endian unsigned integer -- `readUInt64BE(): bigint` - reads 8 bytes as 64-bit big-endian unsigned integer -- `readVarUint(): bigint` - reads a variable length integer as defined in the Bitcoin protocol [docs](https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer) -- `readBigSize(): bigint` - reads a variable length integer as defined in the Lightning Network -- `readBytes([len]): Buffer` - reads the optionally specified number of bytes. If no length is supplied, it reads to the end of the buffer. +- `readUInt8(): number` - reads 1 byte +- `readUInt16LE(): number` - reads 2 bytes as 16-bit little-endian unsigned integer +- `readUInt16BE(): number` - reads 2 bytes as 16-bit big-endian unsigned integer +- `readUInt32LE(): number` - reads 4 bytes as 32-bit little-endian unsigned integer +- `readUInt32BE(): number` - reads 4 bytes as 32-bit big-endian unsigned integer +- `readUInt64LE(): bigint` - reads 8 bytes as 64-bit little-endian unsigned integer +- `readUInt64BE(): bigint` - reads 8 bytes as 64-bit big-endian unsigned integer +- `readVarUint(): bigint` - reads a variable length integer as defined in the Bitcoin protocol [docs](https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer) +- `readBigSize(): bigint` - reads a variable length integer as defined in the Lightning Network +- `readBytes([len]): Buffer` - reads the optionally specified number of bytes. If no length is supplied, it reads to the end of the buffer. ## BufferWriter @@ -58,11 +58,11 @@ This class can intenrally manage a Buffer and will automatically expand to fit the data footprint required. At the end of writing, `.toBuffer` must be called to obtain the underlying Buffer. -- `writeUInt8(number): undefined` - writes 1 byte -- `writeUInt16LE(number): undefined` - writes a number as a 16-bit little-endian unsigned integer -- `writeUInt16BE(number): undefined` - writes a number as a 16-bit big-endian unsigned integer -- `writeUInt32LE(number): undefined` - writes a number as a 32-bit little-endian unsigned integer -- `writeUInt32BE(number): undefined` - writes a number as a 32-bit big-endian unsigned integer -- `writeUInt64LE(number): undefined` - writes a number as a 64-bit little-endian unsigned integer -- `writeUInt64BE(number): undefined` - writes a number as a 64-bit big-endian unsigned integer -- `writeBytes(buffer): undefined` - writes the supplied buffer to the cursor buffer +- `writeUInt8(number): undefined` - writes 1 byte +- `writeUInt16LE(number): undefined` - writes a number as a 16-bit little-endian unsigned integer +- `writeUInt16BE(number): undefined` - writes a number as a 16-bit big-endian unsigned integer +- `writeUInt32LE(number): undefined` - writes a number as a 32-bit little-endian unsigned integer +- `writeUInt32BE(number): undefined` - writes a number as a 32-bit big-endian unsigned integer +- `writeUInt64LE(number): undefined` - writes a number as a 64-bit little-endian unsigned integer +- `writeUInt64BE(number): undefined` - writes a number as a 64-bit big-endian unsigned integer +- `writeBytes(buffer): undefined` - writes the supplied buffer to the cursor buffer diff --git a/packages/checksum/README.md b/packages/checksum/README.md index 5dcb3bfb..325fa304 100644 --- a/packages/checksum/README.md +++ b/packages/checksum/README.md @@ -1,10 +1,10 @@ -# `@node-lightning/checksum` +# `@node-dlc/checksum` Calculates a CRC32C checksum based on RFC3720. ## Usage -``` -const { crc32c } = require('@node-lightning/checksum'); +```typescript +const { crc32c } = require('@node-dlc/checksum'); const checksum = crc32c(Buffer.from("hello")); // 2591144780 ``` diff --git a/packages/crypto/README.md b/packages/crypto/README.md index 58832f7f..3229b2f5 100644 --- a/packages/crypto/README.md +++ b/packages/crypto/README.md @@ -1,13 +1,13 @@ -# @node-lightning/crypto +# @node-dlc/crypto This packages uses the native Node.js crypto for all functionality except secp256k1. This package requires Node.js 10.17+. This package provides cryptographic functions for use in LN Tools: -- sha256 -- hash160 -- hkdf -- secp256k1 ECDH -- secp256k1 private key generation -- secp256k1 public key derivation -- chacha20-poly1305 AEAD +- sha256 +- hash160 +- hkdf +- secp256k1 ECDH +- secp256k1 private key generation +- secp256k1 public key derivation +- chacha20-poly1305 AEAD diff --git a/packages/noise/README.md b/packages/noise/README.md index 58f9dc01..7ef5ff04 100644 --- a/packages/noise/README.md +++ b/packages/noise/README.md @@ -1,7 +1,7 @@ # Lightning Network Noise Protocol Socket (BOLT #8) -[![CircleCI](https://circleci.com/gh/altangent/node-lightning/tree/master.svg?style=shield)](https://circleci.com/gh/altangent/node-lightning/tree/master) -[![codecov](https://codecov.io/gh/altangent/node-lightning/branch/master/graph/badge.svg)](https://codecov.io/gh/altangent/node-lightning) +[![CircleCI](https://circleci.com/gh/atomicfinance/node-dlc/tree/master.svg?style=shield)](https://circleci.com/gh/atomicfinance/node-dlc/tree/master) +[![codecov](https://codecov.io/gh/atomicfinance/node-dlc/branch/master/graph/badge.svg)](https://codecov.io/gh/atomicfinance/node-dlc) Implements a Noise Protocol TCP Socket and Server in Node.js as defined in Lightning Network BOLT #8 for Encrypted and Authenticated Transport. @@ -13,12 +13,12 @@ Implements a Noise Protocol TCP Socket and Server in Node.js as defined in Light Learn more about the Noise Protocol and the Lighting Network version: -- [Noise Protocol](http://noiseprotocol.org/) -- [Lightning Network BOLT #8](https://github.com/lightningnetwork/lightning-rfc/blob/master/08-transport.md) +- [Noise Protocol](http://noiseprotocol.org/) +- [Lightning Network BOLT #8](https://github.com/lightningnetwork/lightning-rfc/blob/master/08-transport.md) ## Requirements -- Node.js 10.17+ +- Node.js 18+ ## Usage @@ -28,13 +28,13 @@ This example shows how to create a `NoiseSocket` that connects to a `NoiseServer To create a `NoiseSocket`, you must provide: -- local private key `ls` which is a `Buffer` with 32-bytes -- remote node's compressed public key `rp` which is a `Buffer` with 33-bytes. +- local private key `ls` which is a `Buffer` with 32-bytes +- remote node's compressed public key `rp` which is a `Buffer` with 33-bytes. These two values are required to create the encrypted and authenticated communication channel. ```javascript -const noise = require("@node-lightning/noise"); +const noise = require("@node-dlc/noise"); // ls is private key as a Buffer(32) defining a point on elliptic // curve secp256k1 @@ -76,7 +76,7 @@ socket.on("data", buf => { ### Creating a NoiseSocket Server: ```javascript -const noise = require('@node-lightning/noise'); +const noise = require('@node-dlc/noise'); // ls is private key as a Buffer(32) defining a point on elliptic // curve secp256k1 diff --git a/packages/wire/README.md b/packages/wire/README.md index f03b53b2..5669bcc6 100644 --- a/packages/wire/README.md +++ b/packages/wire/README.md @@ -1,19 +1,10 @@ -# `@node-lightning/wire` +# `@node-dlc/wire` This repository contains the wire protocol code for the Lightning Network and includes the following functionality: -- Peer which manages a connection to a peer. This class allows sending and - emitting message traffic. It also manages internal state for Ping/Pong message - traffic. [/lib/peer.ts](lib/peer.ts) +- Peer which manages a connection to a peer. This class allows sending and emitting message traffic. It also manages internal state for Ping/Pong message traffic. [/lib/peer.ts](lib/peer.ts) -- Messages defined in BOLT #1 - Base Protocol, BOLT #2 - Peer Protocol for - Channel Management, and BOLT #7 P2P Node and Channel Discovery can be - found in [/lib/messages](/lib/messages). This code includes the message - types and serialization and deserialization methods for each message. +- Messages defined in BOLT #1 - Base Protocol, BOLT #2 - Peer Protocol for Channel Management, and BOLT #7 P2P Node and Channel Discovery can be found in [/lib/messages](/lib/messages). This code includes the message types and serialization and deserialization methods for each message. -- P2P Node and Channel gossip management that can be found in - [/lib/gossip](/lib/gossip). This code contains the - [GossipManger](/lib/gossip/gossip-manager) which controls gossip for many - peers. Gossip synchronization for a single peer is managed through - [PeerGossipSynchronizer](/lib/gossip/peer-gossip-synchronizer). +- P2P Node and Channel gossip management that can be found in [/lib/gossip](/lib/gossip). This code contains the [GossipManger](/lib/gossip/gossip-manager) which controls gossip for many peers. Gossip synchronization for a single peer is managed through [PeerGossipSynchronizer](/lib/gossip/peer-gossip-synchronizer). diff --git a/rocksdb-to-classic-level-migration.md b/rocksdb-to-classic-level-migration.md deleted file mode 100644 index 85817b5c..00000000 --- a/rocksdb-to-classic-level-migration.md +++ /dev/null @@ -1,129 +0,0 @@ -# RocksDB to Classic-Level Migration Summary - -## Overview -Successfully migrated the `@node-dlc/rocksdb` package from using the deprecated `rocksdb` package to `classic-level`, maintaining the same API interface. - -## Migration Context -- **Original Issue**: The `rocksdb` package is deprecated and no longer maintained -- **Solution**: Migrate to `classic-level`, which is part of the official Level ecosystem and actively maintained -- **Important Context**: `node-lightning` is deprecated and being absorbed into `node-dlc` - -## Key Changes Made - -### 1. Dependency Updates - -#### packages/rocksdb/package.json -- **Removed**: - - `rocksdb` (5.2.1) - - `@node-lightning/gossip-rocksdb` -- **Added**: - - `classic-level` (^3.0.0) -- **Updated**: Node.js requirement from v14 to v18 (required by classic-level) - -### 2. Code Changes - -#### packages/rocksdb/lib/rocksdb-base.ts -```typescript -// Before: -import level from 'levelup'; -import rocksdb from 'rocksdb'; - -export abstract class RocksdbBase { - protected _db: level.LevelUp>; - - constructor(path: string) { - this._db = level(rocksdb(path)); - } -} - -// After: -import { ClassicLevel } from 'classic-level'; - -export abstract class RocksdbBase { - protected _db: ClassicLevel; - - constructor(path: string) { - this._db = new ClassicLevel(path, { - keyEncoding: 'buffer', - valueEncoding: 'buffer', - }); - } -} -``` - -#### All Store Files -- Changed imports from `@node-lightning/gossip-rocksdb` to local `./rocksdb-base` -- Replaced all `createReadStream()` calls with async iterator pattern: - ```typescript - // Before: - const stream = this._db.createReadStream(); - stream.on('data', (data) => { /* ... */ }); - - // After: - const iterator = this._db.iterator(); - try { - for await (const [key, value] of iterator) { - // process data - } - } finally { - await iterator.close(); - } - ``` - -### 3. TypeScript Configuration Updates - -#### Root tsconfig.json -- Added `skipLibCheck: true` to bypass type checking issues in deprecated `node-lightning` packages - -#### Root package.json -- Updated TypeScript from 4.2.3 to 4.9.5 -- Updated @types/node from 16.10.3 to 18.11.9 (in all packages) - -## Important Discovery -- RocksDB databases use `.sst` files (RocksDB format) -- Classic-Level uses `.ldb` files (LevelDB format) -- These formats are **incompatible** - existing databases cannot be directly migrated -- User decided to start fresh with new project, eliminating need for data migration - -## Testing Results -- ✅ All 71 tests passing -- ✅ Code coverage maintained at ~66% -- ✅ No API changes for consuming code - -## Build Process Issues & Resolutions - -### Issue 1: TypeScript Errors -- **Problem**: TypeScript 4.2.3 didn't understand `Awaited` type and had React type conflicts -- **Solution**: Updated TypeScript to 4.9.5 and @types/node to 18.11.9 - -### Issue 2: node-lightning Type Conflicts -- **Problem**: `@node-lightning/noise` had incompatible type definitions with TypeScript 4.9 -- **Solution**: Added `skipLibCheck: true` as permanent solution (appropriate since node-lightning is deprecated) - -## Current State -- Migration complete and working -- Using modern async/await API instead of streams -- Database files will be created in LevelDB format going forward -- Ready for use with Node.js 18+ - -## Usage Notes for Dependent Projects -1. Must update to Node.js 18+ -2. Run `yarn bootstrap` to rebuild all packages -3. No code changes required in consuming applications -4. Database format change means starting fresh (no migration from existing RocksDB databases) - -## Commands to Verify -```bash -# Build all packages -yarn bootstrap - -# Run tests -cd packages/rocksdb && npm test -``` - -## Benefits Achieved -- Using actively maintained database library -- Modern async/await API instead of streams -- Cleaner codebase without separate levelup wrapper -- Future-proof solution as part of official Level ecosystem -- Better TypeScript support with updated versions \ No newline at end of file