New alloca by Km1zZzoU · Pull Request #43 · heuristic-team/nop

Km1zZzoU · 2025-08-25T19:41:33Z

new allocator and refactoring

Copilot

Pull Request Overview

This PR introduces a new allocator system with a garbage collector and refactors the memory management architecture. The changes implement a hierarchical allocator with thread management, garbage collection capabilities, and a more sophisticated arena-based memory allocation system.

Key changes include:

Introduction of a hierarchical allocator (HAllocator) with garbage collection support
New thread management system with mutable/immutable phases and thread pools
Replacement of simple Object trait with a concrete Object struct containing size and bitset information

Reviewed Changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
runtime/src/utils/object.rs	Defines new Object struct with size and bitset fields
runtime/src/utils/mod.rs	Introduces reg type alias and exports Object utilities
runtime/src/threads/threadpool.rs	Implements ThreadPool for managing worker threads with state transitions
runtime/src/threads/nthread.rs	Defines thread state management with phases and contexts
runtime/src/threads/mod.rs	Main thread management interface with stop-the-world coordination
runtime/src/gc/mod.rs	Core garbage collector implementation with mark-and-sweep algorithm
runtime/src/gc/markqueue.rs	Queue system for GC marking phase
runtime/src/lib.rs	Main runtime interface with initialization and allocation functions
runtime/src/alloca/mod.rs	Refactored allocator module exports and test updates
runtime/src/alloca/hedgearena.rs	Enhanced arena implementation with GC support
runtime/src/alloca/hallocator.rs	Hierarchical allocator with tiered block management
runtime/src/alloca/cfg.rs	Configuration system for allocator parameters
runtime/src/alloca/arena.rs	Updated Arena3 trait with new lifecycle methods
runtime/src/alloca/allocator.rs	Simplified allocator trait interface

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

chronorose · 2025-08-27T11:37:10Z

+    fn alloc3() {
+        let mut aa = HAllocator::new(config1());
+        let inst = Object {
+            size: 48,
+            bitset: &[0],
+        };
+        for i in 0..100_000_000 {
+            test_alloc(&inst, &mut aa);
+        }
    }


не проходит если поставить размер >=64

при i in 0..2048 ещё всё хорошо а на 2049 уже ломается

пофиксил

chronorose · 2025-08-27T11:38:20Z

+    pub fn popn(&self, n: usize) -> Vec<MarkQueueElement> {
+        let mut lock = self.queue.write().expect("popn");
+        let bound = min(lock.len(), n);
+        let mut result = vec![];
+        for _ in 0..bound {
+            result.push(lock.pop().expect("popn copy"));
+        }
+        result
+    }


не с той стороны

пофиксил

chronorose · 2025-08-27T11:40:38Z

+pub struct Gc<U: Arena3 + Send + Sync + 'static> {
+    pub alloca: Arc<Mutex<alloca::HAllocator<U>>>,
+
+    threads: Arc<Mutex<threads::Threads>>,
+
+    stw_is_done: Arc<Mutex<bool>>,
+    stw_cv: Arc<std::sync::Condvar>,
+
+    heap_is_done: Arc<Mutex<bool>>,
+    heap_cv: Arc<std::sync::Condvar>,
+
+    mark_is_done: Arc<Mutex<bool>>,
+    mark_cv: Arc<std::sync::Condvar>,
+
+    root_is_done: Arc<Mutex<bool>>,
+    root_cv: Arc<std::sync::Condvar>,
+
+    work_is_done: Arc<Mutex<bool>>,
+    work_cv: Arc<std::sync::Condvar>,
+    workers: Vec<JoinHandle<()>>,
+    count_active_workers: Arc<AtomicUsize>,
+
+    root: Arc<Vec<ptr>>,
+    mark_queue: Arc<MarkQueue>,
+}


чето не юзается много че

chronorose · 2025-08-27T11:53:49Z

+                        if !(local_queue.len() & 15 == 0) {
+                            // TODO: make somethink like cfg
+                            (*thread_mark_queue).pushn(&mut local_queue);


пофиксил

chronorose · 2025-08-27T11:59:05Z

+                                loop {
+                                    if (*thread_mark_queue).last_is_end() {
+                                        thread_count_active_workers.fetch_add(1, Ordering::SeqCst);
+                                        {
+                                            let mut work_is_done_flag =
+                                                thread_work_is_done.lock().unwrap();
+                                            while !*work_is_done_flag {
+                                                work_is_done_flag =
+                                                    thread_work_cv.wait(work_is_done_flag).unwrap()
+                                            }
+                                        }
+                                        break 'external;
+                                    } else {
+                                        local_queue = (*thread_mark_queue).popn(16);
+                                        break;
+                                    }
+                                }


там кстати все таки нужен этот луп, там если в очередь не попал конец, и при этом в ней никого нет, то надо покрутиться

Km1zZzoU · 2025-08-27T19:06:03Z

Pull Request Overview

This PR introduces a new allocator system with a garbage collector and refactors the memory management architecture. The changes implement a hierarchical allocator with thread management, garbage collection capabilities, and a more sophisticated arena-based memory allocation system.

Key changes include:
* Introduction of a hierarchical allocator (`HAllocator`) with garbage collection support

* New thread management system with mutable/immutable phases and thread pools

* Replacement of simple Object trait with a concrete Object struct containing size and bitset information
Reviewed Changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 8 comments.
Show a summary per file

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

я пофиксил, го новое ревью

Km1zZzoU · 2025-08-27T19:07:03Z

соре за спам :3

battlemag3 · 2025-08-27T19:47:42Z

Ревью нэ будэ

chronorose · 2025-08-27T22:56:10Z

+                        }
+                    }
+
+                    'external: loop {


а че экстернал не ушёл в итоге

потому что все таки не ушел внутрений цикл который в else, потому что там есть кейс, в котором надо нихуя не делать и сидеть ждать конец

надо этот цикл разнести по методам постараться наверное всё таки, слишком крепко

попозже посмотрю че куда можно будет

chronorose · 2025-08-27T23:00:15Z

+                        while local_queue.len() > 0 {
+                            Self::mark(thread_alloca.clone(), local_queue.pop_front().unwrap());
+                        }


учитывая что это локальная история это просто фолд

chronorose · 2025-08-27T23:03:30Z

+        let bound = min(lock.len(), n);
+        let mut result = VecDeque::new();
+        for _ in 0..bound {
+            result.push_back(lock.pop_front().expect("popn copy"));
+        }


0..min(lock.len(), n).map(|_| lock.pop_front().expect(...))

Не выебывайся

дело вкуса

ну конкретно тут то не только дело вкуса ибо оно ресайзится никогда не будет с мапом, а вот с циклом не факт

chronorose · 2025-08-27T23:05:16Z

+            let thread_root_is_done = self.root_is_done.clone();
+            let thread_root_cv = self.root_cv.clone();
+            let mut thread_root = self.root.clone();
+            let thread_alloca = self.alloca.clone();
+            let thread_mark_queue = self.mark_queue.clone().clone();
+            let thread_count_active_workers = self.count_active_workers.clone();
+            let thread_mark_is_done = self.mark_is_done.clone();
+            let thread_mark_cv = self.mark_cv.clone();


может это в отдельный стракт какой-то логический и просто клонить его сразу весь?

не вижу смысл, тут либо вынести сигнальное говно, но там есть мьютекс+кондд вар которые не нужны именно треду, можно вынести то, что нужно треду, но аллока, count_active_workers и тд нужны не только воркерам, + все равно эти 8 строчек надо будет перенести в метод клон, будто мы все равно метод останется едва читаемым

не, там задерайвить клон можно будет просто, не нужно будет писать эти строчки

так пусть будут нужны никто же не забирает, пусть они просто вместе в одном стракте лежать будут а так останутся на месте

chronorose · 2025-08-27T23:06:29Z

+                    let mut local_queue = VecDeque::new();
+
+                    let count_for_scan = thread_root.len() / count + 1;
+                    for j in min(i * count_for_scan, thread_root.len())..thread_root.len() {


этот min наверное стоит вынести перед циклом всё таки, для уверенности в завтрашнем дне

chronorose · 2025-08-27T23:07:22Z

+                        }
+
+                        local_queue = (*thread_mark_queue).popn(16);
+                        if local_queue.len() == 0 {


is_empty или как-то по другому но точно есть чето такое

Km1zZzoU added 19 commits July 29, 2025 02:31

.

76f243b

threads 0

fb44358

gc 0.0

df86370

add markqueue

9c5adac

rewrite mark colors

bf2714b

add flag in queue in Arena3

d1ff919

refactoring lib.rs

b9b5b08

progress in gc/mod.rs

d1be98c

start -> span_start

aa7c9d0

progress in the gc module

75f3f48

misstake

8935849

fix misstakes

0fd4144

sweep complete

9c42cf0

;

230c05e

rewrite alloca

6b9b569

the worst commit ever, dont check please, rust issue

2ba2994

this fcking bitch compiled

99dd8e5

fix tests

ca5ed15

fix bugs, complete test and stress-test (alloca3)

6db5291

Km1zZzoU requested review from chronorose and Copilot August 27, 2025 10:35

Copilot AI reviewed Aug 27, 2025

View reviewed changes

chronorose requested changes Aug 27, 2025

View reviewed changes

fix mutexs, bug in alloca and etc

32acf95

Km1zZzoU requested a review from chronorose August 27, 2025 19:04

Km1zZzoU requested review from chronorose and removed request for chronorose August 27, 2025 19:06

chronorose requested changes Aug 27, 2025

View reviewed changes

review

6df8596

Conversation

Km1zZzoU commented Aug 25, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Km1zZzoU commented Aug 27, 2025

Pull Request Overview

Reviewed Changes

Uh oh!

Km1zZzoU commented Aug 27, 2025

Uh oh!

battlemag3 commented Aug 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!