You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Cache NumPy field views for 'id', 'logit', and 'p' to bypass expensive property lookups.
13
+
- Refactor copy_logits to use pre-generated ID sequences and cached views.
14
+
- Ensure logical consistency by resetting token IDs every sampling step to counter C++ reordering.
15
+
- Minimize redundant memory allocations during the inference loop.
16
+
17
+
- feat: Add explicit memory cleanup for sampling contexts
18
+
- Implements `close()` and `__del__` for LlamaTokenDataArray and expands LlamaSamplingContext cleanup.
19
+
- Ensures NumPy views and internal C-references are properly released to allow Python GC to reclaim memory.
20
+
21
+
- optimize(memory): reduce scores buffer size and optimize state saving
22
+
- Update save_state and load_state API use.
23
+
- Refactored self.scores to allocate only a single row (1, n_vocab) when logits_all=False, significantly reducing memory usage for large vocabulary models.
24
+
- Optimized save_state to eliminate redundant memory allocations and copies by using ctypes.string_at.
25
+
- Updated load_state, eval, and sampler adapters to correctly handle the dynamic shape of self.scores.
26
+
27
+
- Fix CMake install layout to avoid top-level bin directory in site-packages
28
+
29
+
- ggml: Load ggml library from candidate path list
30
+
- Auto-select lib/ or bin/ directories
31
+
- Add backend loading functions
32
+
33
+
- feat(loader): extend default library search paths on Linux and macOS
34
+
-`load_shared_library` to include a path list feature (allowing you to add custom paths in addition to the default ones). You can later add your own paths to the `libggml_base_paths` candidate list in `_ggml.py`, such as those not commonly used Python paths.
35
+
- fix: Enhance the handling logic for non-existent file paths.
36
+
- Improves reliability of shared library discovery for system-wide installations.
37
+
38
+
- feat: Update llama.cpp to [ggml-org/llama.cpp/commit/abb9f3c42b5e6acee9e8e37836ef691d1a41bdb8](https://github.com/ggml-org/llama.cpp/commit/abb9f3c42b5e6acee9e8e37836ef691d1a41bdb8)
39
+
40
+
- feat: Sync llama.cpp llama/mtmd API Binding 20260219
41
+
42
+
More information see: https://github.com/JamePeng/llama-cpp-python/compare/b03224b2dde3c8cbdd8bf529794e3a41ac7f5751...6b38182a17effd0cedbf8736bee2464dd6c7b4da
43
+
10
44
## [0.3.25]
11
45
- feat: [Refactor Llama class to use new LlamaSampler chain API from _internals](https://github.com/JamePeng/llama-cpp-python/commit/1e6094a327f0fb9dc35d52f84d8ebabc1faa1e95)
12
46
This commit refactors the high-level Llama class to fully utilize the new C++ `llama_sampler` chain architecture via `LlamaSamplingContext`.
0 commit comments