Skip to content

Commit f1e648b

Browse files
chore(mac-bridge): raise MAX_NEW_TOKENS bound 512->2048 (chat answers need it; natural EOS stops first)
Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
1 parent 9064d53 commit f1e648b

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

inference_engine/bridge/manifest.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
# bridge is for evidence runs and debugging, not for monopolizing the
3030
# single Mac with open-ended workloads.
3131
MAX_N_SAMPLES = 50
32-
MAX_NEW_TOKENS = 512
32+
MAX_NEW_TOKENS = 2048 # backstop for chat; natural EOS stops well before this
3333
MAX_BLOCK_SIZE = 16
3434

3535
_ENV_PLACEHOLDER = re.compile(r"^\$\{ENV:([A-Z][A-Z0-9_]*)\}$")

0 commit comments

Comments
 (0)