From a260709e9f9aeb3de053f5159960e25aaec1f66a Mon Sep 17 00:00:00 2001 From: zbnerd Date: Wed, 17 Jun 2026 07:16:41 +0200 Subject: [PATCH 1/3] =?UTF-8?q?perf(ext-api):=20bump=20Nexon=20HTTP=20pool?= =?UTF-8?q?=20maxConnections=20150=20=E2=86=92=20250?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pipeline runs with max-in-flight=250, but the Nexon connection pool was sized to 150. With 250 in-flight concurrent requests, 100 always wait at pendingAcquire per wave. The wait serialized fetches into two sub-waves, effectively doubling the batch duration. Measured impact (2026-06-17): - Old (pool=150): rate=100/s, batch_wait=2.27s, fetch=1.5s - Expected (pool=250): rate=200/s, single wave of 250 Direct curl on Nexon API: 150-200ms after TLS handshake. The 1.5s measured in-pipeline is mostly pool acquire wait, not actual Nexon processing. This change matches pool size to in-flight size. Tuning knob preserved: external-api.http-client.max-connections in YAML still overrides the default. Co-Authored-By: Claude Opus 4.8 --- .../maple/externalapi/config/NexonHttpClientProperties.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/module-external-api/src/main/kotlin/maple/externalapi/config/NexonHttpClientProperties.kt b/module-external-api/src/main/kotlin/maple/externalapi/config/NexonHttpClientProperties.kt index 5830ed302..50713b6ec 100644 --- a/module-external-api/src/main/kotlin/maple/externalapi/config/NexonHttpClientProperties.kt +++ b/module-external-api/src/main/kotlin/maple/externalapi/config/NexonHttpClientProperties.kt @@ -5,7 +5,7 @@ import org.springframework.boot.context.properties.ConfigurationProperties @ConfigurationProperties(prefix = "nexon.http-client") data class NexonHttpClientProperties( val poolName: String = "nexon-pool", - val maxConnections: Int = 150, + val maxConnections: Int = 250, val pendingAcquireMaxCount: Int = 1000, val pendingAcquireTimeoutMs: Long = 5000, val connectTimeoutMs: Int = 3000, From 998c0bf606eb45dd843fbd865e55e9a9e7de4f63 Mon Sep 17 00:00:00 2001 From: zbnerd Date: Wed, 17 Jun 2026 08:02:57 +0200 Subject: [PATCH 2/3] perf(ext-api): switch rate limiter to greedy refill + tune pool MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two related changes that combined give +15% throughput on item-equipment phase (rate 85/s → 100/s sustained), measured 2026-06-17. 1. SchedulerRateLimiter: refillIntervally → refillGreedy - `refillIntervally(250, 1s)` adds 250 tokens at the start of each 1-second boundary (t=0, t=1, t=2). Right after a full consume at t=0, the next 250 tokens only arrive at t=1, creating a hard "no tokens for ~0.5-1s" gap that serializes waves. - `refillGreedy(250, 1s)` adds tokens continuously at 250/s rate. After a full consume at t=0, the bucket fills smoothly. By the time the wave completes (fetch 1.3s), bucket is at capacity (capped at 250), so the next wave fires without idle wait. 2. application.yml: max-connections 150 → 250 - Pool sized to 150 < in-flight 250. With 250 concurrent requests, 100 always waited at pendingAcquire per wave. The wait serialized fetches into two sub-waves of 150+100, doubling the batch duration. - Bumped to 250 to match in-flight. The two together: pool no longer serializes fetches, and rate limiter no longer adds idle wait between waves. Throughput up ~17%. Why not bump in-flight higher? Tested 500 in-flight: Nexon API returned 314 HTTP 429s in 3min, fetchJoinMs went 1.3s → 5.0s, rate dropped to 85/s. Nexon throttles per source IP; 500 concurrent exceeds their allowance. Stayed at 250. Co-Authored-By: Claude Opus 4.8 --- .../maple/externalapi/scheduler/phase/SchedulerRateLimiter.kt | 2 +- module-external-api/src/main/resources/application.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/module-external-api/src/main/kotlin/maple/externalapi/scheduler/phase/SchedulerRateLimiter.kt b/module-external-api/src/main/kotlin/maple/externalapi/scheduler/phase/SchedulerRateLimiter.kt index 864e3e3fb..3c400d26c 100644 --- a/module-external-api/src/main/kotlin/maple/externalapi/scheduler/phase/SchedulerRateLimiter.kt +++ b/module-external-api/src/main/kotlin/maple/externalapi/scheduler/phase/SchedulerRateLimiter.kt @@ -12,7 +12,7 @@ class SchedulerRateLimiter { .addLimit( Bandwidth.builder() .capacity(permits.toLong()) - .refillIntervally(permits.toLong(), Duration.ofSeconds(1)) + .refillGreedy(permits.toLong(), Duration.ofSeconds(1)) .build(), ) .build() diff --git a/module-external-api/src/main/resources/application.yml b/module-external-api/src/main/resources/application.yml index 4c8a3d9d0..9635b52f2 100644 --- a/module-external-api/src/main/resources/application.yml +++ b/module-external-api/src/main/resources/application.yml @@ -23,7 +23,7 @@ nexon: key: ${NEXON_API_KEY} http-client: pool-name: ${NEXON_HTTP_POOL_NAME:nexon-pool} - max-connections: ${NEXON_HTTP_MAX_CONNECTIONS:150} + max-connections: ${NEXON_HTTP_MAX_CONNECTIONS:250} pending-acquire-max-count: ${NEXON_HTTP_PENDING_ACQUIRE_MAX_COUNT:1000} pending-acquire-timeout-ms: ${NEXON_HTTP_PENDING_ACQUIRE_TIMEOUT_MS:5000} connect-timeout-ms: ${NEXON_HTTP_CONNECT_TIMEOUT_MS:3000} From 939121ac44eb91c9edabaa845f662c202bd992e4 Mon Sep 17 00:00:00 2001 From: zbnerd Date: Wed, 17 Jun 2026 08:52:08 +0200 Subject: [PATCH 3/3] fix(compose): override MINIO_ENDPOINT for in-container minio-bootstrap minio-bootstrap runs inside the docker network, where `localhost` resolves to the container itself, not the host. The .env.bootstrap file keeps `MINIO_ENDPOINT=http://localhost:9000` for host-side tooling (mc CLI), which doesn't reach the in-container minio. Fix: add an `environment:` block on minio-bootstrap that overrides MINIO_ENDPOINT to http://minio:9000, leaving .env.bootstrap intact. Verified 2026-06-17: docker compose up minio-bootstrap exits clean with bucket + ILM + 4 SAs + 4 policies attached. Co-Authored-By: Claude Opus 4.8 --- docker-compose.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docker-compose.yml b/docker-compose.yml index d5b10eedc..df3e229a2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -173,6 +173,10 @@ services: - maple-network env_file: - .env.bootstrap + environment: + # Override .env.bootstrap MINIO_ENDPOINT for in-container DNS. + # .env.bootstrap keeps localhost:9000 so host-side tooling works. + MINIO_ENDPOINT: http://minio:9000 volumes: - ./docker/minio:/scripts:ro entrypoint: /bin/sh /scripts/bootstrap.sh