Skip to content

rate limiting to 9 req/sec#86

Merged
robz merged 1 commit into
asmbly-makerspace:mainfrom
robz:fix-429s-again
Apr 1, 2026
Merged

rate limiting to 9 req/sec#86
robz merged 1 commit into
asmbly-makerspace:mainfrom
robz:fix-429s-again

Conversation

@robz
Copy link
Copy Markdown
Contributor

@robz robz commented Apr 1, 2026

We're still getting 429s from neon when fetching membership data. So adding rate limiting to ensure we parallelize as much as possible but without going over 10 qps

2026-04-01 04:41:04 INFO Fetching membership details for 571 accounts
2026-04-01T09:41:04.000Z
Traceback (most recent call last): File "/home/ec2-user/NeonIntegrations/dailyMaintenance.py", line 52, in <module> main() File "/home/ec2-user/NeonIntegrations/dailyMaintenance.py", line 24, in main neonAccounts = neonUtil.getRealAccounts() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ec2-user/NeonIntegrations/neonUtil.py", line 473, in getRealAccounts results = list(executor.map(appendMemberships, accounts_to_fetch)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/concurrent/futures/_base.py", line 619, in result_iterator yield _result_or_cancel(fs.pop()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel return fut.result(timeout) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/usr/lib64/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ec2-user/NeonIntegrations/neonUtil.py", line 123, in appendMemberships raise ValueError(f"Get {url} returned status code {response.status_code}: {response.text}")
2026-04-01T09:41:04.000Z
ValueError: Get https://api.neoncrm.com/v2/accounts/8033/memberships returned status code 429: [{"code":"9997","message":"Get an account's membership history gone over throttle limit (10 calls per second)."}]

@robz robz merged commit e27da55 into asmbly-makerspace:main Apr 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant