Conversation
drahnr
force-pushed
the
bernhard-feat-add-rate-limiter
branch
from
3ea5ce1 to
558e109
Compare
| // TODO double check how to address proxy rate limiting based on i.e. `X-Real-IP`. | ||
| request | ||
| .metadata_mut() | ||
| .insert("forwarded", format!("for={addr}").try_into().unwrap()); |
There was a problem hiding this comment.
we shouldn't panic here right? this is a callback for every request?
There was a problem hiding this comment.
It is a callback per request, unless someone manages to inject a broken/malicious IP in the Linux Kernel and manage to pop it up in our end, it's virtually impossible to break this invariant.
Since we're using SmartIPKeyExtractor* we might get away without this. It's here for ensuring we have a fallback
Mirko-von-Leipzig
left a comment
Mirko-von-Leipzig
left a comment
There was a problem hiding this comment.
I'm unclear on whether we want to apply any gRPC options to our internal services at all. I would tend to say no?
| /// | ||
| /// If the handler takes longer than this duration, the server cancels the call. | ||
| pub grpc_timeout: Duration, | ||
| pub grpc_options: GrpcOptionsInternal, |
There was a problem hiding this comment.
Doc comment is inaccurate now. We also ignore the max connections setting then?
crates/utils/src/clap.rs
Outdated
| /// Number of global concurrent connections. | ||
| #[arg( | ||
| long = "grpc.max_global_connections", | ||
| default_value_t = DEFAULT_MAX_GLOBAL_CONNECTIONS, | ||
| value_name = "MAX_GLOBAL_CONNECTIONS" | ||
| )] | ||
| pub max_global_concurrent_connections: u64, |
There was a problem hiding this comment.
I think global is confusing -- this is just the maximum number of connections?
There was a problem hiding this comment.
Rename to _concurrent_ if that's any better, open to suggestions
| use tonic::service::Interceptor; | ||
| use tonic::{Request, Status}; | ||
|
|
||
| // Extracts the IP for `Governor` |
There was a problem hiding this comment.
Its unclear in this context what a Governor is.
crates/store/src/server/mod.rs
Outdated
| } | ||
| }); | ||
|
|
||
| let concurrency_semaphore = grpc::concurrency_semaphore(self.grpc_options); |
There was a problem hiding this comment.
I think this is incorrect. We don't want to share concurrency between these because it means external RPC requests can DoS our internal requests. I still don't think we want to apply any limits internally at all.
There was a problem hiding this comment.
Removed, only limiting the RPC component now
|
I'd tend to say a global connection limit to self-protect each service is a good thing in general. It's a trade-off of risks, and I don't think there is a good answer. I removed all limits for internal services again. |
I do agree; but at the moment we don't have any good idea on numbers. |
3 participants
Adds a rate-limiter with quotas using
tower_governorandtower::limit::GlobalConcurrencyLimitLayer.Done in scope of our recent fd consumption, generally a useful thing to do.
Makes all parameters configurable and group them into
GrpcOptionsand moves them tomiden-node-utilsto avoid duplication, similarly for limiting layer setup helpers.