Concurrency#18
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds concurrency support to the metrics collection process in the collector. The changes enable parallel processing of metrics while respecting configurable concurrency limits.
- Introduces concurrent goroutine execution for metric collection using sync.WaitGroup
- Implements a semaphore pattern to limit maximum concurrent operations
- Maintains sequential collection logic but executes it in parallel for better performance
| go func(mc config.Metric) { | ||
| defer wg.Done() | ||
|
|
||
| smph <- struct{}{} | ||
| defer func() { | ||
| <-smph | ||
| }() | ||
|
|
||
| if len(mc.SubMetrics) == 0 { | ||
| c.collectSimpleMetric(ch, mc) | ||
| } else { | ||
| c.collectComplicatedMetric(ch, mc) | ||
| } | ||
| }(metricConfig) |
There was a problem hiding this comment.
The semaphore acquisition could block indefinitely if the context is cancelled or the operation needs to be interrupted. Consider using a select statement with context cancellation to make this interruptible.
| if len(mc.SubMetrics) == 0 { | ||
| c.collectSimpleMetric(ch, mc) | ||
| } else { | ||
| c.collectComplicatedMetric(ch, mc) | ||
| } | ||
| }(metricConfig) | ||
| } | ||
|
|
||
| wg.Wait() | ||
|
|
||
| c.logger.Info("Metrics collection finished") |
There was a problem hiding this comment.
The WaitGroup.Wait() call could block indefinitely if any goroutine panics or hangs. Consider adding a timeout or context cancellation to prevent the collector from hanging indefinitely.
| wg.Add(1) | ||
|
|
||
| go func(mc config.Metric) { | ||
| defer wg.Done() |
There was a problem hiding this comment.
Goroutines that panic will not be recovered and could cause issues. Consider adding panic recovery with defer recover() to ensure the WaitGroup is properly decremented and semaphore is released even if collectSimpleMetric or collectComplicatedMetric panic.
| defer wg.Done() | |
| defer func() { | |
| if r := recover(); r != nil { | |
| c.logger.Error("Panic recovered in goroutine", "metric", mc.Name, "error", r) | |
| } | |
| wg.Done() | |
| }() |
No description provided.