BudgetScan

RemoveClients callback invoked while holding write lock can deadlock or lock permanently

high swesweep-fasthttp · lbclient.go · LBClient.RemoveClients · run #4

Description

rc is called while cc.mu is write-locked and without a deferred Unlock. If rc reentrantly calls AddClient, RemoveClients, or Do/DoTimeout/DoDeadline, it blocks on the same mutex and deadlocks. If rc panics, the mutex is not released, permanently locking the LBClient.

Triage

Verdict
INVALID confidence 0.0
Model
qwen/qwen3.8-27b
Rounds
6
Cost
$0.0166 · 823.1 s
[ARBITER] The reported issue is a potential deadlock or lock leak caused by re-entrant calls or panics within the user-provided callback `rc`. Under the provided threat model for a library, the attacker is defined as an entity providing adversarial data (payloads, strings, buffers) to public APIs. The `rc` parameter is a function pointer (callback) written by the library consumer (trusted code), not an attacker-controlled input. A deadlock resulting from the consumer writing re-entrant logic in their own callback constitutes API misuse, not a security vulnerability exploitable by an external attacker. Furthermore, the library cannot reasonably prevent a caller's callback from re-entering the library or panicking without violating performance or design constraints typical for such synchronization primitives.
All rounds
[
  {
    "finding_title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "verdict": "UNCERTAIN",
    "reasoning": "The report claims that `RemoveClients` invokes the callback `rc` while holding the write lock (`cc.mu.Lock()`), which would cause a deadlock if `rc` reentrantly calls `AddClient` or `RemoveClients`, or cause a permanent lock if `rc` panics.\n\n1.  **Bug Pattern Verification**: The code for `RemoveClients` is:\n    ```go\n    func (cc *LBClient) RemoveClients(rc func(BalancingClient) bool) int {\n        cc.mu.Lock()\n        n := 0\n        for idx, cs := range cc.cs {\n            cc.cs[idx] = nil\n            if rc(cs.c) { // rc is called here\n                continue\n            }\n            cc.cs[n] = cs\n            n++\n        }\n        cc.cs = cc.cs[:n]\n        cc.mu.Unlock()\n        return len(cc.cs)\n    }\n    ```\n    The assertion is technically correct that `rc` is called while `cc.mu` is locked. If `rc` calls `cc.AddClient`, `cc.RemoveClients`, `cc.Do`, etc., it will attempt to acquire `cc.mu` (Lock or RLock) and block forever, causing a deadlock. If `rc` panics, the `Unlock` is skipped (no `defer`), leaving the mutex locked.\n\n2.  **Threat Model Analysis**: The threat model defines the attacker as \"Code that calls the public API, and above all the data it passes through... parsers, decoders... receive adversarial payloads...\" and specifies that \"Misuse that violates a documented precondition of the public API\" is **Not a vulnerability**.\n    \n    The function `RemoveClients` takes `rc func(BalancingClient) bool`. This callback is provided by the *application developer* or the *library user* who is managing the load balancer configuration. It is not an \"adversarial payload\" received from a network or file. It is a programmatic hook.\n    \n    The deadlock only occurs if the user-provided callback `rc` itself attempts to re-enter the `LBClient` mutex. This is a misuse of the API by the caller (the developer). The library does not claim to be re-entrant with respect to the user's callbacks. Standard Go mutex usage implies that a function holding a lock should not call back into the user in a way that acquires the same lock, unless explicitly documented as safe (e.g., via RWMutex where readers don't block each other, but even then, a write lock blocks all). Here, `RemoveClients` holds a *write* lock. \n    \n    Since the \"attacker",
    "elapsed": 209.3,
    "tokens": 3890,
    "tokens_in": 3307,
    "tokens_out": 583,
    "cost_usd": 0.0031379399999999997,
    "file": "lbclient.go",
    "round": 1
  },
  {
    "finding_title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "verdict": "INVALID",
    "reasoning": "The reported vulnerability is a potential deadlock in `RemoveClients` if the provided callback `rc` reentrantly calls other `LBClient` methods (like `AddClient` or `Do`) that require the `cc.mu` lock. While the code technically holds the write lock while invoking the user-supplied callback, this scenario requires the caller (the application developer) to write specific reentrant logic within the callback. The threat model for a library explicitly states that 'misuse that violates a documented precondition of the public API' is not a vulnerability. In Go, calling back into a locked object's methods from within a method that acquires a write lock is a classic programmer error/misuse, not a bug in the library's handling of untrusted data (like network packets). The attacker cannot control the source code of the `rc` callback to make it reentrant; they can only control data passed *into* the public API. Since the 'attacker' would have to be the code author who wrote the buggy callback, and the threat model limits the attacker to providing adversarial payloads/data to trusted code, this is not a security vulnerability. It is a code quality/robustness issue regarding API design (lack of `defer` for panic safety), but it does not meet the bar for a security flaw under the provided threat model.\n\nCRUX: Whether a deadlock caused by a user-provided callback reentrantly calling the same object's methods constitutes a security vulnerability when the callback itself is trusted code, or if it is considered misuse violating the API's implicit contract.\nGREP: func (cc \\*LBClient) RemoveClients",
    "elapsed": 154.0,
    "tokens": 4360,
    "tokens_in": 4010,
    "tokens_out": 350,
    "cost_usd": 0.0027342,
    "file": "lbclient.go",
    "round": 2,
    "grep_used": true,
    "grep_results": "GREP `func (cc *LBClient) RemoveClients (simplified to: RemoveClients)`:\n```\nlbclient.go:111:// RemoveClients removes clients using the provided callback.\nlbclient.go:114:func (cc *LBClient) RemoveClients(rc func(BalancingClient) bool) int {\n```"
  },
  {
    "finding_title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "verdict": "INVALID",
    "reasoning": "The reported vulnerability is a deadlock or permanent lock state in `LBClient.RemoveClients` if the user-provided callback `rc` reentrantly calls `LBClient` methods or panics. To exploit this, an attacker must control the implementation of the `rc` function. Under the provided threat model, the attacker is defined as code calling the public API with adversarial *data* (payloads, strings, buffers), not code injecting arbitrary logic that re-enters the library's internal locking state. The `rc` callback is trusted code written by the library user. A deadlock caused by a user calling a lock-acquiring method inside a callback that is itself invoked while holding that lock is a classic API misuse/programming error, not a vulnerability reachable via untrusted input. Furthermore, the threat model explicitly excludes \"misuse that violates a documented precondition\" (or implied contracts of thread-safety/reentrancy not documented as safe for reentry) from vulnerabilities. Since the attacker cannot trigger this by sending a crafted `Request` or `Response`, but would need to write buggy Go code themselves, it is not a security vulnerability in this context.\n\nCRUX: The attacker must control the source code of the `rc` callback to trigger the deadlock, which is outside the scope of untrusted data input defined in the threat model.\nGREP: RemoveClients",
    "elapsed": 137.5,
    "tokens": 4723,
    "tokens_in": 4431,
    "tokens_out": 292,
    "cost_usd": 0.0027370199999999997,
    "file": "lbclient.go",
    "round": 3,
    "grep_used": true,
    "grep_results": "GREP `RemoveClients`:\n```\nlbclient.go:111:// RemoveClients removes clients using the provided callback.\nlbclient.go:114:func (cc *LBClient) RemoveClients(rc func(BalancingClient) bool) int {\n```"
  },
  {
    "finding_title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "verdict": "INVALID",
    "reasoning": "The report identifies a potential deadlock in `LBClient.RemoveClients` where the callback `rc` is invoked while the write lock `cc.mu` is held. If `rc` reentrantly calls other `LBClient` methods (e.g., `AddClient`, `Do`) that attempt to acquire the same lock, the program deadlocks. Additionally, if `rc` panics, the lock is not released due to the lack of `defer`, permanently locking the `LBClient`.\n\nHowever, the threat model for this library explicitly defines the attacker as an entity providing adversarial *data* (payloads, buffers, strings) to the public API. The `rc` parameter in `RemoveClients` is a function pointer (callback) provided by the library's user (the application developer). For the deadlock to occur, the user must write code within `rc` that re-enters the `LBClient` API. This constitutes a programming error or API misuse by the developer, not a vulnerability triggered by untrusted external input. The threat model explicitly states that 'misuse that violates a documented precondition of the public API' is not a vulnerability. Since the attacker cannot control the implementation of the trusted callback `rc` via network traffic or file input, this issue is not reachable through the defined trust boundary. It is a code quality/robustness concern, not a security vulnerability.\n\nCRUX: The trigger for the deadlock requires control over the source code of the `rc` callback, which is trusted code written by the library user, not untrusted data provided by an external attacker.\nGREP: func (cc \\*LBClient) RemoveClients",
    "elapsed": 123.0,
    "tokens": 5113,
    "tokens_in": 4757,
    "tokens_out": 356,
    "cost_usd": 0.0030659399999999996,
    "file": "lbclient.go",
    "round": 4,
    "grep_used": true,
    "grep_results": "GREP `func (cc *LBClient) RemoveClients (simplified to: RemoveClients)`:\n```\nlbclient.go:111:// RemoveClients removes clients using the provided callback.\nlbclient.go:114:func (cc *LBClient) RemoveClients(rc func(BalancingClient) bool) int {\n```"
  },
  {
    "finding_title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "verdict": "INVALID",
    "reasoning": "The report claims a deadlock or permanent lock state in `RemoveClients` if the callback `rc` reentrantly calls `LBClient` methods (like `AddClient` or `Do`) or if `rc` panics. While the code does hold the write lock `cc.mu` while invoking `rc`, the threat model for this library defines the attacker as providing adversarial *data* (payloads, buffers, strings) to the public API. The `rc` parameter is a function pointer (callback) provided by the library's user (the application developer). For the deadlock to occur, the user must write code within `rc` that re-enters the `LBClient` API or allows a panic to propagate. This constitutes a programming error or API misuse by the developer (trusted code), not a vulnerability triggered by untrusted external input. The threat model explicitly states that 'misuse that violates a documented precondition of the public API' is not a vulnerability. Since the attacker cannot control the implementation of the trusted callback `rc` via network traffic or file input, this issue is not reachable through the defined trust boundary. It is a code quality/robustness concern, not a security vulnerability.\n\nCRUX: The trigger for the deadlock requires control over the source code of the `rc` callback, which is trusted code written by the library user, not untrusted data provided by an external attacker.\nGREP: func (cc *LBClient) RemoveClients",
    "elapsed": 101.1,
    "tokens": 5491,
    "tokens_in": 5175,
    "tokens_out": 316,
    "cost_usd": 0.0031215,
    "file": "lbclient.go",
    "round": 5,
    "grep_used": true,
    "grep_results": "GREP `func (cc *LBClient) RemoveClients (simplified to: RemoveClients)`:\n```\nlbclient.go:111:// RemoveClients removes clients using the provided callback.\nlbclient.go:114:func (cc *LBClient) RemoveClients(rc func(BalancingClient) bool) int {\n```"
  },
  {
    "verdict": "INVALID",
    "reasoning": "[ARBITER] The reported issue is a potential deadlock or lock leak caused by re-entrant calls or panics within the user-provided callback `rc`. Under the provided threat model for a library, the attacker is defined as an entity providing adversarial data (payloads, strings, buffers) to public APIs. The `rc` parameter is a function pointer (callback) written by the library consumer (trusted code), not an attacker-controlled input. A deadlock resulting from the consumer writing re-entrant logic in their own callback constitutes API misuse, not a security vulnerability exploitable by an external attacker. Furthermore, the library cannot reasonably prevent a caller's callback from re-entering the library or panicking without violating performance or design constraints typical for such synchronization primitives.",
    "round": 6,
    "file": "lbclient.go",
    "finding_title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "elapsed": 98.2,
    "tokens_in": 3153,
    "tokens_out": 167,
    "cost_usd": 0.00182526
  }
]
Raw scan report for this file

## Security analysis

This file implements a load-balancing client wrapper. The main trust boundary is the public `LBClient` API: `Do`, `DoTimeout`, `DoDeadline`, `AddClient`, and `RemoveClients`. The wrapper forwards caller-supplied `Request`/`Response` values to user-provided `BalancingClient` implementations and user-provided callbacks. The most relevant impact here is availability: panics, nil dereferences, slice out-of-range errors, and deadlocks in the host process.

### Data flow

- `LBClient.Do`
  - reads `cc.Timeout`
  - calls `DoTimeout`

- `LBClient.DoTimeout`
  - computes a deadline
  - calls `get()`
  - calls the selected `lbClient.DoDeadline`

- `LBClient.DoDeadline`
  - calls `get()`
  - calls the selected `lbClient.DoDeadline`

- `LBClient.get`
  - lazily initializes via `init()`
  - reads `cc.cs` under `RLock`
  - selects one `*lbClient`
  - releases the lock
  - returns the selected client

- `lbClient.DoDeadline`
  - calls `c.c.DoDeadline`
  - evaluates health
  - updates `penalty` or `total`

### Notable issues

1. **`RemoveClients` can leave zero clients**
   - `RemoveClients` removes clients based on `rc`.
   - If `rc` returns `true` for every client, `cc.cs` is truncated to length zero.
   - `get()` then executes `minC := cs[0]` without checking `len(cs) > 0`.
   - This causes a slice index out-of-range panic on the next request.

2. **Nil `BalancingClient` is not rejected**
   - `init()` and `AddClient()` store `BalancingClient` values directly.
   - If a nil `BalancingClient` is present, `lbClient.DoDeadline` calls `c.c.DoDeadline` on a nil interface.
   - This causes a nil pointer dereference panic.

3. **`RemoveClients` invokes a callback while holding the write lock**
   - `rc` is called while `cc.mu` is locked.
   - If `rc` reentrantly calls `AddClient`, `RemoveClients`, or `Do`/`DoTimeout`/`DoDeadline`, it can block on the same mutex and deadlock.
   - If `rc` panics, `cc.mu.Unlock` is not deferred, so the mutex remains locked forever.

4. **Lazy init can panic if `Clients` is empty but `AddClient` was used**
   - `init()` checks only `len(cc.Clients)`, not `len(cc.cs)`.
   - A zero-value `LBClient` can have clients added via `AddClient` before the first request.
   - The first `get()` still calls `init()`, which panics because `cc.Clients` is empty.

5. **Nil `req`/`resp` are forwarded unchecked**
   - The public entry points accept `req` and `resp` without nil checks.
   - They are passed to `c.c.DoDeadline` and, during health evaluation, to `c.healthCheck`.
   - If the downstream client or callback assumes non-nil pointers, this can panic.

6. **Lower-confidence concurrency issues**
   - `RemoveClients` does not check whether `rc` is nil.
   - Exported fields such as `HealthCheck` and `Timeout` can be mutated concurrently with method calls, even though methods are documented as safe for concurrent use.

### Non-issues / lower-confidence observations

- `penalty` and `total` use atomic operations consistently.
- `decPenalty` subtracts one from `penalty`; under normal operation this should not underflow because decrements are only created by successful `incPenalty` calls.
- `time.AfterFunc` timers are bounded per client by `maxPenalty`, so they are not an obvious unbounded goroutine/timer leak under normal use.
- No fixed-size buffer copies exist in this file.

```json
[
  {
    "severity": "high",
    "title": "Empty client list causes slice index out-of-range panic",
    "function": "LBClient.get",
    "description": "RemoveClients can truncate cc.cs to length zero if rc returns true for every client. A later Do, DoTimeout, or DoDeadline calls get, which executes minC := cs[0] without checking len(cs) > 0, causing a runtime panic and host process crash."
  },
  {
    "severity": "high",
    "title": "Nil BalancingClient causes nil pointer dereference",
    "function": "lbClient.DoDeadline",
    "description": "LBClient.init and LBClient.AddClient store nil BalancingClient values without validation. When get selects such an lbClient, DoDeadline calls c.c.DoDeadline on a nil interface, causing a panic. If client lists or AddClient input can be influenced by untrusted data, this is a DoS."
  },
  {
    "severity": "high",
    "title": "RemoveClients callback invoked while holding write lock can deadlock or lock permanently",
    "function": "LBClient.RemoveClients",
    "description": "rc is called while cc.mu is write-locked and without a deferred Unlock. If rc reentrantly calls AddClient, RemoveClients, or Do/DoTimeout/DoDeadline, it blocks on the same mutex and deadlocks. If rc panics, the mutex is not released, permanently locking the LBClient."
  },
  {
    "severity": "medium",
    "title": "Lazy init panics when Clients is empty even if AddClient populated cs",
    "function": "LBClient.init",
    "description": "get calls init via sync.Once. init only checks len(cc.Clients), not len(cc.cs). A zero-value LBClient can have one or more clients added with AddClient before the first Do; the later init still panics because cc.Clients is empty, crashing the host process."
  },
  {
    "severity": "medium",
    "title": "Nil request or response forwarded unchecked to client and health callback",
    "function": "lbClient.DoDeadline",
    "description": "DoDeadline, DoTimeout, and Do accept req and resp without nil checks and pass them to c.c.DoDeadline and, during health evaluation, to c.healthCheck. If the BalancingClient implementation or HealthCheck assumes non-nil pointers, an attacker-supplied nil request or response can trigger a panic."
  },
  {
    "severity": "low",
    "title": "RemoveClients accepts nil callback and panics without releasing lock",
    "function": "LBClient.RemoveClients",
    "description": "The rc parameter is invoked without a nil check. Passing nil causes a nil function call panic; because Unlock is not deferred, the panic leaves cc.mu locked. This is a lower-severity caller-error path, but it can be triggered if rc comes from untrusted configuration or code."
  },
  {
    "severity": "low",
    "title": "Exported field mutation can race with AddClient and Do",
    "function": "LBClient.AddClient",
    "description": "AddClient reads cc.HealthCheck while holding the mutex, but HealthCheck is an exported field that callers may assign to concurrently. Do also reads Timeout without synchronization. The type documents concurrent method safety, but unsynchronized exported field updates can cause data races and inconsistent health-checking behavior."
  }
]
```