Unbounded pre-Flush response buffering allows memory exhaustion

[
  {
    "finding_title": "Unbounded pre-Flush response buffering allows memory exhaustion",
    "verdict": "VALID",
    "reasoning": "The reported vulnerability is a lack of size limit on the `w.responseBody` buffer before a `Flush` occurs. In the code, `writer.Write` appends to `w.responseBody` without checking the total size. If a `net/http` handler echoes the request body or generates a large response without calling `Flush` (or `Hijack`), `w.responseBody` will grow in memory. Since `NewFastHTTPHandlerFunc` is a public API that wraps arbitrary `http.Handler`s, and the `http.Handler` can be influenced by untrusted input (e.g., echoing large request bodies), an attacker can trigger large memory allocations in the host process. There is no defense in the `fasthttpadaptor` code itself that limits this buffer size. The threat model for a library includes 'resource bounds (unbounded allocation... reachable through the public API with a crafted payload)'. The ability to cause Out-Of-Memory (OOM) via unbounded buffering is a valid security concern for a reusable library component, especially when the library is used to bridge to potentially unsafe or untrusted upstream handlers. The bug pattern (unbounded append) is real, it is reachable via the public API, and it leads to memory exhaustion. Therefore, it is a valid security vulnerability in the context of the library's threat model.\n\nCRUX: The `writer.Write` method appends to `w.responseBody` without a size limit, allowing unbounded memory growth if the wrapped handler does not call Flush, leading to potential OOM via crafted large requests.\nGREP: w.responseBody = append(w.responseBody, p...)",
    "elapsed": 6.9,
    "tokens": 4835,
    "tokens_in": 4485,
    "tokens_out": 350,
    "cost_usd": 0.0029337,
    "file": "fasthttpadaptor/adaptor.go",
    "round": 1,
    "grep_used": true,
    "grep_results": "GREP `responseBody`:\n```\nfasthttpadaptor/adaptor.go:107:\t\t\t\tl := min(len(w.responseBody), 512)\nfasthttpadaptor/adaptor.go:109:\t\t\t\t\tctx.Response.Header.Set(fasthttp.HeaderContentType, http.DetectContentType(w.responseBody[:l]))\nfasthttpadaptor/adaptor.go:112:\t\t\tif len(w.responseBody) > 0 {\nfasthttpadaptor/adaptor.go:113:\t\t\t\tctx.Response.SetBody(w.responseBody)\nfasthttpadaptor/adaptor.go:136:\t\t\t\tif len(w.responseBody) > 0 {\nfasthttpadaptor/adaptor.go:137:\t\t\t\t\tl := min(len(w.responseBody), 512)\nfasthttpadaptor/adaptor.go:138:\t\t\t\t\tctx.Response.Header.Set(fasthttp.HeaderContentType, http.DetectContentType(w.responseBody[:l]))\nfasthttpadaptor/adaptor.go:206:\tresponseBody []byte\nfasthttpadaptor/adaptor.go:227:\t\tresponseBody: nil,\nfasthttpadaptor/adaptor.go:265:\tif w.responseBody == nil {\nfasthttpadaptor/adaptor.go:267:\t\tw.responseBody = (*w.bufPool)[:0]\nfasthttpadaptor/adaptor.go:269:\tw.responseBody = append(w.responseBody, p...)\nfasthttpadaptor/adaptor.go:352:\tif len(w.responseBody) == 0 {\nfasthttpadaptor/adaptor.go:355:\tout := w.responseBody\nfasthttpadaptor/adaptor.go:356:\tw.responseBody = nil\nserver_timing_test.go:128:\tresponseBody := []byte(\"123\")\nserver_timing_test.go:134:\t\t\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:141:\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:155:\tresponseBody := []byte(\"123\")\nserver_timing_test.go:158:\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:174:\tresponseBody := []byte(\"123\")\nserver_timing_test.go:180:\t\t\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:183:\t\t\t\tctx.Success(\"foobar\", responseBody)\nhttp.go:22:\tresponseBodyPoolSizeLimit = -1\nhttp.go:29:\tresponseBodyPoolSizeLimit = respBodyLimit\nhttp.go:102:\tw     responseBodyWriter\nhttp.go:376:type responseBodyWriter struct {\nhttp.go:380:func (w *responseBodyWriter) Write(p []byte) (int, error) {\nhttp.go:385:func (w *responseBodyWriter) WriteString(s string) (int, error) {\nhttp.go:470:\t\tresp.body = responseBodyPool.Get()\n```"
  },
  {
    "finding_title": "Unbounded pre-Flush response buffering allows memory exhaustion",
    "verdict": "INVALID",
    "reasoning": "The reported vulnerability claims that the `fasthttpadaptor` package allows for unbounded memory allocation (OOM) because the `writer.Write` method appends to `w.responseBody` without a size limit before a `Flush` occurs. While the code analysis confirms that `w.responseBody` does grow unboundedly if the wrapped `http.Handler` writes large amounts of data without flushing, this behavior is a deliberate design choice inherent to the `net/http` `ResponseWriter` interface when buffering is required. In standard `net/http`, if a handler does not flush, the response is buffered. The fasthttp adaptor mirrors this by buffering into `w.responseBody` so that it can set the `Content-Length` header and detect content type once the handler completes, or until a `Flush` is called which switches to streaming mode. \n\nCrucially, the threat model for this library identifies the attacker as 'Code that calls the public API... parsers, decoders... receive adversarial payloads'. The `fasthttpadaptor` is a bridge. The 'untrusted input' in this context is the data passed to the wrapped handler. If a developer wraps a malicious or poorly designed `http.Handler` that echoes a huge request body into the response without flushing, the memory usage will indeed be large. However, this is not a flaw in the `fasthttpadaptor` library itself; it is a consequence of the behavior of the wrapped handler and the semantics of HTTP response buffering. The library correctly implements the contract of `http.ResponseWriter`. Limiting the buffer size would break the `net/http` compatibility (e.g., for handlers that need to build the full response before sending headers) and is not a security vulnerability in the adaptor. The 'bug' is actually the expected behavior for a buffering adapter. Furthermore, any `http.Handler` can be written to stream, but the adaptor supports both modes. The lack of an artificial cap is not a security vulnerability because the memory consumption is proportional to the application's own logic, not an out-of-bounds access or corruption. It is a resource usage issue dependent on the wrapped code, which is outside the scope of a library vulnerability unless the library itself introduces a flaw (like a buffer overflow). Here, the memory is safely managed by Go's runtime. Therefore, this is a false positive / code quality/design choice rather than a security vulnerability.\n\nCRUX: The unbounded growth of `w.responseBody` is intended buffering behavior required to support standard `net/http` handler semantics (setting Content-Length) before the first Flush, and does not constitute a memory safety vulnerability in the library itself.\nGREP: w.responseBody = append(w.responseBody, p...)",
    "elapsed": 10.2,
    "tokens": 6178,
    "tokens_in": 5611,
    "tokens_out": 567,
    "cost_usd": 0.00405762,
    "file": "fasthttpadaptor/adaptor.go",
    "round": 2,
    "grep_used": true,
    "grep_results": "GREP `responseBody`:\n```\nfasthttpadaptor/adaptor.go:107:\t\t\t\tl := min(len(w.responseBody), 512)\nfasthttpadaptor/adaptor.go:109:\t\t\t\t\tctx.Response.Header.Set(fasthttp.HeaderContentType, http.DetectContentType(w.responseBody[:l]))\nfasthttpadaptor/adaptor.go:112:\t\t\tif len(w.responseBody) > 0 {\nfasthttpadaptor/adaptor.go:113:\t\t\t\tctx.Response.SetBody(w.responseBody)\nfasthttpadaptor/adaptor.go:136:\t\t\t\tif len(w.responseBody) > 0 {\nfasthttpadaptor/adaptor.go:137:\t\t\t\t\tl := min(len(w.responseBody), 512)\nfasthttpadaptor/adaptor.go:138:\t\t\t\t\tctx.Response.Header.Set(fasthttp.HeaderContentType, http.DetectContentType(w.responseBody[:l]))\nfasthttpadaptor/adaptor.go:206:\tresponseBody []byte\nfasthttpadaptor/adaptor.go:227:\t\tresponseBody: nil,\nfasthttpadaptor/adaptor.go:265:\tif w.responseBody == nil {\nfasthttpadaptor/adaptor.go:267:\t\tw.responseBody = (*w.bufPool)[:0]\nfasthttpadaptor/adaptor.go:269:\tw.responseBody = append(w.responseBody, p...)\nfasthttpadaptor/adaptor.go:352:\tif len(w.responseBody) == 0 {\nfasthttpadaptor/adaptor.go:355:\tout := w.responseBody\nfasthttpadaptor/adaptor.go:356:\tw.responseBody = nil\nserver_timing_test.go:128:\tresponseBody := []byte(\"123\")\nserver_timing_test.go:134:\t\t\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:141:\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:155:\tresponseBody := []byte(\"123\")\nserver_timing_test.go:158:\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:174:\tresponseBody := []byte(\"123\")\nserver_timing_test.go:180:\t\t\t\t\tctx.Success(\"foobar\", responseBody)\nserver_timing_test.go:183:\t\t\t\tctx.Success(\"foobar\", responseBody)\nhttp.go:22:\tresponseBodyPoolSizeLimit = -1\nhttp.go:29:\tresponseBodyPoolSizeLimit = respBodyLimit\nhttp.go:102:\tw     responseBodyWriter\nhttp.go:376:type responseBodyWriter struct {\nhttp.go:380:func (w *responseBodyWriter) Write(p []byte) (int, error) {\nhttp.go:385:func (w *responseBodyWriter) WriteString(s string) (int, error) {\nhttp.go:470:\t\tresp.body = responseBodyPool.Get()\n```"
  },
  {
    "finding_title": "Unbounded pre-Flush response buffering allows memory exhaustion",
    "verdict": "INVALID",
    "reasoning": "The reported vulnerability is 'Unbounded pre-Flush response buffering allows memory exhaustion'. The code confirms that `writer.Write` appends data to `w.responseBody` without a size limit until `Flush` is called. However, the threat model explicitly defines the attacker as external clients sending data to the public API, and lists 'Misuse that violates a documented precondition of the public API' as NOT a vulnerability. The `fasthttpadaptor` library is a bridge that wraps user-provided `http.Handler`s. The amount of data written to the response is determined entirely by the user-provided handler logic, not by the library parsing untrusted input in a way that it fails to bound. In standard `net/http`, if a handler does not flush, the response is buffered. The adaptor mirrors this behavior to support Content-Length setting and content-type detection. An attacker cannot force the *handler* to write more data than the handler's logic dictates based on the request. While a malicious or buggy handler *could* cause OOM, this is a property of the handler, not a security flaw in the adaptor library itself. The library correctly implements the `http.ResponseWriter` interface. Requiring the library to cap the response buffer would break standard `net/http` semantics (where buffering is expected until flush). Therefore, this is a design choice consistent with the wrapped interface, not a security vulnerability in the library. It falls under code quality/design or potential resource exhaustion due to poor handler design, but not a vulnerability in the adaptor reachable by an attacker bypassing library defenses.\n\nCRUX: The unbounded buffering is a direct consequence of implementing the `http.ResponseWriter` interface semantics for user-provided handlers, not a failure of the library to validate untrusted input.\nGREP: func (w *writer) Write(p []byte)",
    "elapsed": 6.7,
    "tokens": 6657,
    "tokens_in": 6273,
    "tokens_out": 384,
    "cost_usd": 0.00378666,
    "file": "fasthttpadaptor/adaptor.go",
    "round": 3,
    "grep_used": true,
    "grep_results": "GREP `func (w *writer) Write(p []byte) (simplified to: writer)`:\n```\nbrotli.go:119:\t\t// These writers don't block, so we can just use stacklessWriteBrotli\nfasthttpadaptor/adaptor.go:173:\t\t\t// Signal the writer that streaming is ready so Flush() can return.\nfasthttpadaptor/adaptor.go:200:type writer struct {\nfasthttpadaptor/adaptor.go:222:func acquireWriter(ctx *fasthttp.RequestCtx) *writer {\nfasthttpadaptor/adaptor.go:224:\treturn &writer{\nfasthttpadaptor/adaptor.go:235:func releaseWriter(w *writer) {\nfasthttpadaptor/adaptor.go:243:func (w *writer) Header() http.Header {\nfasthttpadaptor/adaptor.go:247:func (w *writer) WriteHeader(code int) {\nfasthttpadaptor/adaptor.go:255:func (w *writer) Write(p []byte) (int, error) {\nfasthttpadaptor/adaptor.go:273:func (w *writer) Flush() {\nfasthttpadaptor/adaptor.go:298:func (w *writer) Hijack() (net.Conn, *bufio.ReadWriter, error) {\nfasthttpadaptor/adaptor.go:331:func (w *writer) Close() error {\nfasthttpadaptor/adaptor.go:340:func (w *writer) status() int {\nfasthttpadaptor/adaptor.go:349:func (w *writer) consumePreflush() []byte {\nfs_test.go:39:func TestNewVHostPathRewriter(t *testing.T) {\nfs_test.go:48:\tf := NewVHostPathRewriter(0)\nfs_test.go:57:\tf = NewVHostPathRewriter(2)\nfs_test.go:65:func TestNewVHostPathRewriterMaliciousHost(t *testing.T) {\nfs_test.go:74:\tf := NewVHostPathRewriter(0)\nfasthttputil/pipeconns_test.go:209:\t\t\tt.Fatalf(\"unexpected error in writer: %v\", err)\nstackless/writer_test.go:75:\t\t\treturn fmt.Errorf(\"unexpected error when re-using writer on iteration %d: %w\", i, err)\nstackless/writer.go:12:// Writer is an interface stackless writer must conform to.\nstackless/writer.go:22:// NewWriterFunc must return new writer that will be wrapped into\nstackless/writer.go:23:// stackless writer.\nstackless/writer.go:26:// NewWriter creates a stackless writer around a writer returned\nstackless/writer.go:29:// The returned writer writes data to dstW.\nstackless/writer.go:31:// Writers that use a lot of stack space may be wrapped into stackless writer,\nstackless/writer.go:34:\tw := &writer{\nstackless/writer.go:41:type writer struct {\nstackless/writer.go:63:func (w *writer) Write(p []byte) (int, error) {\n```"
  },
  {
    "finding_title": "Unbounded pre-Flush response buffering allows memory exhaustion",
    "verdict": "UNCERTAIN",
    "reasoning": "The reported finding suggests that the `fasthttpadaptor` library allows for unbounded memory growth in the `w.responseBody` buffer if a wrapped `http.Handler` writes large amounts of data without calling `Flush`. \n\n1. **Bug Pattern**: The code in `writer.Write` does indeed append to `w.responseBody` without a size check. This is a buffering mechanism required to support `http.ResponseWriter` semantics (specifically `Content-Length` and `DetectContentType`) before the first flush. \n2. **Attack Surface**: The `fasthttpadaptor` is a bridge library. It takes a `http.Handler` provided by the application developer (`h`) and wraps it into a `fasthttp.RequestHandler`. The data written to the response is generated by this developer-provided handler `h`. \n3. **Threat Model Analysis**: The threat model for a library defines the attacker as code calling the public API or untrusted data passed through it. In this case, the `http.Handler` is a trusted component provided by the host application, not an untrusted input. The library is acting as a proxy for the handler's I/O. \n4. **Attribution**: If the handler `h` echoes a large request body without flushing, the memory consumption is a consequence of the handler's logic (or the underlying `net/http` standard buffering behavior). The library is correctly implementing the `http.ResponseWriter` interface. The lack of a buffer cap in the adapter is a design choice consistent with `net/http` semantics (which also buffers until flush). Attributing the OOM risk to the library rather than the application handler logic is incorrect. \n5. **Conclusion**: The ",
    "elapsed": 6.6,
    "tokens": 7193,
    "tokens_in": 6832,
    "tokens_out": 361,
    "cost_usd": 0.00395244,
    "file": "fasthttpadaptor/adaptor.go",
    "round": 4
  },
  {
    "finding_title": "Unbounded pre-Flush response buffering allows memory exhaustion",
    "verdict": "INVALID",
    "reasoning": "The finding claims that the `fasthttpadaptor` library has a vulnerability because `writer.Write` buffers data in `w.responseBody` without a size limit before `Flush` is called, potentially leading to Out-Of-Memory (OOM) if the wrapped handler echoes large input. While the code pattern is real (`w.responseBody = append(w.responseBody, p...)` has no upper bound), the threat model for this library defines the attacker as an external client sending untrusted data (requests) and the assets as the integrity of the host process. The `http.Handler` being wrapped is provided by the application developer and is considered trusted code. The amount of data written to the response is determined entirely by the logic of this trusted handler. In standard `net/http`, response buffering before the first `Flush` (or header send) is inherent to the API semantics (required for `Content-Length` calculation and content type detection). The adapter mirrors this behavior. If the application developer writes a handler that echoes the entire request body without flushing, the resulting memory usage is a consequence of the application's logic, not a flaw in the library's validation of untrusted input. The library does not parse or process the data in a way that amplifies the attack vector beyond what the handler itself does. Therefore, this is a design characteristic of the `net/http` interface rather than a security vulnerability in the adapter library itself. It does not constitute an exploitable vulnerability in the library where an attacker can bypass library defenses to cause harm.\n\nCRUX: The unbounded buffering is a required semantic of the net/http ResponseWriter interface to support Content-Length and content detection before flushing, and the data volume is controlled by the trusted application handler, not the library.\nGREP: func (w *writer) Write",
    "elapsed": 6.7,
    "tokens": 6933,
    "tokens_in": 6553,
    "tokens_out": 380,
    "cost_usd": 0.00389226,
    "file": "fasthttpadaptor/adaptor.go",
    "round": 5,
    "grep_used": true,
    "grep_results": "GREP `func (w *writer) Write (simplified to: writer)`:\n```\nbrotli.go:119:\t\t// These writers don't block, so we can just use stacklessWriteBrotli\nfasthttpadaptor/adaptor.go:173:\t\t\t// Signal the writer that streaming is ready so Flush() can return.\nfasthttpadaptor/adaptor.go:200:type writer struct {\nfasthttpadaptor/adaptor.go:222:func acquireWriter(ctx *fasthttp.RequestCtx) *writer {\nfasthttpadaptor/adaptor.go:224:\treturn &writer{\nfasthttpadaptor/adaptor.go:235:func releaseWriter(w *writer) {\nfasthttpadaptor/adaptor.go:243:func (w *writer) Header() http.Header {\nfasthttpadaptor/adaptor.go:247:func (w *writer) WriteHeader(code int) {\nfasthttpadaptor/adaptor.go:255:func (w *writer) Write(p []byte) (int, error) {\nfasthttpadaptor/adaptor.go:273:func (w *writer) Flush() {\nfasthttpadaptor/adaptor.go:298:func (w *writer) Hijack() (net.Conn, *bufio.ReadWriter, error) {\nfasthttpadaptor/adaptor.go:331:func (w *writer) Close() error {\nfasthttpadaptor/adaptor.go:340:func (w *writer) status() int {\nfasthttpadaptor/adaptor.go:349:func (w *writer) consumePreflush() []byte {\nfs_test.go:39:func TestNewVHostPathRewriter(t *testing.T) {\nfs_test.go:48:\tf := NewVHostPathRewriter(0)\nfs_test.go:57:\tf = NewVHostPathRewriter(2)\nfs_test.go:65:func TestNewVHostPathRewriterMaliciousHost(t *testing.T) {\nfs_test.go:74:\tf := NewVHostPathRewriter(0)\nstackless/writer_test.go:75:\t\t\treturn fmt.Errorf(\"unexpected error when re-using writer on iteration %d: %w\", i, err)\nstackless/writer.go:12:// Writer is an interface stackless writer must conform to.\nstackless/writer.go:22:// NewWriterFunc must return new writer that will be wrapped into\nstackless/writer.go:23:// stackless writer.\nstackless/writer.go:26:// NewWriter creates a stackless writer around a writer returned\nstackless/writer.go:29:// The returned writer writes data to dstW.\nstackless/writer.go:31:// Writers that use a lot of stack space may be wrapped into stackless writer,\nstackless/writer.go:34:\tw := &writer{\nstackless/writer.go:41:type writer struct {\nstackless/writer.go:63:func (w *writer) Write(p []byte) (int, error) {\nstackless/writer.go:70:func (w *writer) WriteString(s string) (int, error) {\n```"
  },
  {
    "verdict": "INVALID",
    "reasoning": "[ARBITER] The 'unbounded buffering' is a required semantic of the `http.ResponseWriter` interface to support `Content-Length` and content-type detection before the first `Flush`. The library acts as a transparent bridge; the size of the response data is determined by the application handler's logic (trusted code) or the client's request (which is typically bounded by server configuration, e.g., max request body size, before reaching the handler). The library itself does not process untrusted input to generate response size without the handler's involvement. Therefore, this is not a vulnerability in the library but rather a responsibility of the application developer to limit response sizes, falling under 'misuse that violates a documented precondition' (implicit or explicit) of managing resource bounds in the application layer, or simply standard interface behavior. The library cannot arbitrarily cap the buffer without breaking compatibility with large legitimate responses or altering semantics.",
    "round": 6,
    "file": "fasthttpadaptor/adaptor.go",
    "finding_title": "Unbounded pre-Flush response buffering allows memory exhaustion",
    "elapsed": 5.1,
    "tokens_in": 6595,
    "tokens_out": 195,
    "cost_usd": 0.0033549
  }
]