ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge
medium swesweep-fasthttp · http.go · ContinueReadBodyStream · run #4
Description
When readBodyWithStreaming detects that contentLength exceeds maxBodySize, the function returns nil (no error) and sets req.bodyStream to a requestStream that will continue reading the full oversized body from the network. The Content-Length header is set to the original (oversized) value. Downstream code that uses Content-Length to pre-allocate buffers or that assumes the body is within the configured limit will process an unbounded amount of data, defeating the purpose of the size limit.
Triage
- Verdict
- INVALID confidence 0.17
- Model
- qwen/qwen3.8-27b
- Rounds
- 6
- Cost
- $0.0847 · 859.5 s
[ARBITER] The reported behavior is the intended design of the `ContinueReadBodyStream` API. The function is specifically designed for streaming request bodies (e.g., when `StreamRequestBody` is enabled or `Expect: 100-continue` is used). In this mode, `maxBodySize` limits the initial pre-read buffer to avoid loading the entire body into memory, but the library intentionally provides the `bodyStream` to the application so the handler can process the body incrementally. The `ContinueReadBody` function (non-streaming) strictly enforces the limit by returning `ErrBodyTooLarge`. The `ContinueReadBodyStream` function returns `nil` with a stream to allow the application to control how much of the oversized body to read/process; it does not silently buffer unbounded data into memory, nor does it defeat the purpose of the limit in the non-streaming context. Therefore, it is not a security vulnerability.
All rounds
[
{
"finding_title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"verdict": "INVALID",
"reasoning": "The report claims that `ContinueReadBodyStream` silently returns an oversized body stream when `contentLength > maxBodySize`, setting no error and allowing downstream code to read the full body. \n\n1. **Trace the code flow**: In `ContinueReadBodyStream`, `req.readBodyWithStreaming` is called with `maxBodySize`. \n2. **Inspect `readBodyWithStreaming`**: The function reads a limited amount of data (up to `maxBodySize` or `8*1024`) into `dst`. It then checks `if contentLength > maxBodySize`. If true, it returns `ErrBodyTooLarge`. \n3. **Inspect `ContinueReadBodyStream`'s handling of `ErrBodyTooLarge`**: The code explicitly handles this error: \n ```go\n if err != nil {\n if err == ErrBodyTooLarge {\n req.Header.SetContentLength(contentLength)\n req.body = bodyBuf\n req.bodyStream = acquireRequestStream(bodyBuf, r, &req.Header)\n return nil\n }\n ...\n }\n ```\n This confirms that when the body is too large, `req.bodyStream` *is* set to a `requestStream` and the function returns `nil` (no error). The scanner is correct about the *behavior*: it creates a stream and doesn't return an error.\n\n4. **Security Relevance Assessment**: \n - **Is this a vulnerability?** The purpose of `maxBodySize` in `ContinueReadBodyStream` is to prevent loading the *entire* large body into memory all at once. By setting up a stream, the library avoids an unbounded memory allocation for the `req.body` buffer. The `requestStream` type is designed to read from the underlying `r` (bufio.Reader) chunk by chunk as the user calls `Read()`.\n - **Does it defeat the size limit?** The `requestStream` will continue to read from the network. If the user (the host application) ignores the fact that the body was too large and simply reads from the stream, they *will* read the full oversized body. However, `ContinueReadBodyStream` is a specific API intended for streaming. The documentation for `ContinueReadBody` (the non-streaming version) says it returns `ErrBodyTooLarge`. The streaming version's contract is different: it sets up the stream for the user to manage. \n - **Crucially**: Does the `requestStream` enforce any limit? `acquireRequestStream` creates a stream that reads from `r`. It does *not* know about `maxBodySize`. So the stream *can* read the full body. \n - **However**, is this a security bug in the library? The library's role is to provide the mechanism. If the caller asks for a stream with a max size limit, and the body is larger, the library *could* either: \n a) Return an error (like `ContinueReadBody` does), preventing the caller from having the stream at all. \n b) Provide a stream that enforces the limit. \n c) Provide a stream that ignores the limit (current behavior). \n \n Let's look at `readBodyWithStreaming` again. It pre-reads up to `min(maxBodySize, contentLength, 8192)`. If `contentLength > maxBodySize`, it returns `ErrBodyTooLarge`. The `ContinueReadBodyStream` function *catches* this error and turns it into a stream. This implies that the library *intends* to allow streaming even when the body is larger than `maxBodySize`, but only up to the point where the pre-read happens? No, `acquireRequestStream` creates a stream that continues reading from `r`. \n \n The key question is: **Is this behavior a vulnerability or intended design?** \n In fasthttp, `ContinueReadBodyStream` is used by servers that want to stream large bodies. The `maxBodySize` parameter in this context is often used to limit the *initial* buffer allocation or to trigger a specific code path. If the server sets a `MaxRequestBodySize`, it usually expects the library to reject or limit the body. \n \n However, looking at the standard usage of `Request.ReadLimitBody`, which is the primary entry point for servers: \n `func (req *Request) ReadLimitBody(r *bufio.Reader, maxBodySize int) error` calls `req.readLimitBody(r, maxBodySize, false, true)`, which calls `req.ContinueReadBody` (not the streaming one). `ContinueReadBody` *does* return `ErrBodyTooLarge`. \n \n `ContinueReadBodyStream` is a separate, lower-level function. It is not the default path for `Read` or `ReadLimitBody`. It must be called explicitly by the user. If a user calls `ContinueReadBodyStream` with a `maxBodySize`, they are opting into streaming. The fact that it returns `nil` and sets a stream that *can* read more than `maxBodySize` is arguably a design choice to allow the user to handle the stream (e.g., to drain it or to read it partially). \n \n But does it allow an attacker to bypass a size limit? If a host application uses `ContinueReadBodyStream` and then checks `if err == nil` to assume the body is within limits, they would be mistaken. But `ContinueReadBodyStream` is an internal-looking helper (it's exported, but the primary API is `Read`/`ReadLimitBody`). \n \n Let's check if `Request.Read` uses this. \n `func (req *Request) Read(r *bufio.Reader) error` -> `req.ReadLimitBody(r, 0)`. Since `maxBodySize` is 0, `readBodyWithStreaming` will not trigger `ErrBodyTooLarge` (checks `if maxBodySize > 0`). \n \n If a server configures `MaxRequestBodySize` (which sets a limit), how does it apply? \n In `server.go`, `MaxRequestBodySize` is typically used to configure the `maxBodySize` passed to `ReadLimitBody`. \n \n If the library's primary API `ReadLimitBody` returns an error, but a secondary API `ContinueReadBodyStream` allows streaming past the limit, is that a vulnerability? \n The threat model says: \"Crash, memory corruption or unbounded resource use reachable through the public API with a crafted payload is VALID.\" \n If an attacker sends a huge body, and the server uses `ContinueReadBodyStream` (perhaps for specific endpoint handling), the server *can* end up processing the huge body. But the server *chose* to use the streaming API. The library's `readBodyWithStreaming` pre-reads only `min(maxBodySize, ...)`. So it doesn't *allocate* the full body into memory immediately. It just sets up the stream. The *allocation* happens as the user reads. \n \n The scanner says: \"Downstream code that uses Content-Length to pre-allocate buffers... will process an unbounded amount of data.\" \n This is a consequence of the API design, not a flaw in the library's memory safety. The library does not *force* the user to allocate unbounded memory. It provides a stream. If the user reads from it, they read the data. \n \n Comparing to `ContinueReadBody`: It returns an error. \n Comparing to `ContinueReadBodyStream`: It returns nil. \n \n Is the *absence* of an error a security bug? If the user *wants* to limit the body size, they should use `ContinueReadBody` (which errors) or check the stream status. However, `ContinueReadBodyStream` *does* check `maxBodySize` to determine the *pre-read* size. It returns `ErrBodyTooLarge` internally, which is then *suppressed* in `ContinueReadBodyStream` to allow streaming. \n \n This behavior allows a server to stream bodies larger than `maxBodySize`. This is a feature, not a bug, for streaming. The `maxBodySize` in streaming context likely limits the *initial buffer* or the *threshold* for switching to file/stream, not a hard cap on total data. \n \n Therefore, this is not a security vulnerability in the sense of memory corruption or unintended bypass of a security control that the library *promises* to enforce against all inputs. The library enforces the limit for `Read`/`ReadLimitBody`. `ContinueReadBodyStream` is a specialized function for streaming, where the caller takes responsibility for handling the body. The \"silent return\" is the intended mechanism to hand off the stream to the caller.\n\n Verdict: INVALID. The behavior is intentional for streaming APIs, where the caller manages the stream. It does not lead to memory corruption or unintended resource exhaustion by the library itself (the pre-read is bounded).\n\nCRUX: Whether ContinueReadBodyStream is intended to allow streaming of bodies larger than maxBodySize by design (returning the stream to the caller) or if it is a bypass of the size limit.\nGREP: func (req \\*Request) ContinueReadBodyStream",
"elapsed": 204.0,
"tokens": 26620,
"tokens_in": 24577,
"tokens_out": 2043,
"cost_usd": 0.01645134,
"file": "http.go",
"round": 1,
"grep_used": true,
"grep_results": "GREP `func (req *Request) ContinueReadBodyStream (simplified to: ContinueReadBodyStream)`:\n```\nserver.go:2527:\t\t\t\t\terr = ctx.Request.ContinueReadBodyStream(br, maxRequestBodySize, !s.DisablePreParseMultipartForm)\nhttp_test.go:1191:\terr := r.ContinueReadBodyStream(br, 0)\nhttp.go:1392:\treturn req.ContinueReadBodyStream(r, maxBodySize, preParseMultipartForm)\nhttp.go:1495:// ContinueReadBodyStream reads request body if request header contains\nhttp.go:1502:func (req *Request) ContinueReadBodyStream(r *bufio.Reader, maxBodySize int, preParseMultipartForm ...bool) error {\n```"
},
{
"finding_title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"verdict": "VALID",
"reasoning": "The report claims that `ContinueReadBodyStream` silently returns an oversized body stream on `ErrBodyTooLarge`, allowing downstream code to process unbounded data. The code analysis confirms this behavior: in `ContinueReadBodyStream`, when `readBodyWithStreaming` returns `ErrBodyTooLarge` (which happens if `contentLength > maxBodySize`), the function catches the error, sets `req.bodyStream` to a new `requestStream`, and returns `nil` (no error). The `requestStream` does not enforce the `maxBodySize` limit; it reads from the underlying connection until EOF or connection close. \n\nTo determine if this is a security vulnerability, we must look at how this function is used in the standard fasthttp server flow. The Grep results show that `server.go` calls `ctx.Request.ContinueReadBodyStream(br, maxRequestBodySize, ...)` where `maxRequestBodySize` is the server's configured limit (e.g., via `s.MaxRequestBodySize`). This is the standard code path for handling request bodies in the fasthttp server (specifically in `serveConn` or similar request processing loops when streaming is enabled or when the body is not pre-read entirely). \n\nIf an attacker sends a request with a `Content-Length` header that exceeds the server's `MaxRequestBodySize`, the server will call `ContinueReadBodyStream`. As shown, this function will NOT return an error to the request handler. Instead, it will set up the body stream. If the request handler then reads the body (e.g., via `ctx.Request.Body()` or `ctx.Request.BodyStream()`), it will read the entire oversized body from the network. \n\nThis defeats the purpose of `MaxRequestBodySize`. A server administrator sets a limit to prevent DoS attacks via large uploads. By silently allowing the stream to be created and read fully without error, the library allows an attacker to force the server to read and process arbitrarily large amounts of data, consuming memory (if buffered) or bandwidth/time. While the streaming nature prevents immediate unbounded allocation in the library itself (it reads chunk by chunk), it allows the host application to ingest data beyond the configured limit without the intended error signal (`ErrBodyTooLarge`) that would allow the app to reject the request. \n\nFurthermore, `Response.ReadBody` has similar logic where `ErrBodyTooLarge` is swallowed if `StreamBody` is set, but for requests, the lack of error return means the standard error-handling path in fasthttp (which would respond with 413 Payload Too Large) is bypassed. The library fails to enforce the size limit at the public API boundary (`Request.Read` / server internals). This is a valid security issue because it allows bypassing a configured resource limit, leading to potential denial of service.\n\nCRUX: ContinueReadBodyStream suppresses ErrBodyTooLarge and allows the full body to be streamed and read by the application, bypassing the server's MaxRequestBodySize limit without returning an error to reject the request.\nGREP: ContinueReadBodyStream",
"elapsed": 215.3,
"tokens": 27517,
"tokens_in": 26866,
"tokens_out": 651,
"cost_usd": 0.013236719999999999,
"file": "http.go",
"round": 2,
"grep_used": true,
"grep_results": "GREP `ContinueReadBodyStream`:\n```\nserver.go:2527:\t\t\t\t\terr = ctx.Request.ContinueReadBodyStream(br, maxRequestBodySize, !s.DisablePreParseMultipartForm)\nhttp_test.go:1191:\terr := r.ContinueReadBodyStream(br, 0)\nhttp.go:1392:\treturn req.ContinueReadBodyStream(r, maxBodySize, preParseMultipartForm)\nhttp.go:1495:// ContinueReadBodyStream reads request body if request header contains\nhttp.go:1502:func (req *Request) ContinueReadBodyStream(r *bufio.Reader, maxBodySize int, preParseMultipartForm ...bool) error {\n```"
},
{
"finding_title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"verdict": "UNCERTAIN",
"reasoning": "The report claims that `ContinueReadBodyStream` allows an attacker to bypass the `maxBodySize` limit by returning a stream that can be read beyond the limit. To verify this, I examined the code path: `ContinueReadBodyStream` calls `readBodyWithStreaming`, which returns `ErrBodyTooLarge` if `contentLength > maxBodySize`. The function catches this specific error, sets up a `requestStream`, and returns `nil`. This behavior is intended for the `StreamRequestBody` feature in the server, where the body is not fully read into memory but provided as a stream to the application. \n\nHowever, the critical security control is not the absence of the error in `ContinueReadBodyStream` itself, but how the server handles the body afterwards. I checked `server.go` for how the server enforces `MaxRequestBodySize`. Specifically, in the `serveOne` function (which processes requests), there is a check after `serveOne` returns: \n`if s.MaxRequestBodySize > 0 && req.body == nil && req.bodyStream != nil && ctx.IsBodyStream() { ... }` \nActually, looking closer at `server.go`, the check for oversized bodies in streaming mode is typically handled by the application or by specific logic if `StreamRequestBody` is disabled. Wait, let me look at the standard non-streaming path. \n\nThe standard path uses `ReadLimitBody` -> `ContinueReadBody`, which *does* return `ErrBodyTooLarge`. The `ContinueReadBodyStream` path is only used when `StreamRequestBody` is true. \n\nCrucially, if `StreamRequestBody` is true, the *application* is responsible for reading the body from the stream. The library's `MaxRequestBodySize` is still enforced in the non-streaming path. In the streaming path, the library provides the stream. Does it enforce the limit? \n\nLooking at `server.go` around line 2527, `ContinueReadBodyStream` is called. If the body is too large, `req.bodyStream` is set. The request handler then accesses `ctx.Request.Body()`. `Body()` calls `bodyBytes()`. `bodyBytes()` calls `copyZeroAlloc(bodyBuf, req.bodyStream)`. This copies the *entire* stream into memory. There is no limit check in `bodyBytes` or `copyZeroAlloc` regarding `MaxRequestBodySize`. \n\nHowever, the threat model states: \"Crash, memory corruption or unbounded resource use reachable through the public API with a crafted payload is VALID.\" \n\nIf a server enables `StreamRequestBody` and sets `MaxRequestBodySize`, an attacker can send a body larger than the limit. The server will not return 413. The application will read the entire body into memory (if it calls `Body()`) or process it via the stream. This defeats the purpose of `MaxRequestBodySize` as a DoS prevention mechanism. \n\nBUT, is `ContinueReadBodyStream` a public API that users are expected to use for limiting? It is used internally by the server. The user configures `MaxRequestBodySize`. If the user enables `StreamRequestBody`, they opt into streaming. The library's documentation for `MaxRequestBodySize` likely implies that the body will be limited. If it isn't, it's a bug. \n\nLet's check if there is a defense. In `server.go`, after `serveOne`, does it check the size? \n`server.go:2527` calls `ContinueReadBodyStream`. \nIf `ErrBodyTooLarge` was returned by `readBodyWithStreaming`, `ContinueReadBodyStream` suppresses it. \nThe server then proceeds to call the `RequestHandler`. \nIf the `RequestHandler` calls `ctx.Request.Body()`, the whole body is loaded. \nThere is no post-check in `server.go` to verify that the read body size was within `MaxRequestBodySize` for streaming requests. \n\nTherefore, the size limit is effectively bypassed for streaming requests. This allows unbounded memory allocation if the application reads the body. This is a security vulnerability (DoS). \n\nHowever, I must verify if `MaxRequestBodySize` is documented to work with `StreamRequestBody`. If `StreamRequestBody` is true, the body is *not* pre-read. The limit might be intended to prevent *pre-reading*. But if it doesn't prevent the handler from reading a huge body, it's a weak control. \n\nActually, looking at `fasthttp` documentation and common usage: `MaxRequestBodySize` is primarily for limiting memory usage by the *library* during parsing. If streaming is enabled, the library *intentionally* does not limit the total size to allow for large file uploads, but this puts the burden on the app. \n\nWait, the prompt asks if the *bug* is real. The bug is that `ContinueReadBodyStream` returns nil on `ErrBodyTooLarge`. This is factually true. Is it a security vulnerability? It allows bypassing a configured limit. \n\nLet's look at `Reviewer 1` vs `Reviewer 2`. \n`Reviewer 1` says it's invalid because it's a design choice for streaming. \n`Reviewer 2` says it's valid because it bypasses `MaxRequestBodySize`. \n\nLet's check the specific code in `server.go` for how `MaxRequestBodySize` is applied. \nIn `serveOne`, `maxRequestBodySize` is passed to `ContinueReadBodyStream`. \nIf the stream is created, the error is swallowed. \nThe request handler runs. \nIf the handler does not read the body, no memory is used. \nIf the handler reads the body, the full size is used. \n\nIs this a vulnerability? If I set `MaxRequestBodySize = 1MB` and `StreamRequestBody = true`, and an attacker sends `100MB`, my server will process `100MB`. My setting was ignored. This is a bypass of a security control. \n\nVerdict: VALID. The `MaxRequestBodySize` limit is not enforced for the total body size when `StreamRequestBody` is enabled, allowing DoS via memory exhaustion if the handler reads the body.",
"elapsed": 150.7,
"tokens": 28931,
"tokens_in": 27590,
"tokens_out": 1341,
"cost_usd": 0.015610799999999998,
"file": "http.go",
"round": 3
},
{
"finding_title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"verdict": "INVALID",
"reasoning": "The report claims that `ContinueReadBodyStream` silently returns an oversized body stream when the body exceeds `maxBodySize`, allowing downstream code to process unbounded data and defeating the purpose of the size limit. \n\n1. **Code Verification**: The code in `http.go` confirms that `ContinueReadBodyStream` catches `ErrBodyTooLarge` from `readBodyWithStreaming` and returns `nil` (no error) while setting `req.bodyStream` to a `requestStream` that can read the full body. This matches the scanner's description of the code behavior.\n\n2. **Threat Model Analysis**: The threat model defines the attacker as code calling the public API (or data passed through it) and the entry point as public functions. The key question is whether this behavior constitutes a security vulnerability (unbounded resource use/DoS) or an intended design choice.\n\n3. **Intent and Context**: `ContinueReadBodyStream` is explicitly a *streaming* API. In HTTP server implementations, `MaxRequestBodySize` (or `maxBodySize`) in the context of *non-streaming* reads (`ContinueReadBody`) is a hard limit to prevent loading large bodies into memory. However, for *streaming* reads, the purpose is often to handle large bodies (like file uploads) without holding them in memory. If the library hard-rejected any body larger than `maxBodySize` in the streaming path, it would make it impossible to stream bodies larger than that limit, which defeats the purpose of streaming.\n\n4. **Defenses and Contracts**: \n - The `requestStream` does not enforce `maxBodySize` on the total read; it allows reading until EOF or connection close. This is standard for streams.\n - The `requestStream` is accessible to the handler. If the handler calls `req.Body()`, it will load the entire stream into memory. However, this is the handler's responsibility. The library provides the *mechanism* for streaming. The `MaxRequestBodySize` in the non-streaming path is a memory safety guard. In the streaming path, the \"guard\" is that the library doesn't *automatically* allocate a huge buffer; it hands the stream to the user. \n - If a server administrator sets `MaxRequestBodySize` to 1MB and enables `StreamRequestBody`, they are explicitly opting into a mode where the body is not pre-read. If they then call `Body()` in the handler, they are violating the assumption that streaming should be used to avoid large memory allocations. The library is not *forcing* the user to read the whole body; the user must call `Body()` or `io.Copy` to do so. \n - Critically, the standard `Request.Read` / `ReadLimitBody` path (used by most non-streaming configurations) *does* return `ErrBodyTooLarge` and rejects the request. The `ContinueReadBodyStream` path is only used when `StreamRequestBody` is enabled or when specifically calling the streaming API. \n\n5. **Conclusion**: The behavior is intentional. The `maxBodySize` in `ContinueReadBodyStream` limits the *initial pre-read* buffer allocation (via `readBodyWithStreaming` reading `min(maxBodySize, contentLength, 8192)`), ensuring the library itself doesn't allocate a huge buffer internally. It then hands off the stream. If the application reads the entire stream, that is the application's choice. This is not a security vulnerability in the library because it does not lead to uncontrolled memory corruption or a bypass of a hard security control that the library *promises* to enforce for *all* inputs. The promise is to limit *internal* memory usage for the header/parsing phase, or to reject in non-streaming mode. In streaming mode, the control is delegated to the application. Therefore, this is a code quality/API design issue (potential confusion for users expecting hard limits in streaming mode) but not a security vulnerability under the provided threat model (no crash, no memory corruption, no unbounded allocation *by the library itself*).\n\nCRUX: Whether ContinueReadBodyStream is intended to allow streaming of bodies larger than maxBodySize by design (returning the stream to the caller) or if it is a bypass of the size limit.\nGREP: func (req \\*Request) ContinueReadBodyStream",
"elapsed": 139.4,
"tokens": 29760,
"tokens_in": 28827,
"tokens_out": 933,
"cost_usd": 0.014906339999999999,
"file": "http.go",
"round": 4,
"grep_used": true,
"grep_results": "GREP `func (req *Request) ContinueReadBodyStream (simplified to: ContinueReadBodyStream)`:\n```\nserver.go:2527:\t\t\t\t\terr = ctx.Request.ContinueReadBodyStream(br, maxRequestBodySize, !s.DisablePreParseMultipartForm)\nhttp_test.go:1191:\terr := r.ContinueReadBodyStream(br, 0)\nhttp.go:1392:\treturn req.ContinueReadBodyStream(r, maxBodySize, preParseMultipartForm)\nhttp.go:1495:// ContinueReadBodyStream reads request body if request header contains\nhttp.go:1502:func (req *Request) ContinueReadBodyStream(r *bufio.Reader, maxBodySize int, preParseMultipartForm ...bool) error {\n```"
},
{
"finding_title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"verdict": "INVALID",
"reasoning": "The report claims that `ContinueReadBodyStream` silently allows oversized bodies, defeating the size limit. While the code does catch `ErrBodyTooLarge` and return `nil` (enabling streaming), this is the intended behavior for the streaming API, which is designed to handle large bodies without loading them entirely into memory at once. The `maxBodySize` parameter in this context limits the initial pre-read buffer, not the total stream size. The threat model considers unbounded resource use via the public API as VALID, but here the library does not force unbounded memory allocation; it provides a stream. If a server application calls `Request.Body()` (which reads the entire stream into memory), that is an application-level choice to buffer the entire payload, not a library flaw. The standard non-streaming path (`ReadLimitBody` -> `ContinueReadBody`) correctly enforces the limit by returning an error. The streaming path is a specialized feature for cases where the application wants to process the body incrementally or reject it manually. Since the library does not introduce memory corruption or force unbounded allocation, and the 'bypass' is a delegation of responsibility to the handler for the specific streaming mode, this is not a security vulnerability in the library itself.\n\nCRUX: ContinueReadBodyStream is a streaming API where the maxBodySize limits the pre-read buffer, not the total stream size; reading the full stream is the responsibility of the caller.\nGREP: ContinueReadBodyStream",
"elapsed": 46.5,
"tokens": 30237,
"tokens_in": 29921,
"tokens_out": 316,
"cost_usd": 0.013514819999999999,
"file": "http.go",
"round": 5,
"grep_used": true,
"grep_results": "GREP `ContinueReadBodyStream`:\n```\nhttp.go:1392:\treturn req.ContinueReadBodyStream(r, maxBodySize, preParseMultipartForm)\nhttp.go:1495:// ContinueReadBodyStream reads request body if request header contains\nhttp.go:1502:func (req *Request) ContinueReadBodyStream(r *bufio.Reader, maxBodySize int, preParseMultipartForm ...bool) error {\nhttp_test.go:1191:\terr := r.ContinueReadBodyStream(br, 0)\nserver.go:2527:\t\t\t\t\terr = ctx.Request.ContinueReadBodyStream(br, maxRequestBodySize, !s.DisablePreParseMultipartForm)\n```"
},
{
"verdict": "INVALID",
"reasoning": "[ARBITER] The reported behavior is the intended design of the `ContinueReadBodyStream` API. The function is specifically designed for streaming request bodies (e.g., when `StreamRequestBody` is enabled or `Expect: 100-continue` is used). In this mode, `maxBodySize` limits the initial pre-read buffer to avoid loading the entire body into memory, but the library intentionally provides the `bodyStream` to the application so the handler can process the body incrementally. The `ContinueReadBody` function (non-streaming) strictly enforces the limit by returning `ErrBodyTooLarge`. The `ContinueReadBodyStream` function returns `nil` with a stream to allow the application to control how much of the oversized body to read/process; it does not silently buffer unbounded data into memory, nor does it defeat the purpose of the limit in the non-streaming context. Therefore, it is not a security vulnerability.",
"round": 6,
"file": "http.go",
"finding_title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"elapsed": 103.6,
"tokens_in": 24637,
"tokens_out": 208,
"cost_usd": 0.010971539999999998
}
]Raw scan report for this file
# Security Analysis: `http.go` (fasthttp)
## Step-by-Step Trace of Untrusted Data Flows
### 1. `readBodyChunked` → `parseChunkSize` → `appendBodyFixedSize` (CRITICAL)
**Flow:** Network bytes → `Transfer-Encoding: chunked` body → `readBodyChunked(r, maxBodySize, dst)` → `parseChunkSize(r)` returns `chunkSize` → `appendBodyFixedSize(r, dst, chunkSize+strCRLFLen)`
**Vulnerability:** `parseChunkSize` calls `readHexInt(r)` to parse the chunk size. If an attacker sends a chunk size hex value that exceeds `MaxInt` (e.g., `ffffffffffffffff`), the integer wraps to a **negative** value. The subsequent check:
```go
if maxBodySize > 0 && len(dst)+chunkSize > maxBodySize {
return dst, ErrBodyTooLarge
}
```
passes because `len(dst) + negative_value` is less than `maxBodySize`. Then:
```go
dst, err = appendBodyFixedSize(r, dst, chunkSize+strCRLFLen)
```
With `chunkSize = -3` (from hex `fffffffffffffffd`), `chunkSize + strCRLFLen = -1`. Inside `appendBodyFixedSize`:
```go
dstLen := offset + n // 0 + (-1) = -1
dst = dst[:dstLen] // PANIC: slice bounds out of range
```
**Impact:** Remote DoS — server process crashes. No authentication required.
---
### 2. `appendBodyFixedSize` accepts negative `n` (CRITICAL)
The function only guards against `n == 0`, not `n < 0`:
```go
func appendBodyFixedSize(r *bufio.Reader, dst []byte, n int) ([]byte, error) {
if n == 0 {
return dst, nil
}
offset := len(dst)
dstLen := offset + n
if cap(dst) < dstLen { // false when dstLen < 0
...
}
dst = dst[:dstLen] // PANIC if dstLen < 0
...
}
```
Any negative `n` causes `dst[:dstLen]` to panic. This is the concrete crash primitive triggered by the chunk-size overflow above.
---
### 3. Decompression Bomb via Default No-Limit Methods (HIGH)
```go
func (req *Request) BodyGunzip() ([]byte, error) {
return req.BodyGunzipWithLimit(0) // 0 = no limit
}
```
All of `BodyGunzip`, `BodyInflate`, `BodyUnbrotli`, `BodyUnzstd`, and `BodyUncompressed` default to `maxBodySize = 0` (unbounded). An attacker sends a ~25-byte gzip body that decompresses to multi-GB. The library allocates the full decompressed size into `bb.B` with no cap → OOM / process kill.
The `WithLimit` variants exist, but the convenience methods that most code uses first are the dangerous ones.
---
### 4. Multipart Form + Gzip with No Size Limit (HIGH)
In `MultipartFormWithLimit(0)` (called by `MultipartForm()`), the streaming + gzip path:
```go
if bytes.Equal(ce, strGzip) {
if bodyStream, err = gzip.NewReader(bodyStream); err != nil { ... }
}
if maxBodySize > 0 {
lr = &io.LimitedReader{ R: bodyStream, N: int64(maxBodySize) + 1 }
bodyStream = lr
}
// maxBodySize == 0 → no LimitedReader
mr := multipart.NewReader(bodyStream, req.multipartFormBoundary)
req.multipartForm, err = mr.ReadForm(8 * 1024)
```
When `maxBodySize <= 0`, the gzip decompressed stream is read **without any size cap**. A crafted `Content-Encoding: gzip` + `Content-Type: multipart/form-data` request with a small compressed payload causes unbounded memory growth in `mr.ReadForm`.
---
### 5. `readBodyIdentity` — Unbounded Allocation when `maxBodySize <= 0` (MEDIUM)
```go
func readBodyIdentity(r *bufio.Reader, maxBodySize int, dst []byte) ([]byte, error) {
...
for {
nn, err := r.Read(dst[offset:])
...
offset += nn
if maxBodySize > 0 && offset > maxBodySize {
return dst[:offset], ErrBodyTooLarge
}
if len(dst) == offset {
n := roundUpForSliceCap(2 * offset)
// no cap when maxBodySize <= 0
b := make([]byte, n)
...
}
}
}
```
When called with `maxBodySize <= 0` (the default via `Request.Read(r)` → `ReadLimitBody(r, 0)`), an attacker who controls connection lifetime (no `Content-Length`, no `Transfer-Encoding`) can send data indefinitely, forcing the server to grow the buffer without bound until OOM.
---
### 6. `ContinueReadBodyStream` — `ErrBodyTooLarge` Leaves Oversized Stream Active (MEDIUM)
```go
if err == ErrBodyTooLarge {
req.Header.SetContentLength(contentLength) // original, oversized value
req.body = bodyBuf
req.bodyStream = acquireRequestStream(bodyBuf, r, &req.Header)
return nil // no error to caller!
}
```
When the pre-read exceeds `maxBodySize`, the function **silently succeeds** and hands back a stream that will continue reading the full (oversized) body. The caller receives `nil` error and a `Content-Length` that exceeds the configured limit. If downstream code trusts `Content-Length` for buffer sizing, this can lead to unexpected large allocations.
---
## JSON Findings
```json
[
{
"severity": "critical",
"title": "Integer overflow in chunked Transfer-Encoding chunk size causes panic (remote DoS)",
"function": "readBodyChunked",
"description": "parseChunkSize() calls readHexInt() which can overflow Go's signed int when the attacker supplies a 16-hex-digit chunk size >= 0x8000000000000000. The resulting negative chunkSize bypasses the 'len(dst)+chunkSize > maxBodySize' check (negative + small <= max), then appendBodyFixedSize() is called with chunkSize+strCRLFLen. When the sum is negative (e.g. hex 'fffffffffffffffd' → -3 + 2 = -1), appendBodyFixedSize executes 'dst = dst[:dstLen]' with a negative dstLen, panicking with 'slice bounds out of range' and crashing the server process. Triggerable by any client sending a chunked body with a crafted chunk size; no authentication required."
},
{
"severity": "critical",
"title": "appendBodyFixedSize lacks negative-size guard, panics on negative n",
"function": "appendBodyFixedSize",
"description": "The function only returns early when n == 0 but does not validate n < 0. A negative n produces a negative dstLen (offset + n), and the subsequent 'dst = dst[:dstLen]' panics. This is the crash primitive reached via the chunk-size integer overflow in readBodyChunked, and could also be triggered by any other code path that passes an unvalidated negative size (e.g. an overflowed Content-Length if header parsing is bypassed)."
},
{
"severity": "high",
"title": "Decompression bomb via default no-limit BodyGunzip/BodyInflate/BodyUnbrotli/BodyUnzstd/BodyUncompressed",
"function": "BodyGunzip",
"description": "BodyGunzip(), BodyInflate(), BodyUnbrotli(), BodyUnzstd(), and BodyUncompressed() all call their WithLimit counterparts with maxBodySize=0, meaning no cap on decompressed output. An attacker sends a minimal gzip/deflate/brotli/zstd payload (~100 bytes) that decompresses to several GB. gunzipData/inflateData/etc. allocate the full decompressed size into a ByteBuffer with no bound, causing OOM and process termination. The convenience (no-limit) methods are the most commonly called entry points, making this the default behavior for any application that does not explicitly use the WithLimit variants."
},
{
"severity": "high",
"title": "Unbounded gzip decompression in MultipartFormWithLimit(0) streaming path",
"function": "MultipartFormWithLimit",
"description": "When a multipart/form-data request has Content-Encoding: gzip and arrives as a body stream, MultipartFormWithLimit(0) (the default from MultipartForm()) wraps the stream in gzip.NewReader but only applies an io.LimitedReader when maxBodySize > 0. With the default limit of 0, the decompressed stream is passed to multipart.NewReader().ReadForm() without any size cap. A small gzipped body containing a high-ratio gzip bomb causes unbounded memory allocation during multipart parsing, leading to OOM/DoS."
},
{
"severity": "medium",
"title": "readBodyIdentity allows unbounded body growth when maxBodySize <= 0",
"function": "readBodyIdentity",
"description": "Called with maxBodySize=0 (the default from Request.Read / ReadLimitBody(r, 0)) for requests without Content-Length or Transfer-Encoding, the function reads from the network until EOF with no size cap. The buffer doubles each time it fills (roundUpForSliceCap(2 * offset)). An attacker who controls connection lifetime (e.g. a slow drip over a long-lived TCP connection) forces the server to grow the body buffer without bound until OOM. No per-request size limit is enforced."
},
{
"severity": "medium",
"title": "ContinueReadBodyStream silently returns oversized body stream on ErrBodyTooLarge",
"function": "ContinueReadBodyStream",
"description": "When readBodyWithStreaming detects that contentLength exceeds maxBodySize, the function returns nil (no error) and sets req.bodyStream to a requestStream that will continue reading the full oversized body from the network. The Content-Length header is set to the original (oversized) value. Downstream code that uses Content-Length to pre-allocate buffers or that assumes the body is within the configured limit will process an unbounded amount of data, defeating the purpose of the size limit."
}
]
```