LF-only header line separation creates parser differential

[
  {
    "finding_title": "LF-only header line separation creates parser differential",
    "verdict": "VALID",
    "reasoning": "The reported vulnerability is a parser differential where `readLine()` accepts LF-only line endings, whereas the HTTP specification requires CRLF. In the context of a reusable library (fasthttp), accepting invalid input (LF-only headers) creates a significant security risk. Specifically, this leniency allows an attacker to craft HTTP requests with LF-only separators. If this library is used to front-end or proxy a backend that strictly enforces CRLF (like nginx or Apache), the fasthttp server will parse headers differently (e.g., folding them differently or accepting them as valid when the backend rejects them, or interpreting header boundaries differently). This enables Request Smuggling attacks, where the client and server disagree on the boundaries of HTTP messages, leading to cache poisoning, authentication bypass, or data corruption. The code in `readLine()` explicitly handles LF as a line break (checking for `line[len(line)-1] == '\\n'` and dropping an optional preceding `\\r`). This confirms the bug pattern exists. Since `headerScanner` is used in public API entry points for parsing HTTP requests (like `Request.Read`), untrusted input can trigger this. The threat model explicitly states that parsers receiving adversarial payloads are in scope, and correct behavior on malformed input is an asset. Therefore, this is a valid security vulnerability.\n\nCRUX: readLine() accepts LF as a line terminator, creating a parser differential that enables request smuggling against strict-CRLF backends.\nGREP: func (s *headerScanner) readLine()",
    "elapsed": 219.1,
    "tokens": 3415,
    "tokens_in": 3092,
    "tokens_out": 323,
    "cost_usd": 0.0022676399999999996,
    "file": "headerscanner.go",
    "round": 1,
    "grep_used": true,
    "grep_results": "GREP `func (s *headerScanner) readLine() (simplified to: headerScanner)`:\n```\nheaderscanner.go:9:type headerScanner struct {\nheaderscanner.go:21:func (s *headerScanner) next() bool {\nheaderscanner.go:76:func (s *headerScanner) readLine() (line []byte) {\nheaderscanner.go:107:func (s *headerScanner) readContinuedLineSlice() ([]byte, error) {\nheaderscanner.go:138:func (s *headerScanner) skipSpace() bool {\nallocation_test.go:124:\t\tvar s headerScanner\nfuzz_test.go:171:\t\tvar s headerScanner\nfuzz_test.go:175:\t\t\t// ReadMIMEHeader normalizes header keys, headerScanner doesn't by default.\nfuzz_test.go:199:\t\t\tt.Errorf(\"unexpected error from headerScanner: %v: %v\", s.err, h)\nheader.go:1307:\t\tvar s headerScanner\nheader.go:2704:\tvar s headerScanner\nheader.go:2992:\tvar s headerScanner\nheader.go:3141:\tvar s headerScanner\n```"
  },
  {
    "finding_title": "LF-only header line separation creates parser differential",
    "verdict": "INVALID",
    "reasoning": "The reported vulnerability is a 'parser differential' regarding HTTP Request Smuggling. This claim requires that the library's parsing logic (specifically header boundary detection) disagrees with a proxy or backend in a way that allows an attacker to smuggle request bodies or headers. \n\n1. **Analysis of `headerScanner`**: The provided code (`headerscanner.go`) implements a low-level scanner for *header lines*. It does not parse the entire HTTP request structure (Method, URI, Status Line) nor does it determine the `Content-Length` or `Transfer-Encoding` boundaries that are critical for request smuggling. It simply iterates through lines to find keys and values.\n\n2. **Impact of LF vs CRLF in `readLine`**: `readLine()` accepts LF as a line terminator. If an attacker sends `Host: foo\\nX-Injected: bar\\n\\n` (LF only), `headerScanner` will parse `Host` and `X-Injected` correctly as two distinct headers. A strict CRLF parser might reject this or parse it differently, but since `headerScanner` is consuming the stream, it correctly identifies the header section boundaries based on the empty line (which `readLine` also handles by detecting blank lines or specific markers). \n\n3. **Request Smuggling Context**: Request smuggling typically arises from ambiguity in `Content-Length` vs `Transfer-Encoding` (Chunked) or malformed request lines (e.g., `GET /\\r\\n...`). The `headerScanner` is not responsible for parsing the request line or the body framing. The fasthttp library parses the request line and body framing in other files (e.g., `http.go` or `request.go`). If the library correctly implements `Transfer-Encoding` and `Content-Length` logic elsewhere, the leniency of `readLine` in accepting LF-only headers does not inherently create a smuggling vector for the *message body* or *request boundary*. \n\n4. **Threat Model**: The threat model states that 'parsers, decoders... receive adversarial payloads'. While `headerScanner` is part of the parsing pipeline, the specific bug (LF acceptance) in header line splitting does not constitute a meaningful security harm (crash, code execution, data corruption, auth bypass) under the standard definition of Request Smuggling. It might lead to a 'Header Injection' if an upstream proxy expects CRLF and treats the whole blob as one header, but fasthttp is a self-contained server library in most contexts, or if used as a client, it sends valid CRLFs (likely). Even if it accepts LF-only headers from a peer, it parses them into separate `Header` objects. This does not corrupt the host process memory, nor does it bypass authentication (auth checks happen on parsed headers). \n\n5. **Verification of 'Meaningful Harm'**: Is there a crash? No, `readLine` handles bounds safely. Is there data corruption? The headers are parsed as intended by the lenient parser. Is there auth bypass? If an auth check looks at `X-Auth-Token`, and the attacker injects it via LF, fasthttp *will* see it. A strict parser *would* reject the request. This is a difference in behavior, but if fasthttp is the final authority (server), it sees the token. If it is a proxy, the issue would be in how it forwards the raw bytes vs parsed headers. Fasthttp typically reconstructs requests using CRLF when forwarding. Thus, the differential is neutralized by the reconstruction step or is irrelevant for direct server use. \n\nTherefore, this is a minor compliance/strictness issue, not a security vulnerability enabling smuggling or code execution.\n\nCRUX: The leniency in header line splitting (LF vs CRLF) does not cause a request smuggling vulnerability because the scanner does not control body framing or request boundary logic, and fasthttp reconstructs valid CRLF-separated requests when acting as a proxy.\nGREP: func (h *Request) Read",
    "elapsed": 327.2,
    "tokens": 4674,
    "tokens_in": 3826,
    "tokens_out": 848,
    "cost_usd": 0.00415092,
    "file": "headerscanner.go",
    "round": 2,
    "grep_used": true,
    "grep_results": "GREP `func (h *Request) Read (simplified to: Request)`:\n```\nheader_timing_test.go:34:func BenchmarkRequestHeaderRead(b *testing.B) {\nheader_timing_test.go:36:\t\tvar h RequestHeader\nheader_timing_test.go:65:func BenchmarkRequestHeaderReadNetHttp(b *testing.B) {\nheader_timing_test.go:86:\t\t\tif _, err := http.ReadRequest(br); err != nil {\nheader_timing_test.go:121:func BenchmarkRequestHeaderWrite(b *testing.B) {\nheader_timing_test.go:123:\t\tvar h RequestHeader\nheader_timing_test.go:124:\t\th.SetRequestURI(\"/foo/bar\")\nheader_timing_test.go:157:func BenchmarkRequestHeaderPeekBytesSpecialHeader(b *testing.B) {\nheader_timing_test.go:159:\t\tvar h RequestHeader\nheader_timing_test.go:171:func BenchmarkRequestHeaderPeekBytesNonSpecialHeader(b *testing.B) {\nheader_timing_test.go:173:\t\tvar h RequestHeader\nheader_timing_test.go:238:\tvar h RequestHeader\nheader_timing_test.go:267:\t\t\tvar h RequestHeader\nheader_timing_test.go:281:func BenchmarkRequestHeaderIsGet(b *testing.B) {\nheader_timing_test.go:282:\treq := &RequestHeader{method: []byte(MethodGet)}\nfasthttpadaptor/adaptor.go:33:func NewFastHTTPHandlerFunc(h http.HandlerFunc) fasthttp.RequestHandler {\nfasthttpadaptor/adaptor.go:53:func NewFastHTTPHandler(h http.Handler) fasthttp.RequestHandler {\nfasthttpadaptor/adaptor.go:54:\treturn func(ctx *fasthttp.RequestCtx) {\nfasthttpadaptor/adaptor.go:55:\t\tvar r http.Request\nfasthttpadaptor/adaptor.go:56:\t\tif err := ConvertRequest(ctx, &r, true); err != nil {\nfasthttpadaptor/adaptor.go:57:\t\t\tctx.Logger().Printf(\"cannot parse requestURI %q: %v\", r.RequestURI, err)\nfasthttpadaptor/adaptor.go:201:\tctx        *fasthttp.RequestCtx\nfasthttpadaptor/adaptor.go:222:func acquireWriter(ctx *fasthttp.RequestCtx) *writer {\nfasthttpadaptor/request.go:13:// ConvertRequest converts a fasthttp.Request to an http.Request.\nfasthttpadaptor/request.go:14:// forServer should be set to true when the http.Request is going to be passed to a http.Handler.\nfasthttpadaptor/request.go:16:// The http.Request must not be used after the fasthttp handler has returned!\nfasthttpadaptor/request.go:17:// Memory in use by the http.Request will be reused after your handler has returned!\nfasthttpadaptor/request.go:18:func ConvertRequest(ctx *fasthttp.RequestCtx, r *http.Request, forServer bool) error {\nfasthttpadaptor/request.go:20:\tstrRequestURI := b2s(ctx.RequestURI())\nfasthttpadaptor/request.go:22:\trURL, err := url.ParseRequestURI(strRequestURI)\n```"
  },
  {
    "finding_title": "LF-only header line separation creates parser differential",
    "verdict": "INVALID",
    "reasoning": "The finding claims that `readLine()` accepting LF-only line separators creates a parser differential enabling HTTP request smuggling. To evaluate this, I must determine if the library uses `readLine()` for determining message boundaries (like the end of headers or body framing) or if it uses a strict CRLF check for those critical boundaries. \n\n1. **Header Boundary Detection**: In `headerScanner.next()`, the initialization logic checks for `strCRLF` and `strCRLFCRLF` (which is `\\r\\n\\r\\n`). The loop `i := bytes.Index(s.b, strCRLFCRLF)` relies on the strict CRLF sequence to find the end of the headers. If an attacker sends LF-only separators, `strCRLFCRLF` will not be found, and the scanner will return `ErrNeedMore` or fail to initialize, effectively rejecting or stalling the request rather than misparsing it. It does not accept LF-only as a valid header terminator in the structural sense that defines the message boundaries.\n\n2. **Line Content Parsing**: `readLine()` is called *after* the header block has been identified. It splits the lines within that block. While it is lenient about LF for splitting lines, the critical security boundary (where headers end and the body begins) is governed by the `strCRLFCRLF` search in `next()`. \n\n3. **Smuggling Vector**: Request smuggling typically relies on differing interpretations of `Content-Length` vs `Transfer-Encoding` or malformed request lines. The `headerScanner` does not parse the request line or body framing. The leniency in `readLine` means that if a header block *is* found (via strict CRLF markers), the lines inside might be split on LF. However, since the scanner is part of `fasthttp` and `fasthttp` is the server (or the parser component), it defines the truth for its own processing. If it were used as a proxy, it would reconstruct the request using standard CRLF when writing to the backend (via `Request.Write`), neutralizing the differential. \n\n4. **Conclusion**: The bug pattern (lenient LF in `readLine`) exists, but it does not lead to meaningful harm like smuggling because the message boundary detection uses strict CRLF, and the library reconstructs valid HTTP messages when acting as a proxy. It is a compliance/strictness issue, not a security vulnerability under the provided threat model.\n\nCRUX: The message boundary detection in headerScanner.next() uses a strict search for strCRLFCRLF (\\r\\n\\r\\n), not the lenient readLine(), preventing parser differentials at the critical boundary.\nGREP: bytes.Index(s.b, strCRLFCRLF)",
    "elapsed": 140.4,
    "tokens": 5741,
    "tokens_in": 5149,
    "tokens_out": 592,
    "cost_usd": 0.00393858,
    "file": "headerscanner.go",
    "round": 3,
    "grep_used": true,
    "grep_results": "GREP `strCRLFCRLF`:\n```\nheaderscanner.go:28:\t\ti := bytes.Index(s.b, strCRLFCRLF)\nstrings.go:20:\tstrCRLFCRLF                 = []byte(\"\\r\\n\\r\\n\")\n```"
  },
  {
    "finding_title": "LF-only header line separation creates parser differential",
    "verdict": "INVALID",
    "reasoning": "The report claims that `readLine()` accepting LF-only line separators creates a parser differential enabling request smuggling or header injection. \n\n1. **Bug Pattern Existence**: The code in `readLine()` (lines 76-95) indeed searches for `'\n'` and optionally strips a preceding `'\r'`. This confirms that the scanner is lenient about line terminators *within* the header block.\n\n2. **Reachability and Boundary Control**: To trigger `readLine()`, the scanner must first initialize in `next()` (lines 21-43). The initialization logic (line 28) strictly requires the presence of `strCRLFCRLF` (`\\r\\n\\r\\n`) to identify the end of the headers. If an attacker sends headers separated only by LF (e.g., `Host: A\nUser-Agent: B\n\n`), `bytes.Index(s.b, strCRLFCRLF)` will fail (return -1), causing `s.err = ErrNeedMore`. The scanner will not proceed to `readLine()`. Therefore, the lenient `readLine()` is only invoked when the strict CRLF boundary marker is already present and verified.\n\n3. **Security Impact**: Since the critical message boundary (end of headers) is strictly enforced via `\\r\\n\\r\\n`, the leniency of `readLine()` is confined to lines *between* the start of headers and the strict end marker. This does not create a differential in where the header/body split occurs, which is the primary vector for request smuggling. While a strict parser might reject the *lines* within the block if they are LF-only, fasthttp parses them successfully. However, because fasthttp is the library defining the parsing behavior for the host process, it does not 'mispars' boundaries relative to itself. If fasthttp is used as a proxy, it reconstructs the request using standard CRLF when writing to the backend (`Request.Write`), neutralizing any differential. Thus, there is no vector for request smuggling, cache poisoning, or code execution. The issue is a lack of strictness/compliance with RFC 7230 (which mandates CRLF), but not a security vulnerability leading to meaningful harm under the specified threat model.\n\nCRUX: The initialization of the header scanner strictly requires the CRLF-CRLF sequence (\\r\\n\\r\\n) to define the end of headers, preventing the lenient line-splitting logic in readLine() from affecting message boundary detection or enabling request smuggling.\nGREP: strCRLFCRLF",
    "elapsed": 120.9,
    "tokens": 5713,
    "tokens_in": 5155,
    "tokens_out": 558,
    "cost_usd": 0.0038391,
    "file": "headerscanner.go",
    "round": 4,
    "grep_used": true,
    "grep_results": "GREP `strCRLFCRLF`:\n```\nheaderscanner.go:28:\t\ti := bytes.Index(s.b, strCRLFCRLF)\nstrings.go:20:\tstrCRLFCRLF                 = []byte(\"\\r\\n\\r\\n\")\n```"
  },
  {
    "finding_title": "LF-only header line separation creates parser differential",
    "verdict": "INVALID",
    "reasoning": "The reported vulnerability claims that `readLine()` accepting LF-only line endings creates a parser differential enabling request smuggling or header injection. While `readLine()` is indeed lenient (it accepts LF as a line terminator), the security impact depends on whether this leniency affects critical message boundaries or allows an attacker to manipulate how the HTTP message is interpreted relative to a strict parser. \n\nAnalysis of the code reveals that `headerScanner.next()` (the entry point for parsing headers) strictly requires the presence of `strCRLFCRLF` (`\\r\\n\\r\\n`) to identify the end of the headers block (line 28: `i := bytes.Index(s.b, strCRLFCRLF)`). If an attacker sends headers separated only by LF (e.g., `Host: A\\nUser-Agent: B\\n\\n`), `bytes.Index` will fail to find `\\r\\n\\r\\n`, causing the scanner to return `ErrNeedMore` or fail to initialize. The lenient `readLine()` logic is only executed *after* the strict CRLF boundary has been established. Therefore, the scanner does not misidentify where the headers end; it only parses lines within a block that has already been demarcated by strict CRLF markers. \n\nFurthermore, fasthttp is a self-contained HTTP server library. When it parses a request, it uses its own internal representation. If used as a proxy, it reconstructs outgoing requests using standard CRLF terminators (via `Request.Write`), neutralizing any differential between the incoming (potentially leniently parsed) and outgoing (strictly formatted) representations. The leniency does not lead to memory corruption, code execution, or a disagreement on message boundaries (smuggling) because the boundary detection is strict. It is a minor compliance issue with RFC 7230 (which mandates CRLF) but does not constitute a security vulnerability under the specified threat model that requires meaningful harm like crash, code execution, or auth bypass via parser differential.\n\nCRUX: The initialization of headerScanner strictly requires the \\r\\n\\r\\n sequence to define the header boundary, preventing the lenient readLine() logic from affecting message boundary detection or enabling request smuggling.\nGREP: bytes.Index(s.b, strCRLFCRLF)",
    "elapsed": 113.3,
    "tokens": 6237,
    "tokens_in": 5742,
    "tokens_out": 495,
    "cost_usd": 0.00389664,
    "file": "headerscanner.go",
    "round": 5,
    "grep_used": true,
    "grep_results": "GREP `strCRLFCRLF`:\n```\nheaderscanner.go:28:\t\ti := bytes.Index(s.b, strCRLFCRLF)\nstrings.go:20:\tstrCRLFCRLF                 = []byte(\"\\r\\n\\r\\n\")\n```"
  },
  {
    "verdict": "VALID",
    "reasoning": "[ARBITER] The vulnerability is a protocol deviation (parser differential) that violates HTTP/1.1 specifications (RFC 7230), which mandate CRLF line endings. While the `headerScanner` uses strict CRLF detection (`\\r\\n\\r\\n`) to identify the end of the header block (preventing classic request smuggling via body confusion), the `readLine()` function is lenient, accepting LF-only (`\\n`) terminators for individual header lines. This leniency allows an attacker to craft headers that fasthttp accepts and parses, which might be rejected or parsed differently by strict proxies or backends (e.g., causing header injection or misinterpretation of folded headers). As a reusable library, fasthttp should enforce strict protocol compliance to prevent such differentials when used in proxy chains or mixed-backend architectures. Therefore, it constitutes a valid security finding under the 'correct behaviour on malformed input' and protocol integrity criteria.",
    "round": 6,
    "file": "headerscanner.go",
    "finding_title": "LF-only header line separation creates parser differential",
    "elapsed": 306.1,
    "tokens_in": 3740,
    "tokens_out": 208,
    "cost_usd": 0.0021948000000000002
  }
]