Too many headers in Phoenix

Published: Dec 14 2023

In a recent Phoenix project, I encountered an unexpected error related to request headers. The Phoenix app was rejecting requests due to excessive headers, as indicated by the following error log:

2023-12-14T09:28:49Z app[786578df46351c] lax [info]09:28:49.798 [error] GenServer #PID<0.3411.0> terminating
2023-12-14T09:28:49Z app[786578df46351c] lax [info]** (stop) :too_many_headers
2023-12-14T09:28:49Z app[786578df46351c] lax [info]Last message: {:continue, :handle_connection}

The root cause was the presence of up to 57 headers in my requests. While it might seem like an extreme number, it is, it was due to using both BunnyCDN and Fly.io (both great companies) which both added several headers to the request.

The problem is thatthere are inherent limitations in both Cowboy (Erlang) and Bandit (Elixir) regarding the number of headers and their size. These restrictions are likely in place for security reasons.

In this post, I’ll guide you through resolving this issue in a Phoenix environment.

Bandit

I was using Bandit as my server and managed to solve it by using a combination of old posts about related issues in Cowboy and the source code of Bandit. What we need to do is add max_header_count for both http_1_options and http_2_options in our configuration.

My config in runtime.exs was change from this default:

  config :test_app, TestAppWeb.Endpoint,
    url: [host: host, port: 443, scheme: "https"],
    http: [
      ip: {0, 0, 0, 0, 0, 0, 0, 0},
      port: port
    ],
    secret_key_base: secret_key_base

To this:

  config :test_app, TestAppWeb.Endpoint,
    url: [host: host, port: 443, scheme: "https"],
    http: [
      http_1_options: [max_header_count: 200],  http_2_options: [max_header_count: 200],
      ip: {0, 0, 0, 0, 0, 0, 0, 0},
      port: port
    ],
    secret_key_base: secret_key_base

It would have been nice if one could set the count for both http/1 and http/2 at the same time but I could not find any such option.

Cowboy

There was some related issues discussed in old Github issues and forum posts but hopefully this can save you some time if you encounter the same problem. In Cowboy we change the code from this:

  config :test_app, TestAppWeb.Endpoint,
    url: [host: host, port: 443, scheme: "https"],
    http: [
      ip: {0, 0, 0, 0, 0, 0, 0, 0},
      port: port
    ],
    secret_key_base: secret_key_base

To this:

  config :test_app, TestAppWeb.Endpoint,
    url: [host: host, port: 443, scheme: "https"],
    http: [
      protocol_options: [max_headers: 200],
      ip: {0, 0, 0, 0, 0, 0, 0, 0},
      port: port
    ],
    secret_key_base: secret_key_base