Note: This post has been updated since discovering this is NOT an Apache issue, and it turns out to entirely be a problem in the request processing framework of the application Apache is proxying requests to. Some frameworks follow old CGI specs that prohibit hyphens (“-“) in request header names. Apache is passing along both it’s header and the client-generated headers, but the proxied framework converts “-“ to “_” which results in a map/dictionary key collision.

As a result, my “Do this” advice has been updated.

While doing doing some Apache TLS configuration this week for work, I came across a security edge case with mod_headers and the RequestHeaders directive.

A fairly common use-case for this is to pass TLS/SSL headers to a proxied backend service when TLS termination is done in Apache. Imagine a case where client certificates are optional but the backend uses information from the certificate, such as the DN, or just validating if a client certificate was used.

Let’s take that last case as an example to illustrate this security risk, where we wish to pass along the SSL_CLIENT_VERIFY Apache variable to a backend, indicating that a client certificate was successfully used and validated. A common, but insecure configuration (which you’ll find in many guides and blogs if you search) is to do this:

RequestHeader set SSL_CLIENT_VERIFY "%{SSL_CLIENT_VERIFY}s"  # Don't do this!

This directive will add the header “Ssl-Client-Verify” to the request passed to the backend service, however this header can be overridden and spoofed by a client!

Instead, use the following configuration, which is not vulnerable to header forgery:

RequestHeader set SSLCLIENTVERIFY "%{SSL_CLIENT_VERIFY}s"  # Do this

Some request processing frameworks follow an old CGI specification that prohibits “-“ in header names and convert these to “_”, so to prevent a client from using a map/dictionary key collision to spoof headers, avoid the use of these characters entirely.

Here’s an example of header forgery, where we can easily override the Apache generated headers when specified like the “Don’t do this” case above:

$ curl --header "Ssl-Client-Verify: SPOOFED" -i https://my.site.foo

With a valid certificate we can still override the Apache generated header:

$ curl --header "Ssl-Client-Verify: SPOOFED" --cert cert.crt --key cert.key --cacert all.cert \
       -i https://my.site.foo

This is easy to test using a simple Python flask backend service with a route like the following (for easy illustration purposes only, of course):

@app.route('/')
def root():
    print request
    print request.headers
    return ''

The resulting output will show that the client was able to override the Apache header if underscores are used in the RequestHeader directive:

Ssl-Client-Verify: SPOOFED

Whereas using either the second or third form, where dashes are used instead of underscores, the client cannot spoof the header:

Ssl-Client-Verify: SUCCESS

Or if client certifications are optional and none was provided:

Ssl-Client-Verify: (null)

This vulnerability happens if the client passes a header that matches the final header of “Ssl-Client-Verify” (case doesn’t matter, so a spoofed header of “SSL-CLIENT-VERIFY” will result in header forgery). Passing a header of “SSL_CLIENT_VERIFY” from the client will not result in a spoofed header, potentially giving a false sense of security in testing.

The security risk is pretty clear - a malconfigured Apache and backend request processing framework that munges header names can result in clients spoofing headers such that a proxied service incorrectly thinks authentication or authorization has been confirmed when indeed it has not.

Be careful, do not use “-“ or “_” for header names in RequestHeader!