ramblings on PHP, SQL, the web, politics, ultimate frisbee and what else is on in my life
back

Auth checks and varnish

These days everybody seems to be using Varnish to speed up their site. Things are quite simple until you have to do authentication. IIRC it was my Liip co-worker Stefan Paschke who come up with a nice and simply solution to the dilemma that while you may have some content cached in Varnish, you still need to figure out if you can serve the content. The solution is as always by leveraging the HTTP specification. When we need to serve protected content, we simply turn GET requests into HEAD requests, send them to our app and check for HEAD requests inside a listener after the auth checks. In case of a HEAD request we then return the response early and Varnish can check the response to determine if to serve the original GET request or not. The good news is that its all nicely implemented in LiipCacheControlBundle, along with various other tools to better leverage Varnish, ESI and all that good stuff that is well integrated in Symfony2.

You can find the code for the listener on github and here is a varnish config that I ripped out of a project of ours. I hope the config is still sane as I didn't do any tests after cleaning out application specific stuff, but it should be enough to figure out whats happening:

backend default {
    .host = “127.0.0.1″;
    .port = “81″;
}

acl purge {
    “127.0.0.1″; #localhost for dev purposes
}

sub vcl_recv {
    # pipe HEAD requests as we convert all GET requests to HEAD and back later on
    if (req.request == “HEAD”) {
        return (pipe);
    }

    if (req.request == "GET") {
        if (req.restarts == 0) {
            set req.request = "HEAD";
            return (pass);
        } else {
            set req.http.Surrogate-Capability = "abc=ESI/1.0";
            return (lookup);
        }
    }
}

sub vcl_hash {
}

sub vcl_fetch {
    if (beresp.http.Cache-Control ~ “(private|no-cache|no-store)”) {
        return (pass);
    }

    if (beresp.status >= 200 && beresp.status < 300) {
        if (req.request == "HEAD") {
            # if the BE response said OK, change the request type back to GET and restart
            set req.request = "GET";
            restart;
        }
    } else {
        # In any other case (authentication 302 most likely), just pass the response to the client
        # Don't forget to set the content-length, as the HEAD response doesn't have any (and the client will hang)
        if (req.request == "HEAD") {
            set beresp.http.content-length = "0";
        }

        return (pass);
    }

    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        unset beresp.http.Surrogate-Control;
        esi;
    }
}

It should be noted that in our application we actually create a token which we can authenticated independently of Symfony2, but for now we didn't want to start writing inline C code to add the validation routines into Varnish itself. We might do so later on if we feel we do not have any other places to tweak ..

Update: Added the critical section where we turn GET requests into HEAD requests before hitting Symfony2 to the varnish config

Comments



Re: Auth checks and varnish

We're working on an ESI implementation for Nginx that will provide significant performance and scalability improvement for dynamic content, not just the static implementation that Varnish supports.
You can find the slides from the Zendcon2011 presentation here.

Re: Auth checks and varnish

Hmm a bit hard to grok things from just your slides. You are adding ESI support into Nginx? And you also want to cache user specific content? You can surely cache user specific content with varnish, its just usually not a good idea if you actually have a lot of users, because your cache obviously does not have infinite size.

For the example you have in your slides I actually prefer javascript based approaches with async calls or better yet client side caching with local storage (and cookies as fallback).

Re: Auth checks and varnish

It seems you're letting your client send a HEAD request. And then varnish converts it to a GET if it was successful.

That's fine, but browsers can't really do it when you click a link. It would be better to convert GET requests to HEAD requests first inside varnish, only if they match some URLs I guess (those you want to do ACL checks on). Then convert back to GET and serve from cache if the HEAD was 200?

I'm not sure if it's possible, but it would allow it to work with any browser/client, and no javascript.

Re: Auth checks and varnish

Ah, let me check, I must have ripped out to much. Indeed thats what we do. We receive a GET request, turn it into a HEAD request inside varnish, read the HEAD response and then respond to the initial GET request.