Symfony2 is full of ESI love. ESI aka Edge Side Includes are a standard for reverse proxy caching which can significantly boost performance. What is even nicer is that ESI really goes well with RESTful API's, which is one reason why I am pushing REST in the Symfony2 world. I must admit that until recently I was totally ok taking short cuts, but I am now being to realize that the short cuts I was taking actually hurt in many ways. So I am warping my mind to be more RESTful. But at the same time I also see great potential in ESI which needs some additional mind bending .. or actually maybe not.

The key thing with ESI is that you need to ensure you keep user specific content separate from unspecific content, so that you can most effectively cache data. One approach is to just keep all user specific data in a separate ESI block. Having multiple such ESI blocks however could defeat the purpose of ESI, since suddenly a single request will lead to multiple, albeit smaller, requests for each of the ESI blocks with user specific (and therefore likely un-cacheable) content. One solution here can be trying to at least group user specific versions according to groups, but this can become pretty tricky and is therefore not really all that feasible in many cases. So one approach might be having just one ESI block with user specific content, which is then placed in different locations in the output via CSS or javascript.

But I am pondering another approach. One draw back here is that it makes the site totally dependent on Javascript. Then again one could simply not offer personalization to users that disable javascript or redirect these users to another site that doesn't use ESI (and as a result might be considerably slower). The fundamental idea is to use cookies to pass user specific data to the browser. In the browser the cookie data can be consumed, written to local storage etc in order to prevent needless traffic, which is oh so critical for mobile web apps.

One common requirement is to have a login/registration button for users not yet logged in. Once the user logs in those links should switch to a name which links to a profile and a logout button. Again one option would be to move this to a separate ESI block. However another approach would be to simply set a cookie with the name to display on login, which is removed on logout. Then the entire logic about if to display login/registration or name/logout is handled in the client and suddenly the entire request can be handled by the reverse proxy. One issue of course is that the logic needs to be triggered which means the content might not be shown immediately on page load.

In AJAX applications which are becoming the norm this is less of an issue since there are fewer full page loads. Then again for these applications one would simply not reload the parts for these navigation pieces. The point being the above is really a solution for full page reloads, which still happen enough to require a solution :)

Another issue are flash messages, which is often shown after a POST. One option is of course is to return the flash message as the response of the POST in case the POST was done via AJAX. A centralized handler can then take the flash messages and put it in the common location for flash messages in the application. For a non AJAX post, one might again fallback to cookies. The POST would set the cookie and tell the client to redirect to the newly created or edited resource. When displaying that page, it would read out the cookie via Javascript, display the flash message and remove the cookie. Again the end result is that no user specific information would be in the response content making ESI more effective.

I must admit that these are all just ideas in my head, which I hope to soon try out in practice. Feedback would be much appreciated.

Update: One thing where I am still a bit fuzzy on is how the entire cookie setting and propagation works. Obviously POST requests are not cached an so the cookie can be set. But I guess I need to ensure that varnish returns any cookie send back to the user. Not sure if this is done by default or requires some config magic.

I do feel that implementing ESI in PHP might be a useful way to go.

Multiple <esi:include>s in a page body could be fulfilled by MemCached::getMulti(), and could just execute a (fragment?) controller directly on cache miss.

@Ren: Symfony2 already did! It can switch seamlessly between using ESI and not for rendering actions. Furthermore it provides a PHP implementation of a reverse proxy, though of course its way more efficient it use Varnish.

Hi Lukas,

I would not bother so much about caching: HTTP has a widely adopted (among consumers) specification, with Sf2 we (services) finally embrace it. That means scaling points by default :-)

Cache channels (http://www.mnot.net/blog/2008/01/04/cache_channels) are another way to scale out without adding nodes to our architecture, which, btw, wouldn't be a bad idea.

Be aware that REST is fully conscious about the fact that the web is weakly consistent.

That said, it is ok not to cache responses to just increase consistency between served and actual data.

ESI becomes powerful because of this: my facebook homepage will never be totally cacheable, but fragments of it will: that's where we should use ESI.

Its awesomeness is more releavant when you think that those fragments are shared among N pages, so ours becomes a more effective gateway cache.