light serving needs

if you ever have the need to serve a few static pages — say munin output, for example — next to your elaborate "plone/zope behind haproxy behind nginx behind varnish" setup :), here's a quick way to run lighttpd via supervisor using buildout.

with a buildout.cfg like:

[buildout]
parts =
  lighttpd
  lighttpdconf
  supervisor

[lighttpd]
recipe = zc.recipe.cmmi
url = http://www.lighttpd.net/download/lighttpd-1.4.23.tar.gz

[lighttpdconf]
recipe = collective.recipe.template
input = ${buildout:directory}/lighttpd.conf.in
output = ${buildout:directory}/etc/lighttpd.conf
directory = /var/www/munin
port = 8080

[supervisor]
recipe = collective.recipe.supervisor
programs =
  10 lighttpd ${buildout:directory}/parts/lighttpd/sbin/lighttpd [-D -f ${buildout:directory}/etc/lighttpd.conf]

and a configuration template (saved as lighttpd.conf.in) like:

server.document-root = "${lighttpdconf:directory}"
server.port = "${lighttpdconf:port}"
server.tag ="lighttpd"
index-file.names = ( "index.html" )
mimetype.assign = (
  ".html" => "text/html", 
  ".txt" => "text/plain",
  ".jpg" => "image/jpeg",
  ".png" => "image/png", 
  ".css" => "text/css",
  ".js"  => "text/javascript",
  ".xml" => "text/xml",
  ".pdf" => "application/pdf",
)

all you need to do is the following:

$ python bootstrap.py
$ bin/buildout
$ bin/supervisord

after which you can access the contents of /var/www/munin at http://localhost:8080/.

please refer to the lighttpd documentation if you need more than this rather minimal setup...

running out of varnish

just a quick note that might be helpful when running a website through varnish: for at least a year now varnish has had an issue with serving large files. from a given size somewhere around 600mb a "stevedore" cannot be allocated anymore, resulting in an assertion error being raised and a successful, but otherwise empty response. the problem can be worked around using "pipe" to serve the content in question. in the case of plone your vcl might look like:

if (req.url ~ "(/at_download/|@@download$|\.mp4$|\.mov$)") {
    pipe;
}

this works, but is rather crude, of course. you'll need to somehow catch & specify all potentially big content, making it likely to miss some. unfortunately though, varnish only allows to "pipe" from inside its vcl_recv, so it's not possible to check the actual size after fetching the object from the backend, for example...