Tuesday, November 10, 2009

write in the vim room?

ever since i first saw it i always liked the idea of writeroom, but since i'm not writing that much on a regular basis i never bothered to get a license and give it a real spin. then, a couple of days ago, the latest macheist gave away free licenses with their "nanoBundle" plus i also had the need to write some slightly longer document...
but essentially from the moment i started i badly missed the vi-bindings i've grown used to — at least when writing text. after a bit of poking i found that macvim also support full-screen mode! however, even after mimicking the color scheme it was still a bit cumbersome to get going: fire up macvim, activate the color scheme, get rid of visually annoying scrollbars, increase the font size and finally hit ⇧⌘F...
so to cut the story short, here's my current workaround:
  • step 1: put the following into ~/.vim/fullscreen.vim
    " vim fullscreen mode
    set background=dark
    set guifont=Monaco:h17
    set guioptions=egmLtT
    set lines=40 columns=80
    " `fullscreen` needs to go after `lines` etc
    set fuoptions=
    set fullscreen
    highlight clear
    highlight Normal     guifg=#00a000      guibg=Black
    highlight NonText    guifg=#002000
    highlight Search     guifg=Black        guibg=#606000   gui=bold
    highlight Visual     guifg=#404040                      gui=bold
    highlight Cursor     guifg=Black        guibg=Green     gui=bold
    highlight Special    guifg=#004000
    highlight Comment    guifg=#008000
    highlight StatusLine guifg=blue         guibg=white
    highlight Statement  guifg=#004000                      gui=NONE
    highlight constant   guifg=#005000                      gui=NONE
    highlight preproc    guifg=#005000                      gui=NONE
    highlight Type                                          gui=NONE
  • step 2: add this line to your ~/.bashrc
    alias fvim="mvim -S ~/.vim/fullscreen.vim"
  • step 3: fire it up...
    $ fvim README.txt
  • step 4: focus!

Thursday, July 16, 2009

light serving needs

if you ever have the need to serve a few static pages — say munin output, for example — next to your elaborate "plone/zope behind haproxy behind nginx behind varnish" setup :), here's a quick way to run lighttpd via supervisor using buildout.

with a buildout.cfg like:

parts =

recipe = zc.recipe.cmmi
url = http://www.lighttpd.net/download/lighttpd-1.4.23.tar.gz

recipe = collective.recipe.template
input = ${buildout:directory}/lighttpd.conf.in
output = ${buildout:directory}/etc/lighttpd.conf
directory = /var/www/munin
port = 8080

recipe = collective.recipe.supervisor
programs =
  10 lighttpd ${buildout:directory}/parts/lighttpd/sbin/lighttpd [-D -f ${buildout:directory}/etc/lighttpd.conf]
and a configuration template (saved as lighttpd.conf.in) like:
server.document-root = "${lighttpdconf:directory}"
server.port = "${lighttpdconf:port}"
server.tag ="lighttpd"
index-file.names = ( "index.html" )
mimetype.assign = (
  ".html" => "text/html", 
  ".txt" => "text/plain",
  ".jpg" => "image/jpeg",
  ".png" => "image/png", 
  ".css" => "text/css",
  ".js"  => "text/javascript",
  ".xml" => "text/xml",
  ".pdf" => "application/pdf",
all you need to do is the following:
$ python bootstrap.py
$ bin/buildout
$ bin/supervisord
after which you can access the contents of /var/www/munin at http://localhost:8080/.

please refer to the lighttpd documentation if you need more than this rather minimal setup...

Tuesday, July 7, 2009

running out of varnish

just a quick note that might be helpful when running a website through varnish: for at least a year now varnish has had an issue with serving large files. from a given size somewhere around 600mb a "stevedore" cannot be allocated anymore, resulting in an assertion error being raised and a successful, but otherwise empty response. the problem can be worked around using "pipe" to serve the content in question. in the case of plone your vcl might look like:
if (req.url ~ "(/at_download/|@@download$|\.mp4$|\.mov$)") {
this works, but is rather crude, of course. you'll need to somehow catch & specify all potentially big content, making it likely to miss some. unfortunately though, varnish only allows to "pipe" from inside its vcl_recv, so it's not possible to check the actual size after fetching the object from the backend, for example...

Monday, April 20, 2009

zope batch undo

here's a quick tip for everyone tired of selecting dozens of checkboxes when trying to undo ZODB changes via the ZMI (or fighting with the batched display of the transactions). i mean, really there should be a "select all" button like in ZMI's folder listing, but until this has been done and released, here's a workaround...

first, and this is only needed once, of course, you bookmark the following javascript-snippet:

then, when you're ready to undo things you get out of the frameset and select a rough number of transactions to undo using a direct link like:
and afterwards simply click your bookmark to select all checkboxes at once. perhaps you'll still need to unselect the last few or else adjust the number in the link and finally click "undo"...

Monday, January 26, 2009


some time ago back at the plone performance sprint in copenhagen i was trying to help optimizing zope start-up time a little more after hannosch's impressive pre-sprint effort.

while profiling i found a recursive implementation of a "list flatten" function, which was called way too often. i figured this could be done better and wanted to see what approaches other people had used before. a blog post titled more on python flatten sounded promising, but the proposed best method somehow seemed overly complicated and longish to me. next thing of course the perfectionist in me crept up, and well, we were at a sprint after all, so why not have some fun? :)

while the first version worked for my use-case and decreased plone start-up time by about 0.2 seconds (which i thought was not too bad for a small change like that), it was a bit oversimplified in that it didn't consider some egde-case. that wasn't too hard to fix, though, and eventually i posted what i considered good enough as a comment in that blog post:

def flatten(l, ltypes=(list, tuple)):
    n = 0
    for i in iter(l):
        if isinstance(i, ltypes):
            l[n:n+1] = []
            n += 1
            l[n:n] = list(i)
    return l
judging by the fact nobody ever replied to it, i guess its workings might have been not too obvious after all. but today, after someone else posted another version i checked back on the post finding that "this one whoops all of the others for speed, nesting level support and elegance!":
def flatten(l, ltypes=(list, tuple)):
    ltype = type(l)
    l = list(l)
    i = 0
    while i < len(l):
        while isinstance(l[i], ltypes):
            if not l[i]:
                i -= 1
                l[i:i + 1] = l[i]
        i += 1
    return ltype(l)
it was referring back to Mike C. Fletcher's BasicTypes library, hence dubbed "mr. fletcher's version" and even compared to ruby's built-in flatten method (implemented in C), almost living up to it.

i got curious and wanted to see how my version was doing in comparison, and surprisingly it it about 33% faster. and that's after i fixed it to be non-destructive with regard to the passed in list:

$ python flatter.py
fletchers_flatten: 9.197 sec
my_flatten: 5.934 sec
comparison: 64.53 %
and without wanting to brag too much, i still think my version is quite a bit slicker, too... ;)

anyway, here's the whole test script in case you want to run it yourself:

# mr. fletcher's version
def fletchers_flatten(l, ltypes=(list, tuple)):
    ltype = type(l)
    l = list(l)
    i = 0
    while i < len(l):
        while isinstance(l[i], ltypes):
            if not l[i]:
                i -= 1
                l[i:i + 1] = l[i]
        i += 1
    return ltype(l)

# my approach...
def my_flatten(l, ltypes=(list, tuple)):
    ltype = type(l)
    l = list(l)
    n = 0
    for i in iter(l):
        if isinstance(i, ltypes):
            l[n:n+1] = []
            n += 1
            l[n:n] = list(i)
    return ltype(l)

# set up a nested list
a = []
for i in xrange(2000):
    a = [a, i]

# make sure it works as advertised
backup = a
assert my_flatten([[]]) == []
assert my_flatten(a) == list(reversed(xrange(2000)))
assert a == backup

# benchmark
from time import time
now = time()
for n in xrange(5000):
    a = fletchers_flatten(a)
fletcher = time() - now

now = time()
for n in xrange(5000):
    a = my_flatten(a)
my = time() - now

print 'fletchers_flatten: %.3f sec' % fletcher
print 'my_flatten: %.3f sec' % my
print 'comparison: %.2f %%' % (my / fletcher * 100)

MJ was right, of course. the fixed version reads like this...

def my_flatten(l, ltypes=(list, tuple)):
    ltype = type(l)
    l = list(l)
    n = 0
    for i in iter(l):
        if isinstance(i, ltypes):
            l[n:n+1] = []
            l[n+1:n+1] = list(i)
        n += 1
    return ltype(l)

Thursday, January 22, 2009

please downgrade my buildout!

i won't go into discussing particular use cases here — that would perhaps be too weird :) — but let's just say sometimes it can be handy to be able to quickly "downgrade" an existing plone 3.x buildout to plone 2.5. here's a buildout config that does exactly this:
extends = buildout.cfg

recipe = plone.recipe.distros
urls = http://launchpad.net/plone/2.5/2.5.5/+download/Plone-2.5.5.tar.gz
nested-packages = Plone-2.5.5.tar.gz
version-suffix-packages = Plone-2.5.5.tar.gz
eggs =

url = http://www.zope.org/Products/Zope/2.9.8/Zope-2.9.8-final.tgz
fake-zope-eggs = true

products =

Friday, January 9, 2009

what a git!

so here's the real reason i started it — nothing too exciting, but after a hunch of a discussion it simply had to go somewhere... :)
since i just figured the below out for the second time (after forgetting what the suitable git commands were since the first time a few months ago) i thought i might as well write it down. perhaps it'll come in handy at some point... :)

imagine you're offline, in a train or plane or cafe, and feel like getting some work done. you're starting to make changes and quickly realize you'd rather split things into several commits than just a single big one. however, you're working against a subversion repository and have — since you're offline — no way to commit things back right away or (conveniently) keep the patches separate. well, except perhaps for copying the whole directory for every single one and later merge/commit them one by one. not really worth the trouble... but git to the rescue!

even without a previously cloned svn repos it's quite easy to locally commit away and later replay your changes. here's a quick step-by-step howto:

  • initialize an empty git repos, propbably best done in the top dir of your working tree:
  • $ git init
  • add the current state to it (you might have to be more explicit here):
  • $ git add *
    $ git commit -m 'import'
  • do your work, i.e. change things, locally commit to git:
  • $ vi ... # or emacs if you must ;)
    $ git commit -a
being offline you leave it at that. once you're back online you wanna transfer those changes to svn, or, iow, apply them to a cloned git repository:
  • format/export the patches you've made (the rev specifier ":/import.." translates to "between a commit starting with 'import' and head) to separate files:
    $ git format-patch :/import..
    this will output numbered files, one file per patch, like:
  • remove your temporary git repos (or move it out of the way):
  • $ rm -rf .git
  • if you haven't done so already, set up the cloned repos (extra options might be required here), which allows you to commit back to svn:
  • $ git svn clone svn://svn-url
  • re-apply the patches:
  • $ git am 00*.patch
  • and finally commit back to svn:
  • $ git svn dcommit