Dabeaz

Dave Beazley's mondo computer blog. [ homepage | archive ]

Tuesday, February 02, 2010

 

A Context Manager for Timing Benchmarks

I spend a lot of time studying different aspects of Python, different implementation techniques, and so forth. As part of that, I often carry out little performance benchmarks. For small things, I'll often use the timeit module. For example:

>>> from timeit import timeit
>>> timeit("math.sin(2)","import math")
0.29826998710632324
>>> timeit("sin(2)","from math import sin")
0.21983098983764648
>>>

However, for larger blocks of code, I tend to just use the time module directly like this:

import time
start = time.time()
...
... some big calculation
...
end = time.time()
print("Whatever : %0.3f seconds" % (end-start))

Having typed that particular code template more often than I care to admit, it occurred to me that I really ought to just make a context manager for doing it. Something like this:

# benchmark.py
import time
class benchmark(object):
    def __init__(self,name):
        self.name = name
    def __enter__(self):
        self.start = time.time()
    def __exit__(self,ty,val,tb):
        end = time.time()
        print("%s : %0.3f seconds" % (self.name, end-self.start))
        return False

Now, I can just use that context manager whenever I want to do that kind of timing benchmark. For example:

# fileitertest.py
from benchmark import benchmark

with benchmark("iterate by lines (UTF-8)"):
     for line in open("biglog.txt",encoding='utf-8'):
          pass

with benchmark("iterate by lines (Latin-1)"):
     for line in open("biglog.txt",encoding='latin-1'):
         pass

with benchmark("iterate by lines (Binary)"):
     for line in open("biglog.txt","rb"):
         pass

If you run it, you might get output like this:

bash % python3 fileitertest.py
iterate by lines (UTF-8) : 3.903 seconds
iterate by lines (Latin-1) : 3.615 seconds
iterate by lines (Binary) : 1.886 seconds

Nice. I like it already!


Comments:
Thanks! It's also a nice instructional use case for a homemade context manager!
 
I use a decorator for this:

def TimeFunction(func):
"""Decorator to time a function."""
def wrapper(*args, **kwargs):
t1 = time.time()
res = func(*args, **kwargs)
t2 = time.time()
print '%s took %0.3f ms' % (func.func_name, (t2-t1)*1000.0)
return res
return wrapper
 
I have a similar tool but I use pystones instead of seconds to make my measures comparables from one box to another.
 
@Tarek: that is interesting. I'm of course familiar with the overall pystones benchmark in the test suite, but not with how to run tests of targeted user code in pystone context. Do you have sample code I can see? Thanks.

@David: handy. I too write those time.time() calls a lot, and this should save me a few keystrokes.
 
Tarek, I'm also interested in this pystone conversion. Is there some description of that or a formula? I know of a general pystones benchmark, but don't know how it applies to more specific tests I might want to do myself.
 
My guess Tarek is referring to this:
http://code.activestate.com/recipes/440700/

I haven't tried it personally...
 
On Windows, time.clock() should be used instead of time.time().
 

Just for future readers of this, you could also use a contextlib contextmanager:
https://gist.github.com/4031964
 
Hi Dave,

thanks for this useful example of a context manager.
I made a small addition that does "the right thing(TM)" for all platforms,
because the timeit module knows what is best:


# benchmark.py
import timeit
class benchmark(object):

˚˚˚ from timeit import default_timer as timer
˚˚˚ def __init__(self, name):
˚˚˚˚˚˚˚ self.name = name
˚˚˚ def __enter__(self):
˚˚˚˚˚˚˚ self.start = self.timer()
˚˚˚ def __exit__(self, ty, val, tb):
˚˚˚˚˚˚˚ end = self.timer()
˚˚˚˚˚˚˚ print("%s : %0.3f seconds" % (self.name, end-self.start))
˚˚˚˚˚˚˚ return False

# it was a bit hard to make the code readable ;-)
# replace the 'ring above' chars by blanks
 
Post a Comment

Subscribe to Post Comments [Atom]





<< Home

Archives

Prior Posts by Topic

08/01/2009 - 09/01/2009   09/01/2009 - 10/01/2009   10/01/2009 - 11/01/2009   11/01/2009 - 12/01/2009   12/01/2009 - 01/01/2010   01/01/2010 - 02/01/2010   02/01/2010 - 03/01/2010   04/01/2010 - 05/01/2010   05/01/2010 - 06/01/2010   07/01/2010 - 08/01/2010   08/01/2010 - 09/01/2010   09/01/2010 - 10/01/2010   12/01/2010 - 01/01/2011   01/01/2011 - 02/01/2011   02/01/2011 - 03/01/2011   03/01/2011 - 04/01/2011   04/01/2011 - 05/01/2011   05/01/2011 - 06/01/2011   08/01/2011 - 09/01/2011   09/01/2011 - 10/01/2011   12/01/2011 - 01/01/2012   01/01/2012 - 02/01/2012   02/01/2012 - 03/01/2012   03/01/2012 - 04/01/2012   07/01/2012 - 08/01/2012   01/01/2013 - 02/01/2013   03/01/2013 - 04/01/2013  

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]