cute_profile

The cute_profile module allows you to profile your code (i.e. find out which parts make it slow) by giving a nicer interface to the cProfile library from Python’s standard library.

What is “profiling”?

(Programmers experienced with profilers may skip this section.)

To “profile” a piece of code means to run it while checking how long it takes, and how long each function call inside the code takes. When you use a “profiler” to profile your program, you get a table of (a) all the functions calls that were made by the program, (b) how many times each function was called and (c) how long the function calls took.

A profiler is an indispensable programming tool, because it allows the programmer to understand which parts of his code take the longest. Usually, when using a profiler, you discover that only a few small parts of his code take most of the runtime of your program. And quite often, it’s not the parts of code that you thought were the slow ones.

Once you realize which parts of the program cause slowness, you can focus your efforts on those problematic parts only, optimizing them or possibly redesigning the way they work so they’re not slow anymore. Then the whole program becomes faster.

Profiling Python code with cute_profile

Python supplies a module called cProfile in its standard library. cProfile is a good profiler, but its interface can be inconvenient to work with. The cute_profile module has a more flexible interface, and it uses cProfile under the hood to do the actual profiling.

Let’s profile an example program. Our example would be a function called get_perfects, which finds perfect numbers:

>>> def get_divisors(x):
...     '''Get all the integer divisors of `x`.'''
...     return [i for i in xrange(1, x) if (x % i == 0)]
...
>>> def is_perfect(x):
...     '''Is the number `x` perfect?'''
...     return sum(get_divisors(x)) == x
...
>>> def get_perfects(top):
...     '''Get all the perfect numbers up to the number `top`.'''
...     return [i for i in xrange(1, top) if is_perfect(i)]
>>> print(get_perfects(20000))

The result is [6, 28, 496, 8128]. However, this function takes a few seconds to run. That’s fairly long. Let’s use cute_profile to find out why this function is taking so long. We’ll add the cute_profile.profile_ready() decorator around get_perfects:

>>> from python_toolbox import cute_profile
>>> @cute_profile.profile_ready()
... def get_perfects(top):
...     '''Get all the perfect numbers up to the number `top`.'''
...     return [i for i in xrange(1, top) if is_perfect(i)]

Now before we run get_perfects, we set it to profile:

>>> get_perfects.profiling_on = True

And now we run it:

>>> print(get_perfects(20000))

We still get the same result, but now a profiling table gets printed:

      60000 function calls in 7.997 seconds

Ordered by: cumulative time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1    0.000    0.000    7.997    7.997 <string>:1(<module>)
     1    0.020    0.020    7.997    7.997 <pyshell#1>:2(get_perfects)
 19999    0.058    0.000    7.977    0.000 <pyshell#0>:5(is_perfect)
 19999    7.898    0.000    7.898    0.000 <pyshell#0>:1(get_divisors)
 19999    0.021    0.000    0.021    0.000 {sum}
     1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

This table shows how long each function took. If you want to understand exactly what each number says in this table, see cProfile.run().

The tottime column says how much time was spent inside this function, across all calls, and without counting the time that was spent in sub-functions. See how the get_divisors function in our example has a very high tottime of 7.898 seconds, which is about 100% of the entire run time. This means that get_divisors is what’s causing our program to run slow, and if we’ll want to optimize the program, we should try to come up with a smarter way of finding all of a number’s divisors than going one-by-one over all numbers.

profile_ready has a bunch of other options. In brief:

  • The condition argument is something like a “breakpoint condition” in an IDE: It can be a function, usually a lambda, that takes the decorated function and any arguments and returns whether or not to profile it this time.
  • off_after means whether you want the function to stop being profiled after being profiled one time. Default is True.
  • sort is an integer saying by which column the final results table should be sorted.

Table Of Contents

Previous topic

cute_iter_tools - documentation not written

Next topic

cute_testing - documentation not written

This Page