Python Profiler Tools

There are different profiler tools for python to help you better estimate the memory and CPU usage.

cProfile

C EXTENSION PROFILE to record the function time spent and so on

cProfile is C extension to analyze the running-time of the program. Basically, you can just import cProfile, and run a simple program. According to the image, you can see that there are 5 function calls, and the details are listed in the output window. Is there any difference if you type a recursion program? Let’s figure it out.

you can see the output said 204 function calls, but 5 primitive calls. It means the program was induced via recursion

We also can see the table here.

ncalls means number of calls

tottime means the time spent on that step

percall means the quotient of tottime divided by ncalls

cumtime means the cumulative time spent

percall means the quotient of cumtime divided by ncalls

Filename:lineno(function) shows the data of each step

pyinstrument

Pyinstrument is another python profiler. With pyinstrument you can record the duration of programs with wall clock time.

My program just have a print(“welcome to Funtern”), and you can see the output said that No sample were recorded, that is because your program executed in under 1ms.

If we want to see the result, we have two options to do. The first one is to set a interval, and second one is to have a more complex function.

Let's try the recursion function in Part A, and set a interval to 0.0001 sec.

So we can see a result on the output window.

yappi

yappi is an alternative Python profiler, which offers profiling for multi-threaded applications and CPU time.

NOTE: before you install and use yappi, you need to have VS C++ on your computer because it is written in C, so make sure you downloaded VS desktop development tools.

The output of yappi is similar to what we get from cProfile, but it can analyze the thread that cProfile cannot do that.

Other Tools

There are other paid tools, like Datadog, New Relic and APPDYNAMICS, etc.

They are more functional to track the performance of python code. You can see the real map when your code is executed. You can sign up for 15 days' free trial.

FUNTERN Associate: Jingrun Zhang