CodeLearn PythonPythonWindows

The Super Short Guide To Built-In Python Profiling Tools

built in python profiling tools

What is code profiling?

Code profiling is a technique to figure out how time is spent in a program. For more details, a profile is a set of statistics that describes how often and for how long various parts of the program are executed. Using the right Python profiling tools – as well as using the right Python Desktop – can save you lots of time chasing down bottlenecks and slow downs in your Python apps.

With these statistics, we can find the “hot spot” of a program and think about ways of improvement. Sometimes, a hot spot in an unexpected location may give you the hint about bugs in your program.

Does Python have a built-in code profiler?

The Python standard library provides two different implementations of the same profiling interface:

1. cProfile

A C extension with reasonable overhead that makes it suitable for profiling long-running programs. cProfile is a recommended practice for most users. cProfile is built based on lsprof, contributed by Brett Rosen and Ted Czotter.

2. profile

A pure Python module whose interface is imitated by cProfile, but which adds significant overhead to profiled programs. If you’re trying to extend the profiler in some way, your task might be easier with this module. Originally designed and written by Jim Roskind.

How do I get the cProfile library?

As cProfile is a built-in Python library, no further installation is needed.

How do I get the snakeviz library, to visualize the Python profiling tools results?

Here is how you can get a stable release of snakeviz using pip:

How to profile a Python function with cProfile?

For example, run the following code, to profile a function  that takes a single argument from the regex library:

The above action would run re.compile() and print profile results like the following (here is the output on PyScripter IDE):

The Super Short Guide To Built In Python Profiling Tools A cProfile screenshot

Or you can use profile, instead of cProfile, like this:

And it would give you the similar output:

The Super Short Guide To Built In Python Profiling Tools cProfile doing its thing

The first line indicates that 214 calls were monitored. 207 of the calls were primitive, meaning they were not induced by recursion. The next line: Ordered by: standard name, indicates that the text string in the far right column was used to sort the output.

The column headings include:

ncallsThe number of calls.
tottimeThe total time spent in the given function (and excluding time made in calls to sub-functions).
percallThe quotient of tottime is divided by ncalls.
cumtimeThe cumulative time spent in this and all subfunctions (from invocation till exit). This figure is accurate even for recursive functions.
percallThe quotient of cumtime is divided by primitive calls.
filename:lineno(function)Provides the respective data of each function.

How to profile a Python script at a runtime?

Even more usefully, we can profile an existing Python script by running cProfile at a runtime. 

Let’s test it to our web scraping script scrapyApp.py that is used in this post:

You can invoke the cProfile when running a Python script, using this command:

The following is the excerpt of the profiling results:

The Super Short Guide To Built In Python Profiling Tools A screenshot of the results from profiling scrapyapppy

How to save Python profiling tools results using cProfile?

Instead of only printing the profiling result on the command line, we can make it more useful to further results by exporting it into a file. 

Here is how you can do it:

The above command would export the profiling results into a .dump file.

The .dump file would result in jumbles of weird characters. So, if you prefer the readability, you could save the result in .txt file, with this command:

Here is the excerpt of the .txt file, when opened in the text editor:

The Super Short Guide To Built In Python Profiling Tools Saving the cProfile results

How to visualize Python profiling tools results using snakeviz?

To visualize your Python code profiling results, call the .dump file with snakeviz, using this command:

It would start a snakeviz web server and would open the visualization results on your default browser. snakeviz web server started on 127.0.0.1:8080 by default.

You can set up the Style, Depth, and Cutoff of the visualization.

Visualize with Icicle style:

The Super Short Guide To Built In Python Profiling Tools A visualization of profiuling results

Visualize with Sunburst style:

The Super Short Guide To Built In Python Profiling Tools A sunburst visualization

Profiling results in tabular format

Excerpt of all the profiling results in tabular format:

The Super Short Guide To Built In Python Profiling Tools Profiling results shown as a table

Amazing isn’t it? Now you can easily find out the bottleneck in your program using cProfile, and visualize them professionally with snakeviz.

Congratulations, now you have learned about built-in Python profiling tools! Now you can implement it to find the bottleneck of your code, and improve it even more!


 Click here to start using PyScripter, a free, feature-rich, and lightweight IDE for Python developers.

Download RAD Studio to build more powerful Python GUI Windows Apps 5x Faster with Less Code.

Check out Python4Delphi which easily allows you to build Python GUIs for Windows using Delphi.

Also, check out DelphiVCL which easily allows you to build GUIs for Windows using Python.


References & further readings

[1] Davis, M. (2021).

SnakeViz Documentation. GitHub. jiffyclub.github.io/snakeviz

[2] PyPI, Python Package Index. (2021).

snakeviz 2.1.1. Python Software Foundation. pypi.org/project/snakeviz

[3] Python Software Foundation. (2022).

The Python Profilers. Python 3.11.1 documentation. docs.python.org/3/library/profile.html 

[4] Shrivarsheni. (2020).

cProfile – How to profile your python code. Machine Learning Plus. machinelearningplus.com/python/cprofile-how-to-profile-your-python-code

[5] Stack Overflow. (2009).

How do I profile a Python script? stackoverflow.com/questions/582336/how-do-i-profile-a-python-script

[6] Yegulalp, S. (2022).

9 fine libraries for profiling Python code. InfoWorld. infoworld.com/article/3600993/9-fine-libraries-for-profiling-python-code.html

Related posts
CodeIDELearn PythonPythonPython GUITkinter

How To Make More Than 20 ChatGPT Prompts Work With Python GUI Builders And Matplotlib Library?

CodeIDELearn PythonPythonPython GUITkinter

How To Make More Than 20 ChatGPT Prompts Work With Python GUI Builders And Pillow Library?

CodeDelphiDelphiFMXLearn PythonProjectsPythonPython GUI

How To Create A Weather App With The Python Delphi Ecosystem and Weatherstack API

CodeDelphiDelphiFMXLearn PythonProjectsPythonPython GUI

How To Create A Music Player With The Python Delphi Ecosystem

Leave a Reply

Your email address will not be published. Required fields are marked *