Python is a high-level, interactive, interpreted, and object-oriented scripting language that programmers, particularly data scientists, use worldwide. New versions of Python are released every year, with the final release occurring at the end of the year after a feature-locked beta release in the year’s first half. Python 3.11 is the latest version of Python at the time of writing.
Python 3.11 has been under development for over seventeen months and was recently published in October 2022. It is now ready for widespread usage. Moreover, with this new version comes many new features that make it even better. The documentation for Python 3.11 contains a detailed list of all the changes and enhancements that are included in this version. In this post, we’ll examine the most interesting and useful new features that Python 3.11 offers and finally take a look at everyone’s’ favorite free python compiler and IDE: PyScripter.
Table of Contents
Is code execution faster in this update?
Out of all the improvements that have been made to Python 3.11, the specializing adaptive interpreter is perhaps the most significant addition to its performance improvements.
Because the type of an object rarely changes, the interpreter now tries to analyze running code and replace generic bytecodes with type-specific ones. For example, binary operations like addition and subtraction can be replaced with specialized versions for integers, floats, and strings.
Python function calls in Python 3.11 also have less overhead. Stack frames for function calls are now designed to use less memory and to be more efficient.
According to the official Python benchmark suite, Python 3.11 is approximately 1.25 times faster than Python 3.10. However, it is important to note that this speedup is an aggregate measure. Some things move much faster, while others move only slightly faster or the same. Still, the best part about these enhancements is that they are free, as no code changes are required for Python programs to benefit from Python 3.11’s speedups.
Has the syntax for asynchronous tasks changed?
Python’s asynchronous programming support has evolved over time. It all began with the introduction of generators in Python 2. Then, Python 3.4 introduced the asyncio library, and Python 3.5 introduced the async and await keywords. The development has continued in subsequent releases, with many minor enhancements to Python’s asynchronous capabilities.
Task groups are the latest addition and provide a cleaner syntax for running and monitoring asynchronous tasks.
The traditional method for running multiple asynchronous tasks with asyncio has been to first create them with create_task() and then await them with gather(). This gets the job done but is a pain to work with. To address the issues associated with child tasks, Python 3.11 introduces asyncio task groups. When you use gather() to organize your asynchronous tasks, part of your code will typically look like this:
|
1 2 3 4 |
tasks = [] for p in params: tasks.append(asyncio.create_task(run_some_task(p))) await asyncio.gather(*tasks) |
Before passing them to gather(), you manually track all of your tasks in a list. By awaiting on gather(), you ensure that each task is completed before proceeding.
With task groups, the equivalent code is simpler. Instead of gather(), you use a context manager to specify when tasks should be awaited:
|
1 2 3 |
async with asyncio.TaskGroup() as group: for p in params: group.create_task(run_some_task(p)) |
You create a TaskGroup object named group in this example, and use its .create_task() method to create new tasks.
Do you get more information as a Pythoneer?
Are tracebacks more informative in Python 3.11?
Python is a popular beginner programming language because of its readable syntax and other user-friendly features. However, interpreting the tracebacks displayed whenever Python encounters an error is a challenge for all, especially for those new to Python.
Traceback enhancement is one of Python 3.11’s most anticipated features and improves your developer experience. For example, decorative annotations are added to the tracebacks to help you quickly interpret an error message.
Let us take a closer look at an example of enhanced tracebacks. Start by adding the following code to a file named log.py:
|
1 2 3 4 5 |
import math def log(number): return math.log(number) print(log(0)) |
The math.log() method returns the natural log of a number. However, because log 0 is undefined, your code will raise an error when you run it:
|
1 2 3 4 5 6 7 |
Traceback (most recent call last): File "/home/User/log.py", line 5, in <module> print(log(0))) ^^^^^^ File "/home/User/log.py", line 3, in log return math.log(number) ^^^^^^^^^^^^^^^^ |
Note that the ^ and ~ symbols are embedded in the traceback and are used to guide attention to the part of the code that is causing an error.
Having this extra assistance in spotting errors is beneficial. If your code is more complex, annotated tracebacks become even more powerful. These may be able to convey information that the traceback alone could not previously.
To better appreciate the power of the improved tracebacks, let’s build a small parser that parses information about some individuals. For example, assume you have a file named people.json with the following content:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[ {"name": {"first": "Ali"}}, { "name": {"first": "Gauri", "last": "Kumar"}, "birth": {"year": 1968, "month": 2, "day": 9}, "death": {"year": 2019, "month": 1, "day": 31} }, { "name": {"first": "Ayesha", "last": "Ahmad"}, "birth": {"year": 2000, "month": 10, "day": 1}, "death": null }, { "name": {"first": "Harsh", "last": "Dutt"}, "birth": {"year": 1981}, "death": {"month": 1, "day": 31} } ] |
It should be noted that the information available about these people is quite erratic. For example, while the information about Gauri Kumar is complete, Harsh Dutt's birth date, month, and death year are missing. In addition, the file only contains Ali's first name.
Now, let us create a class that can wrap this information up. We will start by reading the information from the people.json file we have created above. Let’s create a people.py file with the below code:
|
1 2 3 4 5 6 |
import json import pathlib people = json.loads( pathlib.Path("people.json").read_text(encoding="utf-8") ) |
The pathlib package is used to read the JSON file, and json package is used to parse the data into a Python list of dictionaries. Following that, we’ll use a dataclass to store information about each programmer:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from dataclasses import dataclass @dataclass class Person: name: str lifetime: tuple[int, int] @classmethod def from_dict(cls, info): return cls( name=f"{info['name']['first']} {info['name']['last']}", lifetime=f"{info['birth']['year']} to {info['death']['year']}", ) |
Each individual will be given a name and a lifetime attribute. You also include a handy constructor that can initialize a Person object based on the information and structure in your JSON file. We can now go through our code and look for any errors. Run your program with the -i flag in the terminal to open Python’s interactive REPL, which includes all variables, classes, and functions:
|
1 |
python -i people.py |
Let’s see what happens when we have complete information about a person:
|
1 |
Person.from_dict(people[1]) |
We will get the following output:
|
1 |
Person(name='Gauri Kumar', lifetime=('1968 to 2019')) |
Gauri's information is complete, so you can encapsulate her into a Person object with details like her full name and life span.
Try converting Ali to see the new traceback in action:
|
1 |
Person.from_dict(people[0]) |
Output:
|
1 2 3 4 5 |
Traceback (most recent call last): File "/home/Users/people.py", line 18, in from_dict name=f"{info['name']['first']} {info['name']['last']}", ~~~~~~~~~~~~^^^^^^^^ KeyError: 'last' |
Because last is missing, you get a KeyError. While you may recall that last is a subfield within the name, the annotations make this clear immediately.
What happens with Ayesha? You only have information about her birth:
|
1 |
Person.from_dict(people[2]) |
Output:
|
1 2 3 4 5 |
Traceback (most recent call last): File "/home/User/people.py", line 19, in from_dict life_span=f"{info["birth"]["year"]} to {info["death"]["year"]}", ~~~~~~~~~~~~~^^^^^^^^ TypeError: 'NoneType' object is not subscriptable |
A TypeError is thrown in this case. You’ve probably seen 'NoneType' type errors before. They are notoriously difficult to debug as it is not always clear which object is unexpectedly None. However, as you can see from the annotation, info["death"] in this example is None.
These tracebacks make debugging in Python 3.11 easier than in previous versions, boosting your productivity as a Python developer.
Is the error-handling mechanism enhanced in Python 3.11?
Python’s error-handling mechanism has received many new features in Python 3.11.
Now, Python can handle multiple exceptions that can be raised and handled using the new except* syntax and the all-new ExceptionGroup exception type. This enables the elegant handling of problems when many errors can be raised simultaneously, such as when dealing with concurrent or asynchronous methods or when dealing with multiple failures when you retry an action.
Furthermore, exceptions now have no cost to a program unless they are really raised. This means that the default path for a try/except block is a lot quicker and consumes less memory. In addition, the amount of time required to catch an exception has been lowered by about 10%.
This means that exceptions can be enriched with contextual notes that remain separate from the text of the exception itself.
Let’s look at an example:
|
1 2 3 4 5 |
try: raise TypeError('Bad type!') except Exception as e: e.add_note('Add some contextual information') raise |
When this error occurs:
|
1 2 3 4 |
Traceback (most recent call last): File "<stdin>", line 2, in <module> TypeError: Bad type! Add some contextual information |
What kind of typing improvements should you expect in Python 3.11?
Python’s type-hinting features, which have grown dramatically with each iteration since Python 3.5, make larger codebases easier to maintain and analyze.
Let’s go over the several new type-hinting additions made in Python 3.11:
Self Type
Class methods in the previous versions of Python required verbose and obtuse annotations to be useful. Python 3.11’s all new typing.Self lets you simply annotate the return values of a class method as Self. With this, you get predictable and useful results from whatever analysis tool you may want to use.
Arbitrary String Literal Type
Previously, there was no method to declare that a given variable needed to be a string literal using type annotations (a string defined in source code). The new typing.LiteralString annotation corrects this. Using the new annotation, linters may determine whether a variable is a string defined in the source or a new string made entirely of source-defined strings.
Variadic Generics
Python 3.11 introduces TypeVarTuple, sometimes known as “variadic generics,” which allows you to define a placeholder for a succession of types given as a tuple. This would be especially handy in libraries like NumPy, where you could check for problems like whether a supplied array was the correct shape ahead of time.
So this new feature will come in handy to all Data Science enthusiasts.
What is TOML?
TOML, which stands for Tom’s Obvious Minimal Language, is a configuration file format. It is supposed to translate unambiguously to a dictionary and be easy to read and write due to straightforward semantics that strive to be “minimal.” Its specification is open-source and receives additions from the community.
Does Python 3.11 allow TOML configuration parsing?
TOML was created to be simple for humans to read and simple for machines to parse.
TOML is used as a configuration format in Python (the preferred format for setting the metadata for packages and projects), however you cannot read TOML files as it is not available as a standard library module.
Python 3.11 introduces tomllib to address this issue, as tomllib is now part of the standard library. In addition, this new module adds support for parsing TOML files by adding to the existing and quite popular tomlib third-party library.
The following is an example of a TOML file named units.toml:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
# addresses.toml [home] street = "123 Tornado Alley Suite 16" city = "New Jersey" state = "NY" [office] street = "11 Broadway" city = "New York City" state = "NY" [school] street = "203 Oak Street" city = "Maryland" state = "NY" |
The file contains several sections with headlines in square brackets. Each such section is called a table in TOML, and its headline is called a key. Tables contain key-value pairs and can be nested such that values are new tables.
The example above shows that each table has the same structure, with three keys: street, city, and state. Of course, values can have different types, but in this example, we use only strings. Now, let’s use tomlib package to read a TOML file. Let’s create a Python file test_toml.py:
|
1 2 3 4 5 |
import tomllib with open("units.toml", mode="rb") as file: info = tomllib.load(file) print(info) |
Output:
|
1 2 3 |
{'home': {'street': '123 Tornado Alley Suite 16', 'city': 'New Jersey', 'state': 'NY'}, 'office': {'street': '11 Broadway', 'city': 'New York City', 'state': 'NY'}, 'school': {'street': '203 Oak Street', 'city': 'Maryland', 'state': 'NY'}} |
For using tomllib.load(), you must pass in a file object that is opened in binary mode. This is done by specifying mode= "rb".
Moreover, rather than reading the file with open() context manager, you can parse a string with tomllib.loads(). Now comment out the above code and copy paste the below code to test_toml.py file:
|
1 2 3 4 5 6 |
import tomllib import pathlib units = tomllib.loads( pathlib.Path("units.toml").read_text(encoding="utf-8") ) print(units) |
Output:
|
1 2 3 |
{'home': {'street': '123 Tornado Alley Suite 16', 'city': 'New Jersey', 'state': 'NY'}, 'office': {'street': '11 Broadway', 'city': 'New York City', 'state': 'NY'}, 'school': {'street': '203 Oak Street', 'city': 'Maryland', 'state': 'NY'}} |
In this example, pathlib is used to read units.toml into a string, which is then parsed with loads(). Therefore, TOML documents should be saved in UTF-8 format. To ensure that your code runs consistently across platforms, you should explicitly specify the encoding.
Should you install Python 3.11?
The most significant gains with Python 3.11 are improved error messages and faster code execution, both of which improve the developer experience. These are excellent motivations to improve the environment that you use for local development as soon as possible. This is also the least risky type of upgrade, as any bugs you encounter should have a minor impact. The increased speed is another compelling reason to update your production environment.
However, when upgrading, it is critical to consider improved syntax. You can’t use syntax like TaskGroup() or except* in your code if you’re maintaining a library that supports older versions of Python. The good news is that your library will still be faster for anyone using Python 3.11.
Does PyScripter support the latest Python update?
Well done on getting through this article! A new release of Python is always caused for celebration and acknowledgment of all the effort poured into the language by volunteers from around the world.
This tutorial has new features and improvements, better error messages and handling, faster code execution, task groups, typing features, native TOML support, and much more.
If you have programmed in Python, you must have heard of PyScripter. PyScripter is significantly faster and more responsive than bloated text editors or other Python cross-platform IDEs.
It has a built-in Python interpreter that offers call hints and code completion. This program allows you to run scripts without saving them and keeps track of your command history. For debugging Python code, this IDE also has a remote Python debugger. As a result, it’s possible to see variables, the watch window, and the call stack.
PyScripter’s features include brace highlighting, code folding, code completion, and syntax checking as you type. Furthermore, the use of Python source code tools simplifies programming for programmers. PyScripter also allows you to drag and drop files from Explorer into this IDE, which could save you time.
Since PyScripter is one of the best IDEs for coding in Python, the question naturally arises whether or not this feature-packed IDE supports Python 3.11. And the answer is that it will soon, with PyScripter’s next update, hopefully!
In the meantime, why don’t you master your Python skills in PyScripter, so that once its update is released, you can use Python 3.11 to the fullest? Click here to get some great free Python tools and get started.
