One of Python’s long-standing weaknesses, its inability to scale well into multi-threaded environments, is the target of a new proposal among core developers of the popular programming language.

Developed by Sam Gross she has Suggestion Big change to Global Interpreter Lock or GIL – a key component of CPython, the reference implementation of Python.

If accepted, Gross’ proposal would rewrite the way Python serializes access to objects at run time from multiple threads, and would greatly enhance multithreaded performance.

The GIL has long been seen as an obstacle to better multithreaded performance in CPython (and thus Python in general). Many efforts have been made to remove it over the years, but at the cost of harming single-threaded performance – in other words, by making the vast majority of current Python implementations slower.

The current Python metaphors for dealing with threading and multiprocessing do not make it impossible to achieve high parallelism. But they make it difficult enough that developers often turn to third-party modules like Dask to get the job done.

new proposal Makes changes to the way reference counting works for Python objects, so that references from a thread that owns an object are treated differently than those from other threads.

The overall effect of this change, and a number of others with it, actually boosts single-string performance slightly – about 10%, by some benchmarks made on a forked version of the interpreter versus the main CPython 3.9 compiler. Multithreaded performance, in some benchmarks, is measured roughly linearly with each new thread in the best case – for example, when using 20 threads, 18.1x acceleration on one scale and 19.8x acceleration on the other.

These changes are large enough that quite a few existing Python libraries that work directly with internal Python (for example, Cython) will need to be rewritten. But the rhythm of Python’s release schedule only means that such urgent changes should be made in a major point release rather than a minor release.

Copyright © 2021 IDG Communications, Inc.

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *