Python 101 – Creating Multiple Processes 2 Comments / Python, Python 3 / By Mike / July 15, 2020 / concurrency, Python Most CPU manufacturers are creating multi-core CPUs now. Even cell phones come with multiple cores! Python threads can’t use those cores because of the Global Interpreter Lock. Starting in Python 2.6, the multiprocessing module was added which lets you take full advantage of all the cores on your machine.
In this article, you will learn about the following topics:
Pros of Using Processes Cons of Using Processes Creating Processes with multiprocessing Subclassing Process Creating a Process Pool This article is not a comprehensive overview of multiprocessing. The topic of multiprocessing and concurrency in general would be better suited in a book of its own. You can always check out the documentation for the multiprocessing module if you need to here:
https://docs.python.org/2/library/multiprocessing.html Now, let’s get started!
Pros of Using Processes There are several pros to using processes:
Processes use separate memory space Code can be more straight forward compared to threads Uses multiple CPUs / cores Avoids the Global Interpreter Lock (GIL) Child processes can be killed (unlike threads) The multiprocessing module has an interface similar to threading.Thread Good for CPU-bound processing (encryption, binary search, matrix multiplication) Now let’s look at some of the cons of processes!
Cons of Using Processes There are also a couple of cons to using processes:
Interprocess communication is more complicated Memory footprint is larger than threads Now let’s learn how to create a process with Python!
Creating Processes with multiprocessing The multiprocessing module was designed to mimic how the threading.Thread class worked.
Here is an example of using the multiprocessing module: