Python has an easy interface to use the threading module. Threading allows us to run multiple programs concurrently. There are many methods to run threads. I shared the top four methods with examples here to understand the Python threading concept easily.
Example 1: Using threading Module Directly
In this example, we import the threading module and call the Thread class of the module to start the thread.
Arguments
The Thread() class needs two arguments to be passed in order to initialize a thread.
- target – It is used to call the function we want to execute
- args – we need to pass the arguments to the function we need to execute
Below, I shared a simple program to understand the threading concept. The program will call the function 10 times and each function will print the message after a time delay.
Even though we call the function in order, the output order is not as same as the order of the execution.
import threading
import time
def thread(num):
time.sleep(2)
print(f'this is a thread => {num}\n')
if __name__ == '__main__':
for n in range(4):
threads = threading.Thread(target=thread, args=(n,))
threads.start()Output
this is a thread => 3
this is a thread => 2
this is a thread => 1
this is a thread => 0Example 2: Multithreading
Python threading module is widely used for multithreading purposes. We can understand the multithreading concept better with an easy example. In the below example, I created a program to send an HTTP request to a URL and print the status code.
Here we create a list to store each thread and finally call the join() method in order for the thread to complete the work. If you don’t call the join() method, the program will be executed but it won’t wait for the threads to complete their tasks. So, the program will be terminated before executing all the threads.
import requests
import threading
def send_request(num):
response = requests.get('https://www.google.com')
print(f'thread = {num} and response = {response.status_code}')
if __name__ == '__main__':
threads = []
for n in range(6):
thread = threading.Thread(target=send_request, args=(n,))
threads.append(thread)
thread.start()
for thread in threads:
thread.join() # call join method for the thread to complete the work
Output
thread = 1 and response = 200
thread = 4 and response = 200
thread = 3 and response = 200
thread = 5 and response = 200
thread = 2 and response = 200
thread = 0 and response = 200Example 3: Python Multithreading with Limit
By default, Python doesn’t set any limit for thread limit. We have to set it to avoid program crashing due to high CPU load. We can set the thread limit by using ThreadPoolExecutor class.
For example, if we set the concurrent threads limit to 10 and you initiate 100 threads at a time, Python will process them using the queue. So, the first 10 threads will do the task and once it is complete, the next batch will be executed.
from datetime import datetime
from concurrent.futures import ThreadPoolExecutor
def send_request(num):
print(f'thread = {num}, the time of execution = {str(datetime.now())}\n')
if __name__ == '__main__':
pool = ThreadPoolExecutor(max_workers=5)
threads = []
for n in range(20):
pool.submit(send_request, n)
pool.shutdown(wait=True) # program will be terminated only after all the threads complete the task
Output
In the above example, we set the thread limit to 5. So, at any time, only 5 threads can be run concurrently. Check the time of execution to understand the threading better.
thread = 0, the time of execution = 2022-03-26 01:06:06.365154
thread = 1, the time of execution = 2022-03-26 01:06:06.365154
thread = 2, the time of execution = 2022-03-26 01:06:06.366155
thread = 3, the time of execution = 2022-03-26 01:06:06.366155
thread = 4, the time of execution = 2022-03-26 01:06:06.366155
thread = 5, the time of execution = 2022-03-26 01:06:06.366155
thread = 6, the time of execution = 2022-03-26 01:06:06.366155
thread = 7, the time of execution = 2022-03-26 01:06:06.366155
thread = 8, the time of execution = 2022-03-26 01:06:06.366155
thread = 9, the time of execution = 2022-03-26 01:06:06.367167
thread = 10, the time of execution = 2022-03-26 01:06:06.367167
thread = 11, the time of execution = 2022-03-26 01:06:06.367167
thread = 12, the time of execution = 2022-03-26 01:06:06.367167
thread = 13, the time of execution = 2022-03-26 01:06:06.367167
thread = 14, the time of execution = 2022-03-26 01:06:06.367167
thread = 15, the time of execution = 2022-03-26 01:06:06.367167
thread = 16, the time of execution = 2022-03-26 01:06:06.367167
thread = 17, the time of execution = 2022-03-26 01:06:06.367167
thread = 18, the time of execution = 2022-03-26 01:06:06.367167
thread = 19, the time of execution = 2022-03-26 01:06:06.367167Example 4: Multithreading Using Queue
This example is an extension of the example 2. Here, we use Queue module to collect the data from the threads we run.
This method is very useful if you would like to run threads that return values after execution.
import threading
import queue
def return_number(num):
print(f'this is the output from {num}')
q.put(num) # add value to the queue
if __name__ == '__main__':
threads = []
q = queue.Queue()
for n in range(6):
thread = threading.Thread(target=return_number, args=(n,))
threads.append(thread)
thread.start()
for thread in threads:
thread.join() # call join method for the thread to complete the work
for i in range(6):
print(f'output from queue = {q.get()}') # collect values from threads
Output
this is the output from 0
this is the output from 1
this is the output from 2
this is the output from 3
this is the output from 4
this is the output from 5
output from queue = 0
output from queue = 1
output from queue = 2
output from queue = 3
output from queue = 4
output from queue = 5Conclusion
Among the above four methods, I suggest people the 3rd method that uses ThreadPoolExecutor to limit the threads. This method is easy to implement and works very faster and CPU optimized.
References:
