The concurrent.futures module can be used to run tasks in parallel in Python. Here is an example:
import concurrent.futures
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers = 10) as executor:
futures = [executor.submit(perform_task, task) for task in tasks]
results = [future.result() for future in futures]
The ProcessPoolExecutor class uses a pool of processes to execute tasks asynchronously. The submit function immediately returns a Future object, and you can call future.result(), which will block until the task has completed.
Note that there is also a ThreadPoolExecutor class which uses a pool of threads; however, due to the Global Interpreter Lock, only one thread can execute python bytecode at any one time, which means that you will not achieve any parallelisation in most cases.