Is it not possible to use all threads with DUALCPU in PYTHON?

[penpen]

I want to use all CPU threads. Processing is performed using only one of the two CPUs


import numpy as np
a=np.random.random((25000,5000))
b=np.random.random((5000,25000))
c=np.dot(a,b)

--------------------------

import multiprocessing
multiprocessing.cpu_count()

Out64

---------------------------

psutil.cpu_count(logical=False)

Out32

 

[unwind-protect]

Python does not spread its threads to multiple CPUs/cores.

 

[Jorge Perez]

it's probably limited to a single thread group.

 

[mattventura]

Python code itself is single threaded due to GIL (unless you're using a newer GIL-less version, but that's another story), but native-code libraries like numpy may choose to multithread (especially something like numpy which is intended for number crunching).

This might be relevant? NEP 49 — Data allocation strategies — NumPy Enhancement Proposals

 

[penpen]

thanks

 

[CyklonDX]

You can use ThreadPool (again depends on your code)

concurrent.futures — Launching parallel tasks

Source code: Lib/concurrent/futures/thread.py and Lib/concurrent/futures/process.py The concurrent.futures module provides a high-level interface for asynchronously executing callables. The asynchr...

docs.python.org

You will need to use concurrent.futures.ThreadPoolExecutor(max_workers = X) as executor; where you can first query how many threads/cores your system has available and use it as a var to fill max_workers;


or you can try doing threading.Thread class which is potentially less complicated, as you can specify task within each thread; by simply defining thread, starting it, and joining (if you need to join)

Python ThreadPool: The Complete Guide - Super Fast Python

Python ThreadPool, your complete guide to thread pools and the ThreadPool class for concurrent programming in Python.

superfastpython.com

 

[Sean Ho]

It depends on what function you're using; e.g., most optimization in numpy is really just using LAPACK+BLAS, which are parallelised when using the appropriate libraries.

Is your actual workload primarily generating random numbers? If they need to be cryptographically random, consider openssl instead.