Why doesn’t Python use 100% of the processor?

Question

I’m developing a Python application where in some parts, it requires a lot of CPU for calculations.

However, I realize that even at these "bottleneck" points, the CPU never reaches more than 50% of use.

Of course the program gets slower while it has "spare" CPU that could make it faster.

For example, the below process takes time (on my PC), 15 seconds:

from math import *
import time
ini = time.time()

for x in range(10**8):
    a = cos(x)

print ("Tempo Total: ", time.time() - ini)

But during processing, only a few logic processors are used, with only 1 processor suffering more demand, yet not 100%:

How to make Python use 100% CPU in critical processes?

Answer 1

In your case what is happening is the following, you are running your routine on a single thread on a system (CPU) that has 4 logical cores.

You will need to rewrite your algorithm to be multi-threaded, or else check if Voce can run different instances on specific cores. I don’t know any other way.

Come on..

I set up an example with tabular calculation to show the difference in performance between each of the scenarios.

Scenario 1 - NO Thread Execution

import time
ini = time.time()

def tabuada( threadName, numero):
    count = 0
    while count < 300000000:
        count +=1
        #print(" %s x %s = %s\n" % (numero, count, count*numero))
tabuada("Thread-1",2)
tabuada("Thread-1",3)
tabuada("Thread-1",4)


print ("Tempo Total: ", time.time() - ini)

In that execution I had result:

Total time: 35.65071749687195

Scenario 2 - COM Thread execution running in parallel

import time
import _thread
ini = time.time()

def tabuada( threadName, numero):
    count = 0
    while count < 100000000:
        count +=1
        #print(" %s x %s = %s\n" % (numero, count, count*numero))
    print ("Tempo Total: ", time.time() - ini)    

def tabuada2( threadName, numero):
    count = 0
    while count < 100000000:
        count +=1
        #print(" %s x %s = %s\n" % (numero, count, count*numero))
    print ("Tempo Total: ", time.time() - ini)      


def tabuada3( threadName, numero):
    count = 0
    while count < 100000000:
        count +=1
        #print(" %s x %s = %s\n" % (numero, count, count*numero))
    print ("Tempo Total: ", time.time() - ini)          

try:
    _thread.start_new_thread( tabuada, ("Thread12",2,) )
    _thread.start_new_thread( tabuada2, ("Thread14",3,) )
    _thread.start_new_thread( tabuada2, ("Thread15",4,) )
except:
    print('Erro')

In that execution I had result:

Total time: 10.11058759689331

Total time: 11.129863500595093

Total time: 11.548049688339233

Note. The total time in this case was 11.54s, since, that was the time that took the longest thread. And the total time displayed on each line represents how much each thread took in its run.

As we can notice the time falls a lot when we divide the task, but why? Because when we create different threads the OS interprets as different processes and allocates in different locations and cores within my CPU.

So as I said at the beginning, if you want to use more your CPU and optimize processing time I advise you to use threads

Here are two cool links about the content:

Multi-thread programming in python

Python Threads