1
I’m trying to create a proxy checker using multithreading, this is my code:
import requests
import threading
from time import sleep
from colorama import Fore
url = 'http://google.com'
proxyList = list(open('proxies.txt').read().splitlines())
timeout = 2
def checarProxy(x):
proxy = {
'http': 'http://' + x
}
try:
a = requests.get(url, proxies=proxy, timeout=timeout)
print(' ' + Fore.GREEN + x)
open('workingProxy.txt', 'a+').write(f'{x}\n')
except:
print(' ' + Fore.RED + x)
threads = []
for proxy in proxyList:
threads.append(threading.Thread(target=checarProxy, args=(proxy,)))
for i in threads:
sleep(0.025)
i.start()
The only way I could narrow down the threads was by using sleep
, I wonder if I can narrow it down by number of threads, something like that:
with maxThreads(10):
for i in threads:
i.start
Is there any way to do this? or am I using multithreading wrong?
This Sleep is kind of "suspicious" there. You need it for what? Something else, if you only want 10 make a range with 10 instead of iterating all. Or a loop that sees how many there are and starts one more if you have less than 10
– Bacco
But my thread list has over 20,000 threads, so I need to know if there’s any way to do this
– João
There are infinite ones, and each one depends on its goal. But from what you said about 20000 threads, either you are doing something VERY different, or you are really making a bad use of them (I would bet on the 2nd option, but without you give more details I’m not sure) - Very likely you should create a smaller thread pool and distribute the tasks on a state machine or something.
– Bacco
I’ll put in my full code then
– João
Try to reduce to a [mcve], and better explain the purpose of the code that helps a lot.
– Bacco
Working with threads it will be inevitable that you understand What is Global Interpreter Lock (GIL).
– Woss
I edited it, I hope it’s clearer now
– João