How to download ALL files from a folder with requests?

Question

I found several articles on the internet about how to download a single file, but I always need to specify the name and directory of the file.

The program I would like to do needs to download all the files in this directory (eg: 191.15.5.1/pasta) and save them in the default directory where the app is.

I did something like that, but I couldn’t finish it:

def get_arqchives(url):
    res = requests.get(url)
    if res.status_code == requests.codes.OK:
       for u in res.content:
           with (endereco, 'wb') as target:
               pass
    else:
       res.raise_for_status()

Answer 1

In an HTTP folder we only receive HTML with the list of files. We have to use Beautifulsoup to get the links.

from bs4 import BeautifulSoup
import requests
from pathlib import Path

url = '191.15.5.1/pasta/'

def listFD(url):
    page = requests.get(url).text
    soup = BeautifulSoup(page, 'html.parser')
    return [url + '/' + node.get('href') for node in soup.find_all('a') if node.get('href')]

def downloadFile(file):
    r = requests.get(file)
    path = Path(file)
    with open(path.name, 'wb') as f:
        f.write(r.content) 

for file in listFD(url):
    downloadFile(file)