Error trying to download python BMF zip

Question

I am trying to download the BMF zip files at once (ftp://ftp.bmf.com.br/MarketData/BMF/), but an error appears in which the program does not find the file, however it exists.

follows the code:

from io import BytesIO
from urllib.request import urlopen
from zipfile import ZipFile
import pandas as pd

startdate = '20150130'
enddate = '20150210'
extension = '.zip'
daterange = pd.date_range(startdate, enddate)

for single_date in daterange:
zipurl = 'ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_' + single_date.strftime('%Y%m%d') + extension
with urlopen(zipurl) as zipresp:
    with ZipFile(BytesIO(zipresp.read())) as zfile:
        zfile.extractall('C:/Users/.../.../...')

The error that appears is basically that it does not find the file...

ftplib.error_perm: 550 NEG_BMF_20150131.zip: The system cannot find the file specified..

There’s an easier way to do that?

Answer 1

Your code is almost perfect. Very good!

I decided to test your code with the following dates:

startdate = '20170130'
enddate = '20170207'

When testing with a shorter period, I could notice the following:

• this error only appears when no deals occurred on the day (Sáb, gift or holidays), that is, the file doesn’t really exist!
• you can use a Try/except structure to get around this error

Code Example:

from io import BytesIO
from urllib.request import urlopen
from zipfile import ZipFile
import pandas as pd

startdate = '20170130'
enddate = '20170207'
extension = '.zip'
daterange = pd.date_range(start=startdate, end=enddate)

for single_date in daterange:
    zipurl = 'ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_' + single_date.strftime('%Y%m%d') + extension
    try:
        with urlopen(zipurl) as zipresp:
            with ZipFile(BytesIO(zipresp.read())) as zfile:
                zfile.extractall()
        print("OK: {}".format(zipurl))
    except:
        print('ERRO: {}'.format(zipurl))

    print("="*5)

Output:

OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170130.zip
=====
OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170131.zip
=====
OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170201.zip
=====
OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170202.zip
=====
OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170203.zip
=====
ERRO: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170204.zip
=====
ERRO: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170205.zip
=====
OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170206.zip
=====
OK: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_20170207.zip
=====

---

Checking the calendar, we can see that 04/02/2017 was a sábado and 05/02/2017 one domingo.

And when entering the folder with the supposed downloads, surprise:

Note that I have removed the output path from the extraction p/test purposes. You can reinclude it in your code (zfile.extractall('C:/Users/.../.../...'))

Answer 2

I use a different strategy. I simply lower what they provide. Of course I keep track of what has already been downloaded not to download again.

First I list the files contained in ftp://ftp.bmf.com.br/MarketData/BMF/:

import FTP

def listar_arquivos_ftp_bmf():    
    ftp = FTP('ftp.bmf.com.br') 
    ftp_dir = '/MarketData/BMF/'
    ftp.login()
    ftp.cwd(ftp_dir)
    arquivos = ftp.nlst()
    ftp.close()

    return arquivos

Then I download each of the files:

import urllib

def baixar_arquivo(url, file_name, CHUNK = 16 * 1024):
    """
    url: caminho do arquivo que será baixado. String.
        Ex.: ftp://ftp.bmf.com.br/MarketData/BMF/NEG_BMF_YYYYMMDD.ZIP

    file_name: caminho e nome do arquivo que será salvo. String.
    """
    response = urllib.request.urlopen(url, timeout=120)

    with open(file_name, 'wb') as f:
        while True:
            chunk = response.read(CHUNK)
            if not chunk:
                break
            f.write(chunk)

    response.close() 

I hope I have help.

I have a large base, but there is a hole. Do you happen to have the NEG_BMF_YYYMMDD.ZIP files from February, March, April and May 2017? You can make them available?

Thank you.