How to create a directory structure in memory with Python?

Asked

Viewed 25 times

-1

Hello. I have the following scenario: I am reading a dataframe, which has a column with Xmls. I am writing these Xmls to files. xml separately, to later save them in a certain folder structure and finally zip this structure. But I must do it in memory.

Follow the code attempt to date:

import os
import zipfile
import pandas as pd
import csv
from io import StringIO, BytesIO

df = pd.read_csv(arq.csv, low_memory=False)

version = '1'
year = '2021'
month = 'august'
day = '1'
my_id = 'some_id'
memory_zip = BytesIO()    
memory_xml = StringIO()
memory_path = StringIO()
    
with zipfile.ZipFile(memory_zip, 'w', compression=zipfile.ZIP_DEFLATED, compresslevel=9) as zf: 
     for line in range(len(df)):
          path = version+'/'+year+'/'+month+'/'+day+'/'+my_id+'.xml'
          if (memory_path.getvalue() == path):
               pass
          else:
               memory_path.write(path)
          df.iloc[line,:].to_csv(memory_xml, header=False, index=False, escapechar='\\', quoting=csv.QUOTE_NONE) 
          zf.writestr(memory_path, memory_xml.getvalue())

However, I am getting the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-62-73f2d5a6d220> in <module>
---> 74         zf.writestr(memory_path, memory_xml.getvalue())

~\Anaconda3\lib\zipfile.py in writestr(self, zinfo_or_arcname, data, compress_type, compresslevel)
   1786             data = data.encode("utf-8")
   1787         if not isinstance(zinfo_or_arcname, ZipInfo):
-> 1788             zinfo = ZipInfo(filename=zinfo_or_arcname,
   1789                             date_time=time.localtime(time.time())[:6])
   1790             zinfo.compress_type = self.compression

~\Anaconda3\lib\zipfile.py in __init__(self, filename, date_time)
    347         # Terminate the file name at the first null byte.  Null bytes in file
    348         # names are used as tricks by viruses in archives.
--> 349         null_byte = filename.find(chr(0))
    350         if null_byte >= 0:
    351             filename = filename[0:null_byte]

AttributeError: '_io.StringIO' object has no attribute 'find'

Could you help me, please?

Thank you in advance!

1 answer

0


Fixed: instead of using "/" in the path, just switch to inverted double quotes.

  • Could explain better the solution?

  • 1

    Use the os.path.join(). Something like: path = os.path.join(version, year, month, day, f"{my_id}.xml"). Don’t forget to import the os with import os. With this the Windows "" or the Linux and Mac "/", are abstracted by the method.

Browser other questions tagged

You are not signed in. Login or sign up in order to post.