2
Situation
I have hundreds (and even thousands) of small files (~50KB) on Amazon S3 separated into Buckets per day.
Problem
I need to download through my Java application delivering to the front end of all files for a certain period. My machine in Cloud is limited in memory and disk resources (it has 2GB of RAM and 5GB of disk).
Solution 1
Download one by one the files and transfer them to the front-end? Solution somewhat inefficient, since it comes to thousands of small files.
Solution 2
Download one by one the files and zip (considering the limits of the machine, breaking the zip in parts if applicable) and upload this zip to Amazon S3, delivering to the front only the zip link.
Question
Is there another solution someone has used, some native AWS resource or some more efficient idea to solve this problem?
I believe your solution is here : https://stackoverflow.com/questions/41764836/amazon-s3-console-download-multiple-files-at-once using AWS CLI you have the option to download multiple files.
– Ronaldo Carvalho
Why not just provide S3 links to the file frontend within the desired period? The application will not need to download anything and the client will be able to access which file needs.
– Woss