What exactly is HDFS

Asked

Viewed 39 times

-1

Good morning, I am studying about Hadoop and a question arose me, for example if you talk a lot about HDFS (Hadoop Distributed File System), which would be the file store of Hadoop, my question is the following, where exactly is or where is this HDFS? Is it a specific place? Or can I create folders anywhere on my computer, place files and consider as being a HDFS.

hadoop

  • https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html (in English) and here (in English) https://www.ibm.com/developerworks/br/library/wa-introhdfs/index.html

1 answer

1


To summarize the long story, it’s like a file system just like you have on your machine for windows (FAT32) or linux (ext2, ext3, Reiserfs, XFS, JFS, UFS) but besides this model have support for distributed systems it is also tolerant to failures. It was HDFS that made Hadoop possible, as it comes to a distributed model it runs the same Map and Reduce on each Node where there is a copy of the file , at home of a Node fail , the job will be guaranteed why Hadoop can find the same file on another Node.

  • Gilberto, I’m sorry but it’s still a little confusing for me, the explanation was good, but I haven’t been able to assimilate very well, for example I have a cluster of machines, and let’s assume that in the machine x I have a folder in documents and I want the hdfs to get the files from there, on the other machine y my folder gets on local disk and I want hdfs to pick it up from there, this is the doubt rs I can have it pick up from any folder? Or it has a specific folder that it creates when installed and the files have to stay there

  • Yes, basically, Voce 'sends' files to the HDFS (Node master) and wait for it to do the rest, in the case of Clusterization to the other nodes. the following command allows you to send files from your local machine to the HDFS Fs put --from source_path_and_file --to dest_path_and_file to run this command , Oce needs a complete installation of Hadoop , preferably on linux .

Browser other questions tagged hadoop

You are not signed in. Login or sign up in order to post.