Evolution of Google File System August 08, 2009 07:26:40 EDT There is an interesting interview about the evolution of the Google File System in ACM Queue. I think it is readable by anybody, not just ACM members. One of the morals of this story is that, even if you are building what you think will be the world's biggest, you still will make design decisions that you know are not scalable because you know how to implement them. It is better to get something running right away and start using it. Of course, they also ran into scalability problems that they did not expect. So, some of the evolution of GFS was planned, and some was unplanned.
Sqoop is a tool designed to import data from relational databases into Hadoop. Sqoop uses JDBC to connect to a database. It examines each table’s schema and automatically generates the necessary classes to import data into the Hadoop Distributed File System (HDFS). Sqoop then creates and launches a MapReduce job to read tables from the database via DBInputFormat, the JDBC-based InputFormat. Tables are read into a set of files in HDFS. Sqoop supports both SequenceFile and text-based target and includes performance enhancements for loading data from MySQL.
P. Sethia, and K. Karlapalem. Engineering Applications of Artificial Intelligence, 24 (7):
1120--1127(2011)Infrastructures and Tools for Multiagent Systems.