Why Big Data /Hadoop Implementation Fails

This is a big question which is not been talked by many. Many companies tried to adopt a new technology and getting failed on the implementation . At the end they bury the whole team for the failure. Like wise Hadoop is one of the technologies buried in many companies.

Who uses Hadoop

Hadoop is a fancy technology to be adopted by many organization and they are trying to solving their problem with Hadoop. some of the key problems faced by companies

• Big web logs not able to store and retrieve
• Huge set of Audio files or call logs which needs to be saved some where
• Historical information of the transaction databases

We have multiple problems but is Hadoop is the solution? yes it is. but still why companies are failing.!

Thought process across teams

Senior management : Sees Hadoop as replacement of existing tools and expects it to reduce cost by 50%

Data Architects : They know Hadoop is the highly distributed system but they are not confident to take that path.

Developers : Developer used to work with just one system but they fail to visualize the big picture of the power of executing their code at hundred of servers at a same time.

Aligning the Team expectations

Team should perform the cost benefit analysis of implementing the Hadoop before the implementation such as

• Cost of hardware/cloud nodes for Hadoop
• cost of the software to interact with the Hadoop cluster
• cost of the existing process
• cost expectation or reduction by implementing the new process.