Hadoop Developer Job Description

Hadoop is a framework used for distributed computing, based on a paradigm of processing large datasets across clusters of commodity servers using simple software frameworks.

Hadoop was originally developed by Yahoo! Research in 2004 as a successor to their File System Redeemable Block Store (FS-RBSTM).

In 2009, Yahoo! sold the Hadoop trademark to Hortonworks, which introduced the Hortonworks Data Platform.

Hadoop is already widely used by large organizations for large-scale data processing.

Job Brief:

We’re looking for an experienced Hadoop Developer to join our team. You will be responsible for developing and managing Hadoop applications in our environment. In this role, you will work closely with our data team to ensure that our data is stored efficiently and securely. If you are a self-starter with a passion for big data, then this is the role for you!

Hadoop Developer Duties:

• Implement and develop Hadoop-based solutions to the business
• Recommend and authorize the use or modification of existing/standard code
• Contribute to standard development practice, processes, and guidelines
• Proactively seek opportunities to increase efficiency, automation, and scalability
• Maintain comprehensive understanding of Hadoop ecosystem and best practices for effective development
• Support and resolve production support issues
• Develop efficient, clean, and reusable scripts
• Develop metrics and analysis around performance
• Perform other duties as assigned

Hadoop Developer Responsibilities:

• Bachelor’s degree in Computer Science (B.S.), or an equivalent combination of education and experience
• 2+ years professional experience with Hadoop/Map-Reduce and Java
• Experience in developing Hive queries and programs
• Experience with SQL and relational databases

Requirements And Skills:

• Develop, maintain, and support Big Data solutions using Hadoop and other Hadoop-related technologies
• Create and maintain appropriate data models and analytics
• Write MapReduce programs, HiveQL statements, Pig scripts, and Pig Latin statements
• Analyze data using SQL and Python
• Use and monitor Hadoop monitoring tools, including Hadoop logs and Cloudera Manager
• Build, deploy, and maintain Hadoop clusters
• Work with team members to develop and test new MapReduce jobs, data pipelines, and data tables
• Communicate with teammates and project managers to define, track, and report daily and weekly production metrics
• Design and implement data architecture and ETL solutions, and ensure data integrity
• Assist with ETL development
• Support production environment with operating system management and upgrades
• Conduct data engineering, development, and migration activities

At [Company Name], we celebrate diversity and are committed to building an inclusive team. We encourage applications from people of all races, religions, national origins, genders, and ages, as well as veterans and individuals.