Big Data Engineer Job Description

A Big Data Engineer is an individual who is skilled in the use of big data.

A Big Data Engineer is involved with many aspects of working with big data including, but not limited to, analysis, systems, and implementation and development.

Job Brief:

We’re looking for a Big Data Engineer to work with us on a variety of data-related projects. You will be responsible for helping us to collect, store, process, and analyze large amounts of data. In addition, you will also be responsible for developing and maintaining our data pipeline.

Big Data Engineer Duties:

• Analyze, design, and implement data infrastructure solutions in support of big data initiatives
• Design and implement data management systems, including database schemas and data models
• Apply technical judgment in the planning, development, implementation, and support of data infrastructure
• Provide guidance and direction for data storage, backup, and recovery
• Design, develop, and support data solutions
• Select appropriate tools/technologies and implement data infrastructure and data management solutions
• Perform thorough analysis of data infrastructure, data store, and data management systems to identify current and potential data bottlenecks, as well as scalability and high availability issues
• Perform data modeling, statistics, and database administration tasks
• Support database management systems, including database administration and development
• Optimize database performance, capacity, and backup and recovery
• Develop data flows, data warehousing, and business intelligence systems
• Develop and implement data

Big Data Engineer Responsibilities:

• Develop, design, and define data capture and storage technologies, including data modeling, to ensure data capture and storage support business intelligence and data warehousing needs
• Develop, design, and implement data integration solutions to enable seamless data integration and data discovery; implement software code to capture and transform data across disparate enterprise systems to create a single version of the truth
• Develop, design, and implement data manipulation and data cleansing solutions to ensure that data is valid and structured for analysis and visualization
• Define data governance policies, processes, and frameworks to ensure data accuracy and consistency
• Develop, design, and implement data replication solutions to ensure data availability, performance, and scalability
• Integrate data governance and data warehousing projects with business intelligence and production applications
• Develop, design, and implement data quality solutions to evaluate, develop, and maintain data to become trusted and usable established data
• Develop,

Requirements And Skills:

• Bachelor’s degree in computer science, statistics, or related field
• 2+ years’ proven experience developing and maintaining big data solutions
• Strong knowledge of Hadoop and related technologies
• Experience with Apache Spark and related technologies
• Knowledge of NoSQL databases, including MongoDB
• Strong

At Company Name, we believe that diversity and inclusion are key to building a successful team. We encourage applications from people of all backgrounds, races, religions, nationalities, genders, , and expressions, and ages. We are especially committed to supporting veterans and individuals with disabilities.