Data Engineer

Our client is seeking an experienced Data Engineer to become part of their DataWorks team and contribute to the development of innovative, scalable data solutions. The successful candidate will be responsible for developing and optimising data pipelines, implementing cloud-based data platforms, and managing data infrastructure that supports business intelligence, advanced analytics, and machine learning.
This position requires strong technical expertise across Microsoft Fabric, AWS, and Databricks, together with experience in bringing data together from various systems and sources to create robust, high-performing data environments.

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes to support analytics and reporting requirements.
• Build and manage cloud-based data solutions using platforms such as AWS, Microsoft Fabric, or Databricks.
• Integrate data from multiple internal and external sources into centralized data platforms, data lakes, or data warehouses.
• Develop and maintain data models, data marts, and data warehouse structures to support business intelligence and analytics initiatives.
• Ensure data quality, integrity, and reliability through validation, monitoring, and automated quality checks.
• Work with large datasets and big data technologies such as Spark to process and transform complex data efficiently.
• Collaborate closely with data analysts, BI developers, data scientists, and business stakeholders to understand data requirements and deliver scalable solutions.
• Optimize data processing workflows and improve performance and scalability of existing data pipelines.

Qualifications and experience:

• Bachelor’s degree from an accredited university. A degree in Computer Science, Engineering, Information Technology, or a related field is advantageous.
• Minimum of 7+ years of hands-on experience in data engineering or data platform development. Experience within the financial services industry would be beneficial but is not essential.
• Proven experience building, managing, and optimizing data pipelines and data solutions using on-premise or cloud-based technologies.
• Practical experience working with AWS services such as Glue, Redshift, S3, Lambda, EMR, and Athena, or exposure to Microsoft Fabric or Databricks.
• Familiarity with big data frameworks and technologies such as Apache Spark, Hadoop, or Kafka.
• Strong programming capabilities in Python, Pandas, and SQL for data processing and transformation

The reference number for this position is NG60984 which is a permanent, Hybrid role in Johannesburg offering a salary of up to R1.2mil per annum negotiable based on experience. E-mail Nokuthula on e-Merge.co.za or call her for a chat on 011 463 3633 to discuss this and other opportunities.
Are you ready for a change of scenery? E-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the e-Merge IT website www.e-merge.co.za for more great positions.

Posted on 14 Sep 10:59, Closing date 13 Nov