Data Engineer

SUMMARY:
Our client, based in Sandton, is seeking an experienced Data Engineer to join their dynamic team. The ideal candidate will have 3 to 5 years of professional experience in a Data Engineer or related role. This position involves maintaining, implementing, and innovating current and new projects within a collaborative and fast-paced environment.

POSITION INFO:
Required Minimum Education / Training:

• Bachelor’s Degree in Computer Science, Software Engineering, Informatics, or Information Systems.
• A Degree from an accredited institution is essential to ensure foundational knowledge in programming, algorithms, and software design.
• Equivalent qualifications or certifications combined with significant industry experience may also be considered.
• Relevant certifications (e.g., Microsoft Certified: Azure Data Engineer Associate, Power BI Data Analyst, MCSA/MCSE SQL Server 2012/2014) are a plus.

• 3-5 years of professional experience in a Data Engineer or related role.

• At least 3 years of experience working with Microsoft SQL Server, Sybase, or Postgres SQL.
• Familiarity with NoSQL DB systems.
• Strong T-SQL knowledge.
• Intermediate Python development and familiarity with data-related libraries (Pandas, PySpark, NumPy, etc.).
• Experience with ETL/ELT tools such as SSIS, Data Factory, Synapse, AirFlow, or similar tools for AWS (e.g., Glue, Fire Hose).
• SQL Internals – statistics, execution plans, wait types, isolation levels, DMVs, indexes.
• Understanding of Hadoop, Spark, MapReduce, columnar databases, and Massive Parallel Processing systems.
• Setting up data ingestion processes in Azure, AWS, or GCP.
• Experience with visualization tools such as Power BI, SSRS, Tableau, or QlikView.
• Knowledge of writing and optimizing complex queries with 3-5 years’ expertise.
• Version Control and CI/CD:
  
  • Proficient in using Git and working with repositories like GitHub or Bitbucket.
  • Familiarity with CI/CD pipelines and deployment strategies.

• Advantageous Skills:
  • Experience in C# or Java and its ecosystem.
  • Understanding of microservices architecture and containerization (e.g., Docker, Kubernetes).

• Maintain and Enhance Existing Systems:
  • Identify and resolve issues in existing software systems.
  • Perform system upgrades and ensure compatibility with new technologies.
• Design and Develop New Features:
  • Collaborate with stakeholders to gather requirements.
  • Implement new features and functionalities in both front-end and back-end systems.
• Innovate and Optimize:
  • Explore new tools and technologies to enhance system performance and scalability.
  • Propose and implement improvements to existing workflows and processes.
• Collaborate with Team Members:
  • Work closely with Developers, QA, and Product Managers to deliver high-quality solutions.
  • Participate in code reviews and share best practices.