Data Engineer

SUMMARY:
-

POSITION INFO:

Our client is seeking an experienced Data Engineer. The ideal candidate will have 3 to 5 years of professional experience as a Data Engineer or related role. The role involves maintaining, implementing, and innovating current and new projects within a collaborative and dynamic team environment.

Required Minimum Education / Training

• Bachelor’s degree in computer science, Software Engineering, informatics or Information Systems.
• A degree from an accredited institution is essential to ensure foundational knowledge in programming, algorithms, and software design.
• Equivalent qualifications or certifications combined with significant industry experience may also be considered.
• Relevant certifications (e.g., Microsoft Certified: Azure Data Engineer Associate, Power BI Data Analyst, MCSA\MCSE SQL Server 2012/2014) are a plus.

Required Minimum Work Experience

• 3-5 Years of professional experience

Technical and Behavioural Competencies Required

Technical

• AirFlow etc. or At least 3 years of experience working with Microsoft SQL Server, Sybase or Postgres SQL.
• Familiarity with NoSQL DB systems
• Strong t-SQL knowledge
• Intermediate Python development and familiarity with data-related libraries (Pandas, PySpark, Numpy etc.)
• Working with ETL\ELT tools such as SSIS, Data Factory, Synapse, and similar tools for AWS eg. Glue, Fire Hose
• SQL Internals – statistics, execution plans, wait types, isolation levels, DMVs, indexes
• Understanding of Hadoop, spark, Map Reduce, columnar databases, and Massive Parallel Processing systems.
• Setting up data ingestion processes in either Azure, AWS or GCP
• Experience with Visualisation tools such as Power BI, SSRS, Tableau or QlikView
• Experience with SQL Server and database design, requiring at least 3-5 years’ experience.
• Knowledge of writing and optimizing complex queries with 3-5 years of expertise.
• Version Control and CI/CD:
• Proficient in using Git and working with repositories like GitHub or Bitbucket, with a minimum of 3-5 years of experience.
• Familiarity with CI/CD pipelines and deployment strategies with 3-5 years of experience.
• Advantageous Skills:
• Experience in C# or Java and its ecosystem.
• Understanding of microservices architecture and containerization (e.g., Docker, Kubernetes).

Behavioural

• Problem Solving: Demonstrated ability to tackle complex problems and devise effective solutions.
• Communication: Strong ability to communicate technical details clearly to both technical and non-technical stakeholders.
• Team Collaboration: Proven experience working effectively both as part of a team and independently.
• Innovative Thinking: Capacity to bring creative solutions to technical challenges.
• Leadership: Ability to guide and mentor junior team members, leading by example and fostering a positive team environment.
• Accountability: take ownership and accountability of tasks assigned to you

Key Performance Areas

• Maintain and Enhance Existing Systems:

• • Identify and resolve issues in existing software systems.
  • Perform system upgrades and ensure compatibility with new technologies.

• Design and Develop New Features:

• Collaborate with stakeholders to gather requirements.
• Implement new features and functionalities in both front-end and back-end systems.

• Innovate and Optimize:

• Explore new tools and technologies to enhance system performance and scalability.
• Propose and implement improvements to existing workflows and processes.

• Collaborate with Team Members:

• Work closely with developers, QA, and product managers to deliver high-quality solutions.
• Participate in code reviews and share best practices.