Intermediate/Senior AWS Data Engineer

Recruiter:

PBT Group

Job Ref:

4211453957

Date posted:

Tuesday, August 27, 2024

Location:

Johannesburg, South Africa

SUMMARY:
-

POSITION INFO:

The role involves working with advanced AWS technologies to design, build, and maintain scalable data pipelines and architectures. The ideal candidate will have strong expertise in SQL, Python/PySpark, and key AWS services including Glue, Step Functions, EMR (EC2), S3, Lambda, and Athena. The position is focused on leveraging DevOps practices for smooth delivery and deployment.

Key Responsibilities:

Develop and manage data pipelines using Python/PySpark and AWS Glue.
Implement AWS Step Functions and EMR for orchestrating and processing data workflows.
Work with S3 for efficient data storage solutions.
Leverage Lambda and Athena for real-time data queries and analytics.
Apply DevOps principles to optimise data engineering processes and streamline delivery.
Collaborate with cross-functional teams to align data engineering efforts with business objectives.

Skills & Experience Required:

Proficiency in SQL and Python/PySpark for data processing and analysis.
Hands-on experience with key AWS services (Glue, Step Functions, EMR, S3, Lambda, Athena).
Knowledge of DevOps practices and tools for CI/CD in data engineering projects.
Strong problem-solving skills and ability to work within an agile development environment.
Prior experience in data engineering roles with a focus on cloud-based architectures.

Additional Criteria:

Experience within the insurance industry is advantageous.
Ability to work independently as well as part of a collaborative team.
Strong communication skills for stakeholder interaction and requirement gathering.

Qualifications/Certification:

A degree in Computer Science, Information Systems, Business Administration, Commerce, or an equivalent field.
Relevant certifications in AWS, data engineering, or DevOps are highly desirable.

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

Similar jobs you might be interested in:

Data Scientist – AI Platform - Johannesburg - R370 to R440 per hour
Location: Johannesburg
Salary: 844800
data Scientist – AI Platform - Johannesburg - R370 to R440 per hour
Today

Solutions Engineer
Location: Centurion
Salary: 70 000
Join our dynamic team in Centurion, our client is Â a leading innovator in the IT and Fintech space. We are seeking a highly motivated, experienced, and enthusiastic individual to take on the role of a Solution engineer. In this pivotal role, you will contribute to the development and optimization of our business applications, APIs, data workflows, and automation processes, ensuring seamless opera...
3 days ago

Data Scientist – Centurion – R800k to R1mil per annum
Location: Centurion, Gauteng
Salary: 1000000
Intermediate/Senior Business data Analyst – Johannesburg – R500 to R700 per hour
3 days ago

Intermediate/Senior Business Data Analyst – Johannesburg – R500 to R700 per hour
Location: Johannesburg
Salary: 1344000
Intermediate/Senior Business data Analyst – Johannesburg – R500 to R700 per hour
3 days ago

CAX VDI Engineer (Advanced)
Location: Midrand
Salary:
The CAX VDI engineer (Advanced) is responsible for designing, managing, and optimizing Virtual Desktop Infrastructure (VDI) solutions in a hybrid cloud environment. The role involves driving VDI automation, maintaining high-performance computing tools, and collaborating with global teams to ensure stability, performance, and scalability of VDI systems. Additionally, the engineer will manage comple...
3 days ago

AI Specialist
Location: Johannesburg
Salary:
An AI Specialist within the company plays a pivotal role in integrating artificial intelligence solutions to enhance the capabilities and efficiency of business analysts. This position aims to leverage AI technologies to automate routine tasks, provide data-driven insights, and support decision-making processes. By implementing AI initiatives, the AI Specialist ensures that business analysts...
4 days ago

Data Architect
Location: Pretoria
Salary:
We are looking for a data Architect to build relationships with the company's stakeholders to determine a clear understanding of data related problems and develop relevant solutions. This is permanent role based in Menlyn
4 days ago

Windchill Operations Specialist (Advanced)
Location: Midrand
Salary:
About the IT Hub: The IT Hub South Africa provides IT solutions to a global network, focusing on innovation topics and comprehensive IT solutions such as Development and Operations for SAP, JAVA, Azure, Cloud, and more. Positioned as an internal partner, the Hub ensures collaborative relationships, deep insight into compliance and IT methodology processes, and a vested interest in the success and ...
5 days ago

Software Engineer (Senior) 2600
Location: Pretoria
Salary: Monthly
Software engineer
5 days ago

Python & AWS Software Engineer (Expert)
Location: Midrand
Salary:
We are seeking a **Python & aws Software engineer (Expert)** to join our **Impact Foundry Platform** team. The ideal candidate will have extensive experience in Python and aws, with expertise in API development, ETL, CI/CD pipelines, and cloud infrastructure. South African citizens or residents are preferred.
5 days ago