Intermediate/Senior AWS Data Engineer

Recruiter:

PBT Group

Job Ref:

4211453957

Date posted:

Tuesday, August 27, 2024

Location:

Johannesburg, South Africa

SUMMARY:
-

POSITION INFO:

The role involves working with advanced AWS technologies to design, build, and maintain scalable data pipelines and architectures. The ideal candidate will have strong expertise in SQL, Python/PySpark, and key AWS services including Glue, Step Functions, EMR (EC2), S3, Lambda, and Athena. The position is focused on leveraging DevOps practices for smooth delivery and deployment.

Key Responsibilities:

Develop and manage data pipelines using Python/PySpark and AWS Glue.
Implement AWS Step Functions and EMR for orchestrating and processing data workflows.
Work with S3 for efficient data storage solutions.
Leverage Lambda and Athena for real-time data queries and analytics.
Apply DevOps principles to optimise data engineering processes and streamline delivery.
Collaborate with cross-functional teams to align data engineering efforts with business objectives.

Skills & Experience Required:

Proficiency in SQL and Python/PySpark for data processing and analysis.
Hands-on experience with key AWS services (Glue, Step Functions, EMR, S3, Lambda, Athena).
Knowledge of DevOps practices and tools for CI/CD in data engineering projects.
Strong problem-solving skills and ability to work within an agile development environment.
Prior experience in data engineering roles with a focus on cloud-based architectures.

Additional Criteria:

Experience within the insurance industry is advantageous.
Ability to work independently as well as part of a collaborative team.
Strong communication skills for stakeholder interaction and requirement gathering.

Qualifications/Certification:

A degree in Computer Science, Information Systems, Business Administration, Commerce, or an equivalent field.
Relevant certifications in AWS, data engineering, or DevOps are highly desirable.

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Senior Data Analyst
Location: Johannesburg
Salary: 750 000
Experienced Senior data Analyst Wanted to Drive Strategic Insights | Hybrid | JohannesburgWe are seeking a Senior data Analyst to take a pivotal role in driving data-driven decisions across our organization. As a senior member of the team, you will leverage large datasets to inform leadership and assist team members in making impactful, data-backed choices. Your expertise in analytics, coupled wit...
Today

Data Engineer
Location: Pretoria
Salary:
Exciting Opportunity for a data engineer (Intermediate) – Fully Remote!Are you an experienced data engineer with a passion for building scalable data systems? A leading technology company is seeking an intermediate data engineer to join their fully remote team, supporting international marketplaces. This exciting opportunity allows you to work from anywhere within a 3-hour timezone of South ...
Today

Security Engineer
Location: Johannesburg
Salary: 450000 Annually
In need of Security engineer skills and experience within JHB!
1 day ago

Business Intelligence Developer
Location: Johannesburg
Salary: 420000 Annually
In need of well-versed Business Intelligence Developer skills in JHB!
1 day ago

Security Consultant
Location: Johannesburg
Salary: 300000 Annually
Searching for a Cybersecurity Consultant to practice in JHB!
1 day ago

Software Engineer SNR 2600
Location: Menlyn
Salary:
Job Opportunity: Senior Python Software engineer3-Year Renewable ContractHybrid model (Midrand/Menlyn) Are you a coding genius with a passion for innovation? Ready to tackle complex problems in a cutting-edge global IT environment? We have an exciting role just for you! We’re on the lookout for a Senior Python Software engineer to join our dynamic team and be part of a high-en...
1 day ago

Business Intelligence Developer
Location: Johannesburg
Salary: 600000
Join a newly developed position as a Business Intelligence Developer in Johannesburg!
4 days ago

Senior Solutions Engineer
Location: Johannesburg
Salary:
As a Senior Solutions engineer, you will play a pivotal role in driving the technical evaluation stage of the sales process, ensuring customer success as data and AI become integral to modern enterprises. This role involves acting as the key technical advisor to the sales team and working closely with multiple internal teams, including business units, product management, and engineering. You will ...
4 days ago

Full Stack Software Developer - Pretoria
Location: Pretoria
Salary:
4 days ago

Data Analyst
Location: Johannesburg
Salary:
Are you passionate about turning data into actionable insights? Do you thrive in fast-paced, dynamic environments where your analytical skills can make a real impact? A valued client is seeking a data Analyst to join their team and help shape the future of their business through data-driven decision-making.
5 days ago