Senior Software Engineer - Data, Group Data & ML Platform Tribe

Recruiter:

Recru-it

Job Ref:

PE011100/LM

Date posted:

Thursday, January 9, 2025

Location:

Capetown, South Africa

Salary:

market related Monthly

SUMMARY:
What will you do?
Our client is a newly created engineering organisation on a mission to empower and accelerate autonomous value creation through insights and ML across the group. The company main users are the data scientists and analysts who sit inside the business units and support them in developing and re-developing their financial and digital products to better meet the needs of the company clients and the company shareholders. The company other users are the data management and data product teams who produce the data sets analysts and scientists need to launch straight into their work.

The tribe is made up of 3 different engineering squads covering everything from how the company ingest and store data, to how the company develop and share ML artefacts and insights. The company has one for ingestions, one for data storage and processing and one for insights & ML tooling. The company has one data engineering talent pipeline for the whole tribe, where placements into specific squads are explored later in the interview process based on squad needs and candidate abilities. Data engineers report to the tech leads of the squad they are embedded into.

What will make you successful in this role?
What success will look like for you 12 months into the role.

(Ingestions Squad) Built configurable data ingestion platforms to collect and protect data from any source within two weeks or less. The team has bought and built to assemble a tool soup of persistent and secure ingestion tools, optimized for cost efficiency, with reliability as a non-negotiable standard. Success will be measured by the consistent achievement of the two-week ingestion target while maintaining high reliability and security.
(Data products empowerment squad) Simplified data transformation and product development for the platforms SQL only users: Developed solutions that allow SQL-only users to easily schedule and orchestrate complex data transformation processes, such as through Directed Acyclic Graphs (DAGs). Success will be measured by the 100% adoption of your tools by platform users, the full deprecation of custom solutions and stored procedures, and the reliable orchestration of multi-stage transformation processes underlying our client’s company wide data products.
(Insights & ML tooling squad) Accelerated the development of insights and ML models by analysts and data scientists across the company: Built systems that empower analysts and data scientists to produce the insights and ML models needed to drive customer and commercial success with minimal engineering skills and platform friction. Success will be measured by the number of successful deployments of insights and models per month that directly contribute to improving our business success metrics.
Engineered the platform for observability and performance. Made your systems observable from the outside, taking ownership of their performance. Facilitated real-time tracking of the system’s internal state through logs, metrics, and traces. Defined clear and measurable Service Level Indicators (SLIs) and Service Level Objectives (SLOs) and integrate these into team OKRs to drive excellence. Success will be measured by meeting or exceeding defined performance and reliability benchmarks, and the quick identification and resolution of issues.
Engineer the platform with privacy and security by design.Secured the platform through compliance engineering initiatives that reduced the time and cognitive load related to compliance. Implemented the company’s Data Governance and Privacy policies effectively, leveraging encryption, firewalls, VPCs, Role Based Access Control and IAM integrations such as Active Directory. Success will be measured by the platform’s adherence to security and privacy standards, with minimal compliance overhead for users.
Leveraged containerization and infrastructure as code. Implemented containerization strategies and infrastructure as code (IaC) practices to ensure scalable, repeatable, and efficient deployment of data services. Utilized tools like Docker and Terraform to automate infrastructure management and streamline the development pipeline. Success will be measured by the consistency and reliability of deployments, reduced deployment times, and increased system scalability.
Contributed to technology decisions. Engaged in technology spikes to clarify needs and articulate solution criteria. Evaluated different options through hands-on experimentation, leading to documented evaluations and recommendations that were understood by technology leaders or users across the company. Success will be measured by the quality and clarity of your contributions to technology decisions and the adoption of your recommendations.
Invested in your own development. Executed on a culture of excellence and continuous improvement for both your part of the platform and your own abilities. Lived the company’s culture of ownership, curiosity, and continuous learning. Participated in pre and post-mortems, sharing lessons learned across the organization along with how you are improving based on those lessons. Success is measured by your active participation in continuous learning initiatives and the tangible improvements in platform performance and personal growth.
Technical leadership and coaching. Enable a culture of continuous learning and assist other engineers in their development journey.

Qualification:

Matric
National Diploma in an Information Technology related discipline or a bachelor's degree in computer science, Statistics, Informatics, Information Systems, or a quantitative field will be recommended.

Experience:

5+ years' experience as a data engineer in data platform or service engineering team with at least 2 in a modern context with cloud technologies
Track record of high impact and investing in your own development

Skills and Thinking Preference

Ability to act independently and to deal with high levels of ambiguity.
You look for ways to do, instead of reasons not to. You are high agency, high ownership in everything you do. You know we own our future, only we can make it happen. If you find something that keeps us from our mission, you own it or work collaboratively with others until you find the right owner of it.
Where some see impossible, you see a way.You have the strong conviction in our ability to innovate and bring financial services, and financial confidence to everyone across the continent. You turn obstacles into opportunity, chance into change.
You move fast and fix things. You publish early and often, when you fail you learn and go again. You know that speed gives you the licence to fail, because you’re moving fast enough to correct course.
You love creating more with less. Knowing that the biggest impact is born of the smallest acts, scale is built into your thinking from day 1. This is not about meeting the needs of thousands; it’s about exceeding the needs of millions. And working through small and mighty teams is way we’ll do it.
You seek out and value different perspectives. Believing that when we change how we look at things, how things look for us also changes. Making tomorrow not just different, but better.

Personal Attributes:

Plans and aligns - Contributing through others
Optimises work processes - Contributing through others
Collaborates - Contributing strategically
Resourcefulness - Contributing strategically
Tech savvy - Contributing strategically

Core Competencies:

Cultivates innovation - Contributing through others
Customer focus - Contributing through others
Drives results - Contributing through others
Collaborates - Contributing through others
Being resilient - Contributing through others

POSITION INFO:
What will you do?
Our client is a newly created engineering organisation on a mission to empower and accelerate autonomous value creation through insights and ML across the group. The company main users are the data scientists and analysts who sit inside the business units and support them in developing and re-developing their financial and digital products to better meet the needs of the company clients and the company shareholders. The company other users are the data management and data product teams who produce the data sets analysts and scientists need to launch straight into their work.

The tribe is made up of 3 different engineering squads covering everything from how the company ingest and store data, to how the company develop and share ML artefacts and insights. The company has one for ingestions, one for data storage and processing and one for insights & ML tooling. The company has one data engineering talent pipeline for the whole tribe, where placements into specific squads are explored later in the interview process based on squad needs and candidate abilities. Data engineers report to the tech leads of the squad they are embedded into.

What will make you successful in this role?
What success will look like for you 12 months into the role.

(Ingestions Squad) Built configurable data ingestion platforms to collect and protect data from any source within two weeks or less. The team has bought and built to assemble a tool soup of persistent and secure ingestion tools, optimized for cost efficiency, with reliability as a non-negotiable standard. Success will be measured by the consistent achievement of the two-week ingestion target while maintaining high reliability and security.
(Data products empowerment squad) Simplified data transformation and product development for the platforms SQL only users: Developed solutions that allow SQL-only users to easily schedule and orchestrate complex data transformation processes, such as through Directed Acyclic Graphs (DAGs). Success will be measured by the 100% adoption of your tools by platform users, the full deprecation of custom solutions and stored procedures, and the reliable orchestration of multi-stage transformation processes underlying our client’s company wide data products.
(Insights & ML tooling squad) Accelerated the development of insights and ML models by analysts and data scientists across the company: Built systems that empower analysts and data scientists to produce the insights and ML models needed to drive customer and commercial success with minimal engineering skills and platform friction. Success will be measured by the number of successful deployments of insights and models per month that directly contribute to improving our business success metrics.
Engineered the platform for observability and performance. Made your systems observable from the outside, taking ownership of their performance. Facilitated real-time tracking of the system’s internal state through logs, metrics, and traces. Defined clear and measurable Service Level Indicators (SLIs) and Service Level Objectives (SLOs) and integrate these into team OKRs to drive excellence. Success will be measured by meeting or exceeding defined performance and reliability benchmarks, and the quick identification and resolution of issues.
Engineer the platform with privacy and security by design.Secured the platform through compliance engineering initiatives that reduced the time and cognitive load related to compliance. Implemented the company’s Data Governance and Privacy policies effectively, leveraging encryption, firewalls, VPCs, Role Based Access Control and IAM integrations such as Active Directory. Success will be measured by the platform’s adherence to security and privacy standards, with minimal compliance overhead for users.
Leveraged containerization and infrastructure as code. Implemented containerization strategies and infrastructure as code (IaC) practices to ensure scalable, repeatable, and efficient deployment of data services. Utilized tools like Docker and Terraform to automate infrastructure management and streamline the development pipeline. Success will be measured by the consistency and reliability of deployments, reduced deployment times, and increased system scalability.
Contributed to technology decisions. Engaged in technology spikes to clarify needs and articulate solution criteria. Evaluated different options through hands-on experimentation, leading to documented evaluations and recommendations that were understood by technology leaders or users across the company. Success will be measured by the quality and clarity of your contributions to technology decisions and the adoption of your recommendations.
Invested in your own development. Executed on a culture of excellence and continuous improvement for both your part of the platform and your own abilities. Lived the company’s culture of ownership, curiosity, and continuous learning. Participated in pre and post-mortems, sharing lessons learned across the organization along with how you are improving based on those lessons. Success is measured by your active participation in continuous learning initiatives and the tangible improvements in platform performance and personal growth.
Technical leadership and coaching. Enable a culture of continuous learning and assist other engineers in their development journey.