Senior Data Engineer (Machine Learning (ML))

Job summary

Can you work with technical and non-technical stakeholders to deliver data-driven outcomes?

Do you enjoy solving complex data engineering and machine learning challenges?

Have you built and optimised data pipelines that support analytical and AI use cases?

If so, we’d love to hear from you!

A Senior Data Engineer (ML) works within a rapid prototyping team alongside data scientists to develop, test, and scale data science solutions. They are responsible for ensuring that prototypes are production-ready, scalable, and maintainable. The role bridges data science and data engineering, enabling the team to deliver high-impact, operational AI solutions. Joining our department comes with many benefits, including:

• Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
• 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave), plus 8 bank holidays and a privilege day for the King’s birthday
• Flexible working options where we encourage a great work-life balance.

Read more in the Benefits section below!

Find out more about what it's like working at: MCA Homepage , Civil Service Careers

Job description

As our Senior Data Engineer (ML), you’ll transform data science prototypes into secure, scalable, production-ready solutions. Working alongside data scientists, you’ll lead MLOps, CI/CD and model development and drift practices, optimise machine learning pipelines, champion responsible AI, and establish engineering standards that will support the Agency’s growing data and AI capability

Your responsibilities will include, but aren’t limited to:

• Work within the data science team to deliver rapid prototypes for new solutions, developing new solutions using cutting-edge techniques.
• Collaborate with and translate data science prototypes into scalable, production-grade machine learning systems using robust engineering practices and automation pipelines.
• Design and implement scalable machine learning pipelines using modern MLOps tools and frameworks. Utilising Prototype-to-Production engineering.
• Apply internal data engineering best practices to ensure data quality, lineage, and reproducibility.
• Maintain continuous integration and deployment workflows for machine learning models, including automated testing, validation, and monitoring. Use tools such as MLflow, Databricks, and infrastructure-as-code to ensure reproducibility and reliability.
• Monitor deployed models for performance degradation, drift, and fairness. Implement retraining strategies and version control to maintain model integrity over time.

For further information on the role, please read the role profile. Please note that the role profile is for information purposes only - whilst all elements are relevant to the role, they may not all be assessed during the recruitment process. This job advert will detail exactly what will be assessed during the recruitment process.

Person specification

Required experience:

To be successful in this role you will need to have the following experience:

1. Applied knowledge of Python programming to deliver data science and data engineering products.
2. Experience in productionising complex data science solutions using a Databricks and Azure.
3. Ability to influence stakeholders internally and externally, building stakeholder relationships and acting as an ambassador and champion for data.

Additional Information

This post is offered on a Fixed Term Appointment (FTA) basis to complete a finite piece of work, however this may be extended or made permanent.

Permanent Civil Servants will be appointed on an inward loan and subject to the terms of the agreed inward loan. You must have your home departments approval to return to them at the end of the loan before you apply. Internal candidates will return to their previous post at the end of the loan period, which will need to be agreed with your line manager in advance. If your home department cannot agree to release you on loan, there is the opportunity for you to move across to the department on a fixed term appointment. In this instance the Department for Transport would become your home department and if there is no role available for you at the end of this appointment, you would be subject to the redeployment and redundancy process.

If you're employed by a non-departmental public body (NDPB) by moving jobs this will involve a change of employer and you may break the statutory rules on continuity of employment.

The role is part of the Government Digital and Data (or Government Security Profession Career Framework) profession and utilises an enhanced Capability–Based Pay Framework which provides access to a Digital and Data allowance.

The base pay is £44,241. In addition to this the role includes a Digital and Data allowance of up to £14,756.

The value of allowance awarded will be based on an assessment of your skills and experience as demonstrated through the selection process. Here are more details on the pay framework.

Working hours, office attendance and travel requirements

Full time roles consist of 37 hours per week.

Whilst we welcome applications from those looking to work with us on a part time basis, there is a business requirement for the successful candidate to be able to work at least 32 hours per week.

This role is suitable for hybrid working, which is a non-contractual arrangement where a combination of workplace and home-based working can be accommodated subject to business requirements.

The expectation at present is a minimum of 60% of your working time a month will be spent at either your designated workplace (one of the locations cited in the advert) or, when required for business reasons, in another office/work location/visiting stakeholders. Your designated workplace will be your contractual place of work. There may be occasions where you are required to attend above the minimum expectation.

Occasional travel to other offices will be required, which may involve overnight stays.

If you have a question about hybrid working, part time/job share hours, flexible working, travelling for work, or require a reasonable adjustment, please contact the Vacancy Holder during the recruitment process to avoid possible disappointment later in the process should your working arrangements not be compatible with the requirements of the role (see below for contact details).

Visa Sponsorship

Please note that we will only offer sponsorship for a skilled worker visa where a role has been deemed to be business critical. This role does not meet that category, and we will not sponsor a visa. Therefore, this role is not open to applications from those who require sponsorship – candidates must ensure they have the appropriate right to work in the UK before applying. Should you apply for this role and be found to require sponsorship, your application will be rejected and any provisional offer of employment withdrawn.

Behaviours

We'll assess you against these behaviours dur