Senior Data Engineer

Job summary

Can you design, build and optimise robust, scalable data pipelines in GCP (or similar cloud platforms), applying engineering best practices to deliver high-quality, cost-effective solutions?

Have you used Python, SQL and cloud-based data engineering tools (such as BigQuery, Cloud Functions, Terraform or CI/CD pipelines) to successfully deliver and support data products in a production environment?

Are you experienced working in Agile teams, collaborating with both technical and non-technical stakeholders, and contributing to engineering standards that improve quality, consistency and long-term maintainability?

If so, we’d love to hear from you!

Join a collaborative and forward-thinking data team focused on delivering high-impact solutions that enable better decision-making across the organisation. Working within a modern cloud-based environment, you'll be part of a department that values innovation, continuous improvement and knowledge sharing. The team brings together a range of technical expertise to solve complex challenges, transform data into actionable insight, and drive meaningful change. You'll have the opportunity to work closely with stakeholders from across the business, contributing to strategic initiatives while helping shape the future of our data capabilities. With a strong emphasis on learning, quality and teamwork, this is an exciting environment for individuals who enjoy solving problems, embracing new technologies and making a tangible impact through data.

Joining our department comes with many benefits, including:

• Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
• 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave), plus 8 bank holidays a privilege day for the King’s birthday
• Flexible working options where we encourage a great work-life balance.

Read more in the Benefits section below!

Find out more about what it's like working at DfTc:

DfTc roles: Department for Transport Central - Department for Transport Careers

Job description

Join the Data Engineering team, a collaborative group of specialists working across Leeds, Birmingham, Hastings and London, delivering the department’s critical data services and cloud-based analytical platforms. You'll work closely with data and digital professionals to develop reliable, scalable solutions, support live services, modernise legacy systems, and help drive innovation. This is an exciting opportunity to contribute to high-impact projects that improve data capabilities and support informed decision-making across the organisation.

Your responsibilities will include, but aren’t limited to:

• Collaborate with team members and stakeholders to build and operate DfT’s production‑grade data pipelines and platforms, ensuring reliability and security across our Google Cloud Platform environment.
• Manage live data services; resolving issues at pace to maintain high‑quality data delivery for analysts, policy teams and external commitments.
• Take a leading role in developing and improving legacy systems, ensuring our data infrastructure is modern and meets ongoing needs.
• Support innovation within data engineering, identifying opportunities to modernise tooling, adopt emerging GCP capabilities and introduce new approaches that improve efficiency and data quality.
• Work closely with technical and non-technical stakeholders, translating technical concepts, shaping data‑related decisions, and responding to business need.
• Act as a champion of Infrastructure as Code (IaC), improving and developing repeatable patterns for environments, access, and data services.
• Drive the development of our metadata catalogue, curating business and technical metadata so users can effectively discover and use data.

For further information on the role, please read the role profile.Please note that the role profile is for information purposes only - whilst all elements are relevant to the role, they may not all be assessed during the recruitment process. This job advert will detail exactly what will be assessed during the recruitment process.

Person specification

You will be required to provide evidence that you hold any essential qualifications/licenses at some point during the recruitment process. If you cannot provide evidence, your application will be withdrawn.

Required experience:

To be successful in this role you will need to have the following experience:

1. Hands-on delivery of robust, maintainable data pipelines in Google Cloud Platform or alternative cloud providers. Including building reusable components, optimising performance and cost, and applying engineering standards and high‑quality coding practices.
2. Proficient user of data engineering tools, including relevant languages (e.g. Python and SQL) and GCP or equivalent cloud tooling (e.g. BigQuery, Cloud Functions). Experience working with or knowledge of CI/CD (e.g. Github Actions), logging and testing, and Infrastructure as Code tools (e.g. Terraform).
3. Contribution to engineering standards and best practice, supporting the development of maintainable, consistent technical approaches and improving quality across codebases and services.
4. Experience delivering data engineering work in Agile environments, independently managing components of delivery while contributing to wider team objectives.
5. Working across technical and non‑technical teams to understand requirements, communicate solutions, and support delivery of change and improvement within the data landscape.

Additional Information

The role is part of the Government Digital and Data (or Government Security Profession Career Framework) profession and utilises an enhanced Capability–Based Pay Framework which provides access to a Digital and Data allowance.

The base pay is £44,241 (National) £48,581 (London). In addition to this the role includes a Digital and Data allowance of up to £14,756.

The value of allowance awarded will be based on an assessment of your skills and experience as demonstrated through the selection process. Here are more details on the pay framework.

This role is suitable for hybrid working, which is a non-contractual arrangement where a combination of workplace and home-based working can be accommodated subject to business requirements.

The expectation at present is a minimum of 60% of your working time a month will be spent at either your designated workplace (one of the locations cited in the advert) or, when required for business reasons, in another office/work location/visiting stakeholders. Your designated workplace will be your contractual place of work. There may be occasions where you are required to attend above the minimum expectation.

Occasional travel to other offices will be required, which may involve overnight stays.

Visa Sponsorship

DfTc does not offer Visa Sponsorship for this role.

Behaviours

We'll assess you against these behaviours during the selection process:

• Communicating and Influencing