Lead Data Engineer

Job summary

The Integrated Corporate Services (ICS) is a shared corporate service. It provides corporate services (HR, Finance, Digital, Commercial, Security and Estates) across the Department for Energy Security & Net Zero (DESNZ) and the Department for Science, Innovation & Technology (DSIT).

Our team of over 400 professionals will be leading the way in how these functions will be delivered in the future. Our ambition is to be the leading provider of integrated corporate services for government and set the standard for quality, efficiency, and innovation in our field.

We offer great working benefits including a world-class pension, flexible working options and a career where your learning and development is taken seriously. We are enormously proud to be a Disability Confident Leader employer. We support candidates with adjustments throughout our recruitment process. Information about disability confidence and just some examples of the adjustments that you can request can be found in the reasonable adjustment section below.

Find Out More

You can also follow our LinkedIn Careers Page: https://www.linkedin.com/showcase/desnz-careers/

Job description

Join ICS Digital, where innovation meets expertise to create digital solutions. We are a cohesive digital community who work in the open, solving problems together and fostering a culture of transparency and collaboration. Our commitment to user-focused design and data-driven decision-making ensures that we deliver accessible services tailored to the needs of our customers.

The Data and Development team is part of the D3 (Data, Delivery and Design) function within ICS Digital. We are a data-driven team that delivers data-focused services and products for colleagues across DESNZ and other government departments or bodies. Our close-knit team brings together Data Engineers, Data Scientists, Data Analysts, Data Transparency Specialists and Software Developers, working collaboratively to deliver insight, innovation and value.

We are looking for a Lead Data Engineer to join the team and own the development of the Data Management System, which provides end to end data ingestion, processing and storage capabilities that are used to provide insights and decision-making inputs that are core to the challenges that face the Departments today. For example, we have recently worked on projects that have enabled grants to be paid out to citizens and businesses as part of the Energy Affordability programme. Additionally, we have provided insights into key Energy Efficiency projects that have been at the heart of key Department delivery commitments.

Visit our blog to learn more about us and our work: https://icsdigital.blog.gov.uk/

To find out more about the Government Digital & Data profession visit: https://dsit.pagetiger.com/bzwfhzv/1

Person specification

• Design, build and evolve secure, data pipelines that reliably support complex operational needs.
  
• Define and enforce standards for data engineering to ensure solutions are scalable, maintainable and safe to change.
  
• Provide strong technical leadership to the team, including setting direction, reviewing critical code, mentoring engineers and upholding high engineering standards.
  
• Build a high performing, motivated team by fostering a culture of learning, experimentation and professionalism, by being an effective, supportive line manager.
  
• Owning Data Engineering best practice through building a community of practice across the government departments ICS delivers for.

The above responsibilities are not exhaustive, and the post holders may be required to undertake some additional tasks not listed above, within the remit of their grade, to support the team.

As a line manager, you will be responsible for working with your members of staff to define their objectives, as well as managing their development and performance.

Essential Criteria

• Accomplished expert data engineer with experience of producing, maintaining and updating data models which solve business problems and are optimised for data analysis.
  
• Extensive experience using and managing an Azure based Data Platform with expertise in Azure Databricks, using Pyspark to build data pipelines; and using their experience to continuously improve the codebase and reliability of solutions.
  
• Experience transforming complex structured and semi-structured data into structures and layers that enable analysis, defining and promoting best practices for creating repeatable, reliable and reusable data solutions that therefore drive impact and value.
  
• Can listen to and interpret the needs of technical and non-technical stakeholders, manage their expectations and influence outcomes
  
• Experience of providing technical leadership, guiding and mentoring engineering teams.
  
• Experience in unit testing, end-to-end testing and quality assurance in a Data Engineering context.
  
• Analytical and problem-solving capabilities; self-starter willing to pro-actively problem-solve by researching, learning, testing, and deploying new code solutions independently.
  
• Experience writing code in Python and SQL.
  

Desirable Criteria

• Knowledge and experience of data modelling, analytical technologies, data visualisation.
  
• Experience in DevOps practices including Azure CI/CD, YAML pipelines, git and Terraform.

When completing your CV as part of the application please ensure to include the time you have worked for your organisations/employers, and your key objectives in each role.

It is unusual that all candidates will meet all the desirable criteria. If your skills and experience look slightly different from what we have identified and you think you can bring value to the Lead Data Engineer role and the team, we strongly encourage you to apply. We'd love to hear from you!

Behaviours

We'll assess you against these behaviours during the selection process:

• Making Effective Decisions
• Changing and Improving
• Developing Self and Others

We only ask for evidence of these behaviours on your application form:

• Making Effective Decisions

Technical skills

We'll assess you against these technical skills during the selection process:

• Programming and build (data and analytics engineering) (Level: practitioner)
• Data integration design (Level: expert)

Benefits

Alongside your salary of £56,900, Department for Energy Security & Net Zero contributes £16,483 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides (opens in a new window).

The Department for Energy Security and Net Zero offers a competitive mix of benefits including:

• A culture of flexible working, such as job sharing, homeworking and compressed hours.
• Automatic enrolment into the Civil Service Pension Scheme, with an employer contribution of 28.97%.
• A minimum of 25 days of paid annual leave, increasing by 1 day per year up to a maximum of 30.
• An extensive range of lear