Engineer - Data Test & Assurance

Job summary

Job purpose

As a Data Test & Assurance Engineer, you will operate as a senior technical subject matter expert providing cross-platform data test assurance practice leadership across Homes England’s Azure Databricks Platform.

You will define, embed, and assure automated data and platform testing practices end-to-end across data engineering, analytics, and Data Science / AI‑enabled workloads, ensuring solutions are reliable, secure by design, ethical, and auditable.

The role carries delegated technical leadership responsibility and may require leading and directing the work of contractors and external specialists through technical authority rather than formal line management.

Engineering Community of Practice, mentoring teams and embedding modern ways of working.

This role is crucial to providing the technical leadership and governance required for the Agency to deliver consistent data services, mature its data culture, and respond effectively to strategic and operational priorities.

Job description

Key accountabilities and responsibilities

Cross-Platform Test Strategy and Standards

• Define, own, and continuously evolve the Data Platform Test Strategy, ensuring consistent assurance across infrastructure, CI/CD pipelines, data ingestion, transformation, analytics, and AI/ML workloads. Establish secure‑ by‑design testing standards and patterns adopted across internal teams and suppliers.

Leadership of Contractors and External Resources

• Lead, direct, and quality‑assure the work of contractors and external suppliers delivering data testing, platform assurance, and AI/ML validation capabilities. Set technical direction, review deliverables, and ensure outputs meet Homes England and DDaT standards.

Automated Data, Analytics, and AI Assurance

• Design and implement automated tests for ETL/ELT pipelines, data models, analytical outputs, and machine learning artefacts, including validation of feature engineering, model inputs and outputs, data drift, and reproducibility using Python and SQL‑based frameworks.

CI/CD, Secure-by-Design, and Environment Assurance

• Embed quality and security gates into Azure DevOps CI/CD pipelines. Validate environment parity across Dev, Pre‑Prod, and Prod, and apply secure‑by‑design principles including identity and access management, secrets handling, data masking, and least‑privilege access.

Quality, Observability, and Compliance

• Implement data profiling, anomaly detection, and assurance evidence to support governance, audit, GDPR, responsible AI, and regulatory compliance.

Delivery Tracking and definition

• Maintaining and processing the data test engineering and assurance backlog in collaboration with the product and delivery managers as part of the data service offered to our customers.

Data Community of Practice

• Actively participate and contribute to the Data Analytics Community of Practices from the Data Engineering perspective.

Person specification

Key skills, knowledge and experience

• Hold a relevant degree or professional qualification in computer science, data engineering, mathematics, or a related discipline, or demonstrate minimum equivalent minimum practical experience gained though foundation level Data Engineering Level 5 apprenticeship or higher.
• Strong background in data testing and assurance across cloud data platforms
• Experience with Azure Databricks, Azure Data Factory, Synapse Analytics, and Data Lake Gen2
• Proficiency in Python (PySpark, Pandas) and SQL for automated testing and profiling
• Working knowledge of data science and machine learning lifecycles, including feature engineering and model deployment
• Understanding of AI/ML assurance topics such as bias, explainability, reproducibility, and drift
• Strong understanding of secure‑by‑design principles including IAM, data protection, and least‑privilege access
• Experience embedding testing and security controls into Azure DevOps CI/CD pipelines
• Ability to communicate assurance outcomes clearly to technical and non‑technical stakeholders
• Experienced in managing large, complex structured and unstructured datasets, including collection, validation, analysis, and transformation.
• Able to define solutions for ambiguous or complex problems independently, apply innovative approaches and adopt modern technologies.
• Understanding of project management principles, Agile methodologies, and software delivery practices, capable of leading cross-functional data initiatives.
• Awareness of data governance, security, and regulatory standards, with professional competence equivalent to formal qualifications in data engineering or analytics.
• Experience working within the Homes England sector or similar public sector organisation’s, with an understanding of sector-specific data requirements, compliance, and governance frameworks.

Benefits

Homes England is an executive non-departmental public body with its own terms and conditions, benefits and pension scheme. We are a government agency working in partnership with the private sector and want to employ and retain great people. In return we offer a range of excellent benefits, some of the key benefits include:

Holiday - Homes England offers all staff 33 days annual leave per year in addition to 8 statutory bank holidays.

Working Flexibly - At Homes England we have a strong culture of flexible working and encourage all employees to have a healthy work/life balance. We offer a wide range of potential working patterns, including part time, job share and compressed hours.

Health and Wellbeing - We offer health screening with Nuffield Proactive Health Clinics, a free Employee Assistance Programme which offers confidential support 24 hours a day, 7 days a week as well as access to Occupational Health.

Professional Development - At Homes England we are committed to your continued professional and personal development with learning as one of our core values. We pay for one job-related professional membership subscription per year for our employees.

We also offer an excellent range of family friendly benefits, for example, maternity, paternity, shared parental leave.

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

To apply please follow the links to our careers page.

Security

Successful candidates must pass a disclosure and barring security check.

For more information on the role, please see the attached Job Description.

Successful applicants will be advised of the outcome of their application and interviews held as soon as practical after the closing date. Feedback will only be provided if you attend an interview or assessment. We would like to take this opportunity to thank you for your interest in this role and Homes England.