Staff Software Engineer - Data Platforms

We’re looking for a Staff Software Engineer to help shape and build the shared engineering capabilities that underpin Our Future Health’s data platforms. 

This is an opportunity to combine hands-on software engineering with Staff level technical leadership, working on large-scale data systems and reusable tooling while helping multiple engineering teams solve complex problems effectively. 

Our Future Health is an ambitious collaboration between the public, charity and private sectors, designed to help people live healthier lives for longer through better prevention, earlier detection and improved treatment of diseases. We will speed up the discovery of new methods of early disease detection, and the evaluation of new diagnostic tools, to help identify and treat diseases early, when outcomes are usually better. With over 2.7M volunteers across the UK, we’re now the world’s biggest health research programme of its kind, and our volunteer group is also more diverse than other, similar health research programmes. 

Technology and data are central to our mission. Our systems power web sites, clinics across the UK, secure analytics and research systems, and pipelines that process highly sensitive health and genetic data, and we are continuing to grow our engineering capability to support this ambition. 

Our Data Product squads build and operate the technology that enables us to process and use large volumes of health and research data. As our engineering organisation evolves, we’re looking for a Staff Software Engineer who can help us build scalable, reusable engineering capabilities that can be shared across these teams. 

Essential Duties and Responsibilities 

Day to day work will vary depending on what your focus is at that time, but will include things like: 
 

• Guiding the technical decisions the team makes alongside the squad’s product manager and engineering manager (EM). 
• Represent technology/architecture/engineering in cross-functional project teams throughout a project’s lifecycle (from ideation to implementation). Work within the team to identify optimal solutions for challenges, communicate them outwards to stakeholders and work with others to implement them. 
• Work with the Head of Data Engineering and your peer Staff/Lead Engineers to act as a guiding mind for the engineering/architecture in your business area. Identify areas for improvement and work with other leaders to implement. 
• Lead decision-making on critical technical strategy and implementations which affect your business area. 
• Identify opportunities for and then carry out specific pieces of high-impact hands-on work where it makes sense. 
• Work with partners and suppliers to integrate their systems with Our Future Health’s. 
• Support your colleagues with pairing, peer reviews, coaching, discussion, and other collaborative activities that help everyone reach high standards and result in the delivery of high-quality software products. 
• Participate in live operations support rotations that monitor live systems and quickly and effectively react to potential incidents and provide support to operations teams such as customer support. This could include being part of an out of hours on-call rotation. 
• Anticipate problems that could occur with pipelines and take action to prevent them. Identify and describe problems when they occur and be able to develop solutions that address them for the short and long term.  
• Keep abreast of best practice in software engineering across industry, research and Government, and identify opportunities to bring these into Our Future Health. 

Requirements

We welcome applications from all who may not feel they match the full criteria, so if you have most of the below, we'd like to hear from you:  

Technical Skills 

• You have led the architectural design of systems. 
• Highly proficient in cloud engineering (preferably in Azure; AWS and GCP) 
• Demonstrable knowledge and experience in building solutions centred around moving and processing large amounts of data at pace and scale, using cloud-native technologies such as Kubernetes, Helm and Docker. Experience with storing, searching and filtering large scale data. 
• Highly proficient working with Infrastructure as Code (Terraform, Ansible) 
• Highly proficient in Python. Experience in orchestration tools like Dagster, Prefect and Airflow are advantageous. 
• Experience with Spark, Databricks, data lakes/lakehouses. 
• Knowledge of good data modelling and design patterns; in-depth knowledge of relational databases (e.g. PostgreSQL) and familiarity other types of data store (blob, document, graph). 
• Highly proficient in modern, agile development practices like code review, TDD, CI/CD and pairing using tools like Git and GitHub. 
• Experience in operationally managing software components/service once live, including:  observability best practises, logging best practises, error reporting, debugging and live incident management. Experience using tools such as Grafana, Prometheus, New Relic etc. 
• Experience of working with sensitive personal data. 
• Experience working with a health data (genetic data, clinical data, NHS data) is advantegeous. In particular, any knowledge and experience of FHIR and health data models (OMOP, PCORnet, i2b2, CDISC) would be advantageous. 

Competencies 

• Experience working in/with cross-functional teams consisting of engineers, product, UX and non-technical stakeholders. 
• Proven ability to communicate with stakeholders and audiences at different levels. 
• Understanding of the importance of operational, security and legal factors when designing solutions, with the ability to articulate trade-offs and make informed decisions around design and technology choices. 
• Experience working in small, growing organisations with a high degree of ambiguity, where flexibility is valued. 
• Strong understanding of software quality principles and practices. 
• Experience in mentoring and coaching less experienced engineers. 
• Comfortable in facilitating technical discussions, encouraging collaboration and providing direction. 

Why join us? 

You’ll be joining at an interesting point in the evolution of our Data Engineering organisation. 

We’re looking at how shared tooling and engineering capabilities should work across our Data Product squads, which means there’s an opportunity to influence how we design and build these systems rather than simply inheriting an established model. 

You’ll work on genuinely challenging engineering problems involving large volumes of sensitive health data, while collaborating with engineers, data specialists and architects across the organisation. 

The role combines hands-on engineering with Staff level influence, helping teams make good technical decisions, developing other engineers and improving how we build software across the wider area. 

We’re looking for someone pragmatic, collaborative and comfortable working through ambiguity. You’ll have strong technical opinions, but know when to adapt them; you’ll care about engineering quality without over-engineering solutions; and you’ll be comfortable influencing people without relying on authority. 

Healthcare experience isn’t essential, but an interest in our mission is. It’s an opportunity to apply your engineering experience to technology that ultimately supports health research at an