Data Engineer

Job title: Data Engineer
Location: Surrey (Hybrid working)
Type: Permanent
Salary: £50,000-£75,000
Visa sponsorship: Not available

A UK-based technology business building advanced data-driven capability for a government-facing mission. The team works with real-world sensor and geospatial data at scale, across cloud and edge/on‑prem environments, and is growing due to increased demand and new programmes.

The role

We're hiring a Data Engineer to design, build and run scalable data pipelines and data models that turn multiple complex sources into reliable, accessible datasets for analytics and machine learning. You'll support batch, streaming and interactive use cases, including modern approaches such as image embeddings and LLM-enabled conversational analytics.

You'll work closely with data scientists, analysts, software engineers and platform engineers in an agile (Kanban) environment. The platform operates across Microsoft Azure plus edge and on‑prem deployments.

What you'll do

• Build and maintain robust batch + streaming pipelines (ingestion, transformation, orchestration)
• Monitor and support production data services; investigate data quality, reliability and performance issues through to resolution
• Develop reusable platform components, internal libraries and tooling to improve engineering productivity
• Implement automated testing, validation checks and integration plans for pipelines and data products
• Design data models and storage patterns supporting analytics, ML features, embeddings and conversational querying
• Document data flows, schemas, operational processes and technical decisions

What we're looking for (must-have)

• Strong Python (clean, maintainable, production-grade data engineering code)
• Strong SQL and experience designing relational + analytical data models
• Experience with structured/time-series/high-volume storage (e.g., SQL databases, InfluxDB or similar)
• Experience building batch and event-driven/streaming data pipelines
• ~3+ years' experience in a data engineering / data platform team
• Familiarity with distributed processing (e.g., Spark, Flink, or similar)
• Orchestration tooling to schedule/monitor/recover pipelines
• Understanding of event-driven architecture, message brokers and stream-processing patterns
• Hands-on with Azure data services and cloud data platform patterns
• Comfortable building/debugging Docker containers
• Experience deploying/operating data workloads on Kubernetes (cloud/edge/on‑prem)
• Experience developing/consuming APIs for ingestion, access and integration
• Strong attention to detail; able to prioritise independently; solid communicator

Nice to have

• Degree in CS / data engineering / maths (or equivalent experience)
• TileDB (or similar multidimensional/geospatial/scientific data tech)
• Geospatial + time-series + image/vector search experience (embeddings)
• Exposure to RAG / LLM conversational analytics or ML data pipelines
• Experience designing for intermittently connected edge environments

Benefits (summary)

• Bonus scheme
• Private healthcare + cash plan
• Life assurance (4x salary)
• Share options