VIQU IT
Data Engineer - Document & Retrieval Pipelines - Contract
Data Engineer - Document & Retrieval Pipelines - Contract at VIQU IT, based in North London, London. This is a contract role.
- Pay
- Not given by the employer
- Hours and contract
- Contract · On site
- Location
- North London, London
- Closing date
- Sector
- Data Analyst
Reference 5419294087
About the role
Data Engineer Document & Retrieval Pipelines | SC Cleared Morela is supporting a specialist technology consultancy on a high-profile defence AI programme, and we are looking for a Data Engineer to own the document and retrieval pipelines behind a set of AI-enabled planning tools. This is a contract role on a two-year programme, Outside IR35, based in Central London. Active SC clearance is required. About the Client Our client is a specialist technology consultancy that embeds engineers directly into government and defence delivery teams. They are known for long-term programme work and for giving their engineers genuine ownership of what they build. This is one of their most strategically important programmes. The Role You will join a team building AI-enabled planning tools. The core of the role is document and retrieval pipelines: getting messy, real-world documents in, then making them structured, searchable and measurable. The first increment produced working prototypes. The next increment is about building on them to deliver reliable, production-grade services for real end users, so there will be a strong focus on scaling and hardening the pipelines. You will work as part of a mixed team of engineers, military and civil service colleagues, so being able to represent yourself well and explain technical decisions clearly matters as much as the engineering. Responsibilities
- Build and maintain ingestion pipelines that extract clean text from PDF, DOCX and HTML, including layout-aware parsing of headings, tables and multi-column pages
- Design and build retrieval pipelines for RAG, covering chunking strategy, embedding models, vector search and metadata design
- Own the Postgres layer, including schema design, indexing (including vector indexes) and performance at scale
- Build labelled ground-truth datasets and use them to measure and improve retrieval quality
- Take prototype code to production, hardening pipelines with batching, idempotent re-ingestion and robust error handling
- Containerise services and maintain CI/CD pipelines so changes ship reliably Essential Requirements
- Clearance: Active SC clearance
- Python: Strong Python, with experience building and testing data pipelines and APIs (FastAPI, PyTest)
- Unstructured document ingestion: Extracting clean text from PDF, DOCX and HTML, including layout-aware parsing (headings, tables, multi-column pages)
- RAG retrieval pipelines: Chunking strategy, embedding models, vector search (Postgres with pgvector or similar) and metadata design xjnqpsq
- Postgres: Schema design, indexing (including vector indexes) and performance at scale
- Retrieval evaluation: Building labelled ground-truth datasets and measuring retrieval quality
- Productionising code: Hardening pipelines (batching, idempotent re-ingestion, error handling), containerisation with Docker and CI/CD with GitHub Actions Desirable Experience
- Cloud deployment, particularly Google Cloud; AWS (Bedrock, S3, ECR, EKS) is also used today
- Kubernetes and Helm
- Working with LLM providers (OpenAI, Anthropic, Bedrock) and locally hosted models (Ollama)
- Data labelling workflows (e.g. Label Studio)
- Government, defence or other security-cleared programmes Working Environment Central London, embedded in a mixed programme team with clear delivery milestones. Outside IR35, two-year contract
Reference: 5419294087 · Posted 1 week ago · Closes 4 Jan 2027 · Listed via VIQU IT
Apply for this job
This role is listed via VIQU IT. Applications are handled on the employer's site.
Apply on employer siteOpens the employer's website in a new tab.
Signed in? Every job you apply for counts toward your weekly goal and badges. How it works
Safe applying: a genuine employer will never ask you to pay for a DBS check, training or equipment, or move you onto WhatsApp before you are hired. If this listing does, report it and do not pay anything.