Data Engineer - Databricks

Job Description

Databricks Data Engineer - London (Hybrid) - Databricks - Azure - PySpark - Up to £60,000

\n

Are you a Data Engineer looking to take your Databricks experience to the next level?

\n

Do you want to work on large-scale Data & AI transformation programmes, helping enterprise organisations unlock the value of their data through modern cloud, analytics and AI platforms?

\n

We're partnering with a consultancy that is investing heavily in its Data & AI practice and expanding its Databricks capability across the UK. As a result, we're looking for talented Databricks Data Engineers to join a growing team delivering cutting-edge solutions for some of the UK's most recognisable organisations.

\n

This is an opportunity to work at the forefront of modern data engineering, building scalable Lakehouse platforms that enable advanced analytics, machine learning and AI use cases. You'll collaborate with experienced architects, engineers and AI specialists, gaining exposure to complex projects, modern technologies and clear career progression opportunities.

\n

Whether you're already working in consulting or building Databricks solutions within industry, this role offers the chance to accelerate your career in a highly respected and rapidly growing environment.

\n

What You'll Be Doing

\n

\n
• Designing and developing modern Databricks Lakehouse solutions
\n
• Building scalable batch and streaming data pipelines using PySpark and Spark SQL
\n
• Creating trusted, governed and AI-ready data products
\n
• Implementing solutions using Delta Lake, Unity Catalog and Databricks SQL
\n
• Supporting projects through the full delivery lifecycle, from requirements gathering and solution design through to deployment
\n
• Optimising data platforms for performance, scalability and reliability
\n
• Working closely with architects, cloud engineers, data scientists and business stakeholders
\n
• Supporting machine learning and AI-driven initiatives through robust data foundations
\n
• Applying modern engineering practices including CI/CD, automated testing and Infrastructure as Code
\n
• Contributing to technical best practices and continuous improvement across project teams
\n

\n

Technology Stack

\n

You'll have the opportunity to work with technologies including:

\n

\n
• Databricks Lakehouse Platform
\n
• PySpark & Spark SQL
\n
• Delta Lake
\n
• Unity Catalog
\n
• Azure, AWS or GCP
\n
• MLflow
\n
• Databricks Workflows
\n
• Azure Data Factory
\n
• Generative AI & Machine Learning
\n

\n

What We're Looking For

\n

Essential

\n

\n
• Commercial experience in Data Engineering
\n
• Hands-on Databricks experience in a production environment
\n
• Strong Python and SQL skills
\n
• Experience building and supporting ETL/ELT pipelines
\n
• Understanding of Lakehouse Architecture, Delta Lake or Medallion Architecture
\n
• Experience working with Azure, AWS or GCP
\n
• Strong communication and stakeholder engagement skills
\n

\n

Desirable

\n

\n
• Unity Catalog
\n
• Azure Data Factory, Airflow or Databricks Workflows
\n
• Databricks certifications
\n
• CI/CD and Infrastructure as Code
\n
• MLflow, Feature Engineering or AI-related workloads
\n
• Consultancy or client-facing experience
\n

\n

What's In It For You?

\n

\n
• Up to £60,000 base salary
\n
• Annual performance bonus
\n
• Hybrid working
\n
• Private healthcare
\n
• Competitive pension scheme
\n
• Flexible benefits package
\n
• Industry-recognised training and certifications
\n
• Exposure to enterprise-scale Data, AI and Cloud programmes
\n
• Clear and structured career progression
\n
• Collaborative and inclusive culture
\n
• Opportunity to work with leading Databricks, Data & AI specialists
\n

\n

If you're passionate about modern data engineering, Databricks and the future of AI, and you're looking for an opportunity where you can make an impact whilst accelerating your career.

\n

Apply today