Senior Data Engineer – GCP

One of our growing igamign clients with offices in Dubai and globally is looking to strengthen their Data team with a Senior Data Engineer with experience with GCP for the below position:

As a Senior Data Engineer, you will be a key hands‑on contributor within the modernisation programme that the Lead Data Engineer drives. You will own critical ingestion, transformation and modelling work, influence architecture decisions, and partner with analytics, product and platform teams.

CORE RESPONSIBILITIES

Pipeline Engineering

· Own one or more domain pipelines (bets/payouts/bonuses/taxes/wallet) including schema evolution, data contracts, and runbooks.

· Build, operate and optimise high‑volume streaming (Pub/Sub → Dataflow) and batch (Composer/Airflow) pipelines.

· Apply dbt‑based ELT patterns for incremental models, tests and documentation.

· Tune partitioning, clustering and slot usage in BigQuery to balance latency and cost.

Architecture Contribution

· Collaborate with the Lead Data Engineer to translate business use‑cases into secure, scalable data‑flows.

· Propose storage formats (BigLake/Parquet/Iceberg) and compute patterns appropriate to the workload.

· Advocate medallion‑style layer separation (bronze/silver/gold).

Governance & Quality

· Implement data‑quality tests (dbt, Great Expectations) and monitoring alerts in Cloud Monitoring.

· Enforce policy‑tags, row/column‑level security and authorised views for compliance.

· Maintain lineage metadata and version control for all SQL/Python artefacts.

Ops & Automation

· Codify infrastructure with Terraform modules (projects, service accounts, monitoring policies).

· Participate in pager‑rotation, incident response and post‑mortems for the data platform.

· Introduce contract tests (producer & consumer), ephemeral test datasets, and data diff checks in CI.

Collaboration & Mentoring

· Work with BI analysts to materialise business‑ready data models (NGR, player 360, fraud signals).

· Peer‑review code, share best practices, and coach interns or junior engineers on GCP tooling.

· Communicate trade‑offs and estimates clearly.

MINIMUM QUALIFICATIONS

• 4+ years professional experience in data engineering or analytics engineering.

• Expert SQL plus production‑grade Python (testing, packaging, code review).

• Proven delivery of GCP‑native data solutions—BigQuery, Pub/Sub, Dataflow/Datastream, Cloud Storage.

• Hands‑on with Airflow/Cloud Composer and dbt for transformation and testing.

• Solid grasp of dimensional modelling, incremental loads and basic SCD Type 2 patterns.

• Working knowledge of data‑security concepts (IAM, KMS, masking, policy tags) and compliance constraints.

• Experience monitoring pipelines, cost and performance metrics in Cloud Monitoring/Logging.

PREFERRED / NICE‑TO‑HAVE

• Exposure to real‑time CDC tools (Debezium, Datastream) and exactly‑once semantics.

• Familiarity with Data Vault 2.0, medallion/lake‑house architectures or feature‑store patterns.

• Infrastructure‑as‑code expertise (Terraform, Cloud Build, GitHub Actions).

• Domain experience in gaming, fintech or other low‑latency, high‑volume environments.

• Contribution to OSS data tooling or community meet‑ups.

WHAT SUCCESS LOOKS LIKE IN 12 MONTHS

1) SLO-backed pipelines with <2 Sev-2 incidents/quarter, MTTR ≤4h, and 95th-percentile freshness met for 3 consecutive months.

2) At least 20 production dbt models shipped with tests, docs and lineage.

3) Demonstrated cost trend improvements (scanned bytes/GB/query & slot hours per workload ↓ quarter-over-quarter from baselines).

4) Positive feedback from BI, product and compliance on data reliability and transparency.

5) Documented design proposals adopted by the broader platform architecture council.