Lead Data Engineer – Azure & Databricks

Candidate Description

With over 12 years navigating complex data landscapes, this Senior Lead Data Engineer excels in architecting and delivering large-scale cloud migrations and real-time analytics for organisations such as a Financial Institution, an Insurance Company, a Major Bank, and a Technology Company. Their core expertise spans Azure, Databricks and Apache Spark, underpinned by robust data-modelling acumen and a proven record of optimising pipelines from on-premises Oracle and SQL Server estates to high-performance Azure lakehouse environments.

Adept at orchestration, they leverage Azure Data Factory, Delta Live Tables, Event Hub and CI/CD practices to build resilient, cost-effective solutions. Key achievements include reducing processing times by 40 percent at a Major Bank through advanced ADF optimisation, pioneering Databricks notebooks that halved execution durations at a Financial Institution, and leading a 50 TB secure migration into Azure Synapse. Their toolkit embraces Data Vault and Medallion architectures, strong security posture with Azure Key Vault, and automated governance to ensure data integrity and compliance.

Beyond technical mastery, they demonstrate leadership by mentoring engineering teams, driving cross-functional workshops and aligning data strategies with stringent financial-services regulations. Their adaptability across banking, insurance and consulting contexts positions them as a versatile change agent who consistently transforms raw data into actionable insights and measurable business value.