Data Engineer

Candidate Description

With over four years immersed in large-scale data engineering, this professional combines deep cloud expertise with a strong software-engineering foundation. They have delivered end-to-end pipelines across AWS and Azure ecosystems, applying Python, Spark and Airflow to drive high-volume, low-latency data processing for mining and retail enterprises. Adept at working in Agile environments, they consistently demonstrate analytical rigour and a commitment to code quality.

Their technical acumen is demonstrated through hands-on mastery of Azure Databricks, Azure Data Factory, AWS Lambda and DynamoDB, where they have optimised transformation workloads and reduced processing times by up to 30 %. Leveraging Apache Spark for distributed computation and Airflow for orchestration, they have architected resilient, cost-efficient solutions while enforcing robust testing practices via a project management tool and a test management tool. Successes include migrating complex workflows from ADF to Airflow at a Technology Consulting Firm, achieving a 20 % cost saving, and building pipelines capable of processing more than 1,000 inventory transactions per second.

Collaborative and outcome-focused, they routinely partner with business analysts, data scientists and operations teams to translate requirements into scalable data products. Experience across Consulting and Mining sectors highlights their adaptability, while mentoring sessions and comprehensive documentation underscore leadership qualities. Overall, they bring a rare blend of technical depth, stakeholder engagement and continuous-improvement mindset that accelerates digital transformation initiatives and delivers measurable business value.