Overview

We are looking for a Data Engineer to support the migration and modernization of our existing SQL Server–based data workloads to a cloud-native Lakehouse platform built on AWS and Databricks. In this role, you will design and operate scalable, resilient, high-quality data pipelines and services that empower analytics, real-time streaming, and machine learning use cases across the organization.
Client:
Our client is one of the largest betting communities, having pioneered the betting exchange model back in 2000. Powered by cutting-edge technology, they operate the world’s leading online betting exchange.
Project Overview:

Responsibilities:
  • Migrate legacy SQL Server workloads to a modern Lakehouse architecture on AWS and Databricks.
  • Design, build, and maintain data pipelines for batch and real-time processing.
  • Ensure data quality, reliability, and scalability across all pipelines and services.
  • Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions for analytics and ML use cases.
  • Implement best practices for data governance, security, and compliance.
  • Optimize performance and cost efficiency in a cloud-native environment.
Required Qualifications:
  • Strong proficiency in Python for data engineering tasks.
  • Hands-on experience with AWS services (e.g., S3, Glue, Lambda, EMR).
  • Expertise in Databricks and Spark for big data processing.
  • Solid understanding of SQL and relational database concepts.
  • Experience with ETL/ELT frameworks and workflow orchestration tools (e.g., Airflow).
  • Knowledge of data modeling, data warehousing, and Lakehouse principles.
Nice To Have:
  • Familiarity with streaming technologies (Kafka, Kinesis).
  • Experience with CI/CD pipelines for data solutions.
  • Understanding of data security and compliance in cloud environments.
  • Exposure to machine learning workflows and MLOps concepts.
Note:

✨ Our intelligent job search engine discovered this job and republished it for your convenience.
Please be aware that the job information may be incorrect or incomplete. The job announcement remains the property of its original publisher. To view the original job and its full details, please visit the job's URL on the owner’s page.

Please clearly mention that you have heard of this job opportunity on https://ijob.am.