Data Quality Engineer (Databricks+PySpark)-UAE

Position

Data Quality Engineer (Databricks)

Overview

We are seeking a Data Quality Engineer with strong Databricks and PySpark expertise to design, implement, and manage enterprise data quality frameworks, automated validation rules, monitoring solutions, and data quality reporting.

Key Responsibilities

• Develop and maintain Data Quality frameworks in Databricks.
• Build automated profiling, validation, and cleansing processes using PySpark.
• Design reusable Data Quality rules and controls.
• Implement quality checks across Delta Lake pipelines.
• Develop monitoring, alerting, and reporting solutions.
• Support root cause analysis and remediation activities.
• Collaborate with Data Governance, Data Modeling, and Engineering teams.

Required Skills

• Strong Databricks and PySpark experience.
• Data Quality Framework implementation.
• Delta Lake and Unity Catalog.
• Databricks Workflows, Jobs, and SQL.
• Data Profiling, Validation, and Cleansing.
• Experience with Power BI reporting.

Preferred

• MLflow.
• Azure Monitor.
• Data Governance and Metadata Management.