yesterday
Senior Data Engineer (Spark)
SSC HR Solutions · Cairo, Cairo Governorate, Egypt
WorkableApply on company site
Remote
Senior
Role overview
Description Builds and runs large scale batch data pipelines, keeping jobs fast and affordable as data volumes grow. Works with Apache Spark to build and manage large scale data processing jobs, including performance tuning to maintain efficient and reliable pipeline execution. Supports data modelling for analytics and organises data in a lakehouse so it is usable downstream. Handles automated scheduling, monitoring, and data quality checks, while working with platform and product teams on end to end data flows.
Requirements
Requirements
- Strong hands on Apache Spark including performance tuning, not Spark usage through a managed notebook only.
- Data modelling for analytics and organising data in a lakehouse so it is usable downstream.
- Automated scheduling, monitoring, and data quality checks.
- Works with platform and product teams on end to end data flows.
- Apache Iceberg or other open table formats.
- Trino or similar query engines.
- On premises or self managed cluster experience.
Responsibilities
1Requirements
2Strong hands on Apache Spark including performance tuning, not Spark usage through a managed notebook only.
3Data modelling for analytics and organising data in a lakehouse so it is usable downstream.
4Automated scheduling, monitoring, and data quality checks.
5Works with platform and product teams on end to end data flows.
6Apache Iceberg or other open table formats.
7Trino or similar query engines.
8On premises or self managed cluster experience.
Requirements
1Strong hands on Apache Spark including performance tuning, not Spark usage through a managed notebook only.
2Data modelling for analytics and organising data in a lakehouse so it is usable downstream.
3Automated scheduling, monitoring, and data quality checks.
4Works with platform and product teams on end to end data flows.
5Apache Iceberg or other open table formats.
6Trino or similar query engines.
7On premises or self managed cluster experience.
Skills and tags
EngineeringEGData Analysis
