Data Engineer

Data Engineer [for Project Domino; preferably senior 5 years and up experience] Responsibilities
Design, architect and maintain secure data platforms such as setting up data storage solutions like SQL databases
Implement and maintain optimized workflows, ETL / ELT processes, and pipelines; deliver flexibility for data consumption by designing batch and streaming data use cases
Ensure support for traceability and reproducibility through enabling of data lineage features for ingested data
Work with colleagues such as Data Scientists, Data Analysts, software and data engineers, and architects on implementing solutions involving data on distributed systems and platforms
Create highly available and scalable capabilities that deliver data related tooling and manage data platforms and pipelines for stakeholders
Qualifications
Proficiency and experience in software engineering programming languages such as Python. Experience in Golang is preferred
Expertise in SQL / RDBMS databases and proficiency in setting up these resources. Additional experience in NoSQL databases and setting up these is preferred
Extensive experience in design and implementation of data handling processes and pipelines (ingestion, transformation, modelling, and storage) using Python, Apache Airflow, Spark
Expertise in cloud platforms and technologies (Azure, AWS, GCP) is a plus but not required
Experience with streaming platform setup and consumption such as Kafka, Flink
Experience with consumption from other storages for streaming such as S3, Spark
Experience in containers and container orchestration platforms such as Docker and Kubernetes
Experience in CI / CD and Devops, scripting languages such as shell and automation
Preferably a degree in the related fields of Computer Science
Strong problem solving and solutions engineering mindset
Experience in working and coordinating remotely