Big Data Platform Engineer

 

BIG DATA PLATFORM ENGINEER

 

Important Information

Location: Singapore

 

KEY FEATURES OF THE POSITION

• Operating Global Data Platform components (VM Servers, Kubernetes, Kafka) and applications (Apache stack, Collibra, Dataiku and similar).

• Implement automation of infrastructure, security components, and Continuous Integration & Continuous Delivery for optimal execution of data pipelines (ELT/ETL).

• Develop solutions to build resiliency in data pipelines with platform health checks, monitoring, and alerting mechanisms, quality, timeliness, recency, and accuracy of data delivery are improved 

• Apply DevSecOps & Agile approaches to deliver the holistic and integrated solution in iterative increments.

• Liaison and collaborate with enterprise security, digital engineering, and cloud operations to gain consensus on architecture solution frameworks.

• Review system issues, incidents, and alerts to identify root causes and continuously implement features to improve platform performance. 

• Be current on the latest industry developments and technology trends to effectively lead and design new features/capabilities

Technical Skills

• Bachelor's degree in Computer Science, Information Technology, or related field

• At least 5 years of experience in big data engineering, with a focus on cloud-based infrastructure 

• Strong proficiency in: • DevOps tools like Jenkins, GitLab CI/CD, or similar

• Containerization using Docker and Kubernetes

• Apache Spark and big data processing

• Real-time data integration using Apache Kafka

• Distributed SQL engines like Presto and Trino

• Data science platforms like Dataiku

• Cloud-based object storage solutions like Amazon S3 

• Ansible Playbooks to automate infrastructure provisioning, configuration, and deployment

• Terraform to manage and provision cloud infrastructure resources

• Experience with programming languages like Python

• Experience with cloud migration, including assessment, planning, execution, and validation

• Experience with disaster recovery setup, including backup and restore processes, replication, and failover mechanisms

 

About Encora

Encora is a global company that offers Software and Digital Engineering solutions. Our practices include Cloud Services, Product Engineering & Application Modernization, Data & Analytics, Digital Experience & Design Services, DevSecOps, Cybersecurity, Quality Engineering, AI & LLM Engineering, among others.

At Encora, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality