Senior Big Data Engineer

Description

Bigabid is an innovative technology company led by data scientists and engineers devoted to mobile app growth. Our proprietary ad platforms, powered by machine learning, are the result of that devotion.

We deliver valuable results and insights for a fast-growing clientele of major app developers using elite programmatic user acquisition and retargeting technologies. Our ever-evolving, state-of-the-art machine learning technology analyzes tens of TB of raw data per day to produce millions of ad recommendations in real time. 

As a Senior Big Data Engineer, you’ll join a core infrastructure team shaping the backbone of our real-time data ecosystem responsible for processing ~50TB of data every day. You’ll evaluate new technologies, design and build high-scale, low-latency mission-critical production grade systems at petabyte scale that power our prediction engines and decision-making processes.

This is a high-impact role where you’ll influence architecture decisions, mentor peers, and work with state-of-the-art big data tech.

Responsibilities:

• Architect and implement scalable streaming data infrastructure designed for massive growth in both data volume and complexity, ensuring high performance and fault tolerance.
• Build and optimize a low-latency serving layer capable of handling massive parallel throughput and real-time data access at scale.
• Drive technology evaluation and adoption, leading the selection and integration of cutting-edge tools and frameworks into the data ecosystem.
• Develop a deep understanding of our data lifecycle—how it is acquired, processed, versioned, and made discoverable—ensuring it is easily consumable across the organization.

Requirements

• 6+ years of proven experience as a Data/Backend Engineer with strong coding skills in Scala, Java, or Python.
• Experience with cloud-native architecture, with a strong advantage for AWS.
• Proven experience designing and building data pipelines using Apache Spark, Airflow, RDBMS and NoSQL databases.
• Proficiency in SQL with strong analytical skills.

Advantage:

• Experience with streaming systems (Spark Structured Streaming, Kafka Streams, or Flink).
• Experience with Kafka, Iceberg, Aerospike, Serverless, and Containers (Docker/Kubernetes).
• Familiarity with CI/CD infrastructures such as Terraform, AWS CDK
• Comfortable collaborating on occasional basic DevOps or database operations tasks 

Soft Skills:

• Ownership mindset and ability to work in a fast-moving environment.
• Proven ability to take projects from ideation through design to production in complex systems.
• Excellent English communication skills (verbal and written) for cross-functional collaboration.

Excerpt

We are looking for a highly skilled, self-managing experienced engineer with experience in both batch and real-time streaming data pipelines, along with a problem-solving mindset.