Senior Data Engineer (Databricks)

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.


If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Experience Level: 3 to 6 Years

Work location:  Mumbai, Bengaluru, Trivandrum

Role & Responsibilities:

• Design and implement data solutions (ETL pipelines, Processing and Transformation logic) using Snowflake as a key platform.
• Design Virtual warehouses which are optimized for performance and cost (auto scaling, idle time etc)
• Write SQL , stored procedures to implement business logic, other critical requirements like encryption, data masking etc
• Implement solutions  with features like Snowpipe, tasks, streams,dynamic tables etc
• Design the solutions by leveraging key concepts around the architecture of Snowflake, its data distribution and partitioning mechanism, data caching etc.
• Identify queries which take more time , look at the logs etc and provide solutions to optimize those queries

Must Have Skills:

• 3+ years’ experience of Hands-on in data structures, distributed systems, Spark, SQL, PySpark, NoSQL Databases 
• Strong software development skills in at least one of:  Python, Pyspark or Scala.
• Develop and maintain scalable & modular data pipelines using databricks & Apache Spark
• Experience building and supporting large-scale systems in a production environment    
• Experience working  on either of the cloud storages - AWS S3, Azure data lake, GCS buckets
• Exposure to the ELT tools offered by the cloud platforms like, ADF, AWS Glue, Google dataflow
• Integrate Databricks with other cloud services like AWS, Azure or Google Cloud
• Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks
• Managing and configuring databricks environments, including clusters and notebooks
• Experience working with different file formats - parquet, databricks delta lake
• Experience working with data governance on databricks, implementing unity catalog, delta share
• 3+ years’ experience of Hands-on in data structures, distributed systems, Spark, SQL, PySpark, NoSQL Databases 
• Strong software development skills in at least one of:  Python, Pyspark or Scala.
• Develop and maintain scalable & modular data pipelines using databricks & Apache Spark
• Experience building and supporting large-scale systems in a production environment    

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!