Azure Data Engineers / AWS Data Engineers

Requirements

Experience in Developing Data Pipelines that process large volumes of data using Python, PySpark, Pandas etc, on AWS / Azure

Experience in developing ETL, OLAP based and Analytical Applications.

Experience in ingesting batch and streaming data from various data sources.

Strong Experience in writing complex SQL using any RDBMS (Oracle, PostgreSQL, SQL Server etc.)

Ability to quickly learn and develop expertise in existing highly complex applications and architectures.

Exposure to AWS platform's data services (AWS Lambda, Glue, Athena, Redshift, Kinesis etc.)

such as Azure Data Factory (ADF), Azure Data Bricks (ADB),Azure Synapse Analytics, Azure Active Directory, Azure Storage, Azure data Lake Services  (ADLS), Azure key vault, Azure SQL DB, Azure HD Insight.

Experience in Airflow DAGS, AWS EMR, S3, IAM and other services

Snowflake or Redshift data warehouses

Experience of DevOps and CD/CD tools.

Familiarity with Rest APIs

• Clear and precise communication skills
• Experience with CI/CD pipelines, branching strategies, & GIT for code management
• Comfortable working in Agile projects

Benefits