This role is for one of the Weekday's clients

Min Experience: 4 years

Location: Bengaluru

JobType: full-time

Requirements

Key Responsibilities

As a Data Engineer, you will play a crucial role in designing and maintaining scalable and high-performance data systems. Your responsibilities will include:

1. Data Pipeline Development and Management

Design, build, test, and maintain efficient data pipelines and data management systems.
Develop and manage ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes to integrate data from diverse sources such as databases, APIs, and real-time streams.

2. Data Modeling and Architecture

Design data models and implement schemas for data warehouses and data lakes to support analytics and business operations.
Optimize data storage, access, and performance for scalability and maintainability.

3. Data Quality and Integrity

Implement validation, cleansing, and monitoring to maintain data accuracy, consistency, and reliability.
Define and enforce best practices and standards for data governance and quality.

4. Infrastructure Management

Manage and monitor key data infrastructure components including databases, data lakes, and distributed computing environments.
Apply data security protocols and ensure proper access controls are in place.

5. Automation and Optimization

Automate data workflows and pipelines to improve reliability and performance.
Continuously monitor and fine-tune systems for operational efficiency.

6. Collaboration and Support

Partner with data scientists, analysts, software engineers, and business stakeholders to gather requirements and provide scalable data solutions.
Document processes, workflows, and system designs; support cross-functional teams with technical guidance.

7. Technology Evaluation

Stay current with emerging tools and technologies in the data engineering space.
Evaluate and recommend new solutions to enhance data capabilities and performance.

Education and Experience

Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, Data Science, or a related field.
5 to 7 years of experience in data engineering, software development, or a similar domain.

Technical & Functional Competencies

Required Skills & Qualifications

Technical Proficiency

Programming: Strong experience in Python and SQL.
Databases: Proficient in relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases.
Data Warehousing & Lakes: Hands-on experience with platforms like Snowflake, Redshift, BigQuery.
ETL/ELT Tools: Proficiency with tools like Apache Airflow, AWS Glue, Azure Data Factory, Talend.
Big Data: Working knowledge of Apache Spark or similar big data technologies.
Cloud Platforms: Experience with AWS, Azure, or GCP for data engineering workflows.
Data Modeling: Strong understanding of modeling techniques and best practices.
API Integration: Ability to build and consume APIs for data integration.
Version Control: Experience with Git or other version control systems.

Soft Skills

Analytical mindset with a strong problem-solving approach.
Excellent communication skills for both technical and non-technical audiences.
Team player with a collaborative work ethic.
Detail-oriented with a commitment to data quality.
Adaptability to new technologies and changing project requirements.

Key Skills:
ETL, Data Modeling, Data Architecture, Cloud Data Platforms, Python, SQL, Big Data, Data Warehousing, API Integration