Big Data Developer (Spark/Scala + Python)

Job Summary

Synechron is seeking an experienced Big Data Developer with strong expertise in Spark, Scala, and Python to lead and contribute to large-scale data projects. The role involves designing, developing, and implementing robust data solutions that leverage emerging technologies to enhance business insights and operational efficiency. The successful candidate will play a key role in driving innovation, mentoring team members, and ensuring the delivery of high-quality data products aligned with organizational objectives.

Software Requirements

• Required:
  • Apache Spark (latest stable version)
  • Scala (version 2.12 or higher)
  • Python (version 3.6 or higher)
  • Big Data tools and frameworks supporting Spark and Scala
• Preferred:
  • Cloud platforms such as AWS, Azure, or GCP for data deployment
  • Data processing or orchestration tools like Kafka, Hadoop, or Airflow
  • Data visualization tools for data insights

Overall Responsibilities

• Lead the development and implementation of data pipelines and solutions using Spark, Scala, and Python
• Collaborate with business and technology teams to understand data requirements and translate them into scalable solutions
• Mentor and guide junior team members on best practices in big data development
• Evaluate and recommend new technologies and tools to improve data processing and quality
• Stay informed about industry trends and emerging technologies relevant to big data and analytics
• Ensure timely delivery of data projects with high standards of quality, performance, and security
• Lead technical reviews, code reviews, and provide inputs to improve overall development standards and practices
• Contribute to architecture design discussions and assist in establishing data governance standards

Technical Skills (By Category)

Programming Languages:

• Essential: Spark (Scala), Python
• Preferred: Knowledge of Java or other JVM languages

Data Management & Databases:

• Experience with distributed data storage solutions (HDFS, S3, etc.)
• Familiarity with NoSQL databases (e.g., Cassandra, HBase) and relational databases for data integration

Cloud Technologies:

• Preferred: Cloud platforms (AWS, Azure, GCP) for data processing, storage, and deployment

Frameworks & Libraries:

• Spark MLlib, Spark SQL, Spark Streaming
• Data processing libraries in Python (pandas, PySpark)

Development Tools & Methodologies:

• Version control (Git, Bitbucket)
• Agile methodologies (Scrum, Kanban)
• Data pipeline orchestration tools (Apache Airflow, NiFi)

Security & Compliance:

• Understanding of data security best practices and data privacy regulations

Experience Requirements

• 5 to 10 years of hands-on experience in big data development and architecture
• Proven experience in designing and developing large-scale data pipelines using Spark, Scala, and Python
• Demonstrated ability to lead technical projects and mentor team members
• Experience working with cross-functional teams including data analysts, data scientists, and business stakeholders
• Track record of delivering scalable, efficient, and secure data solutions in complex environments

Day-to-Day Activities

• Develop, test, and optimize scalable data pipelines using Spark, Scala, and Python
• Collaborate with data engineers, analysts, and stakeholders to gather requirements and translate into technical solutions
• Lead code reviews, mentor junior team members, and enforce coding standards
• Participate in architecture design and recommend best practices in big data development
• Monitor data workflows performance and troubleshoot issues to ensure data quality and reliability
• Stay updated with industry trends and evaluate new tools and frameworks for potential implementation
• Document technical designs, data flows, and implementation procedures
• Contribute to continuous improvement initiatives to optimize data processing workflows

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field
• Relevant certifications in cloud platforms, big data, or programming languages are advantageous
• Continuous learning on innovative data technologies and frameworks

Professional Competencies

• Strong analytical and problem-solving skills with a focus on scalable data solutions
• Leadership qualities with the ability to guide and mentor team members
• Excellent communication skills to articulate technical concepts to diverse audiences
• Ability to work collaboratively in cross-functional teams and fast-paced environments
• Adaptability to evolving technologies and industry trends
• Strong organizational skills for managing multiple projects and priorities

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT
 

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice