Senior Software Engineer (Python Scraping Expert)

Purpose:

We are looking for a Senior Software Engineer (Python Scraping Expert) with strong expertise in Python, Scrapy, BeautifulSoup, Selenium, Playwright, and API Integration to design, develop, and maintain advanced web scraping solutions for data extraction and processing. The role involves building and optimizing scraping scripts, handling dynamic websites, captchas, proxies, and APIs, storing structured data, ensuring scalability and compliance, and collaborating closely with stakeholders to define and deliver data requirements.

Qualifications, Skills, And Experience:

• Bachelor’s degree in Computer Science or a related field.
• 3–5 years of relevant experience in Python development with a focus on web scraping and data extraction.
• Expertise in Python, Scrapy, BeautifulSoup, Selenium, Playwright, and API Integration.
• Proven experience working with dynamic websites, proxies, and pagination.
• Strong understanding of data storage formats such as JSON, CSV, and databases.
• Outstanding communication skills with experience in client-facing interactions.

Key Responsibilities:

• Build Scraping Solutions: Design, implement, and fine-tune web scraping scripts using Python frameworks and libraries to extract data efficiently from various sources.
• Tackle Dynamic Websites: Utilize tools such as Selenium and Playwright to manage websites with dynamic content, captchas, proxies, and pagination challenges effectively.
• Integrate APIs: Implement API integrations for data retrieval, ensuring clean, structured, and reliable outputs aligned with project requirements.
• Ensure Data Quality: Maintain data accuracy, reliability, and scalability of scraping solutions while adhering to legal and ethical scraping guidelines.
• Structure & Store Data: Store extracted data in standardized formats (JSON, CSV) and databases, enabling easy access and downstream processing.
• Collaborate Effectively: Work closely with project owners, technical teams, and client representatives to understand data needs, provide timely solutions, and support ongoing objectives.

A Culture of Belonging: At our core, we value diversity and inclusion. As an equal opportunity employer, we are dedicated to creating a workplace where every voice is heard, every person is respected, and everyone has the opportunity to succeed.