Backend developer (Web crawling, Java & Python)

Backend developer (Web crawling, Java & Python)

What makes the Similarweb R&D team awesome?

• We love data! At Similarweb you have the opportunity to work with it on a massive scale, processing huge volumes of data across a wide variety of global marketplaces.
• We work with high‑load, highly dynamic environments, advanced bot‑protection mechanisms, and complex data pipelines running in Kubernetes.
• Work with cutting edge technology - we often beta test the tech that other people will only discover next year!
• We want you to get better and help you set goals and conquer them

Key Responsibilities:

• Maintain, optimize, and modernize legacy monolithic systems during migration to microservices.
• Design scalable backend solutions for high-frequency web crawling and data processing.
• Optimize database performance for MongoDB (primary), PostgreSQL, and ClickHouse.
• Collaborate with data teams to integrate pricing analytics and recommendation engines.
• Ensure code quality via CI/CD pipelines, automated testing, and peer reviews.

Core Requirements:

• 4+ years Java development (Core, multithreading, performance tuning).
• Expert-level Spring Boot (RESTful APIs, JPA, security, batch processing).
• Advanced SQL/NoSQL skills (schema design, query optimization, aggregation pipelines).
• Hands-on Docker/docker-compose for containerization and local development.
• Proficiency in Git/GitLab (branching strategies, code reviews, merge requests).
• Python scripting for crawling (Scrapy), prototyping or automation tasks.
• Fluent English (daily standups, technical docs, client-facing demos).

Critical Assets:

• Experience decomposing monoliths (DDD, event-driven architecture, API gateways).
• Familiarity with Kubernetes (cluster management, Helm charts, scaling).
• Kafka/event streaming for real-time data pipelines (producers/consumers, connectors).
• Monitoring tools (Prometheus/Grafana, ELK stack, error tracking).

Nice-to-Have:

• Web crawling libraries (Jsoup, Selenium) or frameworks (Scrapy, Apache Nutch).
• Understanding of proxy rotation, CAPTCHA bypass, and anti-scraping mechanisms.

We Offer:

• Impactful role in a niche SaaS product serving enterprise-level clients.
• Tech stack: Java 11+, Spring Boot, Kafka, GitLab CI/CD, Prometheus.
• Competitive compensation.
• Team culture prioritizing ownership, innovation, and upskilling.