Senior Machine Learning Engineer II

Pos-29621

We're seeking a Machine Learning Engineer to join our Developer Experience team and own the operational deployment and performance optimization of our AI coding infrastructure. You'll be the expert who ensures our code generation models run reliably and efficiently at scale, powering the systems that help developers write software.

You'll work across the full model lifecycle. This includes fine-tuning open source models for code generation tasks and implementing RLHF pipelines to improve code quality and align with developer workflows, then taking those customized models and deploying them at scale. You'll evaluate and test bleeding-edge code models as they're released, debug distributed inference frameworks like vLLM, SGLang, and Ray, resolve GPU memory allocation issues, manage CUDA dependencies and kernel compatibility, and navigate the ever-shifting landscape of ML library ecosystems.

Your primary focus will be maximizing the throughput and capacity we can extract from our GPU infrastructure, turning experimental code models and fine-tuned variants into production-ready systems that generate code at scale for thousands of developers.

We are looking for people who have:

• A Boston-area location (or willingness to be local) and an interest in working in the office a couple of days each week.
• A track record of deploying and optimizing LLMs in production environments that serve users at scale
• Versatility to work across the stack in multiple languages (TypeScript, Java) as needed to integrate models into production systems
• Rigor to improve model performance through experimentation with inference configurations, fine-tuning, and RLHF
• Experience evaluating different models and inference frameworks to identify the best fit for specific latency, throughput, and quality requirements
• Ability to communicate complex technical tradeoffs to product and engineering teams
• Ownership mindset working cross-functionally to ship model improvements that directly impact developer productivity
• Curiosity to experiment with new model releases and appetite to debug the messy edge cases that come with production ML systems

Minimum Qualifications:

• Bachelor's degree in Computer Science or related field
• 5+ years of experience in machine learning or ML infrastructure roles
• Strong proficiency with LLM inference frameworks (vLLM, SGLang, Ray, or similar)
• Experience with Python and deep learning frameworks (PyTorch, TensorFlow)
• Hands-on experience with GPU optimization, CUDA, and distributed computing
• Experience with model evaluation, fine-tuning, or RLHF for LLMs
• Familiarity with MLOps best practices around model deployment, monitoring, and quality assurance

Preferred Qualifications:

• Master degree in a specialized discipline like Data Science or Machine Learning
• Experience specifically with code generation models or AI coding tools
• Background in developer tooling (static analysis, IDEs, development tools, etc…)
• Contributions to open source ML infrastructure projects

If you're passionate about leveraging machine learning to transform how developers build software, come join us at HubSpot!

Pay & Benefits

The cash compensation below includes base salary, on-target commission for employees in eligible roles, and annual bonus targets under HubSpot’s bonus plan for eligible roles. In addition to cash compensation, some roles are eligible to participate in HubSpot’s equity plan to receive restricted stock units (RSUs). Some roles may also be eligible for overtime pay. Individual compensation packages are tailored to your skills, experience, qualifications, and other job-related reasons.

This resource will help guide how we recommend thinking about the range you see. Learn more about HubSpot’s compensation philosophy.

Benefits are also an important piece of your total compensation package. Explore the benefits and perks HubSpot offers to help employees grow better.

At HubSpot, fair compensation practices aren’t just about checking off the box for legal compliance. It’s about living out our value of transparency with our employees, candidates, and community.

Annual Cash Compensation Range:$191,000—$305,600 USD

We know the confidence gap and impostor syndrome can get in the way of meeting spectacular candidates, so please don’t hesitate to apply — we’d love to hear from you.

If you need accommodations or assistance due to a disability, please reach out to us using this form.

At HubSpot, we value both flexibility and connection. Whether you’re a Remote employee or work from the Office, we want you to start your journey here by building strong connections with your team and peers. If you are joining our Engineering team, you will be required to attend a regional HubSpot office for in-person onboarding. If you join our broader Product team, you’ll also attend other in-person events such as your Product Group Summit and other gatherings to continue building on those connections.

If you require an accommodation due to travel limitations or other reasons, please inform your recruiter during the hiring process. We are committed to supporting candidates who may need alternative arrangements

Massachusetts Applicants: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

Germany Applicants: (m/f/d) - link to HubSpot's Career Diversity page here.

India Applicants: link to HubSpot India's equal opportunity policy here.

About Hubspot

HubSpot (NYSE: HUBS) is an AI-powered customer platform with all the software, integrations, and resources customers need to connect marketing, sales, and service. HubSpot's connected platform enables businesses to grow faster by focusing on what matters most: customers. 

At HubSpot, bold is our baseline. Our employees around the globe move fast, stay customer-obsessed, and win together. Our culture is grounded in four commitments: Solve for the Customer, Be Bold, Learn Fast, Align, Adapt & Go!, and Deliver with HEART. These commitments shape how we work, lead, and grow.

We’re building a company where people can do their best work. We focus on brilliant work, not badge swipes. By combining clarity, ownership, and trust, we create space for big thinking and meaningful progress. And we know that when our employees grow, our customers do too.

Recognized globally for our award-winning culture by Comparably, Glassdoor, Fortune, and more, HubSpot is headquartered in Cambridge, MA, with employees and offices around the world.

Explore More:

• HubSpot Careers
• Life at HubSpot on Instagram

By submitting your application, you agree that HubSpot may collect your personal data for recruiting, global organization planning, and related purposes. Refer to HubSpot's Recruiting Privacy Notice for details on data processing and your rights.