Principal AI Engineer

About the role

In the role of Principal AI Engineer for Concepts Lab, you will report to the VP of the Concepts Lab.

We are considering applicants for the location of Los Angeles, CA (onsite).

As the Principal AI Engineer for Concepts Lab, you will be the technical anchor for everything AI in the lab: designing and building the agentic delivery tooling, evaluation harnesses, and paved paths that let cross-functional pods take ideas from concept to production in days instead of months.

This is a hybrid technical and enablement role. You will move fluidly between hands-on agent and platform development, instrumenting and measuring whether the system is actually working, and partnering with stakeholders across the company to roll those capabilities out. Success in your first year looks like a Concepts Lab where the cross-functional pod model has been stress-tested with real evidence, and where we can point to specific KPIs and say with confidence whether agentic delivery is making us faster and better, or whether we need to course-correct.

Core Areas of Responsibility

• Own the design and evolution of the lab’s agentic delivery system, including the agent harness, CLI and skill interfaces, workflow orchestration, isolation, evaluation, and observability layers that let agents safely read, write, refactor, and test code across our repos.
• Own the measurement layer that tells us whether agentic delivery is working. You will define and instrument the KPIs we will judge this program by, including cycle time (idea to prototype, decision to production), quality and rework rates, integration cost, eval pass rates, cost per cycle, pod throughput, adoption, and direction-stability signals. You will build the dashboards and learning reports that turn the lab’s 2-week sprints into evidence executives can act on, and you will be honest when the numbers say something is not working.
• Lead the integration work that makes agents effective in our environment, including context engineering, repo and tool access patterns, MCP servers, and the connective tissue between agents and the systems they need to reason about (codebases, docs, ownership, incidents, runtime).
• Drive enablement and adoption across Product, Design, and Engineering. You will run the rollouts that take the agentic stack from Concepts Lab into pilot teams and beyond: onboarding cross-functional pods, codifying paved paths and best-practice playbooks, embedding alongside teams during their first cycles, and using the feedback to harden the platform. You will be the person whose job it is to make the tooling actually get used, not just exist.
• Set the technical bar for the lab’s AI work, including design reviews, code quality, eval coverage, observability, cost management, and security practices, and mentor the engineers and contractors who collaborate with us.
• Be the opinionated voice on how cross-functional pods should be designed and how agentic systems scale inside a large engineering organization. You will help leadership think clearly about pod composition, human touchpoints, where to keep humans in the loop and where to take them out, how to roll out team by team without breaking what works, and how to avoid the common failure modes (review overload, shadow product teams, prototypes that become unowned production systems). You will bring a strong point of view, defend it with evidence, and update it when the data says you were wrong.

About You

We get excited about candidates, like you, because…

• You have 15+ years of professional software engineering experience building production systems, with a track record of operating as a senior individual contributor or technical lead.
• You have hands-on experience building and shipping agentic systems in production, including designing agent harnesses, tool-use and MCP integrations, multi-step workflows, and the eval and observability layers that keep them reliable.
• You possess deep expertise with modern agentic coding tools and frameworks (Claude Code, Cursor, Devin, or comparable) used across the full development cycle, and you can speak to where they break and how to extend them.
• You are equally strong in evaluation design, prompt and context engineering, and program-level instrumentation, and have shipped agentic systems where you could point to numbers that demonstrated they were working (or not).
• You write production-grade Python and/or TypeScript and have built developer-facing tooling (CLIs, SDKs, internal platforms) that other engineers chose to adopt because it made their work easier, and you have rolled tools out across teams, not just shipped them.
• You hold strong, well-defended opinions about how agentic systems scale across large engineering organizations, and you have either built that experience firsthand or studied closely how leading companies have done it.
• You have experience working in early-stage, ambiguous environments where you define the problem as much as you solve it, and you know when to prototype quickly versus when to build for the long term.

Nice to Have

• Experience in media, streaming, or gaming
• Experience working in an innovation lab, incubation environment, or R&D organization
• Experience with Datadog, LangSmith, Langfuse, or comparable LLM observability tooling

About the Team

The Concepts Lab is Crunchyroll’s R&D and incubation group, focused on exploring new ideas, technologies, and experiences for anime fans. The team partners closely with engineering, product, and business leaders to test assumptions, build evidence, and transition validated concepts into delivery teams.

Our work spans early discovery through beta execution, with an emphasis on learning quickly, reducing risk, and setting teams up for long-term success.