LLM Engineers

Job Overview:

Master Works is looking for a highly skilled LLM Engineer to join our AI Core Delivery team within the AI & Analytics CoE. In this role, you will develop, deploy, and optimize advanced language model capabilities, including Retrieval-Augmented Generation (RAG) pipelines and agentic AI systems. You will play a key role in scaling AI Core as the central enterprise AI platform by ensuring production-ready performance, robust observability, and seamless integration of LLMs with internal and external systems.

Key Responsibilities

• Model Development & Optimization
• Build and optimize RAG pipelines for efficient document processing, indexing, and retrieval.
• Implement agentic AI logic for tool integration, task automation, and external data access.
• Enhance query understanding, response ranking, and AI-generated output quality.
• Develop evaluation benchmarks to measure LLM performance, accuracy, and relevance.
• LLMOps & Deployment
• Collaborate with API developers and backend engineers to deliver secure, observable LLM endpoints.
• Deploy and manage LLMs in cloud-native or hybrid environments with support for scalability and multi-cloud readiness.
• Implement CI/CD workflows for model updates, rollback mechanisms, and performance tuning.
• Backend & Data Integration
• Integrate LLM capabilities with enterprise data sources and knowledge bases.
• Support multi-source retrieval and embedding strategies to improve AI responses.
• Design pipelines that ensure low latency, high throughput, and fault tolerance.
• Security & Compliance
• Apply enterprise-grade authentication, access management, and encryption standards.
• Ensure compliance with internal governance policies, data privacy, and audit requirements.
• Continuous Improvement
• Monitor system performance, debug issues, and fine-tune models based on real-world feedback.
• Stay updated on advancements in LLM architectures, open-source tools, and evaluation frameworks.

Requirements

• Bachelor’s or Master’s degree in Computer Science, AI, Data Science, or related field.
• Hands-on experience with LLM development and deployment (OpenAI, Anthropic, Hugging Face, LLaMA, etc.).
• Strong understanding of Retrieval-Augmented Generation (RAG), embeddings, and vector databases.
• Experience with cloud environments (AWS, Azure, GCP) and infrastructure-as-code (IaC).
• Proficiency with Python or similar languages for AI/ML pipeline development.
• Familiarity with container orchestration (Docker, Kubernetes) and CI/CD practices.
• Knowledge of observability tools for monitoring AI performance in production environments.

Preferred Skills

• Experience integrating agentic AI capabilities with external APIs and automation tools.
• Strong grasp of prompt engineering, contextual memory systems, and LLM evaluation.
• Exposure to enterprise AI security and compliance standards.