AI Evaluation Engineer - Business & Operations Domain
Posted 20 days ago
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are looking for highly analytical business and operations professionals to contribute to advanced AI evaluation projects focused on realistic operational workflows, business processes, and multi-step decision-making scenarios. In this role, you will help design and evaluate complex tasks that test how effectively AI systems handle real-world operational challenges across business environments.
The ideal candidate has hands-on experience working with business operations, analytics, workflow automation, reporting, CRM systems, financial processes, or cross-functional operational workflows. You will contribute realistic scenarios involving process optimization, business reasoning, operational failures, reporting workflows, and structured decision-making.
This role is particularly well suited for professionals with backgrounds in business operations, management consulting, business analytics, finance operations, CRM and sales operations, supply chain and logistics, operational excellence, process automation, reporting and BI, strategy consulting, or workflow optimization.
CONTRACT: Contractor assignment (5 weeks)
COMMITMENT: Full-time (40h/week) or Part-time (20h/week) with minimum 4h PST overlap
LOCATION: Remote — Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Kenya, Nigeria, Turkey, Vietnam
PROCESS: One technical assessment/interview (~45 min)
Responsibilities:
- Design realistic business and operational workflow scenarios for AI evaluation systems
- Create structured tasks involving analytics, reporting, operational reasoning, and process optimization
- Develop clear task specifications, expected outcomes, and validation logic
- Identify operational edge cases, bottlenecks, and workflow failure scenarios
- Evaluate AI-generated outputs for reasoning quality, usefulness, and accuracy
- Contribute expertise across business operations, analytics, automation, or operational systems
- Review and improve workflow complexity, clarity, and evaluation quality
- Collaborate with reviewers and researchers to refine AI benchmark scenarios
- Help create realistic multi-step business and operational problem-solving tasks
Requirements
- 3–10 years of experience in operations, analytics, consulting, business systems, or related domains
- Strong analytical thinking and operational problem-solving skills
- Experience working with operational workflows, reporting systems, CRM tools, or business analytics
- Good understanding of cross-functional business processes and dependencies
- Experience with spreadsheets, dashboards, operational reporting, or workflow automation
- Strong written communication and documentation skills
- Exposure to AI systems, automation platforms, or evaluation workflows is preferred
- Ability to design realistic and structured operational scenarios for evaluation purposes
Other open roles at Gramian Consulting Group(6)
Gramian Consulting is your partner for accessing the engineering capabilities you need—delivered in the model that fits your business, from staff augmentation and talent recruiting to Build-Operate-Transfer (BOT). We combine the perspective of a software engineer, the rigor of a technical recruiter, and the vision of a business builder, so you get experts who understand your challenges and deliver results the right way. This blend is our signature advantage in providing top-quality services, fast and reliably.
Key team members
Emmanuel Yawson
Pauline Perry
Emad Hassan
Aleksandra Šarac
Jobr aggregates jobs directly from company career portals — no middlemen. Our team applies on your behalf with AI-tailored resumes, reviewed by a human before submission.