Overview
We are looking for a Senior AI Engineer to design and build production grade multi agent AI systems in a multi tenant SaaS environment. You will take ownership of AI solutions across the full lifecycle, from architecture and implementation to evaluation, deployment, and continuous optimization, ensuring reliability, scalability, and high quality outputs. This role requires a senior hands-on builder with deep experience shipping production LLM systems to real users, not only prototypes or proof-of-concept solutions.
Project Overview:
You will work on building production grade multi agent AI systems within a multi tenant SaaS environment. The project focuses on designing advanced agent orchestration patterns, enabling reliable AI workflows, and delivering scalable solutions used by real customers.
- Design and build production grade multi agent AI systems for real world usage
- Develop end to end AI solutions from requirements definition through architecture design and deployment
- Implement agent orchestration using LangGraph or LangChain
- Design supervisor and specialist routing, agent to agent communication, and human in the loop workflows
- Design and integrate domain-specific MCP servers and external tools into agent workflows
- Build and maintain evaluation frameworks including offline and online quality monitoring
- Monitor system performance and optimize reliability, latency, quality, safety, and operational costs
- Implement observability pipelines including tracing, prompt and output logging, and evaluation tracking
- Collaborate with Product Managers and engineering teams to refine requirements and deliver production ready solutions
- Independently propose architectures, evaluate trade offs, and drive implementation with limited supervision
- Proven experience building and operating production LLM systems used by real customers
- Ability to clearly describe a specific production LLM or agentic system personally built and shipped
- Experience designing AI systems end to end from architecture to deployment
- Hands on experience with multi agent architectures and orchestration
- Experience with LangGraph or LangChain
- Experience implementing multi agent orchestration and agent to agent communication
- Experience with tool calling and MCP integration, including domain-specific MCP servers
- Experience building human in the loop workflows
- Experience designing evaluation frameworks including golden datasets, offline evaluations, online monitoring, and LLM as a judge
- Experience measuring regressions and making engineering decisions based on evaluation results
- Experience implementing AI observability including tracing, logging, evaluation pipelines, and latency monitoring as well as guardrail monitoring
- Experience working with production AI systems in AWS environments with a focus on reliability, scalability, and cost optimization
- Ability to independently drive technical decisions and deliver solutions with minimal supervision
- English working proficiency
- Experience with Amazon Bedrock AgentCore
- Experience with OpenAI APIs
- Familiarity with Arize, LangSmith, or Braintrust
- Experience with A2A communication protocols
- Experience working with retrieval systems
- Experience building conversational AI systems
✨ Our intelligent job search engine discovered this job and republished it for your convenience.
Please be aware that the job information may be incorrect or incomplete. The job announcement remains the property of its original publisher. To view the original job and its full details, please visit the job's URL on the owner’s page.
Please clearly mention that you have heard of this job opportunity on https://ijob.am.
