Location: Hybrid Preferred (Minneapolis, MN or Denver, CO) | Remote Considered
Employment Type: Full-time
Are you passionate about ensuring AI solutions continue delivering meaningful business value long after deployment? Trissential is seeking an AI Operations Specialist to join our client's team and support the operational success of enterprise AI agents, virtual assistants, and generative AI solutions in production.
In this role, you will monitor AI performance, quality, adoption, governance, and business outcomes while partnering with Product, AI Engineering, Operations, and business stakeholders. You will serve as the operational owner of AI solutions in production, ensuring they remain accurate, compliant, effective, and aligned with business objectives.
What's in It for You?
Be the operational owner of cutting-edge AI solutions used across a large enterprise environment
Drive measurable improvements in AI quality, adoption, customer experience, and business value
Collaborate with Product, AI Engineering, Operations, Security, Governance, and business subject matter experts
Gain exposure to Responsible AI practices, enterprise governance programs, and AI observability platforms
Work in a highly visible role where your insights directly influence AI strategy and operational outcomes
Help define long-term AI operational standards and best practices across the organization
Opportunity to contribute to enterprise-wide AI transformation initiatives
Your Role & Responsibilities
Own the operational performance of AI-powered solutions after implementation and hypercare.
Monitor AI quality, response accuracy, relevance, compliance, adoption, and business outcomes
Track and report on key AI performance metrics, including response quality, customer satisfaction, adoption, escalation rates, and overall solution effectiveness
Review transcripts and AI conversations to evaluate accuracy, compliance, and overall response quality
Identify, troubleshoot, and escalate AI performance issues, including inaccurate responses, knowledge gaps, and workflow failures
Monitor and improve AI knowledge sources by identifying content gaps, validating information accuracy, and ensuring content remains current
Support AI governance and compliance activities to ensure solutions meet Responsible AI, privacy, and organizational standards
Coordinate incident investigations and identify root causes across business and technical teams
Translate operational findings into prioritized backlog items for Product Owners and AI Engineering teams
Measure and report value realization, including productivity gains, self-service adoption, customer outcomes, utilization metrics, and ROI
Develop operational procedures, governance standards, and knowledge-transfer documentation to support long-term sustainability
Drive continuous improvement initiatives that enhance AI performance, reliability, and business impact
Own the operational performance of AI-powered solutions after implementation and hypercare
Monitor AI quality, response accuracy, relevance, compliance, adoption, and business outcomes
Develop and maintain operational scorecards covering containment, satisfaction, escalation rates, retrieval effectiveness, response quality, latency, and utilization
Review transcripts and AI conversations to evaluate accuracy, grounding, compliance, and overall response quality
Investigate hallucinations, retrieval failures, misrouted interactions, prompt issues, knowledge gaps, and workflow breakdowns
Manage and optimize Knowledge Hub performance through content gap analysis, search quality evaluations, source validation, and freshness monitoring
Support Responsible AI monitoring activities including privacy validation, governance reporting, audit readiness, bias reviews, and guardrail compliance
Coordinate incident investigations and identify root causes across business and technical teams
Translate operational findings into prioritized backlog items for Product Owners and AI Engineering teams
Measure and report value realization, including productivity gains, self-service adoption, customer outcomes, utilization metrics, and ROI
Develop operational procedures, governance standards, and knowledge-transfer documentation to support long-term sustainability
Skills & Experience You Should Possess
5+ years of experience in technology operations, analytics, product operations, quality engineering, knowledge management, conversational AI, or a related field
Experience supporting AI, conversational AI, digital assistant, chatbot, knowledge management, search, or automation solutions in a production environment
Experience evaluating solution quality through transcript reviews, response evaluations, quality assessments, or operational performance reviews
Strong operational analytics experience with dashboards, KPIs, performance reporting, and trend analysis
Knowledge of search effectiveness, knowledge management, content validation, and content-gap analysis
Experience investigating production incidents and driving root-cause analysis efforts
Understanding of Responsible AI principles, governance frameworks, privacy considerations, and compliance monitoring
Excellent communication skills with the ability to explain operational issues and business impacts to diverse stakeholders
Strong documentation, organizational, and continuous-improvement capabilities
Bonus Points If You Have
Experience supporting AI solutions within customer service, customer operations, or contact center environments
Hands-on experience with Microsoft Copilot Studio, Salesforce Agentforce, enterprise chatbot platforms, or related AI ecosystems
Experience with AI observability tools, conversation analytics platforms, and evaluation frameworks
Familiarity with Jira or similar work-management platforms
Background working within regulated, utility, audit-sensitive, healthcare, financial services, or privacy-focused environments
Experience supporting enterprise search and retrieval systems
Education & Certifications You Need
Bachelor's degree in Information Technology, Computer Science, Business Analytics, Data Analytics, Engineering, or a related discipline, or equivalent practical experience
Relevant certifications in Microsoft technologies, cloud platforms, AI, IT Service Management, Product Operations, Analytics, or Data Management are beneficial
Certifications are valued but do not replace hands-on production operations experience
What We Offer
At Trissential, we believe talented professionals should have meaningful opportunities to create measurable impact while continuing to grow their careers. By joining our client's team, you'll help shape the future of enterprise AI operations while working in a collaborative and supportive environment.
- Competitive Compensation – $90-$100 per hour. Final compensation is determined based on skill alignment, years of experience, and fair, market-based rates by geography.
- Comprehensive Benefits for you and your dependents – Extensive Medical Plans, Dental, Vision, HSA with company contribution, company sponsored Life Insurance and Short / Long term disability insurance, a host of voluntary plans including legal, hospital, accident, pet wellness, and 401k with matching
- Paid Holidays and Paid Time Off
- Career Development and Advancement – Expleo is a global company in 29 Countries worldwide with a high focus on development
- Strong Team Culture – Collaborate with experienced professionals who value precision, performance, and teamwork
Please note: This position is open only to candidates authorized to work in the United States.
Ready to be part of something exciting?
If you're passionate about operational excellence, AI quality, governance, and ensuring AI delivers measurable business value, we encourage you to apply and help shape the future of enterprise AI operations with Trissential and our client.
