Full TimeRemote
Engineering Manager – Evaluation & Observability
Lead the team building AI evaluation and observability systems that detect drift, policy violations and prompt injection across enterprise AI.
Apply For This Job
Team
AI Governance Platform – Evaluation & Observability
Salary
USD 170,000 – 190,000
Posted 25 March 2026
Apply by 31 July 2026
Required Skills
AI Governance
Langchain
Google Vertex AI
About the Role
You will build and lead the evaluation, monitoring, and observability layer of our AI governance platform — core to trust, compliance, and auditability.
What You'll Do
•Build systems for model evaluation (accuracy, bias, drift, hallucination detection)
•Build agent monitoring (inputs/outputs, decisions, tool usage)
•Build policy enforcement + audit logs
•Design observability pipelines (real-time + batch)
•Define evaluation frameworks aligned with NIST AI Risk Management Framework
•Work on explainability, traceability, and audit artifacts
•Lead a team of engineers (back-end + data + ML)
What We're Looking For
•8–12 years engineering experience, 3+ years managing teams
•Strong back-end/data systems experience (event pipelines, logging, distributed systems)
•Familiarity with LLM/agent evaluation techniques
•Experience building observability platforms
Nice to Have
•Experience with AI governance / compliance systems
•Knowledge of EU AI Act technical requirements