Hiring: Staff AI Product Analyst Company: Angi Location: Denver, CO Job Posted Time: 2026-09-13 17:57:13 Employment Type: Full-time Target Skills & Keywords: AI Product Analytics, LLM Systems, Agentic AI, Evaluation Frameworks, Golden Datasets, RAG, Multi-Agent Systems, Python, ML Frameworks, Data Governance, Experimentation, Causal Inference, Metrics Design, Data Visualization, Product Strategy Experience: - 10+ years of experience across product analytics, data science, or technical product management within a complex, high-scale operating environment - 1+ years of direct, hands-on experience working with AI/LLM systems in an applied context, with a specific focus on evaluating, trusting, and deploying them responsibly Required Skills: - Deep technical proficiency with Python, ML frameworks, data visualization/analytics platforms, and standard data management/governance practices - Proven track record designing AI evaluation frameworks, including formal evals, golden datasets, production monitoring, or equivalent rigor in a related data discipline - Exceptional communication skills with a demonstrated ability to translate highly technical AI insights and tradeoffs into clear narratives for non-technical business stakeholders - Define north-star and guardrail metrics for flagship AI journeys (adoption, task completion, quality, downstream business outcomes) - Design and maintain formal evaluation sets, golden datasets, rubrics, and regression checks for agent outputs - Taxonomy and failure mode analysis (wrong answers, tool misuse, context gaps, data edge cases) - Monitor live usage of agentic AI analytics workflows (success, abandonment, fallback patterns, human override rates) - Map how agents interact with metrics, raw tables, documentation, and APIs Qualifications: - Bachelor's in Computer Science or a quantitative field (e.g., Computer Science, Statistics, Mathematics, or related fields) - Preferred: Experience building or evaluating agentic analytics workflows, orchestration frameworks, and familiarity with multi-agent systems - Preferred: Demonstrated ability to operate fluidly in ambiguity, run structured experiments or pilots, and default to fast action and iteration - Preferred: A systems-oriented mindset with a healthy skepticism of AI hype, focusing strictly on organizational outcomes and driving technical investments that yield true business value Compensation: - Salary band: $165,000 - $225,000 commensurate with experience and performance (may vary based on cost of living) - Eligible for a competitive year-end performance bonus & equity package - Full medical, dental, vision package - Flexible vacation policy - Pet discount plans & retirement plan with company match (401K) Interested candidates, please apply directly through the job posting on company's career page or try via AI auto apply on this platform. Don’t miss this opportunity to join a forward-thinking team!