In this episode, we sit down with Rohan Katyal and Raghav Sethi, co-founders of Milana, to discuss the shift from passive analytics to the world’s first AI Product Engineer. Rather than just providing another dashboard to monitor, Rohan and Raghav are building an agentic partner that you add to your product to bridge the gap between discovery and deployment. Drawing on their experience at Meta, Yelp, and Airtable, they explore how Milana enables autonomous improvement - turning deep user intelligence into shippable code and structural refinements that act as a tireless extension of your engineering team.
The conversation dives into why session replays — a mature but historically underused technology — are now a powerful data asset thanks to vision LLMs. Raghav explains how session replays are really just high-granularity logging of DOM changes, not screen recordings, and why feeding them through AI unlocks insights that traditional event-based analytics simply can’t capture. The team breaks down how they use just-in-time structuring to extract meaning from dense, unstructured session data without requiring upfront instrumentation.
Rohan shares hard-won lessons from building Yelp’s experimentation platform — including how teams that simply ran more experiments consistently outperformed those with better data resources. They discuss the tension between A/B testing rigor and iteration speed, why most experiments never ship, and how lowering the cost of generating and testing hypotheses changes everything about product development velocity.
We also get into the technical details of semantic clustering across millions of sessions, why video is actually a more compact representation than raw DOM for LLM reasoning, and how Milana analyzes sessions from multiple perspectives — user researcher, PM, founder — to surface real pain points. Plus, a bold prediction: analytics dashboards are dying, and the future belongs to agentic systems that don’t just deliver insights but actually own and drive your OKRs.
Topics covered:
- Why session replays are the ultimate untapped data asset for product teams
- How vision LLMs unlocked AI-powered analysis of user sessions
- Just-in-time data structuring: querying unstructured sessions without upfront instrumentation
- Lessons from building experimentation platforms at Yelp and Airtable
- Why running more experiments beats having better data
- Semantic clustering: separating signal from noise across millions of sessions
- Video vs. DOM vs. events — the best data representation for LLM reasoning
- Analyzing agent behavior through session replays
- The death of dashboards and the rise of agentic growth systems
- User research horror stories and the surprising things users do
Chapters00:00 Introduction to Rohan and Raghav's Journey
04:47 The Importance of User Research
08:03 Making Solutioning a Science
11:09 Understanding Session Replays and Experimentation
14:50 Defining Sessions and Experimentation Platforms
18:54 The Need for Consistent Metrics
22:11 The Role of Events vs. Session Replays
29:46 Leveraging LLMs for Enhanced Insights
35:04 Determinism vs. Non-Determinism in Data Analysis
37:57 Understanding User vs. Agent Behavior
39:47 The Art of Structuring Data
45:25 Semantic Clustering and Its Importance
47:09 Building Infrastructure for Complex Data
51:24 The Future of User Simulation and Experimentation
