When Quora launched Poe.com, the goal was simple: to bring together the best AI models from leading AI companies in a single interface across all devices. Today, Poe has grown into a unified hub where anyone can access more than 100 of the best text, image, video, and audio models, along with millions of bots built on top of them, all in one place. Achieving this hinges on three essentials: reliability, responsiveness, and seamless interactions.
With multiple production deployments a day, the team needed to continuously validate performance, stability, and user experience across their platform. With much of the interactions happening across an extremely dynamic chat interface, they needed a quality solution that can adapt intelligently.
Manual testing couldn’t keep up
Quora’s Product Operations team is responsible for a vast variety of things, including but not limited to product support, product operations, internationalization, and QA.
QA was mostly manual, with almost 500 test cases documented in a Google Sheets spreadsheet. These were key to ensure releases from the engineering team had no regressions.
“It wasn’t sustainable - we needed a faster, more reliable way to keep up with releases.” - Head of QA
Executing critical test cases daily consumed up to 7 hours. With these tests running every weekday, the time investment became substantial. It was time for a change.
The search for the right AI QA solution
What followed was a thorough RFP process, where the Product Operations team evaluated a range of agentic QA solutions based on the following key criteria:
A truly native generative AI solution - Quora needed a modern solution that could simplify testing while handling the dynamic nature of Poe.com’s AI.
Quality of the tool & UI - The AI agent needed to feel intuitive for Quora’s internal teams.
Depth of integration - The tool had to integrate seamlessly with Quora’s existing test case management system to ensure context about bugs never gets lost. After evaluation, Momentic was selected for the project.
Expanding coverage with minimum engineering lift
Quora had a clear goal: give the Product Operations team the ability to build and automate end-to-end tests of critical user flows in Poe.com.
Momentic’s natural language test editor and multi-modal assertions meant the team could describe expected behavior in plain English, critical for a team that was deeply embedded in workflows but never trained in Playwright or Selenium code.
“I really appreciate this AI feature in one of our QA test automation tools - it instantly gives me feedback on my prompt and helps me craft the best instruction on-the-go.” - Senior Program Manager
By the end of the month, the team had automated all critical tests in Momentic with minimal support from the engineering team. As Poe.com continues to grow and evolve, Momentic has played a helpful role in their process, contributing to smoother deployments.
A 14x speed up
Over the next three months, the Product Operations team automated hundreds of critical test cases. Today, the daily test executions take a mere 30 minutes instead of 7 hours. This 14x speedup not only saved time but also streamlined processes and made updates smoother, reducing dependence on manual steps.
