How Atlassian continuously improves Rovo Search quality

The days of walking the halls looking for a subject matter expert or waiting for a clarifying email are over. To stay competitive, modern and globally distributed teams need to access and translate actionable knowledge into progress as fast as possible.

Atlassian’s solution for instant knowledge is the AI-powered Rovo Search, which is tailored to work across the enterprise using Atlassian’s proprietary Teamwork Graph.

By using Rovo Search, teams have access to:

• Contextually smart answers, insights, and links to their sources.
• Related chat prompts so there’s always a suggested next step.
• 50+ connected apps – find information across the Atlassian portfolio and third-party tools like Google Drive, SharePoint, OneDrive, and more.
• Enterprise-grade trust – Rovo combines end-to-end encryption, compliance standards, robust admin and permissions controls, and a zero data retention policy.

In this blog, we’ll share how we define and measure search quality for Rovo Search as well as the latest advancements in AI-powered search that you can now experience.

Our process of elevating search quality

We are always identifying opportunities, running experiments, and launching model improvements to ensure Rovo Search offers the best possible results in real time. Recently, in an A/B test, Rovo Search gave searchers 60% more successful results than the next leading enterprise search tool.

Since January 2025, we have already launched several ranking and infrastructure upgrades that have improved our primary quality metric (by +10%) and reduced latency while loading search results.

How Atlassian measures search quality in Rovo

We established a primary online search quality metric to evaluate the extent to which user needs are fulfilled by Rovo. This metric is a composite that reflects positive user engagement with essential components of the search results page, such as:

• User consumes a search result: To ensure the user clicks on a link and finds the landing page useful, we ensure that after the click, the user spends a sufficiently long duration away from Rovo Search or performs post-click actions (comments, edits, etc.). This is also referred to as the user making a long click. We want to ensure the user is finding results at the top of the page, so we gradually discount when long clicks happen towards the bottom.
• User interaction with AI-generated answers: With AI-generated answers, users can directly get the answer to the query without needing to engage with the AI answer, click through to cited sources, or potentially give explicit feedback. To determine success, we explore the level of answer engagement, dwell time, and post-search actions.

In addition to these composite metrics, we closely track a suite of metrics that cover various aspects of the search page, including clicks, long clicks, click depth, filter usage, and more.

Rovo’s latest improvements to search quality

Atlassian’s enterprise search capabilities are ever-evolving. For example, here are the four latest advancements we just launched in Rovo Search:

• Improved search quality with the incorporation of recently viewed documents of users and their collaborators, signals derived from sharing and commenting activities, and many other ranking model enhancements.
• Increased coverage of AI answers on the search results page and improved source material citations in the AI answers.
• Reduced P90 Latency by 300ms through a language model and serving infrastructure upgrade.
• Improved recall for Google Drive by including viewed content that you have access to, maintaining consistent searchability with permissions.

Continuous improvement means being transparent

By sharing how we measure and improve Rovo, we’re opening the door for feedback, trust, and collaboration. We will continue to identify opportunities for improvement in our ranking and user experience, striving to facilitate seamless knowledge search across diverse data sources, so your teams can work smarter and move faster.