About
What We Do
This platform tracks and analyzes brand mentions across large language model (LLM) responses. We monitor how different AI systems recommend and reference various software products, tools, and services when answering user queries.
Measurement Methodology
Data Collection
We systematically query LLMs such as ChatGPT with Bottom-of-Funnel queries across various software categories (e.g., CRM, Accounting, Project Management). These queries are designed to elicit recommendations and comparisons of different products and services.
Queries are collected via the ChatGPT interface rather than the API to better simulate real user interactions.
Brand Extraction
From each LLM response, we extract and catalog all mentioned brands, tracking:
- Total number of times each brand is mentioned
- How often each brand appears first in responses
- The percentage of queries where each brand appears (coverage)
- Temporal trends in brand mentions over time
Metrics Explained
- Total Mentions: The number of times a brand appears in LLM responses within a specific category
- First Mentions: How many times a brand is mentioned first when multiple brands are listed (often indicates higher relevance or preference)
- Coverage %: The percentage of category queries where the brand appears, indicating how consistently the brand is recommended
- Share of Mentions:The brand's proportion of total mentions relative to competitors in the same category
Data Updates
Data is updated on a monthly basis to capture evolving trends in how LLMs recommend different products and services. Historical data allows us to track changes in brand visibility over time, as well as detect emerging brands gaining traction.
LLM Visibility Score
The LLM Visibility Score is a composite metric designed to quantify brand dominance in Large Language Model (LLM) responses by considering a number of key factors.
Our LLM Visibility Score synthesizes three pillars of performance into a single score out of 100. Within each category, for a given month, each brand assigned a visibility score as follows:
We consider three factors when scoring brand visibility:
1. Presence (Frequency)
- Weight: ~43% of total score
- Metric: Percentage of responses where the brand appears.
- Rationale: This is the foundation of share-of-voice. A brand appearing in 100% of relevant prompts receives the maximum base points for this pillar.
2. Prominence (Rank)
- Weight: ~43% of total score
- Metric: Average position normalized against a baseline value of 9, which we found to be the average number of brands mentioned in a given LLM response after analyzing over 100,000 LLM responses in over 700 categories.
- Rationale: Users rarely read beyond the initial list provided by an AI. Rank #1 contributes maximum value, while Rank #10 or lower contributes zero value, reflecting the sharp drop-off in user attention "below the fold."
3. Primacy (First Mention Bonus)
- Weight: ~14% of total score
- Metric: Ratio of times the brand was listed first.
- Rationale: In conversational AI, the first entity mentioned anchors the user's perception. We weight this at 0.33—significant enough to reward market leaders who define the category, but controlled to ensure a single lucky mention doesn't overpower consistent presence across several responses.
Limitations & Considerations
Important notes about our data:
- Mention frequency does not necessarily indicate product quality or suitability
- LLM responses can vary based on query phrasing and context
- Brand mentions reflect training data and may not represent current market conditions
- Our data captures what LLMs say, not necessarily what users should choose