Skip to main content

Documentation Index

Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

Topics automatically analyze and classify your logs to surface the failures, edge cases, and recurring problems hiding across your traces. Use Topics for:
  • Blind-spot detection: Surface user request patterns and system gaps you didn’t know to look for.
  • Silent failure detection: Catch quality issues that don’t trip explicit checks.
  • Product roadmap signals: Cluster real user requests into themes to inform what to build.
  • Targeted evaluation datasets: Filter classified logs to build datasets for focused evals.
Beta — This feature is subject to change. For self-hosted deployments, Topics is early access with additional eligibility requirements. See the v2.x upgrade guide for details.

​
Enable Topics

Enabling Topics activates three built-in facets that automatically classify your logs:
  • Task: Extracts the user’s intent or goal (e.g., “Creating a dataset,” “Debugging an API error”).
  • Sentiment: Extracts the user’s emotional tone (e.g., “POSITIVE,” “FRUSTRATED,” “NEUTRAL”).
  • Issues: Identifies problems with agent behavior or responses (e.g., “Tool call failed,” “Incomplete answer”).
To enable Topics in a project:
  1. Go to Topics.
  2. Choose whether to Apply to existing traces or only to new traces.
  3. Click Enable topics.
Each facet extracts a short summary from each trace and stores it in the background. Once at least 100 summaries are collected, the daily pipeline clusters them into topics that classify your logs. See Check automation status to track progress, How it works below for pipeline details, or Common issues if results don’t appear.
Topics quality depends on how your traces are structured. The LLM that extracts facet summaries works best when traces capture conversation or session structure with nested spans. If your traces are flat, review your instrumentation before enabling, or plan to write a custom preprocessor to adapt.

​
How it works

Topics runs a daily pipeline on your logs:
  1. Preprocessing — Each trace is formatted into readable text. Messages, tool calls, and nested spans become a narrative.
  2. Facets — For each facet (Task, Sentiment, Issues), an LLM analyzes the preprocessed trace and extracts a short summary describing the trace through that lens.
  3. Topics — Once at least 100 facet summaries are collected, a clustering algorithm groups similar summaries into topics. For example, “User wants a refund,” “Requesting a chargeback,” and “Asking for money back” might all become the topic “Refund requests.”
  4. Classification — For each facet, the trace is matched to its closest topic. These classifications appear in your logs table, where you can filter, query with SQL, and build evaluation datasets.
The pipeline runs on a set cadence:
  • Initially: Existing logs are optionally backfilled with facet summaries.
  • Continuously: New logs are processed as they arrive.
  • Daily: Topics are regenerated from collected facet summaries. Generation requires at least 100 summaries.

​
Models and data handling

Topics runs three model calls during the daily pipeline. All three are served by Braintrust, not by third-party LLM providers:
  • Facet summarization: An LLM reads each preprocessed trace and writes a short summary through the lens of each facet (Task, Sentiment, Issues, or your custom facet). Choose the model in the automation’s Advanced settings.
  • Embedding: An embedding model turns each summary into a vector so similar summaries can be grouped together.
  • Cluster naming: An LLM reads a sample of summaries from each cluster and picks a short, human-readable name like “Refund requests.”
The models used are brain-facet-* for facet summarization, brain-embedding-1 for embedding, and brain-agent-1 (based on Kimi K2.5) for cluster naming. Braintrust serves them on Baseten, which is included in the Braintrust DPA as a subprocessor. Self-hosted deployments call the same Braintrust-hosted endpoints with Zero Data Retention. See Enable Topics for self-hosting requirements. Topics requires built-in models. If your organization has disabled built-in models, topic generation will fail.

​
Next steps

  • Review insights by viewing topic distributions, examining traces, clustering filtered subsets, and tracking trends.
  • Act on findings by building evaluation datasets, scoring logs by classification, and assigning topics for human review.
  • Manage Topics by checking pipeline status, re-generating topics, adjusting sampling, and rewinding history.
  • Create custom facets for domain-specific patterns beyond the built-in Task, Sentiment, and Issues.