All posts

An AI Dashboard That Support and Execs Can Actually Use

IntelliChat’s AI dashboard shows how the assistant is being used, where answers break down, and what support and training need next. It turns chat logs into decision-grade signals without guesswork.

Originally posted on YouTube on August 9, 2023

Key takeaways

  • Time-based slicing (day/week/month/year) turns AI usage into trend data instead of anecdotes.
  • Frequency and sentiment help identify repeat friction points and rising issues before they become ticket floods.
  • Drilling into individual queries and responses connects high-level metrics to concrete operational and product changes.

Support teams already have plenty of data. Tickets, macros, CSAT, call recordings, onboarding checklists. The problem is that most of it doesn’t connect cleanly to what operators need to know: What are people actually asking? Are we getting better over time? And where are the bottlenecks that keep repeating?

IntelliChat’s AI dashboard was built to answer those questions with as little ceremony as possible.

“It provides a lot of insights to a support manager, support staff, or an executive… [to] understand how the AI is being utilized… [and] where the bottlenecks and hiccups within your support and training may exist.” — James Moelk

Start with time, not anecdotes

Most “AI reporting” turns into a screenshot war: someone remembers a bad answer, someone else remembers a good one, and nothing changes.

The dashboard forces a useful first step: choose the time window. Day, week, month, or year. Then set a specific date range.

That sounds basic, but it’s operationally important. Support leaders need to see whether changes correlate with real events: a release, a policy change, a new cohort of hires, a seasonal spike, or an outage. Without a consistent timeframe, you’re just paging through chat history.

Exportable data for the people who live in spreadsheets

Not every organization wants to do analysis inside a vendor UI. Sometimes you need to join AI usage with ticket volume, product telemetry, or staffing schedules.

The dashboard supports CSV export so you can manipulate the data in whatever way is meaningful internally. That includes:

  • Building your own weekly reporting packs
  • Comparing AI queries to ticket tags
  • Rolling AI usage into broader ops metrics

This is also how you keep stakeholders aligned: the same underlying dataset can serve support managers, support ops, training leads, and execs.

Frequency, sentiment, and change over time

Counting queries is table stakes. The more valuable layer is understanding the shape of the demand.

The dashboard looks at:

  • Frequency of questions: what’s being asked repeatedly (and therefore what should be documented, fixed, or trained)
  • Sentiment of questions: where frustration is clustering, even if the topic looks “normal” on paper
  • Time-based shifts: whether something is trending up or down over the selected period

That last point is where teams find early warning signals. If a previously quiet topic starts accelerating over a week, it’s usually not random. It’s a release gap, a confusing UX change, a broken workflow, or an onboarding miss.

Drill into the actual conversations

Aggregates help prioritize. But operators still need receipts.

Alongside top-level metrics like the number of queries (tied to the selected timeframe), you can drill into individual interactions: the exact question and the response the AI gave.

“You can further dig into each query and review what the query was specifically and the response.” — James Moelk

This is the part that makes the dashboard usable across roles:

  • Support managers can spot repeat failure modes and update playbooks.
  • Support staff can see where the assistant helps (and where it doesn’t) without guessing.
  • Product and engineering can connect user confusion directly to the language users use.
  • Training leads can turn real questions into onboarding scenarios.

What this means for you

If you’re running support or training, the goal isn’t “more AI.” It’s fewer blind spots. A dashboard that ties time windows, frequency, sentiment, and real transcripts together gives you a practical loop: observe what’s happening, validate it in the underlying queries, and make a change that sticks.

Full transcript
Hello, I'm James with IntelliChat. I wanna quickly run through the AI dashboard that we created because it provides a lot of insights to a support manager, support staff, or an executive who understand how the AI is being utilized, but also in part to really understand, you know, where the bottlenecks and hiccups within your support and training may exist. So if we look here, you can see that we can select metrics in the terms of the day, the week, the month, or the year. We can select, you know, the specific date range. And within that timeframe that's selected, we can, of course, download a CSV to manipulate the data in a way that's specifically meaningful for us. But we also utilize the AI to dig back in and ask about the frequency of questions, the sentiment of questions, and something to do with the timeframe, you know, so it may, you know, recognize that things are changing, you know, in the last week or so as a result of whatever. But, you know, internally, that may mean something to you. We have the number of queries, and again, this is going to relate to the timeframe that's selected. And then you can further dig into each query and review what the query was specifically and the response. So all this information ultimately ends up being super powerful. It allows you to make decisions, not just from a support standpoint, but from a development standpoint to understand how you may be able to improve on the user experience in a way that's going to be helpful for the end user in the future. But that's it. This is the AI dashboard.