Dialog — Data & AI audit

Building a shared view of Dialog’s data and AI foundations in a matter of days, then turning it into recommendations and a roadmap tied directly to their product ambitions.

Sector
E-commerce · AI
Engagement
Technology audit
Timeline
Summer 2025
Expertise
Data & analytics systems · Applied AI & agents

Context

Dialog builds AI agents for e-commerce companies. By the time we came in, the team already had a rich data and AI environment, fed by several families of data: user interactions, application data, conversations and model responses, agent workflow traces, analytics data, and the needs of their machine learning work.

The point of the engagement was therefore not to start from scratch, but to take a step back from an already advanced foundation, formalise a view of the whole, and prepare what came next.

Two product ambitions shaped the thinking in particular: enriching the analytics capabilities offered to Dialog clients, and preparing for more autonomy in how conversational experiences are configured and optimised.

The engagement started with a full day in Dialog’s offices, to understand as much as possible of the product, the uses, the systems, the technologies and what the teams expected.

Our approach

We deliberately started from the context and the uses before making any technical recommendation.

The goal was to understand how data flows through the Dialog ecosystem today, how it is used by the product, by the data and ML teams and by the AI systems, and only then to identify which changes would best support the next phase.

Immersion
Mapping
Diagnosis
Options
Prioritisation

From observed uses to a decided trajectory

The principle we held to throughout the audit: change what has to change, without calling into question what already works.

Each stage had its material. The mapping covered sources, flows, processing and the analytics, ML and LLM Ops uses. The diagnosis looked at reliability, synchronisation, exploitation, performance, scalability and the new product uses. The options weighed strengthening the components that already earned their place against more structural trajectories, challenging existing choices where that was useful.

What we delivered

The final handover came in four pieces, meant to be used by the team rather than filed away.

A map of the data and AI system

We formalised the life cycle of the data end to end: the main sources, how they are ingested, the analytical uses, what the data and ML teams need, and how the traces produced by the LLM systems are exploited.

That shared view made it far easier to connect technical choices to future product uses.

A trajectory for the data foundation

We proposed a progressive evolution of the existing foundation, to strengthen the reliability of the processing, reproducibility, history management, performance, the ability to evolve schemas, and the connection to data science and machine learning workloads.

The approach was deliberately pragmatic: keep the components that already earn their place and strengthen them before considering more structural change.

Product thinking around analytics

The audit did not stop at infrastructure. We worked on how data could become a product capability in its own right, including a vision for an analytics dashboard aimed at Dialog’s own clients.

The longer-term goal: give them more visibility over user journeys, and prepare the ground for experiences that adjust themselves.

Performance

  • Conversion
  • Revenue
  • Usage

Transparency

  • Sessions
  • Conversations
  • Agent actions

Understanding

  • Recurring questions
  • Points of friction
  • What makes a session succeed
The three levels agent analytics should cover: what the product returns, what actually happened in the conversation, and why. The last one is almost always the missing one, and it is the only one you can act on.

A prioritised roadmap

The final handover grouped the recommendations into three strands. Evolving the data-to-product loop: better synchronisation of the sources, improvements to the data foundation and a progressive build-up of the analytics uses. Strengthening the RAG system: optimise what exists in the short term, then test alternatives in a controlled way when that becomes relevant. Evolving the analytics and LLM uses: complete certain datapoints, improve certain workflows and optimise how the models are used.

What was expected was not an exhaustive list of recommendations, but a clear, prioritised trajectory the team could act on directly.

See Dialog

Do you need to take a step back from your technical foundations before deciding what comes next?

Let’s talk