GLOSSARY
Latency
Latency is the delay between a defined trigger and a system response; for AI, the first response and completed answer are different measurement points.
Choose the measurement point from the work
During a negotiation, a suggestion may lose its value if it arrives after the topic has changed. Preparing a substantial proposal may allow longer waiting times. Define what response is being measured: the first visible text, a complete answer or a professionally checked result. A quick start to text generation can conceal a long completion time. For users, time to genuinely usable support is often more meaningful than one technical measurement within the process.
Make slow cases and demand peaks visible
Alongside the typical result, examine unusually slow cases, concurrent requests and additional tool calls. Inference is only part of the workflow; research, transfer and checking may add delays. Throughput describes the volume processed and answers a different question. An optimisation can improve one measure while worsening another. A practical test should reproduce representative conversations or assignments and measure the relevant interval end to end. Critical situations also need understandable behaviour during delays so users know whether to wait, retry or proceed without the assistance.