GLOSSARY
GPU / graphics processing unit
A GPU is a processor for many parallel calculations, originally developed for graphics and now also used for training and running AI models.
Compute capacity must fit the workload
An organisation considering its own AI infrastructure needs to estimate the models, data volumes and concurrent requests the hardware must support. Theoretical processing power answers only part of that question. Memory requirements, data movement and the operations involved affect usable performance. A conversational assistant may require short response times, while overnight document processing may prioritise total volume. Define these differences before selecting hardware so that a headline performance figure does not obscure the intended workload.
Assess utilisation and operation together
A continuously provisioned GPU incurs costs even when few requests arrive. Variable demand can therefore materially affect the economics of self-hosting. A comparison with a service should include purchase or rental, operation, support and realistic utilisation. Measure latency under concurrent use as well. A useful trial runs the intended model on representative inputs, including longer cases. Only then can the organisation estimate how many reliably completed tasks the infrastructure supports. Hardware specifications alone establish neither product quality nor a cost advantage over an alternative deployment.