LLMs, such as GPT-4 (est. >1 trillion parameters) require more data to flow between memory and xPUs. Each parameter represents a bias that is updated during training, so larger models require more data retrieval, processing and writing, increasing the bandwidth needs for efficient execution.
Therefore, there is a need for low power, high bandwidth, low cost, reliable interconnect solutions to overcome these challenges.