Challenges for AI and Data Centers
As AI technology advances, the demands on data center components are increasing exponentially. Every critical element in the compute chain must rise to the challenge by delivering enhanced performance whilst consuming less power, providing greater bandwidth, achieving higher density, and doing so at a reduced cost.
The Power Challenge
McKinsey predicts that data center power consumption will represent ~12% of total US demand in 2030.
Training large language models (LLMs) requires immense computational demands, often taking weeks or even months to complete. As these models continue to grow in size and complexity, the required resources also increase. Hence, it is essential to develop new technologies that reduce power consumption across the entire compute chain.
AI and LLMs Limited by Memory Bandwidth
LLMs, such as GPT-4 (est. >1 trillion parameters) require more data to flow between memory and xPUs. Each parameter represents a bias that is updated during training, so larger models require more data retrieval, processing and writing, increasing the bandwidth needs for efficient execution.
Therefore, there is a need for low power, high bandwidth, low cost, reliable interconnect solutions to overcome these challenges.
µLED Interconnects Deliver Best-in-Class Energy Efficiency and Bandwidth Density
While electrical interconnects work well over short distances of a few millimeters, they face fundamental limits in terms of reach, size and power efficiency, especially at higher data rates and longer distances. Avicena’s LightBundle™ interconnects use arrays of microLEDs connected via multi-core fiber bundles to Photo Detectors on CMOS ICs. This enables ultra-low power links of < 1pJ/bit with up to 10m reach.
The parallel nature of the LightBundle interconnect is well matched to the wide internal bus architecture of ICs like CPUs, GPUs or switch ASICs, therefore eliminating the need for a power intensive SerDes interface. In addition, Avicena’s LightBundle is a 2D µLED array making it possible for the IO to cover the entire IC area, thereby increasing the BW density further.
LightBundle Achieving Class Leading Bandwidth Density
The figure illustrates how bandwidth density scales with per-lane data rates. With the current 4 Gbps lane data rate and 50 µm pitch , the LightBundle™ achieves bandwidth densities > 2 Tbps/mm.
The next generation of devices, featuring a reduced 25 µm pitch, will enable bandwidth densities exceeding 10 Tbps/mm.
LightBundle™: 10x Better Power and Density
The performance of an interconnect can be expressed in terms of a figure of merit (FoM) combining bandwidth density on the edge of chip (Gbps/mm) with energy efficiency (pJ/bit). Avicena’s LightBundle™ optical interconnects can achieve power and density FoMs which are >10x Better than Existing Solutions.
The chart shows the LightBundle interconnects match the performance of electrical links whilst increasing the reach up to ~10m, allowing more efficient scale-up AI architectures.
Product Roadmap for AI/ML Applications
Avicena’s On-Board Optics (OBO) solution consists of an array of 1.6Tbps OBO transceivers that can be placed close to the xPU, significantly reducing power consumption, boosting bandwith density, and facilitating xPU scale-up in data centers.
