In a major leap forward for high-bandwidth memory technology, SK hynix has announced the world's first 48GB 16-Hi HBM3E memory at the SK AI Summit 2024. This breakthrough puts the company ahead of competitors Samsung and Micron in the race for advanced memory solutions.
Pushing the Boundaries of HBM3E
The new 16-layer HBM3E memory builds upon SK hynix's recent 12-Hi variant, which has already secured contracts with industry giants AMD and Nvidia. The increased density of the 16-Hi version allows for up to 384GB of HBM3E memory in an 8-stack configuration, opening new possibilities for AI accelerators and high-performance computing.
SK hynix reports impressive performance gains with the new memory:
- 18% improvement in training performance
- 32% boost in inference performance
These advancements are made possible by innovative packaging technologies like MR-MUF, which uses melted solder to connect individual chips within the stack.
Timeline and Future Developments
While samples of the 16-Hi HBM3E memory are expected to be ready by early 2025, the technology may have a limited lifespan. Nvidia's next-generation Rubin chips, slated for mass production later next year, are anticipated to utilize the upcoming HBM4 standard.
SK hynix is not resting on its laurels, however. The company is actively developing:
- PCIe 6.0 SSDs
- High-capacity QLC eSSDs for AI servers
- UFS 5.0 storage for mobile devices
- LPCAMM2 modules and soldered LPDDR5/6 memory for laptops and handheld devices
Addressing the "Memory Wall"
To tackle the growing challenges of data processing and storage, SK hynix is exploring innovative solutions such as:
- Processing Near Memory (PNM)
- Processing in Memory (PIM)
- Computational Storage
These technologies aim to bring computation closer to where data is stored, reducing latency and improving overall system performance.
The Road to HBM4
As the industry looks ahead, HBM4 promises even greater capabilities:
- Doubled channel width from 1024 bits to 2048 bits
- Support for up to 16 vertically stacked DRAM dies
- Individual dies with up to 4GB of memory
While Samsung's HBM4 tape-out is expected later this year, reports suggest that SK hynix may have already achieved this milestone in October, potentially giving the company an edge in the development of next-generation memory technologies.
SK hynix's announcement of the 48GB 16-Hi HBM3E memory represents a significant step forward in the evolution of high-performance computing and AI acceleration. As the demand for faster, more efficient memory solutions continues to grow, innovations like these will play a critical role in shaping the future of technology.