Intel Habana Gaudi2 Launched for Lower-Cost AI Training

Intel Habana Gaudi2 Cover

Intel has a new AI training chip, the Gaudi 2. A few years ago Intel Acquired Habana Labs for its AI efforts. That was in the original Gaudi generation. Now, Intel has a new generation of AI training chips called the Gaudi 2. Headlining the feature set is more speed for faster training.

Intel Habana Gaudi2 Launched for Lower-Cost AI Training

The new Intel Gaudi2 is a 7nm OAM module now with 96GB of HBM2E memory. Part of the shrink to 7nm was that the chips got more compute elements and memory. Intel is also using the industry standard OAM form factor instead of a proprietary form factor.

Intel Habana Gaudi 2 Card Image
Intel Habana Gaudi 2 Card Image

For some sense of performance, Intel says that Gaudi2 is faster than the NVIDIA A100. We will quickly note that Gaudi2 is just launching while NVIDIA is about to launch the H100. We have already had our hands on the NVIDIA H100 in Our First Look.

Intel Gaudi2 Performance V. NVIDIA A100
Intel Gaudi2 Performance V. NVIDIA A100

Beyond moving from 16nm to 7nm, Gaudi2 now moves from 8 in the previous generation to 24 Tensor Processor Cores. Tripling compute resources is a big deal enabled by the newer process. We also have media processing engines added to Gaudi2.

Intel Gaudi To Gaudi2 Specs
Intel Gaudi To Gaudi2 Specs

This is important for video analytics workloads, and the inclusion of the hardware accelerator for this is a key enhancement in this generation. As mentioned earlier, not only are compute resources tripled, but onboard memory is tripled to 96GB. SRAM that is on-chip is doubled to 48MB.

Intel Habana Gaudi 2 High Level Block Diagram
Intel Habana Gaudi 2 High Level Block Diagram

Havana’s value proposition extends off chip. The company, prior to its acquisition, was founded on using more open technologies. That includes OAM modules as well as using 100GbE RoCE. As you will notice, there are 24x 100GbE RoCE connections per chip, and eight chips in a system. That requires a ton of bandwidth and so there is the OAM baseboard for the chips (we have already looked at the UBB, but Intel’s key partners are Supermicro and DDN here.

Intel Habana HLBA 225 OAM Board
Intel Habana HLBA 225 OAM Board

Here is the diagram of the platform just to show how it is constructed at the baseboard level.

Intel Habana HLBA 225 High Speed ​​Block Diagram
Intel Habana HLBA 225 High Speed ​​Block Diagram

Compared to NVIDIA that is using SXM and NVLink/ Infiniband, Intel is hoping that using more open/ standard technologies will help to get more market penetration.

End Words

Intel is one of the few companies that has the resources to challenge NVIDIA in AI. The new Intel Gaudi2 is an interesting response. One item that is interesting here is that this is a PCIe Gen3/ Gen4 devices instead of a Gen5 card. That means Intel is pairing the new accelerators with Ice Lake instead of the Sapphire Rapids generation.

Leave a Comment