Nvidia pursues multi-chip module architecture to meet evolving data needs

Why is this important: Currently available deep learning resources are falling behind due to increasing complexity, competing resource requirements, and limitations imposed by existing hardware architectures. Several Nvidia researchers recently published a technical paper describing the company’s pursuit of multi-chip modules (MCMs) to meet these changing demands. The article presents the team’s position on the benefits of a Composable-On-Package (COPA) GPU to better accommodate various types of deep learning workloads.

Graphics Processing Units (GPUs) have become one of the primary resources supporting DL due to their inherent capabilities and optimizations. COPA-GPU is based on the realization that traditional converged GPU designs using domain-specific hardware are quickly becoming a less than practical solution. These converged GPU solutions are based on an architecture composed of the traditional matrix as well as on the incorporation of specialized hardware such as high bandwidth memory (HBM), Tensor Cores (Nvidia)/Matrix kernels (AMD), ray tracing (RT) cores, etc. This converged design results in hardware that may be well suited for some tasks but inefficient for performing others.

Unlike current monolithic GPU designs, which combine all of the specific runtime components and caching into a single package, the COPA-GPU Architecture provides the ability to mix and match multiple hardware blocks to better accommodate the dynamic workloads presented in today’s high performance computing (HPC) and deep learning (DL) environments . This ability to incorporate more capacity and adapt to multiple types of workloads can result in higher levels of GPU reuse and, more importantly, a greater ability for data scientists to push the boundaries of what is possible using their existing resources.

Although often grouped together, the concepts of artificial intelligence (AI), machine learning (ML) and DL have differences. DL, which is a subset of AI and ML, attempts to mimic the way our human brains process information by using filters to predict and classify information. DL is the driving force behind many automated AI capabilities that can do everything from driving our cars to monitoring financial systems for fraudulent activity.

While AMD and others have touted chip and chip stack technology as the next step in their CPU and GPU evolution in recent years, the concept of MCM is far from new. MCAs can be dated as far back as IBM’s bubble MCMs and 3081 mainframe computers in the 1970s and 1980s.


Please enter your comment!
Please enter your name here

Trending this Week