The Tech Poutine #21: Defining NVIDIA скачать в хорошем качестве

The Tech Poutine #21: Defining NVIDIA

Today's gravy is Jeff from @CraftComputing! Everyone else's gravy is going over exactly what @NVIDIA showcased at its annual tech conference. We have new datacenter AI chips, new racks of hardware, new switches, and new workstations. Timestamps (thanks ‪@eyeofthetiger7‬ ) Pre-Show and Introductions 0:02 - Introduction to the Tech Poutine pre-show with hosts in a hotel kitchen. 1:42 - Discussion about NVIDIA selling RTX 5090s at MSRP at GTC event. 2:37 - GTC attendance context - grew from 20,000 people last year to 25,000+ this year. 4:21 - Introduction to "tech boxes" segment for smaller news stories. Main Show Introduction 11:18 - Official introduction to Episode 21 of Tech Poutine with hosts Ian (Tech Potato), George (Chips and Cheese), and guest Jeff (Craft Computing). 12:36 - Discussion of Jeff's entrepreneurial venture: Open Virro environmental monitoring system. 18:28 - Introduction to the main topic: NVIDIA's GTC 2024. NVIDIA Roadmap and Blackwell Architecture 19:28 - Overview of GTC 2024 - felt less packed despite higher attendance numbers. 26:44 - Discussion of Jensen Huang's keynote structure: first hour on strategy, second hour on hardware announcements. 28:48 - Analysis of Jensen's frequent use of "token" instead of "AI" in the keynote. 31:32 - Examination of NVIDIA's updated roadmap extending to 2028. 32:58 - Overview of Blackwell B100/GB200 launch featuring dual die GPUs with HBM3e memory. 34:30 - Discussion of NVIDIA's NVL systems with large numbers of GPUs in a single rack. 36:52 - Detailed explanation of the computational capabilities of Blackwell, including FP4 and FP8 operations. 40:41 - Analysis of how the NVL system works with multiple system images appearing as a single GPU. 44:19 - Discussion of the unusual 10-9-8 rack arrangement for compute and switching drawers. SOCAM Memory and New Memory Form Factor 45:47 - Introduction to SOCAM, a new LP-DDR memory form factor shown at Micron's booth. 48:52 - Explanation of how SOCAM differs from soldered LPDDR memory and makes systems more serviceable. 52:01 - Analysis of Samsung's SOCAM modules with different specs from Micron's version. 56:05 - Discussion of power consumption and bandwidth advantages of SOCAM over standard DIMM modules. Blackwell Ultra and Future Architectures 1:00:24 - Introduction to Blackwell Ultra (GB300) with 288GB of HBM memory. 1:01:42 - Discussion of Vera Rubin architecture (2026-2027) with SMT2 ARM cores. 1:03:42 - Analysis of the custom ARM CPU cores with SMT for Vera compared to standard Neoverse cores. 1:07:03 - Discussion of the Ruben Ultra 576 GPUs with massive 600kW power consumption per rack. 1:11:25 - Debates about the packaging technology constraints for the extremely large Ruben Ultra chips. 1:20:04 - Analysis of Richard Fineman architecture (2028) and unusual naming conventions. 1:24:33 - Discussion of optical connectivity trends and NVIDIA's position on optical vs. copper connections. Silicon Photonics and Optical Networking 1:33:32 - Introduction to NVIDIA's Silicon Photonics program and co-packaged optics. 1:35:57 - Explanation of pluggable vs. integrated optical transceivers in networking. 1:41:22 - Analysis of how optical connections could enable new memory architectures. 1:45:44 - Discussion of NVIDIA's Spectrum X and Quantum X switches with integrated optics. DGX Spark 1:52:21 - Introduction to DGX Spark (formerly Project Digits) - a small AI development system. 1:58:58 - Detailed specifications of GB10 chip with MediaTek-developed ARM cores. 2:02:51 - Connectivity options including 10Gbe and 200Gbps networking on the small form factor. 2:05:24 - Discussion of 170W power consumption through USB-C and cooling challenges. 2:09:39 - Pricing details: $3,000 for OEM versions, $4,000 for NVIDIA Founders Edition. DGX Station 2:11:42 - Introduction to DGX Station workstation with GB300 GPU and Grace CPU. 2:14:05 - Discussion of PCIe slots, networking options, and memory configurations. 2:16:02 - Analysis of the complex power delivery with multiple 12V connections. 2:19:24 - Detailed motherboard examination showing audio chips, fan headers, and connectivity. 2:22:52 - Estimated pricing is around $15,000 for the motherboard alone, $25,000+ for complete systems. RTX Pro 6000 Cards 2:28:48 - Introduction to RTX Pro 6000 professional graphics cards with 24,196 CUDA cores. 2:31:13 - Explanation of three variants: workstation, Max-Q (blower design), and server (passive cooling). 2:36:42 - Discussion about high-speed fans needed for server variants with 600W power consumption. Tech Boxers (Smaller News Items) 2:43:27 - Introduction 2:44:41 - MediaTek's access to NVIDIA's IP portfolio including NVLink. 2:49:12 - Analysis of the MediaTek-NVIDIA relationship and potential implications. 2:55:57 - News about Ampere Computing being acquired by SoftBank. 2:56:55 - Discussion of SoftBank also owning ARM and Graphcore. Closing Remarks 3:03:54 - Final thoughts and acknowledgements.