Gooaye 股癌

用 4090 土炮 AI 伺服器

" 建置小型 GPU 伺服器，尤其是使用 NVIDIA RTX 4090 或新款 RTX 5090 等強大 GPU，可運行大型語言模型如 LLaMA、DeepSeek 和 Mistral。現代開源模型在設計時就考慮到了高效推理，通常使用混合專家 (MoE) 架構，而 4090 可以輕鬆處理這些工作負載。"

Server model: ASUS ESC8000A-E12P
GPUs: 8x NVIDIA RTX 4090
CPU: 2x AMD EPYC 9254 Processor (24-core, 2.90GHz, 128MB Cache)
RAM: 24x 16GB PC5-38400 4800MHz DDR5 ECC RDIMM (384GB total)
Storage: 1.92TB Micron 7450 PRO Series M.2 PCIe 4.0 x4 NVMe SSD (110mm)
Operating system: Ubuntu Linux 22.04 LTS Server Edition (64-bit)
Networking: 2 x 10GbE LAN ports (RJ45, X710-AT2), one utilized at 10Gb
Additional PCIe 5.0 card: ASUS 90SC0M60-M0XBN0

https://a16z.com/building-an-efficient-gpu-server-with-nvidia-geforce-rtx-4090s-5090s/

Andreessen Horowitz

Building an Efficient GPU Server with NVIDIA GeForce RTX 4090s/5090s | Andreessen Horowitz

Building your own GPU server—like the one described here—means no API calls to external services, no data leakage, and no usage throttles.

www.tgoop.com/Gooaye/5054

29.4K viewsApr 4 at 03:41

用 4090 土炮 AI 伺服器

" 建置小型 GPU 伺服器，尤其是使用 NVIDIA RTX 4090 或新款 RTX 5090 等強大 GPU，可運行大型語言模型如 LLaMA、DeepSeek 和 Mistral。現代開源模型在設計時就考慮到了高效推理，通常使用混合專家 (MoE) 架構，而 4090 可以輕鬆處理這些工作負載。"

Server model: ASUS ESC8000A-E12P
GPUs: 8x NVIDIA RTX 4090
CPU: 2x AMD EPYC 9254 Processor (24-core, 2.90GHz, 128MB Cache)
RAM: 24x 16GB PC5-38400 4800MHz DDR5 ECC RDIMM (384GB total)
Storage: 1.92TB Micron 7450 PRO Series M.2 PCIe 4.0 x4 NVMe SSD (110mm)
Operating system: Ubuntu Linux 22.04 LTS Server Edition (64-bit)
Networking: 2 x 10GbE LAN ports (RJ45, X710-AT2), one utilized at 10Gb
Additional PCIe 5.0 card: ASUS 90SC0M60-M0XBN0

https://a16z.com/building-an-efficient-gpu-server-with-nvidia-geforce-rtx-4090s-5090s/

BY Gooaye 股癌