Скачать с ютуб LocalAI LLM Testing: Llama 3.1 8B Q8 Showdown - M40 24GB vs 4060Ti 16GB vs A4500 20GB vs 3090 24GB в хорошем качестве

LocalAI LLM Testing: Llama 3.1 8B Q8 Showdown - M40 24GB vs 4060Ti 16GB vs A4500 20GB vs 3090 24GB

3090 24GB Has joined the lab + Llama 3.1 models released this week. This week we are taking Llama 3.1 8B (https://huggingface.co/mradermacher/M...) at a Q8 quant running 32k of context through the gauntlet with several GPUs: (These are affiliate-based links that help the channel if you purchase from them!) Tesla M40 24GB https://amzn.to/3Yf4yXC 4060Ti 16GB https://amzn.to/3NeSEGT A4500 20GB https://amzn.to/3TXtAYR 3090 24GB https://amzn.to/3Yf57AI Link to blog on Llama 3.1 and memory requirements https://huggingface.co/blog/llama31 Just a fun night in the lab, grab your favorite relaxation method and join in. Our website: https://robotf.ai Machine specs here: https://robotf.ai/Machine_Lab_Specs GPU Bench Node (These are affiliate-based links that help the channel if you purchase from them!) Open Air Case https://amzn.to/3U08Y27 30cm Gen 4 PCIe Extender https://amzn.to/3Unhclh 20cm Gen 4 PCIe Extender https://amzn.to/4eEiosA 1 TB NVME https://amzn.to/4gWFcFb Corsair RM850x https://amzn.to/3NkITa4 128GB Lexar SSD https://amzn.to/3TZYYGh G.SKILL Ripjaws V Series DDR 64GB Kit https://amzn.to/4dAZrWm Core I9 9820x https://amzn.to/47UuIST Nocuta NH-U12DX i4 CPU Cooler: https://amzn.to/3TZ7O6R Supermicro CX299-PGF Logic Board https://amzn.to/3BxbWVr Remote Power Switch https://amzn.to/3BubQOg Recorded and best viewed in 4K Your results may vary due to hardware, software, model used, context size, weather, wallet, and more!