A new gateway to run frontier LLM locally is here and its from Apple

By | March 6, 2025

Apple just released the MAC Studio upgrades for the year 2025, powered by both M4 Max and M3 Ultra. Yes, it will be available in two flavors: M4 Max will be up to 128GB unified RAM and M3 Ultra gets up to 512GB Unified RAM. The M3 Ultra Studio doubles the memory bandwidth (819 GB/s vs. 546 GB/s) and supports up to 512GB of unified memory, making it ideal for memory-intensive tasks like running large language models (LLMs) with over 600 billion parameters entirely in memory. This might be the one that gets to run the Deepseek q4 locally at acceptable token/second.

Comparison of Updated Mac Studios: Mac Studio (M4 Max) vs. Mac Studio (M3 Ultra)

FeatureMac Studio with M4 MaxMac Studio with M3 Ultra
ChipM4 MaxM3 Ultra
CPU Cores14-core (10 performance, 4 efficiency) or 16-core (12 performance, 4 efficiency)28-core (20 performance, 8 efficiency) or 32-core (24 performance, 8 efficiency)
GPU Cores32-core or 40-core60-core or 80-core
Neural Engine16-core (up to 38 TOPS)32-core (estimated ~2x M3 Max’s 18 TOPS, ~36 TOPS)
Unified Memory36GB, 48GB, 64GB, or 128GB96GB, 256GB, or 512GB
Memory Bandwidth546 GB/s819 GB/s
Storage Options512GB, 1TB, 2TB, 4TB, or 8TB SSD1TB, 2TB, 4TB, 8TB, or 16TB SSD
Thunderbolt Ports4x Thunderbolt 5 (up to 120 Gb/s)6x Thunderbolt 5 (up to 120 Gb/s)
Other Ports2x USB-A, HDMI, 10Gb Ethernet, 3.5mm headphone jack, <br> 2x Thunderbolt 5 (front), SDXC slot2x USB-A, HDMI, 10Gb Ethernet, 3.5mm headphone jack, <br> 2x Thunderbolt 5 (front), SDXC slot
Performance (CPU)Up to 1.5x faster than M2 Max (multi-core)Nearly 2x faster than M4 Max (multi-core)
Performance (GPU)Up to 2x faster than M2 Max (ray tracing)Up to 2x faster than M4 Max (graphics workloads)
AI CapabilitiesStrong for on-device AI (e.g., LLMs up to 128GB)Superior for large LLMs (up to 512GB in memory)
Starting Price$1,999 (14-core CPU, 32-core GPU, 36GB RAM, 512GB SSD)$3,999 (28-core CPU, 60-core GPU, 96GB RAM, 1TB SSD)
Target Use CaseHigh-performance creative work, video editingExtreme pro workflows, AI/ML, massive multitasking
Availability March 12, 2025March 12, 2025

Pre-orders are now open at Apple.

Leave a Reply