r/SelfHostedAI • u/AltCtrlCat • 2d ago

Mac Mini / Studio

I saw the preorders this past week for Mac mini and studio. The builds I’d like are I the $3k and $5k range respectively. I know this is def a higher price than building my own rig for a self hosted AI machine.

I am curious what opinions are on a custom built rig with separated hardware vs the “unified memory” architecture of Apple silicon? I recently set up ollama on my 16g RAM m1 MacBook Pro from 2021 and please rly surprised at performance.

1 Upvotes

100% Upvoted

3 comments sorted by

3

u/newz2000 2d ago

I don't think you can get more for your money. nVidia cards with 24gb of ram can cost $3k, and, as someone with a 16gb card, I think 24gb is the minimum to do a variety of tasks. But a Mac with 64gb of Ram can allocate 48gb to AI tools, maybe more.

1

u/tomByrer 1d ago

- smaller models very fast with nVidia
vs

  • bigger models slower with OSX

2

u/vortec350 2d ago

I've spent a lot of money on various GPUs. Arc B60, AMD R9700, 7900XTX. In the end when running a model that fits on a single GPU the performance per dollar is much better than a Mac or other systems that have unified memory available (AMD AI 395+ Max whatever it's called and DGX Spark and its cousins). BUT when running larger models across multiple GPUs the performance just doesn't scale the same or work as smoothly. PCI-E just can't compare to unified memory etc. What I'm trying to say is - for a casual user who just wants to run larger models on a local server and wants something efficient and reliable, you're better off buying a Mac Studio or GB10 system. Personally I do not like MacOS so I'd buy a GB10 PC (the DGX Spark and the related OEM versions from Dell, HP, Asus, MSI, etc - all the same motherboard with different cooling systems and cases etc) over a Mac but it just depends what you are looking to do with it.