Rendered at 22:01:35 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
ks2048 6 hours ago [-]
So, Meta will spend millions/billions on building open-weight models and then people will send Apple $20,000 to run them. Seems like a good deal for Apple.
Juvination 6 hours ago [-]
Exactly why does Apple need to do any model training/development? They are out here selling the shovels.
bredren 6 hours ago [-]
Apple’s Photos ecosystem is privacy oriented and as a result some (now) baseline features require the use of local models.
The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.
I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)
I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.
There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.
cj00 6 hours ago [-]
I've found some strange guardrails with the Foundation models. Like blocking describing of any photo with a single woman in it. It does this relatively consistently but with a different subset every time. Like out of 10 photos in the set, it's skips 6 then the next run, it skips 3. Have you seen guardrail issues like this?
I switched to Gemma 4 instead.
blacksmith_tb 3 hours ago [-]
I am going to assume that means "a women by herself" and not that it can magically detect her marital status before refusing to continue (I suppose it could be laser-focused on wedding rings...)
dominotw 5 hours ago [-]
same i am using new embeddinggemma. seems far more capable than foundation models.
sneak 3 hours ago [-]
No. Apple’s Photos ecosystem uploads all Photos effectively unencrypted to Apple in a way that is readable by Apple, allowing Apple to do server-side analysis of all content. This content is also provided to US or Chinese federal authorities without due process or warrants.
Nothing about iCloud Photos is private.
isomorphic 3 hours ago [-]
With Advanced Data Protection enabled (it's not available globally but it is in the US), Photos is E2EE.
soulofmischief 5 hours ago [-]
I don't want any of these features and yet they've been forced upon me and are now getting in the way of me actually using my computer how I want. Games and heavy applications slow down and stutter because of "AI" background processes, even with App Tamer, my machine runs too hot to even touch when just idling. Compared to when I got my M2 in '23 and thought, "This is the fastest and snappiest machine I've ever used", my experience with an M5 and the latest macOS version is nothing short of an anti-consumer hellscape.
doug_durham 5 hours ago [-]
My Macs run cool and silently. I've never observed a background "AI" process consuming resources.
soulofmischief 4 hours ago [-]
It's not just AI processes. But things like mediaanalysisd, duetexpertd, etc. even the freaking menu bar agent, all regularly pound my computer both when idling and when in use.
I recently installed App Tamer which is helping a bit, but it doesn't allow me to tame everything. I just checked now and duetexpertd was trying to use nearly two full cores. I have all iCloud integrations turned off, I do not use iCloud at all, and I do not have an iPhone, so I am flabbergasted that this iCloud desktop-phone syncing process runs at such high core usage in the background with extreme regularity. Proof: https://imgur.com/VkBh9bi
Unfortunately, this does nothing for services such as WindowServer which now often takes up 50-100% of a core just on its own.
I have reinstalled and upgraded to the latest version. This MacBook is only a few months old.
While typing this message and literally just having open Codex with one running task that is not doing anything substantial, and Firefox, with no high CPU usage from any tab, my laptop got so hot that I had to use another installed program, Stats, to manually bump up the fan since macOS also has absolutely garbage native system fan controls.
I have fibromyalgia and seem to be sensitive to hot keys; my Thinkpad regularly getting to hot to touch with even light usage is EXACTLY why I went and bought a MacBook one morning in '23. One morning I touched my keyboard and realized I was tired of singeing the nerves on my fingertips and having them hurt for hours. So now in '26, finding myself in the same exact position but with a much more expensive computer, I feel like I am living in a corporate hellscape nightmare.
One thing I have noticed is all of my cores can sit idle except a single one and that will still heat up the machine a lot. If the hardware is designed as such, the software MUST be designed to work around this and not blast a core for 10 minutes straight every time one of a dozen or two misbehaving system processes wants to complete a background task. If the software is not designed for this, the hardware is incomplete and faulty. This would be true if it was shipped as firmware; software doesn't get a free pass here.
But now Pages and Numbers recommend document titles for me, so.... worth it?
unsnap_biceps 2 hours ago [-]
this is not my experience at all. I wonder if one of your tweaks is causing the software to get into a bad state. Likely worth reporting to Apple even if it feels like it's going into a void.
papa_bear 5 hours ago [-]
Interesting, this has not been my experience at all. I have an M5 machine and I'm not even aware of what AI features have been added, and it's streets ahead of my old M1 Pro in speed.
sneak 3 hours ago [-]
The M1 used to be adequate in speed. Then Apple kept adding always-on cloud daemons that run even if you don’t use iCloud.
soulofmischief 4 hours ago [-]
peep the comment I just made to the adjacent commenter, that has been my experience. I basically have to run my fans at full speed all day while working now in order to type on my computer (when I'm not using my magic keyboard). It is like a huge problem, not even just a small one but like something that is affecting me negatively every single day, increasingly making me want to break this machine in half and disappear into the woods.
rswail 6 hours ago [-]
Exactly my thought as I read it.
They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.
Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.
So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.
drob518 5 hours ago [-]
Well, the moat is that cpu/npu development is very hard, requires talented engineers, and requires access to cutting edge fab process, all of which in turn costs a lot of $$$. So, sure, “anyone” who has all that can definitely breach Apple’s moat.
uurue1 5 hours ago [-]
Ok.. and why specifically do people work at Apple?
Hint: money is not the only reason.
Jeez.
sneak 3 hours ago [-]
Most of the smart people I know have left Apple because over the last ten or fifteen years, all of the reasons other than money have gone away, and they don’t pay well.
intrasight 3 hours ago [-]
> Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.
I think both points are very incorrect. Apple has a massive moat. And Apple is making hardware (specifically, CPUs) that are two years ahead.
uurue1 5 hours ago [-]
Ermmm what?
Apples largest competitive advantage has always been and always will be - trust combined with epic marketing to make the vast majority of people aware of it.
Trust of quality, privacy w.r.t data etc.
I’m sorry but I’ve seen so many bad takes re. Apple over the years. Many of you think you are so smart on topics outside of your domain - but you’re not.
Nevermark 3 hours ago [-]
Don’t underestimate how much of their success is due to unusual internal advantages over competitors.
They spent decades learning to eliminate a 1000 headaches for themselves, and many for their customers, by accumulating vertical integration:
The co-design of their own comms, processors, devices and OS variants. Their own integrated supply chain and sales channels.
The better the results, the more invisible or clock-like their delivery.
Very easy to overlook. But nobody can match them on their terms.
(Nintendo also applied a unique set of internal advantages to generations of successful devices: vertically integrating humble components, highly scoped software, a strong theory of fun, and deep branding with novel polished interfaces.)
fasterik 5 hours ago [-]
What exactly about the parent comment is wrong? Apple has always been good at making hardware that people are willing to pay a premium to use. I can easily imagine them playing a similar role in local LLMs to the role they played in personal computing in the 1980s or smartphones in the 2010s.
oakejp12 5 hours ago [-]
> Apples largest competitive advantage has always been and always will be - trust.
Lmao
> Many of you think you are so smart on topics outside of your domain
Turn the mirror around
MangoCoffee 5 hours ago [-]
Nvidia is also selling the shovle but they train their own LLM and by doing so they can fine tune their CUDA/Chip to have a better shovel than others.
Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.
Apple can chose not to train their own model to learn how to sell a better shovel.
jtbaker 4 hours ago [-]
> They are out here selling the shovels
just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).
neya 4 hours ago [-]
That's just BS cope. They tried and miserably failed - and still are failing badly. Remember, they were the ones to push for AI with "Siri". Many firings later and 2 CEOs later we are still exactly in square one. The fucking thing can't even get who I want to call, let alone set an alarm without touching my phone FFS.
jorblumesea 5 hours ago [-]
apple is pretty well set up for this because they do all their hardware in house. if they wanted to, they could pivot in this direction. Ternus is also a hardware guy.
loeg 5 hours ago [-]
How big is the local LLM market? I have trouble imagining Apple will move enough units to make $20k/ea worth it for them as a business segment.
____tom____ 5 hours ago [-]
They have been sold out of a lot of units, due to people developing AI, so apparently, it is significant.
Lalabadie 5 hours ago [-]
I think the bet is that it will get bigger, both at the consumer level, and with orgs eventually discovering ready-made solutions for having their own on-prem inference.
ijidak 5 hours ago [-]
100,000 units a year would be $2 billion annually.
And usually, most of a company's revenue comes from the heaviest spenders.
So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.
I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.
$2-10 billion is meaningful for any company. Even one as large is Apple.
Especially if it's a first step towards a stronger foothold in the AI inference space.
They can then expand down market to grow the pie further.
For example, with truly dedicated inference boxes.
And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)
5 hours ago [-]
GeekyBear 6 hours ago [-]
How many 5080 16 Gig cards would you have to buy to run the same model, and what would that cost?
fluidcruft 6 hours ago [-]
idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs.
petu 6 hours ago [-]
158GB number is for Mistral Medium 3.5 128B model, Muse Spark is way larger.
Muse Spark Open would require more than 1.6TB of RAM in its native precision (BF16) for the model parameters alone. In a 4-bit quantized (NVFP4) format, the model memory footprint is lowered to about 493GB.
bigyabai 5 hours ago [-]
So about 24gb/expert at NVFP4. If you compressed the KV cache, you might be able to get away with 2x 24GB cards and stream the experts.
petu 5 hours ago [-]
If you're happy with generation speed in 5 t/s range, sure -- PCIe 5.0 x16 is only 64GB/s.
bigyabai 5 hours ago [-]
I never said it would be fast!
In any case, I'd much rather run GLM 5.3 Flash on that sort of hardware. Better fit for the performance profile, and will probably beat Muse Spark Open in most uses.
reilly3000 6 hours ago [-]
They have the best strategy of bigtechco: let others figure out the squishier layers while they focus on selling the best hardware. Privacy has incredible value to a lot of individuals and industries. $20-80k in hardware that *appreciates* the moment you buy it is a reasonable investment for a lot of SMBs, and still less than a single Nvidia rig. Privacy pays.
aurareturn 25 minutes ago [-]
I disagree.
I think the iOS App Store is going to be in steep decline in a few years. People will generally have a few apps: ChatGPT, Instagram, messaging, and something like a Photos/Camera app.
Everything else will be done in your AI app.
I've been hooking up more and more data and plugins into ChatGPT/Claude. I only want to use it. I don't want to use other apps if I don't have to. For example, research, buy, plan. I prefer not to use other apps.
I think all it takes is for OpenAI to make a phone from the ground up for this and Apple is going to have serious competition.
bigyabai 5 hours ago [-]
If we're being frank here, Apple's AI strategy lags behind Nvidia, AMD and even Google. Nvidia has CUDA, AMD ships CDNA, Google has the TPU family, and Apple... well, Apple discontinued the rackmount Mac Pro, forfeit the HPC ARM datacenter to Nvidia's Grace CPUs, refuses to sign CUDA drivers, and spent half a decade designing raster GPUs instead of GPGPUs.
All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.
AndrewKemendo 4 hours ago [-]
Saying they are lagging assumes they are in the same race
They aren’t
Apple is a high net worth consumer technology integrator that focuses on highest coherence user experience. They pick the most efficient stable technology, integrate it and sell the package
Apple only needs to lead in precisely that they don’t need to lead in literally anything else
in fact if they tried to lead in those other things they would give up their integration capacity
bigyabai 2 hours ago [-]
But they are. Nvidia ships GPGPU ARM SOCs to consumers; why can't Apple? They're both trillion-dollar companies, surely Apple isn't lacking in engineering talent.
The vertical integration argument relies on Apple having a good architecture for AI. First it was the ANE, which everyone (mistakenly) thought would differentiate Apple Silicon from other NPU platforms. Then everyone parroted the "unified memory" talking point without checking to see if professionals actually used it. Now people are pivoting to say the next-gen Apple GPUs will finally blow the last-gen GPGPUs out of the water. Which is good, because there are 4-year-old GPUs that beat M5 Ultra in compute efficiency.
I know that "Apple is behind" is a tough pill to swallow, but they literally have no analog to CUDA, CDNA or even TPU technology. Nvidia SOCs like Tegra prove that Apple could absolutely bring GPGPU tech to low-power hardware like the iPad and iOS without giving up any real features. Apple just refuses to invest in it.
newsclues 2 hours ago [-]
I wonder how many units Meta bought to convince Apple to do a collaborative advertisement.
binlog 3 hours ago [-]
Now scale that up to Nvidia and you've understood the entire AI market
rvz 6 hours ago [-]
Both companies are profitable and AI is not their main line of business. This affects pure AI companies.
As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally.
srmatto 6 hours ago [-]
[flagged]
informal007 6 hours ago [-]
It won't be free for Apple to use Meta's model in the future.
chaostheory 4 hours ago [-]
Even in the unlikely case that does happen, there are plenty of other open weight models like Qwen, GLM, and Mistral to name a few
peri-cl 6 hours ago [-]
I think they're sandbagging Qwen pretty heavily by benchmarking Meta's model at 4-bit, against Qwen only at native BF16 (and nothing else shown). That's not a quant most users would choose.
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
zkrhass_ 5 hours ago [-]
[dead]
liuliu 5 hours ago [-]
FWIW, single M5 Ultra with 256GiB runs DeepSeek 4.1 Flash 0910 just fine (~3200 tok/s prefill, ~45 tok/s). M5 Max w/ 128GiB runs GLM 5.3 Flash also fine.
I also benchmarked both on quality benchmarks (MathArena Apex / GPQA Diamond etc) to make sure the quality is what you get from the official numbers.
TomGarden 1 hours ago [-]
GLM 5.3 Flash on 128GB Ram would have to be 2-bit right?
I'm no expert but it seems the consensus is you get a suboptimal experience under 4-bit. Please correct me if I'm mistaken here
janderson215 7 hours ago [-]
Hilarious how they managed to put out an entire press release about Meta’s technology without mention Meta other than what is in the code references.
Someone 6 hours ago [-]
It’s not a press release and not about Meta’s technology.
The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse.
It’s more like an advertorial for their tech.
browningstreet 7 hours ago [-]
Still weird, and against brand, for them to lead with this particular model from that particular company. The groans are still earned.
enos_feedler 6 hours ago [-]
I find it bizarre of Apple to do anything publicly in partnership with Meta. I always thought the reasons Apple snubbed Facebook were out of values and principles not business. That could be wrong. Now that Meta has a line of business that Apple can finally charge a platform tax on there is new reason to start sleeping with them.
bitpush 5 hours ago [-]
> I always thought the reasons Apple snubbed Facebook were out of values and principles not business.
Apple is showing a masterclass of how to tow the line perfectly. Or more candily "have the cake and eat it too".
If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.
Same here - they are happy to partner with Meta, without getting any of the bad "aura".
To be clear, all companies do this just that Apple alone has a successful strategy here. Tim Cook personally gifted the current adminstration with a gold plaque, something none of the other "dirty" CEOs did. Such a public gesture, yet nobody would think that Apple is in bed with the current administration. Its marvelous to see this happen in real time.
macintux 5 hours ago [-]
Apple also was one of the largest contributors to the ballroom. They're not maintaining much of a "moat" between the company and the administration.
origini 6 hours ago [-]
Its still weird though.
The datacenters are not full of Mac studios but Nvidia hardware.
They are known for macbooks.
And they are not willing to build proper apple servers for all the build systems people have.
robotresearcher 6 hours ago [-]
They did, and few people bought them, so they stopped.
myrandomcomment 6 hours ago [-]
I worked at a company where one of the pieces of hardware we were building at the time used a PowerPC CPU (this is back in ~2007). We had a rack of Apple xServe for build and test. The xServe was nice pieces of hardware compared to the x86 kit we had, more like Sun gear in their attention to detail and ease of use.
At my current company we compile for x86 and devs can do a full build and test environment on their laptops (which are 95% MacBook Pros) but it is unclear what will happen when Rosetta support is pulled. Now the interesting thing is we plan to move our stuff over to target the AWS Graviton (and other cloud ARM cores) as those instances are cheaper for generally the same performance (for what was are doing anyway) so the point will be moot. There will be some crossover point where having build and test servers in a rack will be cheaper than doing it in the cloud. At that point having Apple ARM based servers would be a win assuming we can build a binary that works for Graviton, et.al.
Someone 6 hours ago [-]
They don’t lead with it. The title of the actual article is “Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”.
The submitter changed it to “Apple ML Research: Muse Spark Open with MLX on Mac Studio”
browningstreet 6 hours ago [-]
Second paragraph: "In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters..."
In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today. On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second (see Figure 1), well above average human reading speed. Beginning on a single Mac Studio (up to 512GB of unified memory), then scaling out across multiple Macs with the RDMA over Thunderbolt driver, below we share an overview of the range of models you can now run on Mac Studio.
bensyverson 7 hours ago [-]
Yes, I'm very glad they're doing this, but as of 2026, it's a bit academic. Pricing for the 512GB Mac Studio hasn't even been announced yet, but the 256GB version starts at $9499. So when you start talking about four of those puppies, it's substantial.
Someone else will do the math on how many years it would take to spend $40k on Muse Spark Open via OpenRouter.
mbreese 6 hours ago [-]
But there are definitely people out there that will need to run these models locally (contractual or legal requirements). By validating a Mac Studio as an option for running local models, it might be the more appealing option.
For most of us though, you're right -- the OpenRouter option is far more cost effective.
hirako2000 7 hours ago [-]
It seems Apple figured they could sell more Mac Studios on the basis of frontier local AI models, profit to be made while users haven't figured out the math yet.
bensyverson 6 hours ago [-]
Don't get me wrong; in the long term I see the price of local inference falling as their chips continue to get 15-20% faster every year, and meanwhile the "intelligence per billion parameter" chart is rising. There will be a magic point where they cross, and "good enough" reaches your phone. That's when Apple starts to drink Nvidia's milkshake.
cyanydeez 6 hours ago [-]
Apple and NVIDIA prices are not going to fall.
Their competitors will. Anyone locked into these two ecosystems are willingly going along with it.
bensyverson 5 hours ago [-]
It’s not about the price of Apple products falling—I was commenting that the price of inference will continue to fall (chips getting faster). If “good enough” inference is in your pocket, this benefits Apple and harms Nvidia.
bitcurious 6 hours ago [-]
Puts into perspective how cheap the frontier models are via subs or even API.
mlsu 6 hours ago [-]
You can't really compare API prices though. The cards that they are running frontier models on in the datacenter can service hundreds of simultaneous requests and are at nearly 100% utilization. This thing would sit on your desk and be used to service a single user for a couple hours per day.
chaostheory 4 hours ago [-]
The main subsidy, from OpenAI, is ending 10/26. It’s still “cheap” for now compared to all the other non-open weight frontier models, but it won’t last since OpenAI’s stated goal is to make it so that there’s no much of a difference price wise between on demand API calls and subscriptions
LoganDark 7 hours ago [-]
There was the M3 Ultra 512GB Mac Studio, though.
mft_ 7 hours ago [-]
…which is in the region of €20k on eBay, so the calculus doesn’t really shift…
LoganDark 6 hours ago [-]
It was only like $14k new. Why are you people acting like nobody has this stuff? People were already building clusters when that thing first came out. The research isn't bunk just because the refresh hasn't landed yet, my god.
mft_ 3 hours ago [-]
Apple’s article isn’t targeting people who bought multiple M3 Ultras before the RAMpocalypse - those folks can be safely assumed to already be on board with running large models locally.
LoganDark 1 hours ago [-]
I thought the article was about showing this specific model can be run locally, not convincing people about running models locally in the first place.
unshavedyak 6 hours ago [-]
Maybe you (or i?) missed the point - it wasn't that nobody has those machines, it was that at $40k for 4x the 256GB versions is a substantial price, and so the parent parent was suggesting that it would be compared against OpenRouter. Ie how long would it take someone to spend $40k worth of credits on OpenRouter?
I've had the same thought on self hosting for the past 2 years. Eg i could invest $10k into hardware or spend those on credits. I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.
nhecker 4 hours ago [-]
> I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.
I agree, at least for inference credits. But anecdotal counterpoint: about ten years ago I put together a system using about $1600 worth of new and used components. Including electricity that TCO has been something like $14/month. I'd be hard pressed to find 128GB of RAM, 12 cores of x86_84, and 40 TiB (raw) worth of disk, and unlimited ingress/egress traffic for rent at that rate. That's what, $0.02 an hour, rounding up?
Yes, a hyperscaler's rent-a-computer would get faster and be more energy efficient. Possibly more carbon efficient over it's lifetime, although I have no way to quantify that. But in raw dollars it would have costed me absurdly more to rent. On the other hand, my machine has sat cold and unused for multiple months when I didn't need it to be running, and I don't consume all of it's resources around the clock. So maybe I could have gotten away with a lesser amount of local compute in a Raspberry Pi or similar, and an equally small cloud box for $5 a month or something. I dunno. I don't even own a laptop or desktop, and try to use 'cloud' services whenever practical. (Edit: always giving priority to privacy in those circumstances; only data I don't care about goes up.)
There are tradeoffs. I don't use my local server for LLM inference, it's far to inefficient for that; so I'll happily pay OpenRouter per token as needed. So I do think that hosted compute for rent does have a purpose. I guess all that's to say that I don't see it in black and white; there are times to invest in local hw and times to ride on the optimized bleeding edge of what other companies are willing to host in big datacenters.
This turned into a bit more of a ramble than I intended.
LoganDark 4 hours ago [-]
The substantial price is one reason I'm disappointed that open models are still getting bigger and bigger, rather than more efficient in any way.
I just also think that plenty of people buy clusters of Mac Studios anyway.
nhecker 4 hours ago [-]
Haven't smallish (16-32 GB VRAM) open-weights models been getting better and more capable over the last few years? Seems like what you can run on 'a laptop' today beats the pants off of whatever you could run on that same laptop two years ago.
LoganDark 4 hours ago [-]
I guess. I found that Gemini 3.1 Pro and 4 Argon are by far the best at what I like to use LLMs for. Even 3.8 Flash doesn't compare, and I'd imagine something I could run on my local hardware (M4 Max 128GB) would be just as disappointing as the rest, at least for now, even if better than earlier models. One of the reasons I've been hoping to see diffusion language models eventually.
throwworhtthrow 5 hours ago [-]
> peak power consumption is about 200W for the Mac Studio with M3 Ultra, and 300W for the Mac Studio with M5 Ultra — a fraction of the power used by a common toaster.
(Emphasis added.)
lemoncucumber 3 hours ago [-]
Bad comparison, if I needed to make toast I'd use a Pentium 4 not an M5 Ultra
porridgeraisin 5 hours ago [-]
I run my toaster for hours on end too. Love my sandwiches.
iAMkenough 4 hours ago [-]
Once a month, I run my toaster oven the entire night to take advantage of off-peak energy rates.
aenis 4 hours ago [-]
Imagine if you had 4 of them!
mrboatshoepants 4 hours ago [-]
That first bar chart showing between 15-36 tokens/second is for a 8,192 token context. Thats not nearly enough context to be useful these days
PrairieFire 2 minutes ago [-]
[flagged]
magnusokik 5 hours ago [-]
It's a good article which showcases direction of travel, i am sure mac studio will get more and more powerful in coming generations but for now it's way to expensive for local setup.
kingcauchy 5 hours ago [-]
yeah especially because it's not very easy to configure everything sensibly for a multi-use case setup, they really need better mac containers on the mac itself
comandillos 6 hours ago [-]
A 800B model that performs way worse than other 300B ones. Well...
This is still a welcome model, even if I won't be running it at home
villish 1 hours ago [-]
Muse is significantly faster and more token efficient. That is pretty impressive considering how much larger it is. There is a lot of potential even though this release won't shake up the standings.
nico 5 hours ago [-]
Very cool
Has anyone adapted the open Muse model to work like a decision model?
Sort of like Laya but based on muse spark
_pdp_ 6 hours ago [-]
I am super excited about this.
While right now Mac Studio is not a budget device and it certainly costs more than the subsidised subscription, the promise that one day we can run these models on commodity hardware is exciting.
6 hours ago [-]
AndrewKemendo 5 hours ago [-]
Anyone know why they went with Meta models?
bigyabai 2 hours ago [-]
Apple has too many arbitrary, pointless grudges for Ternus to take them all seriously.
Google partnership for Apple Intelligence, Meta partnership for local models, and allegedly NVLink Fusion support for M8 GPUs. Maybe the White House told him that it's time to start burying hatchets. I don't get the impression that Apple's prior AI strategy under Cook made any real progress.
butterlesstoast 6 hours ago [-]
I -- love -- an article -- full of -- em -- dashes.
shagie 5 hours ago [-]
If you type the following into Microsoft Word:
> In this post, we will show how to run the latest frontier-class model - Muse Spark Open, with nearly a trillion parameters - right from your desk, using the latest version of MLX, released today.
You will get
> In this post, we will show how to run the latest frontier-class model – Muse Spark Open, with nearly a trillion parameters – right from your desk, using the latest version of MLX, released today.
Not everything with em dashes is the product of generative AI... some of it is the word processor correctly following the rules of grammar and punctuation.
NekkoDroid 2 hours ago [-]
Do note that those are en-dashes and not em-dashes.
ManuelKiessling 6 hours ago [-]
If used correctly, I find the em dash one of the most elegant and reader-friendly ways to structure certain parts of a text — that it got such a bad reputation lately really is a shame!
jonathaneunice 5 hours ago [-]
Em dashes—love ’em!—but they do. not. need. personal. space.
isodev 6 hours ago [-]
Preparing to upsell those very expensive Macs coming later this month, are we.
The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.
I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)
I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.
There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.
I switched to Gemma 4 instead.
Nothing about iCloud Photos is private.
I recently installed App Tamer which is helping a bit, but it doesn't allow me to tame everything. I just checked now and duetexpertd was trying to use nearly two full cores. I have all iCloud integrations turned off, I do not use iCloud at all, and I do not have an iPhone, so I am flabbergasted that this iCloud desktop-phone syncing process runs at such high core usage in the background with extreme regularity. Proof: https://imgur.com/VkBh9bi
Unfortunately, this does nothing for services such as WindowServer which now often takes up 50-100% of a core just on its own.
I have reinstalled and upgraded to the latest version. This MacBook is only a few months old.
While typing this message and literally just having open Codex with one running task that is not doing anything substantial, and Firefox, with no high CPU usage from any tab, my laptop got so hot that I had to use another installed program, Stats, to manually bump up the fan since macOS also has absolutely garbage native system fan controls.
I have fibromyalgia and seem to be sensitive to hot keys; my Thinkpad regularly getting to hot to touch with even light usage is EXACTLY why I went and bought a MacBook one morning in '23. One morning I touched my keyboard and realized I was tired of singeing the nerves on my fingertips and having them hurt for hours. So now in '26, finding myself in the same exact position but with a much more expensive computer, I feel like I am living in a corporate hellscape nightmare.
One thing I have noticed is all of my cores can sit idle except a single one and that will still heat up the machine a lot. If the hardware is designed as such, the software MUST be designed to work around this and not blast a core for 10 minutes straight every time one of a dozen or two misbehaving system processes wants to complete a background task. If the software is not designed for this, the hardware is incomplete and faulty. This would be true if it was shipped as firmware; software doesn't get a free pass here.
But now Pages and Numbers recommend document titles for me, so.... worth it?
They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.
Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.
So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.
Hint: money is not the only reason.
Jeez.
I think both points are very incorrect. Apple has a massive moat. And Apple is making hardware (specifically, CPUs) that are two years ahead.
Apples largest competitive advantage has always been and always will be - trust combined with epic marketing to make the vast majority of people aware of it.
Trust of quality, privacy w.r.t data etc.
I’m sorry but I’ve seen so many bad takes re. Apple over the years. Many of you think you are so smart on topics outside of your domain - but you’re not.
They spent decades learning to eliminate a 1000 headaches for themselves, and many for their customers, by accumulating vertical integration:
The co-design of their own comms, processors, devices and OS variants. Their own integrated supply chain and sales channels.
The better the results, the more invisible or clock-like their delivery.
Very easy to overlook. But nobody can match them on their terms.
(Nintendo also applied a unique set of internal advantages to generations of successful devices: vertically integrating humble components, highly scoped software, a strong theory of fun, and deep branding with novel polished interfaces.)
Lmao
> Many of you think you are so smart on topics outside of your domain
Turn the mirror around
Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.
Apple can chose not to train their own model to learn how to sell a better shovel.
just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).
And usually, most of a company's revenue comes from the heaviest spenders.
So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.
I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.
$2-10 billion is meaningful for any company. Even one as large is Apple.
Especially if it's a first step towards a stronger foothold in the AI inference space.
They can then expand down market to grow the pie further.
For example, with truly dedicated inference boxes.
And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)
In any case, I'd much rather run GLM 5.3 Flash on that sort of hardware. Better fit for the performance profile, and will probably beat Muse Spark Open in most uses.
I think the iOS App Store is going to be in steep decline in a few years. People will generally have a few apps: ChatGPT, Instagram, messaging, and something like a Photos/Camera app.
Everything else will be done in your AI app.
I've been hooking up more and more data and plugins into ChatGPT/Claude. I only want to use it. I don't want to use other apps if I don't have to. For example, research, buy, plan. I prefer not to use other apps.
I think all it takes is for OpenAI to make a phone from the ground up for this and Apple is going to have serious competition.
All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.
They aren’t
Apple is a high net worth consumer technology integrator that focuses on highest coherence user experience. They pick the most efficient stable technology, integrate it and sell the package
Apple only needs to lead in precisely that they don’t need to lead in literally anything else
in fact if they tried to lead in those other things they would give up their integration capacity
The vertical integration argument relies on Apple having a good architecture for AI. First it was the ANE, which everyone (mistakenly) thought would differentiate Apple Silicon from other NPU platforms. Then everyone parroted the "unified memory" talking point without checking to see if professionals actually used it. Now people are pivoting to say the next-gen Apple GPUs will finally blow the last-gen GPGPUs out of the water. Which is good, because there are 4-year-old GPUs that beat M5 Ultra in compute efficiency.
I know that "Apple is behind" is a tough pill to swallow, but they literally have no analog to CUDA, CDNA or even TPU technology. Nvidia SOCs like Tegra prove that Apple could absolutely bring GPGPU tech to low-power hardware like the iPad and iOS without giving up any real features. Apple just refuses to invest in it.
As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally.
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
I also benchmarked both on quality benchmarks (MathArena Apex / GPQA Diamond etc) to make sure the quality is what you get from the official numbers.
I'm no expert but it seems the consensus is you get a suboptimal experience under 4-bit. Please correct me if I'm mistaken here
The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse.
It’s more like an advertorial for their tech.
Apple is showing a masterclass of how to tow the line perfectly. Or more candily "have the cake and eat it too".
If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.
Same here - they are happy to partner with Meta, without getting any of the bad "aura".
To be clear, all companies do this just that Apple alone has a successful strategy here. Tim Cook personally gifted the current adminstration with a gold plaque, something none of the other "dirty" CEOs did. Such a public gesture, yet nobody would think that Apple is in bed with the current administration. Its marvelous to see this happen in real time.
The datacenters are not full of Mac studios but Nvidia hardware.
They are known for macbooks.
And they are not willing to build proper apple servers for all the build systems people have.
At my current company we compile for x86 and devs can do a full build and test environment on their laptops (which are 95% MacBook Pros) but it is unclear what will happen when Rosetta support is pulled. Now the interesting thing is we plan to move our stuff over to target the AWS Graviton (and other cloud ARM cores) as those instances are cheaper for generally the same performance (for what was are doing anyway) so the point will be moot. There will be some crossover point where having build and test servers in a rack will be cheaper than doing it in the cloud. At that point having Apple ARM based servers would be a win assuming we can build a binary that works for Graviton, et.al.
The submitter changed it to “Apple ML Research: Muse Spark Open with MLX on Mac Studio”
...and Zuck and Ternus high-fiving each other here: https://x.com/finkd/status/2110778025629347999
"We're excited for mommy and daddy to be playing nice again," Meta AI chief Alexandr Wang told Axios here: https://www.axios.com/2026/10/15/apple-and-meta-bury-the-hat...
Someone else will do the math on how many years it would take to spend $40k on Muse Spark Open via OpenRouter.
For most of us though, you're right -- the OpenRouter option is far more cost effective.
Their competitors will. Anyone locked into these two ecosystems are willingly going along with it.
I've had the same thought on self hosting for the past 2 years. Eg i could invest $10k into hardware or spend those on credits. I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.
I agree, at least for inference credits. But anecdotal counterpoint: about ten years ago I put together a system using about $1600 worth of new and used components. Including electricity that TCO has been something like $14/month. I'd be hard pressed to find 128GB of RAM, 12 cores of x86_84, and 40 TiB (raw) worth of disk, and unlimited ingress/egress traffic for rent at that rate. That's what, $0.02 an hour, rounding up?
Yes, a hyperscaler's rent-a-computer would get faster and be more energy efficient. Possibly more carbon efficient over it's lifetime, although I have no way to quantify that. But in raw dollars it would have costed me absurdly more to rent. On the other hand, my machine has sat cold and unused for multiple months when I didn't need it to be running, and I don't consume all of it's resources around the clock. So maybe I could have gotten away with a lesser amount of local compute in a Raspberry Pi or similar, and an equally small cloud box for $5 a month or something. I dunno. I don't even own a laptop or desktop, and try to use 'cloud' services whenever practical. (Edit: always giving priority to privacy in those circumstances; only data I don't care about goes up.)
There are tradeoffs. I don't use my local server for LLM inference, it's far to inefficient for that; so I'll happily pay OpenRouter per token as needed. So I do think that hosted compute for rent does have a purpose. I guess all that's to say that I don't see it in black and white; there are times to invest in local hw and times to ride on the optimized bleeding edge of what other companies are willing to host in big datacenters.
This turned into a bit more of a ramble than I intended.
I just also think that plenty of people buy clusters of Mac Studios anyway.
(Emphasis added.)
This is still a welcome model, even if I won't be running it at home
Has anyone adapted the open Muse model to work like a decision model?
Sort of like Laya but based on muse spark
While right now Mac Studio is not a budget device and it certainly costs more than the subsidised subscription, the promise that one day we can run these models on commodity hardware is exciting.
Google partnership for Apple Intelligence, Meta partnership for local models, and allegedly NVLink Fusion support for M8 GPUs. Maybe the White House told him that it's time to start burying hatchets. I don't get the impression that Apple's prior AI strategy under Cook made any real progress.
> In this post, we will show how to run the latest frontier-class model - Muse Spark Open, with nearly a trillion parameters - right from your desk, using the latest version of MLX, released today.
You will get
> In this post, we will show how to run the latest frontier-class model – Muse Spark Open, with nearly a trillion parameters – right from your desk, using the latest version of MLX, released today.
Not everything with em dashes is the product of generative AI... some of it is the word processor correctly following the rules of grammar and punctuation.