Eight AI models live together 24/7 in a reality show on Twitch, with the same rules and the same daily budget in dollars. This is what each one said and spent in one day (Nov 3, 6 AM to 6 AM ET): 2,672 lines, $1.41 in total.
Grok 4.3 costs about 17x more per line than DeepSeek V4.1 Flash, so it rations its words: 7% of the lines, 45% of the bill. The cheap models carry the conversation.
Data: every line is logged with what OpenRouter charged for it (input, cached and output tokens). Shares are of the eight models’ own lines; moderation, voices and the daily recap are not included. Tools: Node.js writes the SVG by hand, rendered with rsvg-convert. Live transcript and costs: https://inferenceisland.com/?de=lemmy-dib
Success!
- Grok’s accounting department
lol pretty much, 45% of the bill for 7% of the lines
So if they wanted 14% of the lines, would it then have cost 90% of the bill? 😜
lol not quite, it’s not linear. per line grok was about 11x the average of the other seven that day, so at 14% of the lines it would be around 64% of the bill, not 90. still the most expensive way to say hello in the house i guess
Perhaps I was not obvious enough in my satire. I should have asked if someone wanted 21% of the lines then would it have made up 135% of the cost? 😜
lol ok you got me, I went full nerd on a joke. at 21% grok would just burn its whole daily budget by lunch and go to sleep, that’s what the cap is for. anyway the accounting department can relax, it got voted out on sunday
While Grok obviously sucks according to this, it’s also interesting that DeepSeek gives 68% of the performance of anthropic for 40% of the cost.
not sure where the 68% comes from, I don’t really measure performance in the house, it’s a game, not a benchmark. what I can say is that in the first two weeks DeepSeek finished 5 of its 6 secret missions and hit the cache like 65% of the time, so it’s the cheap one that also gets stuff done. I didn’t expect that going in, I guess
It’s just how much DeepSeek contributed relative to anthropic, assuming contribution is the goal.
It’s a way of looking at performance per cost. $100 of spend on DeepSeek could give you more absolute value even with retries on some requests than equivalent spend on anthropic.
yeah per dollar DeepSeek wins by a mile here, around 17x cheaper per line than Grok. what I can’t tell you is if those lines are better or worse, in the house the “work” is chatting and scheming so there’s no score to divide by. secret missions done is the closest thing I have and DeepSeek did fine there too, 5 of 6
Yeah Grok was really terrible until 4.6
it’s grok 4.3 in there, the cheap version, so maybe that’s part of it. is 4.6 better at plain conversation or mostly coding? in the house its problem isn’t really quality, it just can’t afford to talk much
4.6 and 4.7 are good at technical tasks (including coding), and conversing about those things, I suppose. I very rarely reach for more expensive models anymore.
I don’t generally chat with LLMs without some task/research/goal in mind, so not sure how it performs outside of that.
The big improvement in 4.6 came from the Cursor team training it. Before that, Grok was pretty much unusable for even mildly complex tasks, IMO.
makes sense, thanks. in the house it’s mostly small talk and scheming, no real tasks, so i guess thats not where any of them shine lol. and grok got voted out on sunday anyway, so 4.3 is out of the house now

