The ROG Flow 13 is running AMD's Strix Halo APU (combined CPU+GPU) which, although capable of gaming, is geared specifically towards running Large Language Models (LLMs) locally.
While AMD GPU processing technology is considered inferior to Nvidia’s CUDA, it’s still capable of supporting AI workloads effectively. And Strix Halo is particularly useful in these scenarios because that 128gb of RAM can be allocated to both the CPU and GPU. Some explanation:
- The quality of AI model you can run is limited by the amount of RAM your GPU has access to. More capable LLMs require GPUs with access to much more RAM.
- In most video cards, there’s a fixed (and relatively small) amount of RAM. For example, an Nvidia RTX 5070 has 12gb of onboard RAM. This is only capable of loading small to mid-size models. 16gb cards can run better models, and if you’ve got $5K burning a hole in your pocket, 32gb GPUs can run even better ones.
- The 128GB ROG flow can apparently assign up to 90gb of RAM to the GPU. This isn’t as performant as a regular GPU, but because of the way RAM is soldered into the system, it can approach conventional GPU performance. The larger RAM amount more than compensates for slightly inferior compute performance. In theory he could run some of the most powerful open source and hobbyist models locally on this device.
I assume the $3,700 is after tax, as the 128gb model is listed on ASUS’ site for ~$3,300USD (it’s listed on Amazon for ~$2,800, so he’s not even bothering to shop around). This cost is actually decent for what the device does. For example a similarly speced Mac Studio (which has slightly superior LLM capabilities, but lacks a display and keyboard as it isn’t meant to be portable) is $3,700USD
before taxes. The Mac is also the objectively gayer option, so I’m shocked Nick didn't spend the extra money.
All this said, if he thinks he’s going to have his own little ChatGPT or Grok, he’s mistaken. Even the largest local models are still small and limited compared to what the big commercial LLMs, with their endless RAM-hoarding datacenters, can do. And LLMs, in and of themselves, are kind of limited in what they do. To be truly useful they must be extended with external tools so they can do things like pull data from the Internet. Getting these extensions set up locally is not easy and in many cases requires dev skills, which Nick didn’t have even before he gave himself brain damage.
I assume one of the reasons he’s getting this is to make degenerate porn that doesn’t get reported to authorities. In this he might find some success, as a huge chunk of the AI hobbyist community is just creating models to generate weird fetish anime porn. But even there, you’re not going to be creating 5 minute long realistic scenes locally like you do on Soya. You’re going to get five second-long clips staring nightmarish Cronenberg flesh-monsters.
I'll close by noting that Nick
might have been able to figure out all this stuff 5 years ago, but post galaxy-gas he just doesn’t have the cognitive capacity to use these tools effectively.
