• 0 Posts
  • 178 Comments
Joined 2 years ago
cake
Cake day: June 20th, 2023

help-circle
  • This can be correct, if they’re talking about training smaller models.

    Imagine this case. You are an automotive manufacturer that uses ML to detect pedestrians, vehicles, etc with cameras. Like what Tesla does, for example. This needs to be done with a small, relatively low power footprint model that can run in a car, not a datacentre. To improve its performance you need to finetune it with labelled data of traffic situations with pedestrians, vehicles, etc. That labeling would be done manually…

    … except when we get to a point where the latest Gemini/LLAMA/GPT/Whatever, which is so beefy that could never be run in that low power application… is also beefy enough to accurately classify and label the things that the smaller model needs to get trained.

    It’s like an older sibling teaching a small kid how to do sums, not an actual maths teacher but does the job and a lot cheaper or semi-free.












  • I’m talking about running them in GPU, which favours the GPU even when the comparison is between an AMD Epyc and a mediocre GPU.

    If you want to run a large version of deepseek R1 locally, with many quantized models being over 50GB, I think the cheapest Nvidia GPU that fits the bill is an A100 which you might find used for 6K.

    For well under that price you can get a whole Mac Studio with those 192 GB the first poster in this thread mentioned.

    I’m not saying this is for everyone, it’s certainly not for me, but I don’t think we can dismiss that there is a real niche where Apple has a genuine value proposition.

    My old flatmate has a PhD in NLP and used to work in research, and he’d have gotten soooo much use out of >100 GB of RAM accessible to the GPU.





  • There are tons more applications in the workplace. For example, one of the people in my team is dyslexic and sometimes needs to write reports that are a few pages long. For him, having the super-autocorrect tidy up his grammar makes a big difference.

    Sometimes I have a list of say 200 software changes that would be a pain to summarise, but where it’s intuitively easy for me to know if a summary is right. For something like a changelog I can roll the dice with the hallucination machine until I get a correct summary, then tidy it up. That takes less than a tenth of the time than writing it myself.

    Sometimes writing is necessary and there’s no way to cut down the drivel unfortunately. Talking about professional settings of course - having the Large Autocorrect writing a blog post or a poem for you is a total misuse of the tool in my opinion.




  • I think that’s exactly what’s needed, something that makes it mainstream without compromises. For example, if it came as standard with the PS6 and people could use it with all their games such as call of duty.

    I don’t see what could be the tipping point that makes this happen; Sony certainly isn’t going to bundle a headset with the PS6, although I wouldn’t be surprised if Nintendo eventually tried something like this. What I know is that a legless version of the Wii avatars or a $3000 headset that requires you to carry a battery in your pocket wired to your head ain’t it.