Apple GPT in your pocket? It could be a reality sooner than you think. Apple AI researchers say they have made a key breakthrough in deploying large...
Everyone likes to trash machine learning because the power requirements are high, but what they don’t realize is that we’re in the very first days of this technology (well, first couple decades of the technology being around, first few years of it being advanced enough to have anything to show off). Every technology that got bundled together into your phone was equally as useless when it was first invented. Honestly, compared to the development of most other technologies I’ve looked at, the pace of development in AI has been shocking.
Literally once a week, I see some news story about AI researchers delivering an order of magnitude speedup in some aspect of AI inference. The technique described here apparently allows for a 20x speedup on GPU’s.
Whispercpp works off the ML cores on the m series chips. It’s faster than my 1080ti that I have in a server doing the same things–by orders of magnitude. And it sips power.
Purpose built chips can be super powerful for their specific purposes.
Everyone likes to trash machine learning because the power requirements are high, but what they don’t realize is that we’re in the very first days of this technology (well, first couple decades of the technology being around, first few years of it being advanced enough to have anything to show off). Every technology that got bundled together into your phone was equally as useless when it was first invented. Honestly, compared to the development of most other technologies I’ve looked at, the pace of development in AI has been shocking.
Literally once a week, I see some news story about AI researchers delivering an order of magnitude speedup in some aspect of AI inference. The technique described here apparently allows for a 20x speedup on GPU’s.
Whispercpp works off the ML cores on the m series chips. It’s faster than my 1080ti that I have in a server doing the same things–by orders of magnitude. And it sips power.
Purpose built chips can be super powerful for their specific purposes.