• 1 Post
  • 42 Comments
Joined 9 months ago
cake
Cake day: October 2nd, 2023

help-circle






  • It does help to know what those funny letters mean. Now we wait for regulators to catch up…

    /tangent

    If anything, we’re a very long way from anything close to intelligent, OpenAI (and subsequently MS, being publicly traded) sold investors on the pretense that LLMs are close to being “AGI” and now more and more data is necessary to achieving that.

    If you know the internet, you know there’s a lot of garbage. I for one can’t wait for garbage-in garbage-out to start taking its toll.

    Also I’m surprised how well open source models have shaped up, its certainly worth a look. I occasionally use a local model for “brainstorming” in the loosest terms, as I generally know what I’m expecting, but it’s sometimes helpful to read tasks laid out. Also comfort in that nothing even need leave my network, and even in a pinch I got some answers when my network was offline.

    It gives a little hope while corps get to blatantly violate copyright while having wielding it so heavily, that advancements have been so great in open source.





  • We do not have a rigorous model of the brain, yet we have designed LLMs. Experts of decades in ML recognize that there is no intelligence happening here, because yes, we don’t understand intelligence, certainly not enough to build one.

    If we want to take from definitions, here is Merriam Webster

    (1)

    : the ability to learn or understand or to deal with new or trying >situations : reason

    also : the skilled use of reason

    (2)

    : the ability to apply knowledge to manipulate one’s >environment or to think abstractly as measured by objective >criteria (such as tests)

    The context stack is the closest thing we have to being able to retain and apply old info to newer context, the rest is in the name. Generative Pre-Trained language models, their given output is baked by a statiscial model finding similar text, also coined Stocastic parrots by some ML researchers, I find it to be a more fitting name. There’s also no doubt of their potential (and already practiced) utility, but a long shot of being able to be considered a person by law.


  • I don’t want to spam this link but seriously watch this 3blue1brown video on how text transformers work. You’re right on that last part, but its a far fetch from an intelligence. Just a very intelligent use of statistical methods. But its precisely that reason that reason it can be “convinced”, because parameters restraining its output have to be weighed into the model, so its just a statistic that will fail.

    Im not intending to downplay the significance of GPTs, but we need to baseline the hype around them before we can discuss where AI goes next, and what it can mean for people. Also far before we use it for any secure services, because we’ve already seen what can happen


  • Building my own training set is something I would certainly want to do eventually. Ive been messing with Mistral Instruct using GPT4ALL and its genuinely impressive how quick my 2060 can hallucinate relatively accurate information, but its also evident of limitations. IE I tell it I do not want to use AWS or another cloud hosting service, it will just return a list of suggested services not including AWS. Most certainly a limit of its training data but still impressive.

    Anyone suggesting to use LLMs to manage people or resources are better off flipping a coin on every thought, more than likely companies who are insistent on it will go belly up soon enough


  • The fallout of image generation will be even more incredible imo. Even if models do become even more capable, training off of post-'21 data will become increasingly polluted and difficult to distinguish as models improve their output, which inevitably leads to model collapse. At least until we have a standardized way of flagging generated images opposed to real ones, but I don’t really like that future.

    Just on a tangent, openai claiming video models will help “AGI” understand the world around it is laughable to me. 3blue1brown released a very informative video on how text transformers work, and in principal all “AI” is at the moment is very clever statistics and lots of matrix multiplication. How our minds process and retain information is by far more complicated, as we don’t fully understand ourselves yet and we are a grand leap away from ever emulating a true mind.

    All that to say is I can’t wait for people to realize: oh hey that is just to try to replace talent in film production coming from silicon valley



  • If you think the useless appliances are bad, just take a look at more critical connected devices.

    I needed some POE security cameras, found some foscam ones on the cheap. Plug them up, go to IP, “install our app”… was pleased to find it allowed a local account without the need for an email, but found that half of my network traffic was comprised of requests to their “ivyIOT AI detection”. I didnt measure what data was going through before sectioning them behind a firewall zone.

    My fault for not having looked further into other brands, they were still a bargain and work without issue with my setup, but annoying