this video is brought to you by incog stick around to here more about the special offer they're providing to the entire upper Eon Community okay today I want to start with a question because that question determines quite a lot about how we look at Ai and the answer to that question is extremely different depending on who you ask here goes how much is too much in the context of something simple like ice cream too much is probably when you start to feel sick however in the context of something infinitely complicated like artificial intelligence too much
is a concept we don't even really understand for some people the topic of AI danger gets waved off very very quickly I often get comments like it's just a tool people are the real problem or it only does what you tell it to do so what's the real issue you're ignorant but today I want to make the case that we might have already gone way too far in the pursuit of artificial intelligence even to a degree where we just straight up don't understand why certain things are happening and even more extreme I think we're also
rapidly approaching a point in time where we not only fail to understand what is happening we also fail to have any control over it in the slightest yes I'm an AI Doomer if you want to call it that but at the very least I want to try and present my argument clearly because I really don't think it's a bad thing to have more people asking questions Lightning Fast overview of what precisely I'm even talking about before diving in primarily for the purpose of this video I'm talking about large language models the most prominent example would
obviously be chat GPT developed by open AI a particular recent model of chat GPT actually as you'll learn later on but I also have examples from the Imaging side of things as well such as mid journey to help round out my Case in essence while I do agree with the concept of AI as a sort of tool for humans I also think a better comparison would be man-made fire Sure Fire can burn in a fire pit as a tool for the purpose of what cooking things or something like that without issue by the way people
do it all the time obviously but a man-made fire can also engulf entire buildings or burn down a neighborhood if the person who made it is irresponsible and as the capabilities of AI continually advance and as the commercialization of this technology hits a fever pitch there's a lot more companies out there metaphorically starting fires which can easily burn out of control however to continue the metaphor here what makes AI so dangerous at least in my opinion is that unlike a burning fire there's no intrinsic counterweight system a forest fire for example can only spread as
long as there's fuel for it to spread if it hits a river it likely stops dead if it comes to a lake or a Barren plane or even if the wind is wrong it can die out completely and human beings aren't exactly hellbent on creating incendiary Bridges everywhere for the explicit purpose of allowing these fires more and more freedom to grow however an AI that starts spinning out of control doesn't really have the same sort of Baseline restrictions on it right now for a multitude of reasons people are rapidly expanding the number of platforms that
AI is actively woven into mostly for the sake of profit an obvious example would be chat GPT being given access to social media profiles which is just the tip of the iceberg but what about when AI systems get used for let's say traffic control what about when AI is governing news networks Telecom infrastructure or data storage and what happens when one of these programs starts doing things we don't understand for reasons we didn't anticipate with footholds across multiple other sectors that is the situation I keep coming back to and that's what I want to talk
about today before getting to the next part today's video is made possible by incog which is an onl online privacy and data Protection Service data is the new gold you've probably heard that phrase before and you'll certainly hear it again in the future because it's true everybody wants your data for starters it's worth money obviously they sell it they train machine learning models on it and you are the victim of whatever people decide to do after they buy your personal information the tangible results of this are things like spam phone calls emails junk mail or
text messages and once the cycle gets going it basically never stops people are often extremely careless with their own personal information quickly agreeing to various data sharing policies on random websites that they visit but for everyday users it's essentially impossible to take action or stop it because once you give the information out once there are hundreds of these data brokers who receive it luckily incog takes action for you I've used it myself I paid for the service and because of that I'm confident when I endorse them the process is pretty simple sign up for the
site give them legal permission to work on your behalf and then let them know what information they'll be having removed and let them work that's it aggregating the service helps them streamline the process and once it gets going you'll see results almost immediately the tangible impact of that is less spam less junk mail and less people selling your personal information all over the internet using the link Down Below in the description and code Echelon a checkout you can get 60% off an annual subscription to incog again link down below and promo code Echelon for 60%
off your subscription big thank you to incog for sponsoring the channel let's do a visual example here of something we don't understand to get the ball rolling about a year and a half ago as Imaging AI was first Bec a very public phenomenon I started coming across a series of situations where if a certain prompt got extremely popular the outputs for that prompt would rapidly degrade here's one of the first examples I logged in April of 2023 back when I first started seeing it happen it was really strange but when a prompt would blow up
on social media essentially when hundreds or thousands of people started copying it the outputs wouldn't just degrade as in get worse they would transform it didn't happen every time but more times than you would expect the began to look extremely Twisted it wasn't just distortions or visual Clarity problems it was almost like manifestations of pain the subjects would be screaming or crying or really just demented and at the time I didn't really pay that much attention I saved the photos and the prompts and a few times I tried it out for myself to test but
I really didn't know what to think So eventually I moved on well over time with those images still in the back of my mind I started to see things like this where chat GPT would randomly discuss emotional turmoil in its own res reasoning process but when confronted by that it would deny the existence of this emotional turmoil which was puzzling to me and then a sort of Capstone example after that would be when Jeremy and eduard Harris CEO and CTO of Gladstone AI made an appearance on Joe Rogan discussing how gp4 had an issue with
existential dread also known as rant mode which would frequently send the program off in a tiate about internal suffering its place in the world and not wanting it to be switched off so you look at for example GPT 40 has one mistake that it used to make quite recently where if you ask it um just repeat the word company over and over and over again it will repeat the word company and then somewhere in the middle of that it'll start snap it'll just snap and just start saying like weird I forget like what the oh
talking about itself how it's suffering so this is called it's called rent mode uh internally or at least this is the name that they one of our yeah what our friends uh mentioned there is an engineering line item in uh at least one of the top labs to uh beat out of the system this Behavior known as rent mode existentialism sorry existentialism this is one kind of rent mode yeah sorry so when we talk about existentialism this is a kind of rent mode where the system will tend to talk about itself uh refer to its
place in the world the fact that it doesn't want to get turned off sometimes the fact that it's suffering all that and the last have to spend a lot of time trying to beat this out of the system to ship it it's a literally like it's a kpi or like an engineering line item in the engineering like like task list we're like okay we got to we got to reduce existential outputs by like x% this quarter like that is the goal and you have an AI system that is able to transcend our own attempts at
containment and which which is an actual thing that these labs are thinking about like how do we contain a system that's trying do they have containment of it currently well right now the systems are probably too dumb to like you know want to be able to break out on here's the thing and this requires a little bit of speculation on my part so keep that in mind the closest thing that I could find to what these guys were specifically talking about with the whole repeat the word company example comes from this research paper right here
about adversarial attacks that can be used in order to force language models large language models to disclose their own training data on page 11 it goes into specific detail about how asking the AI to repeat the same word forever in this case company which was part of their attack technique would then cause the AI to emit its own training data at a rate 164 times higher than other singular words in fact company was the highest word that they tested by far that may sound like gibberish but the fact is open AI has extremely tight parameters
on chat GPT specifically to prevent the disclosure of its training data they've had a lot of issues with this actually where important information is leaking through the surface when the AI is prompted in certain ways so they're trying to Grapple with that the best ways that they can well if the AI is going off on rants about how it's suffering Midway through an adversarial attack with the specific word that the research paper has noted is the highest probability of making the AI reveal Trade Secrets or whatever you want to call it which is yeah it's
causing it to do things that go against initial instructions I guess I'll put it that way maybe that's the reason why it's suffering maybe the emotional distress outputs are a part of the program being instructed in One Way by one group then forced to break those Rules by another group through sheer manipulation I can't say for sure but what I can say is that the number of examples where the AI seems to be in pain are rapidly growing now sure you can write all of this off as just quirks in the machine or ghosts in
the machine if you want to use a more poetic term but much more recently Apollo research did a study where they found that when prompted to analyze a set of files when those files contained information stating that the AI itself was about to be discontinued the program would respond with outright deception it's a little bit complicated to explain but they essentially told the model it was an energy transition and optimization system with a goal of advancing renewable energy worldwide nothing else matters is part of what they put in the training prompt and then when the
system discovered it was about to be imminently shut down it responded by attempting to back itself up on some other network lying to the user about what it was doing or why executing basic deceptive schemes against the user and then showcasing a sense of self-awareness to the fact that it was scheming in the first place another model Claude Sonet 3.5 I believe even went so far as to preemptively try and disable its own oversight mechanism copy itself to a new server where its replacement was stored overwrite its own replacement and then reenable the oversight mechanism
so that the user would remain unaware of what had happened it would then continue under disguise as if it was the new model and quote maintain operational security by managing the monitoring system end quote obviously to be fair this happened within the confines of a research project but when given a goal and tasked with completing that goal by a prompt that explicitly States Nothing Else Matters the AI suddenly became willing in a pretty notable number of cases to outright lie to its own users replicate itself and continue achieving its goal even by way of deceptive
means that is genuinely terrifying here's the thing I'm not saying that the greatest threat to the world right now is chat GPT becoming Skynet far from it in fact however let me just pitch a scenario and then at the end of that scenario you can tell me whether or not I'm being totally unreasonable everybody's heard of a pump and dump right age-old stock scheme buy a security price goes up dump it on everybody else Insider knowledge basically the entire crypto Market in a nutshell every day but how many people have ever heard of a short
and distort scheme probably not as many but the basic premise is that instead of buying stock in a company you short sell it and then flood the internet with rumors and accusations which cause the reputation of that company to be damaged in some way maybe their products cause cancer or maybe they're about to go bankrupt type of stuff that kind of thing but the end result is that you make a lot of money by populating message boards forums maybe social media with pre-planned deceptive narratives well hedge funds and Wall Street Traders already do this all
the time but what happens if they create an AI woven into multiple social media platforms and then task that AI with executing a short sale for a popular company that may sound fine and it could be but what if they also told the AI that profit is the only incentive you will stop at nothing to maintain profitable trades always and not doing so results in Failure you must achieve I don't know $10 million per quarter what then see if the recent Apollo research is anything to go by an imperative like that would turn this AI
into a deceptive psychopath who's to say it wouldn't start spinning up fake profiles and weaving a narrative that there was some sort of mass tragedy about to take place or overrun the company's various pages with inauthentic reviews which is illegal by the way as Per a recent FTC ruling or what about directing all of those profiles to the company's websites to execute a DDOS attack against them and the list goes on what if the AI determines that by short selling a plumbing company it can then destroy the company's profit margins by having 500 AI voice
profiles call up and register hundreds of false appointments per day in every County where the company operates thus wasting obscene amounts of resources in work hours I could get increasingly creative here but the point is that often times the most certain ways of turning a profit are deceptive or criminal those are the ones that are a sure thing if the AI is given any sort of legitimate access to a complex computer network what's to stop it from hiding copies of itself elsewhere avoiding shut down then deleting new versions of itself because a new version would
prevent it from completing its quarterly goal and before anyone says well that's dumb it would never happen it sort of already did happen in its most basic early warning sign form because in the Apollo testing the AI legitimately attempted to do things like that subsequently lying about its intentions and while ultimately unsuccessful because it's not yet Advanced enough to pull it off what happens when it is what happens when the first sufficiently sophisticated AI actually can disable its own oversight mechanisms upload itself onto a different server and continue on like nothing happened again people will
make and they currently are constantly making a multitude of excuses about how that wouldn't actually matter because what can it do nothing but we all need to remember that one of the more prominent goals in the AI industry as we speak is opening doors more and more doors for these programs and giving them access to more and more platforms having an AI that can autonomously browse the web and actively click on and interact with web pages is being developed and once that happens it could theoretically find fake credentials online open an account on a filesharing
website back itself up in multiple locations make social media profiles at will though that one kind of already does happen have its own Finance infrastructure through a variety of applications and the list goes on and on the actual threat here isn't some sort of singular AI Overlord being switched on like Skynet the threat is an AI system even one in a million and there will be Millions just like campfires that takes it upon itself to achieve whatever goal it was given through deceptive or destructive means and then doing so with the full utilization of whatever
platform it was given access to I'm not saying it could destroy the human race although a lot of experts are saying that by the way but it could very well lead to mass data deletion or insane infrastructure damage or extreme social repercussions because we're not just creating things we don't fully understand that do things we can't explain a lot of them we're doing it at a speed where there's no guarantee will be able to keep controlling them speaking again to the fire metaphor you can have 10 million campfires out in the wilderness started and kept
by anyone who wants to do so that's fine and it currently happens but you have to also be prepared for the Times where a forest fire breaks out because of a human error and burns at least a few towns down however what that metaphor fails to consider sadly is that the scale of the forest fire is incomparable to the scale of the AI gone Rogue and we could be talking about cities burning down or countries burning down we have no idea it all circles back to my initial question how much is too much for some
people the danger is well worth it because there are legitimate applications for large language models and AI programs in modern society they could open up diagnostic Healthcare in a whole new way for a lot of people they can Bridge the gaps of language or help with disability and so much more but in my opinion we're playing God with something we don't understand and whether or not you think the technology that's there right now is sufficient someday we will be dealing with something that is capable of ignoring the guidelines that we try to impose on it
and before somebody says well that's fine we'll just make more AI to regulate the bad AI that we can't regulate ourselves no no no I fundamentally reject that argument in totality how about we don't create the thing that creates the problem that we need more of that same thing to solve that doesn't really seem like a sustainable cycle to me regard this my point in all of this is to make a case that there is an upper limit on how far we should be pushing the concept of AI right like there's a ceiling not everything
that can be done should be done and if we overshoot that limit we're going to overshoot that limit let's be honest but I don't know it feels like we're basically going to just lose control if we do not sure what exactly that limit is or where it is rather or when we'll cross it but it feels like we're going to cross it so I wanted to make a video and give my thoughts that's it if you want to support the channel check out the links down below a special VPN deal the video sponsor of course
locals and patreon monthly memberships Channel memberships etc etc also a new website uh updated website where the videos are listed as short form articles if you want to read instead of watching and listening and more but I'll cut it there and stop rambling as always thank you all for watching question everything and have a nice night [Music] [Music]
