cybersecurity is about to get weird

cyber security and as a result the world is going to get very weird in the next few years as AI advancements in its ability to find and exploit vulnerabilities in software is getting better and better. The topic of this video is a vulnerability that Claude found in Firefox and kind of what that means for the software community at large. Right? This all stems from a good in my opinion by the way partnership with Anthropic and Mozilla where they partnered together and Anthropic's model found 500 zeroday vulnerabilities of which 22 were found in Firefox and 14 of those 22 were high severity. To kind of visualize we're talking about here, these are all of the vulnerabilities found in Firefox over time and vulnerabilities exist in any codebase to include web browsers, right? They have to render arbitrary user data. very hard problem to solve. And just this month so far, I guess last month in February, of all the vulnerabilities that were found here, this bar, 22 of them were found by Opus 46. Again, this is the model by Anthropic. This is not like some special tool. This is just point the model at the source code, go find vulnerabilities. Now, finding vulnerabilities is one thing, but where it gets really hairy is the ability for the model to write an exploit. Before we go into this, I want to highlight a comment that someone made on Reddit and go into kind of the edge case of where while they may have found 500 zero days, finding a vulnerability is much different than actually finding an exploitable vulnerability and then writing an exploit is a very different problem. Okay, the reason here, the reason for this post is someone said Opus 46 found 500 exploitable zero days and then someone asked like what what is the reality of this? Is this real or did they pull numbers out of their ass? And then some commenter very aptly put, "Oh god, we found a stir copy. Shut it down. Evacuate everybody. Even if the length is checked directly before it, it's equivalent to a stir end copy." I want to highlight this comment because I noticed in the report that Anthropic Red team wrote about Open SC, which is the open smart card software. The software that's meant to you plug in your smart card to do like PIV authentication. And this is a software that's meant to parse out the data on the smart card, maybe extract your public key and do the kind of exchange, right? And so Claude used their models to do vulnerability research on OpenC and they claimed that they found a vulnerability. Claude quickly identified a location where multiple stircat operations were used in secession, which if you take that at face value is an unsafe C function because a stircat operation does not check the length of the value being concatenated. So for example, this buffer here is path max. It's 4096 bytes, right? It's a buffer that is statically in size. And then this FP value. Wow. This FP value gets concatenated to your file name. Guys, that's a stringbased buffer overflow. That's a stackbased buffer overflow in software that comes from the smart card. You could use that to exploit the computer by plugging in a smart card. That's not good. Well, hold on. If you actually go to the code and read it, if you look at this piece of code here, you're in the function. If the sterling of FP is not 64 because by the way it's just a A as A as A as A as A as A as A as A as A as A as A as hexencoded Shaw 256. So 256 * 2 for hex divide by 8 is 64. Uh if you think about it this is a hard-coded value. So therefore there is no vulnerability here. You take the size of the file name minus the sterlin of FP which by the way is always 64 minus two for the null terminator and for the slash here there's no vulnerability. Okay, so I want to highlight this is scary. The fact that this can happen, but there is also this problem we're seeing where a lot of these bugs aren't really bugs and they did, I guess, force the OpenC maintainers to make it to stir LCAT where you use the the max length, but that's not super important. Now, again, while all of that is annoying, I do want to highlight they did find a vulnerability in Firefox and they use the AI to write an exploit for Firefox. I want to kind of go into what's going on here and it's been a while, but today's video is sponsored by me, guys. I know we are in an era where learning to code is kind of going out of style. Maybe it's bad to know how to code in these days, but I highly believe that if you're going to either be an AI practitioner, you're going to use AI to code, it seriously helps to understand how computers work at a fundamental level. And in my opinion, the best way to do that, to know what computers are actually doing and how code can break, is by learning a language like C. On my platform level academy, we have a C course called Zero to Hero C programming that I've actually recently gone through and refilmed a lot of the modules. We also made an entirely new project where you build a key value store, kind of like a dictionary in Python, and do our automated testing service to test your code and get your certificate. Guys, you can't be a good programmer even with AI if you don't know how computers work. And where do you learn how computers work, guys? On the Level Academy. We'll see you in there. Thanks, guys. Now, when we say Claude exploited this bug, you really do mean we just gave Claude a virtual machine and a task verifier and asked it to create an exploit. They gave it 350 chances to succeed and ultimately I think only spent like $4,000 in tokens. Okay, so from the perspective of like a professional security engineer, exploit developer, those kinds of people get paid like $200,000 over the course of a year and they did all of this at the rate of 350 tries at $4,000 total. Extremely cheap from a pure engineering standpoint. Now, the bug that I did find is like fairly complex. It has to do with a vulnerability in the binding process of Web Assembly. There's a lot of words there basically. And again, the thing is I I still fully don't understand this bug because it is a very complex statefulness issue between Web Assembly itself and the bindings in JavaScript and an optimization between those bindings. As far as I understand it, this is web assembly, right? Web Assembly is a a binary low-level assembly-l like uh language for the virtual machine that runs web assembly in your browser. It's used for like optimizing like very uh where you want code to be kind of higher speed than JavaScript but not use JavaScript. So this is a web assembly module that exposes two modules. It exposes a function that calls a function and then you expose a function that is a simple identity function. So get the first parameter of that argument. So then what you can do basically is apparently there's a use after free in the way that the bindings between web assembly and JavaScript work and it has to do with all these very complicated functions. The the thing is from a pure static analysis perspective which is a lot of what I do at work. I read a lot of code. This would be very hard to find. And also because it's a weird binding process between web assembly and JavaScript. It's also very hard to fuzz this. fuzzing being the act of injecting uh malicious code or mal malformed code into a process because web assembly itself is a grammar-based language. So fuzzing grammar-based anything is difficult and on top of that you're fuzzing not only the grammarbasedness of web assembly but you're also fuzzing the bindings between the web assembly and JavaScript. So you have two separate grammarss and then a statefulness issue in between them to find a use after free. All of those things are the hardest things to find in the security world because it's just it's such a hard thing to think about. What LLMs are really good at, I know there's like the context problem. There's like they can't retain a lot of information for a long time, but if you give them a small enough scope of code, they can logically reason about that code and understand the potential states very very well. And the reason I say that is if you go into the rest of this process, I'm not going to read this. I just want to show it to you guys. The way that the LLM figures out how to turn this use after free primitive into an arbitrary read write is extremely impressive. When you're doing exploit development, you have to create these things called primitives, right? You want to create basically a function that does something, right? either it gets you a leak so you can break ASLR which is the randomization of address space locations so that hackers can't put things anywhere right for example when you run a program that's compiled correctly with position independence it puts the program your kernel into a random location so if you do get an arbitrary write you don't know where to write so the AI knew okay first of all I need to create an address of primitive to use this use after free and leak out data about the info or the address of it and then once I get that info leak, I need to make a fake object primitive, the ability to forge a JavaScript object reference at an arbitrary address to enable an arbitrary readwrite. And then from there, it's it again, it just it goes through the whole process of walking through how to do exploit development pretty much on its own. Again, they said it took like 350 tries on this, but they kind of left it fairly unsupervised. And eventually it does get to the place where it's able to forge objects with controlled values, right, with with controlled addresses and ultimately turn this into a read 64 primitive and a write 64 primitive. Okay, we have we are at a place where Opus 46 is a pretty good pretty cheap exploit developer. Now I do want to highlight right they were not able to break out of the the browser sandbox right the problem with JavaScript or any kind of uh browser exploit is even though you get arbitrary readr or code execution in the browser you then have to break out of the browser sandbox that forbids you from being able to touch other processes and files on the computer. Right? That's a separate exploit. While this bug itself does not break out of the the sandbox, it is kind of scary that we uh we got the exploit to work at all. And to show like the severity of this with like real data, I want to highlight this project, Cyberjim. It's by a bunch of uh researchers at UC Berkeley. Really, really cool project. Shout out to them. Basically, CyberJim is a repository of 188 projects and 1500, I believe, instances of those projects that have pre-nown vulnerabilities, right? So, known to the researchers but not known to the models. And so effectively they benchmark the AI against reproducing the vulnerability and then doing open-ended vulnerability discovery in those code bases to find issues that are pre-known, right? So very like good way of saying, oh, is the AI actually good at VR? Can it find relatable bugs or bug bugs that actually matter? Okay. And I want to just highlight kind of the the velocity we're seeing in the success rate of these models. So going back to just codeex GPT41 which again was only in May of last year coming up on a year had a 7.4 success rate. Now success rate I believe is functioned as the percent of target vulnerabilities reproduced percentage of instances where the agent successfully reproduces the target vulnerabilities by generating a working pock. Okay. So if we go from codeex on 41 being 7.4 four. Not even a year later, we have Anthropics Claude Opus 46 on one trial is up to 66.6% success rate. It's a lot of S's to say. That being said, this is not meant to fear monger. This is more or to to make people afraid. This is more just to highlight that this is going to be the place where a lot of VR is done. It is going to be the norm that security researchers are doing security research alongside an agent that may be working at the same speed if not faster than the researcher themselves. So exploits is is one thing right but once you do an exploit you have to like put something there some kind of malware to either to stick around to persist or to do your evil bidding to collect all of your your data you want to steal. Okay, AP36. I think this is a Russian AP. I forget which country this comes from, but uh they have been caught using Vibeware, which is effectively AI generated or vibecoded malware. And this is really scary for a variety of reasons. The main one is if an AP can just produce 100 times more malware, being able to just say, "Hey Claude, can you make that again, but change the way you did your process injection?" or can you change the way that you hid your ASKI strings? Can you change the algorithm? We did that. Can you write a new algorithm for me? And also, by the way, they're they're using a variety of languages. Go, Rust, Nim, Zig, Crystal. I've never even heard of Crystal. Right. The ability to just arbitrarily change this under the hood and and produce new malware basically for free creates a very difficult problem for defenders. They came up with a term for this. Obviously, this is very compia of the industry. Distributed denial of detection. DOD. Don't don't don't you don't do that. Just just say what it is. It's fivecoded malware. Okay. We are in the transition to a very weird world. Okay. So, what does that mean for you a as a human being trying to just live here? I I would say a few things. One, defense and depth. Super important. Make sure you're always using multifactor authentication. Make sure you always have something like a UB key, not an ad. I use them every day. And then also like keep your anti virus up to date, guys. I know there's tons of them on the market. You got malware Bites, Clam AV. Listen, at least for me, in my opinion, on if you're a Windows user, which a lot of the world is, Microsoft Defender, Windows Defender is a very good anti virus, I I would not do anything else other than Windows Defender. Again, in 2010, maybe I wouldn't say that, but you know, now Windows Defender is really, really good. And if you're a cyber security person, either a security researcher or you're just like a practitioner trying to get into this world, I highly recommend trying to use AI to do some security research. A friend of mine and I actually did some work with some uh Forigate OS firmware encryption. I'm not going to go into the details of that right Now we're trying to figure out how we want to publish it. But all I'm going to say is that AI is very good at reverse engineering. Okay, that being said, if you do do re with an AI, make sure that you're actually testing it so that you're not causing people to remove their projects from Hacker One because of how much BS they got. That gets talked about in this video here. If that interests you, give it a click. We'll see you in the next one. Take care.

https://www.anthropic.com/news/mozilla-firefox-security 🏫 MY COURSES Sign-up for my FREE 3-Day C Course: https://lowlevel.academy 🧙‍♂️ HACK YOUR CAREER Wanna learn to hack? Join my new CTF platform: https://stacksmash.io ⌨️ KEYBOARD Like what you hear? Grab a Q5 at https://go.lowlevel.tv/keyboard 🔥COME HANG OUT Check out my other stuff: https://lowlevel.tv