How React took down Cloudflare

There was a vulnerability discovered in React so egregious that Cloudflare in an attempt to mitigate the problem accidentally took itself down. Millions upon millions of requests per second were 500ing all from a React vulnerability. Yes, the library that once was the VN MVC now runs everywhere. And yes, this vulnerability was a remote code execution, meaning that I could execute with no authentication any code on your server. where I could read your environment variables and then curl them out to one of my servers. In fact, I bet you right now that there are still many, many websites that are running right now completely vulnerable to this because they did not upgrade in time. All right, so what is this big problem going on? Well, it is a serialization issue. Yes, the classic serialization issue. Now, I've been personally a part of a team that has hand rolled a service that I played a very large role in in creating a bug so egregious in serialization that a single request could take down an entire instance. Meaning a simple while loop could hold down a multi-billion dollar company. So, I am no stranger to when it comes to these serialization deserialization issues. They're very, very difficult. They can be quite tricky. Now, this one is absurdly tricky. So, I'm going to walk you through it. Okay. So the first thing you need to understand is that React these days, okay? Not your grandmother's React, not my React, but the new kids React, okay? The cool kids, the kids that say 67 and then do that weird hand gesture all the time that I think looks stupid and cringe. They use React 19. Okay? And in this version, the server can communicate out a payload for React to render. But here's the deal is that JSON payload can say, "Hey, this part of the payload's a promise. It hasn't been resolved yet. I'll send you that data later." the client can send stuff back up. This is called the flight protocol. And it is quite complex, but there is a very special part you need to be aware of, which is this right here. I know it doesn't look like a lot, but we're going to come back to this and you're going to go, "Oh, that makes perfect sense now." So, first, let's break it down. Now, this is the minimum viable reproduction payload by, of course, top G from Verscell himself. And in this reproduction, effectively, what happens is that you have two chunks. Now, a chunk is these little bits of data. The first one's denoted as zero. The second one's denoted as one. Now, in this there's these things called these model strings. And you can see that in this little beautiful code that I showed you. If I scroll all the way up to the top, this thing will be called parse model string. All model strings start with a dollar sign. So, this right here is a model string that wants to resolve a promise at position zero. This, of course, would be the promise. This is position zero. You can see right there. Come on, look at it. All right. And so now this is where things get a little bit tricky. The value of this is this thing right here. Now this is a again a model string that has a type of B. Now B is the one that I was telling you about. That's the one we had to be careful of. And if I go all the way back down here is this beautiful piece of code right here. But what needs to happen is that React needs to be able to actually evaluate some stuff. So it first looks to see this status right here. This status says, "Hey, we're resolved. We already have a value react. Don't worry, we got this. Now, this is going into the internal state of React. Unfortunately, the internal state of React allowed the client to set up their own internal state. So, this is where all the problem is. And this internal state right here has a nice response provided. And of that, there's these two fields right here, form data and get. Now, the get has this beautiful line of code. This little line of code, again, using the model, this beautiful model string right here, effectively says, hey, it's a then constructor. In other words, it's the promises.n constructor. So what does that mean? Well, if I go here and I type in a nice little promise right here and I go to the prototype and I go to then, it's a function. It is literally the function definition of then. But if I go to the constructor, what's that? That's the function constructor. Now the function constructor, I can pass in anything I want. I can go like this console.log and then I could pass in something that just says say hello, right? And now I can take that and I get a function out. If I were to execute that function, it actually executes my code. That's the pivotal part. So this line right here is actually pointing to a function constructor. And if you look at how the B, yes, the B is being decoded, it's being decoded by using form data.get. Well, form data.get, what is it? It's a function constructor. And what are we passing into that function constructor? We are passing in the responses prefix plus an ID. That's why whenever you look at these uh vulnerabilities, you'll see that it ends with a comment at the end because it's going to make sure that it doesn't have a syntax error and then whatever is in here will be executed. It is passing around the nuclear bomb and they have absolutely no idea. So, all you have to do is set up the prefix of your choice and boom, you get to execute whatever that code is. Now, you're probably asking yourself, h how how did this affect Cloudflare? I'm super super confused. While Cloudflare was in the process of going, "Oh my gosh, the whole internet exploding. There's this giant uh rce going around. We're going to do some stuff to help out the internet. Hey, we're the good guys. We're not the NSA. We're here to actually help you out." So, they do the classic. They change the size of a buffer. I don't know why so many things always start off with the buffer, but apparently the default size of the W uh HTTP buffer is going to be 128 KB and they go all the way up to 1 megabyte because apparently that is suggested by Nex.js. Now, with that being the case, this change doesn't really do anything to the system. They send it all out, everything seems to be behaving except for one small thing. There's an internal testing tool inside the W that was not behaving. It could not handle a 1 megabyte buffer size. And so they're like, "Ah, it's an internal testing thing. The entire internet's literally being taken advantage of and every single server that has this as a vulnerability is having remote code execution. We can turn off this currently not being used internal testing tool and everything will be fine." Well, unfortunately, this led to a really weird situation inside of the FL1, the front line written in Lua, not the one written in Rust, because that one it remained safe even after all this. You know why? cuz Russ is safe. Russ holds you and tells you you're going to be okay. Russ says, "Hey, you know what? I love you. And if you have a no pointer, I'm not going to let you accidentally put a dot on the end of that. Okay, we're going to prevent any pain you'll ever feel, but we're going to take all that pain and we're going to put it at compile time." So anyways, what ends up happening is that there's a set of requests that have a special tag called execute, which says, "Hey, go off and go execute another set of rules for us." Well, unfortunately, those other set of rules just got just they they just got turned off. So, what comes back out of here? Well, nothing comes out of here. A beautiful nil comes out of here. And then it all comes crashing down. If you look at this beautiful piece of code right here, it goes like this. If rule set action equals execute, then let's just go in there and let's get some of that action results. Okay, we want that extra rule set results. Well, well, unfortunately, this that doesn't exist due to all the previous things. And of course, since this is in Lua, there's no extra protection. And boom, it explodes. Now, first off, you're probably thinking the same thing I am. What? This was written in Lua? Yeah, I actually had no idea. Cloudflare, you're writ that that you you write your stuff with Lu FL1. That's incredible. Second off, uh FL2, obviously the new upgraded version written in Rust. It didn't have a problem. So, congrats for Rust. Hey, Rust boys. I know you guys are out there just high-fiving or doing whatever you're doing. So excited that you actually have yourself something in production and you know it's working well. The outage from Cloudflare, of course, was taking out around 22 to 25 million requests per second or just 500. That's a ton of requests. So, a decent portion of the internet was just experiencing some issues. And this all came back to React having a vulnerability in a serialization deserialization format. Now, Roman points out something that's pretty interesting, which is that Java actually already had this problem. This actually already exists. In fact, there was a paper written in 1994 going over all of the reasons why treating your server and your client as the same machine and you don't actually need to know where objects are will result necessarily in a bunch of disastrous operations and ultimately a bunch of voodoo that you have to do on top of your code to make things work. Absolutely incredible insight from Sun Microsystems. Again, Sun Micros Systemystems, they just were ahead of their time on everything. Hey, I wanted to give a special shout out to Lackland Davidson. Apparently 100 plus hours went into this effort of figuring out this vulnerability. That's awesome. Incredible find. Absolutely amazing that you made the entire internet scramble, but even better. You probably saved the entire internet from a bug that would have existed pretty much forever. I don't think anyone would have discovered it without your snooping. All right, I also only have 9,000 subs left until I hit a million. By the time this video goes out, I'll probably be a lot closer. So, just press the button and if I'm already there, say, "Yeah, say something." A decent hairline. The name is I hope I keep my hairline for a little bit longer. A gen. The two biggest problems in computer science is cash invalidation and pronouncing things. I personally spent all day on teams arguing whether it's SQL or Squeal. Well, I have great news for you. Today, we have a company that solves both of these problems. Convex. Uh, Prime. Pretty sure that's Convex. >> Teach, get out of my yard. Convex lets you ship your backend without a back-end team. Its real-time database makes it easy to keep all of your queries up to date. Convex allows you to stop worrying about complicated SQL and untyped messes. Instead, you can embrace Convex's deep TypeScript integration with type completion and validation and database queries that are just pure uncut TypeScript functions. Try Convex today in your language of choice. TypeScript, Python, and even Rust. Visit convex.dev.

Checkout https://convex.dev today's sponsor! Convex, makes it easy to store your data and keep your backend and frontend synced in realtime, both locally and remotely! https://twitch.tv/ThePrimeagen - I Stream on Twitch https://twitter.com/terminaldotshop - Want to order coffee over SSH? ssh terminal.shop Become Backend Dev: https://boot.dev/prime (plus i make courses for them) This is also the best way to support me is to support yourself becoming a better backend engineer. Great News? Want me to research and create video????: https://www.reddit.com/r/ThePrimeagen Kinesis Advantage 360: https://bit.ly/Prime-Kinesis