cheap servers with AWS Spot

we all love saving money does not matter if you're a broke boy or if you are rich saving money just feels good and I see you all out there complaining about your AWS bills you're always like I'm paying too much money to Jeff Bezos well there is a way to give him a little bit less money it's called AWS spot instances and we're going to talk about how to use them in this video we're also going to talk about the business of AWS and how they sell their servers so you understand why this cheat code even exists and of course we'll get into some technical details on how to actually use it all right let's get into it all right we're going to talk a little bit about business and I know you're just an engineer but I believe in you I bet you can understand the business stuff you can do it so let's give it a shot here uh so let's say you're AWS right you know they have data centers we're going to do big square and a Data Center and this data center is full of servers right and I'm just going to draw a few you know here's some servers in there some smaller rectangles and the bigger rectangle you get the the idea uh and you sell these servers right you sell them by the uh by the time that people use them so this data center is full of servers let's say there are a uh let's say there's a 100 servers in here and they price their servers at let's say you know $1 an hour or $1 a day right um given 100 servers and this pricing there's a certain level of demand for the server servers let's say that they sell this many of the servers right I do I represent this right okay so at this price they're able to sell 50 servers sold 50 50 servers at $1 great the problem is they have all these other servers over here that are just doing nothing so they look at the situ ation they're like well I bet if we discounted these servers we'd probably sell more than 50 so they discount those they figure out a price they look at their data they do some math they do some economics they do some finance and they are able to sell these other 50 servers for 50 CS so now 100% of their data center is being used and they're making lots of money and they're really happy about that right full utilization hooray uh um but this is a little weird situation because the people on the right are just getting cheaper servers like why are the people on the left paying more well it's because this is not a static situation let's say the demand increases so now there's enough demand for 60 servers at $1 but they've already sold 50 at0 50 so they're missing 10 servers that they could be making more which is why they treat the people on the right a little bit differently the thing with the discounted servers is they can reclaim them from you at any time so if more demand comes in they will notify you programmatically hey the server is going to get shut down in 2 minutes and they will reassign this to the more expensive pool so obviously a very simplified explanation but the idea behind this system is they have what's called OnDemand servers and they also have what's called spot servers I don't fully get the naming to be honest but the idea here is on demand's priority if you ask for an on demand server they're going to find it for you they're going to pull it from the people using spot um and then the spot people are paying less with the risk that hey at any given time we might notify you and you have two minutes to you know give the server back to us at which point you can just go if you still need the server you can still go and be an on demand user and purchase it at the larger price but the system is dynamic right the number of servers in these pools change dynamically the costs of the spot instances are also dynamically pric based off of the market but this is why there is an opportunity to pay less for AWS specifically their servers because they always have some unutilized capacity so this goes even further because they also have the concept of reserved instances and this is kind of a legacy concept they're not doing this as much anymore but helps like really drive home this idea idea uh the idea we reserved is there are some people that go further than on demand they're saying Hey I want to actually upfront pay you for a Year's worth of a bunch of servers but they might not use them all right away right so they might Reserve let's say like 60 servers but they're only using 50 so then there's 10 that are just technically reserved but they're not being used AWS again will put that into the uh spot pool and they're basically making a ton money doing this right like they've sold the server multiple times and the moment the customer is not actually using it they're like reselling it again so this is great for them they make a bunch of money but then for people that uh can tolerate this tradeoff with a spot instance There's an opportunity for them to pay a lot less so to fully understand the pricing of spot I use this service from Vantage so Vantage is a great service for tracking your AWS bill they give you a lot of good insights and they also bought this other service that would just track all of AWS pricing stuff and you know they have it hosted here so if we look at the ec2 pricing let's sort it by just your on demand pricing so you can see there's onand pricing for this instance it's $3 a month uh the reserved cost so if you pay for this up front it's going to cost they give you a discount it'll cost you 89 and then they have the spot pricing so spot pricing is the cheapest right this is A1 five um and like I said this changes and and you can go here to kind of get a sense of what the current price is the reason they say minimum cost is because it uh you know it varies depending on the different availability zones and different regions depending on supply and demand there this will vary a bit but you can see in this case you're saving 50% so if you're running a bunch of containers a bunch of computes you can be saving up to 50% of that bill which is pretty nice this isn't like a 5% off little weak ass coupon right this is significant amount of amount of money and if you go and look at some of the bigger instances it's even it's even crazier so this one uh $15 $5 you know 3x cheaper so spot is great um obviously there is this tradeoff of hey they might shut it down at any time but if you think about a lot of the stuff that you're probably building that's generally okay so if you're building an API server or any kind of web service it's unlikely you have requests that take longer than 2 minutes so if you get signaled hey we need to shut this down you can just stop accepting new connections finish processing the requests it's likely by that 2-minute countdown you'll have been done processing all the requests container can shut down new one can come up it might be a new spot instance it might be uh you know on demand instance but it's typically fine with web services you you you pretty much have to handle this anyway if you're doing any kind of rolling Deploy on a new deployment it's the same scenario right you need to spin up some new containers shut down the old ones they need to shut down Gra F so you're not interrupting work and the new instances and the traffic is directed to the new instances so I think if you are building a web service great just do this use spot it's like free money on the table um cases where it's not so good is if you're doing long running batch processing let's say you have some kind of process that takes 20 minutes to complete on minute 19 or minute 18 if it gets interrupted and it doesn't finish uh that's going to be annoying because need to sort over when when it starts um so you know some cases where it doesn't work but for the majority of requests coming in from web requests from the browser bances do work pretty nicely and the best part we've got a really nice way to do this in SST and it's so easy so if you want to use it you really have no excuse uh so we're looking at the AWS cluster spot example in our examples folder and you can see it's a typical you know ad service situation but we've got this new capacity field so if you specify capacity spot that just means it's going to use all spot instan it's going to try to use spot instances for you know everything so if you set set like scaling rules on like this needs to Auto scale up to five containers it'll try to use spot for all those but we've also got more complex configuration so you can make this you can do like some interesting automation here so you can say that okay spot the weight on the spot instances is one and then the on the fargate instan so fargate is like normal that's the on demand instances uh the weight is one so in this case it'll do 50% spot 50% fargate and you can configure this however you want there's also this base property which says like okay I definitely at least want three on demand instances running uh and then from there you can start to start to weight these differently and consider the spot inance and that's all it takes so you specify this and now you are suddenly saving a bun of money like I said a lot of you might just want to consider doing this just have capacity spot uh and then that should work as long as you're able to handle interruptions in this web service which like we said most of you should be able to and that is it look I know it feels bad you look back on all those bills you paid you could have been paying so much less and it feels bad to know that thousands tens of thousands millions of dollars that you might still have but look we got a nice way to do it start using this going forward make it something you consider whenever you deploy a new service and you're good to go all right see you guys in the next one

we're talking about aws spot instances and why you should be using them