That's It, I'm Done With Serverless*

the only thing about me you know I love serverless if you know anything else about me you know I hate slow apps which is why ultimately by the end of this year I'm hoping to have moved off of serverless almost entirely so what am I going to do instead not going to host servers right well you know better I'm not going to host servers I will never touch kubernetes or terraform or any of that chaos around provisioning boxes again I'm just going to move everything to the edge well why though let's talk about it so

the first thing we need to break down is what do I mean by The Edge when we talk about Edge we're often talking about one of two different things the one I'm usually talking about is The Edge runtime but there's also the edge location which is running servers way closer to your users when you use something like Lambda you pick a specific location that the serverless functions run in Like Us West tour Us East one that location is where the actual server is spun up to run your code quick and if you have users in

other places they're all going to be hitting that same server unless you create different regions and Route traffic to those different regions generally that's probably for the best because most databases are also in one region both Edge you don't really think about regions things are distributed all around the world there's I think like 200 plus different locations for Edge functions on cloudflare and many more on other providers too and the benefit here is the function that is getting the request and sending a response is way closer to the user that said I don't care that

much about global Edge functions in terms of the location the functions are in as the infrastructure is much less interesting to me than Edge the runtime because edge of the runtime means I don't have to spin up a Linux box to run my code with Edge you have isolates that can run in parallel on a given box and easily taken code immediately process it and respond whereas on Lambda and serverless you actually have to spin up instances using the firecracker chaos that Amazon invented a while back and as incredible as that technology is the isolates

that you have with something like Edge runtime are much more powerful AWS does have an Edge product they call Lambda at Edge I want to be clear about this though it is not Edge the runtime it is just edged the location and it's not even really at the location because AWS has such a limited set of of locations they deploy servers to so when I talk about Edge and I'm talking here now about moving to Edge what I'm specifically most excited for is moving our functions to the edge runtime so that we no longer need

to hit cold starts to process requests and send responses to our users one additional thing with Edge that makes it really valuable cost the cost of lambdas is not cheap it's not expensive but it's certainly not cheap you're paying per hour of time spent doing requests whereas with Edge you pay for the number of requests and the amount you pay is very small on versel for one million requests you pay 65 cents that is hilarious the cost of doing things on edge is insanely cheap it's hard to do a one-to-one comparison because with lambdas you're

paying per minute of compute and how much compute you do per request can vary but I promise you it is hard to make a workload that is more expensive on Lambda than it is on edge and cost is by far one of the best reasons to move things to Edge if you can and things like rate limiters and bouncers should almost never be run on Lambda if you can avoid it simply because of the cost difference of running on edge instead but is this really worth it it's like what's the difference what's the cost why

am I even bothering making a video well Edge has some gotches from the one megabyte limit and how much JavaScript you can ship to an edge to the distribution by default causing longer round trips there's a lot of things we have to think about and worry about now as we use Edge functions let's talk about a few of those let's say we have a user in this case a circle and a database in this case a square this big circle is the world these two things are far distance from each other that's not the biggest

deal though especially if the user only makes one request but the user isn't going to directly request from the database unless you're using something like Acera don't use something like Acera and as such the user needs to hit a server generally the way we deploy things most of the time for doing things right simple so we have a server that's really close to the database these two things are usually going to be in the same region we'll call this Us West 2. so when the user makes a request the user hits server which can immediately

not immediately that's a form of connection which can take some time but it's in the same region so once it's formed a database connection let's say it has to make like five round trips because it does a query to see if you're off they're not when you come back you're like yes this user's off they can continue so it's like okay they're requesting profile information for this room do they have permission for the database says yes they have permission for this room they're like okay then we want the data for this room at which point

the database responds okay here's the data for that room and if this database is nice and close like it is here that's great let's say the server gets moved really close to the user this looks pretty good because the user is much closer to the server and theoretically they can get a response faster but you might have noticed the actual amount of distance traveled here has gone up a lot because there are more back and forths here between the server and the database than there are between the server and the user so if that is

the case which it is on a lot of services as you waterfall your queries the server being closer to the database is often significantly better than the server being close to the user just by the raw amount of distance that has to be traveled by your network requests so if we have Edge servers that are running all over the world world like we have tons of these all over the place we have one in here but we have one up here too depending on where you're located and what that closest Edge is this can get

really nasty really quickly so how do we deal with this if we're using Edge functions doesn't this look objectively worse well thankfully there is a very cool solution to this now called Regional Edge functions for cells are pushing was pretty hard I'm seeing more and more people taking advantage of it the goal of regional Edge functions is to take your Edge functions and limit them to a specific region so it says okay our Edge functions can only run in and around West too and by limiting in that way yes the user is no longer getting

the edge benefit of having the server really close to them but the user is now getting the edge benefit of the cold start being milliseconds instead of seconds that is a massive massive win but you don't have to think about where this Edge is going and not just put everything in the same region or just do it's fine this effectively makes the behavior location wise nearly identical to Lambda which solves most of our problems this is a project for cell built to test the database latency C for Global Edge versus regional edge with a bunch

of databases in the same region or not depending on the database so in this case we're going to pick Planet scale love them they're a channel sponsored poorly transparent and we can test the latency from a global function versus in a US East Regional function so the regions the same as the database it's not making a database connections using the HTTP proxy or HTTP endpoint we'll talk about that more in a bit but I want to show off what the implications are here so if we run with a single query so this is going to

run one query from The Edge function to the database and then get a response and time it and we'll see that the global latency is a decent bit higher than the regional one for the processing time which is how long the edge function has to run because that edge function has to take more time to send the request and get back its response but if we look at the end to end time which is from when the request happens on the user's device to when they get a response the closeness here is actually significantly closer

because we've taken that one query and moved it from my device to the server so the big difference here is this Gap but the that time is still eaten by us as users regardless but what happens when we change it to two queries so now the edge is going to run a query get some response and then run another after using the data that it got in that first response things are going to look very different now so we can see here with the two query example that the gap on the latency distribution for The

Edge versus US is gotten much wider almost twice as wide silly kind of makes sense looks we've doubled the amount of queries that are being done and as we see here there are spikes for the global but it's still not that big a gap but we are now consistently seeing that the regional Edge functions are fast actually it's a bit bigger Gap than I thought it's like 3x pretty consistently two to three x difference between Regional and Global for your Edge functions so what happens when we bump it to five queries when things get really

interesting and we're gonna see a massive gap between the edge functions running in the region and the edge functions that are not and this is pretty crazy because we're doing five back-to-back queries that are blocking on a function from the database getting fresh data and we're still getting a response back to the user in under 200 milliseconds consistently under 400 pretty much always which is nuts for doing that much data in a blocking Fashion on a server of any form this is insane performance and to consistently be seeing numbers like this when we test using

tools like this one and when I experience it with the things that we've built Regional Edge is a really really good solution to one of the biggest Edge problems which is also one of the biggest Edge benefits the global nature so if the reason you're looking into Edge is faster responses and better connection methods rather than the closeness to the user here you go problem solved what's the catch so we already talked about the region thing and we showed you that it can be worked around thankfully region proximity uh it was uh playing at Scales

what I was testing with originally original proximity solved with regional Edge funks so what else compatibility we solve the regional let's talk about the compatibility the big thing to know about Edge is it's not node so a lot of the things that you're used to using in the node environment are not going to work things like built-in crypto methods things like FS you lose a lot of the stuff that we rely on every day and even I've run into problems with some of our deployments where I can't use FS to read a markdown file to

parse it inside of deployment and instead I have to do a build step to get that file into the JavaScript world before the Apple location ships on the edge these are things that can be worked around and we've been seeing more and more work done to improve this in fact versel just shipped a bunch of polyfills four different things specifically buffer so that packages relying on Buffer will still work on the edge we're making progress here but we're still making that progress but there's something that's similar but not quite compatibility that's important to note I

hinted at it a bit there I should say no native runtime layer what I mean here is you can't shim out to C plus code or run some Rust binary in the background or do a lot of the other things that we're used to with everything from like Cypress to electron to more importantly Prisma in its current architecture Prisma will never be able to run on the edge simply because Prisma runs a giant rust binary that has to run on a real server and you can't run that inside of one of the V8 isolates that

exists within Edge functions so something like a traditional database connection where you're making a SQL connection to a database and maintaining that to send queries and responses back and forth that can't happen on edge that's just not an option which is not just fine it's actually kind of exciting because this has been taken as a challenge by a lot of the companies and individuals we talk about on this channel all the time one of the biggest is our friends over at Planet scale who are actually the ones I was testing in this demo here how

am I using Planet scale if I can't connect to the database well they worked really hard to solve this there's a library I've talked about a bit before database JS by Planet scale and this Library isn't necessarily interesting as a library so much as the infra behind it that powers what it is capable of the TL DR of Planet scale serverless driver is that you can just post a request to an endpoint and get back a response that post can just be a SQL string so in this example we await connection.execute select one from Dual

where one equals whatever you pass for the variable here is a simple syntax to send SQL strings to a server and get back a response without having to form a connection it is hard to put into words how awesome this is and it's really cool to see all the stuff people building on top of it including the Keely driver that is being used here as well as the work that drizzle orm is doing to support all this stuff as well it's really really powerful and I I'm beyond hyped on what I've been seeing what is

Edge Edge is two things Edge run time and Edge location location for Edge is you run lots of servers in lots of different places runtime is you can spin up one of those functions and get a response out of it in milliseconds instead of seconds these are different things people seem to think the word Edge means just one or the other it doesn't it's referring to both of these things which is annoying but I don't care about Edge location anywhere near as much because any real application is going to have to worry about data on

the edge which means any real application is going to have to worry about where these functions are running or they're going to do some crazy with distributed data which is almost never worth it and as a result this becomes non-viable for a significant portion of the applications on the web and that is fine it doesn't mean it's bad it doesn't mean Edge location is never going to be useful to anybody it simply means that we need to consider that edge means two things and when I'm talking about it I am almost never referring to this

so to be very clear I don't talk about this much because I don't give a so if you're trying to sell me an alternative Edge solution that does just this and not this I don't give a I genuinely don't I just couldn't care less if we take a breakdown of a slow request on a T3 app today so here is a request breakdown so you just makes a request user receives response what goes down on this line and to be clear we're doing this for a bad cold started request so we're going to say this

takes 10 s total because this is what it looks like for a bad case on like Ping or kyle.com or any other create T3 app so the first thing that has to happen and we will encode we will encode Transit time between server and client in blue so here is the time spent requesting to the server it's all gonna be napkin math but this is from what I've seen from tracing playing with and doing all these other things so request the server is some amount of time this is the bad case so we're going to

be doing a cold start so here I'll fill this with pink for a cold start time and then we have another thing here Prisma spins up and connects to 2db orange for things the server does request sent to DB and response received and then there's like an additional render step which is going to be I'll make this a tiny bit bigger to be fair compared to this make this minuscule that little piece there is actually render format output and then the same time at the end here send to user cool this is a rough idea

of the pieces that result in a request taking this long on a service like Ping today you have an amount of time that the request takes to get to the server a much larger amount of time that that server takes to cold start if it's not warm which there will be times where it isn't period That's the nature of building things on Lambda you can never guarantee things will always be warm that Lambda once it's gotten all of its code and has everything it needs to spin up now connects to the database alance is connected

to the database it can finally send a SQL request to the database get the response format render do whatever it needs to be it send HTML to the user or send Json to the user and then actually sends it to the user this is not the fastest thing so what does this look like on global Edge so if you're going all in on cloudflare your Edge is now global well let's do Global Insurance let's do the ideal this is the uh Regional Edge I will say One S Max because that's fair so the first thing

is this is going to stay exactly the same because the distance between the user and the server is much smaller or it's not smaller it's exactly the same so this does not get any smaller when you use Regional Edge Regional Edge request to start's the same cold start time there it's not truly a cold start and they'll fight and I know there's going to be cloudflow employees in the chat saying this isn't really cold start so what I'm going to do to suffice them is I'm going to put cold starting quotes don't come at me

next is connecting the DB except funny enough this step no longer exists because we don't connect to the DB we just send it a request so this becomes there and to be fair this will be a tiny bit longer because it also has to sign https and like SSL cert check so we can expand that a tiny bit this will be exactly the same and this will be exactly the same you might have noticed this is a lot shorter now like a lot shorter even though these blue things are still the same size the actual

traffic from the user to the server is the same distance all the other steps have been flattened to be way way way faster so what does this look like on the global Edge so the cold start if you're saying the cold start happens in the handshake cool here to make the Nerds particularly happy because like like to be clear the pushback I get I want I want you all in the comments of YouTube to understand what I go through when I talk to cloudflare employees and such here's what they're asking me to do they want

me to specify that on the other side here in parallel an SSL handshake also happens so as you see here in this step that's so small it doesn't matter two things actually happen in parallel do you feel better now Jacob is this now accurate cool anyways yes your SSL handshake can have it in parallel to the edge starting up awesome regardless we're talking 10 seconds to one second Max here and if you make multiple parallel database requests in here this can get bigger and smaller but it's not going to be based on or it's not

going to be exponentially bigger and smaller like it would would be in the edge here so let's do the next one copy this and paste it so uh Jacob and friends at cloudflare advocating for with global is make this way smaller cool let's do it we've made this way smaller now so the issue here and we'll be very clear about the issue is this request here for every database request we make so we're assuming right now we're making one all the time we saved here is now getting put here instead so whatever the difference was

and I guess we have the difference right there it's that difference but it's that difference twice because it has to account for the other side as well and then finally we got our response so if we make a single database request we have taken that travel distance as I showed in the diagram here if we just make one request and we move the server somewhere else is not a big deal we've just moved who's doing most of the travel but if we're doing multiple requests here like will often happen in almost any service then being

closer helps a lot more because if we go back to the diagram here let's say this does two requests every time you do an additional request it doubles this so if move these guys down we do two requests that's barely a dent on this so you barely even feel it there we go two requests and if we do this here it's worth noting I made this bigger because we have to SSL handshake once but once we've hand shook you could even argue that that's like a different step so we could argue there's like the handshake

step for the initial transaction here I'm not going to label it because I'm lazy but then all of these will be exactly the same as they were otherwise and we end up with much faster overall responses still but here's where the catch happens if we use Global Edge I want to be very clear about this because this is why I think a lot of people push back on edge and I think they have good reason but we had this as the like additional time because of the distance and we do two requests it's a pretty

massive difference because if this request took 40 milliseconds in the same region and over 100 milliseconds in a global region you're going to be eating that for every additional request and the result here is a significant full cycle penalty as we saw in the charts here this is pretty evident to me that unless you have data located on the edge alongside your Edge functions then Global Edge is now net negative for a lot of applications but that doesn't mean we shouldn't use the edge runtime so what am I doing about this why am I talking

about this now and what does it look like for us to start moving to Edge well the first thing we have to do is handle the dependencies that aren't Edge ready yet the biggest one that we've talked about a lot is Prisma I do want to shout out the team I know they're working really hard to fix a lot of the things that make it hard to run on edge specifically finding ways to run the connection layer on a different server so that you can connect to that through the edge function with a fetch request

that will significantly help but we still have a lot of work to do before we're quite there and in the interim there are a lot of different solutions we have right now to start running things on the edge for databases we have kiesley and drizzle which are both phenomenal solutions for auth we have clerk which is then after a little bit of nudging gotten very very edge ready and they've been a reliable off provider for all of our Edge deployments we have upstash which has a redis connection layer that is fully Edge compliant and compatible

and honestly I'm not finding many packages now that aren't ready for the edge every everything I use from next to kiesley to drizzle to Clark to a lot of stuff we talk about here almost all of it is ready for the edge now and as a result the applications we've been building on the edge are just hilariously faster I know I show this one a lot but y'all might have played with the first app I built on stream a while back roundest I want to be able to see how long it takes for the actual

content to show up in the page that was a pretty rough cold start I didn't even have the network tab open but that finished in four seconds for a hilariously minimal application pretty sad but if we compare this to a different app I built which to be fair isn't fetching data from The Edge to start it's loading it in client but the result is just hilarious like this is milliseconds now instead and that like final chunk after that's all from like Chrome extensions if I go here and do it 300 milliseconds 200 milliseconds or if

I uh what's the edge deployment for chirp that I made uh this is a deployment of a Mojo that appears to be broken probably people spamming content they shouldn't but it is getting a database response in that time and that database response is literally coming in 500 milliseconds or less fetching a ton of data so the page is crashing I'll go deal with that later it's on the deployment of played with much recently but even like marker thing marker thinks a new application that we just shipped that also runs on edge I guess it's not

signed in at the moment and you'll see the page response times are absolutely hilarious for fetching data and actually rendering it it's insane the response lines we're seeing from things on edge no matter how much traffic they're getting is just absurd and like the latency that we're seeing is coming from whatever random JavaScript we're loading on the client usually in this case like Twitches JavaScript yeah yeah things have been great I am really impressed with the performance we're seeing overall surprised at how smooth the adoption has been and overall just putting us in a place

where we can take advantage of never ever ever eating one of these again theoretically there's a future where cold starts can be mitigated by Lambda or things around it but until we're in that future I really really really would prefer to never have to worry about cold starts affecting my users again also this one because these two blocks tend to be where most of my applications waste their time running and when a user doesn't get a response back in a second it's almost always because these two things are holding them back so if you want

to make make your applications faster and you want to keep using the awesome technologies that we talk about here these are the problems that keep us from having that type of performance on serverless we have to talk about one more small piece how do I actually deploy this there's two ways that I deploy it one I like more than the other both are very much using versel and nexjs the way that you can make any application on versel to play on the edge it's actually pretty simple in your next config there is a runtime experimental

Edge that said versel does plan to deprecate this because they don't want us thinking about Edge on an application Level they want us thinking about Edge on an individual route level the way they recommend doing this in the way that I've seen it in most of their demos has been on a given page you export const runtime equals Edge and you can put node here if you have something that's incompatible with Edge on just this one route which means that on a route level an API route level or a page route you're able to tell

versel if this code should be thrown on Lambda or on edge which allows you to have pretty hilariously granular control over how how your applications are deployed the result is just hilarious performance I've been beyond impressed with what I've seen but it's so simple to actually move your applications to the edge I yeah I've been really impressed if you're on other systems there are ways to deploy you're probably going to have to spend some time setting up cloudflare yourself though to me versel is the easiest way to deploy on cloudflare's edge so I use cloudflare's

workers Edge functions through versel by adding a key here or honestly usually putting it here until they tell me I can't anymore one last request from versel I personally would really appreciate a way to opt all of my functions into Edge as the default runtime and have to opt into node rather than the other way around which is what it is now where everything is node by default knife to opt-in to Edge y'all were down to make that change or at the very least leave this here for me I would appreciate it because I don't

want to have to manually go through and declare Edge as the method of runtime for every single page route appreciate the help there super hyped and what versatile's been working on here I hope this one was helpful I've been holding this one back for a bit and this is why I'm moving off a serverless for the year and trying to focus all of my efforts on building great applications on the edge if you want to hear more about what makes this database performance so important I have a whole video dedicated just to database performance on

the edge here check that out if you haven't already thanks as always peace