Taco Bell: Order Middleware - Enabling Delivery Orders at Massive Scale

Welcome to, ‘This is My Architecture'. Today, I'm joined by Robbie from Taco Bell. Welcome, Robbie.

Thanks for having me. So, I love Taco Bell, but tell us about Taco Bell. Sure, so Taco Bell is the nation's leading Mexican inspired QSR brand.

We serve over 42 million customers per week in over 7,000 restaurants. That is a lot of tacos. That is a lot of tacos being made.

Awesome. So we're going to talk about a piece of your architecture today. What are we going to talk about?

Yeah, so this is a very exciting new platform that we built that we call the Order Middleware. The Order Middleware sits between different various order sources and our restaurant technology. So our actual restaurant's POS technology, there.

And it allows us to kind of route those inbound orders in an event to get into the store so the food's made for the customers. Okay, so this- Yeah and today what we're going to look specifically at how we handle delivery aggregator orders. Okay.

Yeah. So, this is between me and my burrito? Exactly.

Got it. And so, let's start from the beginning. Where do we start?

Yeah, so you have a hungry customer, that's where it always starts, and they have their phone or their web app or whatever. Right? So they go in the delivery app, and they see Taco Bell, and they want to place an order.

That's where it all starts. We've already sent our menus to them, so that whole menu integration is done. The pricing, the products, everything should be accurate.

The hours. So, that's the first part is the menu integration, now we're on the order integration. Okay.

So the customer will place that order with the delivery aggregator, and the delivery aggregators will send an event that a new order was made. So all that's happening, there. They just send an event to our API Gateway that we set up here.

And we use an HTTP API which is new. It's a great product, and it has a direct service integration to EventBridge, so it forwards that event on directly to EventBridge. So no Lambda or anything in between?

I love Lambda, but if I could save a Lambda I'll do it, right? Less code. So EventBridge is also something we're really big fans of.

It's a very powerful new technology. And for event based systems it's amazing. It should be used by everyone.

For us, it takes that event and it uses a routing system to look at that, and know it's a new order. At that point we use another direct integration to Step Functions. Saved another Lambda.

And we trigger this workflow, here. So this is basically a new order workflow in Step Functions that we've modeled out. We're also huge fans of Step Functions as well.

So Step Functions will trigger, and we have Lambda functions that the Step Function is going to call. For that new order, the first thing it's going to do, it's going to communicate with the restaurant. The POS technology: can I even talk to you, are you online, are you open, did these products, are these even in your store.

So there's some validation that happens. But most of the time it goes very well, and they can create that order on that side And everything looks good, I can actually send an event from that Lambda to the aggregator that accepts the order. So accepting the order is essentially, you're on your phone and it says the restaurant can make your food.

You know, you're like, "Oh, cool. It's going to be made soon. " Right.

So that was the first two major steps of the Step Function workflow, but it's not done yet. However, it doesn't really make sense what it's going to do now, so what it actually does is it waits. So there's a callback task token.

We stash that in a DynamoDB table for a lookup for later. And it's just sitting there and waiting. So if they said they want the food, you know, 45 minutes from now or an hour from now, that's okay.

That Step Function can last up to a year, but for us it's not going to last that long. So, that's just a Step Function that's sitting there waiting for some type of acknowledgment. Yeah, it's kind of modeling the real life workflow of this order.

Okay. It's not done yet, right so? So cool, the timing works out, the driver's available, and he accepts that order on the delivery aggregator side.

So, he's going to get a notification on his phone in his car saying, "Hey there's an order for you. " And so, he's going to click yes, I'll accept this order. Which is another event?

Exactly and he'll get the Taco Bell that he needs to go to to pick up that food. Okay. So, he'll start going, he will drive there, and you know, the step function is still waiting.

Once he gets to a certain distance, you know, the timing kind of works out now, that we may be able to make the food. So that when he gets in, he's going to be able to pick that up right away without waiting. Having to go in there and sit there like, oh man.

So, we want to avoid that. So, because his mobile phone knows where he is geographically and it knows where the store is, they can calculate how long they need in order for him to show up, and that food to be ready and warm when he picks it up. Yeah.

So that's a really cool feature of that. So when he hits that geofence boundary, he'll send an event. His phone basically will send an event back up to them, and they will forward a new event to us that says the driver is nearby the store.

So that event goes to EventBridge, again. Which is great. We look at that message, we know it's not a new order, but it's actually an event based on an existing order in an existing Step Function instance.

So we do that look up the Dynamo I said, we get the call back to task token, and we send it back to the Step Function. So that actually starts it back up again. Which is really cool.

So it's been sitting there waiting, and now once it gets that new event, it can release that order and continue the process? Yeah I love to see it happen. It's pretty cool to see it happen.

So you know that the driver is close by. So what will happen it will release the order, and now we can actually trigger another call to our store to make the food. So it will actually go into the kitchen, and we have our own kitchen technology, and you'll see that order in there.

It will say the delivery aggregator name, they'll make it. Hopefully the timing works out so right when he walks in, they hand him the bag, and it's just a better experience for the customer. That's really cool.

I love that this is such a tactile, you know, visibly tactile event driven architecture. We've got events for the users using their mobile phone, we've got events where the driver's using their mobile phone, and then we have an automatic event where the geolocation is triggering that Lambda to continue. Tell me about developing this.

How did you land on an event driven architecture? So we're serverless-first at Taco Bell. So the first thing we do is can we do this with what AWS has to offer?

Right. Because it's been so successful for us. And we have a team now that's just been building over the last two-three years our serverless knowledge.

And so we're always looking at what's new, and we're not afraid to try the new things. So we tried HTTP API, here, EventBridge. We're all over Step Functions, so we were comfortable with trying something different versus maybe a traditional way of doing this type of thing.

Right. So within two weeks I'd say, we had almost the whole flow, like deployable. That's amazing.

Because you obviously, infrastructure as code with whatever framework you're using it's very easy to push up these serverless frameworks up. Well we talk about agility a lot, that's exactly what we're talking about. That's an amazing results.

What about scalability? Is this scaling to meet your needs? Yeah, so that's one of the other great parts of this system.

We can't break it, so it's going to scale as much as we need to. We've already pumped in like 300,000 orders in an hour which is way more than what we do now. We expect to do that eventually so we have that run way.

We know that this system is going to scale with us. It's very predictable, the cost model. It's very inexpensive to run, but the best part is I know exactly how much it's going to cost when I hit there.

Because you can count all the different things you know, the pay as you use model is great here. So, we're ready for millions and millions of orders, eventually. Alright.

Well I'm going to go place an order. That sounds good. Thank you for sharing your architecture with us, Robbie.

And thank you for watching, ‘This is My Architecture'.