a batching solves the problem of throughput increased throughput now when you're getting a bunch of salt small messages you do not from a performance perspective get higher throughput you not want to send them one at a time you'd rather gather up a bunch of those in one particular batch and then pay the cost of connecting to sending that Network packet over at one time and getting one acknowledgment back right so the batch size you can Define how big these batches are specified either by the maximum number of messages I will get n number of messages
before I consider my batch full or a maximum published latency or a maximum size of bytes so there are three different settings that you have that can control uh when a batch gets triggered right so there once a batch is created they're tracked and stored as a single unit not individual messages this is internally a mechanism so when you're we're talking on day one about sending quote unquote messages and we're filling these internal cues that really when we say messages that's talking about if you have batching enabled that talks about these batches and then inside
these batches are the individual messages themselves so a takeaway from this is when you look at some of these metrics around Pulsar topic stats or if you're looking through like grafana and you're looking at backlog and things like that then that backlog size actually represents a total number of batches because that's the level of granularity we can measure and not the total number of messages right so just just be aware of that you look at say hey I'm 100 messages behind well actually you're 100 batches behind and if your batch message size is a thousand
then you're actually a hundred thousand that's just behind just to be aware of that on the consumer side when the batch is delivered the Pulsar client knows how to handle this batch and have an index and uh you know manipulate these messages individually so it's all transparent to you but under the covers they're being sent back and forth as batches now schedule messages um are not sent as batches because we don't want to intermingle those that are uh intended to be delayed with those aren't so we break those out separately and treat those as individual
messages uh just just so we have that we don't inadvertently delay or reschedule messages that weren't attended so that at that level we have a slower level uh of granularity now the batch itself on the consumer side once you've consumed all of those messages uh there it's that's when you acknowledge those messages by the consumer right so any sort of uh unexpected failure and negative acknowledgments act timeouts uh will result in the redelivery that entire batch because that's the smallest unit we have so again if you're processing halfway through to negatively acknowledge one of those
messages in the batch uh just from a performance perspective that whole batch was going to get redelivered again uh so now if you want to avoid that default Behavior then you can enable this this particular property here uh in your broker.conf configurations file to avoid uh you know this that behavior and just have the actual individual messages that you negatively acknowledged sent back rather than the entire batch itself right so when this is enabled enable the consumer filters out the batch index and sends just sends the back index back rather than the whole message so
it's it's a more granular control allows you but it does result more chatter because you're sending an act back for every message or negative act back so there's a trade-off related to that but that's that's a configuration setting you have uh to to uh you know control that sort of behavior it's not turned on by default and we'll make sure that you uh turn it on to true so you know this uh this way this works so that all of the messages will be sent to the consumer as a batch but the consumer will take
care of the messages that have already been acknowledged right so that's that is the preferred setting for that now the broker maintains the batch index acknowledgment status and tracks the ACT status back so you don't have to worry about again about this when all individuals messages in that batch are acknowledged that and then that entire batch message cursor is moved onward and it becomes acknowledged themselves and then it's eventually slated for for deletion if there's no other once it falls out of the data retention policies and all those other things we talked about yesterday about
your day how long you want to keep acknowledged messages around now batching reduce reduces how it improves it is you know Pulsar Works uh you know in like a pipeline format so you send a message then you send another without waiting for the ACT for the for each individual messages right but if we have bigger messages that are able to process these more you know we're able to processes more efficiently and more easily than if we're trying to process a whole lot of smaller messages back and forth right so there's also uh so just sending
them back and forth of Bruce throughput there's also an impact on bookkeeper as well it's much more effective to write a bigger chunk of data and access it as access your disk once rather than go back and read the same data you know next time right so you're reading these next messages grabbing them is a batch makes them more efficient i o all the way across makes it more efficient depth sync them to disk so it's it's set it's enabled by default and it incurred it's there for performance and throughput reasons you just have to
be aware of it right so uh there are some things to mention about how it impacts the impacts of batching on other features is that if you're using a key shared subscription uh you know if you're using just batching and not carrying back the keys then you're going to get in a situation where your batch can't be split up by key you know that uh you know where you where you'll need to actually manage those keys you you know yourselves so that's you know back you know when you create a batch you may mingle the
keys and then those keys won't be evenly distributed on the key shared subscription if you have batching enabled so that's something to be aware of if you when you're producing these messages on the producer side uh and your intent is to consume them from a key shared subscription then you're then you're going to want to disable batching uh in that particular scenario to make sure that your keys of the same value are always going to be delivered to the consumer on the consumer side so just just be aware that people have run into performance problems
uh with this and then they're missing data so that's something to be aware of and you have to think of it as an end-to-end solution uh as far as from a consumer side when you're writing the data in chunking and batching are polar opposite so you cannot batch chunk messages and when you uh they solve completely different problems batching applies to much smaller messages you want to get them up into a bigger size message to improve you know prove throughput chunking is the is exact opposite of a very large object and you break it into
smaller pieces to send the entire thing over one by one now so there are some configurations you have on the producer side when you're batching right so if you want to make sure that your keys Stay Together uh and so if your batches always have the same Keys then you can say when you say set batch Builder you make sure that it's key based and then it will create separate buckets for you uh on the other end uh when you produce these messages and send them out so that's one way to uh solve that problem
we talked about previously if you do not set this thing by default you just use standard batching your keys get mixed together if you're using keys we'd highly recommend setting this uh property on your producer so that you make sure that all the keys Stay Together now these are the settings we've talked about about when to trigger batch when any whichever of these you set uh to be enabled the first one that occurs is what triggers the batch being quote unquote pull and then it gets sent right so the first one is about Max publish
delay this is to minimize the latency between a message arrives and you decide to send it over to uh Pulsar right so this is a hard timer set in milliseconds you know or seconds or milliseconds by default I think it's like you know uh you know 10 milliseconds by default so it's very very fast if you want to increase this throughput you can set this to higher value basically keep your window open to gather more messages to allow your batch to grow uh again max number of messages that that you're allowed to do so you
can say if I get to a thousand messages it's sort of a default size if I get to a thousand messages I will send that over as well so that becomes a secondary trigger on that there's also a backs batching Max size which comes at it from the from the Q perspective it measures the raw bytes um as well so this this one Max messages just does a raw count of messages the max bytes just does an internal count of your bite size of the messages and if that gets triggered then that will trigger the
uh once that threshold is crossed or approached it will trigger that batch and send it over as well so if you don't know if you don't have predictable size messages and they vary you might want to use the the back patching Max size rather than backs Max messages if your messages are predictable size and you can get away with Max messages as well right and so again if you want to maximize throughput you increase these values if you want to lower latency then you can control that by your published delay it's a few milliseconds so
that you just so you have lower latency when sending the messages you don't hold on to them for longer periods of time waiting for the batches to build up so you know by default enable backseat is true you can turn that off again we talk about the publish delay the default is I thought it was 10 milliseconds but let's say one millisecond uh this is the maximum time so between you publish a message and you send them out uh that's how long your weight that's how long you keep the batch open again the max number
of messages coming in the max max bites more than 120 8K and that sort of what you size your internal buffer as well to that so you make this bigger you're going to consume memory on the client side for every you know uh batch you're creating that's going to be larger and then the batch Builder we've talked about the default one uh is there and then there's the key for keyshare to one key shared one as well
