all right welcome back to something that I've been working on for quite a few days now so I did not think it was going to be as complex as what this ended up but I'm super excited with the results so my goal was to learn and understand more about uh AI agent I have used them quite a bit but I still don't feel like I understand how to fully get something uh amazing out of them so I wanted to do something a little bit more comp Lex and that really is trying to write a full
feature book so the reason that I wanted to do this is really there's more than this but I wanted to learn I wanted to like try to solve a problem that's not easily solvable with AI so something that just a single like prompt to CLA or uh to open a chat GPT could solve so something that needed to orchestrate some stuff I wanted all this to run locally I wanted to keep the story arc on track I wanted to keep character arcs on track and I really wanted to experiment more with like group chatting now
I did try the dynamic approach of group chatting and for what I'm trying to accomplish it didn't do what I wanted so ended up going with a round robin approach and that seemed to work really well and then um another goal I had was to try to figure out the best model for writing and there are so many of those models there's no way I could have tested them all but I did want to go through a decent number of them so first off what are agents so they really are specialized AI systems designed just
to achieve a specific goal autonomously so you give them some kind of context to work with and they go off and do their work I like to think about this and this potential is like I have a writer and I have an editor and then I have a bunch of other ones I'll go through each of the agent I came up with but you kind of think about it like the writer is going to write the book and you're going to have an editor go through and just make sure everything works right they can communicate
with another they can pass context between each other they can work on like um different parts of the problem uh so for example I have one agent that takes the context from the previous two and then out puts it in the correct format that I want filling in the data where it needs to go so it has a very specific task that it needs to do the models I tested were the mistal Nemo 2407 instruct the Gemma 27 billion parameter the llama 3.18 billion parameter the 54 unofficial version and I tried the SC I don't
know how to even pronounce this pionic uh cation Ultra quality 20 billion parameter model I landed on the mral Nemo 2407 I could just get an amazing amount of context with that model locally and it performs fairly well I did get decent results with some of the others but really well the one thing I did learn from this is the prompts are very sensitive when you change models and when you're working across like a long running task like my first book I made took 7 hours to generate which is just insane to me because of
all the iterations I was doing and I had the uh the max round set so high so it would like go through multiple times so still tweaking it at that point but it'll get off track so you'd end up getting like 13 good chapters the 14th is a little off 15th by the time you get to the 25th chapter it's uh talking about it doesn't lose the context but it loses kind of the formatting that you want and it's um it's really tricky to kind of get the pieces broken up enough to get the output
that you want so in a at a high level I think about it like this so I'm going to give it an initial story prompt which I'm going to I'm going to detail very uh deeply which I'll show you here in a minute I then want to generate an outline so what are the what is the main story arc what are the settings what are the scenes what are the characters what are the things that are going to happen in each chapter then I want to go through and make sure each chapter is kind of
planned out at a high level and then ultimately um there will be a final chapter generated the chapter generation process is really um I I want to say complex but it it's not terrible it was tricky to get right and I still think there's tuning I could do on it the big piece is really the memory keeper so the memory keeper agent keeps track of the things that have happened in the previous the previous uh chapters and it creates a summary of those keeps track of the status of characters and the story arc the World
Builder uh I've kind of got that more into the outline process now but really the World Builder is used to kind of feed the context into the chapter so you the writer knows how to kind of build the chapter around that and then the editor and the writer kind of work together so the writer will do something the editor will verify that it fits within the world setting it fits within the the guidelines of kind of how we've set the story up the things that have happened in the past like making sure we don't get
off track it also fix any sort of errors that occur it'll send it back to the writer the writer will make those changes send it back and then it gets approved this is initial prompt this is a lot of code here uh but basically I wanted to give it a decent amount of starting points this is look this is I landed on this about probably 2 Days in like giving it this much I started out a lot simpler and it will generate a story but I had this idea for a story a long time ago
um that I've been really interested in trying to to build and this is well before like AI actually even was a thing so the idea was this was the an engineer that built a prediction model built a way to predict um market and it was insanely accurate to the point that It ultimately um forecast asked something like catastrophic happening in the market so much so that he thought it was a a bug or something wrong with his software that he had built rather than something uh that was actually legit but it turns out it was
legit and I wanted to see if I could give it enough to build a story line around that plot with some of the characters I had thought about and then see if it could stay with that through the 25 chapters and then for the outline creation there really are four different agents you can think about the user proxy agent as me kind of initiating this the the user kind of kicking off the um the system to run so that one that one will do like the initial prompt via codes it's like hey group chat do
these things and then what ends up happening once that runs the story planer will kick in the World Builder will kick in the outline Creator will kick in and that the thinking behind this is the story planner will go through and kind of build a highlevel version of like what the story could be you know how it can evolve over time the World Builder based on that will actually turn out and say like okay because of the story I want it to exist in this type of world this type of setting and then the outline
Creator takes that information combines it together into a very fixed outline which I'll show you an example of here in a minute the other things to touch on here is Max round is four I have played around with setting that to be higher but because it's round robin I only need need each of these agents to run one time I will say I was confused a little bit about what max round meant because I had thought originally that round drop and you know one round would be every agent would go through one time that is
not the case I tested this in many different ways to verify this and I finally found in the documentation where I was able to confirm that the uh so the idea here is each agent is going to execute one time outline Creator will have a very specific format and then I take and I parse that and I save it so that I can pass it into the context of the other chapters that are or the chapter generation chapter by chapter and the chapter Creations process uh this ends up with and I'll talk a little bit
about what that wrer final means here in a second but this ends up with five different ages again the user proxy one is me kicking it off uh memory keeper so important because it pulls in all the context of everything that's happened before and it makes it so that the writer knows has some of that context built into it and then the writer takes a specific chapter it's going to be working on and the editor then makes edits to that and the writer final will actually take any feedback from the editor and and fix it
now this is a place where I really wanted the dynamic piece to work because I wanted the writer and Editor to talk to each other and decide when it's done but it was and I could get it to work but not consistently so I think there's more tuning that could happen here and that would be a really cool thing to do to get sort of the AI to decide okay I'm done now we will move to to this one versus me kind of sequentially kind of building that out and then I'm going to walk through
this a little bit in code but if you look at agents. this is an example of the story planner and as I jump over you can kind of see like how detailed this particular thing actually is like I really wanted it to kind of provide each of the major plot points and I do think more agent could be created here um I think you could create an agent to develop a character I think you could create an agent to develop a scene and I think the number of Agents could go up drastically and I did
actually have some of the scene stuff in there at one point but I found it again to be very complex to combine scene to get the transitions between them to work well but I think if a person had you know a few weeks to actually work on something like this you could build a really really cool system where you have agents that go through and develop individual characters very detailed and then you've got like another agent that takes that and then combines that into the story and then that story kind of follows that Arc and
so on so let me jump over to code here real quick let me go grab the uh book writer so this is the agents. pi file so I have had a couple versions of this so version one and um I ended up generating a few books for that as well I didn't like the way that that was working as much so I ended up coming out with a uh version two of this I currently have one running now cuz I wanted to be able to show sort of the process that this has ended up going
through and maybe before I jump too far I am using LM studio and this one's currently running the mistal Nemo instruct 2407 and I have the context cranked pretty high because I want to be able to like fit a lot in in there especially as like chapters in the story kind of start building a lot so the agents I create so the memory keeper you can kind of see the memory Keeper's responsibilities and it also will flag any continuity issues I added that in there and that that has helped a bit because it'll kind of
correct things that have happened in the past um through the storyline and you know whether that has made enough of an impact to keep it I don't know but this is probably one of the simpler ones but it is very critical to be able to kind of give it you know give the the writer the context it needs to actually be able to write so you notice that it tracks and each chapter's key events and it monitors character developments and it maintains World building consistency the story planner is the one I kind of talked about
a little bit already you can kind of see the way that we kind of break that up into different plot points character Arts story beats key transitions Etc now the outline Creator is the this is the one that takes the story Builder and the World Builder and actually formats it into the right output which I'll show you here in a second an example of one and the World Builder um again it its idea is to really build the atmosphere the setting the locations the anything that you can think about it so you can kind of
see here the sensory detail key feature and I want to I want to stop here for a second I'm not a professional writer so I'm sure if someone like Brandon Sanderson were to come and look at this um he would laugh because this is nowhere near going to be the level of work or replace anyone anything like what he's doing but I really wanted to just kind of understand the complexity of being able to take this big process that can't fit into a single context window and break it up into bite-sized things and try to
come up with a cohesive output the writer is the the one we talked about a little bit before and I actually think a change that I would make going forward would be making there be uh maybe a couple versions of the writer uh one that takes feedback and then the initial writer like the draft writer and the feedback one that way I could change the prompt and tune it specifically to what job I wanted it to do but so far it's been working fairly well and I do think like um there are improvements that could
happen there though and the editor here you can kind of see the things that are happening here it's an expert editor its job is to really check the alignment uh with the outline verify the character consistency um it will return the completed chapter I also have tried getting the chapter to be of a decent length but I can't really get any model to consistently output a length a certain length of of anything more than like a thousand words uh it's been really tricky and I was surprised at how hard I've even tried like increasing that
number more and more and more and more to see if it just didn't wasn't counting it I really do think llm struggle from writing a large block of text um and thinking about an entire chapter which is why I think breaking it up into scenes would actually make more sense because you could you could plan um scenes and basically what the writer could do is write the length of a thousand where it's for like a scene or an event that's happening and then you could do a transition between those scenes and actually have another um
another agent that's like generating scenes between the or uh generating a transition between the scenes and again the user user proxy is pretty simple I just kind of set the work dire and I'm not using document uh or Docker in this particular thing so this is the agent and how that works now what I want to do is kind of show you what a outline looks like so this is really the highle outline and this is done from that outline gener you see the 25 chapters that are generated now one thing that I tried doing
is having the chapter count dynamically picked but it was ending up with something like 12 or 13 and no matter what I did to try to influence the length of that I couldn't get it to be longer so what I've done is in main.py um I'm actually specifying the number of chapters and I just pass it in and basically I tell all the agents we are looking to generate 25 chapters you know we're in the outline we need 25 chapters in the um World Builder the story planner anything that needs that I can pass that
into it the majority of the code really is parsing the data so these prompts are like what's making the agents actually work and then if you look at book generator you know this is a decent size 377 lines of code which isn't terrible uh but a lot of it is like handling failure uh extracting scenes that type of thing verify that it's complete so the the bulk of the work really is just processing the data that the AI is doing I'm not really doing much more than that so while that's happening what I want to
do is take a look at what's in chapter one text here so you can see here um this one actually is doing a great job at generating a pretty long story so let me see I've I've tweaked a little bit since my last run so while we're doing this just we're going to do this live uh to's see if I've actually got the word count actually happening correctly here um so you can see here this is the one I generated before and this one is going to be probably a lot closer to a few thousand
holy smokes I have never had a chapter generate 9,18 words what in the world so the work that I've been doing to increase that is paying off significantly but I'm kind of wondering if there's something wrong with this yeah there definitely is you can see here this is being duplicated the this is like um let look like I've got this one this is all good this is all good that's all good and then I think like all this after this point is actually data that I'm not parsing out correctly I need to go through and
kind of figure out what's going through there can you see it's just repeated over and over again anyway that's uh that's the difficulty of this thing so you make any changes to it and you end up in a situation where you know it's parsing incorrectly there's a there's a bug with it so I just want to see real quick um here I'll pull this over so you guys can see too this is 2790 where that is like the perfect amount for a chapter I would need to read that to make sure that's correct so I'm
pretty sure you know everything up until this point is the chapter and then these are the scenes that are being gener ated so again I'll go through and kind of verify that let's see what chapter 2 does cuz it ended up uh coming out so check this out this even like broke it into like characters so this is interesting like a silence uh so you could add some descriptive text in there very interesting and see this this is the chapter that I'm used to kind of seeing out of the AI and I bet you this
one's under a th oh yeah 500 30 words so yeah I guess I haven't totally cracked uh the length of this stuff yet it's unfortunate if anyone has ideas on how I can like get consistent length outputs that would be awesome I would love to be able to crack that now here's an example of a book that I fully generated um this one I went through I haven't read at all I've read like the first few chapters and then I skipped to the last one so there's all 25 chapters here and and ultimately if I
go to word count here it ended up being 21,1 159 words with 125,000 characters all generated locally via Ai and this story here is a totally different one that I had a had an idea around it's one that basically imagine a world where compute is is at a is a very limited so the idea is that um there's a company that's kind of taken over because of some postapocalyptic you know atmosphere and they actually use humans to like brain scan them and the output of that was not anywhere aligned with what I had originally fed
into it so it didn't work to the degree that I thought I'd be able to influence the story the story is actually pretty interesting because it has elements of it it has my characters in there it has some of the things about Triton Tech which was the name of the the and and Alex Triton they built him out as a character so it did everything except for the brain scanning piece so I don't think I put enough influence on that anyway I am curious what you all think about this I have enjoyed so much working
on uh this project and I've learned I feel like I've learned a million things I accomplished most of my goals I would say if I go back to my goals I accomplished most of them um I do see the story arc and some of the stories get a a little bit off track but for the most part if I read like the beginning all the way down to the end the same characters they being talked about the same world they're in uh so it's actually doing a relatively good job and I think there's a ton
more that actually do there uh I learned a lot about how you know to build better prompts to to have that memory system in there so that I could keep track of things the editor writer combination was really cool and then testing the different models I ended up landing on that mistal Nemo one being the best I don't think there's that much difference I will say some are just better about following instructions and others some will like do crazy stuff like and then the format will be different and things I have to just clean up
things a little bit better so I'm still like very surprised about this chapter one I need to go through and debug that but I've actually I'm actually amazed that it generated like that much like content like it had that much in its context so anyway I hope you guys enjoyed this please let me know if you have any ideas on what I could do to improve it uh if you have ideas on you know maybe a story that you'd want me to feed in there to see what it outputs just let me know anyway until
next time peace out
