AWS re:Invent 2024 - Best practices for customizing Amazon Q Developer (DOP217)

so let's jump in with a little bit of an introduction um I assume many people have seen this slide already this week I know there this is part of a lot of different presentations but this is the the Gen generative AI stack of our services really what I want to get clear here is where we're talking about today so I'm going to start at the bottom at the very bottom we've got ec2 instances with different GPU and other accelerator options and I can build anything at this level in generative AI right I can train models

build models host them um and what we found over time was that most customers were focused in on just hosting a model and being able to ask questions of it and so early last year we launched Bedrock that makes it really easy to host a model still gives you a lot of configuration and customization capability but really makes it easy to host those models and then last year we watched what customers were doing with that and saw some patterns emerging in the things that they were trying to build on top of bedrock and so at

reinvent last year we launched Amazon Q which comes in a few different flavors and Q is a specific application designed to solve those problems that we saw customers trying to accomplish so on the left you've got Q business right this is about building a rag about making it really easy to set up a rag implementation and build out a chatbot on the right what we're going to talk about today is Q developer and we saw a lot of customers building out different solutions to uh to improve the developer experience and so we built an application

that accomplishes exactly that called Amazon Q developer okay um so Q developers really focused on the entirey of the software development life cycle and accelerating all of the different pieces of it I know developers in the name and of course it's there to help developers but it's also going to help the rest of the IT staff that support an application throughout its life cycle so you've heard announcements today about what we're doing with operations and Cloud watch and the ability to diagnose issues and use Q to help you diag know problems we're really expanding that

capability beyond the developer in order to accomplish this we're building Q developer into all of the different experiences that you touch within AWS right so if you're in the documentation trying to learn how to do something Q is there to help you you don't have to read through hundreds of pages of documentation I can just have that conversation with q and ask questions of it um it's in the console when something goes wrong in the console Q's there to help me troubleshoot it pops up proactively it says oh it looks like you're having a trouble

let me show you how to fix that and let me walk you through step byep how to fix the problem that you've got um we built it into the mobile applications it's in slack and teams so if you're talking to your teammates and working on an architecture you can bring Q into that conversation where you're already having that conversation and then of course we have devops platforms you probably heard the announcement with gitlab so we're building into different um devops platforms and of course in the IDE where the developers are working and we're helping to

accelerate the software development process and this is where we're going to focus our conversation today right it's inside the IDE so this works as a plugin to lots of different idees vs code visual studio uh goes into um the Jeet rins IDE we just announced Eclipse a couple weeks ago so all the different places that you're writing code we've got Q developer there to help you now Q developer of course is trained on lots and lots of publicly available code example so that it can help you but the problem that arises is is it doesn't

know anything about things that are unique to you your internal sdks your apis it might be that you've built out uh a really elaborate uh SDK that you need to be working on all the time or it might just be that you're doing something really simple like you've got maybe a a particular logging solution that you always use on your Enterprise projects and so you want to be able to teach Q about those things that's what a customization is all about so the customization allows you to train on your private code basis so that it

can learn those and then be able to make suggestions about things that it wouldn't know about otherwise you can see in this example in this picture um that says at the top The Prompt that caused it to write this code is process a list of unassigned food deliveries around the driver's current location right a very business domain specific problem that Q has learned and is then able to write code for and it's not part of the publicly available data set so let's start with just a really quick demo to see what this looks like okay

and for this demo I'm going to go very simple imagine that within my organization I have a data repository object right I don't talk directly to a database we've built this rapper class that we always use and we're doing movies we do movie reviews um and so what I want to focus in here is in this movie repository class I've got these fields right title year plot and rating these are the things that are important to me and these are the things that I write into my database so now I'm going to go use that

without the customization I've created my movie repository I ask it to add Titanic and you can see it's suggesting the director and the genre which were not part of my data set Q has no idea what my data looks like and so it's doing the best it can it's guessing now I'm going to switch over and I'm going to come down and I'm going to turn the customization on that I've trained it on so I'm going to come in and select the customization and I'm going to grab have this demo customization and I'm going to

ask that same question again but now Q is aware of and been trained on my internal code base and now you see it knows those fields it knows that I have a title a year a plot and a rating because I've seen examples of that it knows what my data structures look like as a result um you can also see that it reformatted it right in the first example it was all on one line now it structured it because when I trained it I gave it a bunch of examples that look like the way I

write code the way I prefer it and it's now mimicking my style too as a result of of this okay this is a really trivial really simple example but what I'm trying to show is kind of the before and after and how it's now learned our structure in addition I could have chats you know in the chat on the left side of this which I didn't demo I can ask questions about my code base and about my data structures and it's able to answer those questions uh just like it just like it did in the

code complete example okay so with that quick example what I want really want to focus on is how to get value out of this and use this really well today and so the first step to doing that is preparing a customization right it's really about getting your data ready so that you can teach Q about the the the different code examples that you're going to show it so first I'll talk about preparation and here you can see this is right out of the documentation but you need to have at least 10 files that's pretty easy

to hit um and you need to have at least 2 megabytes of data and currently uh customization supports Java JavaScript py and typescript so you have to have 2 megabytes of those file types in addition uh just a week or two ago after I recorded this and got the slides ready we did launch support for uh text files like readme Doc readme markdown files uh and that's really more relevant in the chat so when you're having a conversation you want to ask it questions about like how do I use this and how do I install

it and that stuff is usually not in the code but in the documentation that's inter read me so those are included now as well so for Best Practices when we're looking at the code that we want to train on a few things you want to be thinking about first this is going to take a text based prompt so you're really want to be verbose about the text that you're giving it right you want to do things like have really verbose method names that have all the keywords in it that are likely to match okay now

of course often this code already exists you don't have the luxury of going and making these changes and restructuring it just to use Q so the other thing you should be thinking about is having really good really verbos comments in your code and so you can see here right I've added in um a nice structured comment that describes what my fields are I'm also giving it some additional hints like the max length on that field is uh 4096 so Q can learn a lot more from this by giving it good examples and then the final

piece is you want to include a lot of examples of the usage of the class this is actually much more important than giving it the source code to the class right these these generative AI applications mimic human human intelligence and when I am learning something new I want to learn a new framework I don't go read the source code to that framework I go look at how it's supposed to be used I want to see examples of how it's intended to be used and so show it a lot of those examples that's what it's going

to learn from okay and then the final best practice in preparation is you really want a lot of examples we say two Meg is the minimum um you want to have a lot more data than that and really where when I'm working with customers I'm seeing very good results at around 100 megabytes or more of code okay and even much more than that if you have it and you can get them avoid duplication of code showing it a lot of examples of the same thing isn't helpful that's just redundant and there's avoid any non-unique codes

so like including your node modules all the files that are imported from open source packages are meaningless cuz it's already been trained on all of that it's not going to learn anything new from that might help you get to the 2 megab limit but it's not going to add value so one more quick demo here a lot of times when I'm working with customers the code that they're giving it isn't already well documented and so one of the things we'll do is use Q to help with the documentation process before we actually start training on

it and do a little preparation but still use generative AI to accomplish that so here this time I'm going to do this in Java I've got a Le Java application this is from the junit documentation um and there's an add subtract multiply and divide method um and there's a couple tests that go along with this and I'm just going to come into Q here and I'm going to ask it to add comments into all my code for me and I'm going to do this using one of the agents they they talked about agents in the

keynote this morning I'm going to dodev to invoke an agent and I'm going to ask it to write Java dos comments across my code base it's going to happen in two steps so first it's going to go go through and analyze my code and you can see it says okay I looked over your code I found these two files in the first I found these four methods and I'm going to go and document them for you and then it goes on and talks about the test and says I also found five tests I'll go and

document those as well I could ask for changes conversationally here but I just let it roll and so I let that happen give it a i' fast forwarded there about 2030 seconds and it comes back and says okay I've done that work for you and I can come and look at the changes that it made and it just went through and added comments into all of my code so taking a little time and doing some preparation can greatly improve the results you're going to get with the customization later if you've got really well documented [Music]

code okay so that is step one right we prepared and it's really the most important step the next thing I'm going to do is go and create the customization and the way I'm going to do that is I'm just going to connect Q to one of my existing repositories so I can connect it to um fit bucket GitHub or gitlab okay and I can be selective about what repositories I want to pick I'm in the demo I'm just going to pick all of them but I can be selective and it's going to go through and

it's just going to grab the default Branch so best practice here we also support S3 I'm going to show you an example where we ingest directly out of a um out of one of these repositories I'll use GitHub but we do support S3 so first if these are if you're using something other than these you can just put code in in S3 but also when I work with real customers most of the time we put data in S3 because we want to be really selective even within a code base that we want it to learn

there might be some examples that we don't want it to see I'm moving from this Old Log provider to this new log provider and I don't want it to see the old examples until we're done and so I'll be really selective and load things into S3 so as much as I'm going to show you the easy way to do this most of the time you're probably going to do this by loading very selectively into S3 okay it's going to train on that it's going to go through and ingest that and then it's going to give

you a score back and what I really want to focus on is this is going to be depressing your score is going to be terrible they're always terrible um I got a in the example that I was showing earlier in My Demo I got a score of a three which it considers to be poor um don't get discouraged by this just just continue and test it and see how well it works um it is very hard to get into this category if you're maybe an isv with a lot of unique code you can score this

if you're writing Enterprise code that's mostly using react and angular or you know mySQL database providers Q's already learned all of that so it's seeing very little original stuff in your code it's only you know a very small portion of what it's ingesting is actually unique and new and so you're probably going to end up in here if you do improve your comments give it a little give it a few more files and and try to bring your score up as best you can but don't be discouraged if you end up in that uh you

know in that fair category you're probably going to get really good results even with that so let's see a demo of this of the actual process of generating the customization so I'm in my GitHub account here and you can see I've got the AWS connector for GitHub um that's used by a bunch of AWS Services code pipeline code build Etc I'll share that so I've already got it configured and I've given it read access to all repositories okay so that's pre-existing now I'm going to go in and actually set up the customization so I'm going

to come into the AWS console I'm in the Q developer console and I'm going to set up a new customization I'll call this one uh Q customization GitHub and as I said I could select S3 most real world applications we actually use S3 but I'm going to go with the AWS connector and again I can go with um bit bucket gitlab GitHub I'll select GitHub for this I just give that name and I'll go and create this connector I could add tags right if if you have a tagging policy within your organization I could add

them I'm not for the demo but I'll go and create this connection to GitHub that'll take a few seconds and I'm going to come in I could create a new app I already have that app configured the this is the GitHub app I've already got it configured so I'll just select that again I could add some tags but this is good I'll accept that so this connection is now created I'm going to go back into q and I'm just going to select that connection action that I created moments ago of course have the option to

add tags again I'll set up logs so I can debug if something goes wrong I'm going to send these to cloudwatch logs but you can send logs to S3 or Kinesis wherever you prefer to send logs to and then I'm just going to kick this off and this takes a little while like actually going in and ingesting the code and learning and it takes about an hour to do this usually um I fast forwarded through that and you can see here this is created you can see I got a terrible score of a one um

this is just some sample code that's out on my GitHub repository so it's not terribly exciting and then finally I'm going to activate that version so I can have multiple versions I only have one version here I'll activate that and get it going okay and now we've got this customization completed and ready to use the last thing I'm going to do is decide which users are allowed to use it I'm going to Grant permission to some of my users uh for this I'll just give it to all the developers everyone that's in the developer group

will get access to it and can all use that customization okay so with that we've gone through the preparation stage the most important one where we really did a lot of the best practices we spent a little bit of time on the actual creation which was pretty straightforward uh now we're going to move into how to use this and monitoring so first as we're rolling it out the last thing I did in that in the demo was I granted access to all of my developers best practice here you probably want to be a little more

Grant man than that you've probably got different users maybe you've got a data science team and an appdev team and they have very different codebases even though they might use the same language you probably want to have different customizations for them because the things that this team is doing isn't going to be relevant to that team and sharing a customization is only going to end up with things crossing over that doesn't make sense so so think and be a little more granular than I was in this demo once you roll that out it's very easy

for your users to then go and select among the customizations that they have access to and to go back to the default one anytime they want so if they're working on an open source project at night they might choose to just go to the default provider uh and not use the customization and they have the option to switch back and forth and then the last thing you're going to be doing as you roll this out is you want to be able to monitor it so um in the Q console you've got this dashboard I don't

have any data yet because this is just Brand This is a brand new customization but you can refine that and look at the details for just a specific customization how is this perform what's my accept rate for this specific customization so you can look at it and understand how well it's performing if you're not getting what you want out of that you can also go a lot deeper with cloudwatch and go in and do things like look at the the performance of an individual language for a very specific customization and be really granular about what

you're looking at okay and then you can see how it's performing in different situations how are the python teams doing versus the frontend web team that's using typescript and how is it performing in each of these cases okay so we have about 2 minutes left by my clock the last thing I want to do is just point out a couple references here on One internal One external so we use all of these tools internally of course there are a lot of teams within Amazon using Q developer U this comes from Prime video and I think

the important one here is they are seeing a 30% increase in acceptance rates compared to using q without the customization all right this isn't a 30% increase over not using generative AI this is an additional 30% productivity Improvement here on top of what they were already getting by using the tool which is awesome and then the second one from an external customer comes from National Australia Bank okay theyve rolled out to 500 developers and they're seeing an increase of 60% over what were they were getting at of Q before they ran the customization so training

these customizations sees a big lift a huge productivity boost and I really encourage you all to take a look at this if you're using Q if you're not using Q developer of course start with playing with Q developer get that rolled out but then start to look at customizations because it's going to have a huge impact on your organization and get you much better results excellent and with that I am just about out a time thank you very much