Hello everyone, today I will be guiding you step-by-step through the process of training LoRA on the latest state-of-the-art text-to-image generative AI model, FLUX. Over the past week, I have been deeply immersed in research, working tirelessly to identify the most effective training workflows and configurations. So far, I have completed 72 full training sessions and more are underway. I have developed a range of unique training configurations that cater to GPUs with as little as 8GB of VRAM all the way up to 48GB. These configurations are optimized for VRAM usage and ranked by training quality. Remarkably, all of
them deliver outstanding results. The primary difference lies in the training speed. So yes, even if you are using an 8GB RTX GPU, you can train an impressive FLUX LoRA at a respectable speed. For this tutorial, I will be using Kohya GUI, a user-friendly interface built on the acclaimed Kohya training scripts. With this graphical user interface, you will be able to install, Set up, and start training with just mouse clicks. Although this tutorial demonstrates how to use Kohya GUI on a local Windows machine, the process is identical for cloud-based services. Therefore, it is essential to watch
this tutorial to understand how to use Kohya GUI on cloud platforms, even though I will be making separate tutorials specifically for cloud setups. We will cover everything from the basics to the expert settings. So even if you are a complete beginner, you will be able to fully train and utilize an amazing FLUX LoRA model. The tutorial is organized into chapters and includes manually written English captions, so be sure to check out the chapters and enable captions if you need them. In addition to training, I will also show you how to use the generated LoRAs within
the Swarm UI and how to perform grid generation to identify the best training checkpoint. Finally, at the end of the video, I will demonstrate how you can train Stable Diffusion 1.5 and SDXL models using the latest Kohya GUI interface. So I have prepared an amazing written post where you will find all of the instructions, links, and guides for this Amazing tutorial. The link to this post will be in the description of the video, and this post will get updated as I get new information as I complete my new research with new hyperparameters and new features.
So this post will be your ultimate guide to follow this tutorial, to do FLUX LoRA training by using the Kohya GUI. Kohya GUI is a wrapper developed by the Bmaltais that allows us to use Kohya SS scripts very easily. This is its official GitHub repository. Basically, we are using Kohya SS. However, we have a GUI and one-click installers and easy setup to use the Kohya SS scripts. So you will see that I have a zip file here. When you go to the bottom of the post, you will also see the attachments, and in the attachments,
you will see the zip file. I may have forgotten to put the zip file at the very top of the post, but when you look at the attachments, you will always find the zip file. So let's download the zip file. Click here. Move this zip file to any disk that you want to install. I am going to install it on my R drive. Do not install it in your Windows folder, into your users folder. Directly install into any drive, or do not Install it in your cloud drives. Okay, right-click and extract it. Enter the extracted
folder, and you will see all of these files. These files may get updated, and there may be more files when you're watching this tutorial. Don't get confused. I will update everything as necessary. Currently, the main branch of the Kohya GUI doesn't support the FLUX training. You see, there is a SD3 FLUX branch. So my installer will automatically switch to the FLUX branch and install it for you. Once it is merged into the main branch main repository, I will update my installers. To install Kohya GUI we are going to use the Windows_Install_Step_1.bat file. Double-click it. It
will clone the repository, switch to the accurate branch, and it will start the installation. For this to work, you need to have the requirements installed. You'll see that we have a Windows requirements section. We need Python 3.10.11, FFmpeg, CUDA 11.8, C++ tools, and Git. Now, once you install these, you will be able to use all of the open-source AI applications like Stable Diffusion, Automatic1111 Web UI, Forge Web UI, One Trainer, Swarm UI, ComfyUI, and whatever you can imagine, like Rope Pearl, Face Fusion, and such things. So I have a very good tutorial that shows how
to install all of this. Please watch this tutorial. Do not skip it. Please make sure that your Python is directly installed on your C drive. You have installed FFmpeg, CUDA, and C++ tools. These are all important. How can you check whether you have installed them or not? Type CMD and open a command prompt like this. Type Python and you should get 3.10.11. Open another CMD and type Git and you should get Git code like this. Open another CMD and type FFmpeg and you should get FFmpeg. FFmpeg is not necessary for Kohya, but you should install
it because you may use other AI applications. And type nvcc --version, and this will let you see whether you have installed the correct CUDA or not. Everything is explained in this video. Please watch it to avoid any errors. So the automatic installation will ask you these options. We are going to select option 1 and hit enter. Then this will start installing the Kohya SS GUI latest version to our computer. Currently, we are in the accurate branch. As I said, I will update this branch when it is merged into the master branch. Just wait Patiently. You
see that it is showing me I have Python 3.10.11. It is installed, and it is going to download and install everything automatically for me. So the installation has been completed, and I have the new options. You should scroll up and check whether there were any errors or not because the installation step is extremely important. So scroll up and check all of the messages. So as a next step, we are going to set the Accelerate because on some computers it may not be set correctly. So type 5 and hit enter. It is going to ask us
some of the options. We are going to use this machine. Hit enter. We are not going to do distributed training. Hit enter. We are not training on Apple or anything. We are training on CUDA drivers. So no, we are not using Torch Dynamo. No, we are not using DeepSpeed. No, we are going to use all of the GPUs. So type "all" and hit enter. I also say yes to this, but I didn't see any difference. Hit enter. And we are going to use BF16. This one. So select this one and hit enter. And we are
ready. And we are ready. You don't need to do option 2, option 3, or option 4. But we are not going To use option 6 to start yet. So hit 7 and hit enter. Return back to the folder. And whenever you are going to start training or after the initial installation, run this bat file: Update_Kohya_and_Fix_FLUX_Step2.bat This is going to upgrade to the latest libraries. It is going to also update the scripts to the latest version. So we are going to have the very latest version with this bat file. I will keep these files updated. Just
wait for it to install everything. The steps are also written in this post file, but watching this video is better. And the updates have been completed. Verify that they are all working. We are currently using Torch 2.4.0. And it is working slowly on Windows compared to the Linux systems. However, with Torch 2.4.1, hopefully this speed issue on Windows will be fixed. And as soon as it is released, I am going to update my installer scripts. There has been a new development after I have completed the tutorial video, which is Torch 2.5 version. With this version,
there is a huge speed improvement on Windows. It is almost equal to Linux. However, Torch 2.5 version is currently in development. So it may be broken. It may give You errors. I also didn't test its quality yet. Hopefully, I will test it and I will update the Patreon post. You see, it is already updated. The file that you need to use is: Install_Torch_2_5_Dev_Huge_Speed_Up.bat file after completing the installation. As I said, this is experimental and follow the news on the Patreon post. And now we are ready to start using the Kohya SS GUI. So for starting
the Kohya SS GUI, you can either enter the Kohya SS folder and start the gui.bat file. Or you can use my automatic starter file, which is the Windows start KohyaSS.bat file. This will automatically start and open the browser. You see the interface has started. If it doesn't start, you need to open this manually. All right, now this is the interface of Kohya. It is very, very useful and easy to use. A very important thing that you need to be careful of is that you need to select LoRA because we are currently training LoRA. LoRA is
optimization. So it is different from DreamBooth. DreamBooth is basically fine-tuning, training the entire model. But with LoRA, we are only training a certain part of the model. Thus, it requires lesser hardware, But also results in lesser quality. Currently, this tutorial is for LoRA training. Hopefully, I am going to do full research on DreamBooth fine-tuning and publish another tutorial for that. And I am going to publish configurations as well. So this tutorial is so far the combination of over 64 different trainings. This is literal. When you go to this post, you will see the entire research
history. This is a very, very long, lengthy post. You will see all of the tests I have made. You will see all of the results, grids, and comparisons. I am also going to show some of the parts in this tutorial. So read this post if you want to learn how I came up with my workflow and configuration. Also, when you open this post, you will see all of the models that I have prepared up to now. And this is not all. Currently, I am running 8 different trainings on 8x RTX A6000 GPUs on Massed Compute.
You can see the trainings are running right now. Currently, I am testing a new hyperparameter and the clip large text encoder training, which has arrived just today. So after these tests have been completed, I am going to analyze them, Post the results on this research topic, and I am going to update the configuration files. Don't worry, you will just load the configuration. You will just download the latest zip file and load the configuration, and you will get the very best workflow, very best configuration whenever you are watching this tutorial. So that is why following this
post is extremely important. The link will be in the description and also in the comment section. Don't forget that. So now our interface started and I am going to load the configuration for beginning the setup. I have selected the LoRA in the training tab. When you look at the folder, you will see best configs and you will see best configs, better colors. And what do these configurations mean? When you scroll down, you will see the description of each configuration. I have prepared a configuration for every GPU. So if you have an 8GB GPU, you need
to use this one: Rank_9_7514MB.json file. When you enter inside the best configs folder, you will see that the JSON file is there. So according to your VRAM, pick the configuration file. There are slow and fast versions. What do they mean? With the slow Version, we are going to get slightly better training quality. There is not much difference between rank 1 and rank 6. After rank 6, we are going to lose some quality because of the reduced training resolution and also reduced LoRA rank. If you read the research post, you will see the difference of each
configuration. Training under 1024 pixels seriously reduces the quality. Also, I find that LoRA rank 128 is the very best spot. So these three configurations will have slightly lesser quality than these ones. These ones will be very, very close. Since I have an RTX 3090 GPU, I am going to use the very best configuration rank 3. Currently, this is running slow on my PC, but with the Torch version 2.4.1, it will be much, much faster. So go to here and click this icon, this folder icon, enter inside the folder of best config and load the rank
3. Now you may wonder what is the difference with better colors. This is in experimentation. Currently, I am training it to decide whether to fully switch to it or not. But this uses time step shift sampling and it makes a huge impact on the coloring. When you open this file, you will see this very big, Huge grid. And when you look at this grid file, you will see that using the time step shift brings better colors and a better environment like this one. However, it also slightly reduces the likeness. I am still researching it. I
am still training it. When we open this imgsli link, you can see a one by one comparison. So according to your needs, decide the configuration you want. Hopefully, I will complete research on time step shift sampling and I will decide the very best configuration. Currently, we have two different configurations and their difference is like this. But to be safe, you can use the best configs folder right now. However, you can also use best configs better colors too. So look at the grid file imgsli and decide yourself. OK, as a next step, you don't need to
set anything here. This is for multi GPU. I also have multi GPU configs and I will show them hopefully in the cloud tutorial. I am going to also make a cloud tutorial for Massed Compute and RunPod. So what you need to set with this configuration, you need to set pre-trained model path. This is super important. The model links are posted here So you can either download them from here or I have a one-click downloader here. You see Windows download training models files. If you already have some of the models, make sure that they are accurate
models or you will get errors. This bat file will download everything automatically for me. You see, it started downloading already. However, I already have them downloaded here. So I am going to just pick them. If you already have them, you can use them. We are going to use FLUX Dev version 23.8 gigabytes. The FP8 support also arrived, but I haven't tested it. So to be sure, use the FP16 version. It doesn't matter because we are going to use it in FP8 mode and training with 24 gigabyte GPUs and it will cast the precision automatically. So
select this. You see, this is the base model that I have. Now selecting training images is super important and so many people are making mistakes here. If you know how to set up Kohya folder structure, you can already set it up yourself, but don't use it if you are not an expert. So in the dataset preparation section, we are going to prepare our dataset. The dataset preparation is extremely important. I have written Some information here on how to caption them, how to crop them. I already have an auto cropper and auto resize script, but you
can also manually auto crop and auto resize them. So currently my training images are auto cropped and auto resized to 1024 pixels as you are seeing right now. These scripts that you will find are extremely useful. Watch this video and check out this script to automatically crop the subject with focus and then resize to get your training dataset. Once you have your dataset like this, copy the path of this dataset or you can also select the path from here. You see training images directory. Click this icon, go to the folder like this and select the
folder like this. Now this is super important: repeating. What does repeating mean? We are going to set the repeating to 1 because we are not going to use classification / regularization images. I have tested the impact of regularization and classification images. You can see the results in this research post. For example, let's open this post and no matter what I have tried, there is no way to improve the likeness and the quality with classification regularization images. In none of The cases it yielded better results. When I have used the classification regularization images, you see, you
will get mixed faces like this. And if you still want to know what does repeat means, there is a link here. Open this link. In this link, I have asked Kohya to explain the logic of repeating. The logic of repeating is initially made to balance imbalanced datasets. What does that mean? Let's say you have 5 concepts that you want to train at once. And one of the concepts has 100 training images. The other one has 20 images. The other one has 30 images. So in machine learning, you would like to make datasets balanced as much
as equally trained. Therefore, if you have 100 training images, you make the repeating 1. And for other concepts, if there are 20 images, you make the repeating 5 of that concept. However, since I am training a single concept right now, I am going to set repeating 1 and I am not going to use regularization images because for FLUX training, it doesn't improve the results. However, when you train Stable Diffusion 1.5 or Stable Diffusion XL (SDXL), it really improves the quality of training. At the end of this tutorial, I will show you how to load the
very best configuration for SD1.5 and SDXL and train them. Don't worry about that. So we did set the training images directory here. We are going to set the instance prompt. I am going to use a rare token ohwx because it contains very little knowledge. Also, with FLUX training, it is not as important as before, but still use it. And as a class prompt, I am going to use "man." Even if you don't use a class prompt, since the FLUX model has an internal system that behaves like a text encoder and automatic captioning, it will still
know what you are training like you have fully captioned it. So even if you don't provide any instance prompt or class prompt, it will work with FLUX. I have also tested it. In the research topic you will see the training results with ohwx man, with only ohwx, and with only man. And you will see that almost in all cases, it perfectly generates your concept. Still, I find it a little bit better to set the class prompt as "man" or if you are training a woman, it's "woman." If you are training a car, it's "car." If
you are training a style, it's "style." But training for a man, I use "man." Even if you don't use it, it will work. If you are training multiple people in one training, you don't need to set a class prompt. Just give each one of them a rare token like ohwx, like bbuk, and other rare tokens that you can decide yourself, and train all of them at once. But currently, this is for single-person training. So this is the setup. Instance prompt, class prompt, and destination directory. This is super important. The Kohya GUI will prepare the correct
folders and save them at that destination. So I am going to use the installed folder like here. You can use anywhere, and I will save train images like this and then click prepare training data and watch the CMD. You will see that it is done creating the Kohya SS training folder structure. When you enter inside that folder, you see the train images folder generated here. When I enter inside it, you see there is an image and inside the image, there is 1_ohwx man. So what does this mean? This means that the one is the repeating.
This is super important. If you set this to 100, at every epoch, it will repeat 100 times. So this is super important. Kohya will always read the Repeating number from here, and the rest will be simply the caption of my images. If I wanted to caption them, I could caption them, and it will read the captions. If you don't caption your images, it will read the folder name as a caption. So let's also make a captioned example. I will copy and paste this like this. The name of this folder will not be important because I'm
going to caption it. So copy this path, and for captioning, I have an amazing application called Joy Caption. Click here. When you get to this post, you will see the installer for Joy Captioning. This is a very advanced application. It supports multi GPU captioning as well because I am going to caption my entire regularization images dataset, and I am going to make a LoRA on that. There will be a total of 20,000 images that I am going to caption. So it supports multi GPU and also batch captioning. It is already installed. Let me open it
and show you how to caption. So Joy Caption 1. Let's start the Windows application. Let's start like this. The application started. All I need to do is give the input folder path here and just start batch captioning. You can decide on multiple GPUs, multiple batch sizes. All is working. This is a very optimized application. There is an override existing caption file, append new caption file, and max new tokens. You don't need to set these parameters. You can just change the max new tokens. I'm going to make a dedicated tutorial for this application hopefully later. You
can see the progress on the CMD here. So first it is loading the checkpoints to start captioning. The captioning started and it is captioning the files. When I enter inside this file, you will see that it is generating captions like this. When you open the text file, you will see the caption it generated for that image. However, if you do captioning, it will reduce the likeness of your concept. If you are training a person or if you are training similar stuff. Captioning with styles may work better. I am hopefully going to test it, but still,
if you want to do captioning, you can use this application to caption. I have also compared the captioning effect. So let's search the caption in this post and let's see. Yes, here the captioning results are here. Let's open it. And when we look at the caption results, There will be a slightly reduced likeness. However, I didn't see improvement in the environment or generalization or the overfitting. This is because it already captioned itself in the FLUX architecture. So whether I caption or not, it doesn't matter much. The likeness is still there. It's a little bit reduced
and it doesn't improve the overall quality. So for person training, I don't suggest captioning, but for training a style, it may work better. As I said, hopefully I am going to research it and make a tutorial for style captioning. After you did the caption, you still should add an activation token. What was the activation token? "instance prompt". So you can either edit them manually or I have a caption editor. The caption editor is posted here. When you go to this link, you will see the instructions and downloader zip file. It's a very, very lightweight Gradio
application. Let's open the image caption editor, start the application. It doesn't use any GPU. It is just Python code that allows you to edit the images. It is web-based so it can be used anywhere. So you need to enter the input folder path. This was our folder path. Let's Enter it and it will automatically refresh and scan the images. You see, these are the images. When I click here, it will show me the caption. I can either manually edit it here like ohwx. Then I can click save caption and it will save it. Then I
can click the next image. Then I can filter images by processed or unprocessed. This is an extremely useful application. So with processed, you see this was the saved processed and these are the still waiting ones. This is a very, very lightweight application. Let's make this again as "man" because I am going to show you another thing. Let's save the caption. Then in the batch edit options, you can enter the folder and you can replace words like replace "man" with "ohwx man" and replace all occurrences and it is going to override all the files. Apply batch
edit and all captions are edited. Then you can check all the captions to see whether they contain my activation token or not like this check word and you see all captions contain the specific word or phrase. Then when I refresh here and open an image, you see this photograph features a "ohwx man standing." So all the captions are Now ready to use. Since this folder exists in my images folder, it will train both of them, but we don't need captioned images now. So I'm just going to delete this folder. OK, let's return back to the
Gradio interface where we set up. After we clicked, prepare training data, we also need to click "copy info to respective fields." Otherwise, you will see that the image folder is not accurate. You see it is inaccurate. So click "copy info to respective fields" and you will see that it did set the folder like this "train images img." So what does this mean? It didn't give the path of this folder. It gives the parent folder path here so you can have multiple concepts, multiple persons or items, anything inside this folder, and Kohya will read each one
of the folders and whether you have captioned them or you don't have captioned them, it will read the folder name or the captions and it will train based on all of the images. You can also set different repeating numbers for each concept, as I have explained. Let me show you. Let's say I have 3 concepts. This can have 3 repeating. This can have 5 repeating. So the images inside This one will be repeated 5 times. Images inside this one will be repeated 3 times. This is the logic of setting up folders. Let's delete them. Now
we have set the model path, trained model output name, whatever the name you give, it will generate checkpoints with that name. Let's say "Best_v2." OK, this will be the output model name for LoRAs that are going to get generated. FLUX1 selected. Everything is selected. You don't need to set anything and you don't need to use dataset preparation again, but you can prepare multiple datasets by using here. With Stable Diffusion XL and with Stable Diffusion 1.5, we use regularization images. At the end of this tutorial, I will show that too. So I'm just skipping that part.
And the parameters. In the parameters what you need to set are a few things. First of all, how many epochs you are going to train? We are going to train based on epochs and one epoch means that all of the images are trained one time. When we use regularization images, we use a different strategy. We train 200 repeating and 1 epoch. But since we don't use regularization images and since we use 1 repeating, we are Going to use epoch strategy. So if you have 100 images, you can reduce this epoch number. However, you can still
train up to 200 epochs and compare checkpoints to get the very best checkpoint. So as you increase the number of images for a single concept, you may want to reduce the number of epochs. Currently, most people will collect like 15 to 25 training images, maybe 50. So training up to 200 epochs and comparing checkpoints is the best strategy. About the training dataset: this training dataset is not great. Why? Because it contains the same clothing, same background environment, and it is lacking expressions. Many people are asking me how to generate expressions. If you want to generate
expressions, you need to have expressions, emotions in your training dataset. I am preparing a much better training dataset. It is not ready yet. I am still using the same dataset so I can compare it with my older trainings. However, hopefully I will make another video for an amazing training dataset. You will see that. So when you are preparing your training dataset, have different poses, have different distances like full body shots, close shots, have Different expressions that you want to generate like laughing or crying, whatever you want, have different clothing and have different backgrounds. The quality
of the dataset is very important. With FLUX, it is still very flexible. It is better than the SDXL or SD 1.5. However, as you improve your dataset, you will get better results. One another thing is that make sure that your dataset has excellent focus, sharpness, and lighting. This is also very important. Do not take pictures at nighttime. So make sure that your images have very good lighting, very good focus, and very good sharpness. OK, let's return back to Kohya GUI. So I will leave this at 200. You can reduce this based on the number of
images you have, how many checkpoints you want to get. You can make this like 10, and after every 10 epochs, it will generate a checkpoint. And at the end of training, you can compare all of them. I am going to show you how to use and compare them. There is no issue with it. So you can make this 10 and compare all the checkpoints and find the very best checkpoint you liked. So this is the most optimal way of obtaining the very best model and checkpoint. But for this one, let's leave this as 25. Caption
file extension. If you are going to use captions instead of the folder names, you need to select the correct caption extension. By default, it is TXT and TXT is the most used one. You can also use caption and cap. I never used them. TXT is just working fine for me. Then you don't need to change anything else here. By default, we are using 128 to 128 network rank. OK, this is another thing that you need to set. This is super important: VAE path. So for VAE path, we set this ae.safetensors file. It is already downloaded.
Let's go to the folder, which was inside here. I had downloaded here and ae.safetensors file. And we are going to use clip large model. Click here. Let's go to the clip large. OK, it is also set and we are going to use T5 XXL. Make sure to use T5 XXL FP16. It will be auto cast to the correct position. So let's also pick that file from our downloaded folder, which is here. And we are all set. You don't need to do anything else. Just save the configuration wherever you want. So I will save the downloaded
folder by clicking this save. If you want to reload, you click this or Click refresh to reload. You see the refresh will refresh. You can also re-pick, but sometimes when you re-pick, it may not refresh. So click this to refresh. Let's select the saved config again, which was this one. And let's refresh. Yes, I can see the configuration reloaded. Currently, none of the configurations are training text encoder clip large. As I said, I am researching it right now, and hopefully I will update the configuration according to it. The training of clip large will increase the
VRAM usage slightly. It increased by like 800 megabytes on 16-bit precision. It will also slow down slightly. You see from 8.82 seconds IT to 8.56 seconds IT on a 16-bit precision training on a 6000 GPU. OK. As I said, we are ready. And before clicking start training, you need to check your VRAM usage. Try to make your VRAM usage under 500 megabytes. Currently, you see my computer is using 3.5 gigabytes. Why? Because I am running OBS studio. I am running NVIDIA broadcast and some other applications. So try to reduce your VRAM usage to 500 megabytes
if you are on limited VRAM. How can you reduce it? Go to the startup, disable all of the startups, Restart your computer, and check the performance and GPU and see how much VRAM you are using. Alternatively, you can also use another application, which is nvitop. So to install nvitop, pip install nvitop. I already have it done. Type nvitop and it will show you the VRAM usage, GPU utilization, how many GPUs you have. You see I have two GPUs. So this is the way of also seeing exact VRAM usage. You see this one shows 3.8 gigabytes
I am using. This one is showing 3.5 dedicated GPU I am using. So you can use either way. Then click start training and watch the progress on the CMD window of the Kohya. You see it is starting everything. You should read the messages appearing here. If you see this error, xFormers can't load. It is not an issue. Currently, we are not training xFormers. Actually, I had fixed this, but I am going to fix it again. But it is not an issue because currently xFormers is not working better than the default cross-attention, which is sdpa. I
will also fix xFormers later, but you can ignore this xFormers message. It doesn't make any difference. We are not using xFormers. So what are The messages we see here? This is important. You see it has found 1 repeat. It has found 15 images. So one epoch will be 15 steps because the batch size is 1. Increasing the batch size, I have tested it too. It doesn't bring almost any speed gain. And as you increase the batch size, you will get lesser quality training. Batch sizes should be used only for speed. And in this case, it
doesn't increase much. But you can use multiple GPUs to get almost linear speed increase. I have explained the batch size in the Patreon post. So read that section and you see the regularization factor is 1. Total steps 15. Train batch size is 1. Gradient accumulation steps 1 and 200 epochs. So it calculates the number of total steps, which is 15 divided by 1 divided by 1. Why? Because this is the batch size. This is the gradient accumulation steps multiplied by 200. This is the number of epochs and multiplied by 1. This is the regularization images,
which we don't use. So it is 1. When you use it, it is 2. And total, it will be 3000 steps. I hit enter because when you touch the CMD, it will pause the CMD window. And you will also See that it didn't find any captions. It didn't find regularization images. So it is going to use class tokens or ohwx man, which it reads from the folder name. So everything is looking good. And it will load and start training. The speed is not very great. And you will get better speed as it progresses. In the
beginning, it will show not an accurate speed. Once the Torch 2.4.1 published, we should get speed like 4 to 5 seconds IT on Windows with an RTX 3090 GPU. But as I said, don't worry, I'm going to make cloud tutorials. And it is going to be super fast on cloud GPUs. You will be able to rent very powerful GPUs. And it is perfectly trainable. So this is how we do training on Windows. It will be the same on Massed Compute and RunPod as well. The procedure will be the same. Just the starting and installation will
not be the same. So just wait until these checkpoints are saved. And how are we going to use them afterward? Since I already have trained them, I am not going to wait for training. In this repository, I have the training files, which is trained with exactly the same configuration. You see the Best_v2. So I'm going To download several checkpoints to test on my computer. When you are training, there is one thing that is extremely important that you have to be careful of. Open the task manager, go to the performance tab, and go to your GPU.
You see that there is dedicated GPU and shared GPU memory. When you are doing training, if it uses shared memory, it will become slower like 10 times, 20 times. So this is the criterion that you need to be careful of. Compare the shared GPU memory usage before starting the training and during the training and make sure that it doesn't use more shared GPU during the training. If it does, then you will have extremely slow training speeds. This is really important. This is also valid for all of the AI applications you use, like Swarm UI, like
Forge Web UI, like Automatic1111 Web UI, whatever that comes to your mind. When it starts using shared VRAM, that will make your application slower like 20 times because it will start using the memory of my computer and the memory of my computer will be way slower than the GPU memory. Moreover, if you have low RAM in some applications, you may encounter a problem, but it is not an issue. You Can set virtual RAM memory. So you see currently I am training and I am not using any shared VRAM. My dedicated GPU memory still has space
to go like 3 gigabytes. This is super important. And about RAM memory, you can set virtual RAM to have more RAM memory. Memory is usually important when it is first time loading. So if you have limited RAM, you can increase the virtual RAM memory to avoid any errors. To increase the virtual RAM memory, click this PC. In here, click this PC properties. It will open these properties. In here, go to the advanced system settings. And in this screen, you will see the settings in the performance. And in here, go to the advanced and you see
I have virtual memory. Change it. Set one of your fast drives and you can set a custom size. Currently, my virtual RAM memory is set here. You see system-managed size and it will allocate as necessary as virtual RAM memory. OK, this is the way of it. So during the training, you will see that it will save checkpoints like this. And you will see their saved locations and where they will be saved. They will be saved in the outputs folder. You see the output directory For trained model. This was automatically set by the GUI when we
used the prepared training data and copied info to the respective fields. After you clicked "copy info to respective fields," you can change the output directory for training the model. You can directly give the folder path of your Swarm UI, your Forge web UI, wherever you are using your Comfy UI. So my models will be saved automatically here. When we open that folder, we will see the saved checkpoints like this. So these checkpoints are 2.4 gigabytes. Why? Because the FLUX model is very big. It has 12 billion parameters. Moreover, we are saving it as float. So
we are saving it with the maximum precision without any quality loss. If you want to reduce the file size, you can also save it as FP16. I haven't compared it. Some of the followers say that BF16 is working badly. So if you need a lower size, save it as FP16. If you want to have the maximum accuracy, save it as float. Saved precision will not change VRAM usage. Will not change the VRAM usage during training or during inference. So now it is time to use the saved models with the FLUX model Itself. So how are
we going to use them after training has been completed? You can use them with Comfy UI, with Swarm UI, with Forge web UI at the moment. I prefer to use in Swarm UI. I already have a main Swarm UI tutorial. It is amazing. And I also have a Swarm UI tutorial for FLUX. Moreover, I also have a Forge web UI models downloader and installer for Runpod and also for Massed Compute. I will show the cloud service providers in the cloud tutorial, but I will show how to use it on my computer at the moment. So
I will move these files into my Swarm UI models folder. You see it also saves the TOML file and the JSON file wherever you are saving them. So let's select the generated models and move them into our Swarm UI model folder. My Swarm UI is installed here. When you watch the tutorial, you will see it. I will put the generated LoRA files into the models into the LoRA folder here. This is important. This is where you need to put them. Before starting training as I said, you can give this folder path. So after you have
set the copy info to respective fields, go here and change it like this and save your configuration. Then it will save the Generated LoRA files into this folder for you. OK, so I'm going to start my Swarm UI. First of all, I am going to update it. When you are using Swarm UI, you should update it always. You can also use Forge web UI or ComfyUI. I prefer Swarm UI because it is amazing. It has so many amazing features. It also uses the back end of ComfyUI. So I will use the launch windows.bat file to
start it. And my ComfyUI is starting. There is also a trick that I'm going to show you. This will give you a lot of performance boost. If you have an RTX 4000 series GPUs, click this pencil icon and add extra arguments --fast here and save it. This will hugely speed up your generations with Swarm UI when you are using the FLUX model. Backends are still loading. Go to the server, go to the logs. Let's see in the debug. So we can see that it is starting the server. It is loading everything and the data is
getting refreshed. OK, so now I am ready to generate images. For the FLUX model. I am going to use CFG scale 1. When you use CFG scale 1, you see the negative problem is disabled because FLUX doesn't support. I prefer 30 steps. Based on your GPU, You can set it. I'm going to use resolution 1. I'm going to use sampler Uni PC. I find this the best. The scheduler will be normal and you can set the preferred D type here. Since this is a 24-gigabyte GPU, I am going to use this one. However, you can
also use 16-bit on a bigger VRAM-having GPU. This is a good one. There are also quantized models, but I haven't tested LoRAs on them, whether they are working or not. So I don't know. And I can't say they will work. And in the models, I have FLUX 1 version dev FP8. You can also use the FP16 model. They should work exactly the same actually. So to be sure, I am going to use the FP16 model. I already have it here. So let's copy it. And let's copy it into our models folder. It goes into the
Stable Diffusion. Let's see. Yeah, it goes into the UNet because this is not a quantized model. It goes into the UNet. The quantized models go into the Stable Diffusion folder in the Swarm UI. I could also give the path of this file from this folder before doing the training. So you don't need to have duplicate files. What would be the case? The case would be I click this icon. Then I go to My Swarm UI installation, which is here. Swarm UI inside models, inside UNet. And I can select this. So you see, you can give
any file path in Kohya GUI. It will just work. Then refresh the models and the FLUX development arrived. This is an FP16, 16-bit precision model. But since I'm going to use 8-bit, it will auto-cast it. Before starting using the LoRA, let's generate an image. And a very good part is that in the zip file, I have shared some test prompts. So let's open the test prompts from here. And for example, let's generate this image without our LoRA. And let's hit generate. You can watch the progress in the server, in the logs, in the debug. So
it is going to load the model. And you see model weight dtype, manual cast. So it is going to do everything automatically for me. I can watch the VRAM usage here. It is using ComfyUI, thus it is very, very optimized. I have selected the model. You see the FLUX 1 development model is selected. As I said, I am not sure whether it is working with the quantized models or not. I haven't tested yet, but you can test. For now make sure to test with the development model. Then you can test on quantized models as well.
And this Is my VRAM usage currently. Don't worry, you don't need such VRAM. It works as low as 6 GB having GPUs. It is extremely optimized. Since I have more VRAM, it is using more VRAM. When you have lower VRAM, it will use lower VRAM. Moreover, I can also use my second GPU. It is amazing. So all I need to do is I am going to add a new ComfyUI self-starting click here. OK. I will make this like this. Extra arguments, starting script. And I will make this GPU ID 1 and save. So when I
generate multiple images, it will use my second GPU as well. OK. The image is almost ready. I am also using segment to in-paint face. This is not mandatory. In the main tutorial, I explain everything. It is a little bit slow on my GPU, but you can use the cloud always. We can see the IT per second. OK. Where is it? Let's generate another image to see it. Let's also see. Yeah, it is fine. OK. Now it is starting to in-paint the face. OK. Face in-painting speed is 1.5 seconds per IT. And it is doing 18
steps. Why? Because I set it at 0.7 denoise strength. It is called different here. It is called image creativity. So it is doing 70%. And since I use 30 steps, it is doing 21. Actually, it should do 21, But it did 18. OK. This is the image we got. Now I prefer to use FLUX guidance scale 4. I forgotten that. OK. Then how am I going to use my LoRAs? Go to the LoRAs tab here, refresh. And once you refresh, you will see your FLUX LoRAs. You see type FLUX 1 LoRAs. For example, let's use
the 150 epoch and hit generate. So now it is going to use my LoRA. We can see in the back end. It will load. You can see everything here. Swarm UI working much better than the Forge web UI. When you are using LoRAs, when you are using Forge web UI, it first processes LoRAs. It takes extra VRAM. However, with Swarm UI, it doesn't do that. Also, when you have more VRAM, it will be faster. Since I am using some serious VRAM right now. It is using a lot of VRAM. I am recording a video. It
is also making. And let's also close these two. Try to reduce your VRAM usage as much as possible. And I am already using a lot of VRAM when I am recording. We can see the preview here. First, it is generating the image. Then it will in-paint the face to my face with the prompt photo of ohwx man. It is in-painting the face now. This is not mandatory but in-painting the face, Especially in distance shots, will improve the face quality. Moreover, in my training images, I have eyeglasses. You see? But since FLUX has an internal text
encoder, it is able to separate my eyeglasses from my face. Thus, in this image, I don't have eyeglasses. This is also a little bit overfit image. But I am working on a better workflow. Hopefully, I will update it. and it will become much better. So you can add here with eyeglasses. And you will get a more likely image. Let's see. Let's hit Generate and double-generate. So once I click it double times, it will also start using my second GPU. It should start. Let's see. I think first it will load into the second GPU. Then it
will start. OK. Let's see the server back-ends. OK. OK. It shows here running back-end. OK. For some reason, it didn't start. And if preview disappears here, you can go to the image history, refresh. The last generated images will always appear here. You can see them, their features. FLUX guidance scale, the LoRA. You can also set different LoRA weights from here. OK. Image is generated. And you see it is now much more resembling. We can see from the training Images, it has a perfect resemblance. It is, as I said, more overfit. With the fine-tuning, full model
training I think this overfitting problem will be solved. I am also doing more research right now, as I said. So I will update the config as I get better config. Every day something new is arriving. Also, you may use overfit, over-trained model checkpoints. So how are we going to test and find the very best checkpoint? This is a super important part. Let's see also the other generated image. If you have an RTX 4090, it will be way faster than this. It will be many times faster than this. You can always see the step speed here.
It is around 1.5 seconds per IT for me. And the second image also generated a perfect resemblance. OK. So how am I going to find the very best checkpoint? To do that, we go to the Tools. And in here, we select the Grid Generator. In here, in the first tab, I use LoRAs. So find the LoRAs from here. Let's find it. You can also type to filter like this: LoRAs. When you click Fill, it will fill all the LoRAs. Then you can delete the ones that you don't want to test. Let's start testing from the
50 epochs. So these are epochs, Epoch 50, epoch 75. If you save based on the step count, it will have a different naming. So the last one will be the 200 epochs. Then you can test multiple prompts or you can use this prompt for testing. If you want to test multiple prompts, I have already prepared prompts. So these prompts are shared here. You see test prompts, no segment in-painting. This doesn't have any segmentation in-painting. And I have test prompts here, test prompts with eyeglasses. And test prompts, let's see. One of them doesn't have eyeglasses. OK,
this doesn't have. So I am going to change this name. I will fix this. Test prompts without eyeglasses, grid formatted. So you just copy this. By the way, in the grid, this is the prompt separator. You see like this. Then copy-paste it here. You see it has split each prompt. And now when I generate a grid, it will test all of the models for me. Let's see, generate. This time it should use the second GPU as well, I believe. Let's see what will happen. OK, now it started loading onto the second GPU as well. So
when generating this grid, it will use both of my GPUs. However, with my GPUs, this will take huge time because there are 274 generations. This is estimated this will get better. But this will make a huge grid. You should wait for it to update. You can rent for a 6000 on Massed Compute and use all of them at once. It will be much faster. In the Massed Compute tutorial hopefully, I will show you. OK, the first image is generated. The first image for verifying the model sanity. You see it is perfectly able to generate a
supercar image. Nothing like me. So the model sanity is perfect. It is still keeping its sanity. And what kind of results are we going to get after these grids are generated? For now, I will interrupt this with "interrupt all sessions." And I will show you from my cloud computing. So in the Swarm UI Massed Compute tutorial, there are these new instructions. Copy this. First of all, you need to install this. OK, let's copy this and open a new terminal here. New window and paste it and hit enter. This will start Swarm UI on Massed Compute.
But it will give me a cloud URL that I can use on my computer. It will be here. It is Cloud Flare. Let's copy and paste it and open. And this is where I do my testing, my experimentation. When I go to the tools and Grid generator, I have so many previous tests like this. You see, I have 8 GPUs running there. For example, let's open one of them. OK, let's best new 150 epochs. And when I click here, it will show me the grid results. I explain all of this in the main tutorial. In
the advanced settings, you can select to show which prompts you can select, which models to show. And we will get a grid like this where we will be able to compare different checkpoints LoRAs. So by analyzing this grid, you need to find your best liked checkpoint. Usually 150 is good, but it depends on your training data set. So generate a grid, analyze all of the generated images. You will see at the top the model used. Also, when you click the image in the bottom, you will see which LoRA is used. For example, for this one,
the used LoRA is. Let's see this one. Best v1_5e_05, 150. Also, it shows the LoRA used name here. So this is the way of finding the very best LoRA checkpoint that you have. For example, on this Massed Compute, I have 8 running GPUs and I am able to generate ultra-fast images. This is how I do my experimentation. There Is no other way to do these many trainings on a single GPU. So you need to spend a considerable amount of money. Thankfully, Massed Compute is supporting me. So if you are wondering how did I generate those
amazing pictures that I have shown in the intro, I have generated amazing new prompts and I used the wild card feature of the Swarm UI and I have written all these posts in this wild card and I have generated nine thousand nine hundred ninety-nine images until I stopped it. The new prompts are shared in the test prompts in the 340 prompts used as wild cards. OK, we have covered everything and there is also extra information here on how to train on and use on Runpod and Massed Compute. The instructions are already included in the file.
You will see the Massed Compute, Kohya FLUX instructions. You will see RunPod install instructions. Hopefully, I'm going to make separate tutorials for them. Also, I suggest you save your models on Hugging Face if you want to save them. Download them fast. I have an amazing notebook and tutorial for that. Watch it. And it is also asked of me how to do SDXL and SD 1.5 training with the newest Kohya Interface. So everything is the same, almost the same. Therefore, I will quickly show you. For example, let's begin with SDXL. Let's open the best configuration. In
the very top we will see the configuration. Here. Let's download the Tier 1 24 gigabytes Slower V2. There is also a Tier 2 low VRAM version and the configuration loaded. Let's close the Swarm UI and let's open our latest Kohya installation, which is here. Let's start it again. The configuration is downloaded. Kohya is starting. OK, then these are full fine-tuning configurations, not for LoRA. So select the DreamBooth app here. Go to the configuration. Click this icon. By the way, this is SD3 FLUX branch. So there may be some errors. You need to install it the
normal way. If it doesn't work, I don't know. I didn't test it. So let's go to the downloads. Select the file and it is loaded. You see this is DreamBooth. SDXL is selected. We do FP16 training. Now you set everything exactly the same. Trained model. Output name. You put the image folder name. Pre-trained model path. But this time what differs is you should use regularization images. And regularization images Are posted here. I have amazing 5200 images for both women and men. So what you do when you go to the dataset preparation, you also put regularization
images and you put 200 repeats or the number of repeating may depend on your number of training images. Let's say you have 50 training images. Since we have 5000 regularization images, you can make this 100 and you can train 2 epochs. But let's say I have 15. So I make this repeating 200. Put my training images path here. Then I also go to the downloaded regularization images. Let's see. For example, these are 768. I also have 1024. Yeah, here. This is an amazing dataset. Super high quality. I tested and compared the effect of using regularization
images and it's mind-blowing, especially with OneTrainer. And you put it here. This time repeating will be 1. You just type ohwx and man it will auto-generate the folders for us with accurate naming. Let me show you. So let's make the destination an example place like music folder D. Usually you shouldn't Use the users folder. But this is just for an example. Prepare training data. Watch the CMD window because copying may take time. Wait until you see done. Currently it is copying everything. We can see image 200 ohwx man is copied. And it is also going
to copy man datasets. OK, this one copied. And in the reg folder. OK, this one copied. You see the regularization images are put into the reg folder and the training images are put into the image folder. And inside the reg folder man images are named as 1 man because one repeat and man class token and the training images are 200 and ohwx man. So what other thing changes? Now we train only 1 epoch. So how are we going to save checkpoints in this case? To save checkpoints in this case, let's copy info to the respective
folders and let's also select the pre-trained model path. I have already models here. Like let's see SDXL base, for example. By the way, which models I suggest. I suggest you use Realistic Vision Version, RealVis XL 4 for training SDXL and for training SD 1.5. I suggest what was the name: Hyperrealism version 3. So you should select those models. And when you click the print training command, It will show you the total number of steps. And it is now 6000 because I have 15 training images. I use regularization images and I have 200 repeating. Therefore, let's
see if it does show the calculation. Yes, 200 repeats, 15 images. It makes 3000 steps per epoch. Since regularization images are used, it is multiplied by 2. So total 6000 steps. And let's say I want to save every 20 epochs. So normally it would be save every 20 epochs. But now I don't have that option since I train 1 epoch. So I will save the number of steps. When you divide 6000 to 10, because we want to get 10 checkpoints, you will change the save every N steps. Let's see here. So if I make this
601, it will save as equal as saving every 20 epochs compared to the repeat 1. This is the logic of it. This will save 10 checkpoints during the entire training. And this save every N epochs is 1 and epoch is 1. This will save full checkpoints. For SDXL, it will be over 6 GB. For SD1.5, it will be like, I don't remember actually, but it will be big. So if you need LoRAs, do this way of DreamBooth fine-tuning training, then extract LoRA. It Will work much better than training LoRA with SDXL or SD1.5. I have
tested it. For extracting LoRA, I have an amazing post here. It shows how you can extract. Let me also show you quickly. Go to the Utilities. Go to the LoRA. And in here, you will see Extract LoRA. Whether it is SDXL or not, you pick it here. If you don't pick SDXL, it will extract as SD1.5. If you pick SDXL, it will extract as SDXL. Currently, it doesn't support FLUX. I don't see it here. Oh, here, click here. Extract LoRA also arrived. I haven't tested it yet. I will hopefully test it after I train full
fine-tuning. So select your fine-tuned model. This is the generated model. Select the base model. This is like RealVis XL 4. Set the path where you want to save. Save precision. You can save as float. It will double the size. Load precision. You can load as float. Set this minimum difference 00001. So it will also save the text encoder because I train the text encoder as well. And you can set the network dimension 128 to 128. This is it. For SDXL, you just select SDXL. And that is the way of extracting LoRAs and using them when
you do training with SD1.5 and SDXL. In my post, You will see very detailed instructions. So always read the posts on here. Moreover, you should join our Discord channel. I am always on the Discord channel. You can message me there. At the very top of the post, you will see SECourses Discord. When you click it, you will see our Discord channel. We have over 8,000 members, over 1,000 online members. Just click Join Server to join. Moreover, we have a Patreon exclusive post index. In here, you will see all of our amazing sharings. You can just
do ctrl-F to see them. Also, we have Patreon Scripts Updates History. You will see which scripts are updated last and what changes are made. Sometimes I don't write the full changes. And we have Patreon Special Generative Script List. This shows the useful scripts that you can use for other tasks, other jobs you have. And you can also go to our GitHub repository here, Stable Diffusion Generative AI. When you go to here, you should actually please Star it, Fork it, and Watch it. Also, if you sponsor, I appreciate that. You see, we have 2,000 stars. We
have 200 forks and 82 watching. Moreover, we now have a subreddit. Go to subreddit, SECourses. I follow every comment and post made here. I will reply to all of them. I am also sharing a lot of announcements here. And I have a real LinkedIn account. I am not an anonymous person, obviously. You can go to my LinkedIn account. You can follow me. You can connect with me. It is fine. I also reply to every message here. This is it. I hope I have covered everything that you have been wondering. Hopefully, see you in the future
tutorials for FLUX because a lot of tutorials are coming like training on Runpod, training on Massed Compute, and fine-tuning. I think fine-tuning will be way better. And I am going to update this post and write the very newest findings that I have. Also, you should check out this lengthy research post because you will find a lot of information here. Let me show you one thing. For example, FLUX training discussions with lots of information on this post. You can open them. And there is new information that shows why Windows training is currently slower than Linux training.
With Torch 2.4.1 hopefully, we are going to get an amazing speed boost on Windows without doing anything and without Losing any quality. And it is 75% completed. So it is almost there. I will update my installer scripts. Don't worry about that. Hopefully, see you in another amazing tutorial video.