have you ever wondered how the auto-generating subtitles are created well in this video I'm going to show you a simple python script where I use AI to automate this process and produce your next Instagram reel from a video let's get started the plan is at first to get the audio from the video then we put the audio through an AI called whisper which gives us an entire transcription for the audio next we'll take the video and extract all of the frames and crop the frames it's to a horizontal 16 into 9 ratio which is ideal
for the Instagram reels and YouTube shorts next we'll take the segments provided to us by Whisper and slice up the transcription and put it into our corresponding frames and finally we'll gather all the frames and produce our output video at the end first let's open a python file named anything.pi yes we only need one file for this project next we'll import the necessary libraries here oh and if you don't have a particular Library installed simply type pip install Library name in the terminal to install the library you can install multiple libraries in one line by
putting in space after each Library name then we'll create a video transcriber class with the necessary Constructors as we are going to use the whisper AI to get the text transcription from the audio we are going to load the model in our Constructor here we need the rest of the constructors in our methods below so stick with me we now will write the transcribe video method we transcribe the audio using this line and store the text in the result variable the audio Dash path is empty at the moment as we haven't extracted the audio from
the video yet the result variable contains a dictionary with real information about the transcription let's look into its segments key and see what's in there so as you can see there's a lot of information in this segment key like text tokens start time and end time Etc but we don't need all this for all this information we just need the starting time the ending time and the text within it we get the size of the text in pixels by the get text size method from CV2 this will help us divide the text segments into smaller
parts which would fit the screen with the following lines tells us more information about the video like width height aspect ratio and FPS and using some fancy maths we find the length in pixels for a character on our screen next for each of the text segments and using the start and the end time of the segments we find the number of words which would fit onto the screen at a time we find the total number of frames each segment should be shown using the start and the end time with FPS next we create an array
of words from the text segments and iterate over the words we find the length in pixels for each word and subtract that from the width of the screen to find how many words would fit at each given time you can pause here to understand the following lengths of code on how I did that we're keeping the subtitles only in one line for this video this will give us a new array of which has the words the starting frame number and the ending frame number this area will be used to show the text in the screen
for the corresponding frames this marks the end of our transcription now comes the extracting audio part this is done easily using the movie Pi package we load the video in the video file clip function and extract the audio using the audio method then we write the audio file in a given output path our next method would be the frame extracting method here we'll Loop over the frames of the video using opencv and put the text on the frame using the areas we have created earlier basically we are counting the frame numbers and if the frame
number matches between the starting and the ending frame and the associated text will be shown on those frames also we break the loop if the text matches the condition so that we don't have a time complexity of O of N squared rather than o of n finally we'll be writing the method which will create the video using the extracted frames this method creates a folder named frames on Parent Directory if not found already and then calls the extract frames method from the above after the frames are extracted we use image sequence clip function from movie
pi to create the video the function takes in a list of image paths as an input we also set the audio on the video here finally the Right video file method exports the output video to test our script altogether let us first see the example video we'll be using for this the different speech sounds have been molded into sentences such the consonants and usual compounds occur in the vowel combinations which are met with very frequently in English for lack of time to run the script here we specify that we'll be using the base model of
the whisper Ai and we provide the input video as well as the output name and location and here is the final result hello everyone on the 28th of June 2007 we are in the BBC Kingswood war and anakoic chamber auspicious day indeed brain is used to processing is these days absolutely absolutely most uh broadcasting let alone recording is done very very close to the microphone which imparts a completely different kind of the sound there are many things we could do to improve the script so this is just a general way of doing this as usual
I will have the code posting on GitHub and the link can be found in the description below consider liking and subscribing if you like the video and also follow me on my social media so you can talk to me directly what video selection or anything peace out and I'll see you in the next one