Game Engine Programming 011 - Testing entities system and creating the engine DLL | C Game Engine

hello everyone and welcome to the game engine programming series where i write a game engine from scratch i'm certainly glad that you could join me today because we are going to write some easy and fun code in this video the code will not be a brain teaser but necessary nevertheless because we still have to check whether the code that we wrote in the last two videos is actually working correctly therefore i'm going to write a test program which will also be the base for any other tests for the engine that we might be writing in

the future also today i'll write the dll so we can expose the functions of engine library to the editor and i'll also write code to import those functions and use them in the editor and so we'll have the very first interaction between the editor and the game engine in other words today we'll officially be using the game engine which is rather exciting i can't wait to do all this so let's get coding [Music] to write a test program we'll need another c plus plus project that i'm going to add now i'll use an empty project

for this so it's a c plus console or windows empty project again i have these filters that i don't need so i remove them and the first thing i'll add is the main cpp file and here i'll just add the entry function which is main and in this function i'll just call my test routines to test whatever part of the engine i want to test so for that i need an interface for all the tests that i'll be writing and that's something like this this is an abstract class from which all my tests will inherit

and then i will just put this in its own header file so i can include it to each test file that i'll be writing now imagine i have a class that inherits from this test class then in main function i could just call these three methods member functions that i defined here to initialize run and shut down the tests of course it gives me an error because i haven't implemented the interface so for the sake of this example i'll just make a couple of empty member functions for this class now in the main function i

can just call these methods so if the initialization succeeds i can run the test of course i need to make these public and after i'm done with the test i just shut down this test so it can it can clean up anything that it needs to clean up before exiting the program now for example i can have a couple of defines here that would enable or disable certain tests today we are going to test the entity and component functionality of the engine therefore i'm going to define a macro that is just for enabling that particular

test so here i enabled this particular test that is about the testing of entity and components and after this we can put any other tests that we write in the future and if none of those are enabled then we will get a compile error because well we have to have a test to run in the main function and one other thing that i want to do is to put this in its own file so we just include those files that we need for each test so in this file i'll include the test what i want

to do here is to test functions from the engine and those are in a namespace the primal namespace so i'm going to just do and using namespace normally i don't do something like this in a header file because well then it will be propagated through every cpp file in which we include this header so that's not a good practice to do this but this is a test project made specifically for the engine so it's okay to have to get rid of this primal namespace everywhere and because i'm going to use these headers from from the

engine i'll need an additional include path in the project so i'll add that here for all configurations i'll have additional paths to include directory to this common header files to this common folder and i'll be also using the library itself to link to so i'll also have additional library directory but the library itself is in the output directory so i'll just use another micro that's the out directory and that's basically it now here i can just include uh the headers for the entity and transform because those contain the functions that i'm going to test and

here in the main i need to include this header and also tell the compiler which library it needs to use right okay right now we should be able to compile and build this project let's see if that is okay right it tells us that the namespaces that are nested well if we want to use a namespace like this then we at least need c plus plus 17 because that's the feature that is introduced in c plus 17 and later so i'll have to set the language for this project to c plus plus 17 as well

and it worked there are a couple of more things i would like to do here before we start writing the test first i'll just check for any memory leaks that might happen if we do things in the engine so we will catch those here during our tests and one way of doing that in the debug build is a c runtime function that we could set to detect those leaks there are a lot of flags that we could set to detect different kinds of memory leaks the ones i'm going to use for now are this one

and we can r those flags together so these are the flags that i set and what these exactly do well we can just google the documentation and see what uh what this function actually can do for us so there is in this microsoft documentation and here is are the descriptions of each of those flags that we can set and there are a lot of tests that we can do to detect a lot of memory defects that we can have in a in a program and well the ones that i was using is just the allocation

memory df and that's this every time we allocate space from the heap then we also have to release it and that's what it checks and the other one is leak check oh this one is just for error report so if we have any leaks then it will dump but it was leaked in the memory in in the output of visual studio so now for example i can simulate this by allocating memory here and then not releasing it so now i'm allocating space for four floating point values and then i can run this program and see

what happens here you can see that it says okay there is at this location 16 bytes that we leaked and well that's correct because each floating point value is four bytes long and four of them are a total of 16 bytes so that's correct so we can detect all the memory leaks in the engine and further i am done here in the main so we don't really have to do anything except well of course put this in its right place so for example if you have any other tests that we want to do we can

just disable this and then it will pick any other test that would be enabled but right now we just have one and it goes to this error because well now we don't have any tests that we can do so the way i want to test this addition and removal of entities in the engine is to just add the random number of entities to the engine and then remove a random number of entities from the engine and repeat that a couple of times and see if anything goes wrong if i can check whether all the numbers

are same so well we can check if then anything weird is going on so here i have included these two c plus plus headers the first one is for me to be able to output messages to the output window with the c out like so and without this header we won't be able to do that i need c time to generate a new seat for every time we run this program so there is a function the c function that's this random seed if you're familiar with how pseudo random numbers work then you know that every

time we run a program and it uses the same seed then it will generate the exact sequence of random numbers every time so without this line the first number we might get is maybe three and the other one the second one that we asked for is five eight and maybe two and so on and every time we run this program we will get the same sequence of random numbers which might not be what we want so using this we can have a seat so it starts somewhere else in their sequence so the second time we

run this program with another seat then it would produce a different sequence of random numbers and that's what i want for this test to happen and other than that we don't have we don't need any other initialization for this to happen so i just returned true and the same is for shutdown i don't have to free up anything here for this test so the shutdown method is empty i am going to create entities and remove entities so i need a list of entities that i have during this test and i'll just remember how many entities

i added during the test in total and how many entities were removed and at any given time how many entities i have which is just the size of this array right so here i'm going to run this test until the user presses q for example to exit the test and then i'm going to randomly add and remove entities so here i'll have two functions that create a random number of entities and remove a random number of entities and then i'll just remember how many entities i have at this moment now i can go ahead and

implement these two methods and here what i want to do is to add maybe somewhere between 0 and 20 entities to our engine and then see what happens so this random function gives me i think it gives me a number between zero and i'm not sure what the upper bound of the numbers are not that it's really matters but we can check okay it's between 0 and random maximum which is 32 000 uh 767 so every time i want to have like a random number between 0 and 20 i can just use this modulo to

select those but i also want a minimum of a thousand entities in the engine so i can have a bit more realistic amount of entities in there to test actually i just want to add like a thousand entities in the first go so if this entity is empty entity list like this i just add a thousand entities and for those entities i need a transform component so i need to fill in that init information but i just use its default value so this if you remember this is just the position zeros and rotation will be

zeros and scale is one and then i'll have a game entity entity info which i'll fill in with this transform info of course i need to give it a name so now we have an entity information that we can use to call the create entity method now we are adding an entity so the the added will be incremented and one of my conventions in the classes is that i add an underscore to any member variable that we have so let's do that now as well and now i can call create entity method this is it

in this method i create anything between 0 and 20 entities except when the the number of entities that i already have is less than a thousand and i just add the first thousand entities in one go and then after that i just add this number of entities here and then i check if the the entity that was returned is valid and added to the array of entities that i have for remove random it's the other way around basically the first thing i need to do is to determine where i need to remove an entity right

because i have a random number of entities that i want to remove but i don't know yet where i want to remove entities so here we can see that i have at least 1000 entities there might be more entities and so i'm using the total size of the area of entities to determine a random number in that range between the index of zero which is the first element first item in these entities array and the last one one of those will be removed and the way i determine that is to just use the random function

again for this number of entities and it will give me a number between zero and the number of entities so here i determined where i want to remove an entity and then i have a reference to that entity in the array and i can check or assert whether this is valid in two different ways so i can check more functions that i wrote and if that's the case i can try and remove the entity from the array of entities and also from the engine and one more thing i could do i guess is just to

assert if this entity is not alive anymore and here it should be alive and finally i need a method to output the results to the output window okay i think this is the entire test for now and uh let me see maybe i should also explain this so i create and remove entities in batches of 10 000 and that's because i also want to have an error whenever our generation in this id wraps around so if you remember we had this assertion that this generation should be less than the maximum value for the generation that

fits in this number of bits so if that happens then that could be a potential bug in uh in the game if that happens so here in the tests we can see when that happens so now i can run this program it's for some reason taking a long time to run so let's see what's going on i'll just step through the code and see what happens okay so if we are correct we just added thousand entities and that didn't happen let's try again of course i made a mistake here it should be bigger than zero

so we can see right now that this works it's saying that we should remove an entity at position 806 and we have 1000 entities and this one is also number 806 so that looks good so i can here add a watch for transforms at index 806 and then we can see probably that yeah this is an invalid value for an id so that means that this entity is dead that's good and then the next thing is to go here oh i forgot to print the results of course that's why we don't see anything happening i'm

not sure why it says that no entity was deleted that is because we don't set this value so here when we remove an entity i also need to increment removed entities in the first round it re it added 96 000 and it removed 94 000 so we have now about 2 000 entities left now every time i hit enter this number goes up because the total number of entities that we added and removed just increases and at some point this generation thing is going to wrap around and then we'll get an assertion error like this

so here it says that um well there's a debug error so if i press retry then we see that it stopped here and generation is indeed 255 that means that after almost a million additions and removals this generation has wrapped around so it's the decision of the game programmer if or the game engine programmer if this is an acceptable risk to take because if you have like around thousand entities and you added and removed two million in total and you have somewhere an old id hanging around and you ask for that entity then you would

get the wrong entity so the chances of that happening is of course minimal but still it can happen so one thing you can do is to set this number of bits a bit higher so this happens later and if you think this is totally acceptable you could also set this lower so while we are stopped here we can go in the call stack and check what our other values are so one thing i wanted to see is if that the number of added and removed adds up together to the number of entities that we have

now so here if i bring up the calculator i can have 993153 minus the number that was removed 998230 and we are left with 4923 entities and that's not quite this because we were in the process of adding the last patch of entities and we stopped before updating before updating the number of entities so that's why it isn't quite the same number if we want to know for sure then i can check this and that's 12. so 23 minus let me see how yeah well i can't really know how much it was initially because we

increment we decrement this every time so um well if you really want to do to know you have to add another variable here to just remember what the initial value of count was and then you can do this bookkeeping if it really is correct but i trust that it is for now and i can even go further and create game entity and see if these are okay so there is like 6671 entities in here which is not the 4 000 that we saw earlier and that's because we have a lot of free ids here so

in this array of transform there are a lot of holes so if we now subtract these numbers from each other like this is 66 75 minus 1748 then we get that same number that we have here yeah when removing subtracting removed and added then we also got this number so that's correct and well now our test is kind of interrupted because of this assertion fail so one thing i can try is to indeed increase the number of generation bits and see if that increases the number of times that we can add and remove entities so

i made it 10 bits instead of 8 bits and that means that we now have 1024 maximum value for generations so as you can see we are already in one and a half million entities and it's going on pretty well and i predict because we have we can now put four times the number of generations in there as we could before that this will also terminate at around four million five million maybe okay it went on for a longer time that i thought it would but yeah well at 8 million additions and removals it will

wrap around so that's the difference but it also depends on your random distribution and those kind of things if you add and remove the same entity basically all the time then of course it will wrap around quicker so yeah that was the test i think we could do a lot of more tests here but for now i think this is quite representative of the functionality of the code that we wrote earlier and also now we have a base for other kinds of tests that we might want to write later on and yeah i think i'm

happy with this and i want to continue with the dll for exposing the functions from the engine library to the editor so this time i'm going to add a dll project a windows dynamic link library would be nice here it is so here we see that it created a default entry function for the dll and well i don't really need this for the intended purpose of this dll and now the only thing i want to do is actually to also check for memory leaks here and that i can do whenever this dll process is attached

to any other process so the only thing i have to do is to copy paste these three lines from our test to the dll main and we are done i also need to get rid of this pre-compiled header i'll rename this dll main it actually created more files than we care to use so i'm going to delete all these files and delete these filters as well now in the properties i also need to tell the project that it shouldn't use pre-compiled headers and here also i'm going to link to the engine library so i need

to set an additional library directory and now also because i removed all those files it doesn't know that it should include these from the windows header so i'm going to include the windows header again and this project is used exclusively with the editor which is also a windows application so it doesn't really matter that we now include the windows header in a global place so this also gets rid of some less frequently used windows headers so the compilation would take little less time and it also doesn't recognize this crt function anymore so i'm going to

have to include the crt debug header and that takes care of it let's see if it compiles all right okay now i am ready to expose the engine api to the editor so therefore i'm going to add a new cpp file so here i want to export any function that i want to use in the editor the way to do that is to define a macro what this does is whenever we put this in front of a function for example something like this it will export this function from the dll and that means that it

uses this name literally that is what this extern c means because c plus plus compiler normally does some addition to any function's name and that's i believe it's called name jumbling or something like that if i remember correctly name name mangling all right so yeah the compiler will add these kind of at and then numbers and everything and it's according to some scheme or other but the thing is that if we want to refer to these functions from outside the dll we of course don't know the mangled name of the function so we need however

it was declared here in this cpp file and that's what this extend c is for and this part is self-explanatory uh means that well this function is exported to the outside world to anyone who imports or loads this dll will be able to use this function now here again um because we are using this library this engine library i'll have to tell it that it should use that one and again we need an additional include folder as well so now here i can include the common header and entity and transform headers again we are using

namespace primal so i don't have to type that in every time and then i just want to expose these two functions that we can use to create and remove entities here i want to have a function that calls this and one that calls the remove game entity so this one returns an id okay again i need to use the right language version now i'm returning an id type which is just a u32 basically instead of a complete entity because in the editor if we remember how we constructed the entities in a game entity we have

our own class for representing an entity in the editor so what i basically want to do is just to have an integer that remembers what the id is of the entity that is in the engine so what i want to do is just doing something like and then have another property that activates and deactivates the game entity and that means that we need to add an entity to the engine or remove it from the engine i'll actually rename this to entity id and the default value for it is id basically -1 but i still have

to add a little utility here to handle those ids in the editor as well it's not so much work so i can do it right now here in utilities i can add another static class so here is a constant value for an invalid id which is -1 and a method that can check if the id is invalid or not if we want really to do it properly then we also have to have something that is as big as our id type because right now our id type is 4 bytes and an integer in c sharp

is also 4 bytes but if we would change this to 8 bytes so u64 for example are to 2 bytes then in case of u64 then it wouldn't fit here in anymore and then we will we would have a bug but for now i'm just going to use an int i don't think i'll change this anytime soon to anything other than u32 so but that's something that we should be aware of that can happen so going back to the game entity i can now here have this default value of invalid id and this one is

the is active i just used to add or remove this entity from the engine i'm actually not sure if we really need to do this but i'll just leave it there for now so the only thing that i want to do here differently is just to check if active is true that means that this entity should be loaded in the engine and otherwise it should be removed from the engine these are the functions that i'll be importing from the dll and then we can call them like this in the editor code but first before we

do this i have to finish this dll so going back here i can see that here where we were adding this to the entity uh this function will get some kind of description from this game entity it's basically what components it contains right so in this cpp file we have to have some kind of entity description or rather a descriptor is a pointer to entity description and again i would like to have an anonymous namespace to encapsulate all of this and down here i'll put this the definition of this game entity descriptor which again is

just a list of its components and the only type of components that we have now is just a transform component and again i have to define this as well as a structure so the reason that i'm doing this again like you might think okay you already did something like this before here in this uh transform component but you might also notice that rotation here has four components and in this dll it has three that's because the data coming in from the editor which is pointed to by this descriptor has another format that we have to

convert here before we can use it in the engine so that's why i need to redefine these kind of data structures and therefore i also need a method to convert this to a initialization info structure because remember we have here this and what we have to do is basically we receive this from the editor these three fields or three arrays of floating point values and we have to convert this to a format that is accepted by the engine which is this and luckily the only difference is that we have to convert this rotation we have

euler angles from the editor that is contained in this rotation with three components and we have to convert this to quaternion representation of a rotation so that i can do in a method and for this conversion i'm going to use the directx math library for the position and skill we can just copy literally what we receive to this info so this mem copy copies from this address to the array in the info structure and it's this number of bytes that it will copy from this one to this one and we can do the same for

scale now before we can copy the rotation we have to convert it to the quaternion representation here we have this function with this monstrous name from the directx math library that converts the euler angle representation of a rotation to is quaternion representation and it puts the result here this is by the way why i'm not using this camel case or whatever it's called naming conventions because at some point it's just difficult to read i mean if you look at to me this is much more readable than this right especially if you have functions that are

close to each other in their name like suppose you have this function and you have another function that is that is this and they are in some parts of code that's contains a lot of those kind of functions and it's really difficult to make out this one from the other one right well you have here if we use this notation then it's i think it's much easier to to differentiate between these two and these two so yeah i really don't like this and that's why i use in the c plus plus part of the code

i'll use the convention that i'm using uh here to write this code okay let's get back to our method and now the only thing i have to do is to put this back to an array of floating point values this is basically the same as what we have here is just four floating point values in an array and this function just writes the value of this quaternion from the cmd registers or simply memory to this stack memory or memory array on the stack and then we can use this to copy to the quaternion in the

information for transform although this is four values for floating point values like this it's not an array so the way i want to copy these four values to an array is just to give the address of the first variable in this struct and then from there it will copy four floating point values to the rotation array in the info and then we are done and then we can just return this info structure and then i can go ahead and try and implement this create game entity in order to be able to use this function we

need an entity info structure that i filled in here so basically i converted this information from game entity descriptor that we receive from the editor to an entity information structure and then use that to call create game entity and if that happens we can just get its id and return it back to the editor then i have to do one more which is for a removal of entities now i have to create a game entity from well not a game entity but one of these api entity classes that is actually just an id i have

to make one of these in order to be able to call the remove entity because it of course wants one of these entity class instances for that i'm going to write a method so this basically puts this id in an entity id and puts that in an entity that we can pass to this remove game entity and that's it that's all i have to do now for the dll of course we have to import those functions in the editor and i can do that now here i'll add a new folder for our dll imports i'll

call that a dll wrapper and here i'll add a class we need this name to import from this dll okay so i am importing the function that we wrote in this dll file in the dll create game entity and we have to define the equivalent of this game entity descriptor in our c-sharp code as well so here i'm going to have another namespace now i have here this game entity descriptor and i'll have to tell the.net interop services that anything that is in here should be laid out in memory sequentially so we can access it

in order in c plus plus the way we can do that is by having an attribute and the same as we did here in the c plus we have to define a transfer component structure in c sharp as well so again i'm going to use this structure layout sequential and then define a class that's the transform component and then i'll include an instance of this in game entity descriptor and all we have to do now is to define a public function that converts our entity as we defined here in the game entity that would convert

this actually that an instance of an entity to an entity descriptor so would convert a game entity to game entity descriptor so the reason that we just don't use a game entity directly is of course that first we don't need everything in the game entity for example uh this is enabled or a name or a parent scene well we don't have anything for that in the c plus plus code and we are not going to use that right and the second is that all these data is not laid out sequentially per se in the memory

so we can't really point to any of it like this component list and those kind of things we just can't access it like it is now from the engine dll so we have to convert it to something that we can work with in c plus and the way i'm going to do it is this here i have to fill in different kind of components that we will have in the game entity descriptor but i don't have anything in game entity to get any kind of components that we might have so i have to go here

and define functions that i could call to get a certain kind of component from this list i can use this function to get a component of a certain kind for example i can now go back here i can call get component and this would give me a component of this type and then i have to actually cast that to the transform so this component will be of type transform what i also can do is to write another generic function that would do this for me instead of me just casting it every time so now with

this method i can just write the same as i did but it's shorter and now i can get everything that's in this transform component and copy it over to this transform structure for use with a dll function so here i extracted anything that i need from the entity in order to be able to create a game entity in the engine so for now because we only have a transform component i get the information from the transform component and later on we'll have other kinds of components as well and then i'll use this function to get

those types of components and get information from them as well and fill that in in the descriptor and call the dll function to actually create a game entity for us in the engine next i have to do the same for remove game entity except that's actually a bit simpler here i just get the entity id that i remembered in the game entity and call the dll function with that id and it will remove the entity with that id from the engine the only thing i have to do now in order to use this is to

set this is active to true or to false whenever i add a game entity in a scene or remove it from a scene and that happens in the scene class if we go here in scene and here in add game entity and remove game entity i have to make sure that the game entity is set to active or inactive so i'm going to make a couple of changes in the add game entity and remove game entity because here in the undo redo action i'm inserting these entities back to the list of entities but i'm not

setting the is active or inactive and i can better handle that in these functions instead of modifying the list directly here so here in add entity i'll have an index because instead of this insert i just want to call add entity with an index so i'll have an optional index and then because i am adding an entity i'll set it to active and that only happens when the scene is active because only active scenes are loaded in the engine and the inactive scenes are not loaded in the engine and i'm not sure if i'm ever

going to allow an inactive scene to be modified like when a scene is inactive then we shouldn't be able to add or remove game entities from it so yeah i could just do this but to be more flexible i'm just using the is active property from the scene itself and then if this index is -1 that means that i just need to add an entity to the list is a new entity and otherwise it's probably from an undo redo action and then i'll use the insert function instead of the add function and in remove entity

i just deactivate the entity and then modify these undo redo actions and one more thing i should do is also whenever we load the project we should set the entities accordingly so if this is a scene that is active then we should set all these entities to active so they can load into engine that's everything that we need to do to be able to use this new dll so now i can set breakpoints here and our game entity in is active i can put breakpoints here so yeah we have an entity and um i'm not

sure if i have the native debug ability on here let me see debug yeah it's checked out let me turn that on first so enable native code debugging allows me to go from c sharp to the c plus plus code and the other way around so now i'll be able to go in this function and then see that everything is copied over correctly okay now we are in our c plus plus function and this pointer looks okay so we have all zeros here and then we just copy everything over and now if you look at

this transform info you can see that this rotation has a fourth component which is one and that's because a quaternion that represents a rotation is always normalized that means its length is one and well when all other three components are zero then the fourth component should be one so that makes sense and then we go here and well i know that this is going to be fine because we just tested it right we tested that everything addition and removal of entities that was okay so we can now go back new entity should have an id

of zero because that's the first entity that we added yeah that's zero going back now um this id is zero and it's valid and we are fine we have one more entity this one and then two three or and yeah we have five entities and that's number zero one two three and four now if i would remove one of them let's say this is zero one two so the entity with the index with id two if i remove that then we can check if that works so this id should be two that's correct and well

i don't have really to check this but let's do it anyway this seems to be all okay this index is two and i think we are fine now the thing is that um now we removed one entity and i could undo that action and it should add that entity back to the engine so it should reload basically let's see if that happens control z and yes we are trying to create a new entity in the engine so that works nice so the entity that we removed had the id 2 right but now we are going

to add a new entity and if we see what id that one will get its id is five so it's the same entity basically but it's put somewhere else in the array because we have this system of three ids and generations going on here so the same entity can have different ids when we remove it and put it back to the list again okay and it's back again and i can redo that action and then it will be removed that's good and here i can add one more and then undo that yeah it seems to

be working just fine so congratulations we started using our game engine for the first time it took me quite longer than i had anticipated but we are done for today so next time what i want to do is to do actually a beauty pass for our editor so it will be less ugly than it is now hopefully and um yeah i think i'll have one or two videos just doing that uh which will be blue episodes basically so functionality wise we won't have any changes for the coming two videos so feel free to skip those

the only thing that will change is the look of the editor thank you for watching and i hope you enjoyed i hope that you don't think that these videos are boring because i'm enjoying it a lot making these videos and i hope to continue working on these videos for a long time until we have a fully functional game engine it will probably take me a couple of years to reach that point but in the meanwhile i hope to get to know a lot of like-minded people that are interested in the same things and if you

have any tips or comments then you know where to put them namely in the comments of the videos and feel free to like and subscribe that really helps me to grow the channel so i appreciate it a lot if you would consider doing those things and of course i have my patreon page as well so if you want to have full access to the code of the engine then consider becoming my patron and get access to the github repository for the code and there are a lot of other goodies and exclusive rewards for my patrons

as well so feel free to visit the patreon page the link is in the description below and then i'll hope to see you next time [Music] oh [Music] [Music] you