hi I'm Andrew swerski I'm the founder of a company called Bex and we specialize in devops for embedded Solutions one question I see repeatedly in engineering forums is why AI on fbj is still considered relevant after all the AI industry is dominated by chips that are specialized to perform AI whether that be a server side Nvidia GPU or a tiny Edge AI processor when I say aii in this context I mean popular machine learning algorithms such as neural networks and Transformers it's a high volume industry dominated by specialized chips the exact sort of place where
you wouldn't expect fpga yet companies like AMD and altia for the Intel are still placing big bets on AI on fpj's the question I want to answer is why to do that the first thing I need to explain is what is a modern fpj an fpj is a semiconductor chip where the hardware itself is programmable this programmable Hardware means that we can get better performance than if we made the same applic on the CPU what we call acceleration but not as great as if we manufactured devoted Hardware to an application in the old days this
meant that you had a whole field of logical blocks that could be programmed and designers could configure those blocks into any sort of digital circuit that you wanted without the need for building a new chip every single time but nowadays modern fpj are more sophisticated they have blocks of specialized Hardware that perform task more efficiently than before so in this diagram I've shown an example where our blocks are placed into columns with specific purposes in Orange we have input output blocks in blue we have Ram in white we have our logical blocks and in green
we have our arithmetic blocks when we program the hardware we just link these blocks in specific configurations and by hardening these blocks we get far better than performance than earlier simpler fpga but it can get even more complex than that modern day fpj can form a smaller part of a wider system known as a system or chip normally the fpj stuck in the middle and then surrounded by a network on ship as part of that system we see other common ships such as CPUs real time CPUs external memory configurations iOS and gpus just to name
a few examples now what we can do is actually be clever about this and put a devoted AI engine onto that ship and this leads me to the first reason why AI on fpj is still so exciting so the reason number one enter end acceleration if our s so has an AI engine not only can we accelerate that AI we can also accelerate other processes that are happening pre and post AI one example would be taking raw input from a camera feed that might be on a robot processing images on fpj architecture has a lot
of advantages such as guaranteed low latency and so we can process that data from camera feeds to provide for that AI engine the data can then be fed to the AI we can finally run that data from the that AI for control or decision making the advantage of this is that we're able to perform all our processing on a single chip which can help reduce costs as well as avoid performance penalties you take when you transfer between two chips such as increased latency in power consumption but at the same time this feels like a bit
like a workaround after all we've just shifted the workload to the devoted AI engine rather than have the fpj architecture perform the AI processing luckily the next two points might solve this problem and the first one of these is custom Network Hardware reason number two custom AI model Hardware so with any GPU or AI chip the AI engines on those devices are designed to be able to run any machine learning algorithm regardless of how badly that certain Network maps onto it so in a way every GPU and AI chip needs to be a general purpose
AI engine but when we use fpj we can take advantage of our unique programmability and create custom Hardware that is specific for AI models and Hardware not only that but we can optimize that model for specific objectives for for instance if you want your fpj to minimize latency you could optimize for that or you could optimize for power efficiency so we can even take this one step further and instead of just optimizing the hardware towards a specific Network we could also optimize that Network itself and this leads me on to my third and final Point
Cooperative Hardware software optimizations so most AI chips will have a software tool that can take models from Frameworks such as tensorflow or pyo and then optimize the software of those models to better fit the AI chip this could be techniques such as quantization or pruning but with fjs we can take this a level further what we can do is apply a technique called Automated machine learning or automl so this is where we Define the task that we want to achieve and then we just tail our aut ml tool to find the optimized neuron Network for
that task we can take this idea here and tell our Al ml tool to optimize that neural network specifically for a for latency or for power power consumption and then it can perform both software and hardware optimizations at the same time it can then iterate so might try one proposed Network get results of how it fits onto the hardware and then use that um as feedback to how to further optimize the network and using fpj is really provides a huge amount parameters that auto ml tools can work with and simply put this sort of technique
uh can't be performed on other chips such as gpus or other AI chips what I really think is quite exciting about these reasons is that they're all actually relatively new they're all relatively in their infancy many of these are still active research projects especially Cooperative Hardware software optimizations and building custom Hardware specific for specific AI models we're also seeing IND industry take advantage of that endtoend acceleration especially for vision applications so we're seeing these new wave of fpga chips that are coming out and so there's a lot of hope that com by by combining these
techniques with the latest generation Hardware we can actually generate results that best current AI chips and that's it for this video uh if you've had a great time listening please feel free to subscribe and if you're looking at using AI or fpj's or want to automate your own processes please feel free to to get in touch with us through the link in the description below