over the past few years we witnessed an incredible AI Revolution which has been driven by AI chips in fact the demand for computing power has never been higher meanwhile the scaling of classical computer chips has slowed so what's next while Graphine chips Probabilistic Computers and Quantum Computers are still in the making light based computers are already arrived in this episode I will break down a new light-based computer chip which is on its way to data center right now and I can't be more excited about this let me shed some light on it Photonic Computers have been in the making for decades it all started 60 years ago with the development of optical fiber for communication and over time we got excellent at sending information with light now if it works so well why not to use light for computing in fact researchers have been long working on building light-based computers by now you've likely heard this idea that light-based computers are faster than digital computers because light is traveling way faster than electrons well it's true and not true at the same time let's take any conventional chip NVIDIA GPU for example during computation there is an electron that travels through a copper wire and this wire acts as a conductor and this is how it always works in fact the problem here is not the speed of electron but the medium itself the wire one light travels at 300,000 km/ second in this case we are talking about mm/ second and again here it's not a problem because wire is a conductor so it's full of electrons so here we can reach speeds way faster than mm/ second now you see we can't simply say that photons are faster than electrons it's way more complicated than this in reality the real reason why digital computers are slower than light-based computers because in digital computers we need to switch from zero to one from one and zero and this switching requires us to charge and discharge a capacitor and this takes time and this is where the real slowdown is coming from I explained this concept much more in details in my previous episode on Reversible Computing a great episode make sure to subscribe to the channel right now and watch it right after this video so by now we understood that the real slowdown is coming from this switching from charging and discharging a capacitor which is slow so that's where the light-based chips save the day because nothing like this is happening in the photonic world in photonics we compute data without stopping it basically we are computing as a data is flying by and this computation on the fly happening in the range of femtoseconds which is one quadrillion of a second so it's very fast the main feature of light is not light as itself but the main feature of light is that you can realize an Analog Computer and this is the difference it's not so much the light part when it comes to the math it's more the analog nature of light that you can natively exploit that's also why we call it Native Computing and the main advantage here is that you can carry out complicated mathematical functions without digitalization and that's very interesting in fact if we want to perform a simple summation on a digital chip to add up two numbers we need roughly 200 transistors those tiny devices all the digital computer chips are built off so then when we want to do a square root of this number we need another 7,000 transistors and then when we want to do a Fourier transform on this we you need roughly 1 million transistors so you see the more complex function you want to implement on a digital chip the more devices the more transistors the more chip area it will take what's so interesting when we want to implement a Fourier transform with light we can do it on a single optical device so you get much higher computational density and you might be wondering how is this even possible you know people who are wearing glasses if you are wearing glasses you are wearing every day a Fourier transformator on your nose and it performs this function using no energy at all once you understand this you can use the same principle to implement such complex operations on a light-based chip using special photonic elements just think about it we can replace 1 million devices with just one optical device device and it's passive so it means light just passing through it allowing you to do complex math without spending any energy at all and the same applies for multiply operation where on a digital chip we need roughly 1,500 transistors on a photonic chip we can do it with just one device so we get much higher computational density that's the reason why the interest in light-based chips is growing at light speed in practice it took many decades since this concept of computing with light emerged till the time when we figured out how to actually use it for computing purposes one of the main challenges is that light is really hard to control it tends to spread out and scatter and it has taken industry really long time but Q. ANT has finally built a fully functional commercial light-based computer their new computer chip is called NPU (Native Processing Unit) and it's powered by light rather than electricity we are already shipping first service to high performance computer centers and we've decided on that the first processor generations are coming on the standard interface of of the CMOS world mainly PCI Express and what we actually deliver to the customer are fully equipped servers which are compatible with x86 structures so in the end you get a server module you plug in the ethernet cable you plug in the plug power plug and the system operates what's even more interesting their breakthrough technology relies on a special material they're using so called lithium niobate essentially they deposit a thin layer of lithium niobate on top of silicon dioxide which sits on top of silicon and this particular material is Q. ANT proprietary technology which is fundamental for the success of their computer chip in several ways first of all it's the only material which allowed them to build all the required optical components in the chip in one material and this is fundamental for avoiding losses losses of light because losses of light results in the drop of accuracy in computations so we want to avoid it at any costs what are the fundamental features of lithium niobate well the first thing is that the modulators so basically whenever you want to interact with the light we can realize modulators that can operate in the gigahertz regime so very fast we can realize these modulators that no light is lost in the modulators and the last thing is the switching so in the end at the technological granularity level what you're doing you're changing the refractive index of the material this can be done only using a voltage and I know this sounds super technical but it's elementary because when you only need to change a voltage there is no electricity on the photonic part of your processor meaning there is no heat there is no heat dissipation leing again to a very clean signal and we already talked about that clean signals are fundamental to reach for instance an 8-bit precision so this is why lithium niobate is not just another material it's basically the fundamental source of success for building a Photonic Analog Computer in fact the Q.
ANT chip is the first photonic chip which is able to achieve the accuracy of 8-bit precision now to be honest what striked me the most about Q. ANT is that they're having their own fab so they are manufacturing their own chips and basically they own the entire pipeline from design to technology then they manufacture the wafers dice them package them write software stack for them that's a lot of work this is very untypical situation for a startup especially owning manufacturing because this is very assets heavy a question is how this upstart start up is managing it all and the most important why why do they need this fab light chips the structure of the light chips are per physical definition so by the laws of physics are pretty large you can't realize a photonic circuit with a 50 nanometer width because then the light would not be guided so in that sense what we have is we have access to a CMOS foundry an old CMOS foundry from the 90s and we repurposed it with strategic investments of a few tools to serve for the production of our own photonic chips so in that sense yes it's not cheap but in comparison to what you need to invest in the CMOS world it's easy and and this is a big advantage to the future as well because think about there are a lot of outdated CMOS foundries in the world which could be repurposed to build high performing chips for the AI next generation AI Supercomputers but using mature technology from the 90s I mean this on its own is a production paradigm shift this is indeed a paradigm shift very interesting example of turning so to say obstacle into opportunity and seeing all the investments governments are making into the photonic technology and into the photonic fabs and also seeing all tech giants like NVIDIA TSMC AMD going all in this fab's might have bright future let me know your thoughts in the comments now before we discuss how this new light- based chip works what it's capable of and how it compares to the state of the art GPUs for example have you ever wondered how much of your personal private data is floating around online your name address even information about your family members unfortunately it all gets out there thanks to the data brokers that spread this information online and this exposes you to risks of data breaches and of course personal security you've probably heard about cases where databases containing information about millions of users are sold online and sadly this is happening more and more frequently that's where Incogni the sponsor of today's episode comes in Incogni helps you remove your personal data from databases used by data brokers I used Incogni to remove my personal data from those databases and it's surprisingly easy you create an account give them permission to act on your your behalf and they send data protection law compliant requests to these companies forcing them to remove your information from their databases and you can even track the progress of these removals day by day on your dashboard as someone who values privacy I highly recommend that you try out and Incogni and put an end to annoying spam emails and calls use my code INTECH at the link below to get 60% % off an annual plan thank you Incogni for sponsoring this episode back to the bright new world now it's time to discuss applications and how it compares to state of the art GPUs and here honestly I spent quite some time looking into specs trying to make apple to apple comparison but it's really challenging one thing is clear that this Analog Photonic approach offers way better efficiency just think about it there is no wires so no resistance no heat generation so these chips require much less power to operate especially at high frequencies and here we are talking roughly about 30 times better efficiency compared to the conventional digital chips now with respect to scalability I think the fundamental question is can we compete with a GPU cluster because this is in the end what our basically what our direct competitor in the present data center is and to give you a bit outline to the future so in two years from now we're going to have Native Processing Units so processors coming on a PCI Express card that have the same performance than a graphic card in two years on the AI relevant functions but on the same side we anticipate that these systems have a 30x smaller power consumption than a graphic card in the future now what does this mean if you look on a server today you can bring eight graphic cards into one server rack and then you're at the edge of what's being reasonable in terms of power consumption we can bring much more cards into the same space and by that increasing the computational density in the server and since we still have energy budget left we can bring much more servers into a server rack and by this increasing so this is the forecast of today and I might be wrong and it's even better tomorrow but the forecast says that if we equip one of those servers and we plug the same electricity in as they plugin today already we can exceed the computational density in the server rack by a factor of 10 what's even more interesting according to Q. ANT their chip is built for both inference and training of AI models and this is very interesting you know typically we distinguish between two different kind of workloads a more simple inference when we have already pre-trained model we apply new inputs to it and we ask to recognize an object an image for example to recognize a fox and on the hardware level this typically reflects into performing many multiply accumulate operations in parallel and we've decided on that we are fully concentrating on the AI Inference and the AI Training so the layout of our chips is always that an a chip can basically serve both purposes so we can run AI Inferences which in the end is fundamentally saying similar to a vector matrix multiplication and on the AI Training we are basically going a different route because we can in contrast to what training or how training is established using a CMOS equivalent GPU architecture but the chip layout is always the same and this is very interesting because in order to do training we need to constantly update the model weights we need to constantly adjust it to improve its ability to make better more accurate predictions and to do this in photonics might be really challenging as we discussed in the beginning of the video in photonics this concept of capacitance or storing in intermediate results does not exist so nothing like in The von Neumann Architecture where we have this local memory in photonics no storage available in fact it works entirely different here the longer we can make the light to propagate through the chip without stopping it the more we can benefit from the properties of light let's say you want to train a neural network first you encode your weight into the phase of light and as the light propagates through the chip you modify it along the way one by one and at the output you get the final value and only then you convert it back to digital and then save it to memory for that the Q.
