Okay hello everybody and welcome to today's event on the networking channel we will be having an event today called are we living the long awaited AI Revolution for networking with generative AI it will be led by uh JP pressur PhD vice president of engineering at Cisco Cisco fellow JP is somebody who is known in our community as somebody who is a a Visionary in the area of AI applied ass systems and Somebody who has made truly incredible impact on industry since joining Cisco in 1998 he has been working on a number of networking Technologies ring
from IP and NLS to the internet of things as a Chief Architect of Internet of Things security and wireless networks and more from 1992 to 1998 he worked for service providers in large multi-protocol environments uh he is an active member of the internet engineering task force with more than 35 iotf Rfc's um and he's a co-inventor of more than 500 patents in the area of MP PMS Security Internet of things and machine learning and analytics so before we start uh we just want to mention the networking channel is not just one talk it is a
series of talks that happens every two weeks and in two weeks we're going to have a talk on large language models for computer networking education can it be done what are the Challenges and uh we invite you to attend that uh so with that we're going to start the event today uh one of our goals is to keep things informal to make it easy for you all to join and guide the conversation towards your interest so we would like to definitely encourage you to interact with uh JP there is a button on the bottom of
your screen for Zoom Q&A please enter your questions and and thoughts there um you can also type in the chat um and uh if you would like You can enable close captions so you can see what is being said transcribed and text on your screen and I'll turn things over now to JP who will be taking things from here thank you all right thank thank you very much Matt um as always it's a nothing but a pleasure uh to to to do that session with you so what I'm going to do now is to share
the slides and hopefully you guys should be able to see the slides otherwise please shim Me again don't hesitate to interrupt me as Matt said we tried to brainstorm in order to make it uh interactive and so I just put together you know something like 20 slides for uh the next 30 minutes and then we're going to go with a Q&A so never hesitate to interrupt me at any time should you need some clarification and if you disagree what I say you're very welcome to say that as well okay so I'm going to go very
quickly through the Journey we've been On for the past 12 years um just to give you some ideas on how we've been uh applying machine learning to the networking industry uh that's going to give you some some ideas on how one can use these Technologies uh for better networks so the first thing we started with at Cisco in 2010 was the use of machine learning for the internet of things and the goal in this case was we need to use ml to optimize rooting at the edge of a network with a low power And ly
Network such as I a.21 15.4 and so we wanted to see if be used to do better optimized routin in highly constrainted environment we also started to look at artificial neural Nets to detect the deos attacks which are not Bo Force for the iot and then we moved to security and security is without a doubt uh very well known for using so many emotional Technologies including uh you know recently generative a we go back to Generative AI in in few minutes so in this case we D I'm sorry I the slide view isn't showing up
for me for some reason do you have way to maximize the uh the slide view oh yeah sure so let me try to stop um you know don't go in the let me do that okay so do you see the full screen no not yet huh that's funny before okay so let me try to stop sharing and share again all right so I'm sharing Again that's good yeah thank you okay thank you very much M so in this case you know the the idea was for uh zero day attacks to use no detection and models
running on premise at the edge of a network so that the model could start looking at you know what is a normal activity and when there is something abnormal you know it could start raising some anomalies so this is very much in line with unsupervised learning and then at Cisco We built up the cloud uh AI in 2016 uh I just want to mention a few applications you know just to illustrate how one can use machine learning for Wi-Fi as you know Wi-Fi is can be a fairly complex technology and so you've got a lot
of kpis you may want to Monitor and so the old fashion of doing that was basically to use some thresholds that you would try to adjust and of course if you are too conservative you miss many issues if you Are too stringent and you put very low values you start to have Force alarms so the idea was to say can I grab a lot of kpis related to Wi-Fi and then start training some algorithms such as a gradient booster trees or uh you know different kind of forest and like that uh in order to establish
when a kpi is going outside of a prediction and in this case start raising some anies okay so I'm going very very quickly but this is a an interesting use case that could Be applied to many many areas of networking we use also machine learning to detect spoofing attacks when you have a new device on the network that claims to be a camera you can train no networks orm for example to recognize the behavior of the camera and then when you know the device connects to the network we start looking at the behavior using IP
fix for example netf flow samples and then we get the probability that indeed the device is a camera and if it's not a Camera or very unlikely to be a camera it might be sping attack I'm going super fast but 2019 uh we looked at predictive internet as you know the internet has been mostly reactive for decades now so you basically have to detect an issue and then once you detect the issue you route around an alternate path which could be PR provision using some backup technology such as nprs tunnels or you know Optical we
have a plethora of Techniques so it could be based on restoration when you compute the alternate path on the fly but still it's too late right so you first n detect the issue and then you rise the traffic what we wanted to do instead was to have a model that could look at the behavior of the network and start to detect early signs of a upcoming issue in which case you could proactively reward the traffic before uh you know experiencing the issue and that you can improve the SL And turns out to work extremely well
with a very high percentage of issues that could be completely avoided using a machine learning AI uh so that was a very interesting one the last one is called cognitive networks and now the idea is to say how do I can I learn dynamically on what drives the user experience and what we've been doing for decades now is to have some kind of static values related to the latency the Jitter of the packet loads and we all Know that it's much more complex so with this kind of network what we can do is to CH
machine learning detect get some labels on the reply from users such as thumbs up thumbs down which may be subjective obviously then look at the picture across the stack and then try models to tell us what drives the user experience anyway so that was something like uh the 12 years in in five minutes if you are interested you can go on that website and for each of these use cases And many more you can find some videos some white papers and other um just a quick reminder um I was lucky enough to do another session
with uh Matt a few weeks ago on the impact of machine learning AI on networking and the internet of the last decade so you may want to have a look at that video it's on YouTube it's on the channel see Tech Tok also and so you're going to find a lot of content about that so now let's talk about what is coming which is in my Opinion you know probably one of the most exciting um Pace in term of pace of inov very very exciting journey ahead of us but it could be a bit scary
sometimes in term of complexity of the system and I try to go a little bit deeper so what have we seen if you look at mlei for the past decade it's not something new machine learning has been invented uh 50 years ago we started to have a bunch of you know noral networks and then we Start we saw a bunch of winters and we started to see air very big bump in term of adoption with a CNN that could be used for vision for example around 10 years ago and all of a sudden there was
this huge bump with gp4 so what I'd like to do with you today is basically to step back a little bit and tell you how I see the future of applying generative AI for the network so this is what we're going to try to do this is a complex landscape and your Head may be spilling sometimes when you try if you are trying to follow what's going on in this industry of gener TBI for a number of reasons so what I'm going to try to do now and uh this is a new content I never
talked about that um and I was trying to look at all of the areas where the industry research is going related to generative Ai and why it could be confusing sometimes first of all I'm sure you you you saw this race Towards larger models and so we started with a few hundred of millions of parameters and then billions of parameters and we know that gbd4 is probably exceeding trillions although we don't know the exact number but it's pretty much obvious that we're talking about trillions of parameters still far from the you know the human brain
but still this is very very impressive and at some point you know people Started to say the bigger the better and of course if you you have a given number of parameters you need to have a lot of training data using self supervisor learning for example and the rule of thumb that we C sometimes the chinchila is to have about 20 times more tokens so words if you will for llm than the number of parameters of of the model and they are really really good white papers do at that so if you start to think
About it we're talking about we know four models like Lama 70 billion d ma and you see in by the way this is what they do they use about two trillion uh tokens and so we thought it was fairly clear right you know the more the bigger the model the better of course you need to have a lot of tokens for for the training then there was another Trend according which it was most mostly what would drive the quality of the outcome was not the size of the the you know the Model and the number
of the size of a training set but it was the quality of a data being used and there is a very nice white paper U you know from boc and others talking about the quality and so instead of having a humongous amount of tokens what we want is really the high quality uh like a textbook and that's why it's known as TCH textbook are all you need in reference to attention is all you need of course from a Transformer and if You haven't heard of fi now we have a fi 2.0 these are way smaller
models that are trained with their high quality and there are multiple ways by the way of checking the quality of a training set such as using classifier I'm not going to go there but this is a different uh direction if you will where you say no it's not just the amount of data being used for training this is mostly driven by the quality and of course we've seen many PR Strategies you probably heard of Chain of Thought when what you said to the model is you provide some guidance and you want the model to think
a bit more and articulate the reasoning of that um You probably heard of in context learning with en chart learning so you give some examples sometimes you don't even give any example sometimes you give one two three four um people started to think that when you provide a very big uh context window it's better but then We saw that when the context window is too too broad too wide the model may be confused and lately you might have seen gni 1.5 and other models with like Cloe with hundreds of thousands and millions of tokens and
they seem to be um not lost when you have such a context as well and you have many ways for the agent in term of strategies like Tre or thought react graph of thoughts and thing like that okay and that's not it um we talked about knowledge Distillation so in this case what you do you may want for example when you do inss you may want to use a smaller model and in this case the idea is basically to use a very large model that would be used as a teach teacher of a smaller model
which is known as the student model in this case and you're basically trying to replicate the behavior of the teacher okay very interesting one right um we started to see a large multimodel models like you know gini is one of them When in this case instead of looking at the text we may want to have multimodels and so start looking at videos and different things like that okay that's not it people talked a little bit about interpretability and what we mean by that is can I expect to understand how such a complicated model uh you
know manage to provide a given output and so you may want to be able to interpret it so there are multiple areas such as mechanistic interpretability monos Semanticity I'm not going to go there but this is a very interesting area as well we don't know to be frank whether we'll be able to explain that and I'm not even sure that this is a desirable property you know we could argue forever about the human brain and I do believe that Evolution did not force us to have to explain how we come up with with a a
given outcome a given output and so that you know we may not be able to to always explain that but anyway you might have Heard of small language model I mentioned about five for example is one of them and so sometimes people think that instead of having a humongous big model we may want to have smaller models that are specialized and fine- tuned for given tasks and so is it better to for a given task to use a huge model or multiple specialized model it depends it's not so easy synthetic data generation very important as
well and you may want to use some data Augmentation technique so when you have a a set of training a training set you may want to tweak a little bit so that you can generate some synthetic data and we we know now that some models can be trained with a mix between real data and live data high quality and some synthetic data as well I'm personally super excited about new padding such as bio inspired and Mana is one of his companies who is looking at the Heritage from what we understand from the brain In term
of sparcity that could be very interesting we we've been talking about neuromorphic Hardware that try to be as close as possible to the way the brain works and there are multiple areas now like world based models where you're trying to have a model that understands if you will you know the Dynamics of the world I'm not going to go there because it could take a long time we have multiple training strategies you guys probably heard of Knowledge database and how we can encode if you will into high dimensional space like add to we're talking about
15 more than 1500 dimensions and uh how you can encode if you with an object as a vector in such a high dimensional emings as well there are multiple rag strategies and you probably know what rag is about this is really how you retrieve some information on the fly when you ask a question to a large Lang model there are many ways of retrieving the information And I can tell you this is driving you know the overall efficacy big time okay so I'm going a little bit faster meure of expert is also something interesting this
is a different ways of using uh this kind of models when you have a specialized models and if you will each model can be the data could be rooted through a given expert and so this way you can achieve way better performance mix is a very good example of okay so now your head must be Spinning a little bit what am am I trying to do I'm trying to show you that thanks to open Innovation and the number of areas and the excitement around genbi is not a single week of probably maybe days where we
don't see a new area of research with new outcomes but of course you know when you work in the industry and you want to build a system you need to look at that at the D you need to look at the Dynamics and you need to make choices as well okay so you Know we all remember this is a networking channel U and we love networks so we're going to go back to networking very very quickly what I wanted to show you is um you know I'll give you an example I'm not talking about a
Cisco product commercially available but I wanted to show you what is doable today if you want to use gener tvi in term of architecture and I'll show you a demo in a few minutes so in this case this is a Very simple schema so if you haven't worked on Genera you may say Wella it doesn't look like something simple it's not that complicated so let me try to deifer a little bit what's going on there so the demo that I'm going to show you and consider it as a proof of concept and not in any
way final product is basically a situation where you have a user and the user wants to ask a question to the network and the question could be very simple or it could be fair Complex and what you would like to do is that in contrast with existing approach where basically you have a a question you want to solve it what you do you use a pleora of tools networking tools you retrieve some information using some apis you try to gather the data you do your travel shooting for example and then you aggregate this data and
you come up with an answer that's not what we're trying to do in this case what we want to do instead is to ask the Question to an agent and we want the agent to uh perform the task by itself so this agent has got access to apis so you can imagine that you have multiple controllers controller for Wi-Fi controller for the one for the internet for 5G and whatever you can think of um you interact with a controller using some apis you've got a local memory U this local memory is where you store multiple
things if you will some information about the documentation uh It could be a recipe it could be memory of strategies that worked in the past you name it right so this is really local memory and then you ask your you relay your task to llms you know because it might be multiple LM so you may decide that one of the llm can be used to put a plan together of what you need you want to do some people may argue that llm cannot do any plan we can do you know these sort of things but
I'm not going to go in this debate now um And basically what you do you format the question POS possibly differently you provide a lot of relevant feedback to the llm and you basically say here is the original question I'm going to give you relevant information to solve a task and I want you now to tell me what to do with that so what is extremely important is upon receiv receiving the question you may want to retrieve on the Fly the relevant information and and we you have a massive amount of data this this uh
Task according which you you retrieve relevant information before going to the LM which is known as a rag is extremely crucial and that's not something easy to tune and then you've got some potential back and forth between multiple LMS you may have an agent that is trying to put together plan another one responsible for writing code then the code could be run against API so that you retrieve the right data then you do some math and processing and you may have an llm Specialized in code writing you may have another llm specialized in math another
one that is capable of plotting things and stuff like that and so what I'm going to show you is what I believe is going to have a massive impact on networking and the internet because now instead of having to go through their cumbersome uh you know set of taxs tasks if you will to do performance analysis trending travel shooting you may want to have agents that work together have Access to the data and do the work for you so I know that sounds almost too nice to be true but I absolutely believe that this is
what we are going to see pretty quickly when we talk about generative AI you know one of the most common use case is basically a sort of rag as a service in other words you can imagine that you have a humongous data lake with a lot of documentations and you may want the system to retrieve the most interesting Documentation close to the question so that's the concept of embeddings and distances so in this case imagine that you have a huge space with multiple Dimensions you have your question you project this question as a vector in
the embed and you look around you in this High dimensional space all the relevant documentation you do basically a shortest distance if you will and you retrieve what you believe are the interesting documentation the one that Are mostly IR relevant to the question and many people use use it like that without going through something very complex this is already very strong use case we want to go um further and one way is basically to say look the CLI which um has been used for many many years and we all know that for each equipment you
have different kind of commands you need to remember all of that you may want to say no I want to interact with a network in a different Fashion you may want to do performance performance monitoring ask complicated questions ask about correlation of metrics and all these things uh you just ask to the system instead of retrieving so many information well in this case that's super easy you ask the question to the agent the agent will have to retrieve the information pull up the data from a live network do the math and get back to you
and of course Holy Grail is obviously trouble shooting you ask a Complex question and you expect the network sorry you expect the agent the generative based agent or system to be able to solve the problem for you and say okay this is a this is the issue that you have I can tell you what is the root cause and after doing many many subtasks okay so I'm going to show you um you know one example and if you're like me you know I was blown away by this example and believe me it took quite some
time of tuning of a system That is very complex to achieve that goal so I'm just showing you a very simple example here when you've got a user called John John is connected to a rapper which is running asd1 in this case and then you've got multiple tunnels like internet and PLS tunnels you go through a firewall and then you have another user which is located on a different site and now the question might be list any sd1 tunnels at user John sites that have more than 50 Milliseconds of latency in the three hours why
would you want to ask such a question very simple if someone complains about you know issues with voice and thing like that um you know you may want to have such a question and just the back for a minute read the question and you see that what seems to be fair easy for you because you you can process this sentence and say okay I need to know who is John I need to find the site John is connected to on that Side I need to find the router oh guess what the information about the router
is reachable via a given controller then I need to look at all of the sd1 tunnels I need to retrieve information and do some triage over the past three hours filtering and all that kind of stuff and come up with an answer it's probably something that would take you dozens of menes to do and as you can imagine if you are an agent that understands language with natural language Processing you need to do quite a bit of you know thinking in order to solve that issue and I just recorded that video a few days ago
and you see that in this case I'm not going to go through all of the processes but if you have a question I'm showing you the different steps related to the slide that I showed you a few minutes ago where you see that the engine first of all found out who was John could detect could find out the mark address could find out also the IP Address and you see that the the large langage model opposed some code the code was was correct in this case it was executed and then you see the result like
a plot and immediately you you get that hours of savings if you will if you want to do that the next one that I wanted to show you is is Kara so V user reachable from EDS sd1 Gateway so sorry it's not John is head but bit of a title here so in this case it's a bit more tricky because you need to find out Again who is K where is K where is she connected and then you need to find out that reachability means that you probably need to use a different tool which could
be ping or it could be TW route if you guys are networking people you see what I mean you need to find out how to run it from a different location and so you need to find the router and so on so forth and that's exactly the same thing in this case you see that the system managed to find these two guys When you look at the IP address is this is a lab that's why you see a 10 Class A address you write the code you find out what is the right tool you you
basically run the the TR route you analyze the contain of the TR route it's very interesting in this case because you see that the code that was written at the beginning was incorrect and I didn't tell you all the logic but we have agents that work together one of them said your code is Not good and the second one said okay I'm going to fix it so you see the the interaction between multiple agents which is very exciting you know I I believe this is something that will be a game changer in the wrong run
as well so in this case second time the codes was correct and you see that we have found out you know uh look at that you see automatically number of hops that were found out you see the mean latency and you've got the final answer as well okay Um I don't know for you but I was super excited when I when I saw these sort of things it's not to say that the system can solve any type of question it's not to say that is always reliable and thing like that there are many many challenges
now we're going to talk about the challenges so first of all something very close to my heart I you know from time to time I'm trying to write short sometimes a bit longer white papers that You can find on the website this one was really about uh human intelligence and I really wanted to say that because I think this is very important you have a lot of people who basically say are LM intelligent first of all we need to go back to the definition of intelligence there are multiple dimensions and like that some people tend
to say does it match human intelligence and the answer is absolutely obvious of course it does not Yet at least match human intelligence even the one of a dog or a cat obviously does that mean that the system are not useful of course not and sometimes it tend we tend to see um people confused about the two together and say because it doesn't match human intelligence is not useful as someone working the industry I do believe that it could be extremely useful and still not be as smart as a human if you want to know
more about That I'll let you look at that uh this one is I believe very important is about relevant metrics for AI systems um I think this is a core of Genera so if you start working on gener tvi systems what people do today these days is really to say ah there is a new model like Mistral 7D or mial also or L 2 and these sort of things and we have wonderful metrics that have been designed to see the per performance of a model when Executing tasks in a given category could be translation like
the blow score you know we have this kind of score this you will solving math problem like human eval Etc so what people do all the time is to look at these tables and each time we have a new model which of course happens every few hours you look and you say ah this model is beating the other model for this metric and so it's a little bit better and you have a wave of people excited about that and the Reality is that when you build a generative AI system and I'm not talking about a
model I'm talking about a system where you have the local memory you have strategies for a r you're using potentially multiple models you have ways of tuning the models and all that kind of stuff what you want to do is to evaluate the efficacy of the system and that's a bad news there are interdependency between all these components and that's one of the Challenges with llm which is that sometimes it's very hard to interpret and when you change one component you have an impact on the overall efficacy so the point of this white paper is
to say if you want to build a generative AI system you need to define the metric for the problem at stake the problem you're trying to solve don't pay too much attention about all the kind of metrics that have been designed for different tasks you need to Have a look at the metric that is relevant for your problem okay um there are many challenges does that mean that um they are not usable for existing applications or of course not on the contrary you can absolutely use generative AI for a lot of things today but of
course there are challenges we need to be aware of like reliability that's probably the biggest challenge to me and what people called hallucination which is not really Hallucination because that's for vision but you know it's basically when the the marrow is making up things and there are many strategies to mitigate that but it's still an issue especially when you write it for industrial application you want to make sure that the reality is good enough we talked a little bit about interpretability I'm not sure this is going to be a big issue except that you may
want to have a way to check that there are no biases for regulation so There are areas where you want to do interpretability the performance metrics we talked about that power consumption is a big deal um I'm not going to talk too much about catastrophic forgetting but it is an issue and that's something that the brain is doing remarkably well which is when you learn a task a and you're trying to learn a task B guess what you don't forget the task a and that's something that machine learning models tend to do uh and so
this Catastrophic forgetting is really an issue okay we wanted to have enough time for questions and answer so I'm going to rush out to the conclusion now what can we conclude well well first of all if like me you love science you want to be very careful about conclusion because nobody has got a cable and I see many scientists these days you know making very very affirmative conclusions about the future we need to be a little bit careful when you especially when you Look at the pace of change so what I believe is obviously I'm
stating the obvious that's why I'm not doing any no risk there compute will continue to grow we continue to use AI accelerators like GPU although we start to see some very strong use of CPU especially for inference even with the billions of of parameters um we are going to see continuous algorithmic improvements um some people bet on the fact that Genera algorithm like the Transformer will be gone in a few years um I don't think we know for sure but there will be new comments right and that's good the financial models like GPT will continue
to evolve Lama will continue to evolve mix and many other ones we're going to start seeing more small language models I absolutely believe that we start to see model personalization which is kind of cool because the model will adapt to you which is kind of cool but there's no Hope for simpler architectures they are fairly complex there are also many promising areas collaborative agents personally this is my my own opinion we are going to see systems with multiple agents working together and the example that I gave you a few minutes ago is one of these
examples you see that you have one agent writing some code another agent trying to fix the code of the first one yes it is going to come we do see many kind of approaches about Reasoning and planning some people believe that LM may be able to do planning other people do believe that we need to have completely different approaches um it's it's difficult to say personally I I also strongly believe in reinforcement learning and meta learning but these are very promising areas continuous learning we know that the brain when you are learning something new you
don't do a back poop on the neural network all the time it would be Exhausted otherwise um and some some people believe that the synaptic strength is being adjusted when there is a surprise between what has been seen and uh what was predicted by the the brain and this is when we update the weight that's not at all how we training large language models for example okay U there are challenges none of ter seems to be unsurmountable but reliability we talked a little bit about reliability transparency may be required what I mean By transparency is
the ability for M model to say look I don't know I'm not going to make things up but I don't know the issue is is how do you know when you don't know and that's not an issue that is specific to to machines by the way power consumption is a video so there are many ways to improve power consumption during training using incremental fine tuning sft for example with Laura Laura plus using smaller models that tends to be super efficient As well we may want to be more bio inspired with Computing um new kind of
comput if you will a neuromorphic as well as I said before and there are question marks obviously so can we expend the scope of llms and start to do the kind of prediction like Time series I do believe that classic ml is still doing great job in many areas are we going to see more large models very general as opposed to more specialized um you know do we need New type of architecture for learning as well what is the role of transparency you know thing like that so I would probably say to finish that over
the next few months and years I don't see any reason why the uh you is going to slow down I think that these four blocks will be fine tuned but should be very much relevant For an upcoming future and I can cannot be more excited about what's going on because I I do see many many examples where classic ML and Generative AI will help us build a better network but now it's the time maybe to take some questions um and I'm sure Matthew you may want to make some comments as well yeah thank you very
much JP that was a really great talk um and to the audience if you'd like to ask questions you can do so via the Q&A feature at the bottom of your screen um we're getting some questions already I think uh we'll start with a question related to the types of AI that you've been discussing um there's lots of different types of AI out there uh there's a question of you know how do we apply them to networks which ones do we apply do you think you could talk about uh you know compare a bit these
different kinds of AI like generative AI you know there's kind of more classic machine learning uh you know how would you compare generative AI to Classic machine learning um in terms of what they could do or when you use Them or anything lines yeah yeah this is a very very good point that so first of all we have seen many many publication around gener because that's a you know the new thing and people tend to forget sometimes that the classic machine learning algorithm still do a great job so for example if you're looking at an
detection you know what I mentioned about Wi-Fi right and you know you want to see when a kpi is going complet is you start to see an issue in out lier or You see a new behavior on the network and you believe that there is an attack happening now for example there's a pleora of algorithm that do great job at the nomy detection and we start to see this kind of algorith use for Wi-Fi and and and you know Optical networks and all the areas of networking and I think they going to be there to
stay time series forecasting that the same thing we've we've seen many many algorithm that are very much Suited to these kind of issues some people start to look at and say oh maybe I can use gener tvi but still this is a very nice sweet spot each time we we try to look at natural language I think that the the thing to do is really to use generative and large language models multimodel um models as well and they are very much suited to that so it's more of an issue of cherry picking if you win
and say for this type of problem this is probably even more suitable Approach now you know what I think is very interesting Matt I don't know if you have a a different view is um although when you look at Genera AI we're using what at the end of the day we're using databases that are very well known we use uh Transformers which are you know know networks the Dynamics is fair different in my opinion and here is why first of all when you look at the time it takes to come to productization if you look
at ail I you Know usually what you do right you you need to find out to specify the use case the metric and then what what you do you gather some data you do some pinup and then you TR model you may want to use one or two models you optimize the models by looking at the the objective metric and then at some point when you're happy with the efficacy you just ship it so the buyer to entry is very high because you need to have a team with your data Scientist or more a few
machine learning people as well but it's a bit more determined when you look at generative AI is probably something for which you can build a proof of concept extremely quickly because you can rely on some you know apis that will you know that are very easy to use and you provide you know you can use I'm not going to mention too many tools here but you Lang chain for example very popular you just plug L Chain you know this kind of agent to a database like pine cone or comb whatever and then use an an
open API and you can use I don't know GPT whatever and then very quickly you have a pro conern the issue though is when you want to play with it that's okay but if you want to to have a system that will U you know Drive the factory you need to be very much aware of the capability of a system and because it's very opaque with a lot of components it's very complex so in Other words the barer TR Tre is well lower in order to do a PC but I do believe at the time
it will take to make it reliable enough it's likely to to take more time and that's why you need to have to be very clear about the efficacy Matrix whereas in the case of classification or anomaly detection for example you know what the metrics are they are very well known and simple in the case of gener tvi you need to tast and really find you know think about the The Matrix and then you need to fine tune the system and so so they are very different Beast I believe somehow um and in the blog I'm
trying to detail you know how they they differ from each other in term of how easy it is you know to have a very quickly proof of concept the number of components number of Dimensions the type of security issues so there are many many aspects to to have a look at so generi is making use of well-known Technical components but when you build a system I think this is a this is very different and both will buing together I believe yeah thank you that's that's a that's a wonderful answer um there's there's a few other
questions kind of related to uh maybe downsides of generative AI uh you know can can we ask you what do you feel is the Achilles heel of generative AI um are there kind of mistakes that people make when they apply these sort of Technologies do do You have any thoughts along those lines yeah yeah yeah it's a very good very good question I think that the biggest one to me at least is really about reliability and the reason is very simple to explain is that the objective of this kind of model is really to do
next token prediction and because they are highly probabilistic the the system may start to generate um you know answers that do not make any sense and you know we need to to be more careful About how we Define Mak sense but you can you can easily come up with a system that is providing an answer so there are multiple ways of improving reliability for example you may say first of all I don't want to I want to tune the system to be um as deterministic as possible when I'm asking a question I will do an
ensemble strategy and I'm going to ask the same question to multiple llms this way you do some comparison you can do some fact checking there are many Techniques to mitigate the issue of reliability but people tend to uh Overlook The Importance of Being able to assess scientifically what is reliability and if this is an assistant and you want to use it for we forecasting or I don't know something your vacation it's probably okay but otherwise I think this is is this is one of the biggest problem I believe is really the the Reliability and and
we know that power consumption we need to find an to to solve that issue in the long term because now it's like a you know freelance for everybody right everybody's using it but there is a big cost Associated to that so but I'm I'm very hopeful that we we start to see some solutions to both of this problem that's that's great yeah thank you um there's also a few audience questions related to Energy Efficiency um do you have any thoughts related to uh the Energy Efficiency of these models do they require uh large amounts of
energy to run um is a question another kind of related question is um Can these techniques be used to improve Energy Efficiency of networks uh there's also a question related to specifically wireless sensor networks where Energy may be constrained um do you think large language models for example or generative AI have a Place in wireless sensor Networks I don't know if you have any thoughts along any of those lines oh yeah yeah totally so on the power you know ific cany first of all I think people started by saying okay you know we'll see in
a second step but I want to see what such a system can do for me and then of course we start to see many many optimization techniques uh for example for inference we we know that people they they start to well they don't start They've been using CPUs in term of gpus um and so gpus and sometimes what we call AI accelerators they are very much needed when you want to train but what we start to see is first of all you you don't retrain for National models every other day uh what you you can
do is to fine tune so the power consumption is going to be well less first of all because you instead of retraining you just only fine tune and when you do inference you can start using a new kind Of chipset and we know so there are by the way many optimization technique even when you you train you find tune that are way consuming well less energy you can reduce the you know the the Rel I wouldn't say the reliability the Precision I was with quantization using forbit and you know these sort of things so there
are tons of techniques but I think there's still a lot of room for improvement you rais a point that is Very interesting mat which is can we use you know these kind of models to improve uh the way uh you know networks the the the amount of power networks are using absolutely there are already you know a lot of experiment on doing that when you want to downscale the network while uh trying to maintain some SLA so that's the issue and you need to find a way to scale up very quickly so for example on
Wi-Fi you may want to look at the whether there are people in the room and If there's nobody in this case you you want to shut down some some access point while keeping a a mechanism so that if someone comes in you want to scale up again a bit like the elastic uh compute that we have in in the cloud right when on the Fly you want to reactivate but yes we are many areas like that uh on the hardware side in term of power consumption for example at Cisco we have a silicon one silicon
one Is the latest and greatest chipset that is consuming way less energy than before but I think that this is complementary to a control plane that is going to be less greedy and will downscale the network when resources are not uh needed and that could be coupled with forecasting algorithm so you know when you need to downscale when you need to upscale in addition to this kind of on the Fly um you know mechanism to bring up new Resources so yeah there are there are many many ways of optimizing the network thanks to LMS and
machine learning models absolutely okay thank you and then um so we have a a lot of students on the call who may be interested to think of uh research projects to work on and so we wanted to ask you uh what what do you see are the big unsolved problems in applying generative AI to networks do you kind of foresee any interesting Areas to work on or any key unsolved challenges do you see any kind of lwh hanging fruits that may be uh good to work on um do you have any thoughts I'm sure Matt
you have a lot of thoughts because you are also a very deep expert in this area and a very well-known professor in Academia you know there one thing that I would say I don't know M if you agree with me but um I wish I had I was 30 years uh younger starting my PhD Vis this because you Know years ago right you had to scratch your head to find a very good topic but these days I mean there are so many so many areas where you guys can apply where you can start brainstorming about
you know new topics so I'm going to list a few right totally randomly one is do we need to be more bio inspired um look at the brain 85 something like that billions of neurons gells it's about the same number except for Einstein he had more but otherwise It's about that number and it's about 25 watt step back for a minute and still you can do absolutely amazing things that we can't even do today with digital computers so try to look in this area second one is the size of learning I'm I'm super excited about
that what what do I mean by the science of learning for example if you look at reinforcement learning very powerful um but sometimes you may have to show a lot of examples of issues before the system can learn we Know that rain is capable of having some kind of generic algorithm to learn and I think that the meta learning is called sometimes very very important um do we need to have systems that learn from mistake I'm a strong believer is making in making mistakes because that's the only way to learn very quickly do we need
to have this kind of mechanism as well for digital systems what about neuromorphic do you want to work on New Generation of chipset that will Communicate between the neurons using some digital um analog Computing is also a very interesting area I believe multi-agent system do we need at some point to have collaborative um you know communication between for processes that together we try to improve a global system I could go on and on I'm sure Matt you can easily come up with a dozen of examples yourself no what do you think yeah I think so
it just seems like Such an interesting space to work on and you know trying to figure out while these techniques work um yeah hey can we ask you another another question um and maybe kind of the flip side of that of that last question related to uh do you have any thoughts on what uh professors or academics should be teaching students you you know your your company hires so many uh students and gives them such amazing careers and um but are are universities doing the right thing you Know are we still teaching students the right
concepts are there uh classes you would add to our curriculum um what sort of skills would you like to see networking and and system students learn uh to be competitive in a generative AI world do you do you have any thoughts on those yeah thank you for for this question Matt because this is a really question is something very close to my heart because I think that we need to be to do a better job at you know improving The collaboration between the the world of Academia and Industry and we need you I mean this
is a given we absolutely need Academia and look at the number of findings and research papers that have been produced by Academia so I think this is super important on what what could be interesting to teach is um I would say that it must be difficult for you by the way because it takes months to come up with a good program and I remember having seen a few years ago you Know some master in data science and now some people start to wonder okay but is it important to teach data science because we have now
system that can do a lot of task automatically so what would you teach now um in order for Master student for example to be ready in two years when the pace of innovation is unmatched right so we see that the world is going so so quickly so in my opinion what I believe is it's important not to be too Specialized because it's going to be outdated very quickly and so the fundamental of Statistics math are extremely important the importance of data I haven't seen so many courses my Apologies by the way if we have professors
on the call and say what of course we are teaching the importance of data um but I think this is not something that is usually something that um students when they are done and they join Cisco they discover and they like Oh my gosh I have no idea that before jumping to a machine learning model I would have to work for hours and hours to clean up noise in the data and that's why statistical um you know courses are so important important because you you need to understand that the noise that you inject because of
poor quality data is going to impact completely the system so that's what I I would say is very important is focus on the fundamentals Because that's not going to be outdated anytime soon try to keep up with with with all of the papers the problem and by the way you know what Matt we have the same issue because when you look at the development cycle we used to have the very is not slow cycle but uh now you never know when you need to stop and say okay I'm going to ignore what's going to come
up in a few weeks but I need to freeze my development and try to make it a solid product you you you are Very tempted to say I'm going need to wait for a few weeks because there will be a new algorithm or something new you have exactly the same problem in Academia I don't know if you agree with these statements yeah I think so it's when you design a class it's it's it can be challenging to kind of keep it up to date as things change and it just seems like that challenge is getting
you know more and more challenging as time goes On so yeah I totally feel that um yeah and I think uh we may actually be out of time um thank you so much JP that was such an amazing uh talk and your your answers to the questions were so uh helpful I I I learned a lot um uh thank you thank you so much for putting on such an amazing event thank you to the audience for asking such interesting questions that really helped as well um we're going to have a recording of the session up
on YouTube it's going to be Linked off of the main page uh we want the conversation to continue if you have uh further questions for JP you can join our slack Channel um which was just posted in the chat uh JP would love to continue interacting with you and um answering questions if you have further uh questions and uh please join us for our next event which will also be related to uh generative Ai and llms in particular how we can use llms for uh Network education so yeah thank you than You very much and
thank you thank you thank you very much Matt and you know I would say that if you guys have any recommendations topics you would like to see it's very simple you first need to convince Matt and if Matt which is you know if he reach it out to me I will say yes anyway this is my tic um so hope to see you soon Matt thank you so much as always and thanks to all of you bye yeah yeah thank you JP yeah bye by