Uh so hello everybody and welcome to today's event on the networking channel we will be having an event today called digital Twins and networking Next Generation testing and emulation it's going to be organized as a panel and it's going to be led by an incredible expert on the topic Marie Jose monpe she serves as University lecturer at McGill University where she leads Wireless application research in intelligent Networking on iot applications she is highly decorated uh member of our community and her Awards include the MIT technology review tr10 and Motorola technology prize she is joined by
pidm joari principal research scientist at The Institute for wireless internet of things at nor Eastern University he received his PhD from the University of Buffalo and has served as the CTO of an iot technology startup as well as having published extensively in The field of iot uh we're also joined by Ahmed ALB assistant professor at Arizona State University he is a recipient of the 2012 MCD Fellowship from the University of Texas at Austin the 2016 it signal processing Society Young author best paper award uh for his work on hybrid pre-coding and channel estimation in millimeter
wave communication systems as well as the NSF career award and we're also joined by iston David assistant professor on the Faculty of engineering at McMaster University he has previously served as the head of innovation for a multinational quality Engineering Corporation and has conducted extensive research on modeling uh model Transformations and vent processing so uh before we start um I wanted to tell you a little bit about our next uh talk in the series uh the networking channel is a series of talks happening every two weeks and in two Weeks we're going to have a talk
on large isps how do they work and what are the challenges they face and the engineering and CS skills they need uh this will comprise a panel uh including experts from both industry and Academia where we will discuss the internals of large isps their future and open research problems we hope you will join us and uh with that we're going to start the panel for today uh one of our goals is to keep things informal to make it Easy for all of you to join and ask questions and guide the conversations towards your interests and
uh we'd like you to interact uh invite you to interact with the panelists um all our events are interactive you can ask questions and use the zoom Q&A feature at the bottom of your screen uh you can also enable Clos captions if that's helpful for you and uh we also have translation in case you would like to see our words translated into your uh Preferred language and so now I'm going to turn things over to uh Marie Jose who's going to be leading the panel thank you um thank you very much actually you you're saving
me the um the the burden of having to introduce our distinguished panelists uh so thank you very much again Matt for your uh introduction and uh welcome everyone to this this panel on digital twinning um s before we started was mentioning that this is something we've been talking Together for quite a long time and uh it's it's interesting that uh for people like uh isan who comes from the modeling World um digital twins have been there for quite a while actually I've been hearing about digital twinning in um Aerospace and Automotive for about 10 years
now but in our world in our world of networking they're fairly new and they have a lot of promise and a lot of challenges and I think this is what I would like uh this panel to to address And uh I think Matt is also uh very uh right and saying that we would like to have interaction with uh the people on the call and um on the call on the webinar it's a zoom call but it's a webinar too and uh you can actually uh like also Matt said said you can ask questions on
the uh Zoom Q&A and also on the uh slack channel that is um associated with the networking channel so you're not here to to hear about me you heard about uh the the panelists so Um without so I see you guys in an order of isan um bam and akmed so maybe in terms of a short presentation we'll do it in that order so uh please is vanan uh again for people just join isan is a uh is a new assistant professor at Mech mechmaster and I had the pleasure of working with him when he
was a postdoc at the University of Montreal here and uh we worked on some iot and related digital twinning for a very complex system so isan please uh could you talk A little bit about what you think are digital Twins and your experience yeah thank you very much Mar Jose yeah indeed we we were uh we're building digital twins for strawberries yeah so imagine that yeah digital twins are not restricted to to simple physical systems anymore uh very often um use them to to Twin and represent uh biophysical systems nowadays in more complex systems so
basically when talk about a digital twin the idea is that You have a physical system right and uh you want to build a model of it for a very particular reason and the reason is um you want to collect data and instead of just ing next to the physical thing and observing the physical thing and measuring it and experimenting the physical thing you can now move to the digital side of things and now now you can have a cognitive um image a mental image of the physical twin through the abstractions of models yeah so a
model In our terms is just an abstraction of reality with a certain purpose you have a bunch of properties you're interested in here you're not interested in the color of the building maybe the shape of it the dimensions of it you abstract away from from physical reality that you can tackle the complexity of reasoning about a real physical phenomenon so for many people this is where a digital twin stops the definition of the twin stops here this is not a complete picture of Course because this is just a live model yeah so a digital twin
the most vanilla definition of it demands that you also have control in the other way around right so you collect data you do some reasoning on the digital end and then you excert control so this is this Loop is basically what uh what digital twin what a digital twin is what makes a digital twin so you have a digital twin on the digital side and we call the physical system a physical twin so you Have these twin artifacts you can you can experiment with you can reason about and um yeah so the two big ways
to have a look at this yeah so most most people some people come from the modeling world like I do and and we worked on live models so models at run time and that that these things have been there for decades and there are people coming from the mechanical domain or mechatronics who look at this as basically a smart uh closed loop control so was was the big Thing about this right so basically the the old ideas in a new form um of course the advanced digitalization and digital transformation that we've see we've been seeing
out there in Industry gave rise to to to uh to these systems and now building digital TW is actually a feasible uh feasible thing um when we talk about models yeah you can think of any model yeah so for some people it's something discrete like a like a queuing network or or an Automatan or you a graph for some people it's a bunch of differential equations for some people in very very extremely Niche domains it's a bone graph so anybody here working with bone graphs please let me know uh so very much uh it's a
very wide set of of possible representations you you can choose from and yes it is advised actually in the spirit of multi Paradigm modeling that you choose the most appropriate formalisms the most appropriate Languages to describe your abstractions of the physical world and the most appropriate abstraction and language formulism is of course um whatever suits your needs yeah um you typically equip your digital twin with a simulator or some sort of an Advan real time uh reasoning mechanism so that you can um take the human out of the loop and the reasoning is actually uh
carried out by the machine or computer itself and uh and the control can be uh sort of Timum and yeah digital twinning has been a transformational trend in many domains typically uh in in automotive smart cities but nowadays we see smart Agronomy and apparently also network engineering Network design so this is fantastic um there are different definitions just just showing you basic definitions so we talk about digal twin when we have this bidirectional uh link right we have data coming here and control going there some people like to Distinguish uh uh between digital Twins and
digital Shadows yeah when you have data coming to the digital site uh and the control is manual some people call this digal Shadow sometimes you you can you can put the human in the loop and you can build human actuated digital twins when you generate a bunch of instructions and the human with their intelligence can decide well I I sort of believe this is safe this is secure this is this is actually something I can do With my system I won't burn this this building following this instructions yeah uh and basically you have a digital
model when you don't have data coming in so indeed the key element when you build a digital twin is data coming at you at real time in large volumes and then you you you exert control and there are many different Frameworks to to to um to to um you know position your your your setup but something very interesting it's probably closer to to the wireless Um and and network emulation testing world is is this uh this emerging topic of digital uh quadruplets yeah some people say well you have a physical system you build a model
of it just to visualize it in 3D 2D whatever SS you need this is your digital twin and then you have a digal triplet models you can use to analyze your your physical twin but this is not always enough sometimes you want to build a quadruplet yeah a third representation of the reality Which is an actual physical model scale down and simplified model of physical reality so that gives you massive Advantage you know when you want to experiment with virtual and physical things so you can bring the hardware in the loop but you also have
advanced uh digital reasoning capabilities so this is just to show you the many definitions of digital Twins and yeah bedam andmed will show you how they make use of these uh these these um this well models of of A digital Clan yeah why is this important uh there's no going back to the old ways right so digitalization give rise and digital transformation give rise to these these techniques and this year mark the first when uh the nominal GDP d by digital transfer transer companies um is is high higher than you know digitally non transort companies
so we see this trend and if you read the Gartner reports you see digital Twins and deride Concepts Popping up on the hyp curve and uh yeah we see many of digital twin related topics reaching the plateau and becoming important assets for for companies out there so uh so we are quite there now um just a word about what are we doing with digital twins in in our lab um we are a systems engineering uh lab and what we are very much interested in is the end to endend life cycle of um of systems and
how you can put digital twins around these activities so we will talk about Amed and um and and ped will talk about mostly uh testing emulations of these phases but what you might be interested in is you know using a digital twn to collect data along the engineering life cycle and uh use the data in operation yeah so optimize it real time but also to retire your system and you know think about how you can reuse all the data and actual knowledge you accumulated along the life cycle so this is what we this is what
we um um investigate here in the Sustainable citizens and methods La at mcmas University um this is important about 60% of organizations believe digital twins will be critical in improving their sustainability efforts and these are Trends probably also um channeling into into the network uh domain so keep an eye on that perhaps I think this is basically it I hope this gave you right ideas about what digital twins are I think uh if you have questions uh just feel free to ask them In the chat and I pass back the token to Mari Jose thank
you so much yeah I think it was a great introduction for the next two speakers so I will uh now ask uh pedram to talk a little bit about what they've been doing in Coliseum uh I know actually I'm on your mailing list uh I know a lot about col because it's one of the power projects and I'm also collaborating with one of the power projects at Iowa State University called era where we do uh and we hope to be doing digital twins too uh in in the wireless world so pedram uh maybe you want
to start sharing your slides and take over from here of course uh thanks Mari and thanks ston for setting up the floor um let me start with sharing the slides and we can take it from there please please let me know if you can see the slides yes we do wonderful thanks um um Again as Marty mentioned um you know the work that we have been doing on digital twins uh is primarily around our wireless network test platforms um and um again focused on col um what I have today um in this presentation is mostly
you know a brief introduction about this tested platform and how uh we envision uh to um advocate for Digital Dreams using wireless network emulators with Hardware in the loop uh Coliseum uh just as a brief introduction is the World's largest wireless network emulator um as of right now as of today it has 25 racks of uh computers and radios it has 256 software defined radios um and it has a fabric of fpgas which we call them mcam or massive Channel emulation system uh which essentially takes care of emulating RF scenarios uh for you at the
baseband uh but connected to uh software defined radios for the communication part it also has uh Nvidia dgx uh gpus at this Stage we have two of them each with eight gpus uh the A1 100s that's the state of theg gpus with Nvidia and that supports AI native applications for um you know um a variety of different applications in Wireless that these days will see the Nexus of AR icial intelligence machine learning and wireless networks and iot so that part is a component that they've added in the past year to make sure and that users
do have um access to the stateof the art Gpus to do uh AI training U and inference um and of course everything is backed by a software defined Network infrastructure with Sonic that takes care of the networking what you see um down here is a is a high level overview of the structure of it has four quadrants with software radio notes so those those are dsrn each of the nodes do have a dedicated computer it's a Dell server connected to software defined radio by ni that's x310 with two radio Front ends that creates uh the
environment for users to go ahead and you know put in their radio protocol stack and those are software Define radios so we can essentially um install any type of radio on them uh these days we are primarily focusing on open and programmable radio access networks uh the the notion of Oran but we do have containers that are ready to use for all the users uh with Wi-Fi protocol stack with SRS ran with You know 5G 3ppp um compliance uh protocols um as well as you know some of the other Technologies Laura U just to name
a few um and then um we have a you know protocol tuning U framework uh that can take care of you know a a live and realtime connection to a GitHub repo which can be open source or it can be a private Repository of of the users and then what happens in in this colum which we call a you know digital tween essentially automatically periodically Will just fetch the new version of your code it will install it on the radios and you can use them on the Fly um that being said uh this is uh
just a you know a snapshot of some of the results that we have been able to achieve using uh and enabling it as a tool for digital twinning what you see on the left side is an outdoor environment this is a collaboration that we did with the US Department of Transportation uh for a v2x scenario as You can see on the top left uh there's the actual real real word scenario with the streets and whatnot and then on the bottom we have created a 3D model of that uh we have done rate tracing uh to
create the channel models and then that has been installed on csum and as you can see we can you know emulate mobility of the vehicles we can have all those um you know peculiarities of the Chan channels by rate tracing or instead of that we can always uh you know replace That by stochastical channels um and then users can essentially repeatedly uh run the experiments that they want they can on top of that they can you know U install their their protocol stack and U for example if they want to do some deep reinforcement learning
um uh to optimize um some of the components of the network they can do that on the digital train they can do the testing and then they can deploy it in the real world on the right side on the middle column uh There's an indoor environment that's called Arena that's one of the other tested platforms that we have at Northeastern we in an effort we have created a digital train version of that indoor environment also on calcium uh that lets users to um you know have more accessibility and they can repeat U the scenarios with
the exact same conditions as many times as they want and they can make some augmentations for do for just to do data collection uh that's a Necessity for f for AI training and of course the graphs that you see is essentially showing you a high fidelity uh twinning feature of of of of whatever happens in colum compared to what happens in the real world it's a perfect match um we have tools uh that can do Channel sounding on which is a varus network emulator um this is the same practice of Channel sounding in real world
but we do that on csum but with emulated radios uh and we have been able To demonstrate that the channel That You sound on this larger scale R Network emulator is a perfect match to what happens in the real world um same graph here just to um point out a few extra um you know um important notes um again consider calcium as a um as a tool um that is publicly available um to the researchers um and of course we have spent a lot of time uh back to what ston was mentioning in his presentation
we Have spent a lot of time to make sure that this we can um you know accurately U emulate the wireless environment or what we call it the virtual RF scenario so that's a key component as uh Isam was mentioning and ahed is going to talk about that as well with with simulation so that's what we think is the building block you have to have a fully representative version of your physical word in your virtual domain but but that the story does not End there um you know we are in the business of working on
wireless network systems and we would like to have uh you know additional components so uh on top of you know that that building block that physical layer we have developed uh you know full protocol stack um this this is just an example of ues on the left side and then basis stations on the right side and then on top of that of course you can have your applications with the Notions in the oran for example The D apps is a new ocean that we are we working on at the physical ler that that enables realtime
distributed applications with AI with Native AI nml uh so that you can you know receive um IQ samples play with them train AI do the inference uh and then emulate everything on that on that virtual domain um and of course on top of that um you know there's there's the service management orchestration that take takes care of near real time and non-real time Uh apps in the open program of what sever networks uh and we have a very comprehensive library of ran intelligent controllers or rcks uh that are interconnected uh with with this this entire
platform and of course that utilizes the gpus that I was explaining in in in the previous slide to enable the AI nml applications um real quick here I'm not going to spend too much time on this uh but just um you know this is an overview Of what we call a um open 5G or open 6G um reference structure uh what you see here we have a run in the box which is a complete radio access network with um you know fully programmable open soft Riz and virtualized um full protocol stack and then on the
left uh we have you know we have the open R gym uh that's essentially taking care of uh there's there's a full library of these protal Stacks that people can use them but of Course it also has libraries with with with Rick non-real time and real time [Music] and it has an orchestrator uh uh and management system that will take care of uh you know deploying um and utilizing some of these apps and showing how they perform in in an emulated Network that being said on the bottom um as uh Mari was mentioning um at
the beginning of this call um what we do on Coliseum is all based on uh virtualized Linux Containers and that's the beauty of the system so you can test that in your digital tween which is on csum but you can take that Linux container and then put it on real V tested platforms so it's pretty flexible we can move that to arena for example that's an indoor testbed environment with over the a Annas that's a real V tested experimental testbed platform and then after that you can also uh do uh you know some uh some
performance testing in In larger scale test platforms including the uh um the power platforms which is the platform for advanced Wireless research uh uh platforms that has at this point four different locations fully deployed in in in real word cities um and you can uh pretty much migrate you know this this containerized protocol stack of your wireless system uh from the digital twn to the real V and then we have demonstrated uh that the results um are a good match um and Um you know um you don't have to spend millions of dollars doing test
testing and development in the real world but you can do that in the digital Twain and then once it's ready you can deploy it in your real world um I think at this point I will stop right here to give um also Ahmad um a little time to go over his presentation thank you very much uh and I like the way you actually hit a really important point for for everybody on This on this panel and uh on online is this idea that you don't have to spend you know millions of dollars of infrastructure to
be able to test things and I think this is something that we're going to come back to and I think ammed also mentions some of that uh which is one of the big advantages of moving to the virtual world so ammed please all right so last but not least yeah so so you see the slides yes okay perfect let me just hide the Control here okay perfect yeah so hello everyone again and thanks Mar for the invitation thanks everyone for attending um so essentially what I would like to kind of briefly talk about is our
vision for um not just digital twins but but what we think could be real-time digital T assisted Communications and this this work has been done with jointly with my students and has been funded by NSF and number of Industry grants so uh and our actually motivation For for our vision for Real Time digital T start from starts from actually the physical layer so if you look at the current trends in communication system so two key trends are the use of large antenna array so going from myu to massive myu at subx or millimeter W mimore
cellfree mass of mimore intelligent services so all are different ways right of using large antenna arrays and in fact if we want to keep scaling the data rates and we need To keep scaling these myo systems and another key trend is the increasing dependency on higher frequency bands going from sub6 gz to millimeter waven in in 5G and potentially subter her in uh 6G or Beyond so if you look at these key trends in communication systems so they are mainly driven by the need for higher data rates but they come with many other challenges for
Mobility reliability latency and Energy Efficiency so if you Have these large antenna arrays for example they will have a large Channel Rees and in order to Lally leverage uh these these MOS systems we need to estimate the channel and the estimation of these large channel uh metries typically um needs a lot of training pilot training overhead and feedback overhead which makes it hard for these systems to support Mobility uh also as we go to uh higher frequency Bands then the links become very sensitive to Blockages and become more line of sight and very sensitive to
blockages so uh which again challenges the reliability of the Network and as well as also um the latency so if if the link is is blocked for example and disconnected then this user needs to hand off to another base station which may uh anchor some latency uh in addition to Energy Efficiency and the different components of the network as well as as the mobile user Side so if you look at all these challenges and then you can very Broly say that if we have way to acquire the channel knowledge between the different nodes in the
network in a kind of an a genie eded way right I mean without consuming a lot of wireless communication resources that this can address many of these challenges for example Mobility I mean if the base station and user um knows the channel then they can directly design beam Forming vectors they can even with high Mobility uh uh if that and also if if they know that channel not just at the current moment but maybe um in the next few hundred milliseconds then you can maybe the network can proactively predict that the link is going to
be blocked uh in the you know the next maybe 100 millisecond and can make some proactive action for example proactively moving this user to another base station or to U be switch to another beam so Basically we can do a lot of uh um actions and proactive actions to uh to enhance the reliability latency of the network so with that motivation so we we we thought of this idea of of basically realtime digital so if you imagine there is a vehicle in this city that's in in in a Down Town um then it if it
doesn't have any knowledge about basically I mean about the environment then the classical way which is what's currently deployed is that this user will uh if it Wants to connect to a base station then it will just initiate you know will look for maybe SSB beams or uh initial access process and will take a lot of time to connect and also will take a lot of time to find best Beam for data traffic and so on now imagine if this if this uh mobile vehicle has has access to May either on device or at the
edge or Cloud to a 3D map of its environment including the location of the of the base station and maybe some information also about The antenna panel and so on and using and by fusing some multimodal sensing information from whether the vehicle itself or maybe the infrastructure then this this this digital twin of the or or this 3D map now becomes real time 3D map so it has also access to knowledge about the the Dynamics in in the environment so this real-time 3D map of the environment then again that's available device Edge or Cloud depending
on the latency Requirements and complexity and so on uh then this I mean theoretically at least this uee can run uh real-time rate tracing to know not necessarily the channel Matrix exactly but at least some you know information about the channel Dimension so for example it will know that the the main Channel Subspace is probably you know going from this direction or maybe these are the best uh subset of beams that I need to search over instead of searching everywhere Else and this can dramatically reduce the the channel acquisition overhead and then again address many
of these challenge that I mentioned and even though this this may look futuristic but if we look at um the current trends in in in in in technology so if we uh you know with advances in precise 3D maps and High Fidelity sensing which is available now in in autonomous vehicles and so in arvr and other devices um then it it may not be Hard in the future to fuse this multimodal sensing information that are again acquired by different sens sensors distributed in the network and infrastructure and also mobile mobile devices or users in general
in the on into these 3D maps to have more or less real-time 3D maps of the environment and then also with the advances that we see in uh not just rate tracing but real-time rate tracing which is already um you know getting commercialized by Some by some companies so running realtime rate tracing on realtime 3D maps this this essentially you know gives us uh um directly a real-time digital twin of the physical Wireless communication environment right and this is this is important to clarify here that in in in in in what we at least are
interested and is we are looking at not a digital TW of just a netural looking at also a digital TW of the physical environment the 3D maps the buildings The Cs and so on um and now with the Advan also of machine deep learning it can enhance the different uh aspects of of like of these three components right the maps multimodal sensing the the fusion uh of all these sensing information the procing and so on and the rate tracing as as well could be AI enhanced all right now you can think of different uh you
can think of this in different ways like how to uh or different levels This digital TN so either that every in a very kind of local uh or very kind of let's say simple way so this device could just have a local uh digital T so that's only accessed by by that that it's only basically device that has access to it so for again on device Edge your cloud and it fuses this digital TW with its Loc sensing information and then also it uses this uh this digital twin to make its own decision and then
every device basically Will have digital twin or in a more kind of maybe interesting way you can think that all these devices will jointly fuse their sensing information will have access to a global digital twin maybe at the edge or cloud and they will fuse their sensing information to this digital twin to build like a more again uh Global and more uh um uh Rich digit rich or digital twin and then jointly or separately use this digital T either again to make just separate decisions or In the very kind of uh Advanced way they can
also do joint decisions for both sensing and communication for example if two if two devices are closed they don't need to do to sense the same environment again they can maybe coordinate the sensing decision as well as also they can coordinate the communication decisions for example to do some distributed optimization now this can be interesting for many applications over even though Like started by motivating this for physical layer uh because the channel knowledge can directly enhance Channel prediction uh Beam Beam assign beam management and so on but you can also think that this could have
a lot of interesting applications to higher layers and and in service service migration proactive caching and many other interesting applications in the higher layers as well now in order to do any research in these directions we need To do to have um a digit a research platform that that has both the the uh the the real word actually measurements and the synthetic version of of these measurements that's kind the digital TN and for that we with this motivation We Built This research platform that we call Deep verse 6G which will be available soon and it
is based on our previous work on and on deep mimu which is a completely synthetic data set for wireless communication and deep sense Which is a data set we published uh last year that has more than 1.5 million uh multimodal sensing and communication uh data points and we built deep verse scenarios to be synthetic versions or digital twin uh versions of deep sense so for example this is one scenario that we have now which we essentially on the left you see the uh the the real world measurements that are coming from our deep sense data
set which is publicly available now and On uh the second column we show basically the digital twin replica of this of this uh of the same scenario and again this digital twin replica is not just um by mimicking the visual representation of of the of the real world it has also also modeled we measured for example the the the beam patterns of the uh of the phased arrays that we have been using at the Bas station we incorporated this model in the digital so it's kind of it's a model Of the different aspects of the
environment of the environment as well as uh the uh the hardware devices we have all right we have also again uh digital uh basically model sensing information from lighter Rider position and uh other modalities so essentially with that we we think that digital TW could be interesting of course for testing and emulation but also could be interesting for real world communication operations so it can uh if if in the Real time if this if the if the communication network has access to this digital twin then it can make actions maybe not like exactly real time
but maybe near real time for example for the next a few hundred milliseconds it can enable netork to make again proactive actions for handoff for user Association scheduling um proactive uh B management U um and more so that I stop here and look for I look forward to our uh discussions yeah actually you again uh Touched a number of topics that I think are uh extremely important this idea of the twins of twins and I think K van will relate to that because the project we did had a twin of twins actually there was two
at least two parts to it so I think that is a very important uh aspect of of especially on complex communication systems that it's really almost impossible to do the twin of everything but you can actually do the twin of some parts and then uh be able To have them like you said communicating so since I met you were the last person to to talk you'll be the first person I'll ask a question uh you you know what you presented is is extremely uh uh exciting and and actually very very impressive uh but what were
the challenges uh that you ex you uh you had when you were developing this and pedram and and and um and Ean you also have probably some opinions about this but I'll start with atmed so was it so the I I know for example the access to the real-time data is often a big issue the modeling itself uh the latency because you need to compute things so what in in your uh experience what were the biggest challenges yeah thanks thanks Maria this is a great question so in in and to put it maybe also in
the context of what we think um so once we have these real-time digital Twins or or or again this data collected from the real word we can of course use These digital twins either to pre-train machine learning models for the actual system operation or to use them for uh for real-time inference right so now for the challenges I mean of course it becomes more and more uh interesting and also challenging when we talk when we think of these digital twins as real-time digital twins that can help the communication Network that's actually operating uh now right
not not just to kind of for for for preparing or Or or or designing the network um so in order for this to happen now of course we need to have like you know uh I mean moving building basically these digital twins in real time and fusing the sensing information in real time to update and maintain these twins this is challenging for many from many aspects right I mean we need to um you know figure out also how to to to to to to transfer all this data that maybe from different sensors and in a
way that is Eventually efficient for also the communication so if we are building these digital T to help communication then moving also all this sensing information we have a lot of uh data communication uh you know uh burden on the communication Network so so a lot of like uh you know compression of the sensing information uh um maybe distributed learning semantic communication semantic uh communication how to basically focus on extracting the Important information from all these sensors and then to reduce the data traffic uh for updating and maintaining these digital n this is one aspect
another challenge of course is also on the modeling of of these digital TNS so if we are going to use these digital twins to help the communication Network then this this this digital twin need to be calibrated over time to be making you know close predictions to what the network will actually see in terms of For example the wireless communication channel in reality or at least the channel sub spaces so this modeling is is is not triv so how would you model all the way from the 3D model of the environment the the geometry all
the way to uh the modeling of the uh uh of the electromagnetic you know properties of the different walls and and surfaces uh so the the so that we can do actually you know meaningful rate tracing and then all the way to to the rate tracing Itself which needs also to be calibrated based on the environment do you want to chime in on this sure of course I think um Ahmed is raising a really good points as far as you know um um the the size of the data that we need to communicate between the
Twins uh can be really critical uh you know to to design an efficient way that you know most probably you will you may not to transfer all that information but you know there are critical information That digital train needs to know so that it can help uh with for example if you're training an AI model you need to have some information from what is happening in the real world so that you can fine-tune your results and then communicate those control messages back to the real board so that's uh definitely one of the challenges but um
you know from from where we look at this problem um you know um as you very well know Mari um you know we are focusing on Real board um Wireless test platform with actual radios um and you know often than not when you're looking at a large scale um you know Network um think about AT&T Horizons of the world uh they have deployments uh in the real world with Bas stations and many and many of users and then you want to have a realistic you know digital twin version of that and now we're thinking about
having real radios in the loop and that by itself uh um you know can be quite chenging in Terms of being able to emulate larger scale systems which colum is actually doing 256 radios uh which is quite large compared to you other testal platforms uh and then you know there there's that bottleneck of uh you know being able to process because you know when you're out there every single uee they do have their own processors you know it's it's sort of a community resource that's that comes together with that you know larger scale um physical
word but putting that Together in a room with all the compute that you need all the radios that you need and making sure that in real time they can react to whatever happens in in in the in in the real world I think that's one of the critical challenges that these days we are trying to um you know attack uh making sure to make that realtime connection between the twins that are met um is spending I think a large portion of their time doing all these great work making sure that There's a meaningful connection between
the twins rather than just having an offline version of replaying a model that has happened already in the past in in in the real world without with with lack of the information of what is happening right now at the moment I think these are these are these are critical um um challenges to be to to be looked at and you know resolved um I will switch to the Q&A there's actually two questions are Related to the same a little bit of the same topic and I will push those questions to vanan since comes from University
of Montreal the PowerHouse in AIML in the world right now um so one question and I think both are related one says you know how can llms can be used in digital Twins and the other one is what are the principal models uh that use AIML for building digital twins I was going to answer but I will let isan do the answer because for me the AIML is Is tested on the twins but um it could be answer to so isan please yeah I mean I will start with the uh the llm part um these
are very two very distinct topics right llms and digital twist but there's a there's a very specific challenge um in digital twinning is is the question of autonomy uh so first of all you have to have the ability the twin has to have the ability to actuate the system yeah you have to instrument your physical system Accordingly but that's not enough um the human the human stakeholder also has to build a certain trust and give the uh the the um the ability and the Liberty more like so Liberty to act yeah so you can you
should at some point you should let the twin make decisions without you in the loop because you as a human you're the bottleneck in this so the way to to build trust uh in in in digital twins one way is to understand how the decisions are being made so you Might want to get the decisions and and you know Chain of Thought being explained to you so some people work on using llms to actually generate explanations of uh of of the you know Chain of Thought uh you know the digital twin um has when make
certain decisions so that's that's one way to to combine llm and but this is a very Niche Niche problem and there are there are more important challenges right now in indidual twinning as emphasized by Amed And pedram related to scalability maintainability and maybe verification certification about uh the second question what are the principal models for building visual twins so I can tell you about our experience um so if you build if you have a complex system a complex physical system and you want to build a complex you want to build a simulator of it that
is faithful enough to to simulate you know complex properties of the physical system the Simulator itself will be super complex so it is very challenging to build and Main contain a simulator you know manually so what we investigated was how about you build a rudimentary model of your aesthetic model of reality you call that like the first version of your digital twin and then you augment your digital twin with learning capabilities so how about you use reinforcement learning in your digital twin to observe how the physical system changes as you Actuated so in the digital
twn you have you're on a simulation you say okay this is what I expect to happen if I do this in the physical world so you you you check how your expectations align with the actual change in the physical system and as you get closer as the physical systems Behavior the trace gets closer to what you have in your simulations at some point it's it will get so close through gradual refinement of your simulator that you might say okay now I Believe this is It's a quite faithful simulation and quite faithful repres presentation of reality
so we use reinforcement learning for example to learn simulators of digital twins so this is One Direction of using AI for digal twins and then what my Jose already um mentioned in half of a sentence and that actually um pedram also mentioned uh the other direction how can you use digital twins for learning so pedron mentioned you can use Digital twins for reinforcement learning right so what you can do is instead of actuating the physical thing and just going to the physical thing which might be costly dangerous yeah not too secure not too safe um
you can take the physic the digital twin and if you believe that the digital twin is a faithful enough representation of the physical reality how about you generate a bunch of data using the digital twin you can generate bunch of data for deep learning or you Can generate data in an interactive way for reinforcement learning so this question of AI and digital twins it goes both directions it's I'm just very impressed whenever I see such Solutions out there and apparently colosium has these Solutions is fantastic um there's a question that's much more uh specific to
akmed it comes from Professor uh Muhammad Saleem at Iowa State uh he says um given that Liars are provide higher accuracy uh Compared to radar system do you still uh use radar in your multimodal sensing and that actually is an important question uh related to another question I had which was like what how do you get the data how do you make sure that you have the right data so um and that's yeah so I keep telling my students the answer to all the questions and we need more data but uh so ammed how would
you answer the liar versus radar question yeah this is this a very good Question and um yeah we have been working on this kind of multimodal sensing AED communication which is kind of a it's actually what also led us to digital twet end where we use different sensing modalities to directly help communication Network like beam management Channel prediction plug prediction things like that hand off and so on um and yeah of course we you know like there's always this kind of you Know which modality is more valuable I mean it's position maybe position information could
be enough or do we need to have more uh you know information about the environment maybe visual data all the way to visual data but uh which will give us a lot of information but or or lighter or later uh and I think the every modality has its own uh advantag and and and limitations so for example if we look at uh lighter give of course more Information about maybe the uh depth information in the environment uh Vision gives you more richer information about you know the moving objects and so on uh Rader gives us
also more information about dobler and micro dobler and so if we are interesting it's kind of you know a lot of kind of information again from doler processing and so on then rer interesting the other uh maybe Dimension that's also important when we look at these modalities is just From practical uh perspective the the the privacy concerns right so if you if you you know you cannot deploy visual visual cameras or cameras everywhere or ligher also everywhere so in some scenarios U maybe um the information that we get from Rader or even Wireless sensing could
be uh sufficient for the communication uh objective and also PR preser you know uh uh basically has less um uh concerns from privacy um perspective so it's it's a trade-off Between the technical you know the information that every modality can give us and also the uh other limitations from privacy and computational cost and and cost also the cost of the of the hardware equipment itself okay thank you well so continuing in some of the questions that we had prepared um you know one of the reasons that we organized this was this idea that you Know
the world is going software and however there is still something that is very Hardware oriented it's all system testing especially in wireless communication so um one of my uh question would be and I would start with pedram on this this time um what do you think of of using DTS for for system testing and and what I I read in some somebody else has said that you know that the future of system testing was email us your digital twin and keep your Equipment at home uh how do you see this uh is this going to
arrive in our in our lifetime or what are the major challenges to that and and it go back to what it goes back also to what uh isan mentioned uh you know your twin needs to become so good and you then it really gives into the the the the reinforcement learning and stuff but yes so the question is when can I start emailing my digital twin uh for for system Certification that's a great question Mari and you know this this is one of the very interesting um topics if you will that you know people start
to talk about okay so so what what's going to be next what's going to be future when is going to be this future that you're talking about there's going to be digital twins that are functional and working and fingers crossed will happen in our lifetime that's our goal so um I'm older Than you so maybe my lifetime won be as sry well I think we are not too far from but anyways um so but you know I I I want to connect this to your first part of the question talking about you know things are
going software versus Hardware of course there's dependencies to the hardware um as we speak right now we are hosting um this you know open a interface uh um and open source uh communities of orign Alliance today at nor Easter you know we have people from from large industry who are active L talking about this so it's what is the spiritualization you know removing the dependency to Hardware working on the software protocol stacks and then being able to deploy the uh you know th those software oriented solutions that people come up with on you know any
type of Hardware so it's going to be Hardware agnostic but there's going to be the software and of course digital TR comes To play because um you know there's this whole concept of interoperability between the different hardware and also integration of the software and hardware and the virtualization um our goal here with the wireless network test platforms that we have including is you know you know pushing the limits of digital twins towards a fully open programmable and a smart you know wireless network with Hardware in the loop which is agnostic To the to the specific
Hardware that we use in the digital train and that goes back to the last slide that I was showing that okay everything that you do here in the digital Twain if Mari goes ahead and sends me an email with her version of the digital Twain tells me hey ped this is the location on the W that I want to deploy a wireless network system whether it be outdoor and indoor now we have the tools to create that digital version of that physical Bo U we Are able to develop software with the hardware that we have
on top of that for example AI applications that Ison was was was talking about or using all those you know accurate sensor information that comes from the physical word that AED was talking about we combine everything we deploy the network we come up with that solution and then it will be containerized and we can move it to power Tes platforms we can move it to another vendor who has or operator who Have their own hardware and this just completely virtualized and in a container package that you can move around um in different locations so this
is this is our goal and you know there are so many pieces of this that you know um PE panelists in this call and so many other great researchers in the world are trying to solve little pieces of you know this this puzzle uh and the hope is you know everything will come together one day um and then we'll have a fully Realizable you know digital thing in real time thank you um so we have about five minutes so that's probably about one minute each um so um I wouldn't say what keeps you awake at
night because I'm sure all your research is keeping you um awake at night but um about what you're doing now what is the most exciting thing that's going to happen I would say in the next year so I'll start with isan because he's a new Professor thank you I would just go back to one of the remarks of Ahmed who mentioned uh Energy Efficiency being a big problem of course especially if you resource constr uh devices that's that's an obvious uh problem so what we what keeps me awake literally these days is uh we're working
on on on on on the foundations of understanding how uh digital twins can enable more sustainable systems but also how can we make our digital twins more sustainable Yeah so if you deliver a system that's very much sustainable but along the way you accumulate lots of sustainability death yeah it's you merely just mitigate iove the problem from the problem to the process so we're working on this bipartite notion of sustainability where you create digital twins in a sustainable fashion so that you can build sustainable systems and sustainability here means your twin can evolve as you
change the physical Environment you your T can be continuously reconfigured uh you can build it in economic uh economic way so it's it's it's fast and reliable the development process but also the Energy Efficiency of the twin itself because you might want to run it on on on a main frame right but sometimes at some point you will have to push digital twins to the edge so then it becomes a real problem so this is what keeps me awake and you Know finding these these these case studies in in your your Community is just fantastic
yeah so this is basic with my what gives me away ammed I hope you sleep better yeah well I mean there there are like several really interesting research directions related to digital twins that we are looking at in the group so all the way from understanding the the actual gains right of the digital twins if we in in this context that we are More interested in on the physical layer so if you you know will again you know in a certain problem be like 20% 30% and then does this justify all the cost or or
is it like you know uh 200 uh you know you know 200% like so so so we look at like theoretically understanding the the the actual gains of the digital twin in different problems so this is one I think important aspect and then uh another aspect is just understanding also the different applications and use Cases and potential of of digital TW and and and and be forming Channel compression other other venues um and then exactly I mean if you if you would build these digital Twi and this is actually one one point also that I
wanted to mention earlier is is I mean for this digital twin in the context of real time digital twin to be useful we need them to be calibrated so how can we design AI machine learning models that actually keep calibrating all these like Different aspects of the modeling of the digital TW to get closer and closer to the real world so these are all kind of very important I think problems to us to to to address and we have been supp and ped you get to close the whole thing you have about one minute first
thank you I mean um span and ahed brought all the great points um Energy Efficiency fun is is great uh one of the things that we are thinking about is right now you know being able to measure uh the the Energy Efficiency in the digital tween and make sure that in the real world also it it makes sense because you know those are really consuming lots of energy and that's our recent Nia you know funding that we are essentially working on that and back to Ahad applications of course you know this this is what we
think about every single day uh what keeps me awake is writing proposals just by the way I was up until three last night just to give you a sense but it's all about Digital twins so um you know and and and Mari I'm sure that you appreciate the fact this will this will be my closing sentence everything that we do is towards uh making open source and accessible environment for everybody all the researchers and the community within the power platforms and the Affiliated tested platforms such as colio Marina and whatnot making it open and accessible
to everybody and then you know promoting for having something Reliable that's High Fidelity version of the real world so that researchers can develop you know uh the great applications on top of it yeah I think I think that's and it goes back to the first I think at the beginning we're saying you know uh if we have people are very strong innovators and they would like to build these things and they would like to test them and they would like to use them they shouldn't have to again invest millions of dollars we Should be able
to offer them that digital twin that will allow them to become you know the next big thing and wireless uh we're out of time thank you so much to you three uh I think I hope the audience uh enjoyed this this presentation as much as I did uh we'll keep in touch of course uh thank you very much thank you to S to have allowed us to to do this and thank you to the audience thank you for the questions that we had and yes I think we're at the Edge in this world of digital
Twins and and wireless and I think it's just like a really really fun path to take thank you so very much everyone thanks for having us thank you