AI and Data Science in Earth Observation - Introduction

[Music] hello everyone welcome to the Muk course ml for Earth my name is shaanu I am a chair Professor for data science in Earth observation at the tomb today is my great honor to introduce you the introductionary model of the whole course it's called Uh artificial intelligence and data science in Earth observation since the whole course is about Earth observation I will first show you a video to bring everybody to the same page so in Earth observation we are working with the satellites typically the low earth orbit satellite they orbiting the Earth and Ma making

the measurements of the Earth surface as you can imagine why satellites rotate um orbiting the Earth itself is also rotating in this manner we are able to get the measurements of the entire Earth um and also if the sensor would steer to certain je location it is also possible to get a very high resolution measurements of the Earth's surface um the GE information retrieved from Earth observation data has been used for various uh Downstream applications ranging from Global change research uh to Earth system and environmental Sciences Metrology for example is the typic application field of

Earth preservation and also uh um we are following the un's agenda on sustainable development goals EU has been serving as a very efficient way to monitor the progress of these developments and also down to very practical things if we would have very high resolution data like safety security Mobility resource management or even to very practical things like um uh ALB planning we are currently living in the Golden Era of Earth observation and simply because uh uh we are now basically better than ever have a a huge amount of prestigious infrastructure in the space and this

slide just gives you a overview about the iser observation where you can see we have the um classic Metrology satellites and on the left side you see the science explor missions which is more for future Mission Concepts but the real game changer is the middle nine which is the companis course program of the ISA where there is the a whole Fleet of sunlight of Sentinel and they are kind of a mapping machine providing um higher resolution uh data of of the Earth surface on up to weekly basis so um we inevitably arrived in the Golden

Era and uh as I mentioned on the one hand due to this uh uh Sentinel missions and also other National satellites they provide continuous reliable and quality controlled e data and more importantly these data are all free and open to everyone want to use this type of data and also very important aspect is uh the future missions has been already under discussion this means there will be a guaranteed a long-term access of the data and on the other line from the commercial sites uh at the moment is also a blooming time period or for new

space uh there are different companies like uh Planet Labs II capella space and so on uh take Planet laps as example they are uh launching hundreds of small uh cubic satellites and this has typically a size of 20 cm and they are basically orbiting the Earth uh with very high uh temper frequency and can offer even though not the high highest quality but basically a daily coverage of the earth so these are are kind of complimentary to the approaches which is operated by the space agencies so uh triggered by the availability of the big data

and in the community we also see that uh internet Giants and the startups they are entering the EO like uh opit Insight um uh Google Microsoft they also start to produce a global dual product like the building Footprints and um this is basically the big opportunities we have but then it also come along with the uh methodologic challenges because all the classic um analytic methods we have in our observation that would not be any more sufficient uh when we confronted with the big data and this means we need artificial intelligence in Earth observation and also

uh since we have very high quality demand so in EO every image of the pixel is rather consider considered to be a geodetic measurement so this also means uh we cannot simply uh borrow AI methods from computer vision and do the job instead we really need EO specific AI research and the tailored Innovative AI for EO methods so um this is just a uh more or less a motivation slide from a full system I introduced you the observational system on the other hand I also introduced you uh the users who profit from du information extracted

from the asso salite data basically Ai and the data science rather play in the bridge role for the information retriever to turn observed data to Research Services and applications to the downstream users if we talk about Ai and data science in Asos ation there are a few thematic Focus um on the one hand we need to look at explorative modelbased uh single processing methods towards a better retrieval of the information on the other hand uh we need look at data Fusion as you know for any uh Geographic application um today it's a uh more or

less the standard case that we would have various of O preservation data source available uh basically the key question is how to optimally extract information complimentarily from these various data source to serve individual applications and of course uh information mining is important um assuming uh given that uh we are basically acquiring more than 150 paby U terabytes of data every day so it's impossible to uh process all this data instead we need to figure out a way to efficiently mine out the relevant uh data from the Big Data archive which is around um 100 of

pedabytes and of course ma machine learning de learning is the important uh branch of the development where we focus on instead of model driven mod data driven type of information retrieval I will talk about this later and last but not least last uh if we are talking about global applications with sness this means we are easily reaching data with several pabes uh this means uh basically big data analytics and high performance Computing become a um um Lory part of the whole pipeline of data analytics yeah I have mentioned machine learning and uh um uh deep

learning and for this I have to mention of course the new networks as you guys probably already know that uh there was a a big wave of new networks uh in the late 80s um uh which was the power that uh if you would have a neuron networks with more than two hidden lays then basically it is possible to uh um um mimic any long linear process and but of course then after the wave of neur Networks it came the winter of AI uh which was mostly limited by the uh training um demand and the

uh data which is required uh to train such a uh model and this was also basically the game changer happened uh which we now have the um let's say the uh really high phase of deep neural network even though the Deep running is a very fancy ter but uh it's really basically the same as the classic new networks is only that instead of up to two hidden NS you have a lot of hidden NS uh which uh helps to even boost the uh learning capacity of the models but of course in order to profit from

such kind of architecture then it's very important that we have sufficient number of training data there is a sufficient uh um computing power for train such a model and then there is also breakthroughs in designing the training algorithms okay so deep learning has also already become a kind of established tool in remote sensing if you look at uh uh the literature you will see basically starting from around 10 years ago uh you could see the papers related to deep running remote sensing and uh basically uh there is a kind of exponential increase of the papers

which also reflects how active this field is and also U I think the trend is going on here today so you may Wonder deep learning in EO it's a rather Hot Topic or it's a hype so if we look at the development uh in the past 10 years basically we observe three phases when we talk about deep learning remote sensing uh in the first couple of years I would say it's rather a quick wi and quick papers uh people in the remote sensing Community um borrow the models from Ai and then try to use deep

learning for various applications then the theepic uh uh conclusion of a paper would be we can also do it with deep learning and we increase the accuracy from 86% to 89% so this was more or less the first three years and then there is a second phase um the remote sensing Community understands that e all is different from the internet images therefore uh it's more about uh design new tailored architecture for specific problem and also Trend model from scratch and now we are still in the phase three uh basically we are more focused on um

that uh we need to integrate the domain knowledge we had already for decades into the deep learning models uh focusing on opening the black box or turn the Black Box gray but also basically how we can re implant uh physics base and the domain expertise into the learning process such that it become more labor efficient such that it more consistent to the physical world and so on and in um our uh opinion basically in order to be uh successful if you want to apply uh deploring to EO uh it's crucial to understand what actually makes

deep learning special in Earth observation so firstly one thing is in computer vision a majority of the people are dealing with classification problem detection problem segmentation problem however this is just only small fractions of the E problems we have instead we focus on the retrieval of physical or uh biochemical variables so this means we would lead to focus on high accuracy more importantly the results should be uh um reproducible and also as I mentioned earlier are of these pixels are considered to be a geodetic measurements therefore we definitely need to have the quality measures of

the results and also we should not forget that uh instead of just having the data try to learn knowledge from the data in EO we already have de DEC uh expert knowledge which is available and also different uh from internet images we usually have very well controlled uh data acquisition which are radiometrically geometrically spectrometrically uh very well calibrated and also we even know the signature noise ratio of the data and this kind of quality difference must be taken into account another very important aspect is uh different from uh internet image which is uh RGB uh

kind of picture with various quality and our data is uh much much more diverse so we claim that in Earth observation we're actually dealing with the five dimensional data XY z uh is about any location on the Earth but also the T because we have the uh time series data available for any sensor and also Lambda this is then more about the uh different modality of the uh satellites ranging from Optical sensor super spectral hypers spectral to microwave or site data and in addition um um to this kind of salite data we also have unconventional

Geo data from for example CI science uh like a geotex Twitter message or other social media Source like um uh Fleer or any kind of uh image platforms if there would be a du location of available that could serve as kind kind of unconventional sensors uh for remote sensing and uh also different from the CV Community where there is a a huge label data sets like image net and for EO it's typically that we are lacking of sufficient uh training data so with this uh in mind uh we Advocate uh the current uh focus of

of AI foro should be uh basically improve or information retrieval from um us observation Sal data or unconventional Geo data with the AI and the data science methods but we should rather focus on problems which are uh having really high societal relevance so different from leling cats and dogs probably we should rather look at uh the problem of our time than like urbanization climate change or un's sustainable development goals