Use Cases

Automotive and Mobility

VEHICLE INTELLIGENCE AND V2X COMMUNICATION

Vehicle intelligence: While some advanced driver-assistance systems (ADAS), such as a lane keeping assistant or cruise control, are already commercially available. For several additional vehicle automation functions sufficiently efficient and reliable performance must still be developed and implemented before human drivers can be replaced by AI (in all operating domains). Transferring driving tasks successively from human to AI drivers and meeting all requirements with respect to sensing (scene understanding), decision-making and acting, presents a complex technological challenge with respect to both AI hardware and software models. Today it is clear that besides AI, the connectivity vehicle-to-vehicle (V2V) and between vehicles and infrastructure (V2I) will be key to deploying automated vehicles, since it provides the basis for the coordination of vehicles.

AI potential at the edge: Advances toward automated and ultimately autonomous mobility depend on progress in sensor and actuator technology, but most importantly on progress in AI technology. Each vehicle rep- resents an edge node within the mobility system, connected to the cloud for services such as traffic or fleet management or mapping. Transferring AI tasks to the edge offers multiple benefits including improved system performance due to reduced communication and thereby processing latency, enhanced privacy or new functions such as driver authentication. The combination of vehicle intelligence and intelligent infrastructure using, for example, Multi-access Edge Computing (MEC) can provide further significant safety improvements[10].

Challenges: The optimal distribution of intelligence between the edge nodes (cars), the fog computing layer (e.g. traffic lights at an intersection) and the cloud (e.g. traffic management centres) presents a key, and strong- ly debated, topic in the field of vehicle automation. The answer is likely to differ for different operational do- mains, as automated shuttles on dedicated lanes require far less coordination from a central intelligence than an automated vehicle moving through dense mixed traffic (including non-automated, partially automated and fully automated vehicles).

OCCUPANT ACTIVITY “UNDERSTANDING”

Automated driving: Automated driving is one of the four main automotive trends[11], driven by technical developments, market expectations and continual legislative tightening. The technical solutions are focusing on the human-centred component, which covers two challenges: the Human-Machine Interface (HMI) and human perception of automated driving. Advanced HMI is the essential interface for seamless operation between the (semi)automatic system and humans. The EU-funded HADRIAN project[12] is developing a holistic driving solution, focusing on the utility of dynamically adjusting (fluid) human-machine interfaces taking environmental and driver conditions into account. On the other hand, human perception of driving style and safety is crucial for the acceptance of new technology through the increase of trust. The EU-funded TEACHING project[13] explores AI techniques at the Edge to realise the human-centred vision leveraging the physiological, emotional and cognitive state of vehicle occupants for the adaptation and optimisation of the autonomous driving applications.

AI potential at the edge: Managing transitions between different levels of autonomy is fundamental. The AI-based observer is a key point of this system as it detects the behaviour and the mental state of the driver. Edge AI offers the local calculation of the driver states, thus allowing for control of the response time thus preventing personal data from leaving the vehicle and continuous learning to adapt the AI-based observer to each driver and passenger.

Challenges: Understanding vehicle occupants’ physiological state and their ability to take over control of the (semi)autonomous vehicle are crucial for driving safety. Those challenges are further complemented by the need for the most effective interfacing to the driver. Those calculations must be performed at the local level to avoid basic connectivity risks. The inherent conflict between safety and AI is an open challenge, which is also complemented by the need for continuous learning.

AIR PATH CONTROL AND DIAGNOSTICS

Emission control and overall efficiency of the engine: The air path of an internal combustion engine is a crucial component for emission control and overall efficiency of the engine. The goal of air path diagnostics is to detect faults or poor performance and to identify the root cause.

AI potential at the edge: AI helps in efficiently executing the control strategy and in diagnostics of the air- path. For example, the heavyweight processing (e.g. physics-based simulations) used in executing the control strategy can be substituted with ML workloads. Compared to the original simulation model, the execution of the trained model implementation is less demanding. The deployed model can thus be executed on the edge. As sensors are mounted on the vehicle, this requires a split of intelligence between the backend (e.g. crowd- sourcing of vehicle-data to obtain the diagnostic model) and the edge (e.g. the various privacy related aspects).

Challenges: Challenging requirements such as time-predictability, dependability, energy-efficiency, and security need to be fulfilled. In this respect, the aim of ECSEL Joint Undertaking FRACTAL[14] is to create a reliable computing platform node, implementing a so-called Cognitive Edge with industry standards. This computing platform node will be the building block of scalable decentralized Internet of Things (ranging from Smart Low-Energy Computing Systems to High- Performance Computing Edge Nodes).

BATTERY LIFECYCLE MANAGEMENT

Predictive maintenance for battery aging: The in-use phase of a vehicle (road profile, climatic condition, driving, parking, charging) has a significant impact on battery aging. Batteries pose the risk of exploding (“thermal runaway”) in normal use and the existing methods such as strain-, acoustic – and/or temperature sensors to detect thermal runaways.

AI potential at the edge: To better understand the aging behaviour of batteries, data-driven models based on aging experiments enable lifetime simulation and prediction. Predictive algorithms on the edge can crowd source data from vehicles and/or the lab. This provides critical correlations with battery safety and offers the potential of increasing the warning period. Within the ECSEL JU Integrated Development 4.0[15] a digital twin that allows the prediction of state of charge (SoC), state of health (SoH) and/or remaining lifetime is developed.

Challenges: There are a large variety of modelling approaches ranging from models using first principles (e.g. electro-chemical models) to purely data-driven models (needing to collect aging-related data in the lab and while operating the vehicle). These can be applied to the cell, module as well as package-level. Hybrid models have to be developed that aim to combine the models from first principles and data-driven models.

THERMAL MANAGEMENT OF THE POWERTRAIN

Energy control in the vehicle: The perception of the environment is carried out via vehicle and powertrain sensors coupled with the weather data and the traffic information that can be retrieved from dedicated service providers. The computing workload is split between processing in the backend (e.g. crowd-sourced data) and dedicated control units (energy control units) in the vehicle.

AI potential at the edge: AI has a tremendous potential in optimizing the energy efficiency based on the perceived environmental conditions. For example, by considering weather and traffic conditions to accelerate or delay the cooling process, AI-based strategies can augment classical model-based thermal control strategies. In line with such efforts, the Horizon Europe draft work programme 2021/2022, Cluster 5, mentions safe, seam- less, smart, inclusive, resilient, climate neutral and sustainable mobility systems in terms of their expected impacts.

Challenges: The perception of the environment of the vehicle is key to optimize the vehicle’s energy efficiency. Including the powertrain system (internal combustion engine, electric motor, fuel cell), energy storage system (hydrogen, electric), the passenger or cargo air conditioning system, and the traffic information (V2V, V2X). The collection of data offers the potential to characterize the context under which to perform the optimization. However, continuously collecting such data requires highly reliable connectivity (V2V, V2X) and an agreement on common mobility data sharing space[16]. In addition to standardization of data interoperability this includes data-lifecycle management that is designed around B2B, B2C and B2G data sharing.

MASSIVE SENSOR TECHNOLOGY AND NETWORKS

Massive sensor technology and networks: The new concepts for autonomous mobility, digital industry, and decentralized bidirectional and multi-modal energy supply, as well as smart city and smart home applications, require massively more sensor technology and electronics with significantly higher performance in each individual product than today. At the same time, reliability and safety requirements are increasing dramatically, as the operation of automated and autonomous systems are no longer overseen by human operators. Instead, the lives of passengers, the economics of production, and the stability of utilities depend entirely on the functionality of their electronics. The current way to ensure the highest standards of safety and availability relies mainly on redundancy at all levels of integration (including full system redundancy). This is very expensive, resource heavy and sub optimal. The failures can occur without warning and in both the primary and the redundant unit. Therefore, the ultimate fallback solutions have to be used quite often (e.g. emergency stop). They are safe but usually mean the sudden end of operation. This approach would lead to an unreasonably low availability of ultra-complex systems like autonomous cars. New strategies with smart and pro-active safety assurance need to be developed that are based on continuous self-monitoring, remaining life estimation, and active failure prevention in the electronic systems.

AI potential at the edge: An intelligent approach to functional safety will achieve a higher level of confidence and trustworthiness with less redundancy than today. It will require the inclusion of artificial intelligence algorithms as essential elements. Trained by data from comprehensive physics of failure (PoF) studies and big data-driven (DD) analyses, compact AI routines can be developed and implemented directly into the individual products to deliver maximum availability.

Challenges: Despite the limited computational resources at this edge position, the AI routines should cover the current application scenario very well. This can be achieved by developing dedicated meta-models and by resource-optimized programming. In addition, a (non-permanent) connection to the cloud server allows dynamic updates to best adapt to changing scenarios (e.g. from summer to winter conditions) and to continuously improve the meta-models (e.g. by learning from the entire fleet). A number of projects have already started to explore this approach to AI-based smart safety solution for electronic systems, e.g. ECSEL iRel4.0[18] – PoF and DD analyses, ITEA3 COMPAS[19] – compact models for AI, H2020-GV EVC1000[20] – early warning indicators, lifetime estimation. However, the main part of the research work in this area is still ahead.

Energy

SMART GRID

Distributed energy sources: The development of Smart Grids over the past two decades was a necessary response to the fundamental shift from a unidirectional supply of electricity (from power plants to consumers) toward an increasingly decentralized, bidirectional and complex network. The widespread use of renewable energy sources has resulted in a corresponding growth in the number of network nodes. Advances in ICT have enabled smart home applications which, alongside the introduction of electric vehicles, constitute new agents and further increase the complexity at individual network nodes. At the same time, the introduction of smart meters at network endpoints and ubiquitous sensors throughout the grid, have added a digital layer comprising a myriad of sensors and providing large amounts of data. This data availability and the increasing diversification and distribution of energy sources and applications call for an equivalent distribution of intelligence throughout the grid, to maximise network efficiency, optimize grid management and enable new (end-user) applications, including data privacy.

AI potential at the edge: Large amounts of data concerning energy demand and supply accumulate at individual network nodes and must be processed efficiently at the Edge to exploit their full value. Machine learning applications for Smart Grids include classification and clustering models for big data processing, are used primarily by utility suppliers and cloud service providers to group consumers according to their usage patterns and apply predictive models for future demand[21]. Prediction models can be used for the supply of renewable energy when weather forecasts are included. While many of these models can be applied for management, decision-making and control processes at the (micro) grid level can be run in cloud data centres or fog gateways, some applications potential optimisations can only be fully unlocked using edge AI. Cognitive applications of edge computing in Smart Grids include intelligent agents used both for energy market issues (management, pricing and scheduling) and for network management (security, reliability, fault handling and efficiency). Possible use cases include:

  • Combination of AI and blockchain technology for the integration of electric vehicles in power management platforms for Smart Grids[22].
  • Dynamic pricing to balance demand and supply[23].
  • Pre-processing strategy of hierarchical decision-making to optimise resource usage based on service level requirements.
  • Data-driven methods to analyse equipment and end-user behaviour in the distribution network, for example, to provide energy as a service (EaaS)[24].
  • Fault detection and diagnosis in the transmission grid (e.g. video surveillance and scene interpretation using drones).
  • Real-time monitoring[25].

Challenges: A central challenge for the application of edge computing and AI in Smart Grids remains the design and implementation of efficient system architectures that meet the real time and safety requirements on AI at the device, network and application level and distribute tasks as well as required intelligence between cloud, fog and edge.

SMART BUILDINGS

Buildings as networked cyber-physical energy systems: Buildings are a major contributor to the overall energy consumption. Since passive means (e.g. thermal insulation) are nearly fully exploited, smart buildings are envisaged to be the future enabler for further improvement in energy efficiency[26]. The objectives of building energy control systems are multi-dimensional and complex aimed at using a minimum of energy (preferably generated on-site from renewable sources), a prescribed level of comfort and a healthy indoor climate must be provided. Since the components of the building energy systems are integrating more sensors and embedded systems, buildings are becoming networked cyber-physical energy systems – especially larger objects like air- ports, shopping malls or office buildings.

AI potential at the edge: A high number of multivariate sensors are required to exploit the full potential of model predictive control schemes besides standard parameters such as temperature, humidity, CO2 and the occupation of rooms, which are usual inputs to the control system. While the main control system is usually implemented as a centralized controller, there are relevant applications for data analytics and AI on edge devices and smart sensors; for example: the number of persons present inside a room is a relevant input parameter for building controls. Image sensors allow for a precise counting of people. But to enable a required level of privacy, raw image data should not be spread among open data networks. Implementing AI algorithms directly on the device can help to analyse the image in order to extract the relevant information for the control system. Sensors in energy system components, like fans or air filters, are an enabler for predictive maintenance schemes, allowing higher efficiency and reduced maintenance costs. Using wireless technology, easy installation or retrofitting would be possible – especially at places that are hard to reach by the tethered data network. However wireless data transmission from basements can be difficult due to the metal structures in heating, ventilation and air conditioning systems. Data can be reliably transmitted, with a reduced bandwidth, by using data analytics at the sensor to provide only the relevant information on the current status of the component instead of time series data from pressure sensors etc.

Challenges: Only with a large number of sensors can explore the merits of energy savings, i.e. the economic benefit per sensor is rather low. In turn, a smart sensor for a building energy system must be a rugged and a low-cost system. Furthermore, in many use cases, wireless connectivity is strongly demanded. An optimized power consumption ensures long maintenance intervals, imposing challenges on energy efficiency of the on- board data acquisition and processing.

Digital Industry

PREDICTIVE MAINTENANCE

Industry 4.0 and predictive maintenance: Industrial applications of IoT are predicted to generate a significant economic benefit. Predictive maintenance is a popular example enabled mainly by the analysis of huge amounts of data generated by sensors integrated into industrial assets. This is implemented by the classification of the acquired data, with respect to the status of critical components, and using prediction models to enable a forecast of remaining lifetime and, in turn, to optimize maintenance schedules.

AI potential at the edge: Implementing AI on the edge devices near the sensors would offer several benefits: reduction in the transmitted data volume, which is particularly important for sensors generating large data streams such as vibration time series. Data from heterogeneous sensors can be fused on the device. This also enables cross-validation of sensor data, improving the resilience of the system. Local data analysis can reduce the latency of the AI compared to a cloud based solution, this can be an important advantage when detecting critical faults.

Challenges: In order to gain economic benefits from the sensor signal analysis, the accuracy of the algorithms has to be very high. False alarms or undetected failures can cause severe financial losses or even damages to equipment. Another important aspect is the availability of training and validation data. Only for mass production lines, the necessary amount of representative data can be collected in a reasonable time. In cases of more individualized production, algorithms have to cope with small training sets; or the application of synthetic data from simulation model scan be considered.

RELIABLE PREVENTION OF EARLY FAILURES

Reliable prevention of early product failures: Product reliability has a typical characteristic. A relatively high failure rate occurs during the first operating period. These early failures are caused by the small variations in material, shape, or process properties during fabrication. None of these stochastic deviations exceeds their specified limits, so current process control algorithms cannot detect the reliability risk that arises from unfortunate combinations of these variations.

AI potential at the edge: Expanding the scope of process control, by including a larger number of process steps in advanced data analysis using artificial intelligence schemes, can detect a significant portion of these risky combinations of inherently permissible variations. The ECSEL project iRel4.0[27] explores this approach with the ex- ample of microelectronic production. While the core part of AI-based data analysis can be performed by the large computer clusters, that provide general process control at the manufacturing site, additional edge capabilities are required to enable corrective countermeasures to be taken in real-time at all relevant process tools to provide the important data in a pre-aggregated form.

Challenges: The computational edge capabilities are thus an essential part of the overall AI system. The flexibility, latency, and security requirements of advanced process control cannot be met without them.

ROBOT CO-WORKING

Collaborative robots in industrial environments support human workforce in the fulfilment of repetitive jobs or heavy lifting, for instance. Applications can be found mainly in the manufacturing industry, e.g. assembly of automotive parts.

AI potential on the edge: Edge AI enables new possibilities for the cooperation of humans and robots, be- cause in contrast to cloud based systems edge AI is fast enough to handle situations where the robot could inflict harm. To implement these new possibilities sensors need to be deployed that are able to monitor the environment and the movement of humans and animals within range. The data from these sensors are locally processed by AI models in the robot or running on nearby edge nodes. Afterwards, edge AI-based components use the processed data to control the robot allowing for close cooperation with humans in performing com- plex tasks like the manufacturing of custom products in workshops or rescue operations. To implement this vision of close cooperation many challenges need to be solved such as training the robots for new tasks.

Challenges: In addition to more traditional robotic applications, the safety of the human worker has to be considered, since the robot and the human share a common working space. Operational strategies ensuring safety of the worker require advanced sensing capabilities of the robot[28]. In addition, the sensor data, e.g. from an image sensor, has to be processed with low latency in order to enable a quick reaction of the robot in a critical situation. Thus, transferring cognitive and analytic capabilities to the edge, i.e. a single robot, is advantageous. Potential strategies include distribution of AI methods in a network of robotic devices[29].
Finally, reliability and functional safety requirements of the robotic system with integrated AI capabilities have to be met during the design process.

Health and Wellbeing

VITAL SENSING WITH RADAR

Vital sign monitoring based on non-invasive sensors: will be an important component in many medical applications. However, a cloud-based implementation of the sensing would be too slow in time critical contexts. This is not the only problem of cloud systems as storing generated data in them is also a privacy concern.

AI potential at the edge: Issues of latency and privacy can be solved by using edge AI. When the non-invasive data is processed locally, the information about heart rate, respiration and so on are available fast enough to trigger other parts of the system that can save the life of the human. Furthermore, the results can then be deleted or anonymized before they are sent to the cloud.

Challenges: The accuracy of current edge AI implementations of such products is too low to avoid high false alarms rates. Hence, the accuracy of algorithms needs to be improved to enable better adoption of life saving applications.

PERSONALISED MEDICINE

Personalised Medicine: Human physiology can vary greatly from individual to individual. Examples for that in- clude blood pressure or lung capacity. However, these differences need to be considered for accurate medical applications like vital sign monitoring. Due to privacy concerns, it is difficult to process this information in the cloud-based solutions.

AI potential at the edge: Edge AI offers the possibility of maintaining privacy when processing medical data. Furthermore, many medical applications require real time processing, which can be better realized with local AI. By exploiting these two aspects, many medical and consumer applications can be implemented which were not possible in the past. For example, different organisations work on integrating sensors and AI into clothes allowing for feedback loop based training of athletes.

Challenges: Processing data at the Edge does not make it totally safe against malicious access. Hence, the security measures of edge AI processing pipelines need to be further improved to ensure that medical data or applications are not misused.

AFFECTIVE COMPUTING (“AI OF EMOTIONS”)

Detecting and measuring human emotions: Affective computing is interested in automatically detecting and recognizing the emotional state of a human either with remote or “nearable” sensors (visible and IR imagery, audio, physiology), or with sensors in contact (wearables) for physiology, or activity monitoring. Emotions, a classic conceptual representation of which follows a 2D valence (negative / positive) versus intensity (calm / excited) pattern, have an essential role in human behaviour. These influence the mechanisms of perception, attention, decision making, and social behaviour. The purpose of estimating emotional states is to improve understanding of human behaviour. This is the strongest reason as emotional states are both very personal and evolving, very different from one individual to another, and from one situation to another.

AI potential at the edge: The edge AI allows for maintaining the confidentiality of the data inside the measurement device, to guarantee the autonomy of the devices, and to aim for an individual estimator learning over time. The objective of the studies conducted at the CEA LETI is to develop an autonomous and ambulatory stress observer based on physiological signals, aimed at self-assessment and coaching for well-being (see M.O.T.I.O.N project)[30].

Challenges: Privacy and personalisation. On the road to individual guidance –whether medical or for other purpose (wellbeing, sports or emotion management) – local processing of data answers potential issue of confidentiality and data protection. In addition, the use of AI allows identification and adaptation to individual response pattern to the targeted monitoring (activity, treatment…). Once anonymised, this individual response (learned and characterised thanks to the AI) can feed wider models so that it can be shared and benefits to other users/patients and helps them in managing their own activities.

SPORT ANALYTICS

Prevalence of lower-limb injuries: Lower-limb injuries are common among athletes, accounting for 77% of hospitalized sport-related injuries, and are a risk factor for early-onset osteoarthritis. High-impact forces are one of the factors contributing to lower-limb injuries. To decrease the prevalence of lower-limb injuries, and their associated long-term disability and economic burden, multiple injury prevention programs have been proposed. These take into account the study of ground reaction forces (GRFs) in order to enhance athletes’ performance, determine injury-related factors, and evaluate rehabilitation programs’ outcomes.

AI potential at the edge: Together with industry partners, the Tyndall National Institute have developed a miniaturised monitoring system, integrating ultra-accurate accelerometers and neural networks, to estimate the impact GRF forces while running. Besides being a unique solution for multiple injury prevention, the developed solution can be used by elite athletes, sports teams, coaches, scientists, and consumers who would use novel performance monitoring systems to keep pushing the boundaries of their sports and gain performance advantages.

Challenges: Major challenges in the system implementation are related to the development of a neural network that is sufficiently accurate to model GRFs while it is also simple enough to be deployed on a re- source-constrained microcontroller with limited energy consumption. Moreover, an open challenge is related to the deployment of personalized athlete-specific models rather than general-purpose neural networks; this could be achieved by either training a whole network from scratch directly on the wearable unit by relying only on the data collected from the individual athlete, or by adopting a transfer learning approach where a number of layers in the general-purpose network are trained based on the data from all the available subjects and are frozen and deployed on the microcontroller and the data collected from the individual athlete are used to train only the last layers of the deployed neural network.

PHYSIOLOGICAL MONITORING

Health markers: Elevated blood pressure is a major health concern and a risk factor for complicated cardio- vascular morbidities including coronary heart disease, ischemic, and haemorrhagic stroke. WHO reported an estimated 7.5 million deaths due to elevated blood pressure. The accurate measurement of blood pressure is important to timely detect health threats. Therefore, to get a continuous, accurate, and reliable insight into a person’s cardiovascular health condition requires a practical approach.
Sepsis is also another good example, one of the leading causes of death worldwide, with incidence and mortality rates failing to decrease substantially over the last few decades.

AI potential at the edge: Edge computing is experiencing a massive growth in healthcare applications as it helps to maintain the privacy of patients (e.g. data is locally processed, without engaging cloud services in the overall process), and allows a fast and real-time decision support system.

One of the objectives of the HOLISTICS project led by Tyndall National Institute[31], in cooperation with its industry partners, is the adoption of edge analytics into wearable devices for health-related use case scenarios (e.g. blood pressure monitoring). Cuffless blood pressure monitoring devices adopting AI solutions based on the analysis of PPG or PTT signals have shown promising results in recent years.

As an example to illustrate the value of edge-based AI models in the management of vital signs, the model can raise timely alerts pro-actively prompting clinicians without needing time-consuming and costly laboratory tests. AI solutions have, therefore, the potential to be used on wearable devices to predict the prognosis (e.g. blood pressure), and/or detect the pathogens causing an infectious process (i.e. sepsis).

Challenges: A typical challenge of health-related datasets is the presence of a high imbalance in the data. The development of the outcomes for patients with sepsis and recommend the treatment process (e.g. the medi- cations to be used during sepsis), of techniques and approaches able to tackle this problem at a technical level (i.e. data augmentation, resampling techniques) and policy level (e.g. data collection process, data sharing pol- icy, new standards) is diffusing steadily. Moreover, the possibility to provide tailored medical treatment (e.g. personalized medicine) is attracting increased attention over the recent years; however, its implementation and deployment into edge devices in real-world scenarios is still in its infancy.

Agriculture, Farming and Natural Resources

AUTOMATED WEEDING

Chemical weeding to reduce the competition between weeds and crops: Vegetable production imposes a wide variety of farming operations because of the diversity of crops and the related planting parameters such as the seedbed structure, the seeding density, the spacing between rows and the distance between plants in each row. In addition to these agricultural operations, vegetables require early weeding (7 to 15 days after sowing or planting) due to the strong competition between weeds and crop and the increasing difficulty of removing weeds without damaging the crop. Once the crops cover all the row, weeds are stifled as soon as they appear, and weeding becomes less critical. Chemical weeding is the classical solution to reduce the competition between weeds and crops. However, the growing consumers’ demand for product quality and for the absence of phytosanitary residues, is having an increasing impact on agricultural practices. Mechanical weeding (hoeing) is, therefore, increasingly necessary. Nevertheless, it remains difficult to implement weeding within the rows, because destroying the weeds inside a row while preserving the plants is very delicate, especially when the sowing is dense. To date the only mechanized or automated solutions concern inter-row weeding (weeding between two rows).

AI potential at the edge: Commercial AI-based intra-row weeding solutions exist only for crops with significant inter-plant distances (lettuce or cabbage for instance). No automatic hoeing solution exist for carrot, peas, beans, sweet corn, onions, etc. To realize intra-weeding for these crops some AI-capabilities in the weeding machines are required to adapt to changing environment in real time.

Challenges: A stable connection to the cloud cannot be guaranteed on fields all the time. Furthermore, the auto- mated weeding machines should be as power efficient as possible. Both of these requirements could be solved by neuromorphic AI algorithms, due to their lower energy demand compared to standard neural networks at the Edge. Such algorithms and corresponding hardware are explored in European funding project Andante[32] but these topics will require much more work than which can be achieved within one project.

DRONES FOR PRECISION AGRICULTURE

Precision agriculture is one of the scenarios where Unmanned Aerial Vehicles (UAVs) or drones are currently being used and demonstrated. They are equipped with cameras and sensors which allow taking close images of the crops, field operations and of the machines.

AI potential at the edge: This information can be used for tasks such as obtaining Normalised Difference Veg- etation Index (NDVI) maps from multispectral cameras which can support decision making about spraying or perform additional operations in the crops, for example to recognise areas that may be affected by pests and to apply phytosanitary or pesticide treatments. They can even act as a network gateway to collect information from IoT sensors using low-cost and wide area network protocols like LoRaWAN (Long Range Wide Area Network).

Challenges: The deployment and the usage of drones and UAVs in the agriculture domain still presents challeng- es that must be solved, e.g.

  • be intelligent enough to fly autonomously without requiring
    major interventions from specialised human operators
  • be capable of dynamically readjusting the missions based on context information coming from onboard sensors and other sources of data deployed in the crops
  • collaborate with other drones or ground robots to perform more complex tasks in complex and larger terrains
  • guarantee compliance with security regulations and incorporate trustworthy requirements and guidelines.

To address most of the previous points, artificial intelligence processes will be embedded directly on drones and
robots in order to increase their autonomy and real-time capabilities.

SOIL CONTROL

Efficient food production is important to ensure the food supply of mankind. Hence, more and more sensors are deployed around, and in fields to gather data about their state and planted crops. For soil monitoring Biodegradable sensors are being researched. The idea is to mix them into the fertilizer, which is then put on the field. Afterwards, they send their data for between six months and one year to a node near the field and this node transfers the data to the cloud.

AI potential at the edge: The amount of data generated by the field monitoring sensors is very high. However, not all of the data is relevant and can be averaged over multiple sensors e.g. the average soil moisture level of a field. Edge AI can be trained to analyse these large data volumes resulting in lower amounts of data needed to be sent to the cloud as well as lower network load.

Challenges: The critical point about bio-degradable sensors is that they cease to work after a specific amount of time. In addition, other sensors deployed around the field may have a lower average lifetime than sensors in other contexts. This requires the Edge AI solutions for this application to be able to handle fluctuating amounts of incoming data. Such high levels of flexibility are not well explored yet.

Smart Cities

SMART STREETLIGHTS

Lighting systems adjusting the brightness to the individual conditions of the surroundings: Smart street- lights could provide important services for smarter and greener cities in the future.

AI potential at the edge: Using different kinds of sensors and edge AI, the streetlights can detect whether, and at what speed, a pedestrian or motorist is approaching. As long as the person is within the radius of the light, this area is illuminated by built-in LED lamps. If the person moves away, the lighting is reduced. In adverse weather conditions, such as snow or rain, the light output could be increased automatically as required. The edge AI evaluating the sensor data can run on microcontrollers in the lamp or on other nodes in the proximity. This dynamic light regulation saves energy and costs. Smart streetlights could also be used for implementing other important services like the charging of electric vehicles and the measurement of the air quality.

Challenges: A central challenge of this application is managing the access to the results of the Edge AI. The processed sensor data can be of interest to different parties, for example for the police in case of accidents or insurance services that insure shops near the smart street lights. One approach to solve this challenge would be to combine block chain technologies with Edge AI. However, this is a research field which is still in its initial phase.

AIR QUALITY

Improving air quality using gas sensors: Gas sensors currently available on the market are often quite unstable, inaccurate and show large cross sensitivities to other interfering gases. Moreover, they are often very large (not in a portable form factor) and quite costly.

AI potential at the edge: Neural networks at the Edge are crucial to gas sensing especially when it comes to accurately identifying different gases in an outdoor environment. While the sensor technology itself (materials, geometry, temperature modulation, number of sensing fields, etc.) can surely help to improve sensitivity to target gases, algorithms play a very important role when it comes not only to classifying gases but also to quantifying them in parts per billion (ppb). Since gas sensors in most use cases have limited connection to the internet, these algorithms need to be deployed on the sensor node.

Challenges: Recent results already show that traditional neural networks can strike the right balance between accuracy and robustness for air quality monitoring, still many open questions remain on the behaviour of air quality monitoring sensor deployed in the field over a long time. Here, it is even more crucial to ensure long battery life and wider online learning at the Edge for specific use cases and more self-diagnostics on the performance of the sensor.

ALARM SYSTEMS

Increased safety with intelligent Alarm Systems: Edge AI-based alarm systems are a good example of how edge computing solutions enrich existing smart building systems.

AI potential at the edge: While previous alarm systems use a microphone and simple threshold rules to detect glass breakage when an unlawful entry is made into an apartment, the new generation of alarms process information from multiple data sources via data fusion and neural networks. This minimizes the number of false alarm and significantly increases the reliability of the system. This solution can easily be integrated into existing glass breakage alarm systems.

Challenges: As these new generations of alarm system become more widely spread, they will become targets for attacks. Currently little work has been done in the area of securing neural networks, making them responsible for the whole system.

References

[10] https://www.vodafone.com/news/technology/multi-access-edge-computing-to-power-artificial-
intelligence-for-automotive

[11] Armengaud E., Peischl B., Priller P., Veledar O. (2019) Automotive Meets ICT—Enabling the Shift of Value Creation Supported by European R&D. In: Langheim J. (eds) Electronic Components and Systems for Automotive Applications. Lecture Notes in Mobility. Springer, Cham. https://doi.org/10.1007/978- 3-030-14156-1_4

[12] https://hadrianproject.eu

[13] https://teaching-h2020.eu

[14] FRACTAL project: https://fractal-project.eu

[15] ECSEL JU Integrated Development 4.0 project: http://www.idev40.eu

[16] TOWARDS A EUROPEAN DATA SHARING SPACE Enabling data exchange and unlocking AI potential, BDVA Position Paper, April 2019.

[17] A-Swarm project: https://www.tu-berlin.de/?209696

[18] ECSEL iRel4.0: https://www.irel40.eu

[19] ITEA3 COMPAS: https://itea3.org/project/compas.html

[20] http://www.evc1000.eu

[21] Samie F., Bauer L., Henkel J. (2019) Edge Computing for Smart Grid: An Overview on Architectures and Solutions. In: Siozios K., Anagnostos D., Soudris D., Kosmatopoulos E. (eds) IoT for Smart Grids. Power Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-03640-9_2

[22] Z. Wang, M. Ogbodo, H. Huang, C. Qiu, M. Hisada and A. B. Abdallah, “AEBIS: AI-Enabled Blockchain- Based Electric Vehicle Integration System for Power Management in Smart Grid Platform,” in IEEE Access, vol. 8, pp. 226409-226421, 2020, doi: 10.1109/ACCESS.2020.3044612.

[23] S. Chen et al., “Internet of Things Based Smart Grids Supported by Intelligent Edge Computing,” in IEEE Access, vol. 7, pp. 74089-74102, 2019, doi: 10.1109/ACCESS.2019.2920488.

[24] Feng, Cheng & Wang, Yi & Chen, Qixin & Ding, Yi & Strbac, G. & Kang, Chongqing. (2020). Smart Grid Encounters Edge Computing: Opportunities and Applications. Advances in Applied Energy. 1. 10.1016/j.adapen.2020.100006.

[25] Huang, Yutao et al. “An Edge Computing Framework for Real-Time Monitoring in Smart Grid.” 2018 IEEE International Conference on Industrial Internet (ICII) (2018): 99-108.

[26] Jennifer King and Christopher Perry: “Smart Buildings: Using Smart Technology to Save Energy in
Existing Buildings”, American Council for an Energy-Efficient Economy, Report A1701, 2017

[27] https://www.irel40.eu

[28] Michalos, G., Makris, S., Tsarouchi, P., Guasch, T., Kontovrakis, D., & Chryssolouris, G. (2015). Design Considerations for Safe Human-robot Collaborative Workplaces. Procedia CIRP, 37, 248–253. https:// doi.org/10.1016/j.procir.2015.08.014

[29] Hadidi, R., Cao, J., Woodward, M., Ryoo, M. S., & Kim, H. (2018). Distributed Perception by Collaborative Robots. IEEE Robotics and Automation Letters, 3(4), 3709–3716. https://doi.org/10.1109/ LRA.2018.2856261

[30] http://www.leti-cea.com/cea-tech/leti/english/Pages/Leti/Projects%20supported/Motion-project.aspx

[31] https://www.tyndall.ie/news/tyndall-welcomes–8-million-in-disruptive-technologies-innovation- funding

[32] European funding project Andante: https://cordis.europa.eu/project/id/876925/de

Credits

Parts of this roadmap is reproduced from the European Platform on Smart Systems Integration (EPoSS) 2021 Whitepaper.