Hybrid AI White Paper by TNO | Meyer-Vitali et al. 2019
Source: https://repository.tno.nl//islandora/object/uuid:510e0747-30f4-4af4-9447-9330b7bd4b47
Motivation
Current approaches in Artificial Intelligence (AI), particularly machine learning, have recently reached an unprecedented impact in science and society. There are, however, concerns about interpretability and accountability of such AI, which limit its usefulness and trustworthiness. Current learning AI systems operate almost exclusively in a statistical (model-free) mode, which limits their potential and performance [1]. Many works point out that there is a fundamental need for knowledge representation, reasoning and sharing that is integrated with machine learning systems in order to provide sound and explainable models to alleviate these problems [2, 3, 5, 6, 7, 8]
Machine learning for robust learning, semantic models and knowledge representation for reasoning capabilities.
- Current AI systems fail when cases appear in a context not covered by training data
- Huge amount of examples in training data needed for decent performance; strong contrast with human one-shot learning
- Deep learning is shallow (pun intended?): statistical pattern matching without knowing what the data is about.
- Succes in games is not that interesting long term: the real world does not have complete information, but uncertainties, inconsistencies, lack of information.
- Hybrid AI complements learning from data with reasoning using knowledge and (social) interaction
- "The goal of Hybrid AI is to bridge the semantic gap between data and symbolic knowledge (concepts, relationships and rules)" (4)
Example also w.r.t. explainability:
For example, providing pictures of a thus far unseen bird combined with a model that describes its features (two legs, beak, wings, etc.) will allow for a more efficient learning process [2, 8]. These systems will also be far better at explaining how they arrived at a certain conclusion: instead of just detailing input features, such a system will not only be able to recognise the bird, but also explain why it came to this conclusion based on symbolic descriptive models [2, 6, 7]. Such symbolic explanations are required for sharing knowledge and for coordinating tasks with other agents and humans.
Definition
We focus on Hybrid AI systems that aim to assist humans in daily life by providing smart support tools (collaborative decision support) and by operating autonomously on their behalf (autonomous decision making), while considering given goals and norms. Crucial for such AI tools is the ability to interpret the (social) context in which they operate. (5)
Advantages of hybrid AI:
- more general purpose
- operate in dynamic and unpredictable situations
- collaborate with humans and other systems
- suitable for situations where privacy, safety, security and ethical constraints are critical
Semantic Gap
In Peirce's semiotic triangle:
- Semantic gap is difference between symbol and concept
- Gap between concept and object is symbol grounding
- Machine learning provides object's data with a symbol, e.g. a class label, but without the connection to the concept that the symbol refers to.
A route to bridge the semantic gap thus is to connect symbol to the corresponding concept.
Hybrid AI vs. Hybrid Intelligence
Hybrid AI is a component of Hybrid Intelligence
In figure 4, Hybrid AI is defined as the combination of reasoning on symbolic knowledge with machine learning from data (objects) embodied in an intelligent agent. Hybrid AI is part of systems of Hybrid Intelligence (HI), where intelligent agents learn and reason and interact with humans and other agents in relation to the environment that they are situated in. Intelligent agents need to acquire awareness about themselves, their context and users (both human and other agents).
Benefits
This section discusses common issues and chances for Hybrid AI.
The potential benefits of Hybrid AI systems lie in their increased trustworthiness, transparency, explainability, controllability and responsibility. They may facilitate the implementation of legal and ethical norms in future distributed collaborative systems in order to contribute to more human-aware systems. We elaborate on the benefits with respect to controllability, explainability and responsibility of Hybrid AI systems after explaining some common issues.
For the most interesting:
- Hybrid AI can take e.g. ethical constraints and norms into account
- Hybrid AI offers better options for explainability
Communication with Hybrid AI & Conversational XAI:
Communication of humans with Hybrid AI systems is preferably bidirectional and takes the form of a dialogue. Felicity conditions for fruitful, dialogue-based communication include rich, semantic representations of subject matter. This becomes especially important when humans are facilitated to insert domain or world knowledge in an AI system. On the other hand, extracting semantic information from the eventual sub-symbolic contents (e.g. hidden layers) of an AI component is crucial for successful human-AI interfacing. (11)
Common Issues
Skipped
Controllability
Adaptivity and learning can lead to unsafe or unethical behaviour in unforeseen situations. Controllable agents should be able to recognize these situations and include safeguards, enabling human intervention and control if necessary.
Key requirements:
- Operations in an open world
- User control
- accept and learn from the user
- Self-assessment
- agent should determine if it can meet the task objective; or enable a human to determine so
- Self-management
- adapt its own configuration to actual situation
- learn from previous experience to optimise its performance
- Explainability
- Controllable agent must explain to the user what it's plans are and why it's doing what it does
Interaction with human users is a source for never-ending learning:
The user feedback can be considered as top-down symbolic knowledge about the world that gives meaning to bottom-up, sub-symbolic sensor data. (12)
Exploiting Domain Knowledge to improve performance and more efficiently use resources:
The coupling of such contextual world knowledge on the one hand, and sensory interpretation on the other hand, is an important research topic for Hybrid AI systems. (12)
Context awareness: "Awareness about context (top-down symbolic knowledge) is essential for interpretation." (12)
Context adaptivity i.e. recognize when the environment changes:
The AI agent should signal that the environment has changed, so it can take proper actions, such as loading a model with nature classes instead of urban classes, loading another model that is known to work less accurately but more robustly in different conditions, or asking the operator for guidance. (13)
Learning with few examples: often very little or even no training data is available for a ML application, for various reasons. One approach to tackle this is to divide a problem into subcompnents taht do have ample training data. A more high level model (e.g. finite state machine) can then combine the representations/features acquired by the other (ML) models.
For the example of stress assessment: the mid-level representation may be designed to consist of observable stress indicators for which sufficient labelled samples can be acquired. Again, top-down knowledge is combined with bottom-up sensory observations. (13)
Explainability
Various ideas and approaches:
- Bring sub-symbolic closer "to the symbolic world, by forcing neural networks to focus on semantic, interpretable structures in their data."
- This is not hybrid AI, but is an important ingredient because it works towards a more semantic "language" that can be used in human-AI dialogue and for better explainability
- Address "more procedural information exchange between humans and machine learning models"
- Example: attention networks that "emit probabilities linked to its input layer" to show "the parts of the data that attract most of the networks' attention"
- A more low-level human-AI dialogue (that is hybrid) would be a system that allows interaction between the neural attention and human attention
- Explaining which data underlies an AI-produced outcome, and - vice versa -
assessing whether human-provided data is biased, is helpful for bias resolution.
- "For example, the attention mechanism described above may serve to highlight inductive bias by an AI algorithm" (14)
- "Analogy-based explanations (displaying training data similar to test data or its model abstractions of that data) have strong communicative and explanatory value." (14)
TNO's schema for Human-Robot communication contains a step where subsymbolic representations are translated to symbolic representations, that are in turn communicated to a human. This step is the main challenge for hybrid AI.
Additionally, the symbolic representation must be chosen such that it fits the user's mental model for personalised explainability.
This process of grounding a semantic concept into variables known to the autonomous system is done through so-called “semantic anchors” that are implemented inside the autonomous system (cf. fig. 6).
Semantic anchors
For meaningful AI-human communication, the used symbolic representations should relate to a concept, which is grounded in these symbolic representations/variables.
Data-auditable AI as the way to connect data usage and ontology:
A fruitful combination of semantic anchors with data-driven models demands a type of data-auditable AI: AI that can be explained from a data usage perspective, answering questions such as: which data is being used in the latent space of a deep learning network just prior to outputting a decision? Once these data circumscriptions are clear, external knowledge can find its way into a model.
I find this an interesting central thought, but it's still quite abstract to me.
Grounding a semantic concept into variables is done through "semantic anchors" that connect variables to the ontology.
Example (I find this still very abstract):
An example of a more complex semantic anchor could be when the definition of car is not known as such by the system. Perhaps the system only has a camera-feed with pixels that represent that car but not its position or velocity. The semantic anchor may then contain an operation that extracts the position of that car as well as its velocity vector to obtain a complete description of a car according to the ontology. (15-16)
Structured ML approaches
Approaches that use high-level knowledge about phenomena and mid-level features of phenomena are useful for explainability.
Rules and features can be easily explained to the user.
For instance, consider a robot which is collaborating with a human. The robot has a camera and its video AI interprets that the human is waving, because the AI observes that the human’s arm is moving sideways in the air. It can communicate ‘Did you ask for help? I noticed that you waved your arm’. Such communication is much more human-like than using terms of changing pixels or movement in the image itself. (16)
Visual explanations
Example:
For the verification of a hypothesis of an AI agent, a user may want to check on which information the hypothesis was based. Of course this can be done using the lower-level data, but this is hard to interpret. AI agents that operate on images (camera, spectrograms, etc.) are able to communicate their hypotheses in the form of images, possibly with metadata such as overlays (boxes, texts, regions). For instance, a hypothesis about an observed scenario, such as IED placement (see above), may be shown to the user by one snapshot for each of the consecutive activities that were observed and linked (e.g., standing next to a road, digging, placing something in the ground, walking away). (16)
Responsibility
Very interesting overview! But I'm in a rush, so won't take notes here.
Engineering & Design Patterns
Hybrid AI first of all requires knowledge engineering.
Knowledge acquisition:
Knowledge acquisition requires establishing links between learned structures and activations in a neural network (which are the outcomes of machine learning algorithms) and human-understandable concepts (as formulated using knowledge graphs, for example). These links should be bidirectional and continuously updated. (20)
Knowledge models include concepts, properties, relationships and rules, as well as provide causal relationships between actions and effects. (20)
Frank van Harmelen and Annette ten Teije proposed several design patterns for hybrid AI, i.e. ways for combining symbolic AI with machine learning.
- sym -> SR -> sym
- GOFAI
- data -> ML -> data
- typical machine learning system
- sym -> SR -> sym -> (data -> ML -> data)
- HYBRID 1: Use domain knowledge as a prior on the ML system
- OR use it as a meta-reasoner over the ML system
- Example: Logical Tensor Networks, see: https://arxiv.org/abs/2012.13635
- data -> ML -> sym -> SR -> sym
- HYBRID 2: Ontology learning i.e. learn a symbolic structure from data
- Then use the ontology in a reasoning step
- Combine ML and SR into a single system
Examples
This example is double:
Robotic systems consist of many components with limited resources, incl. time. Learning on the fly and adapting to a constantly changing environment is crucial. Robots can perform self-assessment to determine their level of performance, both logically and physically (wear & tear), by comparing learned properties with a reference model of themselves. (23)
Decision support systems:
Decision support systems have the purpose to support humans in their decisions. For example in the health domain, it is important to have explainable systems in which we have a human-in-the-loop. In these systems the human-in-the-loop helps to either improve explanations (for example in using a memory to remember which explanations were effective and which not) or update a machine learning model with the expert knowledge.
Ontology learning:
Creating ontologies is a tedious task. NLP / ML algorithms can extract relations between words from large bodies of texts. With these relations an ontology can be created using data-driven methods. A human expert is, however, needed in the loop to judge whether the relations are correct and relevant. In the Agri domain, we automatically created ontologies comparing several algorithms and qualitatively and quantitatively measured performance. A next step is to use it as a quick start in an ontology creation session. The algorithms can also be used in other domains.