Catholijn Jonker

V: Values, Ethics and Responsible AI

From the beginning of my academic career I found myself quickly in a smaller camp of AI researchers that are not only interested in maximizing utility for the party their AI would represent, but also in increasing social welfare, that are concerned about fairness. From thereon, I have always supported, advocated and joined the research into value-sensitive and responsible AI. It is for that reason that the words "values", "ethics", and "responsible" occur in my research lines and projects. In the figure, this orientation is depicted by the yellow sphere that provides energy on my research like sunshine would do for life on our planet.

R1: Interactive Intelligence for Hybrid Intelligent Systems

To get hybrid intelligence flowing, in most of my projects I focus on researching and developing interactive intelligence, i.e., the intelligence needed to set up and maintain long term collaborations between intelligent beings. That type of interactive intelligence I embed in artificial autonomous agents. To create hybrid intelligence, what should artificial agents know about human about human participatory processes and participatory decision making? Example questions are, what role play emotions and trust? What can agents do to ensure that the humans they collaborate with have an appropriate level of trust in the agent? What type of explanation do they need from the artificial intelligence? How can the agent learn from human intelligence? What type of explanation should the agent ask from humans? How do you foster diversity? What group processes and dynamics hinder or effectively lead to outcomes that are supported by the stakeholders? What interventions would be beneficial in what situations? That modelling in this research line is done with artificial intelligence techniques.

R2: Cognitive Modelling in relation to Collaboration and Participation

For artificial agents to collaborate well with natural intelligent beings, they need (shared) mental models, sometimes even a theory of mind of natural aspects or forms of intelligence. Therefore, in some projects I focus on modelling cognitive aspects that are essential for interaction, coordination and collaboration. Example questions are, how can the agent recognize human emotions? what values and concerns drive people in participatory processes? How can an agent estimate the different perspectives of the stakeholders (preferences, values and concerns? What makes people (dis)engage in participation? How does human information processing work? How type of information do human participants need and how can agents recognize this need? That modelling in this research line is done with artificial intelligence techniques.

R3: Deliberation and Decision Support in Participatory Processes

In other projects I focus on the challenge of creating autonomous intelligent agents or artificial intelligence techniques. In these cases I try to develop artificial intelligence that is complementary to that of humans and that works well when combined with human intelligence. In particular, I focus on artificial intelligence that can support or augment human deliberations and decision making in participatory processes.

Can we support people in their deliberations about the social situation to determine their actions, and be able to form a mental model about their human team mates? I intend to support both agents and humans in their deliberative processes. Mass deliberation support will help citizens and policy makers to take informed decisions in democratic processes. In this endeavour, my aim is to engage people in lasting interests in the policy making of their environments. For this I research together with domain experts what are the strengths and weaknesses of humans in those participatory processes. Based on that analysis I develop artficial strategies and techniques to support the human participants in the aspects that are difficult for them. Furthermore, I research collaborative strategies and protocols for combining the artificial intelligence for the weak aspects with the strengths of human intelligence. I deploy emperical research methods to evaluate the effectiveness of the hybrid intelligence thus created.

R4: Negotiation Support and Automation

Negotiation support systems for complex situations. I initiated and with an international collaboration continuously develop GeniusWeb (old version: GENIUS), the internationally most used platform to support research into negotiation mechanisms, protocols, and intelligent negotiating agents. The Pocket Negotiator, is the concept of a Negotiation Support System for which I received a (NWO-STW VICI grant) in 2008. The experimental version is called the Pocket Negotiator, or go to WeGain B.V., for the commercial version Enigma.

My research on the embedding of artificial intelligence in negotiation comes in two flavours: automated negotiation, and negotiation support. This research line crosses the other research lines, e.g., the best way to improve the negotiation process with support is if the support agent can learn and understand the perspectives of the negotiators, which includes their value profiles, underlying concerns, and their preference profile with respect to the negotiable issues. Furthermore, the strategies for what to offer when and when to accept bids has to be optimized in relation to the wishes of the negotiator that the agent is supporting. I run the automated negotiation research line as a way to underpin the support agent with strategies that are optimized in fierce competition with other strategies. For this purpose, I launched the initiative for the annual international Automated Negotiating Agents Competition (ANAC), which has boosted the research in automated negotiation agents.

Application Domains

The only way to evaluate and validate my research findings is to apply these findings. By trying to apply the research findings provides insights in the assumptions made, for example on the dependencies on other technology, the availability of information, the willingness and ability of stakeholders to use or interact with the artificial intelligence. Typically becoming aware of these assumptions and dependencies leads to more depth and sharper focus in the research. Sometimes this means back to the drawing board and realizing that the time to application (let alone time to market) is still long as more fundamental research has to be done first.

Over the years I was inspired by, learned about, and worked in the following application domains:

• Urban Twinning
• Climate debates
• Lifestyle and Personalized Care
• Cybernetic Incident Management & International Peace Keeping Missions
• Animal behaviour, in particular about dogs and horses