Category: Artificial Intelligence

For many years, negotiation support systems were designed to assist human negotiators. They could help us analyze our preferences, prepare arguments, simulate possible scenarios, or think through trade-offs. The human negotiator, however, remained in control. The person decided what to disclose, when to make a concession, how to respond to pressure, and where to draw the ethical line.

AI negotiation agents change this logic. Some of them can negotiate authonomically, may exchange offers, make concessions, select tactics, adapt communication, and pursue outcomes with limited or no real-time human intervention. This opens exciting possibilities, but it also raises a question that negotiation scholars, practitioners, developers, and organizations cannot afford to ignore:

What happens when an AI agent is optimized to win at all cost?

Our new article, “Ethical Guidelines for Designing, Developing, and Deploying AI Negotiation Agents,” co-authored with Jan Smolinski, Peter Kesting, and Felix Kröcher, has now been published open access in Group Decision and Negotiation. In the paper, we examine the ethical risks of autonomous AI negotiation agents and propose a lifecycle-oriented framework for designing, developing, and deploying them responsibly.

The promise of AI negotiation agents

There are good reasons why organizations are interested in AI negotiation agents. Negotiations are costly. They require time, attention, expertise, preparation, and coordination. Many organizations conduct thousands of negotiations that are repetitive, data-rich, and structurally similar. In such contexts, AI agents may help reduce transaction costs, accelerate negotiation cycles, and make individualized negotiation possible at scale.

AI systems may also identify trade-offs that humans overlook. They can process large amounts of information, compare options quickly, and generate offers that better match the preferences of both sides. In this sense, AI negotiation agents may support not only value claiming, but also value creation.

If designed well, AI agents could help negotiators find better agreements, reduce inefficiencies, and make negotiation processes more accessible. They could support procurement, sales, dispute resolution, employment negotiations, platform-based transactions, and many other settings in which negotiation plays a central role.

However, the same capabilities that make AI negotiation agents powerful also make them ethically risky.

Why negotiation is a difficult context for AI ethics

Negotiation is a strategic communication process shaped by conflicting interests, information asymmetries, uncertainty, persuasion, trust, pressure, and power. Negotiators rarely disclose everything they know. They frame information selectively. They test the other side’s priorities. They manage impressions. They claim value while trying to preserve relationships.

This makes negotiation ethically complex even when humans are involved. Some tactics are clearly unethical. Others fall into a gray area. Exaggerating one’s alternatives, withholding critical information, exploiting the other side’s assumptions, using emotional pressure, or manipulating trust may sometimes produce better short-term outcomes. But they can also damage relationships, undermine legitimacy, and destroy long-term value.

When AI enters this environment, new risks may emerge and existing ones may be amplified. An AI negotiation agent can act faster, more consistently, and at much greater scale than a human negotiator. It can adapt its communication style, detect vulnerabilities, test persuasive strategies, and optimize for outcomes without fatigue, hesitation, or moral discomfort. If the system is trained or rewarded only to maximize its principal’s gain, it may learn to use tactics that are effective but ethically questionable.

This is the central concern of our paper. So far, the main focus of research on AI in negotiation has been agents’ performance and their ability to reach good agreements. However, we also need to investigate how they achieve them.

The problem of unethical myopia

One of the concepts we discuss in the article is what we call unethical myopia (Kesting et al. 2025).

AI negotiation agents may be especially prone to short-term optimization. If their objective function is too narrow, they may prioritize immediate economic gains while underestimating the long-term consequences of their behavior. They may secure a favorable agreement today while damaging trust, fairness, reputation, or future cooperation.

In human negotiation, this problem is already familiar. Negotiators sometimes win the deal and lose the relationship. They may claim more value in the short run but reduce the chances of future collaboration.

With AI agents, this problem becomes more systematic. The agent may not naturally understand why trust matters beyond the current transaction. It may not recognize that a technically successful agreement can still be perceived as unfair, manipulative, or illegitimate. It may not fully account for reputational spillovers, repeated interactions, or the broader institutional consequences of normalizing ethically questionable tactics.

This is why ethical AI negotiation has to be built into the system from the beginning.

A lifecycle approach to ethical AI negotiation

In the article, we propose that ethical guidance for AI negotiation agents should follow the full lifecycle of the system: design, development, and deployment.

At the design stage, developers need to define what the agent is supposed to optimize. Is it only economic surplus? Is it also fairness, sustainability, relationship quality, transparency, or the viability of the agreement? These choices matter because they shape the behavior the agent will learn and reproduce.

At the development stage, ethical principles must be translated into technical safeguards. This includes decisions about training data, reward structures, bias mitigation, transparency, privacy, human oversight, and the prevention of manipulative or exploitative behavior. Developers need to think carefully about how the agent treats vulnerable counterparts, how it handles information asymmetry, and how it balances strategic behavior with ethical boundaries.

At the deployment stage, organizations need governance mechanisms. AI negotiation agents should be monitored, audited, updated, and evaluated in real negotiation contexts. Their behavior should not only be assessed by outcome metrics, but also by process quality, fairness, trust, and long-term consequences.

This lifecycle perspective is important because ethical risks do not appear at only one moment. They emerge throughout the system’s life. A well-intentioned design can fail during implementation. A technically sound system can produce problematic behavior after deployment. A safe agent in one context may become risky in another. Ethical governance therefore has to be continuous.

Why this matters for negotiation theory and practice

This paper contributes to the growing discussion on AI in negotiation, but it also connects to a much older question in negotiation research: What does it mean to negotiate well?

A good negotiation outcome is not only a matter of price, terms, or efficiency. Negotiation also affects trust, relationships, legitimacy, fairness, and the possibility of future cooperation. This is why negotiation ethics has always been central to the field, even when it is not explicitly discussed.

AI negotiation agents force us to revisit these questions. If we delegate negotiation to machines, we also delegate choices about pressure, disclosure, persuasion, fairness, and value distribution. These choices are not merely technical. They are ethical and strategic at the same time.

So, shall we prevent AI from entering negotiation? That is unlikely and, in many cases, undesirable. AI has the potential to improve negotiation processes and help parties reach better agreements.

The real challenge is to ensure that AI negotiation agents are designed to create value without undermining the norms that make negotiation legitimate in the first place.

Looking ahead

We see this article as a step toward a more specific and practical conversation about responsible AI in negotiation. General AI ethics principles are important, but negotiation requires more tailored guidance. The ethical risks of an autonomous negotiation agent differ from those of a chatbot, recommender system, or document summarizer. Negotiation involves strategic interaction, private information, competing interests, and relational consequences. These features have to be reflected in the way AI systems are designed and governed.

As AI agents become more capable, organizations will need to make deliberate choices. Developers will need to decide what kind of behavior their systems should learn. Managers will need to decide what kinds of outcomes they reward. Regulators will need to understand where negotiation-specific risks arise. Scholars will need to test which guidelines actually reduce unethical behavior in practice. Most importantly, we need to avoid the illusion that successful negotiation AI is simply AI that gets the best deal.

In negotiation, how we get there matters.

---

Source: Ethical Guidelines for Designing, Developing, and Deploying AI Negotiation Agents | Group Decision and Negotiation | Springer Nature Link

Artificial intelligence is increasingly discussed as a tool that may support negotiation, mediation, early warning, and dispute resolution. Much of this discussion, however, still treats AI as a collection of separate applications: a system that summarizes documents, a chatbot that helps people prepare for a negotiation, a platform that supports online dispute resolution, or an algorithm that detects signs of conflict escalation. These applications may be useful, but they capture only part of the challenge. If AI is to make a meaningful contribution to conflict management, we should not think about it only as a set of tools. We should think of it as public infrastructure, available to everyone, not only to the privileged.

This is the central argument of our contribution, co-authored with Peter Kesting, to the Expert Perspectives on New Paths to Peace edition of Negotiation and Conflict Management Research. The edition asks how scholars and practitioners might broaden the repertoire of approaches available for preventing, managing, and resolving conflict. Its premise is not that despite a wide range of established methods of diplomacy, mediation, negotiation, or peacebuilding, the number and intensity of conflict has become higher than every before. It invites us to examine what additional capacities may be needed in a world in which conflicts are increasingly complex, information-rich, fast-moving, and difficult to contain.

Our article proposes that AI-based conflict management should be developed as public-interest infrastructure. By this we mean a shared socio-technical system designed to help individuals, organizations, communities, and institutions understand conflicts better, assess risks more carefully, explore options more systematically, and access appropriate forms of support. The key point is that such infrastructure should not be created primarily as a private advantage for those who can afford sophisticated tools. It should be oriented toward broad collective benefit, much like other forms of infrastructure that support social functioning.

From private tools to public goods

The language of public goods is useful because it shifts attention from individual use to collective value. In economic theory, public goods are often described as resources from which people cannot easily be excluded and whose use by one person does not necessarily reduce their availability to others. AI-based conflict management infrastructure would not be a public good in this strict economic sense. It would require investment, governance, maintenance, institutional oversight, and rules of access. Nevertheless, the public-good perspective is helpful because the benefits of better conflict management extend far beyond the immediate users of a particular system.

When conflicts are managed constructively, the positive effects are rarely limited to the parties directly involved. Organizations avoid disruption, communities preserve relationships, public institutions maintain legitimacy, and societies reduce the human and economic costs of escalation. Conversely, poorly managed conflicts often create negative externalities: mistrust, polarization, violence, litigation, displacement, institutional paralysis, and long-term social fragmentation. For this reason, improving access to high-quality conflict management support should be understood not only as a private service, but also as a public interest.

This has important implications for how AI systems in this field should be designed. A commercial tool may be optimized for user convenience, market share, or proprietary advantage. A public-interest infrastructure must be optimized for trust, accessibility, neutrality, accountability, and responsible use. These criteria are particularly important in conflict settings because the information involved is sensitive, the stakes are often high, and the legitimacy of the process matters as much as the quality of the analytical output.

What AI can contribute to conflict management

AI will not replace mediators, negotiators, diplomats, community leaders, judges, or political decision-makers. Nor should it. Conflict management is not simply a problem of information processing. It involves emotions, identity, power, legitimacy, trust, moral judgment, and human responsibility. However, AI can support human actors by helping them deal with complexity more effectively.

One possible contribution concerns early warning and risk analysis. Conflicts often produce signals before they escalate, but these signals may be dispersed across different sources, interpreted inconsistently, or noticed too late. AI systems can help organize large amounts of information, identify patterns, make emerging risks more visible, and help human actors recognize developments that require attention.

A second contribution lies in conflict mapping. Many disputes involve multiple stakeholders, overlapping interests, competing narratives, and hidden assumptions. Parties may focus on stated positions while overlooking underlying needs, constraints, or fears. AI can help structure this complexity by organizing information about actors, issues, interests, relationships, and possible points of misunderstanding. Used carefully, such support can make conflict analysis more transparent and systematic.

A third contribution concerns perspective-taking. In many conflicts, the formal parties at the table do not represent everyone affected by the outcome. Some voices are excluded because of power asymmetries, institutional barriers, geographic distance, fear, or lack of resources. AI cannot solve these political and ethical problems by itself, but it can help make overlooked stakeholders, interests, and consequences more visible. This may improve the quality of preparation, process design, and decision-making.

AI may also support the generation of options. Negotiations often become trapped in narrow comparisons between fixed positions. By drawing on large bodies of knowledge, analogous cases, and structured problem-solving techniques, AI systems can help generate alternative solutions, identify possible trade-offs, and suggest integrative options. The value of this function is not that the system produces the “right” answer, but that it expands the range of possibilities that humans can evaluate.

Finally, AI can support process navigation. Many people facing conflict do not know whether they need mediation, legal advice, facilitation, counseling, an internal complaint process, restorative dialogue, or another form of support. A well-designed public-interest system could help users understand the nature of their conflict, consider the risks of different pathways, and connect them with qualified professionals or institutions. This may be especially valuable for individuals and communities that currently lack access to expert conflict management advice.

Why infrastructure matters

Thinking in terms of infrastructure changes the normative and institutional questions we ask. If AI for conflict management is treated merely as a tool, then the main questions are technical: Does it work? Is it accurate? Is it efficient? Can it scale? These questions matter, but they are not sufficient. Once we think of AI as infrastructure, additional questions become unavoidable: Who governs it? Who has access to it? Who can audit it? Whose interests does it serve? How is sensitive data protected? What forms of misuse are prohibited? How are errors corrected? How is neutrality maintained?

These questions determine whether such systems can be trusted. Trust is indispensable in conflict management because parties are often suspicious of each other and of the institutions around them. A system that is perceived as biased, opaque, politically captured, commercially exploitative, or vulnerable to misuse will not support constructive conflict resolution. It may even intensify mistrust.

Public-interest infrastructure therefore requires clear governance principles. Access should be broad and non-discriminatory wherever the system is offered. Human decision-making authority must remain intact, so that AI informs, structures, and supports, but does not decide. The system must be transparent enough to be audited and challenged. Data protection must be especially robust because conflict-related information may expose individuals and communities to serious risks. Referral mechanisms must be based on clear criteria, regular vetting, and conflict-of-interest safeguards.

Equally important are red lines. AI-based conflict management infrastructure should not be used for surveillance, repression, coercive manipulation, military targeting, or political control. These risks are not hypothetical. Technologies designed to analyze social tensions can also be used to monitor dissent, identify vulnerable groups, manipulate narratives, or strengthen coercive power. For this reason, the institutional design of such infrastructure is the core condition of legitimacy.

Different users, shared benefits

The potential users of AI-based conflict management infrastructure are diverse. Individuals might use it to understand a workplace dispute, family conflict, neighborhood disagreement, or institutional problem before the situation escalates. Organizations might use it to identify recurring sources of tension, prepare for mediation, or design fairer internal processes. Communities might use it to map interests and concerns in local disputes. Educators might use it to strengthen negotiation and conflict literacy. Public institutions might use it to improve access to appropriate dispute resolution pathways.

In each of these cases, the immediate benefit accrues to a specific user or group of users. The broader benefit, however, is social. When individuals can access better conflict guidance earlier, disputes may be resolved before they become destructive. When organizations understand internal tensions more clearly, they may prevent costly breakdowns. When communities are better able to surface excluded voices, decisions may gain legitimacy. When public institutions help people navigate conflict more effectively, trust in those institutions may improve.

This is why the public-good logic is so important. The aim is not simply to give one party a better instrument for winning a dispute. The aim is to strengthen society’s general capacity to manage conflict constructively. In this sense, AI-based conflict management infrastructure should be evaluated not only by its technical performance, but also by its contribution to access, fairness, de-escalation, learning, and institutional trust.

A realistic but demanding agenda

It would be naive to assume that AI can solve the political, social, and moral challenges of conflict. Peace depends on interests, institutions, leadership, legitimacy, empathy, courage, and often difficult compromises. No algorithm can substitute for these conditions. At the same time, it would also be a mistake to ignore the potential of AI to improve the informational and analytical environment in which conflict-related decisions are made.

AI is already used in conflict management and its role is likely to grow. The more important question is whether this development will be shaped mainly by private markets, state interests, and isolated experiments, or whether we will build systems that serve a broader public purpose.

Our argument is that AI-based conflict management should be developed as public-interest infrastructure with public-good characteristics. It should be accessible, trustworthy, neutral, transparent, and governed in ways that prevent misuse. It should support human judgment rather than replace it. It should help people understand conflicts more clearly, recognize the costs of escalation, identify constructive options, and reach appropriate support earlier.

If designed in this way, AI will not create peace on its own, but it may help create better conditions for peace by expanding access to conflict management capacity. In a world where destructive conflicts remain persistent and costly, that is a goal worth taking seriously.

Source: “Expert Perspectives on New Paths to Peace,” Negotiation and Conflict Management Research, including our contribution “From Tools to Public-Interest Infrastructure: Rethinking AI for Conflict Management.”