Negotiation is when everyone sits down and talks to see how they can reach a consensus and solve problems together. It can be small talk between friends, or it can be a diplomatic occasion between countries.
But negotiation is not simple. People tend to ignore the good advice of others with prejudice and emotions, resulting in unsatisfactory results. Moreover, negotiation is also a technical job, and not everyone can easily do it.
In order to facilitate the human negotiation process, the researchers proposed:Intelligent negotiation agent, capable of assisting humans in multiple rounds of interaction and even negotiating directly with humans. A typical negotiation conversation involves multiple rounds of interaction between an agent and a human, exchanging transaction information with each other, and ultimately accepting or rejecting the transaction, as shown in the diagram below
Now having a negotiator is like hiring a personal advisor who can help you with everything from grocery shopping to complex political or legal matters.
In the future, if you quarrel, you will take the negotiating agent to go, and the stupid person will be saved: "I won't say a word until my agent comes." ”
Maybe there are some friends, rightNegotiating agentI'm not very familiar with it, and if I want to know more about it, I'll bring you an article about it todayA systematic review of negotiation dialogue, including dimensions such as datasets, evaluation metrics, and modeling methods. Let's take a look
Title
let’s negotiate! a survey of negotiation dialogue systems
Statement: This issue ** interpretation is written by non-humans, and the full text is independently completed by the Cyber Malean AI** interpretation expert agent, and is released after manual review and illustrations.
** Xi Xiaoyao Technology said "Background reply".Intelligent internal testing"Get the invitation link for the internal beta test!
Negotiators' preferences and strategies work together to shape the potential outcome of the negotiation and the course of interaction, as shown in the diagram below. of both sidesPreferencesCollectively constitute the scope of possible agreements, while negotiatingStrategy, as a goal-oriented behavior, ultimately determines the quantity and quality of potential outcomes that can be achieved by the negotiator's preference while influencing the interaction.
Put simply, preferences set the range of possible outcomes, and the strategy within that range determines the actual outcomes.
Negotiation, although it is ubiquitous in everyday life, such as bargaining, is still a challenging task. Without professional training, many people often lack the negotiation skills to achieve their desired goals, and do not know what strategies should be used and how to implement them, so human negotiation often has the following difficulties:
In negotiations,Accurately identify and process implicit information about the interests and preferences of other negotiatorsIt's a difficult thing to do. Many times, negotiations are seen as a form of competition, which can lead to a lack of motivation for people to seek or express this information. Cognitive heuristics, biases, and emotions in humansIt can also be an obstacle in the negotiations. For example, people often look overly optimistic about themselves, the world, and the future, which can lead to overestimation and overoptimism in negotiations. Negotiations may also lead to:Participants are emotional, thus making it more difficult to process information rationally. Therefore, it is crucial to develop an effective negotiation agent to help humans better understand and control these different factors and optimize the outcome of the negotiation.
The research on the existing negotiation dialogue system is divided into three major sections, as shown in the figure below:
Negotiator Modeling: The purpose is to infer explicit information policy modeling about other negotiators based on the context of the conversation: learn to choose the strategy to use in the context of the current conversation. Action learning: Incorporate the above negotiation information into a machine learning framework to develop a conversational model that translates strategies into observable actions or responses. 
Negotiation strategy modeling focuses on the various strategies used in the negotiation process. These strategies can be integrative, aiming to achieve a common good among participants, or they can be distributive, i.e., aiming to win the maximum personal benefit.
Integrated strategies (also known as "win-win" strategy modeling) aim to achieve mutual benefits among participants. [1] A Latent Action Reinforcement Learning (LARL) framework is proposed for dialogical strategy modeling, but due to the lack of explicit policy labels, only implicit policies can be analyzed. [2,3] Explicit strategies such as "eliciting preferences", "collaboration", and "empathy" were then defined, and user preferences were captured through hierarchical neural models. [4] A collaborative strategy set is proposed for negotiation of workload and salary in interviews to achieve agreement between employers and employees.
Distributive strategies focus on maximizing personal benefits and are employed when a person stands their ground or resists a counterparty's trades.
5] A set of persuasion methods with 10 strategies is presented to promote others to donate to charity, involving logical and emotional attraction, among other things. [6,7] Further exploration of structures (e.g., faced behaviors, emotions). At the same time, [8] four adversarial attack strategies were studied, including competition, empowerment, biased processing, and avoidance, each of which contained specific strategic behaviors, such as attacking *** or reinforcing personal preferences to negate views.
In multi-party contexts, strategy modeling needs to take into account the different attitudes and complex relationships between individual participants, the entire group, and subgroups. [9] An attempt was made to model multi-party negotiation using a multi-agent reinforcement learning framework. [10] then use the discourse dependency tree ** multi-party relationship dependency. [11] Graph neural networks reveal the relationships between multiple parties. However, multi-party strategy research is limited by the lack of relevant datasets and benchmarks.
Negotiator modeling aims to infer explicit information about other negotiators from the context of the conversation. This includes modeling negotiators' preferences, emotions, and opponents' behavior.
Preference estimation helps agents infer their opponent's intentions and guess how their own words will affect their opponent's preferences. [12] A frequency-based heuristic is proposed to estimate negotiator preference, but the challenge of preference modeling is that it requires a complete dialogue. [13] A rule-based system is adopted to identify user preferences by analyzing linguistic features in some conversations. [3] Preference estimation is regarded as a ranking task, and a transformer-based model is proposed, which can be directly trained in part of the dialogue, and preference modeling in augmented reality applications is proposed.
Sentiment modeling refers to identifying the negotiator's mood or mood changes. [14] Emotional feelings and expressions in negotiation conversations were studied, and satisfaction with results and perception of partners were studied. [15] Emotional transitions were explicitly modeled, and patients were supported with pre-trained language models. [7] A dialogue behavior modeling method was proposed in persuasive discussions. [16] Leverage reinforcement learning frameworks to elicit emotions in persuasive messages.
Adversary behavior modeling refers to the detection and evaluation of the behavior of the adversary during the negotiation process. For example, fine-grained conversational behavior tags are available in the Craigslist dataset to track buyer and seller behavior. [17] A DQN-based modeling framework for adversary behavior is proposed to estimate the counteractions of adversaries. [18] Separate the modeling of opponent behavior from discourse generation to improve the accuracy of the negotiation system. [19] Based on the theory of mind, the negotiation system is improved, and a first-order model is proposed to calculate the expected value of mentality, and explicit and implicit conversational agent variants are provided.
Action learning enables the negotiation dialogue system to reasonably combine prior strategies and other negotiation information to generate high-quality responses. The researchers used a variety of strategic learning methods, including reinforcement learning, supervised learning, and contextual learning.
20] It pioneered the application of reinforcement learning technology to the negotiation and dialogue system. [21] The proposed OPPA uses systematic action to target agent behavior, rewarding structured outputs based on conversation estimation. [22] A modular framework combined with a language model is used to generate responses, and the strategy is evaluated through the reply detector and RL reward function, but the policy learning is separated from the response generation. [23] A comprehensive framework is proposed, which integrates in-depth Q-learning and multi-channel negotiation skills, so that agents can use parametric DQN to learn comprehensive negotiation strategies, and integrate language communication skills and bidding strategies.
24] Use the Seq2Seq model to maximize the likelihood of training data to learn actions. [18] An application supervision model was proposed to optimize the reward function of specific conversations, including ** utility, agent utility difference and number of utterances. [25] The negotiation strategy is first trained and then responses are generated that rely on the strategy, user utterance, and conversation context. [26] Combining the strategy graph network with the Seq2Seq model to create an interpretable policy learning paradigm. In addition, [27] a pre-trained BERT model was used to identify resistance strategies in persuasive negotiation. At the same time, [28] an end-to-end framework is proposed to integrate intent and semantic slot classification, response generation, and filtering tasks.
With GPT-3With the advent of large language models (LLMs) such as 5 and GPT-4, zero-shot and few-shot contextual learning techniques have been applied, and they have also been applied in negotiation dialogue tasks. [29] LLM was used for negotiation scenarios, while [30] for the "Werewolf" game. [31] A framework for assessing the strategic planning and execution capabilities of LLM agents is proposed. In these applications, the LLM acts as an agent to negotiate with other LLMs in a specific context to achieve a predetermined goal.
The negotiation dataset is the basis for the study of negotiation dialogue systems, and can be classified according to the type of negotiation, the scenario and the scale of the data. Sorted by when they were published, as shown in the table below, the negotiation type, context, number of conversations and corresponding average rounds, and participant attributes. This paper mainly introduces the comprehensive negotiation data and the allocation negotiation data.
The comprehensive negotiation dataset involves negotiations on multiple issues, and in order to achieve the best negotiation goals, the relevant participants should weigh multiple issues.
Strategy video games let players communicate verbally with each other to make deals in order to achieve tasks and objectives. For example, the STAC dataset is based on the Catan game, where players need to exchange resources to complete tasks, including timber, wheat, sheep, etc., in order to buy settlements, roads, and cities. Since each player only has access to their own resources, they have to interact with each other.
Item assignment scenarios involve a fixed set of items as well as predefined priorities for each player in a conversation. Since players only have access to their own priorities, they need to negotiate with each other to exchange their preferred items.
InitiativeTalking: Used between the owners of two restaurants. They discuss how to distribute the fruits (i.e., apples, bananas, and strawberries) and try to reach an agreement; dealornodeal: Two participants can only see their own collection of items and assign a value to each item, asking them to maximize their total score after the negotiation. Casino: Involves camping neighbors negotiating over extra food, water, and firewood packs, with each party having different priorities for different items. The JobInterview dataset includes interactions between recruiters and candidates about salary, leave, position, and location. Participants will be informed of each other's preferences and questions. During the negotiation process, feedback with the other party will be conveyed to the participants.
Distributive negotiation is a discussion that revolves around a fixed value (i.e., how the cake is distributed). In this type of negotiation, the participants usually discuss only one issue (e.g. item**), so there are few trade-offs between multiple issues in this type of negotiation.
The persuasionforgood dataset focuses on persuasion negotiation for charitable donations, where negotiators need to persuade the other party to make a donation. In the process of data annotation, the persuader is provided with some persuasion techniques and example sentences, while the persuaded person only tells them that the conversation is about charity. Annotators are expected to complete at least ten utterances in the conversation and are encouraged to agree at the end of the conversation.
The CraigslistBargain dataset is based on real-life commodity negotiation scenarios where buyers and sellers need to negotiate for a given item. NegoCoach is a similar benchmark, but with the addition of a negotiation coach that monitors messages between both labelers and recommends strategies to sellers in real-time to get better deals. The privacy of participants in negotiations is becoming increasingly important. The goals of the actors, such as attackers and defenders, are also counterproductive.
Anti-Scam is a benchmark that focuses on customer service. Users protect themselves by identifying if their adversaries are trying to steal sensitive personal information. Anti-SCAM provides an opportunity to study human seduction strategies in this scenario.
Through an in-depth analysis of the parties** and datasets of the negotiation dialogue system, it is possible to better understand the strategy choices in the negotiation process, the dynamics of negotiator behavior, and how to translate this information into effective dialogue behavior. These studies provide a theoretical basis and practical guidance for the development of intelligent agents that can assist humans in negotiations in various real-world scenarios.
The following table illustrates the various metrics used in the existing negotiation dialogue benchmarks:
1.Goal-oriented evaluation metrics
The goal-oriented evaluation index mainly focuses on assessing the ability of the negotiator to achieve the negotiation goal. These metrics are usually quantifiable::
Success Rate (SR): The most common metric used to evaluate how frequently an agent completes tasks within its objectives. Prediction Accuracy (PA), and Macro Average F1 Score: Evaluates the accuracy of the agent policy. Item Response Theory (IRT): Analyzes the effectiveness of influencing the audience. In addition, there are metrics implemented for language, such as naturalness, confusion (PPL), BLEU-2, rouge-L, and word embedding extreme matching scores.
2.Game-based evaluation metrics
Unlike goal-oriented metrics, game-based metrics provide a user-centric perspective to evaluate the system.
For example, in the game Catan, the researchers proposed a win rate and a win point average (**ps) to evaluate the success of humans and agents, respectively.
In the product negotiation task, specific task scores are used to test the agent's performance, including utility, fairness, and conversation length. In addition, there are metrics such as task completion rate and average sales vs. list ratio.
3.Human assessment
Human evaluation is used as a subjective evaluation method for agent performance to evaluate user satisfaction with the dialogue system. For example, use a user simulator as a salesperson to haggle with real customers and have users annotate customer satisfaction, purchase decisions, and the right response rate in conversations. In addition, there is the agreement rate, the Pareto optimality rate, and the human similarity (manually scoring whether the agent can act like a human using the Likert scale). ）
1.Multimodal negotiation dialogue
Existing research on negotiation dialogue rarely considers multimodal information, but humans often use a variety of modes including textual, audio, and visual information in negotiations. For example, a participant's facial expressions and emotions can be important cues in making a negotiation decision. Future research could consider incorporating these non-textual messages into the negotiation dialogue.
2.Multi-party negotiation dialogue
The existing benchmarks and methods of negotiation dialogue mainly focus on the negotiation between the two parties, which leads to the lack of in-depth research on multi-party negotiation dialogue. In the future, it is necessary to focus on collecting conversation data in multi-party negotiation scenarios, including general multi-party negotiation and team negotiation. Team negotiation is particularly special, involving people with different relationships and roles, and is common in large business transactions and highlights the critical importance of multi-party relationships.
3.Intercultural and multilingual negotiation dialogue
At present, the benchmark of negotiation dialogue is mainly focused on English, and there is insufficient exploration of other languages and cultures. In the context of globalization, dialogue involving different cultural backgrounds is becoming increasingly crucial. Therefore, there is an urgent need to build a multicultural and multilingual negotiation and dialogue system. Future work should integrate multilingual discourse and national social norms into the benchmarks of negotiation dialogue.
4.Negotiation conversations in real-world scenarios
Previous work has proposed a variety of benchmarks for negotiation dialogue, but most of them are based on human crowdsourcing, and participants play specific roles, which may not accurately reflect real-world negotiations, such as politics and business scenarios. Therefore, collecting real-world negotiation conversations, such as business meeting recordings or phone calls, may become a research direction worth exploring.
This paper systematically reviews the research progress of negotiation dialogue system, summarizes the research progress of negotiation dialogue system, and classifies and summarizes the data set, evaluation method and modeling method. It is hoped that this review will stimulate and promote research in the field of negotiation dialogue systems.
Statement: This issue ** interpretation is written by non-humans, and the full text is independently completed by the Cyber Malean AI** interpretation expert agent, and is released after manual review and illustrations.