Designing an Intelligent Agent that Negotiates ... - Semantic Scholar

Proceedings of the 42nd Hawaii International Conference on System Sciences - 2009

Designing an Intelligent Agent that Negotiates Tactfully with Human Counterparts: A Conceptual Analysis and Modeling Framework Yang Yinping Institute of High Performance Computing Republic of Singapore [email protected]

Abstract Automated negotiation has attracted growing interest within fields such as e-business, multi-agent systems, and web services. Nevertheless, a majority of automated negotiation research concentrates on tactics for pure agent-agent interaction, which may not be applicable in today’s e-business environment involving human decision makers. We argue that the design of negotiation agents should consider possible participation of human counterparts, thus requiring more sophisticated machine intelligence in strategic decision making. This paper provides a conceptual analysis of negotiation strategies and tactics from psychological and behavioral literature, and proposes a novel set of tactics for building collaborative agents. These tactics - multiple equivalent (simultaneous) offers, monotone decreasing concession, and strategic decision delay - are posited to enhance some aspects of negotiation outcomes without sacrificing others. Furthermore, we present a modeling framework that translates these tactics into computational rules for prototype development. The implications and future research opportunities are discussed.

1. Introduction With the growth of electronic marketplaces and the advancement of agent-based technologies, automated negotiation has received phenomenal attention. Researchers envision that negotiation agents could provide significant benefits including time-saving with lower opportunity costs, fewer negative effects and more efficient settlements than those in human negotiation, thus providing commercial value [34, 45]. It is useful to start by reviewing developments centered on the notion of automated negotiation, where computational agents find and prepare contracts on behalf of the real-world parties they represent [5]. Among other classification schemes [45, 46, 30, 2], we distinguish three basic types of computerized

Sharad Singhal Hewlett-Packard Laboratories CA, United States [email protected]

negotiations focusing on the involvement of software agents in dyadic relationships (we do not consider third parties, e.g., mediators or arbitrators in this paper). 1) Human-to-human negotiation with computer-based decision and communication support, where human negotiators interact with each other without surrendering control to autonomous agents. Many early theoretical and experimental developments of Negotiation Support Systems (NSSs) [e.g., 27, 10, 40] and their extension to web-based systems [e.g., 56, 28, 7] fall into this category. 2) Agent-to-agent negotiation, when a software agent acts on behalf of one party and negotiates with another agent acting for the counterpart. Later evolvement of NSSs and research in the field of multi-agent systems are associated with this category [e.g., 11, 12, 13, 57]. 3) Agent-to-human negotiation, when a software agent acts on behalf of one party and negotiates with a human counterpart. This stream aims to bridge the gap between how negotiation is performed in human and artificial societies. Existing research in this line examine persuasion techniques [19] and argumentation-based protocols [35, 25] that allow agents to accompany an offer with its underlying reasons. From a decision control point of view, the latter two types of negotiations are fundamentally different from the first. When one or both parties are fully represented by autonomous agent(s), it becomes crucial to ensure the agents capabilities in negotiating with the counterpart. In an open environment such as an electronic marketplace, a software agent acting on behalf of its principal may meet another software agent, or perhaps as often, a counterpart operated by a human. Therefore, it is useful to design an agent that embeds enough intelligence to negotiate with both human and agent counterparts. This paper therefore focuses on agent-to-human negotiations (Figure 1). This situation is frequently applicable to online commerce where an autonomous agent deals with a

978-0-7695-3450-3/09 $25.00 © 2009 IEEE

1


potential pool of buyers or clients on behalf of a seller or service provider. buyer or client

seller or service provider autonomous software agent

Figure 1. The agent-to-human negotiation Two key design issues arise in automated negotiation: the protocol and the strategy for the agents [30]. The protocol specifies the public rules of agents interaction, e.g., the sequence of offers and counteroffers that are allowed or the format of offers that agents can make. Both industrial standardization efforts [53, 14] and academic research attempt to address the protocol aspect of automated negotiation [20, 6, 24, 4, 54]. The strategy determines the way the agent behaves in an interaction as regulated by a protocol. In other words, the strategies are agents private decision-making models. As far as agent performance is concerned, strategy is regarded the centerpiece in automated negotiation research [12, 13, 41, 34, 57, 21, 18]. A common motivation for the study of strategies for negotiation agents is on relaxing the assumption of common knowledge [29] among the agents. Unlike classic game-theoretic approaches [33], the agents in the above-mentioned strategy-focused studies are not given any explicit knowledge about the preference structure of their counterpart agents (e.g., utility functions and reservation value), which is in line with the reality where preferences and bottom line are mostly private and confidential to individual negotiators. An earlier work by Faratin et al. [12] under this motivation formalize an agents decision functions under time-dependent, resource-dependent, and counterpart behavior-dependent families. For each negotiation issue, the value of next counter-offer is determined as a simple function of time, resource or counterpart behaviors. A multi-issue negotiation strategy in deciding the next counter-offer is subsequently modeled as weighted combination of each single-issue tactics [12, p. 166]. Follow-up studies [13, 41] extend this formalization and validate a variation of these tactics using simulations. Others examine guessing heuristics and learning mechanisms such that agents beliefs about the counterparts can be updated and adapted [34, 57, 21, 18]. Nevertheless, the tactics proposed in existing automated negotiation literature is mostly examined in the form of simulated agent-to-agent experiments where both agents tactics are assumed to be in a finite set, and some are only applicable in single-issue negotiations [12, 57]. There is, however, relatively

little work that examines more sophisticated agent tactics in dealing with human counterparts [e.g., 19]. As human negotiators typically experience negotiation as social interactions, the socio-psychological elements of agent-to-human encounters become prevalent. Moreover, human negotiators may employ non-trivial tactics and thus behave very differently from singlecriterion-ruled software agents. These realistic challenges motivate us to look into more comprehensive strategies and to build more intelligent negotiation agent. This paper represents an initial attempt to address the following question: can we design more intelligent negotiation agents by translating human wisdom on strategic negotiating skills into machine imitable intelligence? In the next section, key concepts underlying negotiation strategy are discussed. In section 3, we provide a conceptual analysis of strategic parameters and propose a set of tactics pertaining to integrative negotiation. Section 4 presents a modeling framework in translating the tactics into computational rules. Section 5 concludes the paper and discusses several directions for future research.

2. Theoretical foundations Negotiation is a decision-making process by which two or more parties with conflicting interests reach a compromise or agreement. In order to systematically model this process using machine intelligence, one may start by considering generic decision-making levels for a negotiator as a decision maker. According to Tien [48], four decision-making levels (operational, tactical, strategic, and systemic) can be mapped with their resources in terms of data, information, knowledge, and wisdom. Except for operational decisions, tactical and other higher-level decisions require information as processed data. Strategic decisions require knowledge as processed information, coupled with experiences, beliefs, values, cultures etc. Figure 2 presents an adaption of the framework [48].

Figure 2. The decision-making framework

2


Both strategies and tactics pertain to the possible actions that a negotiator performs to achieve his/her goals. Generally, strategies refer to long-term, highlevel plans to achieve a goal; while tactics are the short-term, adaptive moves designed to enact the strategies. In the context of negotiation, we differentiate the two terms and refer to strategy when we mean the higher-level plans according to goals and objectives programmed into the agent, and refer to tactics when talking about the concrete actions taken by the agent in order to pursue the strategy.

2.1. Meta strategies from the principled negotiation Traditionally, people often view bargaining and negotiation as a fixed-pie problem [15] or a win-lose competitive game. In a behavioral theory of bargaining, Walton and McKersie [51] distinguish between distributive and integrative negotiation situations. In single-issue bargaining, people often distribute a pie, and haggle with each other on a scarce resource such as sales price or annual salary measured in monetary terms. This is perhaps the extreme case of distributive negotiation, where the goals of one party are in direct conflict with the goals of the counterpart. Integrative negotiation, also known as soft or win-win negotiation, allows negotiation parties to expand the pie through problem-solving, creativity, and identification of differences in priorities and/or compatibility of interests. In essence, the theory [51] emphasizes avoidance of distributive behavior and advocates the integrative approach to achieve a winwin situation. Ever since the 1950s, integrative negotiation has been a goal of many theorists from diverse perspectives, including those from economics, organizational behavior, decision science, and social psychology. The Nash bargaining theory, a seminal work in game-theoretical approach to negotiation, mathematically proves the existence of the Nash Equilibrium that maximizes the joint gains of two negotiation parties without sacrificing the notion of fairness [32, 33]. Fisher and Ury [15] propose general principles for negotiation practitioners to structure the negotiation process (principled negotiations), such as separating the people from the problem, identifying and focusing on negotiators interests, generating options for mutual gain, and using objective criteria and data. Bazerman and Carroll [3] propose another stream of negotiation study from the perspective of the behavioral decision theory on human cognition in general. Raiffas prescriptive theories on the art and science of negotiation [38, 39] combine concepts from both descriptive and normative studies and

suggest step-by-step systematic guides for negotiators to realize collaborative negotiation. Together, these works advocate the use of rational, principled approaches to obtain mutual gains for both parties, which benefit their well-being in the long run. In real-world business, pure distributive or pure integrative negotiation situations are rare. Most negotiation tasks are mixed-motive, containing certain elements that require distributive bargaining methods and some that require integrative negotiation. Nevertheless, as long as conducting future business is important to both parties, the integrative element should be the theme for negotiation because competitive behaviors may drive people away from making deals in future encounters.

2.2. Strategies from the dual concern model In order to design autonomous agents that can conduct integrative negotiation, it is important to understand the rationale behind ones basic strategic choices. The dual concern model [37] views strategic choices for a negotiator as the product of two elements: concern for one's own substantive outcome (reflecting concern about self interests), and concern for the other side's outcome (reflecting concern about the relationship). Combining these two concerns results in four types of strategies for negotiators: avoidance, accommodation, competition, and collaboration. Figure 3 presents the dual concern model (adapted from [37]). Is substantive outcome important? No Yes Is relationship No outcome Yes important?

avoidance

competition

accommodation

collaboration

Figure 3. The dual concern model Avoidance is a non-engagement strategy where neither the substance of the negotiation, nor the relationship with the counterpart is considered important by the negotiator. This may happen in a situation when it is not worth the time and effort for a party to negotiate, or one has very strong/weak alternatives to engaging in current negotiation, or when a negotiator is in a trainers position [26]. With an avoidance strategy, the negotiator is likely to display a take-it-or-leave-it attitude towards the counterpart, and is typically unwilling to negotiate and make concessions. When the negotiator believes that the future relationship with the counterpart is more important than a substantive outcome, he/she may engage in an accommodation strategy in order to obtain possible reciprocal accommodation (tit-for-tat strategy, see Shakun [42] for a computational tit-for-tat

3


agent) from the counterpart in the future [17]. With accommodative strategy, the negotiator is likely to provide larger or more rapid concessions to the counterpart. However, accommodation may not be greeted with reciprocal return in many situations. A competition strategy, commonly portrayed in distributive or win-lose bargaining, is used where the primary goal of a negotiator is to maximize ones own outcome. In distributive bargaining, the interests of one party are usually in direct conflict with those of the counterpart. Negotiation parties are mostly competing in order to win some limited resource (often reflected in monetary terms, e.g., sales price of a used car or annual salary for an employment contract). A competitive strategy typically results in a strong and negative attitude during negotiation, where the negotiator is unwilling to give concessions. In contrast, a collaboration strategy is often associated with integrative or win-win negotiation where a negotiator attempts to maximize the sum of both parties outcomes. As parties may have different interests over the issues, value gained by one party is not necessarily at the expense of corresponding value lost by the other party. This strategy results in discussion and exploration of each others goals in an attempt to achieve win-win outcomes even though the conflicts may appear as win-lose situations at the initial stage of negotiation [51]. The literature has suggested at least five different methods to achieve integrative agreement, namely expanding the pie, logrolling, nonspecific compensation, cost cutting and bridging (see [26] for a summary). These methods, as meta-level strategies, essentially transform single-issue bargaining into multi-issue negotiation, which enable the materialization of integrative, win-win situations.

3. A conceptual analysis on collaborative negotiations tactics The dual concern model [37] suggests that a negotiating party which adopts a collaborative strategy needs to balance its concern for substantive outcome with concern for relationship outcome. It is therefore a non-trivial task to devise the actual moves in order to achieve its underlying high-level strategic objectives. A negotiation strategy can be typically manipulated by a number of parameters, such as the initial and final offers, and concession frequency and magnitude (see [55] for an example). This section provides a conceptual analysis on variation of strategic parameters to design tactics for enacting the collaborative negotiation strategy.

3.1. Initial offer & opening A very fundamental decision for any negotiator is to determine the magnitude of the initial offer, a question on whether the opening offer should be exaggerated or modest. An initial tough offer creates an anchoring point for subsequent negotiation. Many behavioral studies indicate that negotiators with exaggerated opening offer get higher settlements than those with modest or low initial offers [e.g., 52, 16]. In psychological terms, a tough offer is likely to invoke a positive successive contrast effect, which refers to the enhancement or diminishment of a weight or other measurement when compared with recently observed contrasting object. In the negotiation situation, a relatively tough initial offer makes subsequent offers more attractive to the counterpart. At the same time, an extremely tough initial offer may also have negative impact in negotiations, including a summary rejection by the other party, and a tough attitude being communicated to the other party, which may be harmful for long-term relationships [38]. Nevertheless, these negative effects of a tough initial offer in single-issue bargaining can be alleviated in multi-issue negotiations. When a decision issue is considered with other interdependent issues, there are higher chances that negotiating parties discover different preferences, or weights, over these issues. As a consequence, a high utility offer to one party may be a moderate (acceptable) agreement alternative to the other party. According to Pruitt [36] and Tutzauer [49], tough initial offer gives the negotiator room to learn his/her counterpart, and acts as a meta-message to the counterpart that more concessions are to be made before the agreement. A tough offer is hence a feasible tactic to be employed in multi-issue negotiation, balancing the needs of substantive outcome and portraying a negotiable attitude as a collaborative negotiator. Hence, we posit that Proposition 1a: In the initial offer or first counteroffer, negotiation agents starting from tough offer (associated with high self utility) are more likely to achieve better negotiation outcomes1 , compared with those starting from moderate or soft offer. 1 It is worth mentioning the existence of different perspectives to view and measure the outcomes of negotiations. A truly multi-dimensional measurement of negotiation outcomes synthesizes both economic (e.g., Pareto efficiency, Nash solution, individual/joint utilities, and contract balance) and socio-psychological outcomes (e.g., satisfaction, perceived collaborative atmosphere, perceived efficiency, and perceived fairness). For the purpose of highlevel propositions in this paper, better negotiation outcomes is referred to as enhancement of some aspects of negotiation outcomes without sacrificing others.

4


Besides the toughness of an offer, one can also consider the number of offers to be made at each negotiation round. This tactics involves presenting the other party with at least two (and preferably more) proposals of equal value to oneself [47, p.81]. From the perspective of ones substantial outcome, making equivalent offers maintains the amount of the overall utility a negotiator desires to achieve, before a concession is prematurely made. It allows generating offers that are of comparable utilities to the negotiator but may benefit his/her counterpart (see Faratin et al. [13] for a related proposal that employs similarity criterion to calculate an offer equivalent to ones previous offer). This approach can potentially increase the joint outcomes even when the counterparts preference structures are uncertain. While sequential-offer schemes dominate current agent-based negotiation research [e.g., 12, 13], there are several reasons to consider simultaneous offers to be supported by a negotiation protocol. First of all, by making equivalent simultaneous offers, one enhances the chances that one of the offers will appeal to ones counterpart, without a need to obtain prior information about preferences of the counterpart (as suggested by negotiation scholars in [31]). In autonomous environments where an agent can encounter a diversity of potential buyers, simultaneous offers can increase the likelihood of acceptance, instead of the hazard associated with subsequent offers. Furthermore, contrast effect can be invoked in making simultaneous offers. Simultaneous contrast effect entails a biased perception that the differences between two objects, when viewed together, are modulated. A neutral object may look more appealing when compared to another object. This effect is largely utilized in marketing and consumer behavior research (As Simonson and Tversky [43] illustrated: after the launch of a highpriced product ($429) in a retail shop, sales of a lessexpensive machine ($275) almost doubled. Conceivably, the $275 product seems like a bargain when the $429 one sits next to it). Based on these concepts, we posit that Proposition 1b: In an initial offer or first counteroffer, negotiation agents with multiple equivalent (simultaneous) offers are more likely to achieve better negotiation outcomes.

3.2. Concession making patterns When feedback on the initial offer or a counteroffer is received, whether and/or how a negotiator concedes from his/her initial offer is the next important decision. Concession making is an integral element of collaborative negotiations. Making concessions indicates an acknowledgement of the other party and a

movement toward the others position (concern for the relationship). On the other hand, a collaborative negotiator still has concerns for his/her own substantive outcome and cannot realistically make unlimited concessions. Therefore, it requires some art to handle both needs that are intrinsic to collaborative negotiators. The pattern of concession making combines the magnitude of concession at each negotiation rounds and amount of total planned concessions. Raiffa [38] suggests that skillful negotiators make the intervals between their decreasing offers become successively smaller, signaling that they are approaching a limit. The limit does not necessarily have to be to ones reservation price or BATNA (Best Alternative to a Negotiated Agreement); it could be just ones readjusted aspiration level. In effect, a negotiators concession behavior plays an important role as social cues that influence the process and outcomes of negotiation. As a simple illustration [26, pp. 94-95], a salesperson who concedes at a constant rate (the concession magnitude is $4, $4, $4, and $4 for a total concession of $16) is less likely to be believed by the counterpart when compared to another salesperson who concedes at a monotone decreasing rate ($4, $3, $2 and $1 for a total of $10). The counterpart is more inclined to feel that the latter salesperson has been truthful because her concession pattern appears that she has little room to offer (even though the former has conceded a higher total amount). In multi-issue negotiation situation, the pattern will be even less likely to be outguessed by the counterpart than in price bargaining situation. Therefore, we posit that Proposition 2: In concession-making, negotiation agents with monotone decreasing concession pattern are more likely to achieve better negotiation outcomes than those using a non-monotone decreasing pattern.

3.3. Final offer & acceptance In consideration of various constraints (e.g., time limit and resource limit), a realistic negotiator must find a way to convey the message that an offer is final with no further concessions possible. Experts and scholars recommend that a tactful negotiator should justify his/her stand by making the last concession substantial, rather than making a simple last offer statement without a change of concession pattern. The magnitude of the last concession, therefore, must be dramatically large (to imply that the negotiator is throwing in the remainder of his/her negotiation range) yet not extremely large (so that the other party would suspect that there is room for even further concessions) [38]. By declaring that the offer is final (showing firmness) and making a

5


relatively large concession (showing friendliness), one achieves the strategic consideration of both self and relationship concerns (see [15] on the communication of firm flexibility). This increases the likelihood that the offer is accepted, coupled with a convinced feeling experienced by the counterpart. We posit that Proposition 3a: In proposing a final offer, negotiation agents with larger concession magnitude are more likely to obtain an agreement with the counterpart, compared with those with a small final concession. Besides the magnitude or rates of the final concession, timing of concession is a relatively less studied parameter in negotiation agent design. Earlier wisdom suggests that negotiators should avoid the dual hazards of conceding too slowly or conceding too quickly [36]. A recent experimental psychology study compares the effects of concession timing on counterparts subjective judgment [23]. They concluded that immediate concession is the least successful strategy as it leads the counterpart to low satisfaction and evaluation of lower value/quality of the object (e.g., second-hand car or house). This effect is likely signified in important steps when a party decides to declare the final offer, or accept the counterparts offer. In the final stages, as a negotiator has demonstrated friendliness in earlier concession stages (concern to the relationship), a moderate delay in final offer signals the counterpart that one has made a careful decision to seal a deal (concern to substantive outcome). Compared with an immediate decision, a strategically delayed decision is likely to enhance the negotiation outcomes in socio-psychological terms. Proposition 3b: In proposing a final offer or accepting a counter-offer, negotiation agents with a strategic decision delay are more likely to achieve better negotiation outcomes than those with immediate decision response. Summarizing discussions in sections 2 and 3, we have conceptually analyzed the strategic parameters and suggest the following collaborative agent strategies and tactics for integrative negotiation. Meta Strategy: Decompose or unlink factors in a single-issue decision making task into multiple, interdependent issues Strategy: Use collaboration strategy as much as possible when both substantive and relationship outcomes are of concern Tactic 1: Make relatively tough, multiple equivalent (simultaneous) offers in initial offer or first counter-offer Tactic 2: Make monotone decreasing concession Tactic 3: Use decision delay and/or make a seemingly substantial amount of last concession in declaring final offer or accepting a counter-offer

4. A modeling framework This section discusses modeling techniques to realize the proposed tactics. Following a call to design useful artifacts [50], we adopt a modeling methodology that is straightforward to understand and easy to use in an agent-based negotiation system, but still captures the necessary complexity in the underlying problem.

4.1. Boundary and assumptions Before we present the computational rules for the proposed tactics, it is useful to define assumptions within the model. 1) In general, agents must agree on the negotiation protocol before negotiation proper begins [30]. This paper does not attempt to examine different protocols, but rather focuses on how decisionmaking rules are modeled in generally accepted protocols. That is, negotiating parties are to exchange multi-issue offer packages in alternate turns, until one party accepts the others offer. The agent is assumed to know the rules a priori. 2) Before agents are able to perform autonomous tasks, they must be programmed with knowledge of about their principals. In other words, the agent should have sufficient knowledge of the task in order to rightfully represent its principal, i.e., know the self. Such knowledge includes the exact issues to be negotiated, the preferences for each issue, and the minimum acceptable alternative (BATNA) for its principal (see [22, 39] for preference elicitation techniques). We assume that the agent has been preconfigured by a utility function that fully incorporates the preferences of its principal. 3) Apart from knowing the rules and the self, it is equally important to know the counterpart. Possible learning mechanisms have been introduced to negotiation agent design, including genetic algorithms [34], behavior-dependent tit-for-tat tactics [12], Bayesian learning [57], neural network learning algorithm [44], and kernel density estimation [9]. However, in agent-to-human interactions, a human counterpart may not be patient enough to make 20 or 30 counter-offers for the learning techniques to be effective. Our model concentrates on tactics that take into account the counterparts behavior but not on algorithms that attempt to estimate the counterparts real preferences.

4.2. Computational rules and pseudo code As argued in 4.1, for this discussion we are assuming that the agent has been programmed with a utility function that represents the interests of its

6


principal. We do not assume any specific form for the utility function; except that it can be computed from the values of the underlying issues (see Figure 4 for an example).

Figure 4. Two-issue utility graph We also assume that the agent has been programmed with an initial utility value as well as a sequence of concession patterns as basic strategic parameters. In general, we expect the concession made by the agent in its offers to become smaller with each round of negotiation, with a relatively large concession amount used in the agents final offer. Finally, we assume that the agent is programmed with parameters representing decision delay. Here we anticipate that the agent takes longer to respond to the counterpart as negotiation proceeds, providing the perception of requiring more thought. Figure 5 shows the pseudocode to build the agent. The following example illustrates the proposed framework. Lets consider a scenario in which an agent is being designed for a distributor of laptop computers to negotiate with its counterpart, a buyer performed by a software or human agent, in an e-marketplace. Suppose three issues are defined in this scenario: the unit price of a laptop (dollars/unit), the quantity of purchase (units), and the quality of after-sale support service (support levels). The issues of price, quantity, and support are weighted at 0.3, 0.5, and 0.2 respectively. Value options are assigned as: {1900, 2000, 2100, 2200, 2300, 2400, 2500} for price, {100, 200, 300, 400, 500} for quantity, and {classic, silver, gold, platinum} for support leading to 7*5*4=140 alternatives. The agent maintains a utility function U which can compute a utility score for each value of an issue (see [22, 39] for the additive scoring technique for preference elicitation). Based on the utility scores, the agent can rank the alternatives. In this example, we assume that utility scores are normalized in the range of [0, 100], the agent aspires to reach a utility of 60, and the utility for its BATNA is 35. Figure 6 presents a hypothetical seller vs. buyer utility graph, from a third partys viewpoint (note that neither seller nor buyer is assumed to have visibility of this graph). The points correspond to buyer-seller

utility scores for all the alternatives, which form the solution space for this negotiation scenario. Let I1, I2, … Im be the issues under negotiation. Let x be a vector with elements representing values of the issues under negotiation, i.,e., x = [v1, v2, … T vm] , where v1, v2, … vm are the values of Issues I1, I2, … Im respectively. Let u=U(x) be a utility function that computes the utility of the agent for issue values in x. Let udesired be the utility value the agent aspires to reach during negotiation. Let uk be the desired utility value the agent is attempting to reach in round k of the negotiation. Let Ok be the offer computed by the agent in round k, consisting of a set of N simultaneous offers { xj, | U(xj) ≥ uk, 1 ≤ j ≤ N }. Let Ck be the counter offer received in by agent in response to Ok. Let the utility of the counter offer be ck = U(Ck) = U(xj) | U(xj) ≥ U(xi) , ∀ xi, xj ∈ Ck, i ≠ j. Let tk be the decision delay for round k. Let dk be the concession the agent is prepared to make at the end of round k. Negotiation Steps Set done = false Set k = 1 while (! done and k < maximum rounds) do { Compute offer Ok Send offer Ok to opponent If(opponent accepts offer) then Set done = true } Receive counter offer Ck from opponent Compute counter offer utility ck = U(Ck) Compute interval tk and concession dk Compute utility for next offer uk+1 = uk - dk Wait interval tk If ck >= uk+1 || ck >= udesired then { Accept counter offer ok ∈ Ck | U(ok) = ck Set done = true } Set k = k+1 }

Figure 5. Pseudo-code for the agent After the seller agent is configured with preference structures, it is then able to start the negotiation proper following our three proposed tactics. First, the agent starts with a relatively high self-utility representing a tough offer (Proposition 1a), and proposes a set of simultaneous offers (e.g., 3 offers; Proposition 1b) which have utility values § 80 (see Figure 6). The agent then invites action from the buyer and decides upon its next step. As the three offers may be valued differently across buyers with different preference structures, the buyer is expected to respond differently.

7


If the buyer does not accept any of the agents offers, and instead proposes a counter-offer, the agent will accept if its utility at the opponents counter-offer ck exceeds its desired utility udesired = 60, otherwise it calculates the next set of offers that has utility uk+1 § (uk - dk). Note that dk is calculated as a product of a predefined monotone decreasing concession ratio (e.g., {40%, 30%, 20%, 10%} for round 1, 2, 3, and 4 respectively; Proposition 2) and the total planned concession amount (uinitial ubatna). At the same time, the agent can update ubatna if the buyer offers a value that is higher than its current ubatna. At each step, a decision delay (Proposition 3a) is calculated to accompany that offer. This iteration continues until one party accepts the counterparts offer, or the maximum rounds are exhausted. If the agent reaches the last planned concession round, it makes a final offer based on its ubatna (Proposition 3b) to the buyer.

Seller agent’s three equivalent offers 100

uinitial

Seller's Utility Score

90 80

udesired

70 60 50

ubatna

40 30

negotiation agents. The literature includes a large number of hardball tactics to beat the other party [e.g., 1, 8]. Many tactics, such as a strategic choice of physical negotiation location, are out of the realm of software agents; some rely on the cooperation of other players (e.g., good cop/bad cop tactics); while other tactics may be considered deceptive (see [26] for a summary of distributive tactics). In contrast, this paper advocates that the design of agent decision-making models should foster as much integrative bargaining as possible in a given situation, which is consistent with the motivation of NSSs [10, 27]. Developing agent decision models based on principled negotiation is valuable as such approaches work reasonably well even if one party knows that the other one is using it [15, 38]. The proposed win-winseeking agent presents more positive social and economical impact than purely self-centered agents designed in distributive mode. With the growth of e-commerce, an increasing number of business activities are going online. Many companies have linked their web sales channels to their supply chain (e.g., Dells direct online sale). Other companies use advanced market mechanisms for sales in existing e-marketplaces (e.g., auctions in eBay). The marketplace may utilize intelligent software agents in negotiating more complex transactions over the web. We regard agent-to-human negotiation as the most promising type to be immediately useful in todays Internet-enabled business.

20 10

5. Concluding remarks and future studies

0 0

10

20

30

40

50

60

70

80

90

100

Buyer's Utility Score

Figure 6. Seller vs. buyer utility graph In summary, the proposed agent framework encapsulates possible configurations for the agent to adopt the proposed tactics that potentially produce more integrative solutions. Instead of independently considering a single issue based on a single decision criterion (e.g., time, resource, and counterparts behavior), the proposed tactics allow the agent to consider all the three interdependent criteria at every decision step, striving to reach a mutual agreement without sacrificing each others substantive payoffs.

4.3. Discussion and implications While this paper draws upon the rich literature on human negotiation strategies and skills, it should be pointed out that not all popular ploys and tactics described in the literature are useful in the design of

This paper elucidates a set of theoretically sound and technically novel tactics to advance the design of intelligent negotiation agents. We translate the rich understanding of negotiation skills into a collection of useful propositions for designing agent decision models and present a modeling framework consisting of computational rules and illustrations. Our proposed approach can be considered a complement to existing efforts in the area of automated negotiation. Compared with other tactics proposed in earlier agent-based models that concentrate on single external forces (i.e., time, resource, counterparts offers and counteroffers, outside options), the tactics presented in this paper are rooted in the intrinsic (dual) concerns for a collaborative negotiator and emphasize the artful skills to balance various external constraints. Our key contribution lies in a bridge of knowledge from socio-psychological and behavioral concepts with computer-based modeling technologies. Furthermore, we remark on how the modeling framework of agent tactics can be pertinent to todays e-commerce context.

8


It is worth mentioning that the strategies presented do not guarantee optimal solutions or perfect equilibrium in game-theoretic sense [32, 33]. In real situations such as an e-commerce context, an entity, human or software agent, faces a rich spectrum of uncertainties and constraints, making it difficult to deduce the optimal decision functions. Rather, as argued in Faratin et al. [13], a realistic negotiation agent can be designed with heuristics-based knowledge in order to form a wise decision in each step. Probabilistically, informed agents with heuristics and adaptation methods are able to outperform those uninformed agents. Our propositions are in line with these rationales. The culmination of this research opens several avenues for future studies. First, based on the conceptual analysis and modeling framework, system prototypes can be readily developed, followed by empirical studies to validate the proposed benefits. Currently, we are implementing an agent engine that incorporates the proposed tactics, and at the same time designing a series of experiments involving both software agents and human subjects. The benefits of the proposed tactics for building negotiation agents will be tested via comparing agent performances in negotiating with other software agents as well as with human counterparts. Second, our conceptual analysis focuses on strategic parameters such as concession magnitude and timing. Future versions of the intelligent agent should extend to other influential techniques, such as argumentation and justification mechanisms associated with agents persuasive capability to negotiate with human counterparts. Furthermore, it is of interest to apply this agent framework to service-oriented context, especially in areas such as e-business where human sales agents may not be desirable from a cost perspective. Clearly, in order to have the best negotiation agent, the actual patterns of its decision model and behavior become important both from a socio-psychological perspective as well as from the perspective of the site attempting to maximize its profitability. We plan to explore some of these issues in our future work.

6. Acknowledgements We thank Dr. Terence Hung, programme manager at the Institute of High Performance Computing, Singapore, for his constructive feedbacks.

7. References [1] Aaronson, K., Selling on the Fast Track, Putnam, New York, 1989.

[2] Ayatollahzadeh Shirazi, Mohammad Reza, and Abdollahzadeh Barfouroush, Ahmad, A conceptual framework for modeling automated negotiations in multiagent systems, Negotiation, 24 (1), 2008, pp. 45-70. [3] Bazerman, M.H., and Carroll, J.S., Negotiator Cognition Research in Organizational Behavior, JAI Press, Greenwich, CT, 1987. [4] Bartolini, C., Preist, C., and Jennings, N.R., A software framework for automated negotiation, In F. Guunchiglia, J. Odell, G., Wei (Eds.), Architecting for Reuse: a Software Framework for Automated Negotiation in Agent Oriented Software Engineering III, Springer-Verlag LNCS, 2585/2003. [5] Beam, C., and Segev, A., Automated negotiations: a survey of the state of the art, Wirtschaftsinformatik, 39(3), 1997, pp. 263-268. [6] Bui, T.X., Gachet, A., and Rwth, H.S. Web Services for negotiation and bargaining in electronic markets: design requirements, proof-of-concepts, and potential applications to e-procurement", Group Decision and Negotiation, 15, 2006, pp. 469-490. [7] Chen, E., Kersten, G.E., and Vahidov, R. Agentsupported negotiations on e-marketplace, International Journal of Electronic Business, 3(1), 2005, pp. 28-50. [8] Cohen, H. You Can Negotiate Anything, Lyle Stuart, Secaucus, NJ, 1980. [9] Coehoorn, R.M., and Jennings, N.R., Learning an opponents preferences to make effective multi-issue negotiation trade-offs, In 6th International Conference on Electronic Commerce, ACM, 2004, pp.59-68. [10] Delaney, M.M., Foroughi, A., and Perkins, W.C., An empirical study of the efficacy of a computerized Negotiation Support System (NSS), Decision Support Systems, 20(3), 1997, pp. 185-197. [11] Espinasse, B., Picolet, G., and Chouraqui, E., Negotiation Support Systems: A multi-criteria and multiagent approach, European Journal of Operational Research, 103(2), 1997, pp. 389-409. [12] Faratin, P., Sierra, C., and Jennings, N.R., Negotiation decision functions for autonomous agents, Robotics and Autonomous Systems, 24(3), 1998, pp. 159-182. [13] Faratin, P., Sierra, C., and Jennings, N.R., Using similarity criteria to make negotiation trade-offs in automated negotiations, Artificial Intelligence, 142, 2002, pp. 205-237. [14] Foundation for Physical Agents (FIPA), Standard Status Specifications, http://www.fipa.org. [15] Fisher, R., and Ury, W., Getting to Yes: Negotiating Agreement without Giving In. Houghton Mifflin, Boston, 1981. [16] Hinton, B.L., Hamner, W.C., and Pohlan, N.F., Influence and award of magnitude, opening bid and concession rate on profit earned in a managerial negotiating game, Behavioral Science, 19, 1974., pp. 197-203. [17] Homans, G.C., Social Behavior: Its Elementary Forms, Harcourt, Brace & World Co, New York, 1961. [18] Huang, P., and Sycara, K., A computational model for online agent negotiation, In 35th Hawaii International Conference on System Sciences, Hawaii, U.S., 2002. [19] Huang, S., Lin, F., and Yuan, Y., Understanding agentbased on-line persuasion and bargaining strategies: an empirical study, International Journal of Electronic Commmerce, 11(1), 2006, pp. 85-115.

9


[20] Hung, P.C.K., Li, H., and Jeng, J-J., WS-negotiation: an overview of research issues", In 37th Hawaii International Conference on System Sciences, Hawaii, U.S., 2004. [21] Jonker, C.M., Robu, V., and Treur, J., An agent architecture for multi-attribute negotiation using incomplete preference information, Autonomous Agents and MultiAgent Systems, 15(2), 2007, pp. 221-252. [22] Keeney, R., and Raiffa, H., Structuring and analyzing values for multiple-issue negotiations, In H.P. Young (Ed.) Negotiation Analysis, University of Michigan Press, Ann Arbor, 1991, pp. 131-151. [23] Kwon, S., and Weingart, L.R., Unilateral concessions from the other party: concession behavior, attributions, and negotiation judgments, Journal of Applied Psychology, 2004, pp. 263-278. [24] Kersten, G.E., and Lai, H., Satisfiability and completeness of protocols for electronic negotiations, European Journal of Operational Research, 180(2), 2007, pp. 922-937. [25] Kraus, S.K., Sycara, K., and Evenchik, A., Reaching agreements through argumentation: a logical model and implementation, Artificial Intelligence, 104(1-2), 1998, pp. 1-69. [26] Lewicki, R.J., Saunders, D.M., and Minton, J.W., Negotiation, 3rd Ed, Irwin/McGraw-Hill, Boston, 1999. [27] Lim, L.H., and Benbasat, I.,A theoretical perspective of negotiation support systems, Journal of Management Information Systems, 9(3), 1993, pp. 27-44. [28] Lim, J. and Yang, Y.P. "Videoconferencing NSS and conflict level: an experimental study", In IFIP TC8 International Conference on Decision Support Systems, Prato, Italy, 2004. [29] Linhart, P.B., Radner, R. and Satterthwaite M.A., Bargaining with Incomplete Information, Academic Press, San Diego, 1992. [30] Lomuscio, A.R., Wooldridge, M., and Jennings, N.R., A classification scheme for negotiation in electronic commerce, Group Decision and Negotiation, 12, 2003, pp. 31-56. [31] Will your proposals hit the mark?, Negotiation, (Monthly Newsletter), Program on Negotiation at Harvard Law School, 11(5), May 2008. . [32] Nash, J., The bargaining problem, Econometrica, 18(2), 1950, pp. 155-162. [33] Nash, J., Two-person cooperative games, Econometrica, 21(1), 1953, pp. 128-140. [34] Oliver, J.R., A machine learning approach to automated negotiation and prospects for electronic commerce, Journal of Management Information Systems, 13(3), 1997, pp. 83-112. [35] Parsons, S., Sierra, C., and Jennings, N.R., Agents that reason and negotiate by arguing, Journal of Logic and Computation, 8(3), 1998, pp. 261-292. [36] Pruitt, D.G., Negotiation Behavior, Academic Press, New York, 1981. [37] Pruitt, D.G., and Rubin, J.Z., Social Conflict: Escalation, Impasse, and Resolution, Reading. Addison-Wesley, MA, 1986. [38] Raiffa, H., The Art and Science of Negotiations, Belknap/Harvard University Press, Cambridge, MA. 1982. [39] Raiffa, H., Richardson, J., and Metcalfe, D., Negotiation Analysis: The Science and Art of Collaborative Decision

Making, The Belknap Press of Harvard University Press, Cambridge, MA, 2002. [40] Rangaswamy, A., and Shell, G.R., Using computers to realize joint gains in negotiations: towards an electronic bargaining table, Management Science, 43(8), 1997, pp. 1147-1163. [41] Ros, R., and Sierra, C., A negotiation meta strategy combining trade-off and concession moves, Autonomous Agents and Multi-Agent Systems, 12(2), 2006, pp. 163-181. [42] Shakun, M.F., Multi-bilateral multi-issue e-negotiation in e-commerce with a tit-for-tat computer agent, Group Decision and Negotiation, 14, 2005, pp. 383-392. [43] Simonson, I., and Tversky, A., Choice in context: tradeoff contrast and extremeness aversion, Journal of Marketing Research, 29(3), 1992, pp. 281-295. [44] Soo, V.W., and Hung, C.A., On-Line incremental learning in bilateral multi-issue negotiation, In 1st International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2002, pp. 314-315. [45] Starke, K., and Rangaswamy, A., Computer-mediated negotiations: review and research opportunities, In A., Kent and J.G., Williams (Eds), Encyclopedia of Microcomputers (Vol. 26), Marcel Inc, New York, 2000. [46] Strobel, M., and Weinhardt, C., The Montreal taxonomy for electronic negotiations, Group Decision and Negotiation, 12(2), 2003, pp. 143-164. [47] Thompson, L. The mind and heart of the negotiation, 3rd Ed., Pearson Prentice Hall, New Jersey, 2005. [48] Tien, J.M., Towards a decision informatics paradigm: a real-time, information-based approach to decision making, IEEE Transactions on System, Man, and Cybernetics: Part C. 33(1), 2003, pp. 102 113. [49] Tutzauer, F., Bargaining outcome, bargaining process, and the role of communication, Progress in Communication Science, 10, 1991, pp. 254-270. [50] Walls, J., Widmeyer, G., and El Sawy, O., "Building an Information System design theory for vigilant EIS." Information Systems Research, 3(1), 1992, pp. 36 -59. [51] Walton, R.E., and McKersie, R.B., A Behavioral Theory of Labor Negotiations, McGraw Hill, New York, 1965. [52] Weingart, L.R., Thompson, L., Bazerman, M.H., and Carroll, J.S.. Tactical behaviors and negotiation outcomes, International Journal of Conflict Management, 1, 1990, pp. 7-31. [53] Web Services Agreement Specification (WSAgreement), http://www.ogf.org/documents/GFD.107.pdf, accessed on June, 2008. [54] Yan, J., Kowalczyk, R., Lin, J., Chhetri, M.B., Goh, S.K., and Zhang, J., Autonomous service level agreement negotiation for service composition provision, Future Generation Computer Systems, 23, 2007, pp. 748-759. [55] Yukl, G., Effects of the opponents initial offer, concession magnitude, and concession frequency on bargaining behavior, Journal of Personality and Social Psychology, 30(3), 1974, pp. 323-335. [56] Yuan, Y., Rose, J., Suarga, S., and Archer, N., A webbased Negotiation Support System, International Journal of Electronic Markets, 8(3), 1998, pp. 13-17. [57] Zeng, D, and Sycara, K., Bayesian learning in negotiation, International Journal of Human Computer Studies, 48(1), 1998, pp. 125-141.

10