“Mediation is an empirical process, a science and art not fully understood by even the most masterful of its practitioners.” (Creo, 2012)
Summary
The purpose of this article is to review relevant literature on measuring success in mediations with the intent to offer recommendations for the improvement of mediation practice and to provide information to ADR program design professionals.
Ross (2000) gives three reasons for measuring success in mediation - (a) performance appraisal by employers of mediators, (b) disputants informally evaluate mediators to decide if they would utilize mediation, and (c) establish the effectiveness of mediation in relation to other dispute resolution mechanisms. Presumably, a fourth reason would be to provide mediators with valuable feedback on their practice.
The context of this review is family mediation. The literature reviewed also includes articles drawn from studies of civil mediation programs (McDermott & Obar, 2004; Wissler, 2004) and international conflict (Savun, 2008). Any recommendation needs to be realized in the context of the type of mediation offered, and therefore it is reasonable not to limit the review to articles about family mediation.
This article intends to explore how different authors think of success in mediation. Three articles bring up the topic of mediation styles in relation to success in mediation (Gross, 2013; McDermott & Obar, 2004; Report of the Task Force on Research on Mediation Techniques, 2017) and one article discusses mediator knowledge and skills (Herman, Hollett, Gale, & Foster, 2001). Four authors discuss outcomes as a measure of success (Creo, 2012; Emery, Sbarra, & Grover, 2005; Wissler, 2004; Tjersland, Gulbrandsen, & Haavind, 2015). Only Savun (2008) offers a definition of conflict describing conflict as a dispute about resource distribution, which highlights that other authors do not provide such a definition. Family mediation belongs to the category of ADR interventions that are based on interactionist conflict theories. Interactionist definitions describe three concurrent conditions for conflict; parties depend on each other, they pursue similar goals, and at least one party perceives another party as preventing them from accomplishing their goals.
The feminist critique of mediation discussed by Maxwell (1992) is an example of how success in mediation is defined by the way conflict and mediation are conceptualized. Maxwell comments that mediations take place within the cultural context of society, which is sexist and supports patriarchal goals. Consequently, mediators and parties bring the cultural context into the mediation process. Mediations are necessarily conducted in private, and there is no way to assess if a male partner or non-feminist mediator practices male dominance. As a result, disputes should be settled in the public domain so that the public can monitor the process and exert pressure on decision-makers. These concerns are still brought forward in a 2015 article by Maclean's Magazine referencing the restorative justice mediation process used in the case of the dentistry students at Dalhousie University (Kingston, 2015). In other words, a mediation process must respond to the issues raised by a feminist critique to be considered successful from a feminist perspective.
A second concern is the accessibility of data. Questions such as achieving data validity, deciding between quantitative and qualitative data analysis, identifying resources required to collect data, choosing points in time data is collected, determining clients' use of other dispute resolution mechanisms regarding the same issue, and requiring expertise for a robust analysis of the data collected direct choices in what measures to use. Also, choosing a measure such as "did the parties reach an agreement?" constrains how success is defined by the limitations of that measure, here a focus on outcome orientation. The limitations of one measure emphasize the need to collect other measures as well. To deal with these issues, a convenient representative measure could be used as a proxy by establishing statistical co-relationships of that measure to other measures. These relationships need to be tested periodically, however.
Mediator preferences and behaviors may be related to client and conflict characteristics (Report of the Task Force on Research on Mediation Techniques, 2017; McDermott & Obar, 2004); as well as mediator knowledge and skills (Herman, Hollett, Gale, & Foster, 2001) may be related to the type of dispute. As discussed below, these factors influence clients' level of satisfaction, a measure that multiple authors discuss.
What is success in mediation?
Gross (2013) points out that the agreement criteria for measuring success has both proponents and critics. Transformative practitioners do not push for agreements, and a significant number of facilitative mediators and some evaluative mediators reject the criterion. Gross highlights that many writers suggest client satisfaction as a measure, both in addition to the agreement criteria or as standalone criteria.
This literature review is organized into three distinct categories (a) mediator characteristics, (b) mediator behaviors as perceived by clients, and (c) mediation outcomes.
Mediator Characteristics
Some authors suggest that mediator behavior has an influence on the outcome of mediation. For example, the Report of the Task Force on Research on Mediation Techniques, American Bar Association (2017) links mediator behavior with mediation success measures. The form and depth of mediator knowledge and skills, in turn, determine the behaviors a mediator uses during the mediation process.
Herman et al. (2001) define knowledge as reflecting the understanding of a range of facts and information establishing the "why" or "when" of a skill. A skill defines "how" to perform an action. The authors note that while some knowledge and skill areas overlap others stand alone. The authors continue to provide a detailed definition of each knowledge and skill area (Table 1).
Intuitively, we know that mediation is a profession defined by specific knowledge and skill requirements of the mediator. Further, we know that the interventions a mediator uses are related to mediation outcomes such as the satisfaction level of clients or clients reaching an agreement. What is less clear, as the debate about mediation techniques indicates, is how mediator behaviors at different points in the mediation process combine with client characteristics and with conflict characteristics to produce mediation outcomes.
From an evaluative point of view, the challenge then is to define measures as indicators of a mediator's knowledge and skill level that in turn direct appropriate and effective use of mediator behaviours.
Benjamin (2006), discusses another component of mediator competency that could be described as personality traits of a mediator. In his article, he presents the idea of 'Systematic Intuition', the dynamic interaction of both analytical acumen and intuitive sensibility. He uses the term to describe the unique ability of mediators to be at the same time both disciplined, rational, organized, with a clear strategy and game plan in mind, while at the same time able to feel or intuitively sense what is personally happening for the clients and the shifting dynamics between the parties. Analytical acumen is for Benjamin the ability to assess, analyze and calculate what is required to effectively manage a conflict and design a structure to approach the dispute, and to learn the techniques and skills required implement that strategy. It is the study of conflict from source to management. Intuitive sensibility is the ability to feel the undercurrents of the conflict, what is really going on as opposed to what is being said, and to connect with parties and quickly gain their trust. Systematic intuition needs to be supported by tenacity and optimism, the ability to purse resolution even in the most difficult circumstances and complicated matters.
Mediator Behaviors
Mediators presumably express their knowledge and skills as behaviors or interventions they believe will be useful in helping parties move towards resolution in a specific context. The process of selecting from the mediation skill toolbox the right tool for the job is influenced by preferences, sometimes also referred to as mediation style and the depth of understanding of how to use a particular tool in relation to client and conflict characteristics as well as the timing of the intervention during the mediation process. A mediator with a preference for an evaluative mediation style may understand the use of the intervention "caucusing" differently compared to a mediator with a preference for a facilitative style.
Creo (2012) suggests that a failure to achieve agreements is not necessarily an indication of a lack of success. Instead, he encourages mediators to focus on providing an excellent process. This point of view would be difficult to defend in an environment where outcomes matter because they are the measure of success.
The Report of the Task Force on Research on Mediation Techniques conducted by the American Bar Association (2017) compared mediator behaviors with outcomes. The authors found that generally (a) eliciting disputants' suggestions or solutions; (b) giving more attention to disputants' emotions, relationship and sources of conflict; (c) working to build trust and rapport, expressing empathy or praising the disputants and structuring the agenda; and (d) holding pre-mediation caucuses focused on establishing trust had a more positive effect on mediation outcomes.
The Report of the Task Force on Research on Mediation Techniques offers some specific findings of mediator behaviors that can be helpful for the design of an ADR program. A key assumption of the authors is that the client and conflict characteristics are similar. At a granular level mediation is a complex and multifaceted process. The value of the report is in the aggregation of findings; however, applying the findings to a specific mediation scenario's may not be practical.
Mediation Outcomes
Settlement and Agreements
Emery et al. (2005) found in their study that roughly a quarter of cases using adversarial processes settled outside of court, while 72% ended up in front of a judge, compared to 11% of cases that utilized mediation. Also, cases that did not settle in mediation likely settled outside of court.
Compliance with the mediated agreement is an issue Emery et al. think requires more research. They viewed return to mediation as a success, not a failure, and found that parents who have a mediated agreement in place are more likely to update the agreement compared to parents who have a court settlement.
McDermott & Obar (2004) suggest that charging parties (parties initiating a court action) are more likely to report a higher satisfaction level in facilitative mediations, whereas charging parties in evaluative mediations reported higher monetary payouts if legal representation was involved. Interestingly, evaluative mediations without legal representation resulted in lower settlement amounts compared to facilitative mediation. In other words, the combination of mediation style and client characteristics such as the presence of legal counsel influenced the type of outcome: receiving what the client wanted (satisfaction level) versus have the client's rights met (monetary settlement).
Ross (2000) states that the most common measure in mediation programs is the outcome of mediation: was an agreement reached? He emphasizes that agreements need to be integrative or win-win agreements, rather than agreements for the sake of getting an agreement. This suggests to Ross that agreements should be measured by how they fully satisfy each side's underlying concerns and interests by providing high payoffs. Ross also notes that parties must see the outcomes as fair and as desirable and they should feel that they have reached a satisfactory agreement without undue pressure from the mediator. Thus mediators must be aware of how a mediated agreement is received by the parties. Finally, Ross suggests that parties must implement an agreement. If one party does not implement an agreement or does not implement an agreement fully, both parties may be worse off as a result. According to Ross, research does indicate that mediated agreements are more likely to be implemented compared to resolutions reached in other forms of dispute resolution.
Tjersland, Gulbrandsen and Haavind (2015) analyzed a mandatory family mediation program with a particular emphasis on high conflict clients. They found that the clients' level of hope is positively related to outcome, that 60% of high conflict couples left mediation early after one session, that approximately 30% of high conflict couples achieved agreement compared to 80% of moderate or low conflict couples. They further note that feedback about the mediation process immediately after was good for all groups but that after 18 months, high conflict couples were more critical of the process and considered mediation a disadvantage. One-third of medium and low conflict couples thought that mediation did not make a difference and many parents stated that they had an agreement in place before mediation. Finally, high conflict couples continued to live in high conflict. The authors also noted that in a mandatory mediation context, parents might feel pressured to come to agreements and raised the question of what consequences that would have for the children.
All authors point out that the quality of the agreement as measured by compliance with the agreement is essential, whether clients receive what they wanted (satisfaction level) or alternatively whether the client's rights were met, whether agreements were integrative and whether the agreement was reached without undue pressure by the mediator. There is also an indication that a longitudinal study may offer important information as perceptions of clients change.
Client Satisfaction
Gross (2013) suggests thinking about the failure of reaching an agreement as a success indicator (if warranted) if all relevant information has been effectively exchanged. Sometimes the cost of reaching and implementing an agreement can exceed the cost of not resolving the conflict and end the relationship between the parties. The author also notes that there are situations where resolutions are necessary despite the costs involved. For example, a couple may realize that separating rather than maintaining the relationship is the only reasonable resolution (costs of maintaining a relationship is too high) and that they must also work out a co-parenting relationship for the benefit of their children (need to resolve conflict despite costs).
Wissler (2004) reviewed the empirical research and methodology of 10 studies on mediation in civil cases. Criteria for measuring success were mostly qualitative measures of the behaviour of the mediator such as maintaining neutrality, assisting disputants in understanding each other, and providing a safe environment. Wissler found that mediation and neutral evaluation settled cases and that participants viewed the process and outcome as fair. Litigants perceived mediation as more favourable, reducing the rate of non-compliance, and at least in settled cases mediation had a more positive effect on the parties' relationship.
McDermot & Obar (2004) found in their research that both the charging party and the respondent rate facilitative mediation more favourably than evaluative mediation.
Emery, Sbarra and Grover (2005) found that parties reported greater satisfaction with mediation both on the assumed strength of mediation ("your feelings were understood") and assumed strength of adversary settlement approaches ("your rights were protected"). The authors found that mothers reported high degrees of satisfaction in both adversarial settlement approaches and mediation, whereas fathers rated adversarial settlement approaches lower.
Issue Related Criteria and Relationship Factors
Ross (2000) suggests a decision tree model. If no agreement was reached, evaluators should consider issue related criteria as well as relationship factors. Issue related criteria include for example clarification of the issues, resolution of some, but, not all issues, concessions made, and overcoming heuristic errors (linking irrelevant decision-making criteria to the issue at hand). If parties make concessions on unresolved issues, then both sides are closer to resolving those issues than if they did not. Issue related criteria can also include that in many disputes each side has an initial position, a target or compromise position and a resistance point or limit. If the parties have respective limits that prevent them from achieving a resolution, than the mediator will be unable to help the parties achieve an agreement unless one party adjusts their resistance point or limit. Ross also notes that if the mediator can expand the range of options for the parties, i.e. unlock their respective anticipated settlement options, the mediator may still have performed a valuable service. Finally, Ross warns that repeat business indicating a dependency on the mediator can be a sign of failure, not a success.
Relationship Factors include if a mediator re-orients the relationship between the parties, hopefully preventing future conflicts from occurring, re-establishing rapport between them, reduce the level of hostility between the parties, dealing with power issues, and moving the parties from a competitive to a cooperative motivational orientation. It is self-evident that without establishing a mutually positive atmosphere between the parties, it is difficult to resolve the specific issues between the parties.
Emery et al. (2005) found that in the short-term family relationships and psychological adjustment of the children was not substantially different when comparing both adversarial processes with mediation; however, the long-term outcomes were much improved in mediation. The authors speculate that mediation is a forum for renegotiating relationships as well as negotiating agreements. Concretely, contact with both parents helped the children adjust to the separation.
A Holistic Model
The authors reviewed for this article assess success in mediation in two distinct ways: mediator behaviour and mediation outcome. Mediation outcome is further divided into settlement/agreement, client satisfaction and relationship factors, and issue related criteria. As discussed earlier, mediator behaviours are influenced by knowledge and skills.
The limitation of concentrating mediation success measures on outcomes is that outcomes may be dependent on context. McDermott and Obar (2004) reference Jeffrey Stemple in their work. Stemple sees the debate over facilitative versus evaluative mediation as a debate over suitability in a specific context. The authors further suggest that the suitability of a particular style may be linked to the strength of a case; in a weak case, facilitative mediation may be more suitable, whereas in a strong case, evaluative mediation may be more suitable. McDermott and Obar define suitability as a higher monetary settlement.
Sandu (2013) considers two aspects of mediation as indicators of success – the process and the outcome and gives four measures of success: fairness, efficiency, satisfaction and effectiveness. Fairness suggests an even-handed procedure and an equitable outcome. "Perceived fairness" however, can mean that indicators of fairness such as neutrality and impartiality are not relevant if parties do not perceive the process as fair. Efficiency focuses on the procedural and temporal dimension of the mediation process, e.g. the cost of mediation, resources devoted to it and the duration of the process. If parties are satisfied with the process or the outcome, they are more likely to perceive the mediation as being successful. In the author's view, satisfaction is an emotional response to the achievement of a goal. Effectiveness refers to achieved results, change or behavioural transformation.
Sandu sees mediation as a process of changing the behaviour and attitudes of the parties. What has changed after the mediator entered the process? An alternate view to measuring success in mediation is an analysis of agreements and outcome types and their impact on the conflict in question. Sandu differentiates between resolution and settlement, advising that settlement may be an indicator of success if a resolution cannot be achieved.
A potential model for measuring success in mediation must find a way of organizing measures into a cohesive and related framework. The proposed Holistic Model (Figure 1) relates "Mediator Knowledge, Skills and Characteristics" to "Mediator Behaviours" to "Outcomes (Client Satisfaction/Relationship Factors and Issue Related Criteria)". The model places this framework in the context of client and conflict characteristics. Client context and conflict characteristics may include the level of conflict, the presence of addiction and mental health concerns, history of domestic violence and safety concerns for the children, the presence of supports and legal representation, power balance issues and alternatives to mediation.
Measures of Success
The first step of designing approaches to measure success in mediation is to establish the purpose of the measure and how it will be used to influence practice, in other words, what does success mean in the context of the process. A mediator reflecting on their practice may wish to articulate a research question, for example, "Does caucusing make a difference in my practice?" and journal observations over time. An ADR program designer may want to demonstrate cost savings realized by mediation compared to the adversarial process and as part of that effort track how many clients utilize adversarial processes after commencing mediation. A manager of a mediation program may want to establish a link between mediator practice preferences and client or conflict characteristics and design questionnaires for mediators to complete. Measures of Success in Mediation (Table 2) lists potential areas to measure and measures discussed by the authors reviewed for this article.
A second step would be to identify quantitative or qualitative methods to analyze the data in a way that responds to the question or questions asked. Selecting the right method or methods is influenced in part by the capacity of the organization or the mediator to support the process. Evaluation requires a culture of intent, competency and resource commitment. An important consideration is to consider and "standardize" for client and conflict characteristics. The same intervention for a high conflict couple and a low conflict couple would produce different outcomes.
Finally, evaluating success is always an iterative process. As data is collected, analyzed, and conclusions are drawn new questions and ideas about what success in mediation means will emerge.
Table 2 Measures of Success
Category
Area of Measure
Suggested Measures
Knowledge and Skills
- Knowledge
- Skills
- Minimum requirements for training and experience
- Professional development requirements
- Regular assessments
Systematic Intuition and Tenacity
- Analytical acumen
- Intuitive sensibility
- Tenacity and optimism
- Professional development requirements
- Reflective practice and debriefing
- Personal care
Behaviours
- Eliciting disputants’ suggestions or solutions
- Giving attention to disputant’s emotions, relationship and sources of conflict
- Build trust and rapport, expressing empathy, praising disputants, structuring agenda
- Premediations to establish trust
- Provide safe environment
- Comediations
- Reflective practice and debriefing
- Supervision
- Role plays
Agreement
Quality of the Settlement/Agreement
Was agreement reached?
Measuring client’s perception of
- Implementation of agreement
- Compliance with agreement
- Facilitative approach by mediator
- Settlement amount
- Integrative / winwin agreement
- Meeting stated needs and interests
- Did parents have an agreement in place before entering mediation
- Amount of pressure applied by mediator to agree
Client Satisfaction
Establishing the level of satisfaction with the process at various intervals post-mediation and after some time has elapsed
Measuring clients’ perception of
- Mediator neutrality
- Change in understanding between clients
- Provide safe environment
- Facilitative approach
Issue Related Criteria
Changes in the understanding of the issues by the clients
Measuring clients’ perception of
- Clarification of issues
- Resolution of at least some issues
- Clients made concessions
- Clients overcame heuristic errors
- Concessions made
- Changes in initial position, compromise position and resistance point
Relationship Factors
- Changes in the relationship between the clients
- Adjustment of children to the separation
Measuring clients’ perception whether
- Relationship is reoriented
- Rapport between parties has increased
- Reduction of hostility by dealing with power issues
- Clients moved to a more cooperative atmosphere
- Children have meaningful contact with both parents
Works Cited
(2017). Report of the Task Force on Research on Mediation Techniques. American Bar Association, Section of Dispute Resolution;.
Benjamin, R. (2006, February). Character Traits of Working Dogs and Conflict Mediators: "Systematic Intuition' And Tenacity. Retrieved December 8, 2017, from medaite.com: https://www.mediate.com/articles/benjamin25.cfm
Creo, R. A. (2012). Failure & Me. Alternatives Vol. 30 No. 10, 187-188.
Emery, R. E., Sbarra, D., & Grover, T. (2005). Divorce Mediation: Research and Reflections. Family Court Review, Vol 43 No 1, 22-37.
Gross, A. E. (2013, June). Is Agreement the Gold Standard for Mediation Success? Retrieved September 18, 2017, from Mediate.com: www.mediate.com/articles/grossA3.cfm
Herman, M. S., Hollett, N., Gale, J., & Foster, M. (2001). Defining Mediator Knowledge and Skills. Negotiation Journal, 139-153.
Kingston, A. (2015, April 10). Rehtaeh Parsons, Dalhousie and the wait for justice in Nova Scotia. Maclean's.
Maxwell, N. G. (1992). The Feminist Dilemma in Mediation. International Review of Comparative Public Policy, Volume 4, no 1, 67-84.
McDermott, E. P., & Obar, R. (2004). "What is Going On" in Mediation: An Empirical Analysis of the Influence of a Mediator's Style on Party Satisfaction and Monetary Benefit. Vol 9, Harv. Negot. L. Rev. 75, 75-114.
Riskin, L. L. (1996). Understanding the Mediators' Orientations, Strategies, and Techniques: A Grid for the Perplexed. Harvard Negotiation Law Review [Vol1:7, 8 - 51.
Ross, W. (2000). Measuring Success in Mediation. The Mediation Journal, Vol. 1, Issue No. 1), 1-16.
Ruderman, M. N., Clerkin, C., & Connolly, C. (2014). Leadership Development Beyond Competencies: Moving to a Holistic Approach. Center for Creative Leadership.
Sandu, C. (2013). Mediation. Measuring the Success of Mediation. Conflict Studies Quarterly Issue 2, 30-39.
Savun, B. (2008). Information, Bias, and Mediation Success. International Studies Quarterly, 52, 25-47.
Tjersland, O., Gulbrandsen, W., & Haavind, H. (2015). Mandatory Mediation outside the Court: A Process and Effect Study. Conflict Resolution Quarterly, 1-16.
Wissler, R. L. (2004). The Effectiveness of Court-Connected Dispute Resolution in Civil Cases. Conflict Resolution Quarterly, vol 22, no 1-2, 55-88.