About the Collaboration Health Assessment Tool

What is CHAT?

The Centre for Social Impact (CSI) and Collaboration for Impact (CFI) have developed an online diagnostic tool - Collaboration for Health Assessment Tool (CHAT) - for organisations to assess the ‘health’ of their collaborative relationships.

It is an evidence-based tool that is interactive and user-friendly and can help collaborators understand how well they work together now, and in the future as their collaboration matures.

Due to their complex nature, social problems are often referred to as ‘wicked’ – i.e. they have multiple roots causes and are hard to define. At CSI and CFI, we believe that solving these problems requires collaboration across organisations and sectors. Yet, organisations have no easy way of assessing the health of their collaborative practices, and consequently, don’t really know if the collaboration is going well or not.

Ultimately, this tool aims to facilitate collaboration so that people can work together more effectively towards a shared goal. Over time, we hope to find out which social issues are being covered by collaborating initiatives, and which issues are missing out. Another aim is to develop benchmarks so that collaborations can assess their progress, and ultimately, achieve social impact.

Our vision

CSI and CFI want this tool to facilitate collaboration so that people can work together more effectively towards a shared goal. Healthier collaborations will mean that Australia’s most wicked problems are addressed more efficiently and effectively than in the past. However, we know little about how effective our collaborations are, across different social issue groups and internationally.

We will use deidentified data from CHAT to track the health of Australia’s collaborative initiatives over time. This will help us to identify the key factors that lead to successful collaboration in the social purpose sector and under which conditions. These data will also be used to identify which conditions present challenges for collaborations and provide insights into how these may be overcome.

We also want to use the CHAT data to create benchmarks. This will allow collaborations to gauge their progress against other collaborations at a similar stage. Benchmarks will help collaborators put their CHAT scores in context and help guide their future activities. Benchmarked data will be integrated into the CHAT tool over time, becoming more robust and reliable as more people use the tool.

Finally, we want to know which of our most important social issues are and are not being addressed through collaborative approaches. We will use this information to lobby for greater resourcing in areas that need it the most.

We will also be looking to enhance the usability of CHAT over time. For example, users will be redirected to relevant pages within the Collaboration for Impact website for further guidance and support and will have the opportunity to leave feedback on the functionality of CHAT and any potential content gaps. In the future, collaborations may also have the opportunity to download aggregated results for storage in other software.

Why do we need CHAT?

Solving social problems often requires resources “beyond the capacity of any one agency or jurisdiction” [1], and therefore, collaboration across organisations and sectors [2, 3]. Collaboration has been recognised in both academic and practitioner circles as a way to tackle complex social problems and create systemic change [4, 5]. Although many authors have attempted to define, and sometimes measure, collaboration [For example, 6, 7-11], much ambiguity remains around what it means to collaborate [3].The lack of consensus around the definition of collaboration has affected the development of a comprehensive tool for organisation to ‘health check’ their collaboration.

While there is a myriad of different collaborative assessment tools, none of them fully capture the meaning of collaboration or how it works. In fact, current contributions tend to conceptualise collaboration only taking into account organisational characteristics, thus failing to recognise that collaboration simultaneously occurs, and is affected by factors at the individual, organisational, and environmental level [12, 13]. Organisations therefore have no easy way of assessing the health of their collaborative practices, and consequently, know if the collaboration is going well or not. Without this information, it is also difficult for collaborations to grow and learn from the challenges and capitalise on their strengths. Put another way, a simple tool that can track a collaboration’s health, provides both a point-in-time snapshot and guidance for the future.

If you want to read more about collaboration and working together to achieve social purpose you can also read the The Travel Companion.

Why use CHAT?

CHAT provides collaborating initiatives with the applied tools they need to meet their goals. Users will receive:

  • A score for each of the eight dimensions of collaboration and an overall score[1]. Scores are based on a traffic light system and can be produced for the whole collaboration as well as for individual groups working within it[2].
  • Feedback tailored to your CHAT score. CHAT will generate broad guidelines on how specific elements of the collaboration could be enhanced.
  • A printable report. A collaboration’s results and feedback can be printed as a report for easy dissemination to your stakeholders.
  • The capacity to track your progress over time. Using the collaboration’s unique key (automatically provided upon registration) will allow collaborations to use CHAT now and in the future.

How to use CHAT?

CHAT is a simple online tool for collaborations to measure the health of their collaboration. It requires someone from the collaboration to lead the use of the tool. The CHAT lead for a collaboration will be able to:

  1. Set launch and closing dates for completing CHAT
  2. Identify the groups/teams/individuals working in their collaboration
  3. Identify the goals for the collaboration
  4. Enter the email addresses of their collaborators
  5. Compose an invitation message and send

Once collaborators receive the invitation they are prompted to set up an account and to take the 10 minute-survey. The survey includes 28 questions on collaboration and background questions on the collaboration’s history, its members and goals.

How the tool was developed

Our conceptualisation of collaboration was based on a review of relevant definitions in academic and other literature, as well as an in-depth review of 38 academic papers/reports to identify the different dimensions of collaboration.

Drawing on this review, we define collaboration as both a process and a structure. These two dimensions of collaboration can also be defined as a relationship – with particular traits – operating within a given arrangement; that is, through working together, organisations construct a structure that governs how the process of joint work will progress over time.

  • Structure refers to the ‘rules’ of the collaboration, i.e. what the collaboration will be doing (shared goal; systems change as a purpose); the resources contributed by the various parties involved (shared resources); and the way in which these resources will be expended (shared accountability, shared authority).
  • Process, is about the way we do our work, for example how decision making occurs. In other words, process is about how the collaboration is actually implemented.

The review of collaboration identified 35 dimensions of collaboration. Through expert consultation with Collaboration for Impact (CFI), these dimensions were consolidated into eight components – four structural components and four process components – outlined in Table 1. Each component comprises of sub-components covering the various aspects of each dimension.

Table 1: Dimensions of collaboration

DIMENSIONS

SUB-DIMENSIONS

Structure

Shared goal:

The overarching objective being pursued by the collaboration as a whole, over and above partners’ individual agendas [14].

  • Shared aspiration
  • Shared understanding of challenge
  • Shared understanding of approach
 

Shared resources:

The extent that resources necessary to sustain the collaboration’s activities such as data, funding, and time are available and committed to the collaboration. What is deemed appropriate, depends on the goals of the collaboration.

  • Sufficient resources for coordinating infrastructure
  • Shared data
  • Financial support
  • Shared capabilities
  • Mutually beneficial
 

Shared authority:

The extent that collaborators participate in decision-making, their willingness to delegate decision-making power to others and their capacity to deliver on the decisions they make [15].

  • Participatory decision-making
  • Authority to commit
  • Shared power
 

Shared accountability:

The extent that collaborators understand their responsibilities and take ownership for the results.

  • Tracking progress and impact
  • Shared responsibility
  • Shared ownership of the final products or outcomes
  • Tracking collaboration’s health
 

Process

Whole-system engagement:

Whole system engagement goes beyond diversity of stakeholders. It also refers to the extent that those affected by the issue are participating in the collaboration and informing its activities.

  • Stakeholders/community as co-creators
  • Needs-based response
  • Diversity of stakeholders
 

Communication flows:

The existence and quality of information exchange processes with stakeholders within and external to the collaboration.

  • Dissemination of evaluation data
  • Adequate internal communication
  • Adequate external communication
  • Shared language
 

(Building) Adaptive capacity:

The collaboration’s ability to learn from its environment, as well as and strategies in place to enable learning and discover solutions though experience [14].

 

  • Commitment to seeking innovative approaches
  • Learning culture
 

Holding/authorising environment:

Refers to a setting that enables the collaboration to “work together in ways that lead to agreement. An effective holding environment helps maintain sufficient pressure on the group to accomplish real work without overwhelming participants with too much stress.” [14]

  • Generating support
  • Level of urgency
  • Safety
  • Trust
 

Subsequently, a further review was conducted to find existing measures of these components. Where gaps were identified, new measures were created. This resulted in a list of 135 measures, each mapped to a sub-component, which became our draft tool.

Through an extensive period of refining the tool, including testing with collaborations, the number of measures was decreased from 135 to 27 spanning the eight dimensions of collaboration. At each stage, the measures were reviewed collectively by the research team and CFI, to assess their suitability and relevance. CHAT measures were taken from the literature [10, 11, 14, 15] and adapted where necessary. Where there were gaps, measures were developed by the research team. Where necessary, further changes to the measures were made, including simplification of language.

The development of CHAT also comprised narrative and correlational analysis. The correlation analysis identified positive relationships between responses to the tool and the participants’ own self-ratings of their collaborations’ strength. This suggests that our measures were performing well as indicators of a collaboration’s health. The narrative component, included qualitative interviews with people who trialled the tool, which allowed us to triangulate the tool diagnostics and better understand whether the CHAT questions aligned with the data we aimed to collect.

This led us to a penultimate version of the tool which was piloted with a collaboration working to address educational attainment in NSW. The Collaborators used an early version of the CHAT to identify any technical issues and provided us with general feedback on the user experience, which was fed into the final version of the tool.

What will happen to my data?

The responses you provide for CHAT will be completely de-identified. This means that your responses will remain confidential. No-one in your collaboration will be able to identify your individual responses. Results will only be accessible to the collaboration if there are at least five responses to the survey tool. If collaboration sub-groups or teams are set up in the survey, responses for these will also only be accessible if there are at least five responses in a group. All data collected through CHAT will be stored using standard UNSW security and privacy protocols. On completion of the survey, you will also be asked to consent to your data being used by CSI and CFI for research purposes. This is entirely optional and will not impact on the CHAT diagnosis for your collaboration or your relationship with CSI and CFI. You can also withdraw your responses at any time by contacting the research team: chat@unsw.edu.au.

How can I get more information about CHAT?

We have set up a Frequently Asked Questions (FAQ) section and have produced three instructional videos. You are also welcome to send any questions to our dedicated email: chat@unsw.edu.au

References

1. Luke, J.S., Catalytic leadership. 1998, San Francisco: Jossey-Bass.

2. Williams, P., The competent boundary spanner. Public Administration, 2002. 80(1): p. 103-124.

3. O'Looney, J., Modeling collaboration and social services integration. Administration in Social Work, 1994. 18(1): p. 61-86.

4. O'Flynn, J., The cult of collaboration in public policy. The Australian Journal of Public Administration, 2009. 68(1): p. 112-116.

5. Bryson, J.M., B.C. Crosby, and M. Middleton Stone, The design and implementation of cross-sector collaborations: Propositions from the literature. Public Administration Review, 2006. 66(Supplement s1): p. 44-55.

6. Kagan, S.L., United we stand: Collaboration for child care and early education services. 1991, New York: Teachers College Press.

7. Keast, R. and M.P. Mandell, The collaborative push: moving beyond rhetoric and gaining evidence. Journal of Management and Governance, 2014. 18(1): p. 9-28.

8. Konrad, E.L., A multidimensional framework for conceptualizing human services integration initiatives. New Directions for Evaluation, 1996. 1996(69): p. 5-19.

9. Mandell, M.P. and T.A. Steelman, Understanding what can be accomplished through interorganizational innovations. Public Management Review, 2003. 5(2): p. 197-224.

10. Marek, L.I., D.-J.P. Brock, and J. Savla, Evaluating collaboration for effectiveness: Conceptualization and measurement. American Journal of Evaluation, 2014. 36(1): p. 67-85.

11. Thomson, A.M., J.L. Perry, and T.K. Miller, Conceptualizing and measuring collaboration. Journal of Public Administration Research and Theory, 2009. 19(1): p. 23-56.

12. Emery, F.E. and E.L. Trist, The causal texture of organizational environments. Human Relations, 1965. 18(1): p. 21-32.

13. Gray, B. and D.J. Wood, Collaborative alliances: Moving from practice to theory. Journal of Applied Behavioural Science, 1991. 27(1): p. 3-22.

14. Chrislip, D.D., Essential concepts of collaboration, in The Collaborative Leadership Fieldbook. 2002, Jossey-Bass: San Francisco.

15. Graham, J.R. and K. Barter, Collaboration: A social work practice method. Families in Society: The Journal of Contemporary Human Services, 1999. 80(1): p. 6-13.

16. Audit Commission, A fruitful partnership: Effective partnership working. 1998, Audit Commission: London.

17. Hardy, B., B. Hudson, and E. Waddington, Assessing strategic partnership: The partnership assessment tool. 2003, Office of the Deputy Prime Minister: London.


[1] The eight scores are calculated by averaging the responses corresponding to each of the eight dimensions. The overall score is an average of the eight dimension scores.

[2] To protect confidentiality of collaborators, there must be at least five responses submitted before collaboration scores are provided.

Sign In or Register