Much has been written about measurement and evaluation of various forms of strategic communication including public relations. Despite the plethora of articles, manuals, guides, tools, and impassioned pleas from academics and industry leaders, measurement and evaluation remain under-used and largely focussed on measuring outputs rather than outcomes and impact (Gregory & Watson, 2008; Macnamara, 2015; Michaelson & Stacks, 2011).

Furthermore, most proposed models for measurement and evaluation narrowly focus on outcomes that achieve the goals and objectives of the organisation undertaking the communication. The impact on stakeholders and audiences is seldom considered other than in terms of how they reflect the desires and intentions of the organisation. This organisation-centric approach persists despite conceptualisation of PR and corporate communication as two-way, dialogic, and focussed on relationships.

Rather than repeat advice that is provided elsewhere, this short analysis seeks to advance discussion by introducing two approaches that generate greatly increased data for planning and evaluation as well as improved results of communication.

Lack of listening

Public relations and strategic communication management including corporate communication and government communication are theorised as two-way communication, dialogue, and focussed on engagement and relationships (J. Grunig, L. Grunig & Dozier, 2006; Johnston, 2014; Taylor & Kent, 2014).

However, in early 2016 I released the findings of a two-year, three-country study that showed that organisations listen “sporadically at best, poorly, and sometimes not at all” to their stakeholders and publics (Macnamara, 2016a, p. 236). The Organisational Listening Project (Macnamara, 2016a, 2016b), which is continuing to explore and test various organisational listening strategies, concluded that, under the auspices of ‘communication’, organisations predominantly implement an ‘architecture of speaking’ through advertising, PR, Web sites, events, social media, and other activities focussed on disseminating their messages. From 80 to 95 per cent of the communication resources of organisations were found to be devoted to disseminating organisations’ messages (i.e., speaking).

Creating an architecture of listening

The study called for government, corporate, and non-government organisations to counter-balance this by creating an architecture of listening in order to achieve their stated goals of two-way communication, dialogue, engagement, and relationships. Organisational listening requires more than adding a ‘listening tool’ or doing an occasional survey. My research proposed that the characteristics of an architecture of listening are:

1. A culture of listening;
2. Policies for listening;
3. Overcoming the politics of listening (e.g., the marginalisation of some groups);
4. Structures for listening;
5. Resources for listening (including human resources);
6. Skills for listening;
7. Technologies for listening;
8. Articulation of what is heard to organisation decision making and policy making (Macnamara, 2016a, 2016b).

The Organisational Listening Project may appear to be unrelated to measurement and evaluation of public communication. However, the two practices – listening and measurement undertaken for evaluation – are closely interconnected.

This is because all research involves and requires listening. Whether the method used for measurement and evaluation is surveys, focus groups, in-depth interviews, content analysis of media reporting and commentary, ethnography, experiments, random controlled trials, or statistical analysis of database records, measurement and evaluation implicitly if not explicitly have listening at their core. Measurement and evaluation involve not only the collection of data, but the processing, analysis, and interpretation of quantitative and qualitative data to try to understand audience awareness, attitudes, perceptions, concerns, interests, media preferences, intentions, and behaviour and changes in those variables.

Overcoming organisational centricity

However, the organisation-centric approach of many organisations and many practices in strategic communication mean that the listening undertaken as part of traditional research methods is often functional and instrumental. That is to say, organisations listen to what they want to listen to when they want to listen. Usually this is confined to help them target potential consumers and to measure the effectiveness of their own information and persuasion initiatives, rather than open listening to stakeholders and audiences.

The Organisational Listening Project sought to not only explore the level and effectiveness of listening by organisations but also to identify methods to improve listening to their stakeholders and publics, which often requires large-scale listening and listening across diverse interests. Attention to organisational listening is necessary because listening is much more than tokenistic attention and dialogue is not simply turn-taking at speaking. From psychology, political science, and communication literature, listening is defined as (1) recognising others who have relevant views or interests (not only ‘publics’ decided by the organization); (2) giving acknowledgement when others speak; (3) paying attention to what others say; (4) interpreting what they say to (5) gain understanding of their views and needs; (6) giving consideration to what they say; and (7) responding in some appropriate way, which is not necessarily agreement (Bickford, 1996; Husband, 1996, 2000, 2009; Honneth 2007; Lundsteen, 1979; Purdy & Borisoff, 1997).

Traditional methods for organisational listening

Traditional organisational listening methods include research, increasingly referred to as insights, involving quantitative methods such as surveys and qualitative methods such as focus groups and in-depth interviews, as well as public consultation, and social media monitoring. As well as identifying that these traditional methods can be significantly improved, The Organisational Listening Project identified a number of specialist methods for tuning in to audiences, gaining insights, and creating measurable outcomes.

A new approach – Sense Making Methodology

One of the advanced methods identified is sense making methodology (SMM). While sometimes narrowly understood as a method of making sense of data (i.e., analysis and interpretation), SMM is in fact a broad methodology for research, planning, and implementation that is focussed on creating genuine open dialogue between organisations and their publics in which the organisation listens actively in a spirit of mutuality and reciprocity. In short, it is a systematic approach for the design of dialogue beyond tokenistic efforts that often characterise government and corporate engagement. Leading advocate of this approach, Brenda Dervin, and her main co-author summarise the SMM approach as:

You listen to me, and I will listen to you; you learn from me, and I will learn from you; you trust my narratives about my material circumstances, and I may listen to the narratives within which you wrap your expertise. But you will also have to listen to my expertise. (Dervin & Foreman-Wernet, 2013, p. 159)

This points to one of the key principles that makes SSM fundamentally different to other top-down and expert led approaches to research and planning. SMM is based on an understanding and acceptance that “both organisations and constituencies have expertise to share, common struggles to ponder, and capacities to teach and learn from each other” (Dervin & Foreman-Wernet, 2013, p. 160) and, therefore, SMM “mandates refocussing communication attention on dialogue rather than transmission” of information (Dervin & Foreman-Wernet, 2013, p. 154). Dervin and her co-author state frankly: “in SMM public communication is defined as the means to not merely change constituencies but to change organisations” (p. 160).

As part of this, SMM goes further than facilitating dialogue with major stakeholders. As The Organisational Listening Project found, dialogue and consultation are often restricted to the “usual suspects” – major organisations such as business groups, trade unions, and mobilised lobby groups that represent economic and political elites (Macnamara, 2016, p. 180). SMM requires open listening and engagement of all parties potentially affected by an organisation, which often requires outreach rather than passive engagement such as waiting for submissions to consultations or relying on occasional surveys.

A further fundamental difference in the Sense Making Methodology approach is that it recognises that people who comprise audiences and publics move and change through time and space. Traditional approaches to mass communication and public communication tend to assume or “imagine” audiences as static groups identifiable by stable demographic and psychographic data (Anderson, 1991). SMM recognises that people are in a constant state of growth and change. Therefore, traditional audience segmentation works only for identifying habitual behaviours. SMM tries to understand the person in situation – in other words, the person at a particular point in time within the social, cultural, political, economic context in which they exist. Traditional market research that relies on quantitative statistics such as demographics uses blunt instruments to measure audience attitudes, perceptions, interests, and needs. SMM incorporates greater use of qualitative research to gain deep insights into audiences. This is important because other key principles of SMM are:

• Ultimately the intended audience controls all the gates, as sense making and meaning making of the strategic communication or organisations happens in the minds of audiences. Therefore, in communication everything begins and ends with the audience – not the organisation or channels;
• The everyday experiences of people are valid and valuable sources of information that are often dismissed in favour of ‘expert’ views and external scientific data. Ultimately, audiences know more about their lives than anyone else. So listen to them – often and intently. Dervin and Foreman-Wernet point out that in SMM informants in research are “theorists of their own worlds” (2013, p. 158);
• People (audiences) most trust and rely on their families, friends, and peers, so mass media campaigns will always have limited impact. SMM approaches favour agility, flexibility, multiple channels, participation, and collaboration including community-based engagement.

SMM approaches require well-meaning experts in marketing, research, and communication to “humble” their knowledge and subjugate their views and categorisations in favour of allowing audiences to speak on their own terms in their own contexts (Dervin & Foreman-Wernet, 2013, p. 160). SMM also is cautious towards consensus, recognising that consensus is often not possible and strategies therefore have to deal with diversity and difference.

A final explanation and warning comes from Dervin and Foreman-Wernet who say: “Communication is ultimately a quid pro quo. People are willing to listen to that which collides with or is new to their worlds when those communicating at them change to communicating with them” (2013, p. 153) [italics added].

Approaches such as Sense Making Methodology are rarely used in public relations and corporate communication, but offer much to create a listening organisation, a learning organisation, and an adaptive organisation. The organisation that does not listen, learn, and adapt inevitably faces fractious issues, crises, and potentially an uncertain future. Conversely, open listening informs strategy (formative research) and provides a constant stream of feedback (evaluation), and can build productive relationships and trust.

References

Anderson, B. (1991). Imagined communities: Reflections on the origin and spread of nationalism (Rev. Ed.). New York, NY and London, UK: Verso. (Original work published 1983)

Bickford, S. (1996). The dissonance of democracy: Listening, conflict and citizenship. Ithaca, NY and London: Cornell University Press.

Dervin, B., & Foreman-Wernet, L. (2013). Sense-making methodology as an approach to understanding and designing for campaign audiences. In R. Rice & C. Atkin (Eds.), Public communication campaigns (4th ed., pp. 147–162). Thousand Oaks, CA: Sage.

Gregory, A., & Watson, T. (2008). Defining the gap between research and practice in public relations programme evaluation: Towards a new research agenda. Journal of Marketing Communications, 24(5) 337–350.

Grunig, J., Grunig, L., & Dozier, D. (2006). The excellence theory. In C. Botan & V. Hazelton (Eds.), Public relations theory II (pp. 21–62). Mahwah, NJ: Lawrence Erlbaum.

Honneth, A. (2007). Disrespect. Cambridge, UK: Polity Press.

Husband, C. (1996). The right to be understood: Conceiving the multi-ethnic public sphere. Innovation: The European Journal of Social Sciences, 9(2), 205–215.

Husband, C. (2000). Media and the public sphere in multi-ethnic societies. In S. Cottle (Ed.), Ethnic minorities and the media (pp. 199–214). Buckingham, UK: Open University Press.

Husband, C. (2009). Commentary: Between listening and understanding. Continuum: Journal of Media & Cultural Studies, 23(4), 441–443.

Johnston, K. (2014). Public relations and engagement: Theoretical imperatives of a multidimensional concept. Journal of Public Relations Research, 26(5), 381–383.

Lundsteen, S. (1979). Listening: Its impact on language and the other language arts. Urbana, IL: ERIC Clearing House on Reading and Communication Skills.

Macnamara, J. (2015). Overcoming the measurement and evaluation deadlock: A new approach and model. Journal of Communication Management, 19(4), 371–387.

Macnamara, J. (2016a). Organizational listening: The missing essential in public communication. New York, NY: Peter Lang.

Macnamara J. (2016b). The work and ‘architecture of listening’: Addressing gaps in organization-public communication. International Journal of Strategic Communication, 10(2), 133–148.

Michaelson, D., & Stacks, D. (2011). Standardization in public relations measurement and evaluation. Public Relations Journal, 5(2), 1–22.

Purdy, M., & Borisoff, D. (1997). Listening in everyday life: A personal and professional approach (2nd ed.). Lanham, MA: University of America Press.

Taylor, M., & Kent, M. (2014). Dialogic engagement: Clarifying foundational concepts. Journal of Public Relations Research, 26(5), 384–398.

Who?

Jim Macnamara PhD, FAMEC, FAMI, CPM, FPRIA is Professor of Public Communication at the University of Technology Sydney, a role he took up in 2007 following a distinguished 30-year professional career spanning journalism, public relations, and media research. Jim is the author of 16 academic and professional books and more than 200 academic and professional journal articles. He is Chair of the Academic Advisory Group to the International Association for Measurement and Evaluation (AMEC) and a Fellow of AMEC and has advised a number of organisation on evaluation including the UK Government and the state government of New South Wales in Australia.