Assessing science communication effectiveness: Issues in evaluation and measurement

The practice and theory of evaluation is far too large a topic to comprehensively cover in a single book chapter. We present here a subset of that topic which considers evaluation in a science communication context. Between us, we draw on some 35 years' experience in social research and evaluation which has given us insights into what works and what often gets missed. We will touch on some fundamental elements of evaluation but focus more on offering critiques and pointers that we have picked up through our professional endeavours. We will also touch on the political nature of evaluation, particularly in science communication and in the evaluation of university research and education. With this in mind, this chapter moves from the general to the specific. We begin by presenting and critiquing some fundamental concepts and tools of evaluation before turning to specific challenges facing the evaluation of science and communication endeavours. Examples include a classic evaluation controversy that spawned the Public Understanding of Science (PUS) movement in the UK and a look at the difficulties faced by science centres in determining the effect they may be having on their visitors. As an example of a sector-wide evaluation challenge, we look at an issue of broad concern to science communication because it affects science research in general: the global ranking of universities. At the end of the chapter, we have included a more detailed, but not exhaustive, example of a hypothetical evaluation of a science communication enterprise.