Gathering evidence in consultations was closely linked to achieving an inclusive balance of voices and participants, and having a variety of evidence sufficient to reflect this. It was also seen as essential to good decision-making. Key issues were:
Evidence for decision-making
As the basis for decision-making and recommendations, evidence had to be broad, encompassing a range of views and types, so that decisions could adequately balance a range of interests.
From our perspective, they're looking at getting a wide range of opinions on what a policy change would mean for different parts of the industry. Even if you just look at the creative industry, there are so many parts of that ecosystem, that see policy impact in very different ways. So, I think that what they're looking at is generally getting different industry views, different views from users, different views from NGOs, third parties, government, other regulators. [SH20]
I think the theory holds that taking the responses from stakeholders into account when developing policies, these policies will be better. [SH21]
An openness to different types of evidence, and new perspectives
Accessing and listening to new ideas and perspectives of a topic was key to developing more imagination around policy choices, which participants felt was necessary in a fast-moving industry, and could ‘shake up’ the system if necessary.
You'd like to think when they're consulting with stakeholders who are going to come up with really obvious things that they haven't spotted themselves. […] because what's the point in setting up some legislation if they've missed the main point or they've missed the whacking loophole that everybody is going to immediately jump into? [PU6]
You don't know everything, so you have to be a little bit aware of it, but on the other hand, you have to also keep your naivety to ask questions that are shaking, a little bit, the system more. Otherwise, you're just confirming the whole system more and that's what you see. In the process as it is now, it is people that know who can participate on a really efficient or effective basis, but they are people that are kind of consolidating the process because they don't step outside of the line of thinking. So, I think you have to have a mix of people that are competent in the matter but they have to be balanced with people that are, kind of, the dreamers that are saying, "We should shake up the whole process a little bit to rethink it." [SH22]
High-quality evidence
High quality evidence that could be tested and assessed robustly, was also important given that it was going to be used for policy decisions. However, some participants also valued individual stories, experiences and case studies.
So, the good contribution, or the ones that are actually much more valuable are ones that are not just based on hearsay or on anecdotal things, you know? The more evidence, the better. [SH1]
I suppose that, in terms of principles to be applied, I’d be pretty strongly on the side of you need an evidence base. That evidence base needs to be tested. If you are not working with rigorous testing and feasible evidence then you are unlikely to be doing a very good job. [SH29]
But where [case studies] are very helpful, I think, is if somebody has got a very personal, individual concern about a situation they've been through, this is where the individual responses can be incredibly valuable, because it does help policy makers. "I'm not really bothered about the overarching policy. I needed a licence for my village fete, and I couldn't get one. What are you going to do about it?" [SH9]
Narrow evidence
Some participants expressed concern about narrow definitions or understandings of evidence collected through consultations potentially limiting the scope of the consultation and excluding some voices.
[A consultation] is one way of, kind of, gathering data, but it’s only one way and it’s a way by which the industry is made responsible for gathering the data for the policymaker and presenting it in a form that’s prescribed by the policymaker in order to inform policy rather than, for example, the policymaker going out and conducting surveys or data-gathering through book research etc. [SH13]
What consultations should aim to achieve is that they should test emotion against evidence and weight of voice against minority arguments, which may, for one reason or another, merit more attention than their numerical weight puts in. [SH29]