Understanding the limits of professional perspectives is an important part of becoming a data-informed cultural organization.

Cultural professionals are not great at thinking like regular visitors.

“Wait. What?! I don’t like where this is going…”

This isn’t necessarily a bad thing – and skewed, insider perspectives are certainly not unique to the visitor-serving industry. After all, insider experts – and especially those who work for an organization – aren’t regular visitors. What is a bad thing is not remembering that insider professionals (all of us – myself included) often have skewed perspectives when we “I think” solutions to problems.

Check out the Fast Facts video that accompanies this article for a more snackable, sharable version of this information.

 

Why insider experts are not good at predicting visitor behavior

(…without unbiased data)

If you work for a cultural organization, then you probably know more than the average person about the organization’s content area. You might know your mission verbatim, the areas in which your organization is struggling, the goals it is trying to achieve, and you even know many of the awesome people working hard to achieve those goals.

But you know what visitors don’t generally know? All of those things. And it’s staff knowledge of those things that tend to create organizations’ biggest blind spots.

Simply put, having insider information about the organization or the industry already skews our perspectives. The things that we know or think that we know– and even the things that we hope – influence what we predict will happen, as well as what we believe is already happening in many situations.

Experts being limited and incorrectly guessing outcomes is not at all unique to the visitor-serving industry, and many business schools nowadays have entire lectures or course sections on human vs. statistical decision making. Check out this delightfully meta study by The University of Pennsylvania called, “Prediction of Consumer Behavior By Experts And Novices.” The study wanted to uncover if people with scientific knowledge of consumer behavior were better able to make predictions about phenomena in the field. The researchers asked academics, practitioners, and high school students…and they found that no group preformed better than chance.

I certainly do not intend to imply that experience and expertise are useless. Far from it! I’m simply pointing out that we industry experts (you and me) are human. Because we’re human, we bring a whole lot of misleading, pre-written, and not-fact-checked metaphorical paperwork to every table at which we sit that may influence the starting points and assumptions that we make about cultural organizations. That’s a big reason why audience insights are so important for cultural organizations. Listening to actual audiences provides clues about true perceptions, behaviors, and barriers to acting in an organization’s interests.

I have a colleague at IMPACTS who often makes predictions about which marketing campaigns will respond most favorably to market testing. He states his favorite of the bunch, and then jokes, “Now that I’ve identified my favorite, that is certain not to be the one that is most favorable to the market!” We all laugh, because it tends to be true: He’s usually hilariously incorrect – and he has probably seen more market testing outcomes than anyone else at the company!

Of course he is (at least sometimes) wrong! He knows the nuances of what the organization is trying to achieve and is coming from a different starting point than the general market. His predictions may be subconsciously based upon the things that he knows as an insider. This makes him phenomenal at working with cultural organizations because of his heightened awareness… but it also makes him delightfully (for the rest of us) abysmal at guessing the outcomes of market testing for ad campaigns.

(Thankfully, as a data company, we don’t rely on personal guessing. “Data is data.”)

Simply, the market doesn’t know everything that industry professionals know about an organization – how much hard work went into an initiative, the monetary investment, Wendy who works in the education department and is really banking on the program’s success, or that the Board Chair is quite certain that this particular project will serve as a magic bullet for an organization’s problems.

Are we doomed? No. We need only keep in mind that the market is the arbiter of an organization’s success, and remember that our insider knowledge affects how we see our work and our world. When we remember this, we can be more thoughtful, curious, and – best of all – we can make better decisions.

Overlooking the fact that market and audience research often lends itself to more effective outcomes than “expert guessing” leads to two (of potentially many) dangerous problems.

 

A) Confirmation bias

The first is a psychological phenomenon called confirmation bias. Confirmation bias is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s preexisting beliefs or hypotheses. It’s human – and also rather damaging.

Here’s just one example of confirmation bias at work within cultural organization employees. Before we dive in, I will warn you that it may be a hard one to take (…in the midst of an article discussing the fallibility of your and my expert opinion, which I’m sure already makes you feel like you cannot write me fan mail fast enough).

Here we go: Data suggest that broadly publicized reduced admission days result in attendees that are more educated and have higher household incomes than regular, full-price admission days. IMPACTS collected data from 48 cultural organizations that have a regularly scheduled, free admission day and we gathered data relating to these organizations on free these days and also on full admission days.

The average household income of those who visit cultural organizations on free days is $4,668 higher than the average household income on a paid admission day. Sound like an insignificant difference? Most organizations consider free days to be affordable access programs! The average level of education is also higher on free days than full admission days, and there are also more repeat visitors on free days. So no, free days generally do not encourage visitation from new audiences, either.

“This is certainly not true for my organization! Whenever I attend our free days, I see tons of low income individuals!”

First: What exactly do you mean that you “see tons of low income individuals?” I haven’t noticed that these folks wear signs!

Second: If you’re saying this, it may be confirmation bias at work. Confirmation bias – in this case – may make professionals think that their organization is exempt from this well-proven finding because they saw some people who looked “low income” on an affordable access day when they were looking for people who looked like this. When we believe that something is taking place, we often unknowingly seek out information that confirms our beliefs – never mind that a staff member may see even more people who fit their “low income-looking” bill on a paid admission day, when they are similarly seeking them.

(For those of you on the edge of your seats wondering how this is possible – let alone a general finding within the industry: Cultural audiences tend to be more wealthy and educated than the general public, so it’s wealthy folks who tend to take up the deal when free or reduced admission initiatives aren’t specifically targeted. Simply put, effectively getting the word out to low-income individuals and households is a completely different marketing strategy than those that organizations typically deploy – and it requires major additional marketing investment.)

Now you have a tip-off that may challenge your confirmation bias: Free days generally result in higher educational attainment and household income attendees. The challenge, now, may be to devise an unbiased plan to uncover the extent to which your organization may (or may not) truly be the exception to this rule.

This is but one example of confirmation bias at play. Really, it makes its mark in everything from how big of a barrier we perceive admission cost to be (it’s not as big a barrier as some imagine) to how we approach millennial engagement (millennial trends tend to be indicators of broader market trends). When we don’t know that confirmation bias is human and everywhere, we risk creating biased surveys and drawing false conclusions from mistaken starting points. We build our entire business on shaky, unstable ground.

 

B) Inside-out thinking

The second reason why misunderstanding the limits of insider perspectives holds organizations back is that it reinforces traditional inside-out operations.

Arguably, cultural organizations have traditionally functioned as “inside-out” organizations. That is, we make our decisions with the aid of insider professionals and content experts, and then push those decisions to the outside world. Our insider experts decide what to display, what’s important, what is appropriate for visitors to care about, and what should matter most to them. No doubt, it is our history of functioning as “inside-out” organizations that has contributed – at least in part – to the worrisome perception of cultural organizations as boring, unwelcoming, uppity, and incapable of evolution.

Operating as an “inside-out” organization comes from an assumption that we know more about our audiences then they know about themselves, and that we get to decide that matters to them. Perhaps it should be the other way around.

To be a data-informed organization means operating from the outside-in. It means understanding that we exist to serve human beings. Indeed, human beings are at the core of most organizations’ missions to educate and inspire. To be clear, operating from the outside-in does not mean dumbing down information or crowd-curating every decision. Instead, it means realizing that we can – and should – encourage audiences to play an important role in helping us help them.

When we rely too much on insider experts at the expense of consulting our actual audiences, we reinforce inside-out practices. We take a small hit at industry evolution.

 

In a world increasingly run by data, the cultural industry remains largely an “I think” industry. When we “I think” solutions, and take them to “I think” insider experts, we have a whole lot of thinking and guessing from people who aren’t our target audiences.

If we want to know what’s going on, then we have to ask our audiences. Or, at least we may benefit by considering that we are not our audiences.

Let’s not go overboard here. Experience is critical. Expertise is critical. A professional with experience and expertise hopefully realizes that we live in a connected world with new challenges facing cultural organizations, and expert experience may be more beneficial in informing not what we do (these things are changing), but rather how we approach what we do.

 

It’s good that professionals know what is going on inside of their offices – it’s their jobs! But it’s also their jobs to remember that knowing those things may affect how they see their world.

I propose that we, as an industry, start focusing more on what we know (informed by behavioral economics and market research) and less on what we think we know.

If other industries are any predictor, we aren’t likely to be an “I think” industry forever. We’ll need all hands on deck to make sure that the things we do (and the things we’ve always done) are actually effective in helping organizations reach their financial and mission-based goals.