Sometimes the findings challenge us, too.
There are four steps to becoming a data-informed cultural organization. Many organizations stop at the first one – data collection – and call it a day. But having data is not the same as using it. Data doesn’t need only be collected. Data also needs to be interpreted (someone needs to champion it and decode what it means), accepted (leadership needs to embrace it), and integrated (action needs to be taken that is informed by the data).
At a glance, the first step of data collection may still seem the most difficult. After all, it’s difficult to contemplate integration of data findings when there isn’t reliable data around in the first place!
But as more and more organizations are using audience and market research to inform strategic decisions, we’re not so sure that the collection of data is the most difficult step. It’s often a simple investment of money to collect the data, or time to find and contemplate high-confidence market research (like we share on this site, folks!). In my eight years thus far of working with IMPACTS and aiding organizations in understanding market research, I’ve come to believe data acceptance is the biggest hurdle to becoming a data-informed cultural organization.
Why? Data collection is based on math and science, but data acceptance depends on the fickle nature of human beings. Even our most accomplished leaders are human – and being human means having a whole host of unintentional cognitive biases that stand in the way of progress.
Arguably, the most helpful research involves findings that inform strategy and shine a light on blind spots. Bringing blind spots to light can be staggeringly inconvenient. It requires a willingness to see the realities around messes we may have been unintentionally (or intentionally) avoiding. The best data makes leaders uncomfortable.
If (high-confidence) data makes you uncomfortable, then a cognitive bias may be the root cause. It means that math has entered the war zone of emotions in the fight for becoming a data-informed organization – and the battle could get ugly.
By far the most popular “challenging” data we have is on free admission and discounted pricing. Pricing psychology is an entire section of behavioral economics with whole academic journals devoted to it… and the cultural industry often pretends it doesn’t exist. Free admission does not significantly impact attendance, free admission museums do not attract lower-income individuals, discounts do not bring in new audiences, and the steeper the discount, the less satisfied folks are with their visit. None of this is news to a behavioral economist, but these realities are almost traitorous to acknowledge within the cultural organization industry.
Why? Because of feelings.
We just feel like being free is the same as being welcoming. (It’s not.)
In my role at IMPACTS, I am largely divorced from the process of survey methodology, and the math and technology of our data collection processes. I know what’s going on, but I don’t direct or inform any questioning… for good reason. I know the desired outcomes! Our company separates the data-collection process from the analysts so human emotion does not manipulate the methodology or the findings. By keeping wishful thinking away from the collection process, the resulting data often unintentionally… well… crushes some of that wishful thinking.
I’m a human, too. (Hi there.) So are my colleagues. The three of us who work on the front end to make the findings of the National Awareness, Attitudes, and Usage Study accessible all have a background working in cultural organizations. We work with leaders and board members on a daily basis (and we are those board members)! We know the industry assumptions well… and we know when we uncover challenging findings.
Here are five findings that gave me and my colleagues feelings when we first saw them. These are the findings that made me say, “no way” and – in one or two cases – send it back to IMPACTS, “just to double-check.”
(Unfortunately, high-confidence data does not change just because someone is uncomfortable.)
Ready? Let’s get comfortable being uncomfortable.
1) Local audiences have negatively skewed perceptions of the organizations in their area.
We know, we know, oh goodness do we know: Your local audiences are your best audiences. But are they, really? We consistently find that regardless of region or organization type, local audiences are generally the hardest to please.
IMPACTS tracked 118 visitor-serving organizations and found that on average, people living within 25 miles of the organization indicate value-for-cost perceptions that are 14% less than those of regional visitors living between 25 and 101-150 miles away. In other words, locals believe their experience is less worthy of the admission cost they paid compared to the perceptions of those living further away. Interestingly, locals paid 20% less for admission, on average, than non-local visitors thanks to local discounts and promotions! They are also much less satisfied with their experiences than non-local visitors.
(This isn’t wholly surprising in itself. The steeper the discount, the less satisfied people tend to be with their visits. When we devalue our experiences, people devalue them right back – and local discounts are common among cultural organizations.)
To add insult to injury, non-local members tend to be more valuable than local members as well.
These findings make sense when you consider the data, but they are hard to take in. Many organizations strive to be connectors and treasures for their local communities. And many are! This finding doesn’t counter that notion! But it does shine an important light on locals’ embedded perceptions. Many cultural organizations have unintentionally created a bit of a cycle of entitlement among local audiences, and this plays a role in long-term solvency that we don’t often stop to strategically consider.
2) An average visitor attends a cultural organization type only once every 27 months – and the average member returns to take advantage of free admission only once per year.
The average person who visits an art museum will not visit another for 28 months, on average. The average person who visits a history museum will not visit another for 32 months, on average. In total, the average visitation cycle for organization types that we monitor is 27 months. Here’s more on that data and what it means.
Does this surprise you? If you’re reading this, you’re probably a (cool?) weirdo like me who loves visiting cultural organizations every chance they get. But take a look at the data below (even realizing that you have a skewed perspective) and consider how often you visit some of these organization types. Or think of how often your friends might.
Remember: This is data for people who actively attend these entities. There are a whole lot of folks in the US who do not have interest in visiting cultural organizations at all.
I’ll be honest: The fact that the average visitation cycle of most organization types is over two years was never surprising to me. We see too much data over here on visitation cycles and audience behavior for this to be at all shocking.
But when I saw that nearly 20% of members do not visit, and members who do attend visit only once per year on average, I had serious feelings. I even reversed the order of articles for publication on this site because I needed some time to process the information. There are so many programs and benefits for members! The ability to return for free is a top membership benefit, but folks only use it once, on average. (Critically, this doesn’t mean that folks don’t buy memberships intending to use it more often!)
Subscription-based organizations such as theaters and symphonies: You’ve got it a bit better. Your members visit twice each year, on average.
3) Millennials are not “aging into” caring about arts and culture
This isn’t surprising to me and we have so much on this we’re getting into a “ridiculous” data volume category here, but this shocks other folks, so it’s making this list!
Millennials are not “aging into” caring about arts and culture as a natural function of getting older. Millennials also are not “aging into” other things some entities are banking on, like the belief that dolphins should be kept in captivity. In the data below, you can see that individuals are taking their cause priorities with them as they age, and it is influencing the entire age bracket! This finding deserves a deeper dive, and you can read that here.
In a nutshell, the belief that a whole generation will “age into” caring about something is based on the realities of a previous time in which the web was not omnipresent. Today, we turn on our devices to a myriad of messages and calls to action. A person need not live near the ocean to be made aware of plastic pollution. A person need not have a child reach school-age to care about the public education system. In a less connected age, we arguably encountered certain issues as a function of aging and life stages. Now, we don’t need to age to come face-to-face with critical issues anymore, and this is likely playing a role in these findings.
Millennials are a very important group for cultural organizations to engage. The take-away of these findings is critical: “Let’s just wait for people to think we’re important” is a failing engagement strategy.
4) On average, attendance goes back to baseline 5 years after a major expansion (but operation costs tend to be increased forever).
This finding was pivotal for my colleague, Jim Hekkers, when he served as the Managing Director of the Monterey Bay Aquarium. Projecting the attendance performance of a proposed costly expansion was the first thing that the Aquarium asked IMPACTS to assess some 15 years ago. Jim says that coming to grips with the reality that expansions do not generally increase long-term attendance was his first lesson in understanding what it means to be a data-informed cultural organization. “From that moment on,” Jim says, “I began to actively force myself to shift to analytical thinking with each new data finding and focus on the strategic implications of the data rather than waste time arguing about its validity.”
IMPACTS tracked attendance to 11 visitor-serving organizations that completed significant expansion projects between years 2003-2011. Each of the assessed organizations had been in business for at least ten years before opening its new expansion. The average project cost was $43.6 million. Here’s what we found:
In a nutshell, attendance decreases in the years prior to a major building project as folks defer their visits until after the expansion opens. When an expansion opens, attendance certainly increases – 19.6% compared to the ten years prior! But that increase gradually decreases until attendance levels retreat to the baseline of the ten years prior after only 5 years. And the increased building space also means more staff members, more programming, more electricity, and more ongoing maintenance.
This finding doesn’t mean expansions are bad by any means. They can serve several functions that are meaningful and strategic, like maintaining relevance and updating spaces. However, a “long-term attendance increase until the end of time” is not likely to be one of the outcomes.
If you’re fundraising for or undertaking a major building expansion, make sure that you are clear on your goals and objectives – and that your expectations for long-term attendance and ongoing maintenance are grounded in reality. (As a reminder, an organization need only spend 1% of a major project budget to assess its feasibility.)
5) Mobile applications do not significantly increase visitor satisfaction
If you’ve ever contacted us with a question about the data or a request, then you’ve probably heard from Bethany Corriveau Gotschall, Know Your Own Bone’s awesome Project Coordinator. As a professional with experience working closely with onsite technologies, she says that this particular finding was most challenging to her: People who report using mobile applications onsite report visitor satisfaction scores that are not even 1% higher than people who do not use a museum’s mobile application onsite, on average.
Interestingly, people who use social media onsite in a way that relates to their visit report 7% greater visitor satisfaction scores than people who do not use social media in relation to their visit. Mobile web users experience a 6% bump in satisfaction. Even though all three of these methods (mobile applications, social media, and mobile web) take place on a mobile phone during a cultural organization visit, social media and the web significantly contribute to the visitor experience. Mobile applications do not reliably do this. One explanation for this may be that social media and mobile web “meet audiences where they are” and are examples of onsite technology facilitating the experience. Mobile applications, on the other hand, can be examples of technological intervention in which a visitor must interrupt the experience to figure out how to engage with the technology, or download it in the first place.
When we first published this data in 2017, we got some hate mail from mobile application developers angry at us for shining a light on the unbiased impact (or lack thereof) of most mobile applications. By the time we published a data update in 2019, it seemed the storm had passed. Still, the story of mobile applications and the cultural industry remains a great example of the bandwagon effect and the perils of investing in technology for technology’s sake.
This website is full of myth-busts and reality checks. Trend data is critical for cultural organizations and some findings are more difficult to swallow than others. Instead of proclaiming, “that doesn’t apply to me,” we encourage leaders to say, “Okay. I see that this is true for the industry. To what extent does this finding apply to me and my organization?” It is then that one may confidently assess that theirs is an exception to the finding. It may be. Uncovering this is up to you.
Market research can be inconvenient or uncomfortable, but the things that encourage change and growth – both in life and in the cultural industry – often are.
When you see high-confidence data that challenges you, know you’re not alone in your discomfort. Indeed, that discomfort may be helping us and our organizations grow.
Nerd out with us every other Wednesday! Subscribe here to get the most recent data and analysis on cultural organizations in your inbox.
