Is your organization integrating market research into strategic decision-making processes yet? Here are eight important things to keep in mind.
I just returned from a whirlwind 24 hours in St. Louis for (a portion of) the American Alliance of Museums annual conference. While I was there, I had the honor of serving on a panel to talk data with some rad folks: Rob Stein (American Alliance of Museums), Kaywin Feldman (Minneapolis Institute of Art), Seb Chan (Australian Center for the Moving Image), and Kari Allderedge (McKinsey & Company). We spoke about how data can be used to help inform strategic decisions. Prior to the conference, Rob asked each of us to send him our key takeaways regarding our portion of the discussion. If you couldn’t make it – or if you could and you simply like a good refresher – these were my six takeaways…with two additional points that I want to add on after our discussion.
Here are eight, important things to keep in mind regardless of where your organization is in the process of integrating market research into strategic decision-making processes:
1) The need to make different audiences into regular audiences is urgent.
Cultural organizations are experiencing a phenomenon called the negative substitution of the historic visitor. Negative substitution occurs when the number of people who profile as historic visitors exiting the market outpaces the number of people who profile as historic visitors entering the market. It’s the driving reason for the decline in attendance to museums, zoos, aquariums, performing arts entities, and other visitor-serving organizations (particularly when contextualized by the rate of population growth). Negative substitution is taking place because the market is growing more diverse, while perceptions of cultural organizations as being places for a certain kind of person have remained largely static.
The negative substitution rate for museums shows that for every one historic visitor who leaves the US market (by way of death, relocation, or migration), they are being replaced by only 0.948 of a person (by way of birth, relocation, or immigration). This may not sound impressive – but this is actually a huge difference.
Think of it this way: An organization with a stable attendance of 1,000,000 visitors may keep doing everything right by their current audiences (e.g. marketing, developing exhibits, etc.), and then might reasonably expect to engage 948,000 future visitors…and then 899,000 visitors…and then progressively fewer yet visitors over time absent interdiction. And they will be doing everything right by their current audiences!
Although the negative substitution rate for aggregated cultural organizations is 0.948:1.00, rates are slightly different among visitor-serving organization types. For instance, for history museums, the number is 0.951. For art museums, it’s 0.946, and for science museums it is 0.939. For more about these rates and those for some other organization types, check out this article.
2) Audience research and market research are different. We need them both.
Negative substitution is an important example of a pressing reason why organizations must contemplate market research. Market research includes both visitors and non-visitors alike. It is helpful for spotting trends, informing strategic decisions, reaching new audiences, and providing clues for effective engagement. In order to know what people really think of our organization, we also need to know what the people who do not decide to pay us a visit think.
Market research is the type of research that helps inform our more global reputations and identifies primary barriers to visitation. Asking only current audiences about our reputations would be like Donald Trump solely asking the GOP what they think of him. Of course, the GOP is incredibly important to Donald Trump – they presumably comprise the core of his constituency. But, in order to get a more complete and accurate view of his reputation, he would need to include folks who may decide not to support him as well.
Market research tends to be “bigger” data, and it’s generally harder for cultural organizations to obtain. After all, without significant investment, it can be difficult to reach the folks who are not engaging with an organization. They aren’t likely following the entity on social media, they probably aren’t on the organizations’ email lists, nor are they onsite to survey. Adding these folks to the mix helps us understand the bigger picture of our organization’s effectiveness and reputation. Market research informs strategic decisions, and it helps answer the question, “What should we do?”
Audience research is also incredibly important! It Includes visitors to our institutions and participants in our programs. Once we’ve created a program, it helps us figure out how it can be improved. It can also let us know what current audiences like and don’t like, and what they expect from our organizations in the future. Audience research can help affirm and monitor the efficacy of strategic decisions, and it helps answer the question, “Is what we are doing working for our current audiences?”
Here’s a brief overview video of the difference between audience and market research that provides more information.
3) Market research does not seek to affirm decisions (although it can). It informs them.
Market research functions fundamentally differently than audience research. Simply, it is not a wholly adequate tool for affirming decisions. It’s not a thing to be considered after a decision has been made, but something to consider in the development of programs and initiatives. Market data can be helpful for evolving and altering programs so that they meet market expectations, but it may be better – and much more efficient – to utilize market research to design effective programs in the first place.
Remember: Market research includes high-propensity visitors. High-propensity visitors are people who profile as likely visitors, but they may not necessarily have visited your organization (yet). They represent market potential. Market potential can be different than actual visitation. For instance, some organizations think that their low visitation numbers from adults without children indicate that adults without children do not have interest in visiting the organization. Data suggest that this may be untrue. Moreover, nearly a quarter of potential visitors to cultural organizations fall in a single, ten-year age cohort that may not necessarily match an organization’s attendance – outlining a potential opportunity.
4) Effective organizations prioritize the perspectives of the market.
Organizations that take an “outside-in” approach to strategic decision-making generally outperform those still taking “inside-out” approaches. In other words, it pays to pay attention to the market and listen to its expectations, perspectives, and behaviors. Traditionally, visitor-serving organizations may be more used to the opposite strategy of essentially bestowing upon themselves the responsibility of determining for their audiences what these audiences should care about.
“I think” doesn’t count from an insider professional because we are not the market. We are not our audiences, so we need to ask and observe audiences in order to have a baseline understanding of their needs, wants, and expectations. We benefit by paying attention to target audiences, and realizing that we don’t know our audiences better than they know themselves.
5) Confirmation biases create blind spots. Challenge what you think you know to get real answers.
Confirmation bias may be the root of our industry’s most popular misconceptions. It certainly plays a big role in why Know Your Own Bone is sometimes called “controversial” by industry insiders and generally reads more like common sense to not industry-insiders.
Confirmation bias is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s preexisting beliefs or hypotheses. It’s human – and also prevalent and incredibly damaging.
Here’s just one example. We frequently observe in demographic surveys that more educated and higher income people visit cultural organizations during broadly publicized reduced admission days than during full-price admission days. (As a side, this makes perfect sense. Cultural audiences tend to be more wealthy and educated than the general public, so it’s wealthy folks who take up the deal when our initiatives aren’t specifically targeted). Confirmation bias makes us think that our organization is exempt from this happening because we saw some people who looked “low income” (whatever that means) on an affordable access day…particularly when went out of our way to look for people that we believed were low income.
Confirmation bias is often what makes cultural industry leaders utter that most damaging and defensive phrase, “That data doesn’t apply to me” when it applies exactly to them.
Again, confirmation bias is completely human. I even have coworkers that I run my ideas by for Know Your Own Bone articles or who help me edit them when I worry that my own confirmation biases may be popping up. These folks help me remind myself to question and look into even things that our industry assumes to be unassailable. It’s by looking into them that I’ve realized just how much of our industry best practices are still housed on an unstable rubble of “I think” rather than a sturdy foundation of economics or consumer behavior.
Confirmation bias makes us all blind. If you work with data, remember that you may not be able to completely remove those blinders, but you need to acknowledge that they are there or you may not be able to move forward accurately. We benefit by testing even what we think we know, and even that which will make us most uncomfortable if we’ve been doing it “wrong.”
6) Sometimes data makes people angry, but hard truths can help us evolve.
Real data will be hard sometimes. It’s data. You don’t get to chose the outcomes. Inevitably, sometimes there will be bad or surprising news.
If your organization never receives hard truths from data, then one of two things may be going on: First, your organization may be a miracle of modern cultural business practices, or – more likely – you’re not asking the right questions that lead to growth and learning.
If you’re truly asking good questions and chipping away at engagement barriers, you will get hard answers sometimes. They may be different than the Board Chair’s personal preference. This will be challenging. (Remember: A sample size of one person is not a significant sample – even if that person is the Board Chair.) It downright stinks to deliver data that says that your friend Tim-in-the-Community-Engagement-Department’s pet project isn’t reaching affordable access audiences, or your lunch buddy Nancy-the-Curator’s idea for an exhibit won’t be worth the cost. Though data can be hard to swallow, it’s great to get it. If we don’t learn, we cannot grow. If data suggest that a board member’s idea isn’t the best, then the board member isn’t dumb – they are great for coming up with the idea to test. And your organization is great for testing it first.
Organizations benefit by creating a culture where leadership is comfortable being uncomfortable. It’s the side effect of being thoughtful. It’s also the side effect of creating impact in a data-informed world.
7) Aim to collect signals rather than noise.
Sure, there’s a lot of data that you (or a third party entity on your behalf) could collect – but collecting data for the sake of collecting data without an idea of how it will be used to help you achieve your strategic goals can be a huge waste of time (and even a liability if you’re not careful with data management practices). Data can be a tremendous asset and, indeed, it should increasingly inform strategic decisions. That said, collecting data for data’s sake seems like a vain exercise in pseudo best practices. Measure what counts.
Consider what you need to know in order to best reach your organization’s goals and then use data to uncover the answers. Do you want to know if your organization is trusted? If an expansion project is a good idea? If people believe they are getting a good value for your cost of admission? The average amount of time between visits? If your new exhibit idea will attract new audiences? Why people with reported interest aren’t coming? Start with the questions. Data is important because it helps answer the questions that informs strategic decisions.
Figure out what you want to learn about the market or your audiences and measure signals. For instance, measure your reputation, your intent-to-revisit chronologies, trust, visitor satisfaction, value-for-cost perceptions…things that mean something. It’s easy to get distracted by noise. We get misled by noise because noise is often the easiest to get – never mind that it doesn’t really matter and often wastes time. Noise includes vanity metrics like click-through-rates, web visits, mobile app downloads, social media followers, and other numbers that don’t usually correlate with success in motivating visitation or heightening mission execution. These things can be helpful diagnostic metrics, but they are NOT key performance indicators. They generally do not belong in board packets. They are largely noise that can make the task of truly being a data-informed organization harder in the long run.
8) You do not need to be a data expert to make smart strategic decisions based on data.
You are already presumably an expert at the job that you have. You are probably already pretty darn stretched thin, too. You may be thinking, “We just got social media figured out…and now we need to navigate the ins and outs of data?!” Building a culture based on more data-informed decision-making may mean that it’s increasingly important for folks within an organization to know its value and possesses a baseline understanding of it. This does not mean that everyone in an organization needs to add to their job the work of becoming expert at data collection and management. In fact, that sounds like a disaster sure to result in a lot of noise and some very bad data collected by non-experts. Bad data can be worse than no data.
I work in analyzing cultural organization data every single day. I write about outcomes here every week. I keynoted about it last week in California and I am keynoting about it next week in Australia. I analyze data – but I don’t collect it. Other people in the company with significant academic and professional experiences related to data collection do that. I don’t conduct surveys or create questionnaires for market research. I help let people who are expert at creating unbiased instruments know the questions to which client organizations want answers. I don’t personally know the ins and outs of data collection and privacy laws – other people in the company with law degrees do that. (I hear that they are called lawyers!)
I like to think that I know a lot of things. I certainly don’t know everything about data collection, management, rules and restrictions, creating effective and unbiased survey instruments, the specific details of deploying those instruments, structural equation modeling, or network theory modeling (for starters)…and I am a “data person” who knows a great deal about data and uses it every day.
I believe that people who work in cultural organizations are superheroes in many, many ways. That said, if my full-time job revolves around data and I work predominately in one portion of it (analysis), then I think it may be a bit unfair for you to expect yourselves to become specialized mathematicians, behavioral economists, and lawyers in addition to your regular important work of educating and inspiring the masses.
That’s not how you become a data-informed organization. Becoming a data-informed organization means asking hard questions, challenging how you think and doing it critically, and using data to inform strategic decisions that is developed, deployed, and analyzed by experts.
Here are some ways that your organization can obtain data:
I’ve included a broad range of solutions and this list is nowhere near exhaustive. Organizations have different budgets, capacities, and capabilities.
- Have a good evaluations team for audience research
- Create a specialized data-related department with data-related experts. Some larger organizations already have these and they can be a big help!
- Hire a firm to collect, manage, and analyze market research
- Hire a firm to help collect, manage, and analyze audience research
- Follow market research available on the web to stay informed of trends
- Associations are increasingly investing in and distributing industry data. Follow them.
- Partner with a university. Many have departments that can help!
- Partner with a grant-making entity
- Create a cultural consortium of organizations in the area to share research costs
- Read Know Your Own Bone (Oh hey!)
I cannot possibly do this entire topic justice in one article, but boy did I just attempt a broad overview! I hope that it proved helpful and provided some food for thought. Again, there are many areas relevant to the topic of “data.” On the AAM panel, we spoke at length about data security. That is also relevant and important, and I didn’t even touch it here.
Holy moly did I meet many great Know Your Own Bone readers while I was at AAM! It served to reinforce what a great group of thinkers are perusing this site. I aim to be a data-informed resource for strategic decisions. If you’re reading Know Your Own Bone – and especially if your organization is passing it around – then you’re already creating a culture of data-informed, strategic decision-making. You’re asking hard questions and you’re likely already an “outside-in” thinker.
It’s an exciting time of change for cultural organizations that will lead to more effective operations. Data is a tool that can help organizations do what they already do best: educate and inspire their communities.
