The first step toward becoming a data-informed cultural organization is effective data collection. Here’s some basic groundwork, no matter your role at a cultural entity.
One of the most interesting parts of our jobs here at IMPACTS (in my humble opinion) is helping cultural organizations evolve from “I think” organizations of the past that make guesses about how to engage people… to data-based “I know” organizations of the future. It’s a big shift, and it’s hard work. Our team works with leaders who are turning around big ships, small ships, and many-sized fleets in between. We are constantly in awe of those leaders who take on this monumental – but necessary – task.
But how exactly does an organization start to become data-informed?
One common misconception is that being data-informed simply means collecting some kind of data. And then – POOF! – an entity is data-informed!
If only it were that simple…
Becoming truly data-informed takes a great deal of leadership dedication and often requires significant culture changes. Shifting from inside-out thinking (talking at people) to outside-in thinking (talking with them) takes time and energy. But it’s also how organizations may be most effective and successful today. It’s often a major undertaking. But we can break the process down a bit: In a nutshell, becoming a data-informed cultural organization has four primary parts: (1) data collection, (2) data interpretation, (3) data acceptance, and (4) data integration.
I feel a series coming on…
Today, we’ll provide a broad overview of the first step in this process: Data collection. Then, on September 25th, we’ll provide an overview article on data interpretation and cover data acceptance on October 2nd, with a new KYOB Fast Facts Video to boot. Finally, on October 9th, Jim Hekkers, the former Managing Director of the Monterey Bay Aquarium who aided the organization in evolving to a data-based culture, will share leadership tips on the topic of data integration. We’re excited to share these articles with you! If you haven’t already, make sure to subscribe so you don’t miss any of them.
Do Your Research: Audience vs. Market
Let’s start with the basics: Data collection can generally be divided into two related categories: audience research and market research. Audience research is a subset of market research.
Market research includes everyone – those who are visiting cultural organizations as well as those who are not. Audience research, on the other hand, specifically investigates the perceptions and behaviors of folks who currently visit or have visited, or likely visitors (i.e. folks in a database who may be likely to visit).
On this site, we share market research which includes non-visitors, but we can also drill down into the market research to share audience research. Market research aims to be representative of the market at large (the world, the nation, a state, a city, or a neighborhood), while audience research aims to be representative of audiences currently visiting. Audience research helps us understand who visits, but only high-confidence market research helps us understand the people who do not.
Because market research requires larger sample sizes, it is often more difficult to obtain than audience research. After all, audiences are readily available on-site, in our email lists, or following us on social media. It’s a lot harder to reach the people who never even think to visit. Market research can help us figure out what those people think.
Evaluation may potentially be seen as a third category, although it is closely related to audience and market research and overlaps in many ways. While market research may be most helpful in informing strategies based upon how people think and behave for the development of programs, evaluation can be conducted as a form of audience research in order to assess how well specific programs are working in meeting goals and objectives, or even changing behavior. Simply put, market research tends to be about people and their behaviors, thoughts, and motivations, while evaluation can help organizations understand how those thoughts and behaviors are influenced by or related to a specific program or initiative.
From understanding the people who have interest in visiting but haven’t, to getting to the nitty-gritty of satisfaction with a specific program and how it is changing perceptions, all these kinds of research have value and serve meaningful purposes.
Measure What Counts
Okay, you’ve got some insight into the types of data out there. But the kind of data you want – what will be most useful to your organization – depends on the questions you need answered.
There’s a lot of data that you (or a third party entity on your behalf) could collect – but collecting data for the sake of collecting data without an idea of how it will be used to help you achieve your strategic goals can be a huge waste of time, resources, brainpower, and even a liability if you’re not careful with data management practices. Collecting data for data’s sake can also risk burning people out by confusing fads and trends. If your organization starts trying to change its whole strategy based upon last week’s most popular Saturday Night Live sketch, then it might be picking up too much noise. While capitalizing on fads can be a beneficial tactic to engage audiences (such as timely social media content about sparking joy a la Marie Kondo), trends inform strategic direction over time.
Consider what you need to know in order to best reach your organization’s goals and then use data to uncover the answers. Do you want to know if your organization is trusted compared to others in the nation? If an expansion project is a good idea? If people believe they are getting a good value for your cost of admission? The average amount of time between visits? If your new exhibit idea will attract new audiences? Why people with reported interest aren’t coming?
Trend analysis similar to what you read on this website can help shine a light on how people think and behave, and it can bring big ideas you’ll need to consider in terms of your strategic goals and objectives. For your own data collection purposes, start with the questions. Data helps answer questions that inform strategic decisions. In collecting it, you may come up with new questions. (Indeed, that’s the process of our work at IMPACTS as well! Sometimes we’ll uncover a bit of information that surprises us and informs our next set of inquiries.) Figure out what you want to learn about the market or your audiences and measure signals. For instance, measure your reputation, your intent-to-revisit chronologies, trust, visitor satisfaction, value-for-cost perceptions… things that mean something. It’s easy to get distracted by noise. We get misled by noise because noise is often the easiest to get. Never mind that it doesn’t really matter and often wastes time. Just because you can count something doesn’t mean that metric matters much.
Start with good questions – and you’re more likely to get useful answers.
Count what counts.
How do we get the data?
You have questions, and now you need answers. Luckily, you’ve got options! Here are some ways to obtain helpful research:
Work with a third-party entity specializing in high-confidence research
There are a lot of ways to accidentally mess up data collection – especially if someone is not trained in audience research and evaluation. Given that many cultural organization professionals wear several hats at once, it may be far-fetched to expect everyone to also be experts in the math and behavioral science that goes into sophisticated data collection. If your organization doesn’t have that skillset on hand and needs advanced research, you may want to consider hiring an entity to help. Ask them to deliver only data that achieves 95% confidence or above, as that is the standard within the data collection industry and it can be overlooked by entities that collect research more casually or without the proper background. Hiring outside help may be an especially good idea if you are looking into the perceptions and behaviors beyond those who already visit (non-audience members).
This may sound expensive, but it doesn’t have to be! There are opportunities to partner with universities, obtain aid or sponsorship from your city’s convention and visitors bureau, or team up with other cultural organizations to share costs on a market research study on the behaviors and perceptions held by people in your city or state. (In fact, we at IMPACTS are asked to provide research this way rather frequently.)
Collect your own audience research or conduct evaluations
We’re seeing this done well more and more often – and we like it! Entities are increasingly hiring their own specialists and devoting trained, full-time staff to areas of audience research and evaluation.
Not only that, it’s working! Entities increasingly collecting and acting on their audience research is one of the reasons why a growing percentage of attendance is made up of repeat visitors. Organizations are figuring out what makes people most satisfied and successfully executing those things.
Having dedicated staff to focus on collecting audience research and conduct onsite evaluations positions organizations to become leaders in new and exciting areas of thought. One of my personal favorite brains on the evaluation scene, in particular, is Kathayoon Khalil who works with the Oregon Zoo and is heading up efforts to understand visitor empathy toward animals at zoos and aquariums. How cool is that?! And the Peabody Essex Museum even has a neuroscientist in residence!
Read and reference publications, articles, and reports
You can collect high-confidence research right at your own desk! If you are a subscriber to this website, then you’re already actively taking in market research that may provide signals for your own organization to make strategic changes or investigate specific trends. Reports, articles, and publications can be excellent sources of information and many of them are free – a bonus for famously cost-sensitive nonprofits. (I’m a bit of a Pew Research junkie myself.)
But be aware of who pays for the data you take in. I cannot emphasize this enough. Not every data collection company is hired to provide real data to the public – some are hired to “prove” predetermined hypotheses and the hiring entities omit unfavorable outcomes in reports to the public. (Sometimes entities hire research firms with the intent to use outcomes as advertising.) Or worse, the data collection process is purposefully skewed (framing questions, non-representative samples, etc.) in order to achieve a desired outcome. If a mobile application company pays for data to show how great and successful mobile applications are for museums, for instance, be wary. Find someone without skin in that mobile app company’s game to provide an assessment.
If your organization is utilizing trustworthy sources – and abiding by best practices when collecting your own research – then you’re obtaining helpful information that may aid your organization in making strategic, data-informed decisions.
Having data in the first place is an important prerequisite to being data-informed, but it isn’t the end of the conversation. Once an organization has data, they need someone to interpret it, they must accept it (if it’s high-confidence data), and then actually use it. Collecting data – or having it – is the first step, but it’s not the only one.
Next week, we’ll dive into data storytelling and why data needs a champion and an interpreter or analyst in order to get the message through. (Don’t worry. We’ll show you what we mean next Wednesday.)
We look forward to continuing the conversation next week!
Stay tuned by subscribing here.
This is Part 1 in the series on becoming a data-informed organization.
For Part 2, see The Few, the Proud, the Nerdy – Why Your Organization Needs Data Advocates.
For Part 3, see Accepting Data Can be Hard – These Brain Tricks Make It Even Harder.
For Part 4, see Seven Things Data-Informed Organizations Do Differently.
