You’re a nonprofit leader and someone asks you a question about the impacts of your program that you don’t know the answer to, but really wish you did. Now that you’ve decided you don’t want to be in that position again, what should you do next?
There are many technical resources available to help nonprofits collect data. We’ve compiled our top 5 best practices for data collection to keep in mind as you start to think about how to plan for, collect, and analyze data.
1. Have A Plan
Once you decide you want to better collect your data, it can be tempting to dive in to get information as quickly as you can. However, our number one best practice is to spend some time at the beginning of a data collection effort to make sure you have a documented plan that includes details about (1) what you are collecting, (2) why you need it, and (3) what you’ll do with it.
In this stage, consider these questions:
- What do you want to learn?
- Why do you want to know?
-
- Is this for program improvement? Internal reporting? External reporting?
- Do you need this information for: understanding your program, improving processes, understanding clients or participants, problem solving, decision making?
- When do you need to know?
-
- Do you have specific reporting deadlines you need to meet?
- What is the best way for you to collect this information?
-
- How can getting this information fit into existing data collection efforts, if at all?
- Over what time period will you collect the data?
- Do you want to have numbers, stories, or both?
- Who will be responsible for collecting data?
- Where will you store this data?
- How will you analyze the data you collect?
-
- Who will be responsible?
- When will you analyze the data?
- How do you plan to share this information?
-
- What format will it take?
- Who will the primary audience be?
Having a concrete plan that outlines the answers to these questions will help you engage the right people, develop realistic timelines, and give you a north star to guide your efforts. It can also be useful to have this mapped out for other staff to engage with and to ensure that anyone joining the organization has clear insight into your measurement plans.
2. Be Intentional when Designing Data Collection Instruments
Designing the instruments – the surveys, tools, or discussion guides - you’ll use to collect data is an important part of any data collection effort. We recommend thinking carefully about the instrument design to make sure that you get the information you need while minimizing the time, cost, and effort of data collection for both the respondents and your team.
Whether you’re fielding a survey or holding a focus group, it’s also important to provide a clear, upfront description of your data collection effort to let respondents know why you are collecting their information and what you plan to do with it. In addition to providing transparency, this will give you an opportunity to confirm that you have their consent to participate in the data collection effort.
During this stage, remember that while you are well-versed in the information you want to collect, it may not be obvious to your respondents. The language that you use and how you structure your questions can make a big difference in the quality of the data you receive.
A strong data collection instrument:
- Includes clear definitions
- Uses standard language and avoids jargon and abbreviations
- Has short, simple questions focused on one topic at a time
- Uses categorical drop downs or multiple-choice fields, when possible, to streamline analysis
- Offers mutually exclusive categories for multiple choice questions
- Includes “Not applicable” options when relevant
- Includes only questions that generate the data you need to answer your research questions
There is also a great opportunity in this stage to bring in other voices to create buy-in for your data collection effort. Consider bringing in other stakeholders, partners, and beneficiaries to provide input on your instrument from their perspective. For example, you might draft a set of survey questions and then ask program staff to review them to see if they think respondents will understand the questions or if there is important information that might be missing from the draft.
3. Pre-test Instruments
This is often a skipped step that can have big implications for a large-scale data collection effort. After spending time and energy to thoughtfully design your instruments, it’s important to put the tools in front of the people you want to respond to see if the questions, format, and concepts used are clear and concise.
Identify 2-3 respondents you think would be willing to give you feedback on how the instrument flows and how the questions are worded and ask them to help identify any areas of general confusion. With this information, you can make critical changes to your instrument based on real feedback before fielding it to make sure that you collect the right information during your full data collection effort.
Don’t waste time and energy collecting data that doesn’t meet your needs!
4. Employ a Data Verification Process
You may have heard the saying that the quality of any analysis is only as good as the data going into it. It’s important to ensure that you have a process for reviewing the data as it comes in to make sure that responses are logical given the questions asked and can be used for analysis. This might include identifying processes for reconciling data inconsistencies with a respondent or planning for how to analyze fields with missing data.
Once you are confident in the quality of your data, you can move forward with analysis and start assessing different options for visualizing your data.
5. Reflect and Fine Tune Your Approach
After you’ve collected and analyzed data, created reports or deliverables, and shared the findings with those who need the information, take a minute to reflect on the data collection process with your team:
- Where did things go wrong? Where did things go right?
- Do you want to continue to collect this data on a regular basis?
- Can this effort be consolidated with other data collection efforts?
- Are there any new questions you want to incorporate into your existing data collection instruments?
Data collection can, and should be, an iterative process as part of your own continuous quality improvement efforts.
We hope these top 5 best practices will help you successfully collect the data you need to share your organization’s story!