How does True Impact ensure accuracy of the data? How does True Impact collect outcome data on programs that are not complete? This article provides answers to questions your internal and external stakeholders might raise.
How do we validate the impacts in your nonprofit partners’ social impact reports?
While we do not audit the results (which would be a very expensive process, rarely undertaken in the sector), we take several steps that substantially improve the accuracy of social impact reporting compared to typical approaches.
- First, our reporting platform provides nonprofit users detailed guidance on how to avoid confusion commonly associated with differentiating between outputs, interim outcomes, and end outcomes.
- Second, we require back up and further explanation of each reported impact. This includes details on their direct measurement approach, or the research they are using to make evidence-based estimates (each of these we consider high-quality data). If they are unable to do so, they identify the social impact as a “guess” so you can properly contextualize the data they provide (which we still consider useful, but of low quality).
- Third, we review each submitted draft report in full, testing for internal consistency and reasonableness, and offer feedback and guidance to each nonprofit to help ensure accuracy and transparency.
Self-reported outcomes are not new in this sector. However, our guidance, backup requirements, and review process brings a new level of accuracy to your nonprofit partners’ outcome reporting.
What is quality data?
All data reported through True Impact must be categorized by its source. Top-level categories include:
- Direct measurement of program results, including testing, monitoring, or reporting by the beneficiaries or program implementers, using subjective or objective evaluation criteria.
- Estimate based on data from other, similar programs; indirect tracking of outcomes (e.g., government data); or the program's past results.
- Guess based on experience, anecdotal results, or logical assumptions.
The first two categories – direct measurement and estimate based on data – are considered high-quality (or evidence-based) data, providing some validation for the reporting organization’s assertions. Guesses are considered low-quality data (better than nothing, but an area that should be improved).
How do nonprofits report out on outcomes when their program isn’t complete?
True Impact's reporting system guides nonprofits through creating a social impact report no matter what stage of program implementation they are in.
If a nonprofit is just starting their program or period of philanthropic support, or if their program or support period is underway but not complete, they create an Initial Forecast or Interim Report. In each case, they report the impacts they anticipate generating at the end of the program or support period. In this way, it’s much like a statement of work or business plan.
For programs with a long track record, these projections can be very accurate; for new programs, expectations for accuracy tend to be lower. In either case, the nonprofit knows they will be asked to update the report with actual results when the Initial Forecast or Interim Report transitions to the Final Report stage (at the close of the philanthropic support period). As a result, there is a strong incentive to create projections in the Initial Forecast or Interim Reports that are accurate as possible.
Having access to both the final results and the variance between the projections and actuals presents a rich source of data for program managers and executive leadership alike.
How are the beneficiary demographic details collected from grantees?