Toward Representation in Evaluation Surveys
Think of a time you were serving your community. Perhaps you sent out a survey to town residents at a community gathering, and you received 35 out of 60 answers, making you confident that you had responses to apply to the entire town. However, what if the 25 people who chose not to respond to the survey were people who varied strongly from the rest of the group? Maybe they lived in a different neighborhood than the other 35 people. Without knowing it, you may have gathered responses that are not actually representative of an entire community. And without those 25 responses, it might be more difficult to make decisions that truly benefit the whole community.
The many surveys that nonprofits administer may seem tedious, but they are often necessary to accomplish their missions. However, it is important to remember that each community varies from each other and has unique characteristics. In this blog post, we’ll discuss the importance of making sure surveys are representative of your community.
Why surveys should represent the community
When conducting evaluation, it is important to note how data can be applicable to people. In order for the benefits acquired from data to be helpful to people, the demographics should be reflective of that community. For example, evaluation work done in collaboration with a primarily African-American community might not have the same relevance for a Latino community. Different populations have different cultures and histories that make each group unique. Even within a community, there are differences of opinion that must be recognized. Communities are not homogenous.
An important factor to consider is wealth. If you would like to gather information about your community and send out a survey but only receive answers from those living in wealthier areas, the responses cannot necessarily be generalized to a larger group of people who may be more middle class. If the information you gather does not accurately represent the community you wish to serve, the data you have collected is incomplete—and could lead you to make uninformed decisions.
Unfortunately, there are cases where surveys or information have been collected and applied to a community that is not represented in that data. For example, during COVID-19, an evaluation was done regarding the high need for contact tracing of COVID-19. The contact tracing was needed nation-wide, however, there was a clear class gap, with working class families and individuals not having as much accessibility to contact tracing (O'Donnell et al., 2022). Research has also found that nonresponse bias—when groups of people choose not to respond to surveys—may lead to misconceptions. For example, a nonprofit may use a survey to find out how many teenagers volunteer. If they only receive responses from teenagers who volunteer, and not as many responses from those who do not volunteer, it may make their evaluation inconclusive (Hager, 2013).
The key point is that survey tools need to appropriately reflect the cultural and linguistic diversity of the community served. Ensuring this representation in survey evaluation not only ensures strong community voice, but it also will help you make better, more informed decisions—leading to better care being provided.
Collaborating with Diverse Groups
It’s also important to build trust in communities. When organizations have built social relationships rooted in trust, communities are more likely to respond and participate in activities like surveys and other data collection processes. Benefits can arise for everyone with an increase in trust and collaboration. With this in mind, nonprofits can work to make their surveys and questionaries more responsive to the communities they serve, which in turn will help them advance their missions.
For more information on collaboration and obtaining insights within communities, consider reading the blog “How Focus Groups Can Inform Your Community Health Assessment,” or reach out to Common Good Data for assistance.
References
O'Donnell, C. A., Macdonald, S., Browne, S., Albanese, A., Blane, D., Ibbotson, T., Laidlaw, L., Heaney, D., & Lowe, D. J. (2022). Widening or narrowing inequalities? The equity implications of digital tools to support COVID-19 contact tracing: A qualitative study. Health expectations: An International Journal of Public Participation in Health Care and Health Policy, 25(6), 2851–2861.
Hager, M. (2013). Gathering information from nonprofits: toward representative survey samples. Journal of Nonprofit Education and Leadership,3(1), 47-59.
