There is no single way of doing evaluation. The evaluation process your community develops will depend on your particular needs and circumstances.
A whole community approach to youth suicide prevention involves multiple activities, programs and initiatives, all of which operate in mutually reinforcing ways. Your end-goal is to address highly complex systems and issues. From an evaluation standpoint, this requires looking at the bigger picture. How do the many different parts of your initiative influence one another and evolve over time? There are at least three things to consider in approaching evaluation this way:1
- Ask what: Your community should assess the progress and impact of the whole change-making process. This includes evaluating the quality and effectiveness of your initiative's structure and operations, evaluating your initiative's structure and operations, evaluating your initiative's impact on the system it targets (e.g. youth-serving agencies)and your progress towards your initiative's goals (i.e. ultimate outcomes).
- Ask why: Your community should use the evaluation results to make decisions about adapting and improving the initiative. As your community progresses through the community mobilization process, the evaluation approach you use will vary. This downloadable table will help your community decide on the approach that makes the most sense to you.1 It is important to note that your community can and should use a combination of approaches over time,
- Ask often: Your community should embrace learning with the goal of continuous improvement. You'll want to start evaluating your efforts at the early stages of your initiative and throughout its course - this way you'll be learning and improving from start to finish. To do so, you'll need to set clear evaluation structures and processes and allocate resources appropriately.
There are also general steps to carrying out any evaluation. Although the steps to evaluation may seem linear, it’s important to remember that evaluation is an ongoing process. It doesn't end with the use and communication of results. You need to continually monitor and evaluate your efforts. The results of an evaluation may even lead to a new set of evaluation questions to investigate. The evaluation planning tool can help guide you through the steps to planning for your community’s evaluation.
Ethical principles in evaluation help ensure your evaluation is done in the safest, most respectful way possible for your participants. Depending on the context of your evaluation, you'll have different ethical considerations. For example, are you asking participants questions that could be triggering or harmful in any way? How are you keeping the information you collect confidential at every step of the evaluation?
How can you ensure your evaluation is ethical?
If your evaluation holds any form of sensitive content (e.g. questions about suicidal thoughts or behaviours, mental illness, loss, bereavement), it should always go through a formal ethical review process. Formal bodies exist to review, monitor and approve your evaluation research to protect participant welfare. The main types of formal ethical review bodies are:
- Ethics review committees which usually exist as an internal group of people with different roles within your organization (e.g. directors, managers, clinicians, youth, and family members).
- Ethics review boards which review materials within institutions such as hospitals, universities and other facilities that receive federal research grants.
There are also some guiding principles that you must follow to ensure your evaluation is ethically-sound from start to finish. These guiding principles include:
- Free, informed and on-going consent of all participants of your evaluation. Your participants must clearly understand and be aware of everything you’re planning to do with the information you’re gathering about them. They must also understand that if ever uncomfortable, they can withdraw from the evaluation process without repercussions.
- Privacy and control of the information about your participants. Any directly or indirectly identifying information you’re gathering about your participants must be kept confidential.
- Fair and equitable treatment of your participants. You can’t exclude any participants from your evaluation based on stable characteristics such as race, gender or socio-demographic status, unless you are specifically looking to gather information about a particular subgroup of people.
- No conflict of interest. You have to ensure your participants aren’t put in any perceived or real conflict with respect to their professional, social or personal lives by participating in your evaluation. For example, a young person shouldn’t feel pressured to take part in an evaluation because they were recruited by their own therapist.
- Let’s get ethical! Ethical considerations in program evaluation is a learning module created by the Ontario Centre of Excellence for Child and Youth Mental Health. This module covers the key ethical standards and principles that come into play when working with participants in an evaluation.
A Project Ethics Community Consensus Initiative (ARECCI) is a network of stakeholders dedicated to improving ethics review processes. Their webpage includes guidelines, a screening tool and webinars to help guide ethical thinking in different types of projects, such as in evaluations.
Looking for more information to help evaluate your community mobilization efforts? Check out Tools and resources.
- 1. a. b. Parkhurst, M., & Preskill, H. (n.d.). Collective Impact Forum: Learning in action: evaluating collective impact. Stanford Social Innovation Review. Downloaded May 4 from: http://www.ssireview.org/supplement/collective_insights_on_collective_impact