5.1 About Evaluation
What is Evaluation?
Put simply,
evaluation means making a judgement about the value of something
- in our case, the value of a mental health promotion initiative.
Evaluation is the process of assessing what has been achieved
and how it has been achieved. It means looking critically
at your mental health promotion
project, working out what was good about it, what was bad
about it, and how it could be improved.
When you
evaluate your initiative, you are gathering information to
help you draw conclusions about the efforts of your group.
By using that information, you can decide what aspects of
your plan are working, and which areas need improvement. You
can then make any necessary changes to your action plans so
that you will be more likely to reach your goals.
The evaluation
information you gather can be used to improve your current
initiative or the next community initiative you undertake,
and can also be useful to help others who are trying to develop
similar initiatives in other communities.
But there’s
more to evaluation than simply finding out if you did your
job. It’s also important to use your evaluation data to improve
your initiative along the way. This is especially true of
mental health promotion projects; it’s essential that the
process, as well as the outcome of your project promotes the
mental health of those who are participating.
Back
to top
Why Should
You Evaluate?
You need
to be clear about why you are evaluating your work, because
this will affect the way you do it and how much effort you
put into it. It’s a good idea to evaluate your mental health
promotion initiative:
to improve
your work
Completing
evaluations helps you to build on your successes and learn
from your mistakes. Evaluation can document your success
with facts, figures and examples, which in turn will lead
to more support and encouragement from the community.
Evaluation
can also help to point out where you have fallen short
of your goals, so that you will learn from your mistakes.
Negative evaluations should be seen as an opportunity
to learn about what works, not as a failure.
to help
other people improve their work
It’s
important to think about and plan for disseminating your
findings, or sharing the results of your initiative with
others. Completing a thorough evaluation will help you
to present your project in a way that will help other
people learn from your experiences. It is important to
publicize failures as well as successes, to prevent other
people from repeating your mistakes.
to justify
the use of resources
You
will need to be accountable to those who provided your
initiative with monetary or in-kind resources. Most funders
require you to conduct a thorough evaluation as a part
of your reporting procedure. A positive evaluation also
provides evidence to support the case for doing mental
health promotion work in the future.
to recognize
the value of your work
Engaging
in the process of evaluating your initiative gives you
the satisfaction of knowing how useful or effective your
work has been. Being able to see the success and value
of your work will boost your spirits and motivate you
to continue with your work.
to identify
unexpected outcomes
Your
mental health promotion initiative may produce some unplanned
or unexpected outcomes. Whether those outcomes contribute
or detract from the goals of your initiative, conducting
a thorough evaluation will help you detect these outcomes
and respond to them.
Who's Afraid
of Evaluation?
Despite
all of the benefits that evaluation can bring to your initiative,
you may still meet with some
resistance
in carrying out an evaluation of your project In order to
gain the greatest benefits from evaluation, you might have
to overcome some common misperceptions about it.
Evaluation
can be threatening to many people. Generally, their fears
fall into three broad categories - "I don’t know how",
" I don’t have time", or "The results might
be negative and hurt us". All of these are valid concerns,
but they shouldn’t be so discouraging as to outweigh the benefits
of doing an evaluation. Here are some responses to those concerns:
"I don’t
know how to do on evaluation. It’s too complicated"
While
it may be true that you don’t have an extensive background
in evaluation, you can still do a good job evaluating
your mental health promotion initiative. This chapter
will provide you with some practical ideas and tools to
help you become familiar and comfortable with evaluation.
"I
barely have time to keep track of the different activities
in the project I don’t have the time to do an evaluation on
top of all the other things I have to do".
If you
have gone through a careful and thorough planning process,
you have already completed many of the tasks involved
in conducting an evaluation. Once you begin to record
the kinds of information you will need for your evaluation,
it will become just a regular part of the everyday life
of your mental health promotion initiative.
Although
it can take some time to plan, an evaluation can end up
saving you time by pointing out potential problems while
they’re still small, instead of waiting until they become
disasters.
"What
if the evaluation ends up being negative?"
Although
this is a possibility, it is fairly unlikely if you start
to evaluate early on, and pay attention to what the evaluation
is telling you. Remember that any negative results you may
find should actually be helpful to you, at least in the
long run. They will help you to improve the overall quality
of your mental health promotion initiative.
How Does Participatory
Evaluation Help?
By incorporating
participatory approaches into your evaluation, you will be
able to overcome many of these common fears. Participatory
evaluation is the direct involvement of group members and
other stakeholders in a way that enables them to learn from
their experience.
In participatory,
or empowerment evaluation, participants work together to identify
and implement appropriate criteria and methods for assessing
the process and impact of their efforts. This approach has
many potential benefits for mental health promotion projects.
Participatory evaluation:
- legitimizes
community members’ experiential knowledge:
- recognizes
the role that values play in carrying out evaluation research;
- empowers
community members;
- demystifies
the concepts and process of evaluation;
- enhances
the relevance of evaluation data for communities.
Given the
emphasis on participation in mental health promotion projects
overall, the evaluation also needs to involve community participation.
You will want to engage your members, community partners and
stakeholders in designing and carrying out your evaluation
and in disseminating your evaluation findings.
How Will the
Results Of Your Evaluation Be Used?
Who will
be using your evaluation data, and what will they do with
it? The answer to this will no doubt affect the kinds of questions
you ask, how much depth and detail you go into, and how you
present the information you gather.
If you are
conducting the evaluation solely for use by your own group,
in order to find out how to
improve
a session or workshop for the next time, you could probably
rely on observation and informal feedback as a means to assess
how well it went. In most mental health promotion projects,
however, the evaluation results will be shared with a wide
range of community partners and networks, so a more formal
component might also be necessary.
If you are
writing a report for your funder, or a community agency that
has supported your work, you will need to think through what
questions those people will expect to be answered, and how
much detail they will require.
The following
tips might help you deal with questions about how to tailor
your evaluation to so that it’s useful for everyone who is
interested in your results.
consider
your stakeholders
Now
is a good time to think back to what we talked about in
the section on planning your evaluation: Who are your
stakeholders? What kind of information do they want to
get out of the evaluation? Being clear on the answers
to these questions is essential to devising an effective
evaluation.
change
the emphasis
When
you communicate the results of your evaluation, you can
stress different aspects, depending on the needs and interests
of your stakeholders. Whereas community members may be
most interested in whether the participants felt the project
was beneficial, your funding source might want to see
if you reached all your objectives.
share
your results broadly
You
should be prepared to share the results of the evaluation
with all of your stakeholders, and potentially, anyone
else who is interested in your project. We will talk more
about sharing your evaluation and other aspects of your
project in the next chapter, Disseminating your Results
and Ensuring Continuity.
Back
to top
What Are The
Different Types Of Evaluation, And How Are They Used?
Although
there are many different types of evaluation, we will highlight
only two of them, process and outcome evaluation, because
they are particularly relevant to community mental health
promotion initiatives.
Process
evaluation looks at the activities that take place during
your mental health promotion initiative, to help you determine
how well things are going. Documenting the implementation
of your project by conducting process evaluation not only
helps the operation of your program, but it can also help
to make sense of the outcome evaluation.
Outcome
evaluation is very important for mental health promotion
initiatives, because it highlights the changes that happen
in your community as a result of the work done by your mental
health promotion initiative. These changes, which are tracked
using outcome measures, can take several forms, including
impacts on individual participants, and impacts on the broader
community.
In this
section we’ll discuss why it’s important to keep track of
both process and outcomes in your evaluation.
Back
to top
A. Process
Evaluation
The task
of promoting mental health in your community is a complex
one, and the immediate results of your efforts may not always
be clear. It is often a challenge to frame mental health promotion
projects in terms of outcomes, because many of the results
of such projects are things like increased empowerment and
self-determination - results that are very hard to measure
in concrete terms.
Evaluating
the process of your initiative, therefore, has special significance
for mental health promotion, for it provides an alternative
way of tracking your progress toward your objectives.
From the
beginning of your initiative, you need to collect information
that shows how you are doing in terms of fulfilling your objectives
and reaching your goals - basically, whether you’re on the
right track. Collecting this information is part of process
evaluation.
The information
gathered through process evaluation serves a number of key
purposes:
provides
positive reinforcement
Process
evaluation provides the reinforcement that you’ll need to
keep your project going; there is nothing like positive
feedback to boost your morale and that of your group members.
highlights
errors
It will
reveal the errors and miscalculations that are bound to
arise. Although it may seem threatening to have your mistakes
highlighted in this way, the negative feedback that comes
up through the course of the process evaluation can actually
provide you with great insights, and help to ensure the
quality of your project.
promotes
self-analysis
Process
evaluation sets up a healthy climate of self-analysis and
reflection that is essential in any community-based mental
health promotion initiative. Sometimes it’s helpful to set
up regular review sessions, where you can monitor your progress
on reaching the goals and objectives of your project.
provides
a historical record
If your
project achieves good outcomes, those wishing to replicate
it would need a clear description of exactly what was done,
and how it was done. If your project does not achieve its
objectives, keeping track of the implementation will help
you to pinpoint why - whether your objectives were unrealistic,
or whether the implementation did not go as planned.
keeps
you on track
Without
conducting frequent internal checks, it’s easy for community-based
mental health promotion initiatives to lose their momentum
and drift off target. Regular, informal evaluations at the
end of meetings (How are we doing? Is everyone feeling well/being
heard!) or even more formal (brief evaluation forms for
participants) will help to keep your project on track.
Back
to top
B. Outcome
Evaluation
Outcome
evaluation focuses on the impact your work is having on the
participants and the community. It helps you to know if your
work is effective, and if your have achieved the objectives
you set out to at the beginning of your initiative. An outcome
evaluation is most meaningful when contrasted with baseline
measures, so that you have a point of comparison that allows
you to gauge the impact your mental health promotion initiative
has had on participants and on the broader community.
The information
gathered through outcome evaluation serves a number of key
purposes:
justifies
use of resources
Conducting
an outcome evaluation helps you to justify the effort
and resources that went into your project, and to demonstrate
to the community, to the others who worked on it, and
to the funding source, that it was worthwhile and effective.
demonstrates
accountability
Evaluating
the outcomes of your project demonstrates that you are
accountable to those who supported your initiative - the
community (and perhaps funding agencies) contributed a
lot into your project. Outcome evaluation provides you
with a way of showing that they are getting something
of value out of it.
Back
to top
5.2
How to Evaluate >
|
