Guest Writer | James McAra

Important to any proposal is an indication of the benefits resulting from changes that accrue given participation in a program.  The link must be made that any positive changes or outcomes will not occur without the support of a particular agent or potential supporter.  The means by which programs are seen as a suc­cess directly links (although not causal in relation) to positive outcomes meas­ured as a cyclic element of both program design and delivery.  By demonstrating beneficial outcomes, a program transfers accountability from delivery agents to supporters.  The transfer of successes also presents an opportunity for support­ers (funding agents) to transfer accountability into their structure thus reinforcing their decision to provide grant funds or other resources to programs showing beneficial outcomes to a larger community.  This flow of benefit and accountabil­ity provides an element of validation against the inherent risks of failed program success.  In other words; program agents, clients and funders can tell others that their participation and/or support lead to the greater good of the community.

 Outcomes may be defined as what has actually happened to a participant (group or individual) as a result from their participation in a program.  The results may be physical, psychological (behaviour, etc.), social or cultural.  Outcomes are a participant’s expression of change.  The measurement of outcomes should show a change that reportedly results from a participation in a specific program or organization.  Outcomes should not be viewed as just the increase or decrease in outputs, such as more trees planted, more food hampers delivered or more bums in seats.  Outputs and the direct results, or what a program actually does as illustrated by the United Way Program Outcome Model (below).  It is important to remember that outcomes do not represent a direct correlation or causal relationship to a program’s efforts.  There may be however, the beginning of a deeper understanding and thus the impetus to direct a more formal analysis of an overall program’s accomplishments that may ultimately lead to critical results.

 United Way Program Outcome Model^[1]

Inputs

Activities

Outputs

Outcomes

This is a complex and evolving topic in which significant developments have taken place over the past few years.  Across North America funders are incorporating Outcome Measures as a key element for all proposals.  The benefit of positive change steps beyond quantification of results and into a demonstration of positive changes resulting from program delivery.  More is not necessarily better, but getting better is now seen as getting more.  There are many excellent resources for the development of programming; it is recommended that a complete approach to program design include the development of a logic model, performance framework program outcome model (Figure 2).  A framework not only for program delivery but also for required resource accounting is a key component to the successful development of any program.

_{Figure 2:  Outcome Model: Evaluation Components}^[2]

In Canada the Voluntary Sector Initiative used Outcomes and Outcome Measurement as an element of accountability.  Looking at how an organization accounts for the resources it has received as well as how those resources have made a difference in participants.  Program success given measurable resources goes beyond a simple tally and comparison of numbers.  Outcomes demonstrate clearly that the resources being applied (in many cases specifically looking at how funding is used in program delivery) are having a difference and improving (hopefully) the conditions set out in the program design.

It should be cautioned that outcomes and their measurement is a relatively new development in the voluntary sector (non-profit).  There are many proven tools that reliably and accurately record and present outcome measures.  These tools (surveys, interview questions, response cards and other data gathering processes) have been developed and evaluated in accordance with statistical methodologies to ensure reliability and validity of the tool.  There are many tools however that are just data recording techniques against sub­jective criteria that are not based on sound statistical methods of design.  There are also many people that have caught the “Outcome” bug but are not able to distinguish between soundly designed tools for assessment and re­porting from a list of words that sound good.  Outcome measurement and analysis steps beyond output and provides data measuring change.  Out­come measures can be fed back into a program though a specific design loop to enhance desired effects or reduce the undesired.  The tracking and use of data that shows meaningful impact on the people in a program can have a consequence of care to the benefit of participants.

Resources

• Logic model development:  Canadian Outcome Research Institute www.hmrp.net/CanadianOutcomesInstitute

• Overall research and guidance:  United Way of America Outcome Measurement Resource Network www.national.unitedway.org/outcomes

• Refer to sections on accountability and financial management:  Voluntary Sector Initiative www.vsi-sbc.ca

• FCSS Program Logic Model:  City of Calgary www.calgary.ca and www.calgary.ca/docgallery/bu/community_strategies/fcss/fcss_calgary_program_logic_model.pdf

Guest Writer

_{James McAra is currently Executive Director of the Calgary Inter-Faith Food Bank.  He believes that being actively involved in building our community is a reward in and of itself.  Beginning with candy selling for school needs in Junior High School, James is constantly involved with family, friends and community.  We could list things that have benefited or suffered from his involvement but won’t.}

_{James received a Bachelor of Physical Education in 1990 and a Master of Arts in 1997.  He has held professional and voluntary positions as Executive Director, Committee Chair, President and Gardener.  He volunteers at his children’s schools and in his community association.}