## What is Outcome Measurement?

- Inputs
- Resources used in a program
- Money
- Staff
- Volunteers
- Facilities
- Equipment
- Supplies

- Inputs
- Resources used in a program
- Money
- Staff
- Volunteers
- Facilities
- Equipment
- Supplies

What program does with inputs:

- Feed and shelter homeless families
- Provide job training
- Educate public about signs of child abuse
- Counsel pregnant women
- Create mentoring relationships for youth

Products of a program

- # of classes taught
- # of counseling sessions conducted
- # of education materials distributed
- # hours of service delivered
- # participants served

Benefits or changes realized by program participants

- New knowledge
- Increased skills
- Changed
- Attitudes or Values
- Modified Behavior
- Improved Condition
- Altered status

- Initial Outcomes ñ Benefits participants experience during or directly after
- Often changes in participants knowledge, attitude or skills
- Intermediate Outcomes ñ Links Initial Outcomes to Longer term Outcomes
- Changes in behavior due to participants new knowledge, attitude or skills
- Longer-Term Outcomes ñ Ultimate Outcomes a program desires to achieve
- Represent meaningful changes for participants often in their condition or status

- Initial Outcomes
- If youth are mentored by adults who stress importance of education
- Then youth will see education as important
- Intermediate Outcomes
- If youth see education as important
- Then they will attend school more regularly
- If youth attend school more regularly
- Then they are more likely to graduate
- Longer-Term Outcomes
- If youth graduate
- Then they are more likely to become employed and not involved in criminal activity

Outcome Targets: Numerical objectives for a program’s level of achievement on it’s outcomes.

- Measuring Program Outcomes gives agency ability to demonstrate effectiveness:
- Retaining/Increasing Funding
- Enlisting and Motivating Volunteers
- Attracting new participants
- Engaging Collaborators
- Gaining favorable recognition
- Garnering support for innovative efforts
- Winning designation as a model or demonstration site

Task 1: Assemble and Orient an Outcome Measurement Workgroup

Task 2: Decide which programs to start with

Task 3: Develop Timeline

Task 4: Distribute your game plan to Key Players

Task 1: Gather ideas for what program outcome are

Task 2: Construct Logic Model for Your Program

Task 3: Select Outcomes that are Important to Measure

Task 4: Get Feedback on your Logic Model & Outcomes Selected for Measurement

Task 1: Specify at least one indicator for each Outcome

Task 2: Decide what factors could influence participant Outcomes

Level of success on Outcomes will be different for different participant groups based on:

- Demographics (age, gender, educational level, income level, disability, single parentÖ)
- Level of Difficulty (very difficult to help, moderate difficulty, minor difficulty)
- Level of Involvement (high, moderate, low participation)
- Organizational unit (if more than one service delivery facility)
- Service Delivery (group session vs. 1-on-1, live vs. taped)
- Avoid collecting everything and anything

Where will you get your data? Will data provide useful, reliable information related to the Outcome?

- Records
- Specific Individuals
- General Public
- Trained Observers
- Measurement

- Clearly state your intentions with the research.
- Include instructions with your survey questionnaire
- Don’t ask for personal information unless you need it
- Keep the questions short and concise
- Ask only one question at a time (the double barreled question)
- Make sure the questions are unbiased
- Present the questions in a clean and organized layout
- Test the survey questionnaire

- I would have worded the question differently
- I should have made the instructions more clear and easier to follow
- I would have better defined the scale
- I should have asked more open ended questions
- I should have dedicated specific persons to do all of the input
- I should have made the survey one page

Task 1: Develop a trial strategy

Task 2: Prepare data collectors

Task 3: Track and collect outcome data

Task 4: Monitor the outcome measurement process

During trial you will likely identify issues such as :

- Overlooked outcomes
- Inadequately defined indicators
- Cumbersome procedures
- Analysis and reporting dilemmas

Easing the data entry process:

- Code the data
- Review the completed questionnaires
- Assign a survey number
- Determine links
- Establish a process for open-ended questions

Steps to data Tabulation

- Count total number of participants for whom you have data
- Count number achieving each outcome status
- Calculate percentage achieving each outcome status
- Calculate other descriptive statistics (averages, medians, modes)

Write the numbers in order

Put all the numbers in numerical order

If there is an odd number of results, the†median†is the middle number

Example: 3, 5, 12

What is median?

The “mode” is the value that occurs most often.

If no number in the list is repeated, then there is no mode for the list.

3, 7, 5, 13, 20, 23, 39, 23, 40, 23, 14, 12, 56, 23, 29

In order these numbers are:

3, 5, 7, 12, 13, 14, 20,†23, 23, 23, 23, 29, 39, 40, 56

The mode is?

- Comparing two or more variables: same unit of measurement, comparable sizes
- Show
**how much** - Consider bar chart when your axis labels are too long to fit in a column chart

- You want to show data trends over a long period of time.
- You have too many data points to plot and the column or bar chart clutters the data.
- When you want to show how much has changed over a period of time.

- You want to show the breakdown of data into its parts. What ìpiece of the pieî is it?
- You have only one data series.
- The data points represent the parts of the whole pie. What does the whole picture look like?
- The parts are of comparable sizes.

- You want to show the trend of composition.
- You want to emphasize the magnitude of change over time.
- You have more than 8 data points to plot.
- The data points represent the parts of the whole composition.

- Analyzing and reporting relationship/correlation between two variables.
- When you want to show ëwhyí. For example: # of hours doing homework is related to grade in class
- When there are more than 10 data points on the horizontal axis.
- There are two variables that depend on each other.

- You want to demonstrate or identify areas of opportunity.
- You want to show
**where**

- Consider the needs of your audience: what information are they looking for?
- Keep it Simple
- Include a summary of major points
- Donít crowd too much on a page
- Define unfamiliar terms
- Define each outcome indicator
- Highlight points of interest with bold type, circles or arrows
- Use color to help highlight key findings
- Label charts and tables clearly ñ titles, rows, columns, axes
- Identify source and date of the data and note limitations
- Provide context (history or comparisons)
- Add variety to data presentation by using bar or pie charts
- Internal repots should be much more detailed than external

Task 1: Review Your Trial Run Experience, Make Necessary Adjustments, and Start Full-Scale Implementation

Task 2: Monitor and Review your system periodically

U$E 1: Identify Outcomes that need attention

U$E 2: Identify Client groups that need attention

U$E 3: Identify Procedures/Policies need improvement

U$E 4: Identify improvements in service delivery

U$E 5: Communicate Program Results

U$E 6: Hold regular program reviews

U$E 7: Identify training and technical assistance needs

U$E 8: Recognize staff and volunteers for good outcomes

U$E 9: Motivate clients

U$E 10: Identify Successful Practices

U$E 11: Test program changes or new programs

U$E 12: Help planning and budgeting

U$E 13: Inform Board Members

U$E 14: Inform Funders (current and potential)

U$E 15: Report to Community

- A brief description of the outcome measurement process
- Outcome highlights, including successes and disappointments
- Explanations of both successes and disappointments
- Actions the organization is taking or planning to take to address problems.

- Don’t jump to conclusions based solely on the data
- Assess the impact of sample size and composition
- Assess the response rate (Aim:50%, Scientific:80%)
- Consider what proxy measurements you are using
- Confidentiality

#Follow us on Instagram