Image Credit: Change Jam Questions | by love2dreamfish Change Jam Questions | by love2dreamfish
January 28, 2016–When CECP staff presents on trends drawn from Giving in Numbers and our hundreds of company conversations, we find many in the audience don’t feel confident in their understanding of impact’s true definition (exhibited by lots of note taking). CECP is raising some questions based on what we’re hearing in these sessions.
The phrase “measuring impact” (Impact being the “I” word) is said and written all the time in corporate societal engagement, but in how many cases is impact trulymeasured? Should our field use the “I” word as much as we are? It’s problematic to misuse the term because as the grand finale of the logic model, it is vital to evaluating effectiveness.
Logic Model: Why and What
A logic model is a measurement and evaluation tool. It outlines steps to effect change. Similar to how prioritized to-do lists help for the most productive days, a logic model helps improve effectiveness by articulating the change the work is trying to produce in a step-by-step way. Teams writing down and agreeing to a logic model can help improve results. It can improve what is tracked, how staff members spend their time, and how the budget is spent.
Experts in evaluation define five elements or steps (see examples below) in a logic model. Each one relies on the previous; one leads to or produces the next. Some say the five elements can be read using “If…Then” phrasing between them to see if the flow makes sense.
“I” Word Definition
CECP defined “impact” in Measuring the Value as “the change occurring in organizations, communities, or systems as a result of program activities over the long term.”
Examples for Understanding: From the Real-World
To begin, let’s take a real-world example: the change a group of New Yorkers will create by fulfilling their new year’s resolutions to get organized.
Input: $ spent at The Container Store, $ spent on sessions with a professional organizer, # hours spent organizing;
Activities: Meetings with professional organizer, Organizing personal documents and materials, Updating online file storage and calendars;
Output: # of hours spent organizing, # lbs of paper clutter removed from home, # of files moved to the cloud;
Outcome: # of hours freed up for new activities, # of familial arguments reduced;
Impact: Increased citywide productivity, Reduced NYC subway delays due to less people running late and holding the doors, Reduced professional hours wasted due to people missing appointments.
While not drawn from a real study (my first draft was on toddler potty training which was too real for print) a personal example is a great, low-pressure way to test your understanding of logic models and thus of the “I” word.
Examples for Understanding: From the Field
Through CECP’s E-Community, we crowd-sourced some favorite logic model examples from our companies.
One company focused on expanding knowledge of molecular biology:
Inputs: $13+ million invested, Expertise to build labs;
Activities: Building hands-on molecular biology labs, Training biology teachers;
Outputs: # of teachers trained, # of students engaged;
Outcomes: % improved biology test scores;
Impacts: Increased interest in biotechnology, Increased teacher confidence, Increased study competencies {CECP might add: new biotechnology innovations}.
Another company focused on building mentoring opportunities for employees into their nonprofit partnerships.
Inputs: $1 million invested, Business-education partnerships, Employee time;
Activity: Mentoring sessions by employees;
Outputs: # of mentoring hours, # of students;
Outcomes: Improved self-esteem, Increased healthy relationships;
Impacts: Increased career-readiness, Improved acceptance rate into secondary education {CECP might add: increased national measures of workforce productivity}.
Imagine trying to measure the impacts in these examples. Sometimes the issue is that the impact is not naturally quantitative. In other cases the issue is that the impact occurs too far in the future for measurement to be feasible. The examples show why this post began questioning how commonly we say “measuring impact.”
In the Giving in Numbers survey, companies identified the “success metric” for their focus areas. CECP classified eight companies’ examples into a best-fit logic model step below. Take note of how often impact appears:
Output: # of young people trained in work skills,
Outcome: # of young people that secure employment;
Output: Number of clients served,
Outcomes: Increased savings, Credit scores, % attaining financial goal;
Output: Number of meals served,
Outcomes: % of clients with access to healthy food, cooking, nutritional skills;
Outcome: Increased enrollment in STEM programs;
Outcomes: Children prepared for kindergarten, Reading by 3rd grade;
Outcome: More affordable housing,
Impact: Decreased unemployment for at-risk populations;
Impact: Increase in low-income youth that complete college education;
Impact: Decreased homelessness.
If not the “I” word, then what?
CECP, and many other leading organizations and practitioners, have begun to use the phrase measuring results more often. It is true your programs and departments seek to drive social impact at scale. When it comes to measurement though, “results” better reflects the practice.
This matters because CECP hears a lot of demand for our field to improve measurement and evaluation. CECP’s recommendation is for the field to make sure we have a foundational shared understanding of measurement terminology so when someone is about to present on how they’ve “measured their impact” everyone puts their phones down and listens closely to absorb an exceptional study of social change.
Share your point of view on this topic! Use the comment section below or emailcperez@cecp.co. If your company is part of CECP, click here to join the E-Community. The E-Community is a measurement and evaluation sub-group of CECP’s network interested in receiving curated resources and peer sharing. Some evaluators may argue whether all the “impact” examples above are truly impact. Stay tuned for my next post to explain why!
CECP is a coalition of CEOs united in the belief that societal improvement is an essential measure of business performance. Founded in 1999, CECP has grown to a movement of more than 150 CEOs of the world’s largest companies across all industries. Revenues of engaged companies sum to $7 trillion annually. A nonprofit organization, CECP offers participating companies one-on-one consultation, networking events, exclusive data, media support, and case studies on corporate engagement.