The following blog is the first entry in an occasional series examining how speakers can be leveraged to drive behavioral change in your organization. Is your agency’s leadership regularly committed to internal speaking engagements with no strategic end-goal? Are they attending the “right” internal events to meet strategic Organizational Change Management (OCM) plans? I am an evangelical believer that managed speakers are the best strategy to employ in a strategic communications toolkit and leveraging the following best practices will take you far in reaching your OCM objectives.
Speakers Bureaus are Resource Intensive But Worth the Price
Changing attitudes and employee behavior is one of the toughest obstacles executives will face when rolling out an OCM plan. However, discourse-based approaches, i.e., utilizing a Speakers Bureau for example, for understanding, communicating and managing change dynamics, has been largely underutilized by researchers and practitioners , and to a greater degree mismanaged, misunderstood and neglected by executives.
Leaders sometimes are not aware intensive change requires significant communications to generate the behavioral shifts in employee segments and leveraging a Speakers Bureau to communicate to an internal audience is one of the key strategies to employ. Executives often cite under-utilization of speakers due to time constraints, travel, hard dollar costs and the resulting lost productivity while a segment of the organization is away from work areas. However, it does not need to be that way.
A Speakers Bureau does require significant resources, but when used properly and couple with a data-driven methodology to measure Return on Investment (ROI), it can become a powerful tool in the constant battle a strategic communications team faces in shaping and mitigating negative stakeholder perceptions. Kirkpatrick’s Levels of Training is one of those tools. It is a data-driven approach and a best practice to leverage, that executives can use in evaluating and justifying the additional resources a Speakers Bureau might need.
Leverage Kirkpatrick’s Levels of Training
Kirkpatrick is best known for creating a model for training evaluation consisting of four levels. You might be thinking at this point, why leverage a training methodology to measure the effectiveness of speakers? Training is inherently a methodology where the key outcome is encouraging new behavior. And part of the feedback mechanism established to measure Speakers Bureau Measures of Effectiveness (MOEs) should, by definition of the outcome, utilize Kirkpatrick’s Levels of Training. The four levels of Kirkpatrick's evaluation model essentially measure:
Level One- The Reaction of the Student: This is the lowest level and measures what attendees thought and felt about the training and to what degree participants react favorably to the training. Training attendees have usually seen a Level One via an end-of-training handout (Smile Sheets) asking the trainee basic questions: How was the speaker? Was the room too hot? Was the PowerPoint presentation too long?
Level Two – Learning (The resulting increase in knowledge or capability): To what degree participants acquire intended knowledge, skills, and attitudes based on participation in the learning event/training. Level Two is the omnipresent testing for knowledge and a level that should not be used to measure speaker effectiveness in OCM -- unless an executive understands the organizational culture and that culture is open to testing.
Level Three- The Behavioral Change (a measurable shift in behavior): The extent of behavior and capability improvement if a gap analysis has been conducted. To what degree participants apply what they learned during training on the job. A Level Three evaluation is a powerful tool and measures stakeholder perceptions and the resulting behavioral shift after the speaking engagement. This is the reason why a Strategic Communications team exists in the first place -- it measures shifts in behavior that can be attributed to Strategic Communications engagement strategies.
Level Four- Return on Investment (ROI) Calculation: The effects on the business or the culture of the organization resulting from the trainee's “improved” performance or improved attitudes towards a particular OCM initiative. To what degree targeted outcomes occur, as a result of the learning event(s) and subsequent reinforcement. This is the ROI attributed to the sought after measurable shifts in behavior generated via strategic communications strategies.
Although Kirkpatrick provides a methodology for training evaluation, OCM and other transformation initiatives are all a series of communications tactics that “train” or persuade stakeholders to act in a different way. Do different things. Or understand a new way of doing business. All these strategies communicate and train a shift in behavior.
Putting It All Together
Below are some immediate tactics aligned with Kirkpatrick that you can use to maximize ROI on using internal speakers for change behavior initiatives.
You can include a one-pager asking basic questions about the speaker (Level One evaluation) in the speaker’s kit to handout along with other marketing/branding collateral, and collect the feedback before the speaking engagement ends. Once collected, use Level One measurements to modify your key messages and the presentations. Using the right questions, a Level One measurement can be powerful to generate actionable feedback; do not waste the opportunity by asking non-actionable questions, or worse, forgetting to collect them after asking stakeholders to fill them out.
Remember Level Three tactics will allow you to measure the sought-after behavioral shift for which you are leveraging a speaker bureau in the first place. Measure Level Three metrics via surveys, skill observation, interviews, work reviews, etc., in order to see a tell-tale sign that a shift in behavior is occurring and that the communications strategy is working. Level Three efforts require careful planning. Evaluating the behavioral transfer that the engagement seeks can be expensive and a thankless task involving control groups and quantitative statistics, but Level Three metrics pack a punch if done right.
To measure effectiveness, Level One and Level Three measurement analysis are vital but the Speakers Bureau coordinator should stay away from Level Twos – they are not worth it for the headaches and stakeholder friction they can create if stakeholders are suddenly faced with a test to evaluate attentiveness during the speaker engagement. However, executive leadership should be able to gage the organizational culture to ascertain whether to use a level 2 evaluation.
If resources and time permits, conduct a Level Four ROI calculation. At this level, and if the team has the ability to isolate and quantify the impact the speakers bureau had in a particular stakeholder segment, leveraging a Level 4 ROI calculation will give strategic communicators a data-driven decision-making culture that can validate the expenditure of scarce resources spent on speakers.
(1) Beneath and Beyond Organizational Change Management: Exploring Alternatives Andrew Sturdy; Imperial College London, UK firstname.lastname@example.org