How to measure the impact of your gathering
Billions of dollars are spent every year on motivating, engaging, and improving employee performance. The largest spending often comes from our gatherings. Not just in cost, but in attendee time.
There are offsites, training classes, leadership development programs, new hire orientations, All-Hands, conferences, and more.
But despite our reliance on this tool to bring about change, many of the organizations I speak with struggle to meaningfully measure the impact of these hefty investments.
In this article, I’ll break down some of the reasons why this is, provide measurement examples, and tips on how we as change makers can work alongside the business to showcase the importance and ultimately, the ROI, of our efforts.
4 reasons why we struggle to measure gathering impact
We are rewarding the wrong things
The saying, “what gets rewarded gets done” applies here.
If we are rewarded for participation numbers in training classes or the quality of the hotel booking at our company offsite there can be little reason (and rightly so) for a more in-depth measurement of our efforts and their impact. This affects not just what we measure but how we design our employee experiences.
Sometimes this happens because our organizations lack fluency on what impact our programs can and should meaningfully have, and it’s up to us to influence and educate accordingly. It starts with measuring the desired outcome, not just our sentiments about the tool we used.
Carpenters don’t care if we liked the hammer they chose, they care if the bookcase they created for our children's room meets our needs.
Lack of psychological safety
Even if we do have in-depth measurements of our gatherings, will people respond honestly and constructively? Or will they tell us what we want to hear? How do we trust the results?
Concerns like these are all too common and one reason we keep our surveys at the surface level focused on reactions instead of results. How we communicate, respond, and react to, and close the loop on feedback people share is one way we can make it safer for people to join us as partners instead of just participants.
Survey fatigue
Another survey? Pumping out survey after survey without purpose or purposeful follow-through is a surefire way to tire out your employees and reduce meaningful responses.
But the truth is that people don’t get survey fatigue; they get a lack of action fatigue. All talk, no action is a big no-no. This is one reason why the more meaningful our surveys, the more meaningful the change we can make. Emoji reactions need not apply.
We don’t know what good looks like
A typical post-event conversation sounds a lot like this: Eager organizer (either through a feedback form or conversation): “What did you think of the gathering”. Pleasing audience member: “Yeah (pause). I liked it ''. This description is unfortunately about as helpful as telling a mechanic that your car “doesn't run well.” This happens in part because we don’t know the right language to use or what cues and signals to look for. We just know that something is getting in the way of the outcome we want, like, a car that works.
How to create a survey template for your next gathering
Diana, a VP of People at a 200-person start-up is working on a leadership offsite and wants to design a measurement strategy. Johan, an internal communications manager seeks to improve how they survey employees on their weekly All-Hands. Here’s what they should do, step by step.
Note, surveys are one measurement tool but not the only one. In lieu of a more rigorous measurement strategy, surveys are a great start.
Define your desired effect
The crisper our definition of success, the crisper our ability to measure it.
Don’t save your survey design for when the gathering is done. Design it in tandem with the event itself. Ask yourself and your stakeholders these questions:
What will be different because of our time together?
What do we need because of the people we’re trying to affect?
How will we know we’ve succeeded? Can we tell a before and after story?
Essentially we want to start from critical incidents or examples happening in the organization and unpack how these will be different after the off-site and All Hands, and how we will know. (Do we see why measuring satisfaction with food and room set-up may not be beneficial here?).
For our All-Hands we can ask ourselves:
What level of knowledge are we trying to build?
What attitudes are we trying to improve?
What do we want employees to understand after the Town Hall? What we do want their takeaway to be?
Create and send out a pre-event survey
Once we have clarity on the 3-5 objectives we want to meaningfully effect, send a baseline pre-event survey to participants (ideally 8-10 weeks before an offsite), and be sure to communicate the context as to why you’re asking these questions.
Not only does this data help us understand what to focus on in our event but it also helps achieve buy-in by pulling in participants via their ideas and insights early on in the process. It shows the gathering is being done with them, instead of at them which leads to more engagement.
For example, if the desired effect of the off-site or all-hands is to align employees on the mission and strategy of the organization, ask anonymous questions such as:
How comfortable are you in describing the current vision and strategy of the organization?
To what extent do you believe we are behaving in a way that is congruent with our stated vision?
If 90% of participants indicate they are already comfortable describing the current vision and strategy that tells us perhaps we can focus our efforts on different or adjacent objectives.
Be sure to include open-ended survey questions like “what would you like to see us focus on at our upcoming off-site or weekly All Hands?”. Consider adding multiple-choice options to help make these questions less abstract.
Communicate the feedback
Begin your off-site or All Hands by sharing a high-level summary of the feedback you received and how you have (or soon will) implemented what you heard.
For example, if 50% of respondents shared that they wanted to hear from Leadership on international strategy point to the agenda changes you’ve made to reflect this feedback. Build trust by transparently sharing the feedback you received and appreciated but do not plan to implement at this time.
Send a post-event survey and communicate the results
Rinse and repeat your pre-survey within 48 hours of the gathering and communicate what you heard to respondents. See what goals you moved the needle on and what needs more attention. For example, you may find that the percentage of employees who are comfortable describing the current vision and strategy rose from 90% to 97%.
Together with the pre-survey, qualitative feedback, conversations, and other business metrics (for example the number of new inter-department projects formed), you’re now prepared to tell a meaningful data-driven story about the impact of your work.
Gathering measurements should be based on results instead of reactions
Initial excitement about a gathering doesn’t always translate to sustained engagement. In fact, they can create false positives.
They happen when we are too quick to judge a gathering as successful based on initial reaction instead of the desired outcome. “Yes, we did it!”. “Good, they liked it”. “Check”! In the short term, they seem successful. Then, engagement can drop when people don’t know how or if they can apply what was shared outside of the context of the gathering.
That’s why it's important to not conflate reactions with results and to learn to ask about the outcome instead of the tool itself.
Positive reactions are nice, but lasting transformational success is measured and diagnosed less in the immediate reaction and more in what people did or didn’t do afterward. When our leadership teams come to us to ask, “was it worth it?”, we want to not just explain yes or no, but demonstrate impact with data.
Lindsey Caplan is a screenwriter turned organizational psychologist who helps HR & business leaders create experiences that boost motivation, engagement, and performance