Emotion. Surprise. Beauty. These elements of sensation are what make design compelling in immersive experiences. This week’s guest is Ellen Lupton, a curator, educator, designer, and expert on multi-sensory design. In this episode, Ellen discusses her design approach to influence emotions, behavior, and memory. So whether you’re a designer, a business owner, or simply someone interested in learning more about creating engaging and captivating experiences, this episode is for you.

Feelings aren't Facts with Joe Federbush
This week on Matters of Experience, we chat KPIs to ROIs in a way that won’t make you ZZZ. Metrics maestro, Joe Federbush, President & Chief Strategist of EVOLIO Marketing, joins the podcast to discuss humanizing data, measuring emotion, and searching for sentiment to help brands develop the most impactful experiences.
About Joe Federbush
Show notes
Transcript
[Music]
Abby: Welcome to Matters of Experience, a podcast that explores the creativity, innovation, and psychology driving designed experiences and encounters. If you’re new here, welcome. We’re happy to have you, and to our regular listeners, thank you for tuning in again. A brief reminder, my name is Abigail Honor.
Brenda: Hello, everybody. This is Brenda Cowan.
Abby: Today, we’re speaking with Joe Federbush, President and Chief Strategist of EVOLIO Marketing. Joe works with the entire event ecosystem plus partners with experiential agencies, organizers, and associations. Joe is most known for his expertise in measuring event success, data storytelling, and helping brands develop the most impactful experiences, all with one goal in mind, providing actionable insights that help brands deliver greater return on experience, investment, and objectives. I sound a bit like a walking ad for you. I clearly love your company and everything you do. Joe, welcome to our show.
Joe: Thanks for having me. I’m excited to be here.
Brenda: Joe, could you tell us a little bit more specifically about what it is that you do? How do you go about doing your work?
Joe: Yeah, I mean, let’s face it, experiences, events, you know, they’re, not a lot of money is being thrown at them these days, we’re fighting for budget, you know, we’re fighting for attention in time with other marketing mediums and activities. So what we really do is those who are creating any type of experience, whether it’s in a museum, whether it’s at a convention, in a park, you name it, what we really want to do is just help them measure success by identifying KPIs, key performance indicators that are going to be the most important metrics to help them understand are they meeting, hopefully exceeding their objectives for that investment, for that brand, for that experience.
Abby: Wow, that’s cool. So, Joe, one of our focuses is storytelling, whatever it is we’re designing. So, I heard that notion of data storytelling, and it sort of piqued my interest. So, what specifically is that, and why is it important?
Joe: Yeah, data storytelling is extremely important, and it kind of plays off of your January 11th, 2023, episode about design experiences in storytelling, right? Now, we look at it the same way, but with numbers, basically. And you know, we’re inundated with data these days, especially when we went to virtual because everything was being captured. It almost did like a 180. Prior to the pandemic, there was not enough data or not enough good data. So now you have, you know, the whole challenge of rudimentary data, too much data. But, like, what’s compelling? What does it mean? What kind of decisions are they going to make from it? So, we look at it is whether it’s surveys, whether it’s, you know, digital data, you know, just what does it all mean? And most importantly, back to your point about the storytelling, what are we going to do with this data?
Actually, a good friend of mine, Victor Torregroza, he’s like an experiential expert at Intel. Years ago, he told me why you helped me so much is because you humanize data. You don’t need me to tell you how to read a pie chart, right? It’s about what are we doing with this information. Give you a perfect example, what seems like rudimentary data upfront when you look at like basic demographic of who’s coming to your event or experience, etc… You know, we look at it as, okay, well, what percentage are executives or what percentage are technical or tactical? Because your delivery has to really align with the audience, with your visitors, with their viewers.
So when you’re talking, alright, 60% of your audience is, you know, say, executive C-suite people, you want to make sure that you’re telling the story about how you can help their business versus, like if you’re talking to like technical or tactical people, you want to maybe show more product features and benefits.
Brenda: You know, I would love to play off of the, the human element that you were just mentioning and think about data collection. So, in my line of work, I’m pretty familiar with qualitative data, collecting focus groups, interviews, and all that kind of thing, and could you share a little bit, what does qualitative data collection look like for you?
Joe: Sure. So for us, you know, we are a full-service market research firm, and so when we look at qualitative, there are several aspects like you mentioned, focus groups, in-depth interviews, but now there’s also amazing AI tools that will take verbatim comments, whether it’s audio or it’s text and, really group it, combine it, code it into the right categories so you can start taking qualitative and actually convert it to quantitative to some degree, of course, depending on the volume of data.
But when we look at qualitative, you know, it’s just it’s getting that like emotion and that sentiment that you’re not going to necessarily get from a statistic. It’s that human connection and emotion.
Brenda: Listeners, you just missed two really alarmed or shocked or amazed expressions on Abby’s and my face when Joe started talking about AI. That’s incredible.
Joe: Yeah, we’re just scratching the surface now, and it’s mind-blowing. It saves and shaves so much hours of work, you know, which either A helps your margins or B, lowers your price so the clients get more value.
Abby: Are you worried you won’t have a job?
Joe: That’s the fear, right?
Abby: See, you didn’t even address that question! That was really slick. He’s like, I’m always going to be here, Abby.
Brenda: I’m not afraid, AI. Bring it on.
Joe: What I’ve learned about AI, the beauty of it is the information has to exist first. Allegedly.
Brenda: Well, but, and you were just talking about the emotional element and the human element, I don’t, I do not see how it is that you could possibly, when we’re talking about data collection in particular and when we’re talking about emotion, and when we’re talking about, you know, the nature of experience, I’m not concerned about AI sort of supplanting that human element.
Joe: I’m not yet either.
Brenda: Yet.
Abby: Yeah.
Brenda: Listeners, you heard that yet, right?
Abby: So when you think about an organization, how do they determine what they value, and how do you sort of quantify that? You know, what are the kinds of conversations you have to help people get at really defining or describing what value means to them?
Joe: Yeah, that is from my experience, in my opinion, that’s like one of the biggest parts of the project, actually. It’s not how to write a questionnaire or how to conduct the interviews. It’s literally all that due diligence and upfront work that has to be conducted to really get inside your stakeholders, your customers, and even their customers’ heads to understand what’s most important and what’s actionable.
You know, a lot of the time clients will come to us, for example, and they have a questionnaire already made, and you know, we kind of snicker at it like, what is this? What are you going to do with this information? So we kind of reverse engineer the process, looking at the end and working backwards. What decisions are you going to make based on, you know, this, this measurement research, etc.? Then let’s design A, the methodology to your point. You know, whether it’s a focus group, maybe it’s a survey, maybe it’s observations, maybe it’s behavioral tracking, maybe it’s social media. There’s so many data sources. What are the best data sources going to be to actually capture the right kinds of actionable information and help our clients really make true business decisions from?
Abby: What I was taught in school when I was in high school doing statistics.
Joe: Couple years ago.
Abby: Oh, I love you, Joe, was that, you know, you can sit there, you can look at a statistic, but it can be interpreted many different ways. So how do you make sure that you’ve asked the right question and you’re interpreting it in the right way?
Joe: Yeah, absolutely, and that’s why, you know, I was saying before about, like it always starts with your objectives and your KPIs so that you don’t really go down a wrong path. I mean, I love it actually a lot of the time when the data, you know, sheds light on something you didn’t think about. But the point is, you know, you don’t want to use data in a dangerous way. You know, it shouldn’t be malicious. Of course, we all go into it hoping for a certain hypothesis or a certain outcome, but really more, it’s about understanding how did we get there? Did we get there, yes or no? What worked, what didn’t? And let’s continuously improve so that the data does get us to where we want to be for continuous improvement.
It’s rare that anyone’s going to get it right the first time and then just be able to like use that as their BKM, their best-known method and just keep repeating, repeating, repeating because things change. And I think, you know, prior to the pandemic, the economy was good, the event industry was exploding, you know, so we were a little, getting a little complacent. Some, not all, of course, but you know, we had a real rude awakening and now with the significant increase in costs for experiential and event marketing, we all have to be smarter, you know? So, we do need data more than we’ve ever needed it before, to your point, Abby, to make those really strong correct decisions and not try to twist the data to do something that it’s really not saying.
Abby: Does it ever put you in conflict? If you’re just responsible for the data and I use the word small J. If you’re doing all data and you’re obviously analyzing somebody else’s work, right, somebody created this event. There’s a whole team or teams of vendors. Are you ever in conflict?
Joe: Unfortunately, yes. Fortunately, unfortunately, however you want to look at it. But the point is it’s not going to fix itself, right, so you may as well identify it when hopefully, it’s just a small problem before it becomes a much larger problem. But there has definitely been instances where, you know, it’s, it’s a little challenging to communicate results that aren’t so great. And that’s, you know, that’s another reason why, you know, EVOLIO’s an independent third party. So, we’re not measuring our own work, you know, so we are going to be truly unbiased in what we present.
Abby: I want your job. You come out always smelling the roses.
Joe: Be careful what you wish for.
Brenda: Well, I’m curious about when we’re measuring success and along the lines of what we’re talking about, companies need to make sure that there’s a return on investment, that they’re spending their money wisely. And I’m curious to know what are the pieces that need to be taken care of, attended to, in order to make sure that the ROI is healthy.
Joe: So our model or framework or methods to help get there are to really understand, you know, first start with again, what are you trying to accomplish? You know, so many events and experiences are designed around different factors and different goals, like whether it’s sales, whether it’s education, whether it’s information, whether it’s, you know, societal things like DEI and sustainability. You know, we see it all the time where we want to measure our ROI, and I have to ask most times, well, how do you define ROI?
Brenda: Sure.
Joe: And a lot of the time, their definition of ROI is not even really about ROI. It’s about ROX, the return on experience, the return on emotion, ROE, you know, the things that we, that you mentioned in the introduction, and then we find out, well, you know if you’re not going to deliver a great experience, why would you expect ROI? So I try to convince a lot of the people that we work with, let’s not start with ROI and look at that as the end result. Let’s look at that as the next phase, making sure that we’re truly delivering great experiences, connecting with the right people in the right way.
Brenda: So you said one of my favorite words, which is emotion, and I would love for you to just give us a quick definition. When you’re talking about return on emotion, what’s in your thinking with that one?
Joe: Yeah. So, couple of ways to go about it because everyone’s always like, oh, that’s intangible, you can’t measure emotion, and there’s so many ways you truly can. It starts with sentiment. And you know, if you think of it from like a survey question point of view, if you do like an intercept survey as people are leaving a museum or an event or activity, say, hey, I would love to ask you some anonymous questions about what you thought about the event. And you have very specific messages designed that they could rate their level of agreement with. And of course, you want to make sure those sentiment statements are not biased. So how happy did this event make you? That’s not the way you ask the question. You know, how do you feel towards this event? Did it make you, you know, on a scale, say, of, you know, excited to bored? And then, obviously the higher end of the scale is where it’s about the positive emotion.
That’s the easiest way to measure return on emotion, in my opinion. Now, the beauty of it is there’s, kind of going back to AI and some other tools, there’s cameras and things that really capture sentiment from like recognition from your biologics, and that can track your, your expressions on your face. And we’re starting to see those now being used at events as well. And the cool thing about that is that it’s capturing information of people walking by, walking by, walking by, stopped, looked, came in, checked it out. Cameras facing them, they could see their like, reactions, and it’s truly now generating the most accurate levels of return on emotion you can get.
Abby: You mentioned a good experience and talking about measuring it, but what is a good experience? What makes a good experience for a visitor?
Joe: Yeah, you know, first is connecting, you know, having that true organic natural connection where it’s not forced, you know, you’re not forcing people into a room to do something that they may or may not feel comfortable doing, seeing being around too many people at the same place, at the same time, obviously after the last couple of years, what we’ve all been through. So what makes a good experience is really like attracting and engaging with the right people, but having the engagers be truly engaging, you know, not to beat up the word, but, you know, so many times we walk into experiences that are designed so impeccably and beautifully, and then it falls flat because the people working it are like, not that excited to be there.
So you know, you’re halfway there, and then I think to get full-way there is, yes, you design something that connects with the right people that you’re trying to attract and engage, and then having that kind of support staff, that team, those ambassadors, whatever the case may be, that really know how to walk you through and treat you as if you’re having a VIP experience. That, to me, is the perfect experience. Now, you know, when you work backwards from that, it’s okay, well, who are the right people, and what do they like to see? And it’s not one size fits all. So you have to make sure that, you know, you’re not setting false expectations of what it is you’re visiting.
Brenda: Joe, I’m particularly curious to know about the arena of social impact and how it is that social impact and social action is being embraced by brands. And I’d love to know: how do you go about measuring whether or not there’s an awareness that social action has truly been prompted?
Joe: It’s huge right now, and I’ll start with those who are not yet doing it. We encourage them to look at ways of measuring, you know, societal metrics versus just business metrics. DEI and even DEIB. If you’ve heard of DEIB, the B is belonging, and this is something I think is so important, especially when you look at it from like an experiential element. If someone who’s not white walks into a room of white people and they just, it’s homogenized, you know, that’s not a good look for that event, for that experience, for that brand. You know, even when we look at it from like a speakers perspective, if you’re at a conference and there’s multiple speakers, multiple tracks, you name it, there has to be diversity with the speakers. It can’t always be equal, but the point is there has to be that introduction that, you know, it just can’t be this homogenized event. And you know, same thing on the sustainability side, you know, especially in the event world. Let’s talk about trade shows for a second, you know, there’s a lot of churn and burn, a lot of carpet, a lot of, fortunately, a lot of recycling, but, you know, reducing carbon footprints in events is extremely important more than it’s ever been because it’s got to go somewhere and it can’t keep going to dumps.
Abby: Now, that’s true. I completely agree with that. Even in terms of all of the design builds we do for all these events. They have to be recyclable. They have to be reused. They have to have that flexibility where especially when you got to take care of the environment. Yeah, I completely agree.
Joe: I mean, in some countries are still 100% build and burn. You know, they haven’t even thought about it yet. And that’s scary.
Brenda: Oh, that is, that is crazy scary.
Joe: Yeah.
Brenda: You know, I’m really curious to know, are there things that we’re talking a lot about, you know, what should be measured and how it could be measured, and I’m curious to hear, what should not be measured?
Joe: Yeah, I think that’s a great question.
Brenda: Where do you draw the line?
Joe: Yeah.
Brenda: Or where should we know to draw the line?
Joe: Yeah, I think, like what shouldn’t be measured are things that are obnoxious and pushy, you know. So, when are you going to buy this? You know, like, and a lot of it, it’s the delivery of how you’re asking. So, you know, how likely would you be to buy this product in the next X number of months is a nice way to say, will you buy this or not? You know, collecting the level of information you’re collecting about people, like personal information, that’s always a turnoff. So it’s like, let it be more of a natural opt-in process. Like, so if I’m answering a survey, that survey should be anonymous because if you want unbiased responses, you don’t want to start off by asking, you know who I am and what’s my email and what’s my job title, what’s my income, and how many kids do I have? And like, you know, like that’s a whole other story. But, you know, you could save those demographics for the end, but just for classification purposes. So we know, am I talking to a Gen Z or am I talking to a boomer because they’re obviously very different types of people, some of them, most of them, not me, but I’m in that gray area between. But yeah, so things that are just aren’t pushy or invasive.
Abby: So I really believe in making mistakes because without making mistakes, you can’t learn, and without learning, there’s no growth. So tell us some mistakes, some horror stories you’ve had along the way, and also some mistakes that clients often make so that anybody listening can try to avoid them.
Joe: You know, and I look through, like, all throughout my career, and I’ve been involved in, you know, event measurement for a pretty long time. And I look back, and it’s not even like those mistakes were always like 20 years ago, you know, just some even more recent where we let the client kind of dictate how to measure. So, there’s been instances where I’ve kicked myself, allowing the client to dictate to us what to do and how to do it. You want to, of course, get the business to help the client. So sometimes, you’ll sacrifice budget to get it done, and then you find yourself taking just a huge hit or a huge loss on the project. And I look at those as the biggest mistakes because then you’re setting the precedent for any future work where they’re like, oh, well, you did it this way last time, and you know, so then you’re kind of digging your own grave, so to speak.
Abby: Yeah, totally relate. Completely understand.
Brenda: Well, we would love to hear what you think in your own personal work, what you think is most successful about what you do.
Joe: I love what I do every day. And so, this is an easy question to answer, fortunately.
Brenda: Softball.
Joe: Yeah, right. Thank you. Thank you for the softball.
Brenda: A soft pitch. Is that what it is?
Joe: Yeah. So, I mean, I just love the fact that, especially if we look at more in the short term compared to the long term, but it’s always been the case. Budgets for event marketing and experiential marketing is always tight. And the fact that we can provide actionable data that helps our clients truly make decisions, it’s such a joy.
Actually, just yesterday, we were on a call with a client, and they did something very different with their experience, and they were really kind of sticking their neck out. We hope this works. So, they had the data because we did the research on-site post-show. The data supported that bringing all their brands together was an extremely successful experience, not only for them but for their customers. So in the past, they had like four different brands in four different locations. So the fact that A, they saved money by consolidating, B, they had more cohesiveness between their brands but still had their unique brand identity for their brands. But they had the data now to say, we did it, it worked, but it wasn’t 100% perfect. And then they realized, Wow, you know, next year we’re going to change this, and we’re going to do this more efficiently. So, it’s more like modifying and tweaking versus like, do we need to reinvent the wheel again?
Brenda: What are some of the lofty goals that you have for yourself for the next few years?
Joe: Yeah, I mean, for, for myself, you know, it’s just really grow the business and be the industry standard when it comes to measurement. You know, we’re working now on building experiential online, in-person, event B2B, B2C benchmarks, for example. So we’ve literally just compiled over 400,000 surveys that we’ve conducted since 2017. So over half a million responses now to create benchmarks for all of those normalized KPIs that a lot of clients want to know, but then, more importantly, be able to filter it between age and gender and geography and event type in industry and things like that. So that’s what we’re really looking forward to continue growing and expanding so that there truly is like normalized data out there so people can really feel good about, if we exceeded the industry benchmarks, congratulations if we’re close to it, congratulations. But if we’re under it, what are we going to need to do to improve our performance and outcomes so that we are exceeding benchmarks?
Brenda: So if you’re looking at years since 2017, I’m curious, what about the impact of COVID? How do you sort of take that into account when you’re looking at longitudinal data analysis such as that?
Joe: Yeah, and that’s a big piece of that segmentation with the benchmarks is, you know, the glory days prior to 2020. Of course, the 2020 through 2021 and a half where it was like, oh, and then now that things are rebounding because, you know, from 2021 when things really started coming back to in-person pretty strong till today, you know, there’s still that momentum that’s being rebuilt. You know, a lot of events, event organizers, associations, museums, you name it, they’re trying to get those bodies back. And now that, you know, social distancing is kind of being put aside, and we can get more bodies back together and not have the limitations of capacity limits and things like that, so we’re starting to see that rebound, and it’s rebounding pretty quick. I mean, I anticipate unless something drastic, God forbid, happens, I anticipate that early 2024 we’re going to be back to normal, whatever back to normal means. But, you know, we’re not going to be in any kind of state of fear, hopefully. But, you know, I speak with some people, and they’re like, oh, it’s not going to be for like two or three more years. I speak with other people, and they’re like it was yesterday, and it depends on your industry, of course, you know, but I think for, collectively and most globally, not all, but I think we will be in a good spot in probably like 9 to 12 more months.
Abby: So thinking about, you touched earlier on the changes that we saw happen due to COVID. How many of them are good and will stick around, and how many do you hope we’ll never see again?
Joe: So, great for QR codes, let’s call it, let’s say that.
Abby: That’s true.
Joe: Love it, digital payments, you know, things just connecting digitally.
Abby: Yeah.
Joe: But understanding where connecting digitally has its limits, you know, networking and all these things that were trying to be done digitally. Very, very difficult. And during the pandemic, we had measured literally hundreds of events, and, you know, it’s great for brand opportunities, excellent for education.
But from that building the connections and networking, you know, no matter how many matchmaking tools and things you tried and did, there were very, very few that were actually successful. I think building community so an event isn’t just three days, say, like now we, we, we understand how to make an event longer for the digital part of it. But yeah, the networking, connecting part, it just fell flat and hopefully, we never have to try to see it again.
Abby: Good. So, in-person still wins.
Brenda: Face to face.
Abby: People want to be with each other.
Joe: Without a doubt.
Brenda: I just remember the conversations back in 20, like 2012, 2013, 2014. Everybody was like, oh, we’ve got to go digital. We don’t need all this expense of face-to-face and, argh, getting people together, it’s so complicated.
Joe: Yeah.
Brenda: Well, here we are.
Joe: Yeah.
Brenda: We want to see each other in person.
Joe: Yeah.
Brenda: I find that very optimistic.
Joe: Hugely optimistic. And you know, digital has its place. We find it’s great for, like, keynotes and, you know, press launches and things like that, so, you know, for, for the masses. But for those that really want that connectivity and that build those relationships, it’s just so hard online. Yeah.
Abby: That intimacy. So three things you love about your job, Joe.
Joe: I love helping my clients with data versus, what is it, feelings aren’t facts. That’s my favorite line to my clients. Feelings aren’t facts.
Abby: I’m stealing that. I’m going to totally steal that.
Joe: Please steal it. I’ll put a little TM over it.
Brenda: TM.
Joe: Feelings are not facts. That’s my number one favorite thing. My number two favorite thing is owning a company and just having a team of employees that I truly love and care about, and I will do anything for them. And number three is just the hurdles. I love the hurdles. You know, everything can’t be easy all the time. And we’ve gone through some hurdles in, whether it’s education or experiential agencies. They only make us smarter and stronger.
Abby: Would you say then, because you’re clearly an entrepreneur, you own your own company, you’re running it very well? It’s growing year after year. Do you think it’s part of who you are? Wanting to be heading a team, wanting to be successful, and also wanting to be challenged? I think it’s like core to what I’ve seen in serial entrepreneurs is that they thrive off it. As soon as everything’s working, it’s boring.
Joe: Yeah. Oh, my employees hate that about me. When it’s working well, then it’s time to change it.
Brenda: Evolve. Let’s use the word evolve.
Joe: You know, that’s funny because EVOLIO literally means evolving your event portfolio, that’s, or experiential portfolio. So that’s where it came from, EVOLIO. It’s about evolution and constantly evolving. So, thank you for that.
Brenda: You betcha. I’m here for you, Joe. I’ve got your back.
Joe: You’ve got my back.
Abby: Yeah, well, you know, in some of my research, before we chatted to you, I did a few surveys.
Joe: There you go.
Abby: And one of them actually gave me a $5 Amazon gift card.
Joe: Only five?
Abby: I thought that was pretty good.
Joe: Oh, no. We do ten.
Abby: Really? Send me some surveys!
Joe: Yeah, between five and ten.
Abby: Is that the norm?
Joe: Yeah, yeah.
Brenda: Look at that. I don’t even like to check the box that says I agree to terms.
Joe: I know, right?
Brenda: That would have been like 10 hours of my time right there, which is how do I feel about these terms.
Joe: Exactly.
Brenda: You know, it took me ten years to get married, so I guess that you know.
Joe: Good things come to those who wait.
Brenda: Yeah, well, that’s for sure. That’s for sure.
Abby: I’ve learned so much today, Joe, from you. It’s incredible. Now I even know what data storytelling is.
Joe: All right!
Brenda: Oh my goodness, Joe, thank you so much for everything.
Joe: Thank you for having me.
Brenda: The work that you’re doing is amazing.
Joe: Well, thank you. And you guys as well. You just bring in some brilliant minds with brilliant people and your perspectives to it, and so thank you for having me.
Abby: Oh, thank you, thank you.
Brenda: Thank you.
Abby: And thanks, everyone for listening. If you like what you heard today, subscribe for more episodes of Matters of Experience wherever you listen to podcasts. We’ll see you next time.
[Music]
Producer: Matters of Experience is produced by Lorem Ipsum Corp and recorded at Hangar Studios. Tune in next time for more fun discussions about experience design.


