Kevin Lyons
CEO & Co-Founder
FlashVote
LinkedIn
Pizza, beer, & public input. Kevin Lyons, CEO & Co-Founder of FlashVote, joins the podcast to discuss community engagement. He highlights common mistakes made with various types of communication channels as well as mistakes organizations make in town halls, social media, online forums, and surveys. Kevin also shares with resources to improve community engagement.
Host: Alyssa Dinberg
| Subscribe: |  |
 |
 |
 |
Learn More
We learned how (and how quickly) online engagement goes bad
Why is Community Engagement Important?
8 tips to improve Community Engagement in Local Government
North Andover Seeking Public Input On Stevens Estate
Considering the future of recycling
Council forgoes referendum on Plymouth Creek Center expansion project, continues with design process
Citrus Heights Launches Flashvote: An Easy Tool For Ongoing Civic Engagement
Episode Transcription
Alyssa Dinberg 00:10
Coming to you from Denver, Colorado, this is Gov Love, a podcast about local government. Gov Love is produced by ELGL, the Engaging Local Government Leaders network, we engage the brightest minds in local government. I’m Alyssa Dinberg, Recovery Coordinator for Clear Creek County and today I’m joined by Kevin Lyons, Co-Founder and Chief Scientist at Governance Science Groups and Co-Inventor of Flash Votes, which gives states and local governments statistically valid community input in 48 hours so they can make great data driven decisions all year long. Flash vote is changing the community engagement market by providing a scientific way to survey and engage communities that produces immediate and timely results. Welcome to Gov Love, Kevin.
Kevin Lyons 00:54
Thanks a lot for having me on. Yeah.
Alyssa Dinberg 00:57
We’re super excited to have you here. I have been hearing a lot about you. So I’m excited to learn more about your input on public engagement.
Kevin Lyons 01:07
Well, yeah, let’s dive in. Hopefully it’s a good thing.
Alyssa Dinberg 01:11
Oh, absolutely. Definitely good things. No, I’m just really excited because I, the listeners will hear but your your, yeah, well, I don’t want to spoil it so.
Kevin Lyons 01:22
I’m looking forward to disappointing you and the listeners.
Alyssa Dinberg 01:25
No, no. All right. Well in today’s episode, Kevin’s going to talk through the do’s and don’ts of public engagement. We’re going to learn about what’s useful, what’s useless and what’s dangerous with different communication channels. And we hope that you’ll come away with a new lens for thinking about how to get better input with less time and effort. So as we do with every episode, let’s get started with a lightning round. Are you ready?
Kevin Lyons 01:52
Oh, yeah, I forgot about that.
Alyssa Dinberg 01:54
Yeah. Okay. So um, because it’s COVID. And none of us are really doing any traveling. I have been asking pretty much everyone in my circle this, what is the most beautiful place you’ve ever traveled to?
Kevin Lyons 02:09
That’s an interesting question. It’s, it’s actually about a mile from where I am right now in Lake Tahoe. East shore of Lake Tahoe has unspoiled beaches, crystal clear water, and four feet of snow falling right now. So if you like, if you like winter, it’s a beautiful place to be.
Alyssa Dinberg 02:28
That is a part of the country that I’ve never been. And it is probably like, number one on my list for in America. So hopefully I can get out there sometime soon.
Kevin Lyons 02:38
Be warned. There are a lot of people I’ve met here who came here to visit never left.
Alyssa Dinberg 02:45
I’ve been told that about Colorado as well.
Kevin Lyons 02:47
well, you’re you’re one of those places. Yeah, I have several classmates, engineering classmates who went to Vail and never left.
Alyssa Dinberg 02:54
Yeah, and it is, definitely catches you. You get the bug, can’t leave.
Kevin Lyons 03:00
Excellent.
Alyssa Dinberg 03:02
All right. So next question. What was your favorite toy growing up?
Kevin Lyons 03:08
Yeah, so I think I probably was on the engineer track early. But I gotta go with like Lincoln Logs and erector sets. Those are pretty, pretty awesome.
Alyssa Dinberg 03:20
I love toys like that. I always loved building things. I remember like, when I was super little, my mom was throwing a watch away. And I was like, No, I want it. And I remember taking it apart and putting it back together. And I was just so proud of myself. It was awesome.
Kevin Lyons 03:33
That’s awesome. Yeah. My daughter is super into legos. We just got her some something for Christmas as like 2400 pieces. And yeah, it’s, it’s fun.
Alyssa Dinberg 03:45
That’s super fun. Um, what would you do if you could be invisible for a day?
Kevin Lyons 03:53
That is something I was just talking about with someone else today. I think that the right the right answer is I would actually be a fly on the wall and hear the real conversations going on between politicians. Yeah, different different topics and issues. You know, what’s really going on behind the scenes? That would be fascinating to me.
Alyssa Dinberg 04:14
I agree. I think a lot of people feel that way. Because it’s hard to know what’s really happening.
Kevin Lyons 04:20
Yeah, you have the sort of the public public statement, public view. And then what’s actually going on behind the scenes, which is something I’ve been interested in for 30 year for 30 years, basically.
Alyssa Dinberg 04:30
Yeah, I agree with that one. So this next question is a really funny one to me. I’ve never asked anyone this, but I think you’re up to the challenge. It’s something that I was just talking to my boyfriend about, because we have random conversations like this. What is your favorite smell in the whole world?
Kevin Lyons 04:51
Good. Actually, today, I was just out sledding with my daughter. I gotta go with hot chocolate with marshmallows in it.
Alyssa Dinberg 05:02
That’s a good one.
Kevin Lyons 05:03
That is pretty good. Any other season it would be, it would be some kind of barbecue smoke. That would be awesome. You’re making me hungry though.
Alyssa Dinberg 05:14
I know.
Kevin Lyons 05:15
Hopefully this one isn’t about food.
Alyssa Dinberg 05:18
It is.
Kevin Lyons 05:20
Excellent.
Alyssa Dinberg 05:21
Yeah, I ask this in every episode, I’m kind of known for it with the Gov Love followers. If you were a vegetable, what would we be and why?
Kevin Lyons 05:32
Oh, God. I don’t eat a lot of vegetables. But I gotta go, that’s the barbecue, I’ll go with carrots. Kind of kind of tastes good, kind of underrated. It’s supposed to help you see better even. I could go with that.
Alyssa Dinberg 05:51
That’s one of those vegetables that I can just eat and not even really think about but then like, feel good that I’m eating it because it’s a vegetable. It’s kind of like a chip. You just might mindlessly eat chips. Like I could do that with carrots.
Kevin Lyons 06:02
That’s right. You can generally eat carrots, exactly, like little ones. We get the little ones and I just kind of pop them while I’m watching football or something. You know.
Alyssa Dinberg 06:11
Yeah, my dog is also obsessed with carrots.
Kevin Lyons 06:14
Oh, well, there you go. probably good for them too, we hope.
Alyssa Dinberg 06:16
Yeah. Well, that’s a good answer. Okay, good. Well, thank you for participating in the lightning round. I appreciate it.
Kevin Lyons 06:25
Good questions, I think. I don’t know what it revealed about me, but we’ll see.
Alyssa Dinberg 06:29
No, that’s okay. Um, if you are ready, let’s dive into public inputs. So, first, why why are you here talking to us about the topic of public input?
Kevin Lyons 06:45
Um, you know, it’s it’s an interesting question, I, been a long and winding road to get here. But over the last year, I started teaching public input, it started with the great program out of TCU, called the certified public communicator program. And I’ve gotten to know the director there. And she asked me to teach a class on on the science of public input, we kind of do a lot of things. And maybe we just kind of do them because we see other people doing them. And we’d actually broken down the whole thing at some point, I made it into a one hour class. And I’m actually teaching again on Friday, that led to actually a bunch of invitations, including some keynote talks to to teach the same thing. And I think it’s a pretty dry topic on the surface. But I think we’ve been able to figure out enough ways to explain it in terms of pizza and beer. That is interesting to people. And, and also really helpful, I think, if especially if they’re a communicator, or they have to deal with the public or attend public meetings, or anything else you can imagine. Look at a Facebook page every once in a while even or next door.
Alyssa Dinberg 07:57
So pizza and beer, can you tell me a little bit about that?
Kevin Lyons 08:02
Yeah, we’re back to food, which is awesome.
Alyssa Dinberg 08:04
I know, I know. And it’s dinnertime. So I’m hungry.
Kevin Lyons 08:06
We’re recording this right before dinner. It’s gonna be so rewarding to actually bite into some carrots later. And yeah, yeah, certainly. The the pizza stories I’ve been testing for a while and how to how to explain things about data, kind of more complicated concepts. And it turns out that one of the most, I guess, well tested stories is that, you know, if I say to you, we’ll role play here. Okay, so there’s 10,000 people, they’re all going to pitch in five bucks to get pizza. And your job, we’re going to pick you you have to order the toppings. Oh, gosh, that’s your job. Right? So what are you going to do? What are you going to do to figure out what topics to get?
Alyssa Dinberg 08:56
Like, you want me to answer this?
Kevin Lyons 08:57
Yeah, let’s try it, go ahead.
Alyssa Dinberg 08:59
Um, if I had to order pizza for 10,000 people, I would do a mix of both just plain cheese and meat. And I would definitely have some meat because we’re gonna have meat lovers. Okay. So like a lot of cheese a lot of pepperoni. A handful of like the supreme and a handfull of vegetable.
Kevin Lyons 09:21
Right. So you’d go with the, we call it the the guessing approach, right? Sort of, you guess and you kind of go with-
Alyssa Dinberg 09:29
Yeah, because that’s a lot of people.
Kevin Lyons 09:30
Well, and you go with some kind of lowest common denominator, right? You know, everyone eats cheese of whatever and you end up with kind of people are like eh, pizza is, okay, but I didn’t really get what I wanted or, you know, that whole tons of people maybe had a favorite topic they didn’t get. So that’s, that’s kind of the the no input approach. And obviously, you can do a lot better than that. Sometimes it can work out just fine. In that case, you already have a pretty good idea from real experiences, what people actually, you know, like or don’t like. Then there’s the input approaches. And if I say okay, well, how are you going to, suppose you’re gonna want to want to get some input, you want to engage the pizza people? What if you have a, let’s have a public meeting, and we’re going to announce it, it’s going to be at five o’clock on Tuesday, and people are going to come and talk about what pizza toppings to get, what do you think’s gonna happen in that case?
Alyssa Dinberg 10:30
You’re gonna get a lot of different answers, and it’s gonna be a big pain in the butt to order the pizza. And it’s probably going to be expensive, because you’re gonna have some, like, really bougey orders.
Kevin Lyons 10:41
That’s it. Yeah. And, and what else might happen? Perhaps the the Friends of pineapple pizza, might rally their, their, their crew, and then you’re sitting at the meeting, and then like, 90% of people are coming up and saying, We need pineapple pizza, you got to order pineapple pizza. And, you know, at least one of the elected officials is going to be taking notes like, oh, wow, nine out of 10 people want pineapple pizza, we got it, we got to do that. And and then you’re making quote unquote, data driven decisions, and you get the pizza, and then everyone hates it right? Or you, maybe, do you even know? Maybe you’re in a community that actually is 90%, pineapple lovers.
Alyssa Dinberg 11:23
Right.
Kevin Lyons 11:24
So that’s doesn’t sound like a great idea, right? So then you go, Ah, here’s what we’ll do, we’ll put something online, we’ll do a forum. And actually, you might get 10 times as many people. So you go from two to 20, or from 10 to 100. And maybe now the anchovy people happen to have an online forum. And so they’re going to be the ones that organize, or you put out an online survey. And they find it, they take it because they’re interested in the topic. And then they share it with their friends in the anchovy circles. And you come back and your data says 90% of people want anchovy pizza. That’s all you have right, it’s your only data. Yeah, so you’re in a tough situation where you know, without anything else, if you go that kind of route, you know, what are you going to do have a meeting and then tell people go pound sand, you know, or do some online engagement, and then completely ignore it? Because you know, it’s wrong. Yeah. So yeah, it’s, you get a lot of these tricky situations, and you kind of make it worse by doing certain types of online engagement. If you are using individual input, individual points of feedback to try to extrapolate, you know, try to imagine or create this community that, what the community wants. So, ultimately, there’s a right answer, the best answer is to take everyone’s order, individually, right? aggregate it, and then take advantage of those great, you know, savings by doing a bulk order, and ever gets pizza they want. There’s a second best solution though, can you guess what that is?
Alyssa Dinberg 13:08
Order one kind of pizza and don’t ask anyone what they want.
Kevin Lyons 13:12
And just deal with all the backlash. People bring, have people bring their own toppings? There’s that’s actually it really is a possibility. Well, I mean, that’s kind of the idea of government, right? It’s like we try to make it operate at the level that everyone gets value from it. So maybe in that case, you know, the idea is set up a pizza oven, bring your own dough, if you want the gluten free, the non, different cheeses, whatever. But yeah, at some level, there’s an optimum there. But the in terms of the data on pizza toppings, a scientific survey would do the job. So that’s, that’s actually we have there is data on that on America’s least favorite pizza topping for example, which is take a guess.
Alyssa Dinberg 13:56
Pineapple.
Kevin Lyons 13:57
That’s a good guess. That’s my guess.
Alyssa Dinberg 13:59
I am team, no, I am team pineapple. And that is a very controversial feeling. But I am very much team pineapple. Is it, if I had to guess, maybe anchovies?
Kevin Lyons 14:09
Anchovies, yep. America’s least favorite pizza topping. I mean, you don’t really see an offer in the West. It’s kind of an East Coast thing. Have you ever had corn on pizza?
Alyssa Dinberg 14:19
Yeah, it’s so good!
Kevin Lyons 14:20
Yeah. It’s totally underrated. I has it in Japan, and you just look at it like what is this? And it’s pretty good.
Alyssa Dinberg 14:28
Have you ever had string cheese on pizza?
Kevin Lyons 14:31
No, no, I could see that going, huh?
Alyssa Dinberg 14:35
It’s delicious.
Kevin Lyons 14:36
Okay, we’ll give that a shot. Yeah. Yeah. So so the pizza is an intuitive way to think about what’s actually a pretty complicated data problem and even human behavior challenge. How do you get, you know the right input from the right people? And so, that’s one of the little bits if you will, that people seem to like and they’re able to use it to explain it to others. You know, why do we need better input, different input, or input at all? And so there’s a lot of other versions of explaining things in terms of pizza, which I’m sure I’ll bring up. Whether you like it or not later.
Alyssa Dinberg 15:12
No, that’s why you’re here. I love it. I’m gonna have to order pizza for dinner.
Kevin Lyons 15:18
Yeah, I mean, where this was, I might not even wait.
Alyssa Dinberg 15:22
So your whole conversation about pizza and serving people on their topics, or toppings, brings up my next question. When do you need public input for good decisions? Because if I’m ordering pizza for an event, I’m probably not going to ask for input on what pizzas to order, especially if it’s for 10,000 people.
Kevin Lyons 15:45
If it’s a small event, yeah, you might I mean, you might. But it’s, you know, it’s, it’s a really good question. There’s, again sort of a, Oh, we should, we should get public input for everything mindset. Which is like a healthy mindset in the right direction, but if I, if I think about the pizza, the pizza again, of course. If you go to a restaurant, and you sit down, they don’t just bring you pizza, usually, they will ask you what topping you want. They might even ask you what what crust you want, you know what kind of flour you want in the dough. But they don’t kind of invite you back into the kitchen and ask you how to, you know, source the flour, or what kind of grinds you want or anything else having to do with the pizza, right? It’s kind of a, there’s a, there’s the input at which you need to tell them something in order for them to make pizza that you’re gonna like. And then beyond that, there’s that delegation to their expertise, you know, to the chef on how to make pizza. You know, what temperature set the oven and everything else. And it’s pretty unlikely that you’re going to have any kind of helpful input to offer in the kitchen, unless you happen to be a pizza expert. So there’s a lot of things like that, you know, that the common metaphor in government is the, you know, the rowing versus steering, have you heard that one? You know the council versus the staff.
Alyssa Dinberg 17:21
Yes, yeah, yeah, yeah, yep.
Kevin Lyons 17:23
It’s like everyone needs to know exactly what their job is, so that you can coordinate the activities efficiently. And there’s another, there’s another level above that, of course, in your org chart, which is the public, right? You got the public, the line down to the elected officials picked to be the representatives to act as the public, and then the line down to the you know, the managers and the staff and everything. And the part besides the rowing and steering is picking the destination. Where are we gonna go, right? So that’s, that’s the level at which the public input is really helpful. As you get into the the steering, and then down into the rowing, it’s not helpful, not super helpful, usually, although there are still cases where there are those pizza experts out there, you know, rare, of needles in haystacks that might give you some really a really interesting fact or idea or something, something that you can use to make a better decision.
Alyssa Dinberg 18:23
Do you have any government related examples of when we should not be asking the public for input?
Kevin Lyons 18:32
Ah, yeah, they usually fall under-
Alyssa Dinberg 18:34
That is frequently asked.
Kevin Lyons 18:35
Yeah. So I mean, I think they, a lot of them fall under micromanagement. Right? It’s, it’s kind of like when you’re delegating to the expert, who actually knows what they’re doing, you don’t don’t, it’s not really helpful to ask, you’re gonna get pulled in different directions by people who don’t know what they’re talking about. And so, you know, of course, people are free to say whatever they want at a meeting or online and make these kind of suggestions. But they’re generally not helpful. So one way to think about it is, you know, if you want to kind of actively listen, all the stuff that you that comes at you at meetings, online, you know, social media, whatever is, you know, is it is what I’m hearing, is it new, true, and for you. So, is it something I didn’t know? Is it actually true, right? Oh, the best way to make pizza is throw in the microwave for two minutes. And then it’s fantastic, probably not right? Yeah. And is it for you? Like is it, are you, is it something about something we do? Or is it something about something we don’t even do? That’s another, the state government, or the or the city, or the town, or not the county or whatever? So yeah, there’s a lot of a lot of filters that the information has to kind of get through to be be helpful.
Alyssa Dinberg 19:55
Okay, that makes sense. Thanks. So there are as we all know, In 2021, there are a lot of communication channels. And we do our best to use all of those communication channels to reach our residents. So what are some of the biggest mistakes people use with each communication channel? And if it’s okay with you, I’d love to just like go through each communication channel. Let’s start with public meetings and workshops,
Kevin Lyons 20:26
Public meetings and workshops. Yeah, so you know, each channel matches pretty well to the type of input that’s useful. So I mentioned like the individual facts, ideas, even questions, questions and answers you can do that fit, especially the helpful stuff, that’s new, true, and for you. There is one whole category of helpful stuff like that, that you can actually get anywhere, you know, some, I remember being in a meeting, and we were talking about renting one of our parking lots to a drift show producer, like literally the guys with the, you know, overinflated tires that just do donuts in a parking lot. Yeah. And it’s like, Oh, that’s interesting, get some free money for that, basically. And then we can resurface it because it needed resurfacing. And then, completely randomly, one guy was at the meeting is like, Oh, hey, I just happened to have retired, but I spent 30 years producing, you know, like, NASCAR and other car racing events. And you’re like, Oh, my God, that’s amazing. And he had some really helpful things to say about the insurance you actually need and, you know, safe distances and all kinds of stuff that no one in the room had any idea about. It was just one random person. It didn’t matter if he lived there or not of course, I think he was I think he maybe was here part time or something. But that that useful information can come from any person anywhere. Yeah, so it’s, it’s all good. The stuff that comes in like that. The problem is, when you, then there’s, you know, think of that is that one type of input. The other type is the community preferences, wants and needs, values, you know, the community as a whole. And those you simply, there’s no way you can get that kind of information from a meeting, or workshop. And yet, because we’re human, we react to the people in front of us. And we can’t help but you know, when we’re doing, when 10 people yell at us, we think, gosh, this is a really bad idea. And, you know, I actually became a public official to in part make sure I could feel the pain directly. And I’ve been yelled at by 10 people, they took off their tinfoil hats, right before they came in the meeting, I’m pretty sure. But they were. Yeah, they were telling you and yelling at us about a cell phone tower, which was going to cause cancer. Well, they had cell phones in their pockets.
Alyssa Dinberg 23:03
Yeah, that sounds very familiar. I’ve experienced that same scenario.
Kevin Lyons 23:09
And they self organized. And they all came and one of them, you know, even was happy enough to point out, and by the way, there’s 10 people at this meeting, and nine of them are, are against this. Okay, what does that tell me? Well, I mean, emotionally, yeah. It actually makes you feel like everyone’s against it. Even though you got that rational statistics part of your head going, this is nonsense. But it’s, you know, it’s tough to overcome in a meeting so that that’s a real danger of meetings. And the other thing is, with workshops, they can be really useful for, as part of a planning process, updating a plan, getting feedback, feedback on a particular topic. And that’s great. But what happens is what people forget is that group that’s there, just like the meetings is a very self selected group. And so if you’re, we were working on a park and rec plan, and our partner in that was doing these live facilitations. And it was basically nothing but pickleball people showed up, nothing against pickleball. It looks cool. But the only thing we want is pickleball, really? I mean, is that, is that the takeaway? And so, you can get that information, you could have really actually it is interesting, back and forth conversations and build consensus and you know, get a smarter output. But you all then have to validate that against the actual true broader community. That’s where that you know, valid sample is really.
Alyssa Dinberg 24:39
That makes sense. I think all of our listeners have probably experienced everything that you’re saying right now. I’m sitting here listening to your talk and just remembering things that have happened. Yeah.
Kevin Lyons 24:51
I mean, these these are real, real pains that we’ve all felt, right? It’s a, they get in the way of really good projects sometimes. We’ve heard horror stories about stuff getting torpedoed. I was talking with a city manager, really smart guy in Texas. And he, and because we’re really crappy at marketing, because we’re too busy figuring out ways to use pizza and beer to teach stuff. He said, You know, I god, I wish I’d heard about you a year ago, because we have this golf course, that’s super underutilized. And there’s like eight golfers that use it. And we want to turn into a park for the city of 100,000 people. And they prevented it literally, by just coming to meetings relentlessly. And, you know, bothering hounding people by email, by phone on social media, and making it seem like everyone was against this thing on social media. Which is easy to do, unfortunately.
Alyssa Dinberg 25:48
Yeah, that it reminds me of actually a project I’m working on right now where we have a group of residents who very, very strongly believe that we should go about this project. And there’s about, I would say, maybe 12 of them. And I just don’t know what the right decision is because I have them in my left ear, and I have our public health director in my right ear. And it’s just yeah, it’s just hard to think clearly, when you have residents lobbying for an idea.
Kevin Lyons 26:18
That’s a great way to describe it, it makes it harder to think clearly. That’s exactly right. It, because the emotional part of your brain gets super engaged, because you care about these people a lot, you know. Like when these people are yelling at me, there’s sincerity, I’m feeling their pain, like, you know. And I did a full Bill Nye the Science Guy thing for them, you know, to sort of explain why they didn’t have to worry about the new types of stuff, and so on, because I’ve just felt bad. But yeah, it’s it’s tough. In fact, I mean, it’s interesting, one of the new things we’re doing this year is training, and you know, how to how to do like process structured deliberation, and teach that so that you can kind of stay in a box where the emotions don’t, you know, lead you to make a bad decision one way or another. And you can kind of march march down a clear path that can, yeah, gives you a lot of confidence that you’re doing the right thing too. It takes away that kind of pit in your stomach when you’re like, I guess we’re gonna do this. I hope it’s not…
Alyssa Dinberg 27:21
Well, we should talk about that.
Kevin Lyons 27:24
It’s never Happened, not that that’s ever happened to anyone ever anywhere in government in the history of ever. But just in case, yeah, I’ve been there. It’s awful.
Alyssa Dinberg 27:34
So you briefly talked about it. But as we know, social media is everyone’s favorite communication channel. We love to use our social media accounts. And I guarantee you, there are a lot of mistakes that are made. So can you talk about some of the biggest mistakes that people make or not people, that organizations make?
Kevin Lyons 27:54
Yeah, hypothetically, these things happen. Yeah, yes, of course. Actually, I have a few tips on this one. You know, it’s too late for most people, I had one of our customers who said, I wish we’d never gotten on Facebook. Because we basically just created a platform for crazy people, you know, angry people. And that is not really the right idea. Because it’s a good way to push stuff out. And actually next door is even better, because they don’t block. But you know, one one tip there is turning off the comments on next door. You know, there’s no, there’s really no point in in in general, it’s not a great place to, to engage beyond kind of the answering questions, I think you’ll see the good social media policies, you have to think of this as what, uh, what’s the what’s the public purpose of doing this? Right? What are we doing here in this channel to serve the public as a whole? And generally, it’s not engaging with trolls or, you know, stuff like that. It’s answering questions so that more than one person can see the answer, you know, stuff like that, that’s actually really helpful. And what you’ll see people try to do with it is use it as more like, again, almost like a community engagement, like, oh, let’s find out what the community thinks. Throw out a poll or something. And of course, you get garbage from that technique. Because people, you know, see it, share it and so on. And so, you know, it’s a, there’s a lot of a lot of very smart cities that are thinking about how to how to scale back that platform very seriously. And do it in a very Yeah, very disciplined way, if you will, and and but in a transparently disciplined way, right, like, here’s what we’re gonna do, this is what we’re here for, you know, there’s other places for you to go do whatever else you want to do. But, you know, we didn’t create this platform to soak up staff time and not really achieve anything with it for the public. Does that make sense?
Alyssa Dinberg 29:57
Yeah.
Kevin Lyons 29:57
So yeah, it’s it’s kind of like the great reining in maybe maybe starting now with with governments. And we’ll see.
Alyssa Dinberg 30:06
So what does that look like in terms of, because it’s a fine line, you know, you want to rein it in so that you’re not having unnecessary conversations with trolls. But you also want to make sure you’re getting the information out there.
Kevin Lyons 30:21
Well, that’s exactly right. You know, and there’s, there’s two directions to think about here. So for outbound, I, you know, I tell people, yeah, you know, outbound is great again, do whatever you can to get it out, however, people are listening to you, that’s where you want to be. It’s kind of like vaccine distribution, you want to you want to put that wherever people are already going, right? That’s as easy as possible for them to, to connect and you know, do what they need to do. So with, with social media, it’s a really good way to push stuff out to a lot of people, because there’s a lot of eyeballs there. And obviously, Facebook is, they keep making changes, but they throttle now and that it’s almost like they’re moving towards a pay to play model there. Next door doesn’t throttle. So there’s usually a good chunk of your community to push stuff out to there. The problem there is, if people will talk about next door as Twitter for old people, which is not a positive spin on next door, but it’s really good for outbound and it’s horrendous for inbound, I actually have got a great quote from a guy in Colorado I was talking to he said, the thorn in my side is when someone says something on next door, and council thinks that’s what the whole community thinks. And that actually sums up in a nutshell, like all the potential problems with that.
Alyssa Dinberg 31:43
We just started using next door in my community. And I’m not sure if we really have many residents that are engaged on it. But in my previous community, it was definitely an engagement tool that we used, and it absolutely reaches a different market than Facebook and Twitter do.
Kevin Lyons 32:03
Yeah, it does. And because this will be the highlight of the whole show, if you don’t know about the feed, the best of next door.
Alyssa Dinberg 32:13
Oh, yeah, it’s so great. We had her on the podcast.
Kevin Lyons 32:15
High comedy. Yeah. Oh, great. Awesome. Of course, you did, this isa hip group, duh, of course.
Alyssa Dinberg 32:20
Ya, Kirsten interviewed her.
Kevin Lyons 32:23
Love it.
Alyssa Dinberg 32:24
Maybe two years ago or a year ago? If you haven’t listened to it, you should listen to it.
Kevin Lyons 32:30
No, I’m going to that’s why I love ELGL. It’s like there’s a certain energy and you know, good healthy amusement at that kind of stuff. There is, yeah, yeah.
Alyssa Dinberg 32:42
So, how about online engagement? And this is different than social media. And it could include like forums or questionnaires.
Kevin Lyons 32:55
Yeah, that’s, you know, that’s, um, it’s an interesting area, that when we were first getting, I was working on a project for a government, overseas, actually. And I was doing a quick survey of, Okay, well, you know, how do you, how do people get the, you know, community input when there’s decisions to be made? And the first answer, I talked with a bunch of people like, Oh, well, we have meetings. I was like, okay, no, that’s, okay, seriously, how do you actually get from the whole community? Because I’ve been to meetings. It’s like the same two people, same 10 people, there’s a ton of amazing nicknames. Yeah. Active pest is a good one. Yeah, that’s, I have to look up that list for a future podcast. But the, the, it’s not the community. It feels like the community, not the community. So then we said, Oh, what there’s got to be some technology. But oh, oh, you mean online, you know, forums, engagement, all kinds of different names, right? community engagement tools. And it’s the same thing you, basically you do go from, literally, I think, in our county of 400,000 people, they went from, you know, four people at meetings to like 40 people giving input online, which, which is actually great. If you’re, you know, you have 10 times a chance of getting something useful, maybe. But it’s nowhere near what you need for any kind of community level of input, where you really are looking for 250 to 600 responses from a, you know, random uncorrelated sample ideally of people in the community. And so as we kind of got into this, we’re like, wait a minute, there’s there actually isn’t a way to get you know, that fast, rapid input, and Oh, dear. People are actually using these, some people, most people actually kind of understand they’re not scientific, but some people are using these as a way to get the, all the community quote, unquote, right? That’s a real problem. And we’ve even seen consultants do it. There’s actually a town in Massachusetts that paid a feasibility consultant about $60,000 to start working on a new community center. And they asked them to figure out what people are willing to pay for the usual kind of willingness to pay stuff. And they did a survey monkey, I think, nothing against Survey Monkey, it’s good for other stuff. And I think that 400 something responses, and 85% of people said that they would pay $100 a year for the community center, which is actually pretty plausible for this very wealthy community. And it turns out the real data was 33% in favor.
Alyssa Dinberg 35:44
Interesting.
Kevin Lyons 35:45
Yeah. And it’s exactly what what you could imagine basically, the people who use it the most or hang out with people, their senior center built into it, along with some some recreation, they take, they find out about survey, first, they’re following the issue. No one else is, they find out about the survey, they then take the survey at a really high rate much higher than other people. And then they share it with their friends, which means they take it at a much higher rate. And they can even take it multiple times. And so by the time you’re done with all that we were we are able to see these comparisons or make these comparisons, we’ll see Yeah, 200 300% error like that, which basically flips it from Oh, everyone wants this to actually, no wrong everyone wants this. And that’s pretty terrifying. If you think about, yeah, I mean, basically, yeah, millions of dollars wasted by free data. The right way to say it, I think is like free data is costing us billions. Because it’s bad data.
Alyssa Dinberg 36:51
I’m just thinking about how every every listener that works for a community definitely has those frequent flyers. And we, we always have the same people at every single meeting. And we’re always trying to figure out ways to broaden that reach. So everything you’re saying is definitely, I’m sure it’s resonating with a lot of people do.
Kevin Lyons 37:15
You know, what’s a really interesting tip on that that group, is, if you think of it as like a mini panel of people, like if you have a small group of 10 people, The Hateful Eight, for example, it’s another one of my favorite nicknames. If they hate everything, and you do something, and they like it, that’s actually useful information. Because it’s like a different signal. And so sometimes that’s actually helpful.
Alyssa Dinberg 37:43
Yeah.
Kevin Lyons 37:43
If you don’t have any other data.
Alyssa Dinberg 37:46
Hmm, yeah.
Kevin Lyons 37:47
Not often. But the little little tip right there.
Alyssa Dinberg 37:50
So last communication channel that I was hoping you could talk about, and if you have more, I’m more than happy to hear them. And I’m particularly interested in this one. I actually put a put a question out to our ELGL Members, I think it was this week, I don’t know, all the weeks are running together. But I believe it was this week about how to write good surveys, and how to make sure you’re asking questions that not only have a benefit to the decision you’re trying to make, but also are understandable and resonate with the people filling them out. And also don’t take too much time. So what do you think some of the biggest mistakes people make with scientific surveys? And this could be question design, mail, phone panels, etc.
Kevin Lyons 38:42
That is, that is a great topic, that’s actually a whole podcast on its own.
Alyssa Dinberg 38:47
It is, yeah.
Kevin Lyons 38:48
It’s a fantastic topic. And we’ve accumulated 23 points of quality control over the years to keep ourselves from screwing up. Because there’s so many ways, even if you have a perfectly random sample, uncorrelated, and you get a really high response rate in that, which I’ll talk about in a second, which is super important, because that’s how you eliminate that self selection, you know, people self organizing to take a survey, you can still completely screw it up by the questions. And, you know, one of the questions I’d like your listeners to answer is, how great was this podcast? Great or really great?
Alyssa Dinberg 39:29
Mm hmm.
Kevin Lyons 39:30
Yeah, what’s wrong, might be missing a few choices there. Hopefully not too many. But there’s all kinds of balanced answer sets, leading questions, you know, pros without cons, cons without pros, all kinds of ways that questions can get messed up. There’s a there’s another tip I like to share which is, you know, if you’re gonna do an unscientific survey tool, that’s basically just a questionnaire. Yeah, survey monkey is the same as Google forms, the same as you know, any number of engagement tools that are out there, you just you want to avoid any kind of percentage questions where you’re kind of asking people to choose a, b or c, right? Because when you add it all up, it doesn’t mean anything. But what is really helpful that you can do is ask open ended questions. Because again, maybe there’s someone out there with a real nugget for you. And the more the merrier. So, for unscientific surveys, the thing is to ask questions, you know, if you could change anything about x, what would you change? Or do you have any suggestions how to improve this? Or stuff that you can kind of like read as a one line and go, Oh, my gosh, that’s a great idea. You know, Oh, my gosh, I didn’t know we were doing that, right? We actually were doing a survey on a giant Park. And one of the comments, we always closed with open ended to make sure we capture this gold. It was like, Hey, I don’t know if you guys have been in this area where we’re proposing the park. But after six o’clock, the mosquitoes come out. And you kind of go, oh, my gosh, If I’m only go to the park during working hours, how would I possibly know that? You know, even if I work there every day, they go over and visit it, but I don’t live there. And who knows? I mean, there’s that kind of stuff you can get from open ended questions, open ended engagement. It’s really helpful.
Alyssa Dinberg 41:19
Yeah, that’s a good point. Yeah.
Kevin Lyons 41:21
But as, But back to the scientific questions, it’s really, you know, making sure that you are getting the three things you need for valid data is large number to get to a small margin of error. 250 to 600 is plus or minus 6% to plus or minus 4%, 400 is the number usually here. That is a kind of a gold standard, plus or minus 5%. And then the second thing is the response rate. This is a tricky one. So guess what the response rate is for phone surveys now?
Alyssa Dinberg 42:01
Oh, I actually just saw this.
Kevin Lyons 42:03
Oh, cool.
Alyssa Dinberg 42:05
Isn’t it like 16%?
Kevin Lyons 42:07
No, it’s actually lower, Believe it or not. Yeah, so like, random digit dial you might get the number I saw recently was 0.9%.
Alyssa Dinberg 42:16
Really?
Kevin Lyons 42:17
Yeah. Yeah. And it was 20 years ago-
Alyssa Dinberg 42:20
That’s crazy.
Kevin Lyons 42:21
Ya, I know It’s crazy. But 20 years ago, it was 60%, six, zero, and another 20 before that, it was like 72% or something. And so what’s, what’s awesome about that is when you’re randomly picking a person to respond. You’re, you’re getting them, almost always. And so you know, you there’s no risk that Oh, gee, who’s actually answering the phone at when I’m calling this? And does that sample of people that I’m actually hearing from look different from the people I’m not hearing from, which is kind of the core thing to think about? And so yeah, with phone surveys, that’s really low. Mail surveys, it’s kind of usually 10 to 20%, maybe, and heavily biased towards older people. So maybe, yeah, you know, yeah, like maybe half the half the responses come in from people 65 or older. So there’s a lot of challenges like that. And, and that’s why panels have taken over, you know, Gallup, Pew, that’s what we do, mostly, to get that, you know, high response rate from a well defined subgroup. So that’s really important. And then, of course, there is the questions, you know, getting the right questions, and one of the most important things is the citizen expertise, what is actually in their head that they know, it’s like full circle back to your first question, right? How do you get the information that they have in their head into something that is really useful, reliable, actionable data, when it’s aggregated? Yeah. And so you can’t ask them things they don’t know. If you if you if you do, you get junk. You can’t ask them kind of opinionated questions. You have to give them some context. You have to, you know, frame it in a way that they’re thinking about, that they can give you an answer. And that’s where you want to avoid, you know, what is called signaling type questions where people might just give you some, you know, political style answer, a lot of political polls have that problem.
Alyssa Dinberg 44:15
So if a community is dead set on sending a scientific survey, do you recommend them using all forms of communication? mail, phone, panel?
Kevin Lyons 44:31
Well, yes. If they want community input that’s, they actually, for decisions. You know, as opposed for sort of fun, engagement or something like that, or that individual input. Yeah, you have to do something scientific. And then what’s interesting is depending on what kind of data you’re trying to get, the response self selection problem may be a big problem or a small problem. So the hotter the topic, the worse the problem. Yeah, it’s actually, one of my favorite surveys of all time is a backyard chicken survey. And we watched the results, we kind of, we broke it down minute by minute, we watched them as they came in. And it took about 10 minutes for some pro chicken person to find the survey, you know, 48 hours survey. So it’s not a long time, imagine how it is after 3 or 4 weeks, typically, for an unscientific survey. And they shared it. And then by the end of the survey that open nonscientific sample was running, like 70% of people care a lot about chickens, and only like 27% of people in the regular, the real scientific sample did. And it was just running off, increasing every hour. And so that’s, you know, that can be a problem with phone surveys and mail surveys as well. If you say, Hey, this is a single topic survey. And we want to know what you think about recycling or the airport or anything really, that might have supporters or antagonists on either side, because those people are going to take that survey at a much higher rate. And because you’re in that single digit or even low double digit overall responses, they can they can bias the results in a way that’s not going to show up if you look at demographics or anything else, either. Because it’s based on interest. Yeah, so it’s really tricky to do that. So we recommend some kind of, being really careful about how you do the invitations to not select by topic and a lot of things associated with that. There’s, as long as they’re working with a, you know, the with a good pollster, if they’re doing something like that. There’s ways to get decent data, or at least get the best data you can.
Alyssa Dinberg 46:56
Yeah, that makes sense. Yeah, I wish we had had this interview, like the beginning of last week before I sent my survey out.
Kevin Lyons 47:05
Oh, no. Well, I mean,
Alyssa Dinberg 47:07
It didn’t go bad, It, I mean, it didn’t go poorly. It was just a very short 5 question survey about the program I mentioned earlier, trying to gauge interest in the program in the business community. And what you had been talking about is exactly what we had happen, we had a bunch of businesses lobby for the program and skew the results and we also had a bunch of businesses that aren’t eligible for the program apply or fill out the survey.
Kevin Lyons 47:37
Huge problem with open open unscientific surveys is a big chunk, typically, like 20, or 30% of the people don’t even live in your community. And if it’s a hot topic, I was talking with a special district General Manager. And they were doing something pretty mundane like cutting some of the trees that were cutting into the power lines, basically, in one part of town, and apparently it worked.
Alyssa Dinberg 48:03
Pretty contentious.
Kevin Lyons 48:04
Well, it turned out it was, you’re absolutely right. I mean, it absolutely is right? They’re probably pretty old, pretty beautiful trees. By the time they’re they’re really getting way up there and knocking into power lines. And apparently, the word got out. And people came gotten cars like 100 miles away from like Berkeley and Oakland, California, where I used to live and they caravanned to this meeting and just raised raised a ruckus. And yeah, I mean, that kind of thing happens, right? It’s certainly anything online gets shared, you know, the better way to think about online services is really like more like petitions.
Alyssa Dinberg 48:39
Yeah, that’s exactly, that is very true. That is very true.
Kevin Lyons 48:44
Yeah. And, you know, what does the petition tell you? If 100 people sign it tells you there’s 100 people that want that doesn’t tell you what the other 10,000 or 100,000, or a million people want at all, you have no idea.
Alyssa Dinberg 48:55
So do you have any advice or I guess advice, because as you know, staff are directed by councils or boards to do different types of things for community engagement. And for let’s just continue my example. The survey that I sent out was driven by our board, they wanted to have a better understanding of what the community, the business community, felt about the program. So I created the survey and sent it out. Do you have any advice on how staff can kind of guide elected officials down a pathway that’s going to give the best feedback from the community?
Kevin Lyons 49:39
I love that question. The you know, it’s very similar that, you know, thorn in my side next door comment. They’re going to be influenced by things that they shouldn’t be. And that’s actually one of the reasons we came up with the pizza story is to kind of be like, Hey, this is what we’re doing. It’s really bad, isn’t it? Is this what we want? You know, kind of is this what the data we want? Or do we want something else? And a lot of them will use that word, the community, really loosely. Well, I mean, if the community wants this or this, if the community wants this or this? And you kind of go great, how do we know? You know, how do we know what the business community wants? And so yeah, the, there’s some, we’ll have to, we’ll have to create some video modules on this or something that people can just use or send, we’ve created a few. We have a video actually a little animated video of some elected officials, one in a town that does get real input one that does it. And there are electoral implications, of course, which get their attention, sometimes. At the end of the day, it’s really just starting from that, that big picture of what are you actually trying to do? What’s the decision on the table? And talking through that with them? You know, what’s the input going to be used for? And then as you get into that, it’s, well, is this input, we’re looking for ideas, suggestions, you know, whatever? Or are we really trying to figure out what, quote unquote, the business community wants? And if they really want to know that that latter thing, you have to do it in a scientific way and it’s, it’s a, it’s a different different animal.
Alyssa Dinberg 51:24
That makes sense.
Kevin Lyons 51:25
They just have to decide what they actually want.
Alyssa Dinberg 51:27
Yeah, that makes complete sense. Yeah. All right. Well, we’ve talked a lot about things that are done poorly. Do you have any advice or resources for our listeners on how to do better?
Kevin Lyons 51:47
That is a Yeah, I think we do have some things on that. In fact, we have a webinar that I did for CCCMA. And for Tamio, actually, for the conference, the keynote was canceled. So we did a webinar instead. And we can put some of that in the show notes or something if you’d like.
Alyssa Dinberg 52:10
Sure. Yeah. I think that would be great.
Kevin Lyons 52:12
We’ll pull together, you know, we’ll put together some resources that would be helpful for people because it’s a really, it’s a really interesting topic. And I think there really are probably, there’s a joke that economists have, you know, there’s no $20 bills laying on the sidewalk. Because if they were, they’d already be picked up. So So if you see one, don’t, the joke is, you know, the senior professor is like, Oh, don’t pick that up. It’s not real. Yeah. It’s like the junior person’s like, oh, I’ll get it anyway, they pick it up. And I actually found by the way, I found a $20 bill in a stairwell in a hotel, because it was like the, everyone was using the elevator. And it was kind of wonky that day, and I happen to use the stairwell and probably no one had used it in months. And there was, so yeah, there’s low hanging fruit, $20 bills lying on the sidewalk, however you want to think about it, that you can do just by changing a few things in how you do your input, by changing how you how you filter the input that comes in. That new true and for you. There’s another little tip framework, which is thinking about what am I What am I actually listening for, facts about opinions, facts about facts, or opinions about facts, or opinions about opinions, which is the worst, everyone wants this, no one likes this, right? And only the top two are useful. The bottom two are not. So the more you can kind of rule stuff out as it comes in, you can you can get those Nuggets to, to shake out. And and you know, may actually make some better decisions with whatever input you have.
Alyssa Dinberg 53:50
Awesome. Yeah, I would love if we can put some resources in the show notes. I think even if you are doing a fantastic job at getting public input, we all have room to improve. So I would love to be able to put that in.
Kevin Lyons 54:04
Yeah, you give me another idea, which is for years, we’ve talked about the Museum of bad surveys. And we’ve been collecting museum a bad survey questions. I think that could be a fun, that might be a fun, fun podcast.
Alyssa Dinberg 54:21
I have a former coworker who collects bad charts.
Kevin Lyons 54:26
I love that. I would love to see that.
Alyssa Dinberg 54:28
Shout out to Eric Roche.
Kevin Lyons 54:30
That’s awesome.
Alyssa Dinberg 54:31
He collects bad charts and yeah, it’s great. They’re super bad. Interesting.
Kevin Lyons 54:35
It’s such a great way to learn, you know, yeah, to really, to look at something and go Okay. Well, you know, for example, most people don’t don’t even think about the movie rating scale. You know, the one through five. Yeah, right. Poor, fair, okay, good, great or whatever, or good, great, excellent or something like that. Yeah. So if you picture that as a seesaw fair, right, is the middle. And yet there’s only one star on one side of fair. And there’s three sides, three stars on the happy side of fair. And so that’s a biased answer set of course, that’s what the movie people created when they invented it back in the 30s or something. Yeah, but it’s so engrained we don’t even notice it.
Alyssa Dinberg 55:21
I’ve never really thought about that before.
Kevin Lyons 55:23
Right, there’s a lot of examples like that things that you do because you’ve seen them done before. And if you actually think about it for a minute you go Oh, god, that’s terrible. Why are we why are we doing that?
Alyssa Dinberg 55:34
Yeah, I’m never gonna think about one through five again, the same.
Kevin Lyons 55:38
Well there you go. Yeah, so, there’s a lot of stuff like that we, I think we’re gonna create some content on that to make it to make it really easy. Because, you know, there’s no reason to You know, Friends don’t let friends Use Survey Monkey but there’s also no reason to make bad questions. People people recognize them right away.
Alyssa Dinberg 56:02
All right. I have one last question for you. If you could be the Gov Love DJ for this episode, what song would you pick as your closing music?
Kevin Lyons 56:13
Oh, God. I was on Pandora. I was cleaning out the garage and my third Gen ska station. mighty mighty bosstones if anyone’s into that I got that. I’m from Massachusetts, but I got into it really when I lived in Santa Barbara. It turned me on to electro swing and there’s some really great mix of 20s and tech that was getting into and I got to go with Booty Swing by Parov Stelar. Yeah, that would be a good one, either that or crushing groove or something like that. But yeah, yeah, go with Booty Swing.
Alyssa Dinberg 56:53
That fits this episode perfectly.
Kevin Lyons 56:57
I don’t want to dive into that one any more than we need to.
Alyssa Dinberg 57:01
Pizza, beer, and public input.
Kevin Lyons 57:02
Pizza, beer, and booty swing.
Alyssa Dinberg 57:04
Yeah, that’s perfect. All right. Well, thank you so much. That ends our episode for today. I really appreciate you taking the time to talk to me. It was great.
Kevin Lyons 57:12
Oh, my pleasure is that was a fun interview.
Alyssa Dinberg 57:15
It was. Gov Love is is produced by rotating cast of ELGL volunteers. ELGL is the Engaging Local Government Leaders network. You can reach us at ELGL.org/GovLove or on Twitter at @GovLovePodcast. Thanks for listening. This has been Gov Love, a podcast about local government.