Daily Archives

2 Articles

Posted by Sonne Taylor on

The Spectrum Of User Surveys

Most advice about user surveys is really bad.

I know, because I’ve been part of the problem on this one.

Don’t believe me? Here’s a video I found and reuploaded from a few years ago. It’s in a folder with a very strongly worded readme all about what was going on at the time.

The trouble with advice like this, is that user surveys happen in one of two ways.

You’ve either got a highly mechanized feedback system (think coupon codes on the bottom of Fast Food receipts) or you’ve got an incredibly laid-back “user-interview.”

I don’t think it will be very controversial to assert that contemporary thinking has largely evolved to consider this a waste of time. (That is ‘coupin interviews,’) The process is too impersonal. The feedback is too inconsistent. If you’re still relying on this method, it might be well past time to start considering an alternative.

Still, the fundamentals are sound. The fine folks at ChartMogul put together a handy runthrough of NPS, I find myself passing along rather frequently.

There’s been a lot of great work generated on the topic of user interviews however. Eleonora Zucconi put together a fantastic collection of “46 Interview Questions For User Experience Researchers…” Teo Yu Sheng breaks out how you should think about asking questions in “5 Steps To Create Good User Interview Questions…” and I’d be remiss if I didn’t mention the vibrant discussion on Charles Liu’s “Never Ask What They Want

Learning to write in this way is a good skill to pick up, but it won’t keep you from making one of the biggest mistakes I see teams run into during the interview process.

Formulating your questions to get real feedback takes practice, but once you’ve figured it out it can be tempting to use a slightly different framing with each person you interview.

The trouble is, inconsistent surveying tends to generate inconsistent data.

That’s why it’s so important that you set your goals with the understanding that feedback happens on a spectrum.

In some circumstances, automated feedback may actually have some utility. For example, it’s probably ideal for “transactional” uses. In other cases (like say exploring a new feature roadmap) you might want to conduct something far less formal than a regimented user interview.

Trying to force each application into one predetermined feedback rail is a mistake. You won’t have the same answer every time, and that’s the point. Instead, try to focus on picking the right feedback mechanism for the task at hand.

The Spectrum Of User Surveys was originally published in Marketing Experimentation.

Posted by Karl Taylor on

“But Ads Don’t Work On Me.”

Why It May Be Time To Rethink Your Skepticism

I still remember the first time I studied ads in school — it helped that it was eighth grade english.

The unit took a few weeks, and in hindsight it had to have been written by advertisers. I say that because one of the key learnings was that the most important part of any advertisement was the “call to action.” My teacher explained that by studying the call to action you could “get behind” the persuasion of any advertisement, and by thinking critically about the world around you blah blah blah blah blah.

If I had known then what I now know, I’d have ruined that class by suggesting that the only thing you’d get out of paying attention to the call to action is a lifetime of disappointment and sucker’s bets.

The truth is, if you think “ads don’t work on you,” I can probably tell you a handful of other things you think about yourself too. You likely prefer to dress in earth tones, you aren’t likely to be reached in a radio audience (unless it’s a campus station or the current,) you strongly identify with statements like “I go my own way,” or “Never Settle For Anyone Who Makes You Feel Like an Option…” I could keep going.

You wouldn’t ever identify with all of the traits I might list, but you’d identify with enough, you might start to wonder how original your rebellion really is.

That’s the trouble with paying attention to the call to action, it’s often the least insightful part of any advertisement.

This is the case for two reasons.

The first is the simplest, if marketers like me had our way, we would never ever run another ad “selling the product,” ever again.

Widget, 5$” used to be everything. When you clearly define what a product is and who it is for, you also define what it is not, and who it is not for.

I have a pretty unique diet, and I’ve got relationships with brands that only my “unique diet friends” know about. You can see this happen all over the place if you have a few friends who are all into different things. Have a friend that’s “really into guitars?” they’ll likely have very identifiable divergences from the “average consumer” pattern.

There’s nothing wrong with this, of course. You can’t sell a product people don’t understand, but carried to an extreme the impulse to make this kind of ad is also responsible for some of the worst examples of advertising.

The truth is, we make “ads” for a wide variety of reasons, and some of the best reasons to do it have nothing to do with convincing you to buy something…at least not right away.

So that’s the first problem, but to be honest, the second is a bit more nuanced.

I’ve noticed that generally, people inside of marketing and people outside of marketing have different definitions for what an “ad” is.

To be honest, that’s likely the fault of how rapidly that definition has changed.

These days, I most frequently encounter “ad” as a description of the smallest discrete purchasable quantity of inventory.

If I buy :30 of airtime, that’s an ad.

Some digital inventory products (say Snapchat Filters) actually work in a fairly similar way.

If you were looking at the world as being made up of only this sort of ad product, only looking at calls to action might make sense.

The truth is modern digital media platforms don’t always work this way.

Some platforms allow you to bid on impressions based on audiences of users. What that means is that while you may only make one “ad,” that “ad” is in reality being shown in a number of places in a number of different ways.

Many times the ability to create ads in this manner proves to be a valuable feature of the publishing platforms that offer the capability.

That means that on a platform like Facebook it’s not uncommon for an advertiser like myself to use the same image and copy to create hundreds and thousands of ads. Each ad is part of a search for just the right balance of customers.

When we find that balance on an audience centric platform, we can use the insights we generate to plan bigger and better marketing.

This is one way large brands can identify exactly which celebrity, figure or performer they ought to approach to promote a product. It’s not the only way, and it doesn’t just work for big businesses.

I started this article by explaining that I didn’t think that class was very good because it must have been written by advertisers. The truth is, that if you only look at the clicks, you’ll miss all of this nuance that I’ve pointed out. If you wanted to start to notice modern ads in the same way, it might be better to ask “why am I seeing this, here, now?”

I added that storm of commas on purpose.

I think when you look at an ad, you should ask yourself three questions.

First, try to figure out what it is you’re seeing. It isn’t often the case, but from time to time motivations aren’t as clear as they appear to be. An article might be paid for by a company. An interview could be conducted by a partial host. You won’t always be able to see between the lines, but with a little creative Googling, you can figure out who is behind most major advertisements.

The second question you need to ask yourself is why is this ad placed where it is.

You may find that you’re seeing a banner ad on multiple websites. If you only look at “calls to action,” you might be inclined to think an ad is following you — and sometimes it is. Other times however, an ad can be running to all of the media properties across a family of brands. Unless you know who owns the property you’re looking at, it’s hard to know what’s going on.

In addition, the choice of “Facebook vs Twitter” (or “Facebook vs A Billboard” for that matter) has real consequences. You should consider them.

The third and final question you need to ask yourself to understand a modern ad is why am I seeing this ad right now.

We spend a lot of time thinking about the different ways we can reach potential customers at exactly the right moment — but I don’t encounter many people who think about the fact that the news they get at 12:00 is different than the news they get at 5:00. Every advertiser I’ve ever met has an opinion on dayparting.

It might still make sense to think about the “call to action,” but if you can try and remember those three questions, I think you’ll keep yourself from falling prey to most of our dirty tricks.

“But Ads Don’t Work On Me.” was originally published in Notes On Digital Marketing.