English Español Deutsch Français Italiano Português (Brasil) Русский 中文 日本語

Advanced PPC #4: Advanced Ad Testing from Small to Enterprise accounts


Topics covered:

  • Single- vs multi-ad group testing,
  • Some simple ad templates,
  • Metrics to use,
  • Ad rotation.



Joel: Okay. So we're live. I'm very excited about this session tonight. A big core part of SEMrush's business is search advertising, which of course is Google Ads, paid search.

Tonight, we have Brad Geddes with us, who is actually one of the authorities on the subject from a long time ago. I've actually known that name from back when I was a junior doing PPC. And I'm very excited to hear what he has to say about advanced ad testing, which actually isn't just for someone who is running very large enterprise campaigns.

The methodology that he's going to present to us, I think will apply to anyone, regardless of campaign size.

We have Brad Geddes, we have Navah Hopkins and we also have Aaron Levy.

The format for this session today is going to be Brad doing his great presentation, which is going to be pretty quick, it'll be about 20 minutes or so.

After the presentation, we're going to have a time for questions and answers, where this panel of experts we have over here, as well as myself, will be addressing anything that you have to ask.

To do a formal introduction, the presentation is going to be given by Brad Geddes, he has been involved with PPC for over 20 years, which is about as old as PPC is. He is a co-founder of Ad Analysis, which is a PPC recommendation engine for Google ads and also Bing ads, that makes day-to-day management of your accounts easy.

Also joining us is Navah Hopkins, who is part of the customer success and thought leadership team at WordStream, that's where she empowers small businesses and also agencies, which serve them to win with Google, Bing, and Facebook.

Aaron Levy, he oversees the Mid-Atlantic Midwest teams at Elite SEM, which is a full-service digital marketing agency. He's been in search advertising since he was given the keys to the car, managing SEM for a Fortune 100 chemical company in college and he also hasn't looked back since.

Ad Testing Guide: Improve Your PPC Ads

Brad Geddes: So... we're going to talk about ads and ad testing and then we're going to stop and do some commentary with Navah and Aaron and Joel and then at the end, we'll come back to some more general questions.

So, when we think about what makes a great ad, one of the first things we think about is how we construct it.  And so our headline is usually related to our ad group and then we've got a benefit or call to action.

So one of the first things though is, we think of paid search. We have the group of people who are fantastic at math and the people who are great at ideas. And while there are people great at both, they're fairly rare. So what often happens, we have to think about ad inspiration, where do we pull ideas from?



Whether it's social; I mean Twitter used to be fantastic for this, because its length was the length of an ad, they were identical. You've got interviews, of course, with customers, you have competitive research. So step one is often going outside of your little bubble to say, where do we pull in information?

PPC Competitor Analysis


Second, we have to think about who our competitors are and how we differentiate them. We compete for impressions, we don't actually want each other's clicks. So this is often where we think about the ads, we're saying how do we make sure we're different from them, so someone can self-select correctly? Often our first step is saying who's actually a competitor and how to make ourselves different.

The next, we want to dig into everyone's ads and say where should we start with ideas? So... you want to do some searches for your top keywords and then say what's good and bad?

So funny enough, this (ad) is for London and the first thing that jumps off is there are dollar signs in these ads. So we didn't get the currency right.

Second, wow, there are some people doing some really good structured snippets. So when someone does a good job, we might want to copy that a little bit. And other times, we’re saying “how do we localize better than other people” because localization is very poor here.

So that's some high-level stuff, so when we think Aaron, Navah, Joel, when you start with your competitor research and ad ideas, where do you turn?

Aaron Levy: So for me, competitive research and SEM is important to find areas where you differentiate yourself from being a commodity product versus a real product. So you think about trademark registration, to a degree, it's a commodity product, you either get it right or wrong.

What I tend to do is look at the messaging that people have on their site and I also really like looking into social profiles to see where they're the most active and seeing if they respond to customer complaints on Twitter or Facebook, seeing what they're saying in their ads.

We also love using tools like SEMrush. What we like to look for is ad consistency. So if we see a competitor that has had the same-ish ad running for 180 days, 365 days, that ad clearly works. Whereas if we see a competitor has 1,000 new ads that are brand new, that probably means they changed their entire strategy, so whatever they were doing before isn't working.

So what we like to do is see where everyone's the same and then figure out where our product can be a little bit different.

Navah Hopkins: I begin the conversation with surf analysis, looking not only is it a transactional search result page, do Google and Bing actually think that it's worth serving ads here, but also is the cohort a reasonable cohort to be serving in?

Another thing that I really encourage folks to think about, are you laying out an ad that will be interesting when everyone else is being boring? So if you see that there are particular queries that folks are just not investing the time, to be interesting, to be engaging, to speak to the actual pain point of the user, I'm much more interested in investing effort there, than I am on a page where even if I love the page, there are three other really interesting compelling ads that are aligned with that prospect.

So it really comes down to where can I make the biggest bang and is the page actually worth investing the effort on?

Joel: That makes a lot of sense and it's actually something that I follow as well, but I do actually want to add: if I'm starting a new activity and I'm at the point where I'm ready to start working on ads, I actually avoid looking at anyone else's ads before I come up with a few ideas.

The reason being is that I feel that like if I don't look at what other people do first, I'm going to come at it with different angles. If I start like with researching what everyone else does, my first like reaction of course will be when I look at people's ads, that are really click competitors, people who are in the same market as me, as well as advertisers of the same keywords, I'm going to start thinking like they do, because they're going to kind of pollute my thoughts a bit.

The second thing I do is I actually do look for weaknesses.

Single vs Multi-Ad Group Testing

Brad Geddes: So when we think about testing, there are two testing types, we've got single ad group testing, which means we're testing in an ad group to find the best ads for that targeting method, be it a keyword, an audience, whatever targeting method. We then have multi-group testing, this is where we're testing an idea across multiple ad groups. Both of these have their place, which we're going to get to.

So multi sometimes confuses people, we're basically saying is, hey, we've got two ideas, we're going to put them in two ads in X ad groups. It could be 10, it could be 10,000, that's all we're looking at.

So when you think about where you want to start, multi is about customer insight, because you're finding the best ad across a large swath of targeting. It could be your entire brand campaign ad groups, it could be a product set one, it could be something else.

So when you've got market research questions, price sensitivity, call-to-action differences, luxury versus commodity cheap, that's really where the multi is good. You're finding out overall it's better, but in any one ad group, it might be worse, because we're looking at the average says it's better overall.


So single ad group testing is best for brands. If you have an ad group and something goes wrong, you get fired, that's where single is because you want the best ad possible there. And those ad groups that just do really well, that you don't want an average to work with.


Now where multi is great, if you're a small account, you may never get enough data in an ad group to test, so you can aggregate information together, it's better for templated ads.

Do you guys have different ways that test or like to think about your segments or how would you definitely segment out different ad tests?

Navah Hopkins: So in terms of tests, I make folks pick what variable they're going to look for. A lot of folks will jump right to the ad without considering the landing page that's going to follow. So when you're looking at the ad test, when you're looking at the performance, we always make sure that we build in the time to figure out not only is the ad copy right but also landing page and swapping those out.


The other piece that we look to, is if there is a point of differentiation that we really want to test, we will always focus much more on headline testing than we will on the description line testing, just because whatever you throw in the description, having the descriptions is great, it doesn't actually matter so much what's in there, just so long as you have them. Whereas the headlines, you can really see is a bit of copy going to move the needle.

Joel: So what I do is, first of all, let's say, I'm advertising a website that's sold beach gear, like beach chairs, beach umbrellas, beach towels and so on.

The first thing I'll do is I'll segment like the types of ads, based on all those different categories. The ad group to pay attention to the most is going to be the one that actually gets the majority of the traffic for the beach chairs campaign. It would actually be people searching for beach chairs, without any really real modifier.

And then to get more data to help me understand what works for the other groups and as far as chairs go, I'll then aggregate things, such as descriptions and call to actions. I'll aggregate them to get more data, but then sometimes I'll take it with a grain of salt and I'll see how certain groups have differentiators.

PPC Ad Testing Metrics

Brad Geddes: So let's talk about metrics, one of the harder discussions. When we think about metrics, I can't stand CTR and conversion rate testing. Everyone feel free to disagree with me when you've got your chance to speak.

The problem with something like CTR is that it just says I want the most clicks, even if no one actually looks at my website. And conversion rate says once they get to my site, how often they convert but ignores how often somebody even clicks to get to the site.

So that's the problem with a lot of testing. If I said, hey, ad one is 10% CTR and ad two is 20%. People go, 20% is better than 10, but what we don't have there is ad one has a 10% CTR, but a 10% bounce rate. And ad two is 20% CTR and a 50% bounce rate. So ad one with the lower CTR actually brings in more traffic and that's where when we think about metrics, one that's useful is just conversion per impression.

So this is the potential amount of conversions that exist. Every time your ad is displayed, you have a chance of a conversion. That's good in lead gen or good when you don't have variable sale amounts, but you would never want to use this in e-commerce.

When we get into e-commerce, then we're really thinking about, hey, what's our revenue per impression? Some people do revenue, which is just my revenue divided by impressions. Others want to take out the cost.

So they'll do, here's my revenue minus ad cost, divided by impressions, assuming your CPCs aren't that different, which in most cases, they aren't, but sometimes there's some weird cause for variables. The numbers usually work out very close. While these are two good metrics that say this is the most potential conversions or potential revenue, they do ignore when the advertiser tells you can't break $35 CPA or you must have a 400% ROAS.

So then often we have what we call filter metrics, which says all we must have a 300% return on ad spend, we're testing four ads, two of them are below that metric, so neither one can be a winner because we would make our client unhappy by keeping them. Of the two that are left, what is the highest revenue per impression?

If your only goal is Quality Score, which should not be most people's goals ever, but if it happens to be just Quality Score, that's CTR. CTR is a great tiebreaker metric, you have two that are incredibly close, well, let's pick CTR to build up more audiences and do other things.

So what are your thoughts on metrics?

Joel: To answer your question, like yes, I normally look at the way you just described it, however, if I am having trouble with my Quality Score, I will focus more in the CTR side. Not necessarily if it's a tiebreaker as well. I will do it if the CTR is higher when the conversion rate is only a bit lower.

Aaron Levy: Our thought process usually, look for CTR winner first, then look at conversion rate winner. If there's no conversion rate and there's a CTR winner, great, we're all done. If they fight, we'll look at conversions per impression, going back for. We don't want to get a million clicks of people who aren't our target of customers.

Navah Hopkins: I appreciate that you put the plug in there for let's not blindly follow Quality Score. Brad's heard me go on that rant many times.

The one thing I will add is when I'm looking at metrics, a really important component is actually segmenting to see what the click type was. Because I'll value click-through rate and conversion rate on a site link or a price extension a little bit more and get insights for potentially new ads, new ways of expanding the account than I will necessarily a headline click.

In addition, kind of going back to Joel's point about the average position, if I'm seeing that consistently mobile is winning, I may be a little bit more forgiving of an ad that maybe costs more, it's in that first position because mobile needs the fuel.

Brad Geddes: So let's keep going. This may be the most contentious setting in all of Google: rotate versus optimized, oh wow.

Rotating Ads vs Optimized Ads

So I know Google's got their advice of, hey, just keep making ads or create at least five ads. We find and we look at huge amounts of data when you break three, Google seems to be like rolling dice for which ad gets served. It seems to no longer be based on any metrics whatsoever.

So if you're here, watching this webinar, I hope you're doing ad testing, which means you should be using rotate or do not optimize.

So what do you guys think? What do you use? Feel free to disagree, this is the most contentious setting probably in everything Google does.

Navah Hopkins: This is another passion point of ours, collectively, but also of mine. There comes a point where you can't actually have a reasonable amount of data behind any individual ad and when you start going beyond three, you're not going to be able to pick a winner, much less a computer.

In terms of rotate versus optimize, I agree with you ... if you have no time whatsoever to manage, optimize is fine. The other use case for optimize is if you are going to be on a push for like Quality Score or a lot of the vanity metrics that are like Google first.

I'm very much aligned with rotate evenly and then make decisions, as opposed to optimize.

Aaron Levy: I will remove my tinfoil hat for once and look at this from a Google perspective or Bing perspective or both, for the next month. So when we do ad rotation, depending on how you have your account built out, there are a lot of decisions that go in to optimize that we can't see.

But what we found is the more segmented the account, the worse it does with optimize. So if we have an account where we have single keyword ad groups segmented by device, segmented by audience, segmented by location, segmented by favorite flavor of ice cream, whatever, that's an area where us having control is good, because the account is built for control.

Whereas if you have an account that's built with putting tinfoil hat back on, Google's best practices, where you have everything smushed together in one giant campaign, that's an area where optimized tends to do better.

So I think there are places for both. We tend to defer towards rotate, but I will say that while optimized looks like it makes decisions a little bit quickly, Google's smart. I don't think that they're that silly that they're just like we're going to pick the one that spends the most. I'm sure that there's science behind it, we just might not always be able to see it.

Expanded Text Ad (ETA) vs Responsive Text Ad (RSA)

Brad Geddes: So we now have responsive ads, you give Google lots and lots of ad assets, they do all kinds of stuff for you and serve the magical ad over time.

So when we think about ETAs (Expanded Text Ads) versus RSAs (Responsive Search Ads), number one, we see almost always RSAs blow away ETAs in click-through rate. When it comes to every other metric, they usually lose.

We looked across 29 accounts that had at least 100 ad groups with an ETA and RSA in it and said how often does an RSA win CTR versus the RSA one conversion rate? Overall, RSA usually wins CTR, but they usually lose converse rate.

There are exceptions to this and so I wanted to know, I'm going to do discussions in a second, I wanted to know why they lost. So we started looking through all the headline data, what combinations you can come up with. And started to say how often does headline three show for ETAs?

And we noticed there are two sorts of groupings here and so we said why do they have these groupings where these show headline threes almost never and these show headline threes a whole lot? Character limits. There's only so much space Google renders stuff.

So if you're on a mobile device and you're on a small pixel screen, you don't see the third headline. If you're on a desktop, you'll often see headline threes more often, but not always. And so you generally see, the shorter your headlines are, the more often you get the third headline. They show description twos all the time, we'll skip that.

So I want to know, if headline three is being shown so often in these combinations, why are they performing poorly a lot of times? So we saw ads that when headline three was displayed, they did worse than two headlines. It's more characters, you would think it's more time for marketers to do stuff, you'd think it'd be better.

It turns out, it has a lot to do with cohesive messaging. So when you think about ways you can use your third headlines, your headline one is assumptions here, related to ad groups, our very first screen of the day, headline one should be saying, hey, we're relevant.

So we think of headline two and three, we got calls to action, we have benefits, we have features. So we'd see ads with two calls to actions in them or two features in them. So we labeled and did a huge amount of data, which I will spare you going through the methodology here, to say actually, what's our time?

Most calls to actions are horrible. If you go into Google and you make a responsive display ad and you go to more options and say select, there are nine options. Nine. Shop now, buy now, call now and you're probably sick of these calls to actions.

Be creative. The call to action is one of the places you can be creative and really stand out in an ad.

And so this is when we look at RSAs they can be problems because you've got two features, two benefits, two calls to action in an RSA. Google's going to try to figure out, hey, what if I serve two CTAs, what if I serve two benefits? So an ETA, we can actually lay out an ad properly to say, well, we should have a headline, very useful, the benefit, call to action or do we do call to action & benefit?

Headline three is not always displayed, this is what makes everything tricky. So your headline three should only support your headline two, in fact, I wish that Google would drop headline two and not headline three. You want to know how your ad ends.

What's the last thing we say? The end of the sentence, you don't know how your sentence ends and so you have to rely on the headline being strong because it will sometimes end the sentence. But your headline three, you can't make the same claim as you do in headline two. So headline three really needs to support the headline two.

And so this is why RSAs, especially in low volume ad groups, Google can't go through and make 10,000 combinations, when you do 1,000 impressions a month. They can't figure out what to show, which is why hopefully, you're paid well to write good ads. That's your job as a marketer is to understand things.

So, thoughts on ETAs, RSAs, should we use RSAs?

Joel: I agree with a lot of it. I don't like RSA campaigns at all and actually, I haven't really heavily used them at all, because I do like being in control. I think if you are running something very huge, you should definitely run standard ads for the words that matter the most because there's always going to be a few keywords that are driving the most of your campaigns, so at least manually manage those.

And by the way, one thing that I think Brad said is that you're always, you talked about the combination of benefits and features, that's something that I kind of like to test individually. When I first really got into PPC, when I first fell in love with the profession, I first read Perry Marshall's book which is An Intro to Adwords. And then when that wasn't enough for me, I then bought Brad's book.

But one thing that Perry Marshall said that really resonated with me is when people are searching for something, they're having an inner dialogue in their mind about what they're looking for. So if someone is searching for let's say a Lenovo Yoga laptop in the back of their mind they must be thinking, I want something powerful and super portable. Or maybe they're thinking, I want something that's really thin, that could fit in my small laptop case.

I would test different benefits, such as speed and portability, well, versus, fits in small cases. The headline is easy because the headline reflects the search term, but the benefit, that's where I really try to like think of what they want and that's what I experiment with the most and that doesn't really work well with RSA ads and I think you lose a lot with it.

Aaron Levy: So a lot of what we're doing is congruent with what Brad mentioned. Right now, responsive search ad reporting is bad. It's not good. We can't find out what works, we just know that this collection of stuff does something. It's getting better and I'm sure it will get better in the future.

Brad Geddes: Mark over at WordStream put together some data on our RSAs and I think he said and I may not have a number right, but I'm pretty close, that if you use all the headlines and descriptions in an RSA, there are more than 42,000 ways Google can display that ad. So if you think you even want 100 impressions per time to read that, that's over 4 million impressions, just to get a baseline.

Navah Hopkins: I genuinely dislike RSAs for one very specific reason. Google encourages you not to lock in that human intent. So you actually can give some guidance, but they tell you it's not recommended.

And the problem I have with that is that they're actually getting in the way of you allowing your human intelligence, your business acumen, the subtlety that goes into buying personas, what is going into that ad group and saying, no, we don't want your advice, trust us. And the problem with that is that you came up with those lines.

Navah Hopkins: It's the same reason why I love DSAs or dynamic search ads because it teaches me how my people search and what headlines could be interesting. It's a good starting point, but I never let DSA be the full breadwinner, it's the same reason why I would never allow RSAs unless they magically, for whatever reason, allowed human input into the algorithm and we could actually... I don't know, that would actually be really interesting if you could set certain filters for how RSA worked.

But until that day, I genuinely dislike them.

Recap on Advanced Ad Testing

Brad Geddes: So we have a recap.

Navah, of all the wonderful things you said today if someone remembered a single point, what would it be?

Navah Hopkins: Own your star power when it comes to testing, you know more about your business and your profit lines than Google or Bing or Facebook. Don't let them wrest that control away from you.

Brad Geddes: Joel?

Joel: When you opened up with talking about differentiating between impression competitors and click competitors, I mean it's something that I haven't thought about in that way before. I really, really do like that. So I think that's one thing that was very interesting to me.

Also, the way you use combined metrics, I found interesting as well. I think that's something that does resonate too.

Brad Geddes: Aaron?

Aaron Levy: Speak customer language, not marketer language. Don't look at our best practices, don't look at a brand guide, don't look at the words that we want to say. Listen to the customers, what they're saying through their search queries, through click-through rate, through ad testing, figure out what makes people tick, not what some persona research said.

Brad Geddes: Awesome.

Joel: What makes people tick is what they click on.

Brad Geddes: So Joel, do you want to wrap this up?

Joel: It was really great. I had never had a webinar to go 15 minutes over before, but it was totally worth it.

It was really a lot of fun, it was my pleasure doing this and I think that's it for the evening.

Navah Hopkins: Bye guys.

Brad Geddes: Enjoy.

Aaron Levy: Thanks, everyone.


Check out other webinars from this series