Search Ads Evaluation General Guidelines

Prévia do material em texto

Search Ads Evaluation General Guidelines
General Guidelines Overview
Introduction
Search ad rating involves interpreting a user query. A user query is the set of keywords that a user enters into the Google search engine. When rating a search ad, perform the following steps:
1. Review the Google search results page, try to understand the user query, and form an opinion about what the user hopes to accomplish by using a search engine.
2. Use the evaluation criteria found in the following instructions to analyze an advertisement and the advertising experience the user will have if he or she clicks on the ad.
User Intent
An understanding of the user intent is necessary to accurately rate a search ad. The user intent is what the user hopes to accomplish by using the Google search engine. Note that users use the search engine to look for a variety things, and there are many user intents.
Some queries are very easy to understand, others are more difficult, and some may seem impossible to understand. Regardless of its meaning, you must research the query and form an opinion about the user intent.
We strongly advise you to review the Google search results page to determine user intent. In order to objectively determine how promising or unpromising an advertiser offering is for a particular user query, it is important to form an opinion about the user intent before beginning an analysis of the advertisement.
Queries & User Intent
Queries with Multiple Meanings
If a query has multiple meanings, please consider that all meanings can be placed on a spectrum between plausible meanings and highly implausible meanings. When analyzing an advertiser offering, consider what meaning the advertiser uses and where it falls along this spectrum. This will help you determine the appropriate search ad rating.
Plausible Meanings
If a query has several plausible meanings, it is important to consider them all. If an advertiser assumes a particular meaning in an ad or on a landing page, and it is a reasonable meaning, assume that is the meaning that the user intended.
Refer to the following example to better understand plausible meanings. For the user query [java], this query could refer to an island, coffee, or a computer language. With no additional information available, it is impossible to say which meaning the user intended. Ads that respond to any of these meanings are acceptable since all three meanings are reasonably plausible.
Possible but Unlikely Meanings
If an ad or landing page assumes a meaning that is possible but not very likely, this is a secondary interpretation. An ad or landing page that addresses only a secondary interpretation of the query is given a lower rating than an ad that addresses a plausible meaning. Use the Secondary Interpretation of Query flag in this case. Generally, rate an ad or landing page that responds to a secondary interpretation negatively.
Refer to the following example to better understand possible but unlikely meanings: For the query [paris], while there are a number of cities called Paris, unless there is some reason to think that one of the smaller cities is meant, a query mentioning Paris is probably referring to Paris, France. So, an ad for hotels in Paris, Texas instead of Paris, France is probably incorrectly comprehending the user intent. Even if the ad is otherwise a good one, rate it on the negative side of the scale and use the Secondary Interpretation of Query flag.
Implausible Meanings
If an ad or landing page assumes a meaning that is completely implausible, treat it as completely wrong and choose a very negative rating. Do not use the Secondary Interpretation of Query flag if the meaning is clearly implausible.
Refer to the following example to better understand implausible meanings. For the User Query [paris], the query probably refers to the city of Paris, France. If an advertiser interprets the meaning to be plaster of paris, it is almost certainly not addressing the user query. Use a very negative rating.
Misspelled Queries
Users often misspell queries. When evaluating a query, if it is clear what the user means, and the misspelled version of the query has no meaning, ignore the misspelling.
Analysis is more difficult if the query appears to be a misspelling, but the misspelled version has a unique meaning. First consider the query as the user entered it, and then consider if it may be misspelled. If advertisers respond to misspellings, ratings may need to be adjusted.
The following are the 4 possible scenarios for Misspelled Queries:
1. Advertiser Responds to Actual Spelling in Query;
2. Advertiser Responds to a Plausible Correction of Spelling in Query;
3. Advertiser Responds to a Possible but Unlikely Correction of Spelling in Query;
4. Implausible Spelling Correction.
Refer to the following example to better understand misspelled queries: User query: [goodnight moom]. There is a famous children’s book called Goodnight Moon. It is very possible that the user means to type [goodnight moon] but types [goodnight moom] instead. However, there is actually a novel titled Goodnight Moom. While the novel is quite obscure, it might be what the user wants.
1 – Advertiser Responds to Actual Spelling in Query
If the advertiser assumes that the query is correct as it stands (in the example above, assumes the user meant [goodnight moom], treat the advertiser’s query interpretation as acceptable.
You would then need to decide separately how promising the ad or landing page are.
2 – Advertiser Responds to a Plausible Correction of Spelling in Query
If the advertiser assumes that the query is misspelled and addresses a corrected version of the query (in the example above, assumes that the user mistyped and meant [goodnight moon], judge for yourself whether this was a good assumption. If you think it was a reasonable assumption, the ad and landing page are treated as if this were the user intent. Don’t modify your scores to account for the spelling correction, and don’t use the Secondary Interpretation of Query flag.
3 – Advertiser Responds to a Possible but Unlikely Correction of Spelling in Query
If the advertiser assumes a corrected spelling that you think is possible but not very likely, this is a secondary interpretation. An ad or landing page that addresses only a secondary interpretation of the query is given a lower rating than if it had responded to a likely or plausible meaning. The Secondary Interpretation of Query flag must be used. An ad or landing page that responds to a secondary interpretation is generally rated negatively.
4 – Implausible Spelling Correction
If the advertiser’s interpretation of the query is based on a completely unreasonable assumption, treat it as completely wrong and give it a very negative score. The Secondary Interpretation of Query flag is not used in this case.
Continuing with the previous [goodnight moom] example, both query interpretations are reasonable. The only way to know this is to research the query and analyze how the advertiser interprets it.
Queries for which a Reasonable Ad is Impossible
Sometimes a query is either so hard to interpret or so non-commercial in nature that no ad will be a good match. Be careful in these cases—rate the ad and landing page according to how well they actually respond to the query, and do not worry about how hard it would be to show an appropriate ad for that query.
Do not give an ad positive ratings if a better ad for the query cannot be determined or if it seems like it was a good try. Rate it positively only if it addresses the query intent. If the query intent cannot be determined, the ad must be rated negatively. For example, a query of [www] or [when did] is not complete enough to serve a proper ad.
Unrateable Queries
In some rare cases, a query may appear that is the result of an error in how the task was added to the evaluation system. For example, a query may appear in the incorrect rating language, or a query of jumbled characters may appear that, after research, hasno discernible meaning.
Do not attempt to provide Ad creative or Landing page ratings for queries like this. Instead, use one of the flags provided in the Query Flags section. These flags are only present on the first page of a Search Ads task, the Ad Creative rating section.
Location and Queries
Approximate Query Location
Many tasks will include a map indicating the geographical region where the query was entered. If you are unfamiliar with the location in the map, you can zoom in and out of the map to familiarize yourself with it. Knowing the approximate location where the query was entered may help you decide how relevant the Ad Creative or Landing Page is to a user. Some ads will be more relevant given the location, some will be less relevant. For some ads, knowing the location will not make any difference.
When Location Matters
Sometimes the query, the ad, or both may refer to a specific geographic location. Even when the approximate query location is available, it can sometimes be difficult to determine how to handle a location specific advertisement.
Location Mismatch between Query and Approximate Query Location
If the query contains geographical information, you should use the information available in the query instead of the map, especially if the two conflict. For example, if the query is [pizza in new york], but the map indicates Los Angeles, pizza restaurants in Los Angeles are a bad result.
Query Specifies a Location but Ad Does Not
If a query specifies a location, take it into account when evaluating the ad. Sometimes research is required to determine whether the product or service is compatible with the location. This research is required before you can choose an Ad Creative rating score.
User query: [pizza Santa Monica]. If an ad pointed to the main Round Table Pizza chain homepage, but didn’t mention this Californian city, this might initially seem useful. However, if, upon using the location finder on the site, there are no locations within a reasonable distance of Santa Monica, this ad is probably not useful.
Ad Specifies Location but Query Does Not
If the query does not specify a location and the approximate query location is also not available, then evaluate the ad as if the user were in an appropriate location. For example, if the query is [pizza] and the ad is for a pizza restaurant in Barstow, California, assume that the user was in Barstow, California.
Neither Query nor Ad Specify a Location
If the query does not specify a location and the ad does not either, then assume that the user can be anywhere in the target country of the rating language. Ignore the approximate query location, if one is provided.
Location Mismatch between Query and Ad
Assuming there is a match between the product or service and the query and ad, take the location proximity into account in evaluating an ad. If the ad for a given query specifies a different and incompatible location, this makes it a worse ad. For example if the query is [pizza Santa Monica] and the ad is for a pizza restaurant in Manhattan, this is very unpromising.
However, an ad returned for pizza in a different but nearby location, like a neighboring town, could be useful, and this ad might not be as bad as the previous example. Use common sense to determine if the ad exceeds a reasonable distance for the user, and an acceptable distance may vary depending on the query.
For example, a user may be willing to travel farther to buy a new car than to get a haircut or go to the supermarket. For certain queries, serving an ad with a completely different location may still be promising. For example, if a user in the United States is looking for [vacation in Australia], then an ad for “vacation in New Zealand” is not necessarily a bad ad since the user is likely to be willing to travel a long distance for a vacation.
Search Ad Rating: How Promising is this Ad?
Evaluate the Ad Creative. A promising ad is one that seems like it will give the user what he or she wants. An unpromising ad looks like it will be disappointing, unhelpful, dangerous, or irrelevant.
Use the slider bar to select from four possible ratings:
· Very Promising
· Somewhat Promising
· Somewhat Unpromising
· Very Unpromising
Very Promising and Somewhat Promising are positive ratings: use them for ads that look like they’re good ads for the user to see that would be worth clicking on. Somewhat Unpromising and Very Unpromising are negative ratings: use them for ads that look like they’re bad ads for the user that aren’t worth clicking on. Consider the following factors while evaluating an ad:
Satisfying the User
Does the ad offer something that will satisfy the user? An ad that has nothing to do with what the user wants is always very unpromising. An ad cannot be promising if it is not relevant. However, being relevant is not enough to make an ad good.
Correct Meaning of the Query
Does the ad address the correct meaning of the query? An ad for a car dealership does not address the query [cars 2] that is the name of a movie. Even if it would be a good ad for some other car related query, it is a completely terrible and unpromising ad for that query.
Clarity and General Appeal
Is the ad written in a clear, appealing way? An ad that makes sense and does not have mistakes, hard to understand language, or awkward phrasing can be good; and, an ad that looks stupid, looks like it was written by a machine, is unintentionally funny, or just does not make sense is usually bad.
Does the ad clearly state what the advertiser offers? A good ad is easy to understand. A bad ad may be overly vague or may not communicate enough information to conclude the ad will lead to a positive advertising experience.
Potentially Scammy Ad
Does the ad look like a scam? An ad that seems too good to be true, sleazy, or deceptive to users is usually bad.
Promise of Additional Links
Some ads contain multiple links to different sections of the website. You do not need to click on these links, and when you are rating the creative, you cannot click on them. However, if these links look promising or useful, this may be a reason to increase your Ad Creative score. If links look unpromising, confusing, or useless, this may be a reason to decrease the Ad Creative score. See the Ads with Additional Features section for more detailed guidance.
Advertiser is a Different Merchant or Provider from Query
Sometimes a query will specify both a product or service and a particular merchant or provider. If the ad offers the desired product or service from a merchant or provider that is not the one specified in the query, it should not be considered a negative user experience unless there is another issue with the ad.
Analyzing the Advertiser Visible URL
The web address (also called the visible URL) displayed in the ad can provide clues about how promising or unpromising an ad may be. The visible URL can affect your evaluation in the following three ways:
· If you are familiar with the advertiser based on the URL displayed in the ad, you may use your background knowledge when rating the ad. Just use the Used Prior Knowledge In Judging Advertiser flag.
· If you aren't familiar with the merchant, assume the merchant is legitimate, even if that's not how you behave in your own online activities. Important note: this only applies if there's nothing in the URL that looks suspicious (see next bullet point).
· If the URL itself makes you suspicious, don’t hesitate to mark the ad bad. For example, an ad for online book shopping that looks very good except that the URL of the merchant is “www.amazom.com” is pretty suspicious, that looks like the merchant is trying to trick you into thinking you’re going to “amazon.com”.
This is likely a scammy ad, and if you think an ad is scammy, it deserves a bad rating. (Other tricks of this sort in addition to misspellings in URLs include adding random numbers, unexpected extensions, or subdomains to create URLs like www.amazon22.com, windows7onet.in, or windows7.customersupport­us.net).Do not assume an ad is promising just because it contains the same words as the query. Do more than note that the words match—machines can tell us this. We need human judgment: tell us whether a human being will find an ad appealing. If a user is looking for [ blue pants ] an ad that says “PANTS BLUE BLUE PANTS www.bargainautoparts.com” is likely a bad ad even though it has the words “blue pants”.
Distinguishing Between Very Promising and Somewhat Promising
If an ad looks like it will lead to a page that satisfies the user intent, it deserves a rating of Somewhat Promising or Very Promising. A good ad deserves one of the ratings described in the following two sections.
Very Promising
A Very Promising ad should look like it points to a page where a user can find almost exactly what is described in the query. If the user is looking for a particular product, the ad should appear to point to a page for that product. If the user seeks a particular kind of store, the ad should appear to point to a store of that kind. If the landing page of a Very Promising ad does not satisfy the user intent, it will be a surprise and a disappointment to the user.
Somewhat Promising
A Somewhat Promising ad should also appear to take the user to a page where the product he or she is looking for can be found; however, rather than appearing to point to a page where the user can find exactly what he or she wants, a Somewhat Promising ad might do one of the following:
· It might look like it points in the right direction but not exactly at the target. For example, if the user seeks a specific model of camera, an ad that looks like it will point to a reputable camera store’s main page is Somewhat Promising.
· It might look like it points to something that might satisfy the user intent but is not exactly what he or she wanted. For example, if the user seeks a particular model of camera, a Somewhat Promising ad might point to a slightly different but reasonably similar model of camera.
· Sometimes it is just not possible to be confident about what the user seeks. If an ad seems to point in the right general direction but there is no way to tell exactly what the user wanted, Somewhat Promising is the highest possible rating.
Distinguishing Between Positive and Negative Ad Creative Rating
Evaluate the Ad Creative and weigh the criteria for Very Promising/Somewhat Promising against the criteria for Somewhat Unpromising/Very Unpromising. If you find that positive elements and negative elements both seem applicable to the creative you’re evaluating, ask yourself which side of the positive/negative division seems to be a more reasonable fit and choose a rating on that side.
Distinguishing Between Somewhat Unpromising and Very Unpromising
It is especially important to distinguish between ads that are simply bad and ads that are very bad for the user entering the query. The following section provides additional guidance to distinguish between the Somewhat Unpromising and Very Unpromising Ad Creative rating.
Somewhat Unpromising
A Somewhat Unpromising ad generally isn’t a great ad to show the user, but it is likely that some subset of users may find it useful. Even if the creative is not of the exact same topic as the query, as long as there is some clearly related task or intent, there are some users who may find the creative appealing.
· One example is if the user seeks [weight loss pills] and the ad is for “diet tips” or “exercise machines”. These types of ads should be rated as Somewhat Unpromising. They don’t directly provide what the user is looking for, but could be somewhat useful to the user so don’t deserve the lowest ratings.
If it is not really clear whether users will find the ad useful, rate it as Somewhat Unpromising.
One example is if a user is searching for some information and the ad asks the user to search for the same information again elsewhere. It is hard to know in these cases whether the ad will be able to provide anything useful to the user, since he/she is being asked to repeat the same action again possibly just to get similar results.
Please view the Google search results for the query to get an understanding of what the user currently sees and what information he/she currently has access to. If you believe that the ad won’t provide any additional information from what is already presented to the user, rate it as Somewhat Unpromising.
However, if you believe that clicking the ad will provide additional useful information to the user, don’t rate it as Somewhat Unpromising­­ give it a higher rating.
One example that would deserve a higher rating than Somewhat Unpromising is if the user is searching to buy a particular item and the ad is asking the user to search for that particular item across numerous stores and merchants. Another example that would again deserve a higher rating is a query for some specific industrial machinery part, and an ad inviting the user to repeat the search on a search engine devoted to machine part sales and manufacture.
· Sometimes a query specifies a location, and the ad targets a different location. For these specific examples, please refer to the When Location Matters section.
Very Unpromising
There are several cases where Very Unpromising is the only appropriate rating.
· Very Unpromising ads have no reasonable chance of satisfying the user. Try to put yourself in the user's mind-set ­ is it possible at all that the Ad Creative offers something useful to the user? If there is no reason at all to think that the user will find the Ad Creative useful, rate it Very Unpromising. (Note: you might think “It’s always possible that someone might find anything useful, even though it has nothing to do with the query.” Don’t go that far!).
· If the Ad Creative looks like a scam, or leads the user to harm, rate it as Very Unpromising.
· If the Ad Creative falls into one of the categories listed in the Machine Generated Ads section, rate it as Very Unpromising.
· If the creative promises to do the impossible, such as selling a person or city, rate it Very Unpromising.
· Just because there is a strong term overlap between the query and Ad Creative does not mean the ad is a good match for the query. If the user is searching for [homeowners insurance] and the ad is for “medical insurance”, the user will very likely not find the Ad Creative useful and you should rate it as Very Unpromising.
Machine‐generated Ads
Some ads are partially auto generated to take words from the query and place them in the creative text. There is nothing wrong with this in itself. For example, if the query is [xbox 360 used] and the creative says “Buy a used xBox 360 on eBay”, that’s a good ad. Unfortunately, sometimes these machine generated ads turn out very badly. Very Unpromising Ad Creatives may have some of the following issues:
· Things offered that cannot be bought:
User Query [san diego, ca]. An ad that says “Buy San Diego cheap on eBay” is ridiculous, you can’t buy a city. Ads that are unintentionally ridiculous, horrible, or offensive, by suggesting that you can buy concepts, human beings, body parts, criminal acts, or similar things are Very Unpromising.
· Part of the query removed, substantially changing the meaning:
User query: [roses lime juice]. An ad that offers the action, “Buy roses”, has left out so much of the query that the entire meaning has changed. By taking only part of the text of the query what remains substantially changes the meaning.
· Part of the query removed, resulting in overly general ad:
User query: [how do I remove gum from satin]. An ad that offers “Get information on how to remove”, is nearly meaningless, too much has been removed from the query. By taking only part of the text of the query, the result is far too general to be promising for the user query.
· Nonsensical, jumbled, or ungrammatical Ad Creative:
User query: [how do I remove gum from satin]. An ad that says “Search for how do I remove gum” or “Find how do I remove gum from satin” is awkward and ungrammatical.Ads that end up nonsensical, jumbled, or ungrammatical because a query has been crammed into a space where it doesn’t really belong is Very Unpromising.
· Be on the lookout for these types of ads. If you’re not paying close attention to how the ads actually look and sound, it can be easy to think these look fine—but to a user who is actually reading the text, they can look laughable, annoying, or foolish, and in some cases, deeply offensive or hurtful. Even those that just look sort of silly or awkward are very bad.
Ads with Additional Features
Some Ad Creatives are just text and a single link to the advertiser page. Other pages contain additional features that may or may not provide something valuable to the user. Ad Creatives may contain maps, videos, images, star ratings from customers, multiple links to specific pages on the advertiser site, and a variety of other features.
These additional features may affect the quality of an Ad Creative. If the special features add to or detract from the appeal, informativeness, or usefulness of an ad, the Ad Creative score can be raised or lowered.
For an ad that contains only text and a single link to an advertiser page, use only the previous criteria in making a decision. For ads that contain anything in addition to text and a one link, consider the following factors, and decide whether to raise or lower your score:
· If an ad does not deserve a score of Somewhat Promising or Very Promising based on the previous criteria, be cautious about giving it a positive rating just because of additional features.
· An ad that deserves a score of Somewhat Promising or Very Promising based on the previous criteria can be given a negative rating if additional features detract from it severely.
· Use common sense when deciding whether additional features improve or detract from an ad enough to move it between Somewhat Promising and Very Promising scores in each category.
· An ad that is scammy or harmful can never be improved by additional features.
· Additional feature should relate to the user intent in a sensible way. If the user is looking for information about a current movie, a movie trailer in the Ad Creative relates to the user in a sensible way, but a map to the movie studio where the film was made does not. The trailer probably improves the experience, but the map detracts from it.
· Where an additional feature is relevant to the user intent, it should be informative, easy to use, and clear. If it is confusing, boring, or hard to figure out, it may either detract from the experience or just fail to improve it.
· An ad may have multiple additional features. Consider all of them together when determining your Ad Creative score.
· Raise or lower your rating by a small amount if the additional feature has little impact on the Ad Creative. Raise or lower it a large amount, according to the previous criteria, if the additional feature has a big impact.
· Do not consider the physical size of an Ad Creative rating it. If the size of an Ad Creative causes it to display incorrectly in the ads rating interface, alert an administrator but ignore the issue while rating it.
Search Ad Rating: Does Landing Page Satisfy User Intent?
Use the slider bar to select from the following four rating categories while determining how likely it is that a landing page will satisfy the user intent. Only consider the user query and the landing page, and ignore the Ad Creative completely.
· Satisfaction Likely
· Satisfaction Possible
· Dissatisfaction Possible
· Dissatisfaction Likely
Satisfaction Likely and Satisfaction Possible are positive ratings that satisfy the user query. Dissatisfaction Possible and Dissatisfaction Likely are negative ratings that do not satisfy the user query.
Only rate the landing page that opens after clicking on the Visit Landing Page button. Do not base your score on pages that are accessible by clicking on links in the body of the Ad Creative. NEVER copy and paste a link to visit the page, and NEVER manually change the URL.
The Principle of Landing Page Evaluation
The fundamental principle of landing page evaluation is this: the user starts a search on Google.com with a goal in mind. The user then enters a query and reviews Google’s search results and ads. The user then clicks on the ad currently being reviewed, and that ad takes the user to the landing page. Keep in mind that in order for a user to have a positive experience with an advertiser landing page, he or she should be closer to the goal expressed in the query, otherwise it is a negative experience. The section below helps frame how distance from the user’s goal helps determine a landing page rating score.
Distance from the User’s Goal
Carefully review the Google search results page to determine the distance from the user’s goal. Does the Landing Page move user closer to his or her goal, further from the goal, or neither closer nor further from the goal?
If the user is closer to the goal, the landing page deserves a positive rating. For example, if the user is hoping to buy a specific camera, and the landing page is a store offering that camera for sale, the user has come closer to accomplishing his or her goal.
If the user is further from the goal, the landing page deserves a negative rating. If the user is hoping to buy a specific camera, and the landing page is a store offering pet food, this is a dead end. The user will need to go back to the search page or start a new search, so he or she is actually further from the goal than before clicking on the ad.
If the user is neither closer to nor further from the goal, the landing page deserves a negative rating. If the user is on a Google search results page and clicks on an ad that just takes them to a page of similar search results, which overall did not provide any additional value, no progress has been made; the user is no closer to or further from the goal than before clicking the ad.
Deciding this is not an exact science. Rely on good judgment. The following guidelines more deeply explain how to generally rate landing pages, but they do not explain how to rate a landing page in every situation.
Distinguishing Between Satisfaction Likely and Satisfaction Possible
Satisfaction Possible and Satisfaction Likely are positive ratings. If the landing page offers the user exactly or very nearly what he or she wants, use a Satisfaction Likely or Satisfaction Possible rating.
Satisfaction Likely
To receive this rating, a landing page must offer just what the user looked for. If the user wants car reviews, it should offer car reviews. If the user wants car reviews about a specific model, it should offer car reviews about exactly that model. If the user wants a category of product, the landing page should be devoted to or include that exact category of product. For a Satisfaction Likely rating, what the user is looking for should be apparent with no additional action needed by the user. It is permissible, however, to click on a link to get detailed information.