Last update: 2019-03-25
Instructions
In this task you will evaluate the quality of an ad in the context of a user’s search experience. Ads are typically comprised of both an ad creative and a landing page; however you may also encounter tasks where only one or the other is provided, or where the ad experience is more interactive. In either case, you will have to do some research, jot down your preliminary findings in a series of worksheet questions, and finally provide an overall rating that is informed by your worksheet responses. In more detail, you will:
- Understand what the user wants.
Behind every query is a real person who wants Google to help them find something. Click on the query to open Google’s search results page and use the links presented there to try to understand the user’s intent and likely interests. If the user’s intent is ambiguous, you’ll indicate this in the worksheet.
- Think about what the ad offers.
An ad creative is considered good if it anticipates what the user wants and makes it obvious how the ad content would be useful. You’ll compare what the ad offers to what the user seems to want and add your observations to the worksheet.
- See what it’s like to follow the ad.
A landing page is considered good if it helps the ad deliver on its promises when the user chooses to interact with it. You’ll follow a link to the ad’s landing page and answer a few more worksheet questions about the experience.
- Assess the quality of the ad.
In the previous steps, you will have jotted down some preliminary answers to factors that affect whether the ad (i.e., ad creative and/or landing page) has the potential to enhance the user’s search experience. In this last step, you’ll use your research from the previous steps and your best judgment to determine a final rating for whether the ad enhances the user’s search experience.
Interpreting the query
Research the user query by looking at search results and anything else you need to understand the user’s intent. To help you determine how useful an ad might be for a query, there are three considerations that you will be asked to assess.
1. Interpretation of the query
Many words or phrases have more than one interpretation, and you need to determine whether the ad matches the meaning that the user has in mind. This consideration is not about how good the ad is, but simply whether the ad is related to a likely or unlikely interpretation of the query
For example, the query [highlander] can have multiple meanings.
- There are at least three different likely interpretations: Many people may be looking for the Toyota Highlander SUV or the 1986 fantasy movie Highlander, and without more context it’s hard to know which one the user means; or the user might mean the original definition, an inhabitant of the Highlands of Scotland.
- A possible but unlikely interpretation is the Highlander Research and Education Center in New Market, Tennessee: it is realistic to think that under some circumstances the user might intend that meaning (such as if the user lives in Tennessee and had just been browsing a list of organizations that mentioned it), but without further context, it’s much more likely the user meant another interpretation.
- You will also come across unrealistic interpretations, such as misinterpretations, where it is not realistic to imagine the user had that meaning in mind; an example is if the user searches for [transformers movie showtimes] and sees an ad for “Custom power transformers”; despite the matching word, it is clear the user meant the Transformers movie franchise and not power transformers.
2. Closeness of intent
Once you have determined that the ad is related to a likely or at least a possible meaning of the query, you need to assess whether it addresses the user’s expressed intent or a different one. Note that this is a judgment about what problem the ad offers to solve; you don’t need to consider how well the ad solves the problem.
- An ad might address the user intent exactly; note this doesn’t necessarily mean it is a good solution for the user. For example, the user query [outdoor work boots women] intends to find women’s boots for working outdoors; an ad that directly offers such boots for sale and an ad that reviews and compares such boots from various other sellers may offer different kinds of value to the user, but they both directly address the expressed intent.
- An ad that addresses a nearby intent does not solve exactly the expressed need, but it does offer something that often goes together, or offers an approximate solution. For example, on the query [outdoor work boots women], an ad offering extra-durable laces for work boots would address a nearby intent because someone looking for work boots might also be interested in better laces to go with the boots.
- An ad can be topically related but not address a nearby intent at all; for example, on the query [outdoor work boots women], an ad offering a book on the history of work footwear has some topical overlap, but it does not address any intent that typically accompanies buying work boots.
3. Substitutability
An ad may offer a product or service that has some differences from exactly what the user is looking for. It is important to consider whether the solution offered and its properties are appropriate substitutes or alternatives for what the user seeks. This consideration is where you will make the most direct judgments about how well the ad satisfies the user’s needs.
- When considering any requirements expressed or implied by the query, you’ll need to judge whether they are necessary or important to meeting the user’s need, and whether users might be willing to accept the substitute or not.
- An ad might offer a reasonable alternative, which isn’t exactly what the user seeks but most likely could fill the same need in a satisfactory way. For example, a user searching for [Comcast high speed internet price] might find an ad for a similar AT&T internet package reasonable, since AT&T as a competitor to Comcast likely provides a similar service. In general, if the ad seems to offer what the user is looking for except it’s from a competing brand or provider, you should consider it reasonable unless the user’s query makes it clear they are not open to alternative brands.
- An ad might offer a radically different alternative, which tries to meet the user’s general need in a different way. For example, on the query [Pampers 72 count diapers], an ad for a reusable diaper service offers the user a very different but potentially useful way to address the need of diapers. Most users would not consider the two interchangeable, but some users might be interested in the alternative.
Tips
- Worksheet questions all have their own instructions and examples. Review these as frequently as necessary when you’re completing the task.
- You’re free to answer however you’d like on the final Ad Quality question, regardless of what you put in the worksheet, but we’ll ask you to double check your rating if it seems to disagree with your worksheet findings.
- If there’s a technical problem with any of the inputs, use the appropriate unrateable flag to skip over part of the task.
