Is Google playing with a new search result display… and also testing image recognition?

Guest blogger Jenise Uehara Henrikson is in internet marketing, focusing on urban clothing and streetwear.

Update: This article is NOT about Google’s “universal search” which includes images, as well as news and video mixed in the site results. This post documents something new that Google Search appears to be testing only in smaller markets. What’s different about it? Read the (entire) post.

I was doing some research on google.se (Swedish localized Google) when I came across some interesting search results.

In case your Swedish is rusty 🙂 “vigselringar” means “wedding rings”.

How is the above different from the image results in Google’s universal search?

1. Unlike universal search, the images above are all from the same website.
2. The images are listed in-line with the URL listing. Universal search displays images as a separate result.
3. When any of the images are clicked, all go directly to the site URL listed, not Google Images.
4. The images displayed are smaller than images shown in Google Universal Search.
5. Notice that there are no other Universal Search elements: no news articles, no videos.

Why do the images show for the third search result and not the first?

I’m hazarding a guess that Google is using a separate algorithm in order to determine which listing gets their thumbnails displayed:

Attempt to identify keyword-relevant, onpage images.  Add them up. The site with the most KW-relevant, onpage images = winner.

Total number of unique, onpage images:

1. vigselringar.se: 32
2. vigselbutiken.se: 32
3. hellkes.se: 33
4. handsweden.se: 11
5. brollopsguiden.se: 38 (note this site has the most images, but NONE of wedding rings)
6. vigselring.nu: 11
7. juveler.com: 21
8. guldgruvan.se: 18
9. guldbolaget.se: 5
10. forlovningsringar.se: 2

Number of images where image focus is a wedding ring:

1. vigselringar.se: 12
2. vigselbutiken.se: 8
3. hellkes.se: 19 = winner
4. handsweden.se: 1
5. brollopsguiden.se: None (not including ads)
6. vigselring.nu: 2
7. juveler.com: none
8. guldgruvan.se: 4
9. guldbolaget.se: uses flash
10. forlovningsringar.se: 1

How is Google selecting which thumbnails to display in the results?

Some possible criteria:

• Selection by popularity.
• Random selection.
• Google image labeler. I did not do a comprehensive search, but it appears google.se does not have the image labeler game available on the google.com site.
• Selection by relevancy to head keywords. This could indicate Google is using some kind of image recognition technology, because most of the thumbnails above do not have keywords in the alt text or file names.

Below are the 9 of 34 images that Google picked to be listed in the search results. (One image has a question mark, as it is actually slightly different from the one displayed in the search results, although it appears to be the same ring. Possible it was changed recently.)

When searching for different keywords, I found that the thumbnail selection is identical for the same site.

Below are search results for “hellkes”. Hellke is a common Swedish surname and has no intrinsic meaning linked to jewelry or rings. Same images are displayed.

Let’s try a different category. Below is a search for “horoskop” (“horoscope”).

In this search, there are two thumbnail displays for two different listings. Note again most of the images chosen are directly relevant to the keyword, as they are representations of astrological signs.

The thumbnails do not appear in every search on Google.se. Here’s universal search results in the usual format for keyword “Skansen” which is a famous Swedish museum and zoo.

More data:
1) Not logged in to Google
2) The thumbnails only show in my Firefox browser, but not in Internet Explorer
3) The thumbnails do not show in other computers’ browsers on same IP
4) Only observed on first page results, if they appear at all

Is Google Search experimenting on google.se? I look forward to the SEO experts following up on this.