search
top

Google Video Test Results: Captions Vs Description Vs Speech Recognition

I had a thought after Google Voice came out: If Google has voice recognition capabilities, shouldn’t they be able to recognize keywords said in videos? And if so, might SEOs get involved in keyword research as part of the scriptwriting process? It was a passing thought, which was quickly dismissed at the time. But then Google announced automated captioning on YouTube:

“To help address this challenge, we’ve combined Google’s automatic speech recognition (ASR) technology with the YouTube caption system to offer automatic captions, or auto-caps for short.”

As of right now you still have to upload the text for captioning to work, but the fact that YouTube can sync up the timing of the words to the part of the video where those words are being said – even without time codes – tells us that they do indeed have ASR technology up and running. So again… why wouldn’t Google use that technology to help improve the relevancy of their YouTube, Google Video, and Universal Search results?

So Nathan Joynt and I decided to run a little test to see if automated captioning and ASR have an impact on search rankings (or to be more precise since this test doesn’t give the factors a relative importance next to other factors) if they are indexed and used as ranking factors at all. Here is what we did…

The Test

  • We made three different videos, which are all linked to below.
  • Each video was given its own three-word phrase that, at the time, produced zero results on YouTube, Google.com and Video.Google.com.
  • One video had its phrase only in the video itself (and nowhere else); one video had the phrase in its description as well as the video itself (but nowhere else); and one video had the phrase in the captions as well as the video itself (but nowhere else).
  • Videos were titled Test1, Test2, and Test3 and each was tagged with its own title; nothing else.
  • We linked to each video once from a single blog post using their URLs as anchor text.
  • We turned off embedding, didn’t allow commenting, video responses or any other user-generated aspect that could muck up the test.

The Keyword Phrases

Video #1 was our test to see if Google is automatically indexing keywords said within videos by using their speech recognition technology. Here are the exact words that are said in the video, which also describe how the test was structured:

This is test number 1 in which pizza apartment hat is said by Mr. Cutts. No transcript will be uploaded. These words will not be in the description.

Video #2 was a control showing, obviously, that Google indexes keywords within the video description. Here are the exact words that are said in the video, which also describe how the test was structured:

This is test number 2 in which foot glasses banana is said by Mr. Cutts. No transcript will be uploaded. These words WILL be in the description.

Video #3 was our test to see if Google indexes keywords from captions when automated captioning is turned on. Here are the exact words that are said in the video, which also describe how the test was structured:

This is test number 1 in which coffee necklace iphone is said by Mr. Cutts. We will upload a transcript and turn on captions. These words will not be in the description.

Our Predictions

Our best guess was that Video #2 and Video #3 would be found in the YouTube search results for their keyword phrases, but not Video #1. While it is quite obvious that Google uses speech recognition technology on YouTube, we weren’t convinced that they’ve gone as far as using it for ranking purposes on every video that gets uploaded – YET. We also hypothesized, since Google Universal leans more toward traditional ranking factors (eg page rank), that we probably wouldn’t see ANY of our videos in the universal results on Google.com. However, we figured we would see the same thing on Google Video search that we would see on YouTube.

The Results

Out predictions were correct. YouTube and Google Video search both index and factor into rankings the keywords found in Caption files but not (yet) the keywords said in videos without captions or transcripts. So there isn’t anything “groundbreaking” here, but it’s always better to know for sure than to just guess. We can also assume that uploading captions files, even if it is just a text file without any time codes, helps you rank for words that are said in the video. Then again… so does a transcript in the description.

The Videos (Yes, they’re kinda freaky)

Screengrab from Video Three

Test Video #1 – Youtube Video Description: “This is test number 1 in which **##* @@@*# **!@# is said by Mr. Cutts. No transcript will be uploaded. These words will not be in the description. They ONLY appear in the video as voice.” This video did NOT rank for it’s phrase ”pizza apartment hat” on YouTube,¬† Google Video Search, or on Google universal search. The only place in which the phrase appeared was verbally in the video. This shows that Google is NOT automatically indexing the speech content of videos without the user first uploading a caption file / transcript, or at least turning on auto-captioning in the few profiles that have that ability so far. I still believe it is only a matter of time before Google does this, however.

Test Video #2 - Youtube Video Description: “This is test number 2 in which foot glasses banana is said by Mr. Cutts. No transcript will be uploaded. These words WILL be in the description.” This video ranked for it’s phrase ‘foot glasses banana’ on YouTube and Google Video Search, but not on Google universal search. You would think that Google would at least show it as a normal web result, but they probably didn’t due to lack of page rank and, possibly, the title “Test2.mov”. If this video had more page rank and a better title would it have a chance at ranking in the universal search results? Yes. And this begs the same question of the video below…

Test Video #3 – Youtube Video Description: “This is test number 3 in which #(#(##, %^@%@%@ , **@@ is said by Mr. Cutts. We will upload a transcript and turn on captions. However those three words will not show up in the description. Big G do you look at and index captions for ranking.” This video ranked for it’s phrase ‘coffee necklace iphone’ on YouTube and Google Video Search, but not on Google universal search. It proves that Google Video and YouTube both take caption text into account when indexing and ranking videos. A few things worth noting: #1 We uploaded a caption file in the form of a txt document without time stamps. This is different than having your video automatically captioned without uploading a file. As of yet, normal user profiles have to upload caption¬† files or transcripts to get captions. In the future I’d like to see if automated captioning works in the same way. #2 A YouTube clone/scraper site called zetaspace did manage to rank for this phrase. It was a tag archive page that managed to include the caption text as on-page, spiderable text.

The Screen Grabs

4 Responses to “Google Video Test Results: Captions Vs Description Vs Speech Recognition”

  1. Troy Dunn says:

    Excellent post! Our Video Department has been slowly improving our video SEO Program and knowing this information will help. Thanks for taking the time to test this. I was waiting to see what your results were after your post last week.

  2. Anna says:

    That is such an elegant experiment! The biology major in me couldn’t stop reading… :-)

  3. Russ says:

    Excellent research.

  4. Dave says:

    Nice to meet you at SMX West this week, good luck w/ the farm, and great case study.

  5. [...] Google Video Test Results: Captions Vs Description Vs Speech Recognition [...]

top