ACR (Automatic Content Recognition)

What is ACR

Automatic content recognition (ACR) is an identification technology to recognise content played on a media device or present in a media file. This enables users quickly obtain detailed information about the content they have just experienced without any text based input or search efforts.

ACR can help users deal with multimedia more effective and make applications more intelligent.

How ACR works

Take the most popular music recognition and discovery app Shazam for example. The ACR workflow is as follows.

The fingerprinting algorithm processes the signal of audios and extracts digital features called fingerprints for each audio.

Fingerprints are very discriminative so the system can use them to identify the audio they belongs to.

Fingerprints are also robust which means they can resist the environment noise and this make it possible to identify recorded audios in rather noisy environments.

If the system finds matched fingerprints of the query snippet, it can determine the most like audio in database and gives the position of the snippet in the source audio

The mission of ACR is completed when the system finds the match.

ACR Algorithms

Algorithms of ACR include but not limited to Audio Fingerprinting, audio watermarking, image watermarking.

After long-term exploration by research institutions and companies, Audio Fingerprinting has been proved to be the kind of most stable, effective and widely applicable algorithm. 

The lack of watermarking is that it requires inserting inaudible or invisible digital tags containing information about the content into the content itself, prior to distribution. For example, a broadcast encoder might insert a watermark every few seconds that could be used to identify to broadcast channel, program id, and timestamp. And this is not realistic and flexible for most applications. 

For more information, please refer to ACR.

ACR Applications

Content Identification

ACR technology helps audiences easily retrieve information about the content they watched. For smart TVs and applications with ACR technology embedded the audience can check the name of the song which is played or descriptions of the movie they watched. In addition to that, the identified video and music content can be linked to internet content providers for on-demand viewing, third parties for additional background information, or complementary media.

Broadcast Monitoring

For advertisers and content owners, it is vital to know when and where their content has been played. Traditionally agencies or advertisers have to manually audit the presentation. At scale it only can be checked through a statistical sampling method. ACR technology enables automatic monitoring of the content played in TV. Information like the time of play, duration, frequency can be achieved without any manual effort.

Content Enhancement

Because devices can be “aware” of content being watched or listened to, second screen devices can feed users complementary content beyond what is presented on the primary viewing screen. ACR technology can not only identify the content, but also it can identify the precise location within the content. Thus, additional information can be presented to the user. ACR can enable a variety of interactive features such as polls, coupons, lottery or purchase of goods based on timestamp.

Audience Measurement

Real-time audience measurement metrics are now achievable by applying ACR technology into smart TVs, set top boxes and mobile devices such as smart phones and tables. This measurement data is highly essential to quantify audience consumption to set advertising pricing policies.