Automatic Content Recognition
Automatic content recognition enables the identification of the media streams and content actually viewed or listened to based on captured audio signals. In audience measurement, it relies on complementary technologies to identify TV, radio, streaming, replay and podcast consumption.
How Audio Helps Identify the Streams and Programs People Watch
In a media landscape where consumption patterns are fragmented across television, radio, Streaming Streaming A technical method of digitally transmitting data to enable the playback of a continuous flow of video and audio material. Unlike terrestrial or satellite TV broadcasting, this technology requires a two-way network (internet) to send periodic requests to the streaming server in order to receive video or audio micro-files. Find out more , live broadcasts, Catch-up TV Catch-up TV This refers to the TV viewing modes whereby a programme is watched on demand after its live broadcast. See Watch again, Catch-up, Time-shifted viewing. Find out more , and Podcasts, the automatic identification of streams and content has become a key challenge for Audience Measurement Audience Measurement Quantitative study of the frequency of use of media. Find out more . By leveraging the audio signal captured by Audience Audience The set of listeners, viewers, or TV viewers in which each individual is equally weighted, regardless of their listening/viewing duration or frequency of contact. Find out more Meters, Automatic Content Recognition (ACR) technologies make it possible to identify the content that is actually being consumed, regardless of the mode mode In statistics, the mode is the most common value. Find out more of broadcast broadcast A mode of transmission whereby a transmitter sends the same audio and video stream to many receivers within a coverage area. In the context of computer networks, broadcast mode is contrasted with “multicast” and “unicast” modes, which respectively refer to the transmission of a stream to a given group of recipient devices and to a single recipient device. OR : Putting a show or a programme on television or radio. This term refers to a programme that is broadcast live as opposed to on-demand. Find out more .
This automatic content recognition relies primarily on two approaches: audio Watermarking Watermarking The technology used in television and radio audience measurement. Watermarking consists of inserting an (audio) tag into broadcasts that is inaudible to the human ear. This tag identifies the channel airing the programme, as well as regular markers of the broadcast time. Audience meters installed in panellist homes detect these signals. Find out more and audio Fingerprinting Fingerprinting A technology used to identify audio-visual content. It involves taking a "sample" of content that has been viewed or listened to on any platform whatsoever and then comparing it against a reference database containing all the programme extracts for a channel or a catalogue of programmes. If this “sample” turns out to belong to that database, then the content from which it was sourced can be identified. Find out more . They share the same objective—to reliably identify content, accurately, and in a way that is usable for Audience Measurement—but address different constraints depending on whether the content is Linear Linear This is when a live television program is watched exactly at the time it airs, in timeshifting (control over live TV) or private delay (personal recording). Find out more , non-linear, or detected in a real-world listening environment (such as portable Audience Measurement).
Qu’est-ce que l’ACR audio et quelles technologies sont utilisées ?
L’ACR audio, ou reconnaissance automatique de contenu à partir du signal sonore, regroupe ici deux grandes approches technologiques. Toutes deux visent à identifier automatiquement un flux et/ou un contenu diffusé, mais elles ne reposent pas sur les mêmes principes de fonctionnement.
- Le watermarking watermarking Technologie utilisée pour la mesure d'audience de la télévision et de la radio. Le watermarking consiste à insérer une marque (un son) inaudible à l'oreille humaine, dans les émissions. Cette marque contient l'identification de la chaîne qui diffuse le programme et des repères réguliers sur l'heure de diffusion. Les audimètres installés chez les panélistes reconnaissent ces informations. Find out more audio consiste à insérer un marqueur marqueur Le marqueur (ou "tag") permet de mesurer la fréquentation d'un site internet. Cela consiste en l'ajout d'un code informatique dans les pages du site à mesurer. Lorsqu'une page est appelée par un navigateur, le code envoie une requête à un serveur de comptage qui comptabilise le trafic ainsi généré. La technologie par marqueur s'adapte aussi à la mesure des contenus vidéo et audio en streaming, des applications mobiles, des podcasts etc. Find out more inaudible directement dans le signal audio. On parle aussi de tatouage audio. Ce marqueur permet une identification déterministe déterministe Le terme de donnée ou data déterministe désigne un processus d'identification d'un individu reposant sur un identifiant unique qui peut par exemple être un numéro d'identifiant d'un appareil mobile ou un numéro d'abonné à un opérateur téléphonique. Find out more , précise et à faible latence (en quelques secondes après la captation du signal audio).
- Le fingerprinting fingerprinting Technologie d'identification de contenus audiovisuels. Elle consiste à "prélever" un morceau de contenu visionné ou entendu sur une plateforme quelle qu'elle soit, puis de le comparer avec une base de données de référence contenant tous les extraits des programmes d'une chaîne ou d'un catalogue de programmes. Si ce "prélèvement" s'avère appartenir à cette base de données, alors le contenu dont il provient est identifié. Find out more audio repose sur l’analyse des caractéristiques propres du signal afin de produire une empreinte sonore, ensuite comparée (en back office) à un référentiel d’empreintes de programmes connus.
Le watermarking audio est particulièrement adapté aux contextes qui exigent rapidité et précision, notamment pour les flux live. Le fingerprinting audio est, quant à lui, pertinent lorsqu’il s’agit d’identifier des contenus déjà catalogués, sans insertion préalable d’informations dans le signal.
How does audio Watermarking work with SNAP?
Médiamétrie uses SNAP, an audio Watermarking Watermarking The technology used in television and radio audience measurement. Watermarking consists of inserting an (audio) tag into broadcasts that is inaudible to the human ear. This tag identifies the channel airing the programme, as well as regular markers of the broadcast time. Audience meters installed in panellist homes detect these signals. Find out more solution developed by Fifty5Blue. This technology involves the inaudible insertion of information into the audio signal, enabling rapid and reliable recognition of the stream and the content being viewed and/or listened to.
The technical principle is based on phase modulation of the audio signal. Specifically, information is inserted into controlled frequency frequency In electronics, the frequency characterises a vibrating motion, such as an electromagnetic wave: it is the number of times the phenomenon is produced per second (the unit of measurement is the Hertz). In statistics, it is the number of times an event takes place, expressed as a percentage. In computing, it expresses the power of a computer's processors. For example: Pentium III 650 MHz Find out more bands without any alteration to the user’s listening experience. This approach therefore aims to balance the discretion of the marking, the robustness of the embedded watermark, and the speed of its detection.
Why is audio Watermarking particularly well-suited for rapid detection?
Audio Watermarking Watermarking The technology used in television and radio audience measurement. Watermarking consists of inserting an (audio) tag into broadcasts that is inaudible to the human ear. This tag identifies the channel airing the programme, as well as regular markers of the broadcast time. Audience meters installed in panellist homes detect these signals. Find out more offers several advantages for Audience Measurement Audience Measurement Quantitative study of the frequency of use of media. Find out more :
- high perceptual transparency of the audio watermark;
- very good robustness against audio processing processing Means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction Find out more in broadcast broadcast A mode of transmission whereby a transmitter sends the same audio and video stream to many receivers within a coverage area. In the context of computer networks, broadcast mode is contrasted with “multicast” and “unicast” modes, which respectively refer to the transmission of a stream to a given group of recipient devices and to a single recipient device. OR : Putting a show or a programme on television or radio. This term refers to a programme that is broadcast live as opposed to on-demand. Find out more chains;
- compatibility with environments involving ambient noise and microphone recording (as in the case of a portable Audience Meter Audience Meter A device for measuring the audience of radio or television programmes. Audience meters automatically register whenever a TV or radio is switched on or off, as well as each change of channel, and each time recorded content is played on the decoder, or catch-up / preview services are viewed. They also allow for individual audience measurement via a remote control with push buttons; each member of the household has their own button which they use to state their presence in the room. The audience meter is connected via the telephone network to a computer that collects and processes the data gathered. Find out more ).
The watermark typically carries two main pieces of information
- a unique identifier assigned to the channel channel A subset of the brand URLs that overlap with a single theme (e.g., news, e-mail, etc.); the channel audience contributes to the overall audience of the brand, which itself contributes to the overall audience of the Parent. Find out more , station, or platform being measured;
- a timestamp.
This architecture enables the reliable identification of streams and content, in both linear linear This is when a live television program is watched exactly at the time it airs, in timeshifting (control over live TV) or private delay (personal recording). Find out more and non-linear use cases:
- in Linear mode mode In statistics, the mode is the most common value. Find out more , using a temporal reference provided by the inserted timestamp;
- non-linearly, through content recognition using its identifier and relative timecode.
Fifty5Blue’s detection algorithms were natively designed to balance identification quality and robustness with compatibility for portable Audience Audience The set of listeners, viewers, or TV viewers in which each individual is equally weighted, regardless of their listening/viewing duration or frequency of contact. Find out more Meters with limited resources (CPU, memory, and power power Indicator for evaluating formats in the context of a media plan. This is the prioritisation of formats according to their target audience. Find out more ).
How does audio Fingerprinting work with UNITAM?
Médiamétrie also uses UNITAM, an audio Fingerprinting Fingerprinting A technology used to identify audio-visual content. It involves taking a "sample" of content that has been viewed or listened to on any platform whatsoever and then comparing it against a reference database containing all the programme extracts for a channel or a catalogue of programmes. If this “sample” turns out to belong to that database, then the content from which it was sourced can be identified. Find out more technology provided by Nielsen. Unlike Watermarking Watermarking The technology used in television and radio audience measurement. Watermarking consists of inserting an (audio) tag into broadcasts that is inaudible to the human ear. This tag identifies the channel airing the programme, as well as regular markers of the broadcast time. Audience meters installed in panellist homes detect these signals. Find out more , this approach requires no prior insertion into the audio signal to be measured.
Identification is performed based on the audio fingerprint of the captured content.
The system analyzes the characteristics of the audio signal, generates a Signature Signature An electronic code inserted into a digital or analogue signal, making it possible to authenticate the transmitting source. Find out more that is as compact as possible, and then compares it to a centralized database database All the homogeneous data collected over the course of several surveys and whose structure provides the potential for tailor-made consultations and inquiries. Find out more of reference content fingerprintsIn which situations is audio fingerprinting relevant, and what are its limitations?
Fingerprinting Fingerprinting A technology used to identify audio-visual content. It involves taking a "sample" of content that has been viewed or listened to on any platform whatsoever and then comparing it against a reference database containing all the programme extracts for a channel or a catalogue of programmes. If this “sample” turns out to belong to that database, then the content from which it was sourced can be identified. Find out more algorithms generally rely on longer observation windows to improve the statistical robustness of the comparison between fingerprints (those observed by Audience Audience The set of listeners, viewers, or TV viewers in which each individual is equally weighted, regardless of their listening/viewing duration or frequency of contact. Find out more Meters and those used as references) and thereby limit false positives.
This approach is particularly well-suited for identifying content that has already been cataloged and previously referenced.
However, it has certain limitations, particularly when identifying identical live streams or situations involving rapid Channel Hopping Channel Hopping Frequently changing television channels. In the narrow sense, this change is done to avoid advertisements (in Canada, they call it “zapping”). Among the different techniques for measuring an audience, only audience monitoring can capture this phenomenon. We can distinguish Flipping (changing the channel during a show) from Switching (changing the channel during a show in order to watch a previously chosen programme). Find out more .
Like any Fingerprinting system, its performance depends heavily on the quality of the captured signal, the acoustic measurement environment, the comprehensiveness of the reference catalog, and how up-to-date that catalog is.
Identification is based on a Server Server A system composed of computer hardware and software, that makes databases or programmes available to its users. Find out more -side comparison, using a probabilistic probabilistic The term probabilistic data refers to a process of identifying an individual based on a probabilistic model rather than on an identifier considered to be “infallible” thanks to its uniqueness. Find out more approach, between the fingerprints observed by the Audience Meters and those already calculated in the reference database database All the homogeneous data collected over the course of several surveys and whose structure provides the potential for tailor-made consultations and inquiries. Find out more .
In Audience Measurement Audience Measurement Quantitative study of the frequency of use of media. Find out more , audio Watermarking Watermarking The technology used in television and radio audience measurement. Watermarking consists of inserting an (audio) tag into broadcasts that is inaudible to the human ear. This tag identifies the channel airing the programme, as well as regular markers of the broadcast time. Audience meters installed in panellist homes detect these signals. Find out more and audio Fingerprinting are not mutually exclusive: they address different and often complementary needs. Using both allows for adapting automatic content identification to the diversity of broadcast broadcast A mode of transmission whereby a transmitter sends the same audio and video stream to many receivers within a coverage area. In the context of computer networks, broadcast mode is contrasted with “multicast” and “unicast” modes, which respectively refer to the transmission of a stream to a given group of recipient devices and to a single recipient device. OR : Putting a show or a programme on television or radio. This term refers to a programme that is broadcast live as opposed to on-demand. Find out more environments, usage patterns, and measurement constraints. It is this combination of technological precision, operational robustness, and methodological integration that enables the most accurate tracking tracking Activity which aims to measure the profile of a visitor to a website, their browsing path, interests, origin in the network, etc., in order to tailor content to them. Find out more of content actually consumed.
FAQ — Understanding Automatic Audio Content Recognition
What is automatic data collection in Audience Measurement?
The goal is to automatically identify the content that is actually being consumed based on the audio signal captured by the Audience Audience The set of listeners, viewers, or TV viewers in which each individual is equally weighted, regardless of their listening/viewing duration or frequency of contact. Find out more Meters.
What is the difference between audio Watermarking and audio Fingerprinting?
Audio Watermarking Watermarking The technology used in television and radio audience measurement. Watermarking consists of inserting an (audio) tag into broadcasts that is inaudible to the human ear. This tag identifies the channel airing the programme, as well as regular markers of the broadcast time. Audience meters installed in panellist homes detect these signals. Find out more involves inserting an inaudible Tag Tag Tags are used to measure visits to a website. To do this, a computer code is inserted into the pages of the website for measurement. When a page is requested by a browser, the code sends a request to a counting server which adds up this generated traffic. Tag technology can also be adapted to use in the measurement of video and audio content streaming, as well as measurement of mobile apps, podcasts, etc. Find out more into the signal and the content being measured, while audio Fingerprinting Fingerprinting A technology used to identify audio-visual content. It involves taking a "sample" of content that has been viewed or listened to on any platform whatsoever and then comparing it against a reference database containing all the programme extracts for a channel or a catalogue of programmes. If this “sample” turns out to belong to that database, then the content from which it was sourced can be identified. Find out more involves calculating and analyzing the audio fingerprint of the measured content.
Why is audio Fingerprinting less suitable for live broadcasts or Channel Hopping?
Because it generally relies on longer observation windows and a Probabilistic Probabilistic The term probabilistic data refers to a process of identifying an individual based on a probabilistic model rather than on an identifier considered to be “infallible” thanks to its uniqueness. Find out more comparison with a content repository. It is a technique that is inherently content-centric
Why is audio Watermarking useful for measuring live streams?
Because it enables fast, deterministic deterministic The term deterministic data denotes a process for identifying an individual based on a unique identifier, which may, for example, be an identification number of a mobile device or a telephone operator subscription number. Find out more detection, which is particularly well-suited for live applications and rapid changes in streams and content (such as Channel Hopping Channel Hopping Frequently changing television channels. In the narrow sense, this change is done to avoid advertisements (in Canada, they call it “zapping”). Among the different techniques for measuring an audience, only audience monitoring can capture this phenomenon. We can distinguish Flipping (changing the channel during a show) from Switching (changing the channel during a show in order to watch a previously chosen programme). Find out more ).