Frequently Asked Questions

Realeyes is a leader in computer vision and emotion AI that empowers companies to create remarkable customer experiences that drive growth. Realeyes uses front-facing cameras and computer vision technology to measure the attention and reactions of opt-in audiences as they experience digital content.

• Performance – our models have the highest performance in unconstrained, real-life situations and are light enough to even run continuously on most consumer mobile devices

• Fairness – our models have the least bias across user dimensions like skin tone, gender and age

• Legal/Privacy safe – our models are trained on proprietary visual data that have been obtained in accordance with the highest legal, privacy and ethical standards

• Speed – our data infrastructures and testing automations allow clients to receive results as early as 3 days after launching their tests

• Future proof – our computer vision team pioneers and incorporates the latest advancements, with new models released and updated periodically across our products

Realeyes has pioneered and patented a deep-learning AI approach to measuring visual attention that surpasses nominal gaze and eye-tracking.

Instead, it assesses attention in a more holistic manner, as a human would do. It picks up the same subtle behavioral cues that the human brain instinctively processes to gauge someone’s attention to content on screen.

Realeyes’ technology was developed in collaboration with leading AI experts from Imperial College London and University of Pittsburgh.

The approach has been validated through a blind reviewed academic paper [1] and the accuracy and fairness of the models have been independently validated by two major technology and video platforms.

Realeyes holds 25 patents, with 15 pending, in computer vision, emotion AI and attention AI.

In the measurement industry alone, Realeyes has won awards from I-COM, the ANA and the ARF [2].

[1] Schulc, A., Cohn, J. F., Shen, J., & Pantic, M. (2019, May). Automatic measurement of visual attention to video content using deep learning. In 2019 16th International Conference on Machine Vision Applications (MVA) (pp. 1-6). IEEE.

[2] Salo, M., Hajiyev, E., Schulc, A. (2019, Dec.). Computer-implemented system and method for determining attentiveness of user (United States 20190379938). U.S. Patent and Trademark Office.
https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/20190379938

Respondents are recruited through a GDPR-compliant consent process.  Their attention levels and facial reactions are measured using their device's front-facing camera as they view your content within your chosen media platform.

Realeyes attention and reactions measurement is a proprietary technology for analyzing facial images. Mobile or desktop camera recordings are sent to Realeyes servers to be automatically processed in a privacy-safe manner. The data pipeline, powered by a Realeyes proprietary neural network, processes the submitted images one by one in several steps: 1) face detection, 2) face cropping and 3) parallel attention & reaction detection.

For each video frame, when a face is detected in the image, the following signals are produced:

• Visual Attention
  Visual Attention quantifies focus, interest, and engagement by tracking head pose, face direction, eye lid openness, and gaze direction.

• Smile
  A smile is formed from the cheeks rising and the corners of the mouth pulling up respectively.

• Surprise
  A surprised expression is formed from a combination of raised eyebrows, eyes wide open (raised eyelids) and the jaw dropping to reveal an open mouth.

• Frown
  A frown is formed with the lowering of the brow, a raising and narrowing of the eyelids, and a tightening of the lips.
  
  
• Disgust
  Disgust is expressed by the nose wrinkling, a downturn of the lower lip and the corners of the mouth moving downwards.
  
  
• Contempt
  Contempt is formed by a slight curling of the lip corners, often accompanied by a tightening of the jaw muscles.

The training data for our models comes from Realeyes’ proprietary dataset, collected entirely by Realeyes over 12+ years of working on facial coding models development. Camera recordings are collected across different countries, age groups and sexes, to achieve fair and balanced representation globally.

Those recordings are annotated for attention and facial reactions by a large pool of local, qualified psychologists and annotators. Employing a “wisdom of the crowds” approach, we label video frames only if the employed 3-7 annotators reach a majority agreement on each specific frame. As a next step, attention and reaction labels go through a standardized quality assurance review.

The Realeyes dataset, regarded as the largest authentic facial dataset in the world, includes:

• 6 million+ respondents, recorded from 93 countries
• 220k+ hours of video recordings
• GDPR compliant with user opt-ins covering Realeyes’ usage
• Age, Gender, Country, Device Attributes on all data
• 2billion+ annotations: Skin tone, Head Pose, Occlusion, Facial Expressions, Attention, Landmarks, and Facial Attributes.
• Annotated by local annotators, to account for cultural nuances

Using that dataset, our computer vision team builds AI models that reproduce or outperform the capabilities of human annotators. These models predict attention and emotion signals from video recordings without human interference. To avoid biases, models are trained on carefully selected subsets of the whole annotated dataset, with balanced samples across relevant dimensions.

Our models improve continuously, allowing Realeyes’ models to perform close to human skills. Labeling camera recordings for attention and reactions is not an easy task, especially for humans, since images may be blurry, too dark or too small. The chart below features performance comparisons of humans versus vision AI, based on Balanced Accuracy.

Those recordings are annotated for attention and facial reactions by a large pool of local, qualified psychologists and annotators. Employing a “wisdom of the crowds” approach, we label video frames only if the employed 3-7 annotators reach a majority agreement on each specific frame. As a next step, attention and reaction labels go through a standardized quality assurance review.

The performance of our attention model is systematically assessed across the following dimensions:

• Skin tones
• Countries
• Head poses
• Partial face occlusions
• Device types
• Age groups
• Genders

Model performance for attention is stable across all of these dimensions, with Balanced Accuracy well over 75% for most segments.

Realeyes’ models are used on task-based tests and live in-the-wild. Realeyes products include:

• Embedded Vision AI – Realeyes' AI models are interoperable with any mobile and desktop application through our Experience Platform SDK. This enables on-device, live results. Some use cases include: identity verification for panel providers, wellbeing of mobile device users and engagement with interactive mobile games.

• Ad Testing – Realeyes’ latest product on the marketing landscape, PreView, provides actionable reporting to maximize the impact of ads. Analyzing attention and reaction levels to your creatives across social and video platforms allows you to understand how your campaign impacts audiences and achieves business outcomes.