Skip to content
>> Home > Research by Themes > Diversity and Inclusion > Using computer vision to improve on-screen representation

Using computer vision to improve on-screen representation

camera operator looking through view finder

Explore the research series

close up of camera lens
Part one: Gaps in the evidence base

From presence to prominence: How can computer vision widen evidence base of on-screen representation…

TV camera with view finder screen display switched o n
Part two: A framework for measuring on-screen representation

From presence to prominence: How can computer vision widen evidence base of on-screen representation…

TV camera operator filming concert crowd
Part three: An illustrative demo measuring prominence of TV characters

From presence to prominence: How can computer vision widen evidence base of on-screen representation…


A series about improving issues of diversity in the characters that we see on our screens

The UK screen industry has a diversity (specifically, representation) problem which it has found difficult to solve. Progress has been made but structural inequalities remain. 

Representation is commonly defined as how the media portray individuals, social groups, events and issues to an audience. Computer vision is an interdisciplinary field that develops techniques to help computers ‘see’ or make sense of the visual world. For example, models can be trained to detect objects like faces in an image or video.

Due to the persistent nature of the diversity issue, it’s crucial that we continue to evidence it. In this series of blogs, we explore whether and how computer vision might provide a new method of measuring on-screen representation. In particular:

  • We argue that diversity evaluation needs to consider more than on-screen presence – it should also consider prominence and portrayal. 
  • We discuss the ethics of applying computer vision to study on-screen characters, via a conceptual framework for measuring on-screen representation. We argue that computer vision can be usefully applied to identify character occurrences, rather than demographic classification. 
  • We provide an illustrative example of measuring character prominence using a short video clip.

This blog series is an output of a pilot project conducted by Nesta in partnership with Learning on Screen. The project team consists of Raphael Leung (Principal Data Scientist, Nesta); Bartolomeo Meletti (Education and Research Executive, Learning on Screen); Dr Cath Sleeman (Head of Data Discovery, Nesta); Gabriel A. Hernández (Chief Digital Officer, Learning on Screen); and Gil Toffell (Academic Research Manager, Learning on Screen).  

This output builds upon a presentation that Raphael and Bartolomeo gave at the International Federation of Television Archives (FIAT/IFTA) Conference 2020 and a short paper by Raphael for the IJCAI 2021 AI for Social Good workshop. It is also inspired by PEC research on generating more subtle, nuanced measures of creative diversity using machine learning, e.g. She said more, which analysed gendered pronouns that are male and female in news articles about the creative industries.

Image by Lê Minh


Related Research Reports

Baltic Art Gallery courtesy of Lorna Simpson
Measuring the economic value of cultural and heritage sites

A major contribution of new guidance, evidence and research on the economic value of culture and her…

religious artwork
AI and the Creative Industries: The art in the artificial

AI and the creative industries Artificial Intelligence (AI) is the goal of developing machines with …

image of urban industrial building with signage on exterior - courtesy of Alyssa Bossom
Creative radar: Mapping the UK’s creative industries

Major urban centres are the sorts of places we would expect to find creative businesses clustered to…

skyline of town - courtesy of Benjamin Elliott
Geographies of Creativity

A Creative PEC State of the Nations report Place matters when it comes to creativity. Where do creat…

image of woman in cafe looking at tablet screen
Streaming culture

Looking back at the rise of streaming Over the past decade, digital technologies have brought huge c…

abstract artwork
Creativity and the future of skills

At a time when all jobs, whether in a coffee shop or a bank, can seemingly be described as creative,…

image of fashion photo shoot with model and crew
The creative digital skills revolution

A common finding in studies on the future of work is that employers will increasingly dema…

image of person drawing on paper - courtesy of Retha Ferguson
For love or money

Graduate motivations and the economic returns of creative higher education inside and outside the cr…

film crew filming in busy street
Improving young people’s access and progression in work in film, TV and games

Download the executive summary Young people can face significant barriers to achieving their full po…

abstract artwork
The migrant and skills needs of creative businesses in the UK

Download the Appendices This report details the results of a survey of employers commissioned b…

Aerial shot of shipping contrainers
12 facts about the UK’s international trade in creative goods and services

Worldwide exports of creative goods exceeded 500 billion USD in 2015, with a 150% increase since 200…

actors on stage behind a red theatre curtain
Charities speak

Using data science to understand the arts and cultural charities sector in England and Wales Chariti…

Author

Sign up to our newsletter