From presence to prominence

17 March 2021

How can computer vision widen the evidence base around on-screen representation

Quote 1: “The goal must be that the diversity of employment in film should represent the diversity in the population as a whole. [...]  although the problem of under representation has been recognised and discussed over a number of years, very little has been done.”

Quote 2: “The screen industries do not currently reflect the UK population – neither in their workforce nor the content they produce. [...] Even without research, industry is clear that film is nowhere near representative of the UK population and has a lot of work to do to change this.” 

19 years separate the two quotes above but they convey very similar points. The first quote is from 2001, from the first report from the Committee for Ethnic Minority Employment in Film, set up by then-DCMS Secretary of State Chris Smith. The second quote is from January 2020, from the Initial Findings report on the British Film Institute (BFI) Diversity Standards. The UK screen industry has a diversity (specifically, representation) problem which it has found difficult to solve. Progress has been made but structural inequalities remain. 

Representation is commonly defined as how the media portray individuals, social groups, events and issues to an audience. 

Computer vision is an interdisciplinary field that develops techniques to help computers ‘see’ or make sense of the visual world. For example, models can be trained to detect objects like faces in an image or video.

Due to the persistent nature of the diversity issue, it's crucial that we continue to evidence it. In this series of blogs, we explore whether and how computer vision might provide a new method of measuring on-screen representation. In particular:

  • We argue that diversity evaluation needs to consider more than on-screen presence - it should also consider prominence and portrayal. 
  • We discuss the ethics of applying computer vision to study on-screen characters, via a conceptual framework for measuring on-screen representation. We argue that computer vision can be usefully applied to identify character occurrences, rather than demographic classification. 
  • We provide an illustrative example of measuring character prominence using a short video clip.

This blog series is an output of a pilot project conducted by Nesta in partnership with Learning on Screen. The project team consists of Raphael Leung (Principal Data Scientist, Nesta); Bartolomeo Meletti (Education and Research Executive, Learning on Screen); Dr Cath Sleeman (Head of Data Discovery, Nesta); Gabriel A. Hernández (Chief Digital Officer, Learning on Screen); and Gil Toffell (Academic Research Manager, Learning on Screen).  

This output builds upon a presentation that Raphael and Bartolomeo gave at the International Federation of Television Archives (FIAT/IFTA) Conference 2020 and a short paper by Raphael for the IJCAI 2021 AI for Social Good workshop. It is also inspired by PEC research on generating more subtle, nuanced measures of creative diversity using machine learning, e.g. She said more, which analysed gendered pronouns that are male and female in news articles about the creative industries.

Image by Lê Minh

Below you can read the three pieces of work in this research series.