Personalized data generation and summarization using learned ranking models

Abstract

In the last decade, the users of many web platforms provided a massive flow of multimedia data content, which opened the possibility of enhancing the user experience in these platforms through data personalization. The personalization of data consists of presenting customized data representations to different users according to their preferences, and with the recent advances of deep learning models in computer vision, it becomes intriguing to explore the impact of these models on personalization tasks. For this purpose, this thesis aims to study the possibility of building personalized deep learning models that benefit from the users' preferences to provide customized applications. The main challenges in this thesis reside in defining the context, methods, and applications of the studied data personalization tasks. Also, the representation of the users' preferences should be considered according to the available benchmarks in the literature.

Our work in this thesis focus on the personalization of generation, exploration, and summarization tasks using three main types of multimedia data: 2D images, 3D shapes, and videos where we define the users' preferences at two levels: categorical annotated labels and comparison-based semantic attributes that are more intuitive given that it is much easier to compare objects rather than assigning a specific label from the users' perspective.

We begin our studies by investigating the usage of generative adversarial networks in 2D image generation tasks according to semantic attributes. These semantic attributes represent subjective measures via pairwise comparisons of images by the user for customized 2D image generation and editing. As an extension to this work, we explore generative adversarial networks for 3D meshes, where the user defines subjective measures to browse and edit 3D shapes. Lastly, we tackle the video summarization task, where we suggest a conditional ranking-based model that generates personalized summarizations given a set of categorical annotated labels selected by the user, which enables the possibility of providing flexible and interactive summarizations.

Date of Award	17 Jan 2022
Original language	English
Awarding Institution	University of Bath
Supervisor	Yongliang Yang (Supervisor) & Wenbin Li (Supervisor)