What and where: A context-based recommendation system for object insertion

Song Hai Zhang, Zheng Ping Zhou, Bin Liu, Xi Dong, Peter Hall

Research output: Contribution to journalArticlepeer-review

7 Citations (SciVal)


We propose a novel problem revolving around two tasks: (i) given a scene, recommend objects to insert, and (ii) given an object category, retrieve suitable background scenes. A bounding box for the inserted object is predicted in both tasks, which helps downstream applications such as semiautomated advertising and video composition. The major challenge lies in the fact that the target object is neither present nor localized in the input, and furthermore, available datasets only provide scenes with existing objects. To tackle this problem, we build an unsupervised algorithm based on object-level contexts, which explicitly models the joint probability distribution of object categories and bounding boxes using a Gaussian mixture model. Experiments on our own annotated test set demonstrate that our system outperforms existing baselines on all sub-tasks, and does so using a unified framework. Future extensions and applications are suggested.

Original languageEnglish
Pages (from-to)79-93
Number of pages15
JournalComputational Visual Media
Issue number1
Publication statusPublished - 31 Mar 2020


  • bounding box prediction
  • image composition
  • object recommendation
  • object-level context

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design
  • Artificial Intelligence


Dive into the research topics of 'What and where: A context-based recommendation system for object insertion'. Together they form a unique fingerprint.

Cite this