Abstract:

The paper proposes a novel probabilistic generative model for simultaneous image classification and annotation. The model considers the fact that the category information can provide valuable information for image annotation. Once the category of an image is ascertained, the scope of annotation words can be narrowed, and the probability of generating irrelevant annotation words can be reduced. To this end, the idea that annotates images according to class is introduced in the model. Using variational methods, the approximate inference and parameters estimation algorithms of the model are derived, and efficient approximations for classifying and annotating new images are also given. The power of our model is demonstrated on two real world datasets: a 1 600-images LabelMe dataset and a 1 791-images UIUC-Sport dataset. The experiment results show that the classification performance is on par with several state-of-the-art classification models, while the annotation performance is better than that of several state-of-the-art annotation models.

Key words:

image classification, image annotation, probabilistic model, variational inference