Under Material Skin Lie the Bones of Identity

This paper explores the automated recognition of objects and materials and their relation to depictions in images of all kinds: photographs, artwork, doodles by children, and any other visual representation. The way artists of all cultures, ages and skill levels depict objects and materials furnishes a gamut of ``depictions'' so wide as to present a severe challenge to current algorithms -- none of them perform satisfactorily across any but a few types of depiction. Indeed, most algorithms exhibit a significant performance loss when the images used are non photographic in nature. This loss can be explained using the tacit assumptions that underlay nearly every algorithm for recognition. Appeal to the Art History literature provides an alternative set of assumptions, that are more robust to variations in depiction and which offer new ways forward for automated image analysis. This is important, not just to advance Computer Vision, but because of the new understanding and applications that it opens.