A number of recent theories of semantic representation propose two-way interaction of semantic and perceptual information. These theories are supported by a growing body of experiments that show widespread interactivity of semantic and perceptual levels of processing. In the current experiment, participants classified ambiguous colors into one of two color categories. The ambiguous colors were presented either as a color patch (no semantic context), as an icon representing an object that is strongly associated with one of the color options, or as a word referring to such an object. Although the iconic and lexical contexts were incidental and irrelevant to the color categorization task, participants' responses were consistently biased toward the context color. These results extend previous findings by showing that lexical contexts, as well as iconic contexts influence color categorization.