Although nouns are frequently used to refer to just a single category member, nouns name entire object categories. Thus, categorizing items is a prerequisite for deciding which particular item is an intended referent. We investigated the categorization process by examining the listeners use of contextually-bound referential domain restrictions during initial reference resolution pitted against a powerful determinant of category membership: visual similarity. Anaphoric one picks out a single item from its antecedent category, so how people determine its referent reveals how they have categorized the items. We examined participants interpretation of another one in a computerized referent selection task while being eye-tracked. Items to be selected varied by their visual similarity and referential domain membership to the antecedent of another one. The eye-movement data reveals that referential domain has a strong effect on object categorization in the earliest moments of reference resolution, modulating the effect of visual similarity.