Children often fail to use informative non-geometric features to recover orientation in cases where adults succeed. The present work demonstrates that adults can also fail to use distinguishing information if that information is not encoded categorically. Adults were shown an object being hidden in a corner of a rectangular room rendered with an immersive virtual display. They were then disoriented and asked to locate the object. The short walls of the room were discriminably different colors so that they could be used to uniquely specify the orientation of the viewer with respect to the enclosure. When the colors were members of different major color categories, participants were more likely to succeed in the task. In fact, participants succeeded in the task if and only if they encoded the color difference lexically. This indicates that categorical encoding plays an important part in reorienting with non-geometric features and implicates language as a default medium of categorical encoding.