How are concepts represented in the human mind? Vector space representations based on the instantaneous firing rates of neurons have been used with great success. However, there is growing evidence, both empirical and computational, that relevant information is encoded in spatiotemporal patterns of spikes called polychronous neuronal groups (PNGs). In this paper, we consider the philosophical implications of PNG representations with regard to their temporal extension, grounding, compositionality, and similarity. We suggest that the temporally extended nature of PNGs implies that conceptual-level dynamics may only be coherent at coarse time scales. We introduce the notion of PNG trigger sets as a way to ground the meaning of PNG representations, and we discuss potential approachs to compositionality. Finally, we identify the open problem of how to define an appropriate similarity metric for PNG-based mental representations.