Recurrent connectionist models, such as the simple recurrent network (SRN, Elman, 1991), have been shown to be able to account for peoples ability to process sentences with center embedded structures of limited depth without recourse to a competence grammar that allows unbounded recursion (Christiansen & Chater, 1999). To better understand the connectionist approaches to recursive structures, we analyze the performance of a single layer network architecture that employs decaying lexical context representation on three kinds of recursive structures (i.e., right branching, cross serial and center embedding). We show that the model with one input bank can capture one and two levels of right branching recursion, one level of center embedded recursion, but not two levels and cannot capture a single level of cross serial recursion. If one adds a second bank of input units with a different decay rate, the model can capture one and two levels of both center embedded and cross serial recursion. Furthermore, with this model the interclass difference of doubly cross serial patterns is greater than it is for center embedded recursion, which may explain why people rate these patterns as easier to process (Bach, Brown, & Marslen-Wilson, 1986).