It has recently been shown that neural networks can learn particular mathematical groups, for example, the Klein 4-group (Jamrozik & Shultz, 2007). However, there are groups with any number of elements, all of which are said to instantiate the abstract group structure. Learning to differentiate groups from other structures that are not groups is a very difficult task. Contrary to some views, we show that neural networks can learn to recognize finite groups consisting of up to 4 elements. We present this problem as a case study that exhibits the advantages of knowledge-based learning over knowledge-free learning. In addition, we also show the surprising result that the way in which the KBCC algorithm recruits previous knowledge reflects some deep structural properties of the patterns that are learned, namely, the structure of the subgroups of a given group.