We explore the coevolution of punishment and prosociality in a population of learning agents. Across three models, we find that the capacity to learn from punishment can allow both punishment and prosocial behavior to evolve by natural selection. In order to model the effects of innate behavioral dispositions (such as prosociality) combined with the effects of learning (such as a response to contingent punishment), we adopt a Bayesian framework. Agents choose actions by considering their probable outcomes, calculated from an innate, heritable prior distribution and agents' experience of actual outcomes. We explore models in which an agent learns about the dispositions of each individual agent independently, as well as models in which an agent combines individual-level and group-level learning. Our results illustrate how the integration of Bayesian cognitive models into agent-based simulations of natural selection can reveal evolutionary dynamics in the optimal balance between innate knowledge and learning.