Great progress has been made in understanding the genetic architecture of phenotypic variation, but it is almost entirely focused on how the genotype of an individual affects the phenotype of that same individual. However, in many species the genotype of the mother is a major determinant of the phenotype of her offspring. Therefore, a complete picture of genetic architecture must include these maternal genetic effects, but they can be difficult to identify because maternal and offspring genotypes are correlated and therefore, partially confounded. We present a conceptual framework that overcomes this challenge to separate direct and maternal effects in intact families through an analysis that we call "statistical cross-fostering." Our approach combines genotype data from mothers and their offspring to remove the confounding effects of the offspring's own genotype on measures of maternal genetic effects. We formalize our approach in an orthogonal model and apply this model to an experimental population of mice. We identify a set of six maternal genetic effect loci that explain a substantial portion of variation in body size at all ages. This variation would be missed in an approach focused solely on direct genetic effects, but is clearly a major component of genetic architecture. Our approach can easily be adapted to examine maternal effects in different systems, and because it does not require experimental manipulation, it provides a framework that can be used to understand the contribution of maternal genetic effects in both natural and experimental populations.