It’s a rich time for the study, exploration and reimagining of play and gender roles.

Two recent, very successful crowd-funding efforts backed dolls that reject unhealthy body types and gendered play patterns. Lammily’s realistic proportions are promoted with the tagline “Average is Beautiful,” and “IAmElemental” depicts female action figures with careful attention not to sexualize the design.

Always launched an ad asking why doing something “…like a girl” has become an insult. A Verizon ad shows how a lifetime of small discouragements can turn a girl off to science exploration. Google, with a host of partner organizations, invested $50 million in “Made With Code,” an initiative to get girls involved in programming.

The Washington Post ran a commentary about the importance of investing the “time and money” to develop substantive female characters for video games.

Into this mix, however, came a study that proves one can be in complete philosophical alignment with a hypothesis, but have qualms about the methods used to test it.

“Boys Can Be Anything: Effect of Barbie Play on Girls’ Career Cognitions” found that 4-7 year old girls who played with a Barbie (dressed either as a doctor or in fashion clothes) had narrower views of career options for girls, compared to those who had played with Mrs. Potato Head. After a brief play period, the subjects were shown pictures suggesting different careers, and asked whether they personally, or a generic boy, could have these jobs when grown up. Most of the jobs were classically “gendered” (e.g., firefighter and construction worker skew male; teacher and nurse skew female).

I don’t doubt that cultural influences (media and playthings) can have substantial influence on gender views, even for quite young children. In this case, however, questionable design and significance may generate great headlines – “Barbie vs. Mrs. Potato Head” – but aren’t necessarily good science.

Let’s start with the easiest critiques. The study was very small, involving only 37 girls from one community, far too few to draw conclusions for a diverse national or global population. Moreover, the call for subjects revealed to parents that the study was about the possibility of a connection between fashion dolls and perceptions of career options in young girls, potentially biasing the type of parent who would respond to the solicitation.

There was no indication of pre-testing to discover other potential influences on the girls’ attitudes (aside from whether and how much they already play with Barbie). Did the girls have older siblings? What media content do they consume? What jobs do their parents have?

Moreover, the study findings were built on incredibly brief exposure to the variable condition. While “recency” can sometimes have an effect on a subject’s memory or thinking, it seems far-fetched to think that five minutes of play could trump a lifetime of family, friend, media and societal messaging.

In framing their study, the authors propose that some toys, in addition to being a broad venue for gender role socialization, also communicate messages of objectification and sexualization. In particular, fashion dolls such as Barbie are physically formed and costumed to communicate messages of appearance-focus and sexualization. Such messages contrast with the nurturant expectations conveyed by baby dolls.

If the contrast being made is between sexualized and nurturing imagery and play, then why did the study use Mrs. Potato Head as their “neutral condition,” instead of the referenced baby doll, or at the very least a more realistically shaped humanoid doll? Alternatively, a different type of study might have given girls free access to a variety of doll types and then looked for any correlation between their choices and career views.

The authors wrote that Mrs. Potato Head is similar to Barbie in the color and texture of plastic that makes up the doll, is a feminine doll with a well-known female persona, and is marketed with clothing and accessories similar to Barbie. Toys, however, aren’t just about appearance, but also play pattern; do 4-7 year olds do similar things with Barbie and the Potato Heads? There may be a hint about play pattern in the accessories that were available to the children in the study: both conditions included a purse, but while Barbie featured clothing and a hairbrush, Mrs. Potato Head came equipped with extra eyes and ears.

The experimental design section of the study reported that they had used the dolls exactly as marketed and sold; however, it also noted that the girls could engage in “doing” play (changing clothes or accessories), but didn’t mention fantasy or role play that might have revealed more nuanced insights into their social psychology.

I question, also, the authors’ breadth of understanding of Barbie’s dual role as children’s toy and adult collector’s item. The study notes that although Barbie is available in a variety of costumes, including athletic wear (which might suggest muscularity or agency) and those relevant to many careers (which might suggest intellectual capacity), her most commonly sold style of dress consists of a form-fitting sparkly evening gown and high heels (in fact, her feet are molded for high heels). The massive attention earlier this year to the limited edition “Sports Illustrated Swimsuit Barbie,” plus the annual holiday collector dolls, suggests that the most glamorous or fashion-oriented Barbie dolls may never leave their boxes.

The second part of the study – career identification – raises questions, as well. As noted above, 4-7 is decidedly old enough to have been influenced by societal and cultural messages about gender, but is it old enough to have a clear understanding of careers and “when you grow up”? A quick Google reveals that kids this age want to be a dog, a ninja chef, a “big girl” and more, among more traditional answers. In their report, the researchers acknowledge, as well, that it’s very different to ask the girls what they personally could imagine doing (a reflection of their individual interests and experiences), compared to what a generic “boy” might grow up to do. Subsequent studies would do well to match the forms of the question.

One great thing about research is that it’s almost never definitive and final. Even with its flaws, “Barbie vs. Mrs. Potato Head” can serve as a stepping-stone to more effective exploration of similar territory. The current multi-prong attention to forging stronger role models, offering more diverse and divergent playtoys, and broadening learning tools for girls makes clear that there is hunger among children and parents, and openness to new ways in industry.

Word of the Post: Access – One reason for the gap between industry and academia is that most scholarly research appears in refereed journals, available only to subscribers, academic institutions, or subscribers to services like JSTOR. The single study written about in this post costs 30 UK pounds (just over $50). As a result, most who might make market use of such research will only ever see news reports. As high-school age researcher Jack Andraka noted at the 2014 Sandbox Summit: what does it say when the latest Beyonce video costs 99 cents, but a scholarly study is more than $35.