Stanford Medicine investigators reported on their development of a new artificial intelligence model that in tests was found to be more than 90% successful at determining whether MRI scans of human brain activity were from a man or from a woman. The findings, the investigators suggest, help to resolve a longstanding controversy about whether reliable sex differences exist in the human brain, and also indicate that understanding these differences may help scientists better understand neuropsychiatric conditions that affect women and men differently.
“A key motivation for this study is that sex plays a crucial role in human brain development, in aging, and in the manifestation of psychiatric and neurological disorders,” said Vinod Menon, PhD, professor of psychiatry and behavioral sciences and director of the Stanford Cognitive and Systems Neuroscience Laboratory. “Identifying consistent and replicable sex differences in the healthy adult brain is a critical step toward a deeper understanding of sex-specific vulnerabilities in psychiatric and neurological disorders.”
Menon is senior author of the team’s published study in PNAS, titled, “Deep learning models reveal replicable, generalizable, and behaviorally relevant sex differences in human functional brain organization.” In their report, the investigators concluded, “Our findings underscore the crucial role of sex as a biological determinant in human brain organization, have significant implications for developing personalized sex-specific biomarkers in psychiatric and neurological disorders, and provide innovative AI-based computational tools for future research.” The lead authors are Srikanth Ryali, PhD, and academic staff researcher Yuan Zhang, PhD.
Sex plays a significant role in early brain development, adolescence, and aging, the authors noted. Moreover, they pointed out, “Sex is an important biological factor that influences human behavior, impacting brain function and the manifestation of psychiatric and neurological disorders … Consequently, knowledge of sex differences in the human brain is critical for understanding both normative behavior and psychopathology.” In fact, the extent to which a person’s sex affects how their brain is organized and operates has long been a point of dispute among scientists, the scientists suggested.
While we know the sex chromosomes we are born with help determine the cocktail of hormones our brains are exposed to—particularly during early development, puberty, and aging—researchers have long struggled to connect sex to concrete differences in the human brain. Brain structures tend to look much the same in men and women, and previous research examining how brain regions work together has also largely failed to turn up consistent brain indicators of sex. “… previous research on how brain organization differs between males and females has been inconclusive,” the investigators pointed out. “… our understanding of sex differences in human functional brain organization and their behavioral consequences has been hindered by inconsistent findings and a lack of replication.”
For their newly reported study, Menon and colleagues took advantage of recent advances in artificial intelligence, as well as access to multiple large datasets, to pursue a more powerful analysis than has previously been employed. First, they created an end-to-end spatiotemporal deep neural network (stDNN) model, which they trained to classify data from resting-state functional MRI (rsfMRI) brain images from the Human Connectome Project (HCP). As the researchers showed brain scans to the model and told it that it was looking at a male or female brain, the model started to “notice” what subtle patterns could help it tell the difference. “Our stDNN model uncovered reliable sex differences with over 90% cross-validation classification accuracies, outperforming previous studies,” the scientists stated.
The team assessed the replicability of their predictive model on additional datasets, without further training. When tested on around 1,500 brain scans, the stDNN model could almost always tell if the scan came from a woman or a man. It also demonstrated superior performance compared with models used in previous studies, in part because it uses a deep neural network that analyzes dynamic rsfMRI scans. This approach captures the intricate interplay among different brain regions. “Critically, our model outperformed previous studies in both test and independent dataset,” the team noted.
The model’s success suggests that detectable sex differences do exist in the brain, but that they hadn’t been picked up reliably before. The fact that the model worked so well in different datasets, including brain scans from multiple sites in the United States, and Europe, makes the findings especially convincing as it controls for many confounders that can impact studies of this kind. “This is a very strong piece of evidence that sex is a robust determinant of human brain organization,” Menon said.
Until recently, a model such as the one employed by Menon’s team would have helped researchers sort brains into different groups but wouldn’t have been able to provide information about how the sorting happened. However, researchers today have access to a tool called explainable AI (XAI), which can sift through vast amounts of data to explain how a model’s decisions are made.
Using explainable AI, Menon and his team identified the brain networks that were most important to the model’s judgment of whether a brain scan came from a man or a woman. They found that the “hotspots” that most helped the model distinguish male brains from female ones included the default mode network (DMN), a brain system that helps us process self-referential information, and the striatum and limbic network, which are involved in learning and how we respond to rewards.
Notably, they pointed out, the DMN, striatum, and limbic network are also “loci of dysfunction in psychiatric disorders with female or male bias in prevalence rates, including autism, attention deficit disorders, depression, addiction, schizophrenia, and Parkinson’s disease all of which have sex-specific sequelae and outcomes.” Their findings, the team suggested, “… may therefore offer a template for investigations of sex differences in vulnerability to individual psychiatric and neurological disorders.”
The researchers in addition wondered if they could create another model that could predict how well participants would do on certain cognitive tasks based on functional brain features that differ between women and men. To do this they developed sex-specific models of cognitive abilities. One model effectively predicted cognitive performance in men but not women, and another in women but not men. The findings indicated that functional brain characteristics varying between sexes have significant behavioral implications. “Critically, the brain features identified by XAI that reliably distinguished functional brain organization between sexes also predicted unique cognitive profiles in females and males,” the authors commented.
In summary of their findings, they concluded, “Our approach using spatiotemporal DNNs and XAI techniques identifies replicable, generalizable, and interpretable sex differences in human functional brain organization across multiple datasets and independent cohorts and, furthermore, reveals that functional brain features that differ between sexes are behaviorally relevant.”
Menon added, “These models worked really well because we successfully separated brain patterns between sexes. That tells me that overlooking sex differences in brain organization could lead us to miss key factors underlying neuropsychiatric disorders.”
While the team applied their deep neural network model to questions about sex differences, Menon says the model can be applied to answer questions regarding how just about any aspect of brain connectivity might relate to any kind of cognitive ability or behavior. The scientists plan to make their model publicly available for any researcher to use. “Our AI models have very broad applicability,” Menon said. “A researcher could use our models to look for brain differences linked to learning impairments or social functioning differences, for instance—aspects we are keen to understand better to aid individuals in adapting to and surmounting these challenges.”
The investigators noted that this work does not weigh in on whether sex-related differences arise early in life or may be driven by hormonal differences or the different societal circumstances that men and women may be more likely to encounter. Nevertheless, they wrote, “The finding of robust functional brain features underlying sex differences has the potential to inform quantitatively precise models for investigating sex differences in psychiatric and neurological disorders. This work paves the way for more targeted and personalized approaches in both cognitive neuroscience research and clinical applications.”