Edited by: Reza Lashgari, Shahid Beheshti University, Iran
Reviewed by: Alex Clarke, University of Cambridge, United Kingdom; Alessandro Principe, Hospital del Mar, Spain
This article was submitted to Brain Imaging Methods, a section of the journal Frontiers in Neuroscience
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Brain connectivity analyses have conventionally relied on statistical relationship between one-dimensional summaries of activation in different brain areas. However, summarizing activation patterns within each area to a single dimension ignores the potential statistical dependencies between their multi-dimensional activity patterns. Representational Connectivity Analyses (RCA) is a method that quantifies the relationship between multi-dimensional patterns of activity without reducing the dimensionality of the data. We consider two variants of RCA. In model-free RCA, the goal is to quantify the shared information for two brain regions. In model-based RCA, one tests whether two regions have shared information about a specific aspect of the stimuli/task, as defined by a model. However, this is a new approach and the potential caveats of model-free and model-based RCA are still understudied. We first explain how model-based RCA detects connectivity through the lens of models, and then present three scenarios where model-based and model-free RCA give discrepant results. These conflicting results complicate the interpretation of functional connectivity. We highlight the challenges in three scenarios: complex intermediate models, common patterns across regions, and transformation of representational structure across brain regions. The article is accompanied by scripts (
To study the neural underpinnings of cognitive processes, we need not only to characterize the response of individual brain regions but understand the functional connectivity between them. This is critical to understand how brain regions interact in giving rise to cognition (
One key feature of RCA is that, rather than activations (
From a broad perspective, model-free RCA (
We generated multidimensional patterns of activity using scripts from the Matlab RSA toolbox (
Arrangement of the simulated conditions in the representational dissimilarity matrices (RDMs), model and neural RDMs and their pairwise correlations.
We used Pearson’s (linear) correlation for comparing RDMs. Accordingly, we only considered significantly positive correlations as indicating representational connectivity. We performed significance testing using a one-sided Wilcoxon’s signed rank test (
Model-based RCA is designed to test whether two ROIs are related with regards to a specific model
Despite the potential benefits of model-based RCA, it has some limitations that should be considered with caution. For example, consider the scenarios depicted in
An intermediate model RDM can lead to the wrong conclusion that two uncorrelated ROIs (RDMs) are connected. RDMs can be represented as a vector emanating from the origin in the N-dimensional space [N = number of elements in the RDM, i.e., n(n–1)/2 with n being the number of conditions].
Conversely, consider the case depicted in
These examples show that model-based and model-free RCA can easy lead to different results. It follows from the fact that model-free RCA is based on direct comparison of neural RDMs in the original dissimilarity space and that model-based RCA is based on comparison of projected RDMs on a line (projection), defined by the model. The degree of inconsistency depends on the neural RDMs and the direction defined by the model.
In Simulation 1 we present a scenario (similar to
The general simulation details are provided in section “General Simulation Details.” For model-based RCA, we used a model RDM that incorporates both aspects of the stimuli (i.e., position and object category). This “intermediate” model hypothesized a larger pattern dissimilarity for two conditions that are different in both position and object category. The model had equal level of correlation/similarity to the neural RDM in each of the two ROIs (
Two ROIs that represent statistically unrelated information are not connected and do not have any shared information. However, looking at the two ROIs through the lens of an intermediate model in model-based RCA can leave the impression that the ROIs are connected. There was significant positive correlation for the two ROIs with the intermediate model, and the correlations between the model and the two ROIs were not statistically different (Wilcoxon’s signed rank test;
To avoid false conclusion about connectivity across ROIs, it is important to evaluate it using both model-based and model-free RCA, see that if the results agree, and interpret accordingly. Where possible, for model-based RCA, it may also help to use minimal models where only one, rather than several, aspects of information is captured. In our simulation, the fact that our intermediate model had components from both aspects of stimuli (i.e., position and category) made it possible to capture variances explained by different processes, i.e., independent encoding of each aspect. Simpler models, for example models that correspond to simple hypotheses, might help to untangle representational connectivity along different dimensions of information transfer. However, it might be difficult to know these models in advance, unless the tasks are simple, and the underlying representations are already well characterized.
It is of note that, while we implemented a simplified version of RCA here, implementations in the literature have incorporated other parameters, such as time and delay, and other techniques such as multi-linear regression and partial correlation (
There can be situations where common uninformative patterns are present along with the informative representations in the pair of ROIs considered for connectivity analysis. The common patterns can be as simple as measurement or neural noise which might be statistically dependent across areas and/or the leakage or feeding of activations from a third ROI to both ROIs as a result of proximity and/or poor spatial resolution (e.g., in EEG and MEG). On the other hand, it can also be the case that the two ROIs encode/represent some shared aspects, which are either task-irrelevant or not the target of study. For example, both position-selective early visual area (V2) and the semantically selective area (ITC) can be sensitive to low-level image statistics such as the spatial frequencies of the stimulus due to connections from V1. This shared information may lead to apparent connectivity if their RDMs are directly compared (as in model-free RCA), but may not reflect shared information of interest to the researcher. In general, we are not interested in capturing commonality in noise, and may not be interested in capturing this low-level information (i.e., spatial frequency) which are represented in both ROIs, but rather by the particular information for which we have hypotheses. In this simulation we ask whether model-based RCA is robust to this type of shared information and allows us to draw a specific conclusion about the shared information of interest to the researcher.
Below we simulate the impact of adding common patterns of activation to a pair of ROIs which otherwise represent distinct information, and show how model-free RCA, and some implementations of model-based RCA, can be affected. We show that using appropriate models that match the dominant representations of the two ROIs can mitigate the false connectivity.
The neural patterns generated here are the same as Simulation 1 (with no connectivity between the two ROIs) except that now we also include the time course of representations to be able to implement more realistic model-free and model-based RCAs (rather than the simplified ones implemented in Simulation 1). We added the temporal dimension so that correlations could be computed over time. Please note that, however, ROI representations at different time points were consistent with the same structures depicted in
We performed model-free RCA by calculating the direct correlation between RDMs of the two ROIs at every time point and then averaging the resultant correlations over the simulated time window.
In this Simulation (and also the next simulation), we consider two versions of model-based RCA that have different motivations. In either case, we first obtained the correlation between the neural and the corresponding model RDM of each ROI at every time point and then calculated the correlation between the time courses of neural-model correlations for the two ROIs.
In the first version, we considered a common model for the two ROIs (similar to Simulation 1) and in the second version we used ROI-specific models (i.e., one model per ROI). The motivation for the first approach (1-model RCA) was that the experimenter might simply want to evaluate the information exchange reflecting a single known aspect of information (e.g., familiarity information across occipital vs. frontal areas:
We added a non-structured (noise) pattern to both ROIs and evaluated its impact on connectivity (
The results are shown in
Addition of common pattern to pair of ROIs can make them look connected in model-free and 1-model RCA but not 2-model RCA if the ROIs are distinct enough.
This result was especially interesting for 1-model RCA because the method required the two ROIs to be temporally correlated to show connectivity. This confirms that the common pattern has not only correlated the patterns of the two ROIs on every time point, but it has also added temporal correlations to the patterns of the two ROIs making them fluctuate similarly over time (which is key for our model-based connectivity). We also observed that this spurious connectivity for 1-model RCA was not specific to the particular model we used and remained when using any arbitrarily defined random models (results not shown). Specifically, we observed that even models unrelated to the representational structure of one of the two ROIs (e.g., position and/or semantic categories) could lead to false connectivity in 1-model RCA. This can be explained by the fact that the time-locked common input will make the RDMs of the two ROIs be more similar to each other and also to any random RDM.
Finally, despite the correlations imposed on the contents of representations and the temporal patterns across ROIs, the 2-model RCA (correctly) showed negative correlations between the ROIs after the common input suggesting no connectivity (
Both model-free and model-based RCA are affected by common-inputs to the two ROIs. This is particularly important for model-free RCA and 1-model RCA where it will always be the case, and should be taken into consideration when interpreting results. However, in 2-model RCA, where the two ROIs originally represented two distinct aspects of the task, the results were robust to the added common noise. This result was dependent on the chosen model RDMs not being positively correlated. Therefore, for cases where two regions are hypothesized to represent distinct information, the use of 2-model RCA with orthogonal or negatively correlated models can avoid spurious connectivity caused by common patterns of activation such as correlated noise.
There can be situations where the structure of the information is transformed from one ROI to the next. In fact, it seems unlikely that information remains intact (‘‘copied’’) between any two ROIs in the brain. Therefore, direct comparison of neural representations, as implemented in model-free RCA, can miss such potential connectivity simply because the statistical relationship may be lost in transformation. However, model-based RCA may allow us to detect the connectivity between two areas, which encode distinct information, based on their temporal statistical congruency. Below we simulate two ROIs that represent two distinct aspects of information, with dynamics that are either temporally congruent or incongruent between ROIs. Specifically, the information about the stimulus position initially appears in the source ROI (V2) and is followed by the semantic-category information which appears in the destination ROI (ITC)
We used simulations to investigate the transformation of information using model-based RCA. The details of information representation in the two ROIs in this simulation are identical to Simulation 2, with the exception that the information does not appear throughout the simulation window but rather for a fixed period of time in each ROI (samples 30–60 in ROI 1; solid black curve in
2-model RCA allows us to detect transformation of information if the temporal dynamics across ROIs are statistically related/congruent.
Our assumption here is that two ROIs that encode/represent statistically unrelated information can be considered connected if their temporal information-encoding profiles are statistically related/congruent (representations appear in the destination after the source ROI at around the hypothesized delay). We ask whether such a relationship would be detected using model-free, 1-model and 2-model RCA.
Simulation results show that model-free RCA did not detect any connectivity between the two ROIs (
Note that in these simulations, we incorporated the delay in our analysis and the two ROIs followed the temporal profiles of representations shown in
Model-free RCA is only sensitive to direct statistical relationship between neural RDMs, and fails to detect the connectivity if the two ROIs do not statistically relate. However, 2-model RCA allows detection of congruent inter-ROI statistical dependencies by having models that capture the representational structure of each ROI. Importantly, as 2-model RCA relies on hypotheses about the representations in source and destination areas, it will be less affected by confounders such as noise which are generally represented similarly across the two ROIs. Similar to the observation made in Simulation 2, it might be that common task-irrelevant patterns in both ROIs obscure the shared information as captured by 1-model RCA or the transformation of information as captured by 2-model RCA. A solution to this would be to remove their contribution by regressing out the RDM of the common pattern from the RDM of each ROI at each time-point. However, for this one needs the knowledge about the structure of the common patterns, which is not often known
Another solution to the failure of model-free RCA to detect connectivity under transformed representations might be to use non-linear mapping functions. Such functions allow more flexible relationships to be detected between areas despite drastic transformation of the representational structure. Such non-linear mapping functions include distance correlation (
Multi-dimensional connectivity is a rapidly developing area of brain connectivity analysis. One of the approaches to multi-dimensional connectivity is representational connectivity analysis (RCA). RCA quantifies the similarity of inter-relationship between the neural representations across experimental conditions for distributed patterns of activity of two brain regions (
Different types of inference about functional connectivity: top left and top right show the response patterns for N experimental conditions in two ROIs. Larger activations in a voxel are shown by lighter colors. One classical approach would be to reduce the dimensionality of data in each ROI to 1, and summarize the rich patterns of activity by a single vector containing one number for each experimental condition (or time-point for the case of resting-state data). Significant correlation between these vectors implies co-activation, i.e., that activations in ROI1 and ROI2 co-vary. Multi-dimensional connectivity methods that we consider in this article characterize the response patterns for different conditions by a representational dissimilarity matrix (RDM). Direct comparison of the RDMs (model-free RCA) tests for shared information (i.e., whether the two sets of response patterns in the two ROIs have any shared information with regards to the experimental conditions). Incorporation of models, i.e., model-based RCA, when a common model is used for both ROIs (1-model RCA) tests for shared information about a specific aspect of task/stimuli. This hypothesis in RCA is specified in the ROI-common model. Finally, model-based RCA with ROI-specific models (2-model RCA) detects potential transformation of information.
Model-free and model-based RCA can potentially provide inconsistent results in certain circumstances. These inconsistencies depend on many factors, some of which are the spatiotemporal structure of neural representations and the choice of the model(s) used in the analysis, and inform interpretation. Here, we focused on three simulations where model-free and model-based RCA provided opposing connectivity results.
First, we simulated a situation where the neural representations across a pair of regions showed unrelated information. As expected, model-free RCA showed no connectivity between the pair of regions. Interestingly, however, we observed that using model-based RCA with an intermediate model, which contains information about the representations in both regions, can leave the false impression that the two regions are connected. Specifically, the two regions showed almost equal, positive and significant correlation to the intermediate model suggesting that from the “lens” of the selected model, the two regions appear to be connected.
There are a few considerations. First, although for simplicity we did not directly implement either of the two published methods of model-based RCA (
In the second simulation, we modeled a situation where the addition of statistically related patterns of activity to a pair of statistically unrelated regions imposed a statistical relationship between them. This led to apparent connectivity in model-free RCA and when using 1-model RCA. However, the common pattern did not affect apparent connectivity when using 2-model RCA, as long as the two models were orthogonal and the two ROIs represented distinct information. Please note that the added common pattern can be non-structured or structured. Although we have seen that both common noise (non-structured) and structured patterns (data not shown) led to similar results, the structure of the common pattern can affect the connectivity as a result of interaction with the representations in the regions and models. It is also of note that the addition of common patterns does not always inflate the connectivity (e.g., in model-free or 1-model RCA); it can also decrease it leading to missing the connectivity. For example, if two regions are perfectly correlated, the addition of common noise (if not perfectly identical but only statistically related across regions) could lead to a decline in model-free RCA as a result of distorting the patterns. Generally, both model-free and model-based RCA can be affected by the noise as a result of the complex interaction between the representation in each region, the structure of the added pattern, the models, and the temporal dynamics of representations. Therefore, despite the situation shown in Simulation 2, these methods we are still far from remaining immune to common noise. We can, however, understand where we are most susceptible to it. One simple remedy for the effect of common patterns would be to regress out its contribution from the RDMs of the two ROIs prior to computing connectivity measures. This is in spirit similar to our recent implementation of model-based RCA (using partial correlation), where we partialled out the effect of additional low-level image statistics from the two regions under study (
In the third simulation, we showed a situation where two regions encoded different types of information that were either temporally congruent or incongruent. In other words, the information initially appeared in one region and after some delay in the other region (temporally congruent). Model-based RCA with proper choices of models can capture this relationship. This may be useful as transformation of information seems an integral part of brain connectivity as it seems unlikely that information would remain intact from one brain region to another (
This simulation also demonstrated the importance of the delay in connectivity analysis matching the data. The delay in the analysis potentially captures the neural lag in information transfer in the brain (
It is generally desired that a connectivity method determines the transferred
Additionally, one could also consider other extensions to model-free RCA. Similar to “information connectivity” (
One limitation of the current study is that we only evaluated connectivity using linear, rather than non-linear, relationships. While this simplification allowed us to make more intuitive predictions about the relationship between brain responses and the models, a more general approach would be to incorporate non-linear connectivity between areas as well. While we believe that the cases evaluated in Simulations 1 and 2 will not be affected by using a non-linear connectivity metric, non-linear mapping functions in Simulation 3 (
This work takes initial steps toward better characterization of the model-free and model-based RCA approaches that have been increasingly used in recent years. We tried to make the simulations as general and ideal as possible (no nuisance factors, e.g., measurement noise, leakage incorporated), so that the insights can be generalized to different implementations of the two general classes of model-free and model-based RCA. Therefore, the points made here can provide insight when studying brain connectivity using variety of neural recording modalities such as EEG, MEG, multi-electrode electrophysiology, and fMRI. Specifically, apart from Simulation 1, which presents a conceptual point applicable to all multivariate imaging/recording modalities, the methods implemented in Simulations 2 and 3 can directly be applied to EEG and MEG data.
The dataset used in this study is auto-generated using the Matlab script available at
HK-R: conceptualization, methodology, software, formal analysis, funding acquisition, writing – original draft, and writing – review and editing. AW and RH: conceptualization, methodology, and writing – review and editing. HN: conceptualization, methodology, software, formal analysis, writing – original draft, and writing – review and editing. All authors contributed to the article and approved the submitted version.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer AC declared a shared affiliation, with several of the authors, HK-R, AW, and RH, to the handling editor at the time of the review.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
This research was funded by the UK Royal Society’s Newton International Fellowship NIF\R1\192608 to HK-R and MRC intramural funding SUAG/052/G101400 and SUAG/046/G101400 to AW and RH, respectively.
We use the term “model” in a general sense: it can be a conceptual model, a computational model or a third brain region, etc.
We consider the case where the information is reliably transferred from one ROI to another as temporally congruent and cases where there is no transfer of information or the transformation is not reliable/consistent as temporally incongruent.