Keywords: fMRI, visual quality assessment, visual perception, brain mechanisms

Overview

Our goal is to explore the perception mechanism of visual quality and to identify the differences in activation levels and response patterns of various visual regions in the human brain when processing low-quality and high-quality images. To this end, we conduct an fMRI experiment in which 14 participants view natural images of three visual quality levels (low, neutral, and high quality) and three content categories (face, object, and scene) and perform quality assessment (QA) and content classification (CC) tasks. Each task condition includes 144 unique images (16 per category across nine categories), with each image presented twice for each participant.

All images are selected from KonIQ-10k, an image quality assessment (IQA) database including images with authentic distortion and MOS for their visual quality.⁴² We systematically categorized visual stimuli into three quality levels based on perceptual and semantic discriminability: low-quality images containing prominent global distortions that substantially impair semantic interpretation; neutral-quality images exhibiting localized distortions that preserve overall semantic recognition while compromising fine details; and high-quality images maintaining minimal perceptible distortions with sharp details, vivid colors, and unambiguous semantic clarity to serve as the perceptual reference standard. For details, see methods and Figure S1.

Blood oxygenation level dependent (BOLD) activity refers to changes in MRI signal caused by variations in blood oxygen levels within the brain. It reflects neural activity indirectly, as increased neuronal firing leads to localized increases in oxygenated blood, which alter the magnetic properties detected by fMRI.⁴³ We analyze fMRI BOLD data using multiple methods, as shown in Figure 1A. First, we employ univariate analysis and functional connectivity analysis to demonstrate that quality assessment is more complex than content classification and prove differences in the activation patterns of visual regions when understanding low-quality and high-quality images. The RSA then shows that visual regions of the brain do not encode visual quality directly, but visual quality significantly impacts the semantic encoding in related brain regions. Moreover, when understanding low-quality images, visual-related brain regions compensate for the loss of semantic information in the images. Subsequently, prediction models between the response patterns of ROIs are established, proving that the mapping relationship between ROI response patterns varies with visual quality and identifying brain regions most significantly affected by image distortion and those compensating for semantic information loss in low-quality images. To validate that low-quality signals trigger compensatory mechanisms and lead to differences in pattern prediction models between ROIs, we decode visual quality using combinations of different ROIs, demonstrating the feasibility of decoding visual quality from fMRI data and identifying the optimal brain region combinations for representing visual quality. Finally, based on the conclusions drawn from the fMRI experiment, we propose an artificial neural network multi-layer feature fusion strategy that significantly improves the performance in visual quality prediction.

Statistical analyses

To unravel the brain mechanisms underlying IQA, we first perform univariate analyses to explore the brain regions activated in the quality-assessment task. We focus on which brain regions are more sensitive in the quality-assessment task in contrast to the usual high-level task focusing on image semantics, so the contrast “QA vs. CC” is conducted, as shown in Figure 2A. We address the statistics and results in Table S1. In particular, we observe significantly increased BOLD activity in the visual pathway, such as the middle occipital gyrus, lingual, fusiform, and cuneus in the QA task compared to the CC task. However, increased activity in the CC task compared to the QA task is not found. To explore the physiological basis of quality of experience, we focus on whether images of varying quality would activate distinct brain regions. The contrast “high quality vs. low quality” is conducted, and its statistical results are detailed in Table S2. Both quality levels activate the precentral gyrus (left for low quality, right for high quality) according to the button placement for ratings. Similar activation patterns are observed in the insula, with high- and low-quality images activating the right and left insula.

In contrast “high-quality > low-quality” subregions of the lingual gyrus, inferior occipital gyrus, and middle occipital gyrus respond at a higher level to high-quality images, mainly involving regions Brodmann area 17 (BA17) and BA18. In addition, significant activation can also be observed in the bilateral inferior parietal lobule and bilateral cingulate gyrus, as shown in Figure 2B. In contrast “low quality > high quality,” significant activations can be observed in the subregion of bilateral fusiform, middle occipital gyrus, and superior occipital gyrus, mainly involving regions BA19 and BA37. Other activations are found in the bilateral middle temporal gyrus, inferior temporal gyrus, inferior frontal gyrus, middle frontal gyrus, left superior parietal lobule, and inferior parietal lobule.

Given the responsiveness of specific brain regions to stimulus quality, our study investigates regions showing positive or negative correlations with BOLD signal intensity related to quality. We conduct the contrasts “high quality vs. neutral quality” and “neutral quality vs. low quality,” identifying overlapping regions consistently activated across both contrasts. We scrutinize the beta parameters estimated by the general linear model (GLM) within these overlapping regions across high-, neutral-, and low-quality conditions. Results shown in Figure 2C indicate increased BOLD responses in bilateral BA18 subregions with higher image quality. By contrast, regions within the superior occipital gyrus and middle frontal gyrus show the opposite trend, with lower quality enhancing BOLD signal responsiveness, as shown in Figures 2D and 2E.

The univariate analysis above shows that the response amplitude in the superior occipital gyrus increases as the image quality degrades, which underscores the crucial role of the superior occipital gyrus in understanding low-quality images.

To further investigate the activity of the superior occipital gyrus under low-quality conditions, we conduct a seed-based functional connectivity analysis using the superior occipital gyrus as the seed. The results, as illustrated in Figure 2F, indicate that in the comparison of “low quality < high quality,” two clusters show significant positive connectivity with the superior occipital gyrus, located at the right occipital pole and the right inferior frontal gyrus (voxel threshold , cluster threshold ). This suggests that under low-quality conditions, the superior occipital gyrus enhances cooperation with brain regions associated with advanced visual cognition, memory, and scene construction to better parse and understand the content of low-quality images.

Representation similarity analysis

To analyze the response patterns of each visual region under different quality conditions, RSA and t-distributed stochastic neighbor embedding (t-SNE) are employed, which are generally accepted in neuroscience for multi-voxel pattern analysis.⁴⁴ We aim to identify brain regions that encode visual quality, similar to how the fusiform face region responds to faces.^45,46 If no such region is found, we will examine differences in response patterns across visual regions under varying quality conditions.

Representations in visual regions for images of different qualities

Voxel-level BOLD activity patterns are extracted from each ROI for each presented image, with beta maps averaged across multiple presentations. This analysis combines data from all tasks and images without distinguishing between tasks. For each participant (number of participants N = 14), representation vectors are constructed using beta values from the 100 most reliable voxels in each ROI, including the calcarine, cuneus, lingual gyrus, fusiform gyrus, and superior, middle, and inferior occipital gyrus in both hemispheres. Vectors from the left and right hemispheres are concatenated for unified representation. The correlation distance between vectors for all images is calculated to create a representation dissimilarity matrix (RDM) for each ROI.

The t-SNE algorithm embeds high-dimensional data into 2D space, modeling similar data as near points and divergent data as far points, thus preserving similarity relationships. To visualize the similarity between representation vectors of images with different semantics under varying quality conditions, we calculate average RDMs across all participants for each quality level and use t-SNE to visualize these RDMs. See the methods section for details about the experiment setup.

Figure 3A shows the average RDM and the t-SNE results for the superior occipital gyrus under high- and low-quality conditions. The RDM demonstrates that representation vectors of semantically congruent images form discernible clusters in the t-SNE-derived 2D space, particularly for facial stimuli within the superior occipital gyrus. However, no analogous spatial clustering is observed for vectors representing quality-matched images in any examined brain region. This suggests that, unlike semantic information, visual quality attributes are not robustly encoded in the spatial patterning of neural responses within these regions. For full t-SNE results across quality conditions, see Figure S4.

Figure 3A also reveals that in low-quality conditions, the clustering of representation vectors of the images with similar semantics in the superior occipital gyrus is less distinct than in high-quality conditions, suggesting more disorganized semantic encoding. RDMs and t-SNE results for other ROIs are available in Figure S3.

Quantifying the content of semantic information in the representation vectors of each visual region

To quantitatively analyze the impact of visual quality on response patterns in visual brain regions, we quantify the quality and semantic information content in representation vectors of each ROI. RDMs are computed for each brain region of each participant in three quality conditions, and their similarities to the ideal quality and semantic RDMs are measured using Pearson correlation coefficients, as shown in Figure 3B.

We first analyze the information content of visual quality within each brain region, with results shown in Figure 3C. The analysis reveals low-quality-related information content across all examined brain regions, with no regions explicitly encoding quality information. In contrast, semantic information content is more substantial across brain regions and varies under different quality conditions. As shown in Figure 3D, the content of semantic information is richer in high-level visual regions, consistent with the human brain’s ventral visual stream. Specifically, the semantic content in the calcarine, cuneus, and superior occipital gyrus shows a significant correlation with visual quality, with lower quality leading to reduced semantic content (Welch’s ANOVA: calcarine ; cuneus ; superior occipital gyrus ). As information moves to higher visual regions, these quality-induced differences become non-significant statistically. In the middle occipital gyrus, inferior occipital gyrus, and fusiform gyrus, semantic content remains statistically unaffected by quality. The power analysis is shown in Table S4 and the section “supplemental statistical power analysis” in supplemental notes.

The average RDMs of each ROI across participants in each quality condition are computed and their similarities to ideal semantic labels measured. As shown in Figure 3E, averaging reduces random noise and individual differences, leading to more robust results. All ROIs show higher semantic content across all quality conditions than individual analyses. The difference in semantic information content is largest between high- and low-quality conditions in the calcarine, decreases between the superior and middle occipital gyrus, and remains low in the middle occipital gyrus, inferior occipital gyrus, and fusiform.

Noise and information loss in low-quality images impact response patterns in primary and secondary visual regions, weakening cognitive functions in image recognition, while high-level visual regions continue to encode semantic information effectively from low-quality images with information loss. This indicates that there are compensatory mechanisms within the visual pathways that address the loss of information in low-quality visual signals, correctly encoding semantic information to ensure proper image understanding.

Building the response pattern prediction model between ROIs

As illustrated in the section “representation similarity analysis,” the semantic content in primary visual regions decreases as visual quality degrades, while it remains largely unaffected in high-level visual regions. Based on the above observations, we hypothesize that the intermediate part of the ventral visual pathway is actively adaptive to low-quality conditions, leading to changes in the way information is processed and transmitted between the primary visual regions and high-level visual regions. Consequently, the mappings between response patterns in different ROIs of visual pathways exhibit differences in response to varying qualities.

To validate the hypothesis and identify brain regions with adaptability to low-quality conditions, we implement a system to model the predictive relationship between response patterns from different brain regions under three quality conditions: high quality, neutral quality, and low quality. Specifically, each ROI is used to predict the response patterns of the ROIs with higher levels in the visual processing pathway, learning the mapping relationships and analyzing their dependency on visual quality.

Let be the representation vector of images in ROI , and be the representation vector in ROI . There exists a mapping system such that . Should our hypothesis be supported, we expect to identify the ROI pairs for which the mapping system is not invariant to visual quality, i.e.,

where .

Initially, images and their corresponding representation vectors under high-quality, neutral-quality, and low-quality conditions are divided into training and testing sets. A support vector regression (SVR) model is then trained using the high-quality training set to establish a mapping model between the representation vectors of ROIs and . This model’s performance is subsequently tested on the test sets of the three different quality conditions. Performance is quantified by the linear correlation between the predicted and the true representation vectors of ROI . To mitigate the impact of shared signals between brain regions, we employ cross-prediction using data from two repeated measurements. The noise ceilings are estimated by calculating the linear correlation between the response patterns of each ROI across the two measurements.

Significant performance discrepancies should emerge across test sets of different qualities if the mapping between ROIs and varies with quality. Furthermore, the system’s performance on the test set corresponding to the low-quality training condition may exceed that under other conditions, suggesting a particular adaptability to low-quality images.

The experiment is conducted independently for each participant, using ROI representation vectors consistent with the previous RSA. Seven ROIs (calcarine, cuneus, lingual, fusiform gyrus, and superior, middle, and inferior occipital gyrus) are paired to make predictions from A to B. Low-quality training datasets are used to train the SVR model, mapping representation vectors between ROIs A and B using an approach similar to that described above. A model adapted to low-quality images would perform best on low-quality tests. As shown in Figure 4A, the predictive performance of the model predicting the middle occipital gyrus and the inferior occipital gyrus from the other ROIs, as well as the model predicting the lingual from the cuneus, is significantly correlated with quality conditions, with significantly better performance on low-quality tests than on neutral or high-quality ones. However, after training under high-quality conditions, we did not identify any ROI pairs with significantly higher performance on the high-quality test set, as shown in Figure 4B. Detailed data for each ROI pair can be found in Figures S6 and S7. The power analysis is shown in Table S5 and the section “supplemental statistical power analysis” in supplemental notes.

This result is consistent with the well-known free energy principle (FEP), which posits that the brain processes information by generating predictions about sensory inputs and minimizing prediction errors.^47,48 Our research provides a concrete example supporting this theory. The findings presented in “representation similarity analysis” demonstrate that the brain can ensure semantic understanding even under conditions of low-quality visual signals. The phenomenon in this section indicates that the brain continuously refines its predictions to reduce uncertainty, in line with the error minimization principle proposed by the FEP. Furthermore, this experiment illustrates how the brain adjusts its internal models to adapt to changing environmental conditions, a behavior anticipated by the FEP.

Predicting visual quality using response patterns of ROI pairs

According to the results in “representation similarity analysis,” we find that individual ROI response patterns contain minimal visual quality information. However, a compensatory mechanism within the visual processing pathway ensures that low-quality visual signals are adequately interpreted, and the prediction models between ROIs exhibit variations depending on visual quality, as illustrated in the section “building the response pattern prediction model between ROIs.” Therefore, we have reason to hypothesize that, although no brain region explicitly encodes visual quality information, it is feasible to decode visual quality by combining information from multiple ROIs. To validate this, we predict visual quality from fMRI signals using representation vectors of individual ROIs and concatenate vectors from pairs of seven vision-related ROIs, as shown in Figure 4B. We also concatenate vectors from all seven ROIs, reflecting the joint predictive performance of all ROIs. Randomly generated predictions for visual quality demonstrate the difference between predictions from brain regions and random results. Experiments use the same vectors as in RSA analysis, conducted independently on each participant’s data, maintaining consistent visual quality labels. SVR models are used with four-fold cross-validation and repeated ten times to minimize random effects on conclusions.

Predictive performance is measured using Spearman’s rank correlation coefficient (SRCC) and Pearson’s linear correlation coefficient (PLCC). SRCC primarily reflects the consistency and monotonic relationship between the predicted visual quality ranking and the ground truth, while PLCC primarily reflects the linear correlation between predicted visual quality and ground truth. Higher SRCC and PLCC indicate greater consistency with human subjective perception of visual quality. The mean and variance of performance across all participants are reported. We use a two-tailed independent-samples t test to analyze whether there is a significant difference in the mean predictive performance of each brain region or ROI pair compared to the joint predictive performance of all ROIs across all participants (threshold , false discovery rate [FDR] corrected).

The experimental results, as illustrated in Figure 5, show that joint prediction using all ROIs achieves the best performance. Among them, the prediction results of a single ROI are generally lower than those of the joint prediction using two ROIs. The ROI pair lingual-middle occipital gyrus achieves the best performance among all ROI pairs, and ROI pairs including calcarine and lingual generally have high prediction performance. There are some ROI pairs, such as calcarine-middle occipital gyrus, lingual-superior occipital gyrus, and lingual-middle occipital gyrus, whose joint prediction performance is not significantly different from using all ROIs for joint prediction.

We calculate the SRCC and PLCC between the ground truth and the average of all participants’ predictive results, designated as the group prediction performance. The results, illustrated in Figure 5, demonstrate that the group prediction significantly surpasses individual predictions. This improvement is likely due to the attenuation of random noise effects. Furthermore, the discrepancy between the predictive performance of individual ROI pairs and the combined predictions using all ROIs is notably reduced. The best-performing ROI pair in the group prediction is the lingual-superior occipital gyrus and lingual-middle occipital gyrus, which coincides with the dominant ROI pair identified in the above participant-level predictions.

The results in Figure 5, both for individual and group predictions, show that decoding MOS from participants’ fMRI signals performs worse than existing deep-learning-based IQA algorithms, which predict visual quality from images. We attribute this phenomenon to two main factors. First, the fMRI signal inherently contains considerable noise and physiological variability, whose substantial influence is widely acknowledged in many fMRI decoding studies.^23,49,50 Second, while MOS is an explicit quantification of human subjective experience, it may not fully capture variations in user perception. This issue is also commonly observed in studies using EEG to measure visual quality perception.^38,41 The decoding performance of each participant can be found in Figure S14 and Tables S6 and S7.

Brain-inspired multi-layer feature fusion strategy for artificial neural networks

It has become common practice in recent human fMRI research to consider convolutional neural networks (CNNs) as computational models of the human visual system. This view is supported by fMRI studies demonstrating that representations in the lower and higher layers of CNNs correspond to neural activity patterns in the early and higher-level visual processing regions of the brain, respectively.^25,26,51,52 In particular, object category encoding learned by deep convolutional networks trained on object classification tasks serves as a robust proxy for neural representations.²⁶

As demonstrated in “representation similarity analysis,” visual quality information is not directly encoded within a single ROI. In “building the response pattern prediction model between ROIs,” we prove that the prediction models between ROIs vary with visual quality, indicating that quality information is reflected in the relationships among ROIs. Based on this discovery, we successfully decode visual quality information from fMRI signals in the section “predicting visual quality using response patterns of ROI pairs.” The results indicate that the combined prediction performance of two or more ROIs is better than that of a single ROI. This implies that different visual regions interact and collaborate during the processing of visual quality. Consequently, we propose a multi-layer fusion strategy to mimic this process in the CNN pre-trained on object classification tasks, aiming to explore whether the findings from fMRI experiments can be validated in CNNs.

The architecture shown in Figure 6A incorporates a backbone network for feature extraction and additional paths (paths 1–4) that branch out from intermediate layers to capture multi-scale features and fuse them at the regression head. It aims to integrate information from different levels, similar to how the brain regions work together in the visual pathway. This strategy is expected to improve the performance of visual quality prediction by leveraging the unique contributions of features at different levels, just as the brain utilizes the cooperation of different visual regions for effective VQA. We selected SqueezeNet,⁵³ ResNet,⁵⁴ and EfficientNet⁵⁵ as backbone networks to evaluate the generalizability and adaptability of our proposed feature fusion method across architectures with varying complexity and design principles. SqueezeNet represents a lightweight model optimized for efficiency with significantly fewer parameters, making it suitable for deployment-constrained environments.⁵³ ResNet introduces residual learning, which has proven effective in stabilizing deep network training and improving performance across a wide range of vision tasks.⁵⁴ EfficientNet, on the other hand, leverages a compound scaling strategy to achieve high accuracy with balanced computational cost.⁵⁵ The inclusion of these three models allows us to assess the robustness of our fusion strategy under different architectural paradigms. For details of the network architecture and parameters, refer to the methods section.

The backbone networks are initially pre-trained on ImageNet,⁵⁶ a widely used large-scale benchmark dataset in image classification that enables the acquisition of robust visual semantic representations exhibiting similarities to human visual cortex processing.²⁷ Subsequently, these networks are fine-tuned on each IQA dataset to adapt specifically to the characteristics of IQA. The experiments use a five-fold cross-validation scheme across three authentic distortion IQA benchmark datasets: LIVE-Challenge (CLIVE),⁵⁷ KonIQ-10k,⁴² and BID.⁵⁸ These datasets are selected to balance content diversity, distortion types, and annotation quality, ensuring a thorough validation of the network’s generalizability and performance. For detailed information on dataset sizes, distortion types, and other experimental settings, please refer to the methods section.

Each path (paths 1–4) and the backbone are independently trained and tested while maintaining identical dataset splits, training protocols, and hyperparameters to ensure a fair comparison of predictive performance. As shown in Figures 6B and S8, the results suggest that incorporating features from path 2, path 3, and path 4 generally leads to improved prediction performance (measured by PLCC and SRCC) compared to using the backbone alone. While the differences in correlation are relatively small, path 2 shows a consistent advantage on the KonIQ-10k and BID datasets, potentially benefiting from the high-level semantic information it encodes. In addition, path 3 and path 4, which leverage intermediate and lower-layer features, perform comparatively well on CLIVE, indicating potential robustness to complex quality variations. These findings imply that both high-level and low-level features may provide complementary information for quality prediction.

The statistical significance tests confirm the effectiveness of the fusion strategy, with path 2, path 3, and path 4 providing consistently strong and reliable results across different datasets and backbones. This indicates that features extracted at different levels of the network contribute uniquely to understanding visual quality. Integrating features from different levels can enhance the network’s performance in predicting visual quality. Furthermore, we compare the prediction performance of the network with multi-layer feature fusion to other IQA methods. As shown in Table 1, EfficientNet with the multi-layer feature fusion strategy demonstrates superior visual quality prediction performance. This result indicates that the multi-layer feature fusion strategy not only validates the conclusions of the previous fMRI experiment in CNNs but also holds practical application value.

From the results, it can be observed that the multi-layer feature fusion strategy exhibits varying performance across the three networks and datasets. A detailed analysis of the reasons behind the differences can be found in the discussion.

To validate the generalizability of the multi-layer models across datasets with different sources, we evaluate the models on the CLIVE and BID datasets, which are pre-trained on KonIQ-10k. The KonIQ-10k dataset, being larger in scale compared to CLIVE and BID, is more suitable for pre-training purposes. As shown in Figures S10 and S11, the results demonstrate that incorporating features from path 2, path 3, and path 4 consistently improves prediction performance (measured by PLCC and SRCC) compared to the backbone alone. The statistical significance tests confirm the effectiveness of the fusion strategy in enhancing the generalizability across datasets with different sources. For detailed training procedures and hyperparameters, please refer to the methods section. The loss curve for each network on each dataset can be found in Figure S9.

fMRI participants

Seventeen healthy right-handed university students participated in this study. Three participants were excluded for excessive motion, leaving 14 for analysis (seven males and seven females, ages 20–27, mean age 23.28, standard division 2.05). Among the 14 participants, four had extensive experience in VQA experiments. Excessive motion is defined a priori as >2 mm translation or >2° rotation. All participants completed a Snellen visual acuity test prior to the experiment and confirmed that they have normal or corrected-to-normal vision, defined as a Snellen acuity score of 20/20 at a distance of 6 m.⁸³ All participants have no known neurological condition. The institutional review board at Shanghai Jiao Tong University approved the experimental protocol, and all participants gave written informed consent. After being informed about potential risks and screened by an institution’s physician, participants gave informed consent before participating. All data were processed anonymously.

Stimuli

All images are manually selected from KonIQ-10k, an IQA database including images with authentic distortion and MOS for their visual quality.⁴² The KonIQ-10k contains 10,073 images selected from the large public multimedia database YFCC100m,⁸⁴ covering a wide range of distortions in brightness, color, contrast, noise, sharpness, and other quality dimensions. The subjective scores in the KonIQ-10k dataset are collected through a crowdsourcing protocol, with each image rated by at least 120 unique participants. The ratings followed the standard 5-point absolute category rating scale: bad (1), poor (2), fair (3), good (4), and excellent (5). After quality control and filtering, the scores are averaged across participants, normalized to a [0–100] scale, and aligned with expert opinion to derive the MOS for each image. KonIQ-10k ensured content diversity by sampling 1 million images from YFCC100m based on machine-generated tags, which are assigned by a deep neural network and represented the top predicted categories for each image. When selecting stimulus images, visual quality is categorized based on the MOS provided by the dataset. Scores below 35 are classified as low quality, between 35 and 70 as neutral quality, and above 70 as high quality. All images are cropped to fit a -pixel frame. When classifying image content, we manually inspect each image to ensure that its semantic content is unambiguous and can be uniquely assigned to one of three categories: faces, objects, or scenes.

All stimuli are classified into nine categories according to content category (faces, objects, or scene) and image quality (bad, neutral, or excellent). These images are prepared for two tasks: IQA and content classification. Under each task condition, there are 16 images per category across nine categories, resulting in a total of 144 unique images. Each image is presented twice, leading to 288 trials per task for each participant.

fMRI experiment procedure

The experimental process is divided into eight runs, each containing four blocks. Participants in each block perform one of the two tasks: quality assessment (bad, neutral, or excellent) or content classification (faces, objects, or scene), and each block contains 18 trials, as shown in Figure S2. A single continuous scanning run lasted 384 s and comprised four blocks, each lasting 96 s. Each block included an initial task instruction period (4 s), presentation of 18 images (3,500 ms for each image), with inter-stimulus intervals (ISIs) randomized between 500 and 4,500 ms to prevent anticipation effects on perception. The 3,500 ms image presentation time, as empirically justified, allows sufficient duration for participants to both perceive the image and perform semantic classification or quality-assessment tasks.⁸⁵ Although the intervals are randomized, the total duration of each block is controlled to remain exactly 96 s.

The systematic counterbalancing of task block order across runs is implemented to ensure that the tasks are presented in a balanced manner across runs, effectively controlling for task order confounds. Specifically, in odd-numbered runs, the task block sequence followed a QA-CC-CC-QA pattern, while in even-numbered runs, the order is reversed to CC-QA-QA-CC. This systematic counterbalancing allows for a robust counterbalancing between the tasks, ensuring that the influence of task order is minimized.

The order of three quality levels and three content categories is also counterbalanced across runs within participants. The stimulus sequences are carefully designed to integrate category-level temporal optimization with item-level repetition control, ensuring both experimental rigor and statistical efficiency. For each task, optimized randomized sequences of nine stimulus categories are generated using FreeSurfer’s optseq2 tool. ISIs are jittered by randomly selecting durations from 0.5, 2.5, and 4.5 s to reduce temporal autocorrelation and improve event-related signal estimation. Within each category, individual stimuli are presented twice. To manage stimulus repetitions, each stimulus is duplicated, and a constrained randomization procedure employing a backtracking shuffle algorithm is applied. This algorithm enforced a minimum lag of one stimulus between repeated presentations, preventing immediate repetitions and mitigating potential habituation or carryover effects. The resulting item-level sequences are then mapped onto the category-level optseq2 sequence, preserving the optimized timing and ISI structure. The full stimulus sequences are segmented according to block lengths and allocated accordingly to runs and blocks, thereby achieving systematic counterbalancing of task order while maintaining a fixed stimulus presentation sequence across participants.

A fixation cross is presented for 12 s at the beginning and 30 s at the end of each run. Scan data from the last 30 s of each run are treated as the resting state for subsequent analyses. All participants complete eight runs. More information about the fMRI experiment including the instruction for participants is provided in the section “supplemental fMRI experimental setup” in supplemental methods.

fMRI data acquisition

Visual stimulation is presented using the SINORAD SA-9939 Brain Functional Audiovisual Stimulation system, integrated with E-Prime, with a 40-inch LCD screen for visual stimulation. E-Prime initiated the stimulus task and waited for a synchronous trigger—generated by the MRI device at fMRI sequence onset, transmitted to the main control box—to execute the program, ensuring precise alignment with the brain’s visual and semantic processing timing. The distance between the LED screen and the participants’ eyes is 160 cm, in accordance with the International Telecommunication Union Recommendation ITU-R BT.500-14.⁸⁶ The ambient lighting conditions are maintained constant throughout the experiment. The participants view the display through an angled mirror (45°) attached to the head coil. The stimulation system is synchronized with the MRI system to provide a time reference for the stimulation task and MRI imaging. The stimulus program is written using E-Prime 3.

All MRI data are obtained on a 3T Siemens Prisma scanner equipped with a 32-channel head coil at the Department of Radiology, Renji Hospital School of Medicine, Shanghai Jiao Tong University, Shanghai, China. A gradient echo-planar imaging sequence is employed with the following parameters: repetition time (TR) = 2,000 ms, echo time (TE) = 30 ms, flip angle = 90°, matrix size = 64 64, field of view = 192 mm, slice thickness = 2 mm, no inter-slice gap, and 70 axial slices covering the whole brain. The phase-encoding direction used is anterior-posterior. In addition, T1-weighted (T1w) 3D structural images are acquired by using an MPRAGE sequence (TR = 1,800 ms, TE = 2.28 ms, flip angle = 8°, voxel size = mm), and T2-weighted 3D structural images are acquired by using a turbo spin echo sequence (TR = 9,560 ms, TE = 90 ms, flip angle = 150°, voxel size = mm).

fMRI data pre-processing

All MRI data are converted into brain imaging data structure using dcm2niix (v.1.0.20220720). Results included in this article come from pre-processing performed using fMRIPrep 23.1.0.⁸⁷ which is based on Nipype 1.8.6.⁸⁸

Each T1w image is corrected for intensity non-uniformity, skull stripped. Brain tissue segmentation of cerebrospinal fluid (CSF), white matter, and gray matter is performed on the brain-extracted T1w image. Brain surfaces are reconstructed using recon-all.⁸⁹ Volume-based spatial normalization to one standard space (MNI152NLin2009cAsym) is performed, using brain-extracted versions of both T1w reference and the T1w template. Functional images are slice-time corrected, motion corrected, co-registered to the structural image, and normalized to MNI space. See the section “supplemental fMRI data pre-processing” of the supplemental methods for details of anatomical and functional data processing.

After pre-processing by fMRIPrep, the functional data are spatially smoothed with an isotropic 4-mm full-width-half-maximum (FWHM) Gaussian kernel only for univariate analysis and functional connectivity analysis. All analyses in the sections “representation similarity analysis,” “building the response pattern prediction model between ROIs,” and “predicting visual quality using response patterns of ROI pairs” are based on non-smoothed data.

Both smoothed and non-smoothed functional data are denoised using a standard pipeline⁹⁰ that includes regression of potential confounding effects characterized by white matter time series (five CompCor noise components), CSF time series (five CompCor noise components), motion parameters and their first-order derivatives (12 factors),⁹¹ outlier scans (up to 89 factors),⁹² and linear trends (two factors) within each functional run, followed by band-pass frequency filtering of the BOLD time series⁹³ between 0.008 Hz and 0.09 Hz. CompCor^94,95 noise components within white matter and CSF are estimated by computing the average BOLD signal and the largest principal components orthogonal to the BOLD average, motion parameters, and outlier scans within each participant’s eroded segmentation masks. Motion parameters, first-order derivatives (12 factors), and outlier scans are confounds extracted in the fMRIPrep analysis stream. Based on the number of noise terms included in this denoising strategy, the effective degrees of freedom of the BOLD signal after denoising are estimated to range from 370.6 to 477.9 (average 434.1) across all participants.⁹⁶

Univariate analysis

The classical univariate statistical analysis is performed on SPM12.⁹⁷ In the first-level analysis, we use the quality-assessment task and content-classification task as regressors of interest to explore brain mechanisms for quality-assessment tasks and use high quality and low quality as regressors of interest to find brain regions sensitive to image quality. A high-pass filter of 128 s and an AR(1) model are used for signal drift correction and serial correlations, respectively. The statistically significant thresholds for reported group-level analysis results are set at a voxel-level and FDR-corrected p < 0.05. The contrast matrix can be found in the section “supplemental univariate analysis” in supplemental notes.

Seed-based functional connectivity analysis

Results of seed-based functional connectivity come from analyses performed using CONN⁹⁸ and SPM.⁹⁷ After pre-processing and denoising, psychophysiological interaction analyses are used to study the changes in functional connectivity across good, bad, and rest conditions. Seed regions include two automated anatomical labeling (AAL) ROIs.⁹⁹ Separately for each pair of seed and target areas, a generalized psychophysiological interaction model (gPPI^100,101) is defined with seed BOLD signals as physiological factors, boxcar signals characterizing each task condition convolved with an SPM canonical hemodynamic response function as psychological factors, and the product of the two as psychophysiological interaction terms. Functional connectivity changes across conditions are characterized by the multivariate regression coefficient of the psychophysiological interaction terms in each model.

Group-level analyses are performed using a GLM.⁹⁰ For each voxel, a separate GLM is estimated, with first-level connectivity measures at this voxel as dependent variables (one independent sample per participant and one measurement per task or experimental condition, if applicable) and groups or other participant-level identifiers as independent variables. Voxel-level hypotheses are evaluated using multivariate parametric statistics with random effects across participants and sample covariance estimation across multiple measurements. Inferences are performed at the level of individual clusters (groups of contiguous voxels). Cluster-level inferences are based on parametric statistics from Gaussian random field theory.^90,102 Results are thresholded using a combination of a cluster-forming voxel-level threshold, and a family-wise corrected cluster-size threshold.

Single trial betas and patterns of each ROI

To extract single-trial fMRI response patterns for each ROI in a given run, we first convolve the onset times of each trial with the hemodynamic response function corresponding to each trial; we then conduct a GLM analysis to extract the beta weights for each voxel within that ROI for each trial. It is important to emphasize that the fMRI data used here are pre-processed and denoised using fMRIPrep but are not spatially smoothed. These voxel beta weights are used as the fMRI response patterns for that trial during the run.

We extract visually evoked responses from seven vision-related ROIs, including the calcarine, cuneus, lingual, superior occipital gyrus, middle occipital gyrus, inferior occipital gyrus, and fusiform gyrus. The boundaries of these ROIs are determined based on the AAL template,⁹⁹ which provides a consistent and reproducible method of quantifying brain activity and structure across different individuals and studies.

To reduce the dimensionality of response patterns within each ROI and enhance their reliability, facilitating the establishment of mappings between ROIs, we select the 100 voxels with the highest reliability from each unilateral ROI. To calculate the reliability of each voxel within each ROI, we use a GLM to compute the beta weights corresponding to 18 conditions (derived from two task conditions, three image content categories, and three quality levels, thus ) for each run. The data are split into odd and even halves by run, and we average the data across runs within each half. We correlate the beta weights from all conditions between the two halves for each voxel. Subsequently, the top 100 voxels exhibiting the highest correlations are selected from each unilateral ROI. Masks composed of these voxels are then applied to the beta weights corresponding to each image to derive the response patterns for the respective ROIs. For more detailed information, see Tarhan and Konkle.¹⁰³ Additionally, the methodology has been employed in works such as those by Xu and Vaziri-Pashkam²⁷ and Cadieu et al.,¹⁰⁴ where Xu and Vaziri-Pashkam²⁷ specifically selected 75 voxels demonstrating the highest correlation in each ROI.

We use the beta weights of these selected voxels to characterize the response pattern of the unilateral ROI to each image. The response patterns from the left and right sides of the same ROI are then concatenated to represent the response of that ROI to each image. We apply Z-score normalization to the averaged pattern for each condition in each ROI to remove amplitude differences between conditions and ROIs.

Representation similarity analysis

To investigate how visual quality impacts response patterns within the visual pathway, we employ RSA⁴⁴ to quantify the semantic information content across various ROIs with RSAtoolbox.¹⁰⁵ Initially, semantic labels for faces, objects, and scenes are encoded as three one-hot vectors, i.e., (1, 0, 0), (0, 1, 0), and (0, 0, 1), with each image uniquely associated with one semantic label. The quality labels for bad, neutral, and excellent are encoded as (0, 1), (1, 1), and (1, 0), respectively, with each image uniquely associated with a quality label. Subsequently, we calculate the correlation dissimilarity between the ideal semantic labels of the images as the ideal semantic RDM and the Euclidean dissimilarity between the ideal quality labels as the ideal quality RDM. For each participant, the representations extracted as described above corresponding to the repeated presentations of the same image are averaged to serve as the response estimate for that image.

Similarly, we compute the dissimilarity between the response patterns of all image pairs within each ROI, independently forming each ROI’s RDM for each participant. The dissimilarity between response patterns is measured using the Pearson correlation coefficient, which is as reliable as Euclidean-like dissimilarities.¹⁰⁶ The correlation distance depends not only on the differences between two patterns but also on shared overall activations, such that additional shared activity, which does not impair decoding, still reduces the correlation distance.

Subsequently, the similarity between the ideal semantic RDM and the RDM of each ROI for every participant is calculated, serving as a representation of the semantic information content within each ROI for each participant. Results on similarity obtained using the Spearman correlation coefficient are presented in Figure S5 and lead to conclusions consistent with those derived from the Pearson correlation. Similarly, the similarity between the ideal quality RDM and the RDM of each ROI for every participant is calculated, serving as a representation of the quality information content within each ROI for each participant.

In this study, we employ Welch’s ANOVA and the Games-Howell post hoc test to evaluate the effects of three levels of image quality on the semantic content associated with ROI. The use of Welch’s ANOVA is necessitated by the lack of homogeneity of variances among the groups. Welch’s ANOVA is utilized to detect any statistically significant differences across the image-quality groups, and, where significant differences are found, the Games-Howell test is applied for post hoc pairwise comparisons to identify specific group differences.

Modeling the mapping relationship between response patterns of any two brain regions

This study utilizes an SVR model with a radial basis function (RBF) kernel to build the mapping between ROI response patterns. The model is configured with a regularization parameter (C) of 1 and an epsilon of 0.1, balancing the trade-off between fitting the training data and preventing overfitting while enhancing robustness to outliers. This configuration is chosen for its effectiveness in capturing non-linear relationships within the data. The models are independently trained for each dimension of the representational vectors, ensuring a robust method for modeling the complex mappings between the ROI response patterns. The SVR model’s input is the representational vector of ROI A, and the output is the representational vector of ROI B, both consistent with the representational vectors extracted in the section “single trial betas and patterns of each ROI,” each with a length of 200.

To mitigate the impact of shared signals between brain regions, we employ cross-prediction using data from two repeated measurements. Specifically, the response pattern of ROI A from the first measurement is used to predict the response pattern of ROI B from the second measurement and vice versa.

The linear correlation between the actual and predicted representational vectors of ROI B is calculated to assess the model’s predictive performance. Let be the actual representational vector of ROI B and be the predicted representational vector of ROI B. The linear correlation can be expressed as

where is the covariance between and and are the variances of and , respectively.

In this study, each mapping relationship for each participant is trained independently. For each participant, the data under each quality condition are randomly divided into four folds and subjected to cross-validation. The reported performance is the average result from multiple rounds of cross-validation, where the data for each participant are randomly divided into four folds ten times, with each split being independently trained. It is important to note that we trained the model using data from only one quality condition (either high or low quality) and then tested it across all three quality conditions (high, neutral, and low). The test set for each of the three quality conditions corresponds to the same fold number. For example, if the model is trained using the first, third, and fourth folds of the high-quality data, the second fold is used for testing across all three conditions.

To examine whether visual quality significantly affects the distribution of model performance among participants, we conduct a Welch’s ANOVA analysis followed by Games-Howell post hoc tests. Mappings with p values less than 0.05 are marked in the figures.

To explore the theoretical performance ceiling of the predictive model, reflecting the inherent variability of the data itself, we calculate the noise ceiling for each participant and each ROI. The noise ceiling for a given participant and ROI is computed by evaluating the linear correlation between the representations of repeated measurements for each image. For each image , we compute the linear correlation between the representations of repeated measurements. This can be expressed as

where is the representation of the image for participant in ROI is the repeated measurement representation for the same image. Next, the average correlation across all images for participant and ROI is calculated:

where is the total number of images.

In Figures 4, S6, and S7, the average noise ceiling represents the average of these correlations across all participants for each ROI is then taken as the noise ceiling for that ROI:

where is the total number of participants.

Predicting visual quality using response patterns of any two brain regions

To validate the feasibility of decoding visual quality information from fMRI data, considering the available data scale, SVR models with RBF kernels are chosen to regress visual quality scores from the representational vectors of various ROIs. The input to the predictive model for a single ROI consists of the representational vector for that ROI, while for combined ROIs the input vector is formed by concatenating the representational vectors of ROI A and ROI B. All representational vectors are consistent with those used in the previous RSA. The SVR models are configured with a regularization parameter (C) of 1 and an epsilon of 0.1. The actual quality scores of the images are quantized into discrete integers 3 for high quality, 2 for neutral quality, and 1 for low quality, while the output of the predictive model is a continuous floating-point number.

The performance of the predictive model is characterized using the SRCC and PLCC. The SRCC measures the strength and direction of the monotonic relationship between the predicted and actual quality scores and is defined mathematically as

where is the difference between the ranks of the th observation in the predicted and actual scores, and is the number of observations. The PLCC, on the other hand, assesses the linear correlation between the predicted and actual quality scores, given by

where and are the predicted and actual scores, respectively, and and are their respective means. Both SRCC and PLCC are essential for evaluating the accuracy and reliability of quality decoding models, with SRCC focusing on rank correlation and PLCC on linear correlation.

The predictive models for each combination of ROIs for each participant are trained independently, with all models undergoing ten rounds of four-fold cross-validation. The predictive performance for each combination of ROIs for each participant is calculated as the average result across these multiple rounds of cross-validation.

Implementation detail of brain-inspired multi-layer feature fusion strategy

Based on the conclusions from the fMRI experiments, we propose a multi-layer feature fusion strategy to enhance the visual quality prediction capability of CNNs by combining features from different stages. The effectiveness of this strategy is validated on various CNN backbones on various IQA datasets (CLIVE, KonIQ-10k, and BID).

We choose SqueezeNet, ResNet, and EfficientNet as backbones to evaluate the adaptability of our feature fusion method across architectures with different complexities. SqueezeNet offers lightweight efficiency, ResNet introduces effective residual learning, and EfficientNet balances accuracy and computational cost through compound scaling. This selection enables a comprehensive assessment of our method’s generalizability. The detailed architecture of the backbone networks used in this study is as follows.

• (1)

  ResNet-50⁵⁴: ResNet introduces residual learning via identity shortcut connections, which enable the network to learn residual functions relative to the input. The architecture of ResNet-50 consists of 50 layers, organized into four stages, each containing a series of residual blocks. The first stage includes three residual blocks, the second stage has four, the third stage contains six, and the fourth stage comprises three. Each residual block employs skip connections that bypass one or more layers and are added to the output, facilitating the flow of gradients during backpropagation and enhancing the learning process.

• (2)

  SqueezeNet⁵³: SqueezeNet is a lightweight convolutional neural network designed to achieve AlexNet-level accuracy with significantly fewer parameters and reduced model size. Its architecture is built around “fire modules,” which consist of a squeeze convolution layer using 1 × 1 filters followed by an expand layer that combines 1 × 1 and 3 × 3 convolutions. This design efficiently reduces the number of parameters while maintaining representational capacity, making SqueezeNet suitable for resource-constrained environments without compromising performance.

• (3)

  EfficientNet-V2-M⁵⁵: EfficientNet-V2-M is a convolutional neural network architecture that improves upon the original EfficientNet by optimizing both model accuracy and training speed. It employs a compound scaling method that uniformly scales network depth, width, and resolution, combined with progressive learning techniques. The architecture consists of mobile inverted bottleneck convolution (MBConv) and fused-MBConv blocks, enabling efficient feature extraction with reduced computational cost. EfficientNet-V2-M achieves a strong balance between accuracy and efficiency, making it suitable for a wide range of vision tasks.

By selecting these three architectures, we cover a broad spectrum of model complexity—from lightweight to deep and efficiently scaled networks. This diversity allows us to comprehensively evaluate the generalizability and robustness of our proposed feature fusion strategy across different backbone designs and computational budgets, thereby strengthening the validity of our approach.

The CLIVE, KonIQ-10k, and BID datasets are selected to validate the effectiveness of the multi-layer feature fusion strategy. These three IQA datasets consist of images with authentic natural distortions encompassing diverse distortion types and varying scales, making them well suited for comprehensive evaluation of the proposed strategy’s effectiveness and generalizability. The details of the datasets used in this study are as follows.

• (1)

  CLIVE⁵⁷: the LIVE In the Wild Image Quality Challenge database is a comprehensive IQA dataset comprising 1,162 authentically distorted images captured from various mobile devices. Unlike traditional datasets with synthetic distortions, it features real-world distortions such as compression artifacts, noise, blur, and color distortions. The dataset’s subjective quality scores are collected via a large-scale crowdsourcing study involving over 8,100 unique observers, with each image rated by approximately 175 participants, yielding over 350,000 opinion scores.

• (2)

  KonIQ-10k⁴²: KonIQ-10k is a large-scale no-reference IQA dataset designed to support the training and evaluation of deep-learning models. It comprises 10,073 images, each annotated with subjective MOS collected from approximately 120 independent raters per image, totaling over 1.2 million ratings. The images exhibit a wide variety of authentic distortions common in real-world scenarios, including compression artifacts, noise, and blur, reflecting diverse everyday capture conditions. Sourced from the YFCC100M dataset, KonIQ-10k ensures content diversity and naturalness.

• (3)

  BID⁵⁸: the BID (Blurred Image Database) is a specialized no-reference IQA dataset focusing on blur distortions. It contains 586 images exhibiting various types of blur, including out-of-focus, simple-motion, and complex-motion blur, captured with consumer-grade digital cameras to reflect realistic photographic conditions. Each image has been rated by multiple human observers to obtain subjective MOS, which serve as ground truth for training and evaluating no-reference IQA models. This dataset is particularly valuable for developing algorithms aimed at assessing image quality where blur is the primary degradation.

We primarily use pooling or fully connected (FC) layers as markers to divide the network into five stages. Pooling layers are selected because they typically signify the end of a block, merging information and passing it to the next block.^27,107 In the absence of pooling or FC layers, the last layer of a block is chosen as the marker for stage division. Features from each stage during the information propagation process are extracted, adjusted to a uniform dimension through convolution layers, and fused to form multi-layer features, which replace the final layer’s features of the backbone. The features involved in multi-layer feature fusion across each path in each network are shown in Figure S12. These fused features are passed into a regressor to produce the final quality prediction score. The regressor of each backbone is designed to closely match the regressor structure used with its corresponding backbone in the ImageNet classification task, ensuring the final output dimension is 1.

All backbones are pre-trained on ImageNet.⁵⁶ All networks are independently fine-tuned and tested on the aforementioned IQA datasets using five-fold cross-validation. The dataset split and all training parameters remain consistent across backbones and datasets. Each network is trained for 30 epochs using a batch size of 64 on each dataset. Input images are first resized to 384 pixels and then center-cropped to 320 pixels, maintaining the aspect ratio. The initial learning rate is set to 0.0001, which decays by a factor of 0.5 every 5 epochs to facilitate stable convergence. All experiments are implemented using the PyTorch framework on a server equipped with an NVIDIA RTX 4090.

Lead contact

Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Guangtao Zhai (zhaiguangtao@sjtu.edu.cn).

Materials availability

The authors declare that the raw fMRI data in BIDS format will be made available on https://openneuro.org/datasets/ds006483.

Data and code availability

• •

  The authors declare that the main data supporting the findings of this study, as well as the raw fMRI data in BIDS format, are available on https://openneuro.org/datasets/ds006483.¹⁰⁸

• •

  The codes that support the findings of this study are available at figshare.¹⁰⁹

Document S1. Figures S1–S14, Tables S1–S5, supplemental methods, and supplemental notes

Table S6. The SRCC of visual quality predicting models

Table S7. The PLCC of visual quality predicting models

Document S2. Article plus supplemental information

• 1.Ericsson (2024). Mobile traffic update. https://www.ericsson.com/en/reports-and-papers/mobility-report/dataforecasts/mobile-traffic-update.
• 2.Shahid M., Rossholm A., Lövström B., Zepernick H.J. No-reference image and video quality assessment: a classification and review of recent approaches. EURASIP J. Image Video Process. 2014;2014:1–32. [Google Scholar]
• 3.Zhai G., Min X. Perceptual image quality assessment: a survey. Sci. China Inf. Sci. 2020;63:211301. [Google Scholar]
• 4.Chen Y., Wu K., Zhang Q. From QOS to QOE: A tutorial on video quality assessment. IEEE Commun. Surv. Tutorials. 2015;17:1126–1165. [Google Scholar]
• 5.Wang Z., Bovik A.C., Sheikh H.R., Simoncelli E.P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 2004;13:600–612. doi: 10.1109/tip.2003.819861. [DOI] [PubMed] [Google Scholar]
• 6.Sheikh H.R., Bovik A.C., Cormack L. No-reference quality assessment using natural scene statistics: Jpeg2000. IEEE Trans. Image Process. 2005;14:1918–1927. doi: 10.1109/tip.2005.854492. [DOI] [PubMed] [Google Scholar]
• 7.Mittal A., Soundararajan R., Bovik A.C. Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 2013;20:209–212. [Google Scholar]
• 8.Zhai G., Wu X., Yang X., Lin W., Zhang W. A psychovisual quality metric in free-energy principle. IEEE Trans. Image Process. 2012;21:41–52. doi: 10.1109/TIP.2011.2161092. [DOI] [PubMed] [Google Scholar]
• 9.Mantiuk R. High dynamic range imaging: towards the limits of the human visual perception. Forsch. Wiss. Rechnen. 2007;72:11–27. [Google Scholar]
• 10.Mozhaeva A., Vlasuyk I., Potashnikov A., Mazin V., Streeter L. 2024 39th International Conference on Image and Vision Computing New Zealand (IVCNZ) IEEE; 2024. Video quality metric compatible with PSNR considering recent knowledge of peripheral characteristics of human vision; pp. 1–6. [DOI] [Google Scholar]
• 11.Gu K., Zhai G., Yang X., Zhang W. Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia. 2015;17:50–63. [Google Scholar]
• 12.Mozhaeva A., Streeter L., Vlasuyk I., Potashnikov A. 2021 28th Conference of Open Innovations Association (FRUCT) IEEE; 2021. Full reference video quality assessment metric on base human visual system consistent with PSNR; pp. 309–315. [DOI] [Google Scholar]
• 13.Mazin V., Cree M.J., Streeter L., Nezhivleva K., Mozhaeva A. 2023 33rd Conference of Open Innovations Association (FRUCT) IEEE; 2023. Research and application of the adaptive model of the human visual system for improving the effectiveness of objective video quality metrics; pp. 192–197. [DOI] [Google Scholar]
• 14.Mantiuk R., Krawczyk G., Myszkowski K., Seidel H.P. Perception-motivated high dynamic range video encoding. ACM Trans. Graph. 2004;23:733–741. [Google Scholar]
• 15.Zhang W., Ma K., Yan J., Deng D., Wang Z. Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans. Circuits Syst. Video Technol. 2020;30:36–47. [Google Scholar]
• 16.Sun W., Duan H., Min X., Chen L., Zhai G. IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) 2022. Blind quality assessment for in-the-wild images via hierarchical feature fusion strategy; pp. 01–06. [DOI] [Google Scholar]
• 17.Zhu H., Li L., Wu J., Dong W., Shi G. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020. Metaiqa: Deep meta-learning for no-reference image quality assessment; pp. 14131–14140. [DOI] [Google Scholar]
• 18.Wu H., Zhang Z., Zhang E., Chen C., Liao L., Wang A., Xu K., Li C., Hou J., Zhai G., et al. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024. Q-instruct: Improving low-level visual abilities for multi-modality foundation models; pp. 25490–25500. [DOI] [Google Scholar]
• 19.Gauthier I., Tarr M.J., Anderson A.W., Skudlarski P., Gore J.C. Activation of the middle fusiform ‘face area’ increases with expertise in recognizing novel objects. Nat. Neurosci. 1999;2:568–573. doi: 10.1038/9224. [DOI] [PubMed] [Google Scholar]
• 20.Cichy R.M., Pantazis D., Oliva A. Resolving human object recognition in space and time. Nat. Neurosci. 2014;17:455–462. doi: 10.1038/nn.3635. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 21.Yamins D.L.K., Hong H., Cadieu C.F., Solomon E.A., Seibert D., DiCarlo J.J. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. USA. 2014;111:8619–8624. doi: 10.1073/pnas.1403112111. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 22.Grill-Spector K., Weiner K.S. The functional architecture of the ventral temporal cortex and its role in categorization. Nat. Rev. Neurosci. 2014;15:536–548. doi: 10.1038/nrn3747. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 23.Huth A.G., De Heer W.A., Griffiths T.L., Theunissen F.E., Gallant J.L. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature. 2016;532:453–458. doi: 10.1038/nature17637. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 24.Huth A.G., Nishimoto S., Vu A.T., Gallant J.L. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron. 2012;76:1210–1224. doi: 10.1016/j.neuron.2012.10.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 25.Cichy R.M., Khosla A., Pantazis D., Torralba A., Oliva A. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci. Rep. 2016;6 doi: 10.1038/srep27755. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 26.Eickenberg M., Gramfort A., Varoquaux G., Thirion B. Seeing it all: Convolutional network layers map the function of the human visual system. Neuroimage. 2017;152:184–194. doi: 10.1016/j.neuroimage.2016.10.001. [DOI] [PubMed] [Google Scholar]
• 27.Xu Y., Vaziri-Pashkam M. Limits to visual representational correspondence between convolutional neural networks and the human brain. Nat. Commun. 2021;12:2065. doi: 10.1038/s41467-021-22244-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 28.Zhang Z., Wu W., Sun W., Tu D., Lu W., Min X., Chen Y., Zhai G. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023. MD-VQA: Multi-dimensional quality assessment for UGC live videos; pp. 1746–1755. [DOI] [Google Scholar]
• 29.Sun W., Min X., Lu W., Zhai G. ACM International Conference on Multimedia. 2022. A deep learning based no-reference quality assessment model for UGC videos; pp. 856–865. [DOI] [Google Scholar]
• 30.Yang L., Duan H., Teng L., Zhu Y., Liu X., Hu M., Min X., Zhai G., Le Callet P. IEEE International Conference on Image Processing (ICIP) 2024. Aigcoiqa2024: Perceptual quality assessment of AI generated omnidirectional images; pp. 1239–1245. [DOI] [Google Scholar]
• 31.Sun W., Min X., Zhai G., Gu K., Duan H., Ma S. Mc360iqa: A multi-channel CNN for blind 360-degree image quality assessment. IEEE J. Sel. Top. Signal Process. 2020;14:64–77. [Google Scholar]
• 32.Duan H., Zhai G., Min X., Zhu Y., Fang Y., Yang X. IEEE International Symposium on Circuits and Systems (ISCAS) 2018. Perceptual quality assessment of omnidirectional images; pp. 1–5. [DOI] [Google Scholar]
• 33.Li C., Zhang Z., Wu H., Sun W., Min X., Liu X., Zhai G., Lin W. Agiqa-3k: An open database for AI-generated image quality assessment. IEEE Trans. Circuits Syst. Video Technol. 2024;34:6833–6846. [Google Scholar]
• 34.Liu X., Min X., Zhai G., Li C., Kou T., Sun W., Wu H., Gao Y., Cao Y., Zhang Z., et al. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024. Ntire 2024 quality assessment of AI-generated content challenge; pp. 6337–6362. [DOI] [Google Scholar]
• 35.Zhang Z., Li C., Sun W., Liu X., Min X., Zhai G. IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2023. A perceptual quality assessment exploration for AIGC images; pp. 440–445. [DOI] [Google Scholar]
• 36.Qiu J., Zhu Y., Shi X., Wenzel F., Tang Z., Zhao D., Li B., Li M. Benchmarking robustness of multimodal image-text models under distribution shift. Arxiv. 2023 doi: 10.48550/arXiv.2212.08044. Preprint at. [DOI] [Google Scholar]
• 37.Bosse S., Brunnström K., Arndt S., Martini M.G., Ramzan N., Engelke U. A common framework for the evaluation of psychophysiological visual quality assessment. Qual. User Exp. 2019;4:1–10. [Google Scholar]
• 38.Arndt S., Antons J.N., Schleicher R., Möller S., Curio G. Using electroencephalography to measure perceived video quality. IEEE J. Sel. Top. Signal Process. 2014;8:366–376. [Google Scholar]
• 39.Arndt S., Antons J.N., Schleicher R., Moller S., Scholler S., Curio G. IEEE International Symposium on Multimedia. 2011. A physiological approach to determine video quality; pp. 518–523. [DOI] [Google Scholar]
• 40.Liu X., Tao X., Xu M., Zhan Y., Lu J. An EEG-based study on perception of video distortion under various content motion conditions. IEEE Trans. Multimedia. 2020;22:949–960. [Google Scholar]
• 41.Hu S., Duan Y., Tao X., Li G.Y., Lu J., Liu G., Zheng Z., Pan C. Brain-inspired image perceptual quality assessment based on EEG: A QOE perspective. IEEE Trans. Pattern Anal. Mach. Intell. 2024;46:8424–8441. doi: 10.1109/TPAMI.2024.3408684. [DOI] [PubMed] [Google Scholar]
• 42.Hosu V., Lin H., Sziranyi T., Saupe D. Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Trans. Image Process. 2020;29:4041–4056. doi: 10.1109/TIP.2020.2967829. [DOI] [PubMed] [Google Scholar]
• 43.Ogawa S., Lee T.M., Kay A.R., Tank D.W. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. USA. 1990;87:9868–9872. doi: 10.1073/pnas.87.24.9868. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 44.Kriegeskorte N., Mur M., Bandettini P. Representational similarity analysis-connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2008;2:4. doi: 10.3389/neuro.06.004.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 45.Kriegeskorte N., Formisano E., Sorger B., Goebel R. Individual faces elicit distinct response patterns in human anterior temporal cortex. Proc. Natl. Acad. Sci. USA. 2007;104:20600–20605. doi: 10.1073/pnas.0705654104. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 46.Nestor A., Plaut D.C., Behrmann M. Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis. Proc. Natl. Acad. Sci. USA. 2011;108:9998–10003. doi: 10.1073/pnas.1102433108. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 47.Friston K., Kilner J., Harrison L. A free energy principle for the brain. J. Physiol. Paris. 2006;100:70–87. doi: 10.1016/j.jphysparis.2006.10.001. [DOI] [PubMed] [Google Scholar]
• 48.Friston K. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 2010;11:127–138. doi: 10.1038/nrn2787. [DOI] [PubMed] [Google Scholar]
• 49.Liu Z., Lin Y., Cao Y., Hu H., Wei Y., Zhang Z., Lin S., Guo B. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2021. Swin transformer: Hierarchical vision transformer using shifted windows; pp. 10012–10022. [DOI] [Google Scholar]
• 50.Gong Z., Bao G., Zhang Q., Wan Z., Miao D., Wang S., Zhu L., Wang C., Xu R., Hu L., et al. Neuroclips: Towards high-fidelity and smooth fMRI-to-video reconstruction. Adv. Neural Inf. Process. Syst. 2024;37:51655–51683. [Google Scholar]
• 51.Güçlü U., van Gerven M.A.J. Increasingly complex representations of natural movies across the dorsal stream are shared between subjects. Neuroimage. 2017;145:329–336. doi: 10.1016/j.neuroimage.2015.12.036. [DOI] [PubMed] [Google Scholar]
• 52.Khaligh-Razavi S.M., Kriegeskorte N. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS Comput. Biol. 2014;10 doi: 10.1371/journal.pcbi.1003915. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 53.Iandola F.N., Han S., Moskewicz M.W., Ashraf K., Dally W.J., Keutzer K. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5 mb model size. ArXiv. 2016 doi: 10.48550/arXiv.1602.07360. Preprint at. [DOI] [Google Scholar]
• 54.He K., Zhang X., Ren S., Sun J. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2016. Deep residual learning for image recognition; pp. 770–778. [DOI] [Google Scholar]
• 55.Tan M., Le Q. International Conference on Machine Learning (ICML) PMLR; 2021. Efficientnetv2: Smaller models and faster training; pp. 10096–10106. [Google Scholar]
• 56.Deng J., Dong W., Socher R., Li L.J., Li K., Fei-Fei L. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2009. Imagenet: A large-scale hierarchical image database; pp. 248–255. [DOI] [Google Scholar]
• 57.Ghadiyaram D., Bovik A.C. Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 2016;25:372–387. doi: 10.1109/TIP.2015.2500021. [DOI] [PubMed] [Google Scholar]
• 58.Ciancio A., Targino da Costa A.L.N., da Silva E.A., Said A., Samadani R., Obrador P. No-reference blur assessment of digital pictures based on multifeature classifiers. IEEE Trans. Image Process. 2010;20:64–75. doi: 10.1109/TIP.2010.2053549. [DOI] [PubMed] [Google Scholar]
• 59.Chen F., Fu H., Yu H., Chu Y. Using HVS dual-pathway and contrast sensitivity to blindly assess image quality. Sensors. 2023;23:4974. doi: 10.3390/s23104974. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 60.Kang L., Ye P., Li Y., Doermann D. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2014. Convolutional neural networks for no-reference image quality assessment; pp. 1733–1740. [DOI] [Google Scholar]
• 61.Bosse S., Maniry D., Müller K.R., Wiegand T., Samek W. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 2017;27:206–219. doi: 10.1109/TIP.2017.2760518. [DOI] [PubMed] [Google Scholar]
• 62.Li D., Jiang T., Lin W., Jiang M. Which has better visual quality: The clear blue sky or a blurry animal? IEEE Trans. Multimedia. 2019;21:1221–1234. [Google Scholar]
• 63.Su S., Yan Q., Zhu Y., Zhang C., Ge X., Sun J., Zhang Y. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020. Blindly assess image quality in the wild guided by a self-adaptive hyper network; pp. 3667–3676. [DOI] [Google Scholar]
• 64.Zhang Z., Wu H., Zhang E., Zhai G., Lin W. Q-bench ++: A benchmark for multi-modal foundation models on low-level vision from single images to pairs. IEEE Trans. Pattern Anal. Mach. Intell. 2024;46:10404–10418. doi: 10.1109/TPAMI.2024.3445770. [DOI] [PubMed] [Google Scholar]
• 65.Zhang L., Zhang L., Mou X., Zhang D. Fsim: A feature similarity index for image quality assessment. IEEE Trans. Image Process. 2011;20:2378–2386. doi: 10.1109/TIP.2011.2109730. [DOI] [PubMed] [Google Scholar]
• 66.Tops M., Boksem M.A.S. A potential role of the inferior frontal gyrus and anterior insula in cognitive control, brain rhythms, and event-related potentials. Front. Psychol. 2011;2:330. doi: 10.3389/fpsyg.2011.00330. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 67.Hartwigsen G., Neef N.E., Camilleri J.A., Margulies D.S., Eickhoff S.B. Functional segregation of the right inferior frontal gyrus: evidence from coactivation-based parcellation. Cereb. Cortex. 2019;29:1532–1546. doi: 10.1093/cercor/bhy049. [DOI] [PubMed] [Google Scholar]
• 68.Hong H., Yamins D.L.K., Majaj N.J., DiCarlo J.J. Explicit information for category-orthogonal object properties increases along the ventral stream. Nat. Neurosci. 2016;19:613–622. doi: 10.1038/nn.4247. [DOI] [PubMed] [Google Scholar]
• 69.DiCarlo J.J., Zoccolan D., Rust N.C. How does the brain solve visual object recognition? Neuron. 2012;73:415–434. doi: 10.1016/j.neuron.2012.01.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 70.Felleman D.J., Van Essen D.C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex. 1991;1:1–47. doi: 10.1093/cercor/1.1.1-a. [DOI] [PubMed] [Google Scholar]
• 71.Greenberg D.L., Rice H.J., Cooper J.J., Cabeza R., Rubin D.C., LaBar K.S. Co-activation of the amygdala, hippocampus and inferior frontal gyrus during autobiographical memory retrieval. Neuropsychologia. 2005;43:659–674. doi: 10.1016/j.neuropsychologia.2004.09.002. [DOI] [PubMed] [Google Scholar]
• 72.Fletcher P.C., Henson R.N. Frontal lobes and human memory: insights from functional neuroimaging. Brain. 2001;124:849–881. doi: 10.1093/brain/124.5.849. [DOI] [PubMed] [Google Scholar]
• 73.Wagner A.D., Paré-Blagoev E.J., Clark J., Poldrack R.A. Recovering meaning: left prefrontal cortex guides controlled semantic retrieval. Neuron. 2001;31:329–338. doi: 10.1016/s0896-6273(01)00359-2. [DOI] [PubMed] [Google Scholar]
• 74.Rahate A., Mandaokar S., Chandel P., Walambe R., Ramanna S., Kotecha K. Employing multimodal co-learning to evaluate the robustness of sensor fusion for industry 5.0 tasks. Soft Comput. 2023;27:4139–4155. [Google Scholar]
• 75.Guo R., Wei J., Sun L., Yu B., Chang G., Liu D., Zhang S., Yao Z., Xu M., Bu L. A survey on image-text multimodal models. arXiv. 2023 doi: 10.48550/arXiv.2309.15857. Preprint at. [DOI] [PubMed] [Google Scholar]
• 76.Long Z., Killick G., McCreadie R., Camarasa G.A. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024. Multiway-adapter: Adapting multimodal large language models for scalable image-text retrieval; pp. 6580–6584. [DOI] [Google Scholar]
• 77.Sweller J. Cognitive load during problem solving: Effects on learning. Cogn. Sci. 1988;12:257–285. [Google Scholar]
• 78.De Jong T. Cognitive load theory, educational research, and instructional design: Some food for thought. Instr. Sci. 2010;38:105–134. [Google Scholar]
• 79.Chen O., Paas F., Sweller J. A cognitive load theory approach to defining and measuring task complexity through element interactivity. Educ. Psychol. Rev. 2023;35:63. [Google Scholar]
• 80.Winding M., Pedigo B.D., Barnes C.L., Patsolic H.G., Park Y., Kazimiers T., Fushiki A., Andrade I.V., Khandelwal A., Valdes-Aleman J., et al. The connectome of an insect brain. Science. 2023;379 doi: 10.1126/science.add9330. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 81.Chang N., Pyles J.A., Marcus A., Gupta A., Tarr M.J., Aminoff E.M. Bold5000, a public fMRI dataset while viewing 5000 visual images. Sci. Data. 2019;6:49. doi: 10.1038/s41597-019-0052-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 82.Allen E.J., St-Yves G., Wu Y., Breedlove J.L., Prince J.S., Dowdle L.T., Nau M., Caron B., Pestilli F., Charest I., et al. A massive 7t fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nat. Neurosci. 2022;25:116–126. doi: 10.1038/s41593-021-00962-x. [DOI] [PubMed] [Google Scholar]
• 83.Holladay J.T. Visual acuity measurements. J. Cataract Refract. Surg. 2004;30:287–290. doi: 10.1016/j.jcrs.2004.01.014. [DOI] [PubMed] [Google Scholar]
• 84.Thomee B., Shamma D.A., Friedland G., Elizalde B., Ni K., Poland D., Borth D., Li L.J. Yfcc100m: The new data in multimedia research. Commun. ACM. 2016;59:64–73. [Google Scholar]
• 85.Gao Y., Min X., Zhai G. Exploring image quality assessment from a new perspective: Pupil size. arXiv. 2025 doi: 10.48550/arXiv.2505.13841. Preprint at. [DOI] [Google Scholar]
• 86.International Telecommunication Union (ITU) ITU-R; 2012. Methodology for the Subjective Assessment of the Quality of Television Pictures. Recommendation ITU-R BT.500-14 ITU Radiocommunication Sector.https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.500-14-201206-I!!PDF-E.pdf [Google Scholar]
• 87.Esteban O., Markiewicz C.J., Blair R.W., Moodie C.A., Isik A.I., Erramuzpe A., Kent J.D., Goncalves M., DuPre E., Snyder M., et al. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat. Methods. 2019;16:111–116. doi: 10.1038/s41592-018-0235-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 88.Gorgolewski K., Burns C.D., Madison C., Clark D., Halchenko Y.O., Waskom M.L., Ghosh S.S. Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python. Front. Neuroinform. 2011;5:13. doi: 10.3389/fninf.2011.00013. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 89.Dale A.M., Fischl B., Sereno M.I. Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage. 1999;9:179–194. doi: 10.1006/nimg.1998.0395. [DOI] [PubMed] [Google Scholar]
• 90.Nieto-Castanon A. Hilbert Press; 2020. Handbook of Functional Connectivity Magnetic Resonance Imaging Methods in CONN. [Google Scholar]
• 91.Friston K.J., Williams S., Howard R., Frackowiak R.S., Turner R. Movement-related effects in fMRI time-series. Magn. Reson. Med. 1996;35:346–355. doi: 10.1002/mrm.1910350312. [DOI] [PubMed] [Google Scholar]
• 92.Power J.D., Mitra A., Laumann T.O., Snyder A.Z., Schlaggar B.L., Petersen S.E. Methods to detect, characterize, and remove motion artifact in resting state fMRI. Neuroimage. 2014;84:320–341. doi: 10.1016/j.neuroimage.2013.08.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 93.Hallquist M.N., Hwang K., Luna B. The nuisance of nuisance regression: spectral misspecification in a common approach to resting-state fMRI preprocessing reintroduces noise and obscures functional connectivity. Neuroimage. 2013;82:208–225. doi: 10.1016/j.neuroimage.2013.05.116. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 94.Behzadi Y., Restom K., Liau J., Liu T.T. A component based noise correction method (compcor) for bold and perfusion based fmri. Neuroimage. 2007;37:90–101. doi: 10.1016/j.neuroimage.2007.04.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 95.Chai X.J., Castañón A.N., Ongür D., Whitfield-Gabrieli S. Anticorrelations in resting state networks without global signal regression. Neuroimage. 2012;59:1420–1428. doi: 10.1016/j.neuroimage.2011.08.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 96.Nieto-Castanon A. Preparing fMRI data for statistical analysis. arXiv. 2022 doi: 10.48550/arXiv.2210.13564. Preprint at. [DOI] [Google Scholar]
• 97.Penny W.D., Friston K.J., Ashburner J.T., Kiebel S.J., Nichols T.E. Elsevier; 2011. Statistical Parametric Mapping: The Analysis of Functional Brain Images. [Google Scholar]
• 98.Whitfield-Gabrieli S., Nieto-Castanon A. Conn: a functional connectivity toolbox for correlated and anticorrelated brain networks. Brain Connect. 2012;2:125–141. doi: 10.1089/brain.2012.0073. [DOI] [PubMed] [Google Scholar]
• 99.Tzourio-Mazoyer N., Landeau B., Papathanassiou D., Crivello F., Etard O., Delcroix N., Mazoyer B., Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage. 2002;15:273–289. doi: 10.1006/nimg.2001.0978. [DOI] [PubMed] [Google Scholar]
• 100.Friston K.J., Buechel C., Fink G.R., Morris J., Rolls E., Dolan R.J. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage. 1997;6:218–229. doi: 10.1006/nimg.1997.0291. [DOI] [PubMed] [Google Scholar]
• 101.McLaren D.G., Ries M.L., Xu G., Johnson S.C. A generalized form of context-dependent psychophysiological interactions (gPPI): a comparison to standard approaches. Neuroimage. 2012;61:1277–1286. doi: 10.1016/j.neuroimage.2012.03.068. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 102.Worsley K.J., Marrett S., Neelin P., Vandal A.C., Friston K.J., Evans A.C. A unified statistical approach for determining significant signals in images of cerebral activation. Hum. Brain Mapp. 1996;4:58–73. doi: 10.1002/(SICI)1097-0193(1996)4:1<58::AID-HBM4>3.0.CO;2-O. [DOI] [PubMed] [Google Scholar]
• 103.Tarhan L., Konkle T. Reliability-based voxel selection. Neuroimage. 2020;207 doi: 10.1016/j.neuroimage.2019.116350. [DOI] [PubMed] [Google Scholar]
• 104.Cadieu C.F., Hong H., Yamins D.L.K., Pinto N., Ardila D., Solomon E.A., Majaj N.J., DiCarlo J.J. Deep neural networks rival the representation of primate it cortex for core visual object recognition. PLoS Comput. Biol. 2014;10 doi: 10.1371/journal.pcbi.1003963. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 105.Nili H., Wingfield C., Walther A., Su L., Marslen-Wilson W., Kriegeskorte N. A toolbox for representational similarity analysis. PLoS Comput. Biol. 2014;10 doi: 10.1371/journal.pcbi.1003553. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 106.Walther A., Nili H., Ejaz N., Alink A., Kriegeskorte N., Diedrichsen J. Reliability of dissimilarity measures for multi-voxel pattern analysis. Neuroimage. 2016;137:188–200. doi: 10.1016/j.neuroimage.2015.12.012. [DOI] [PubMed] [Google Scholar]
• 107.O’Connell T.P., Chun M.M. Predicting eye movement patterns from fmri responses to natural scenes. Nat. Commun. 2018;9:5159. doi: 10.1038/s41467-018-07471-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
• 108.Yiming, Z., Yitong, C., Ying, H., Xu, H., Zhenhui, X., Xingrui, W., Yan, Z., Xiongkuo, M., and Guangtao, Z. (2025). Neural mechanisms of visual quality perception and adaptability in visual pathway. OpenNeuro [dataset]. doi: 10.18112/openneuro.ds006483.v1.0.0 [DOI]
• 109.Yiming, Z., Yitong, C., Ying, H., Xu, H., Zhenhui, X., Xingrui, W., Yan, Z., Xiongkuo, M., and Guangtao, Z. (2025). fMRI for VQA code. figshare. 10.6084/m9.figshare.29609510.v1. [DOI]

Supplementary Materials

Data Availability Statement

• •

  The authors declare that the main data supporting the findings of this study, as well as the raw fMRI data in BIDS format, are available on https://openneuro.org/datasets/ds006483.¹⁰⁸

• •

  The codes that support the findings of this study are available at figshare.¹⁰⁹