Flow experience is a critical lens for investigating the role of website design on consumer satisfaction in online shopping. Since flow is a latent variable that cannot be directly observed, most studies use a questionnaire survey with self-reported data as the predominant method of data gathering. Although many researchers believe that the self-reported approach as a sole measurement method is subject to common method bias (CMB), scholars have different interpretations of this effect. This study designed an experiment with flow experience as a mediator to investigate the relationship between website quality and user satisfaction from a behavioral survey and an electroencephalography (EEG)-based neuroscience method. By adopting Harman's one-factor test and the marker variable technique, the results indicate that the EEG neuroscience method reduces CMB and CMB is not an important issue when considering EEG results.