Introduction: Although virtual reality (VR) is increasingly deployed as a tool for education, little is known about how learning modality influences cognitive processing or whether wearable neuroimaging technologies can accurately classify cognitive load in this context.Methods: In this study, 21 first-year medical students were randomly assigned to learn cardiac anatomy using either traditional 2D materials (n = 10) or immersive 3D VR (n = 11). Prefrontal hemodynamic activity was continuously monitored using functional near-infrared spectroscopy (fNIRS), while cognitive load was assessed across three learning periods using the Klepsch questionnaire. Learning outcomes were evaluated through pre- and post-learning assessments.Results: Results demonstrated that both groups had comparable knowledge gains and reported similar levels of intrinsic, extraneous, and germane cognitive load. In contrast, fNIRS revealed modality-related differences in neural dynamics: learners using 2D materials showed significantly longer time-to-peak oxygenation across multiple channels, with large effect sizes (Cohen’s d = 1.18–1.66). Deep learning classifiers successfully distinguished high from low cognitive load using fNIRS features under leave-one-subject-out validation, with extraneous load achieving the strongest performance (91% F1-score and 92% accuracy), followed by intrinsic cognitive load (63% F1-score and 73% accuracy) and total cognitive load (60% F1-score and 78% accuracy).Discussion: Together, these findings highlight the value of combining neurophysiological and subjective measures and demonstrate the feasibility of fNIRS-based cognitive load classification, offering a foundation for future adaptive and learner-responsive instructional systems in medical education.