The present study aimed to examine the impact of the level of physical activity on prefrontal cortex activation in older adults during single- and dual-task walking. Thirty physically inactive and 36 active older adults (60–85 years old) performed six 2-min tasks on a treadmill: two static cognitive tasks, two single-task walking tests, and two dual-task walking tests. Hemodynamics at the level of the prefrontal cortex were measured continuously using functional near-infrared spectroscopy to evaluate cortical activation. The perceived difficulty of the task, cognitive performance, and gait parameters were also measured. During the walking tasks, the level of prefrontal cortex activation, the perceived difficulty of the task, cognitive performance, and motor parameters were not significantly different between active and inactive older adults. This unchanged activation with physical activity was likely the consequence of a similar motor and cognitive load and cardiorespiratory fitness in both active and inactive older adults.