Gaming experience has been suggested to lead to performance enhancements in a wide variety of working memory tasks. Previous studies have, however, mostly focused on adult expert gamers and have not included measurements of both behavioral performance and brain activity. In the current study, 167 adolescents and young adults (aged 13–24 years) with different amounts of gaming experience performed an n-back working memory task with vowels, with the sensory modality of the vowel stream switching between audition and vision at random intervals. We studied the relationship between self-reported daily gaming activity, working memory (n-back) task performance and related brain activity measured using functional magnetic resonance imaging (fMRI). The results revealed that the extent of daily gaming activity was related to enhancements in both performance accuracy and speed during the most demanding (2-back) level of the working memory task. This improved working memory performance was accompanied by enhanced recruitment of a fronto-parietal cortical network, especially the dorsolateral prefrontal cortex. In contrast, during the less demanding (1-back) level of the task, gaming was associated with decreased activity in the same cortical regions. Our results suggest that a greater degree of daily gaming experience is associated with better working memory functioning and task difficulty-dependent modulation in fronto-parietal brain activity already in adolescence and even when non-expert gamers are studied. The direction of causality within this association cannot be inferred with certainty due to the correlational nature of the current study.