The present study compared activations to prototype, nonprototype, nonphonemic, and cross-category vowel pairs during vowel discrimination, category discrimination, 2-back, and visual tasks. Our results support previous findings that areas of the superior temporal gyrus (STG) are sensitive to the speech-level difference between prototype vs. nonprototype and phonemic vs. nonphonemic vowels. Further, consistent with previous studies, we found enhanced activations in anterior-posterior STG and inferior parietal lobule (IPL) during the vowel discrimination and 2-back tasks, respectively. Unlike the vowel discrimination task, the category discrimination task was associated with strong IPL activations. Our results provide evidence that activations in STG and IPL strongly depend on whether the task requires analysis of detailed acoustical information or operations on categorical representations. Based on previous studies investigating activations during categorical pitch and spatial tasks, we argue that this distinction is probably not specific to speech.