How does brain activity in distributed semantic brain networks evolve over time, and how do these regions interact to retrieve the meaning of words? We compared spatiotemporal brain dynamics between visual lexical and semantic decision tasks (LD and SD), analysing whole-cortex evoked responses and spectral functional connectivity (coherence) in source-estimated electroencephalography and magnetoencephalography (EEG and MEG) recordings. Our evoked analysis revealed generally larger activation for SD compared to LD, starting in primary visual area (PVA) and angular gyrus (AG), followed by left posterior temporal cortex (PTC) and left anterior temporal lobe (ATL). The earliest activation effects in ATL were significantly left-lateralised. Our functional connectivity results showed significant connectivity between left and right ATL, PTC and right ATL in an early time window, as well as between left ATL and IFG in a later time window. The connectivity of AG was comparatively sparse. We quantified the limited spatial resolution of our source estimates via a leakage index for careful interpretation of our results. Our findings suggest that the different demands on semantic information retrieval in lexical and semantic decision tasks first modulate visual and attentional processes, then multimodal semantic information retrieval in the ATLs and finally control regions (PTC and IFG) in order to extract task-relevant semantic features for response selection. Whilst our evoked analysis suggests a dominance of left ATL for semantic processing, our functional connectivity analysis also revealed significant involvement of right ATL in the more demanding semantic task. Our findings demonstrate the complementarity of evoked and functional connectivity analysis, as well as the importance of dynamic information for both types of analyses.