Non-neural approaches to argument mining (AM) are often pipelined and require heavy feature-engineering. In this paper, we propose a neural end-to-end approach to AM which is based on dependency parsing, in contrast to the current state-of-the-art which relies on rela- tion extraction. Our biaffine AM dependency parser significantly outperforms the state-of- the-art, performing at F1 = 73.5% for com- ponent identification and F1 = 46.4% for re- lation identification. One of the advantages of treating AM as biaffine dependency parsing is the simple neural architecture that results. The idea of treating AM as dependency parsing is not new, but has previously been abandoned as it was lagging far behind the state-of-the-art. In a thorough analysis, we investigate the fac- tors that contribute to the success of our model: the biaffine model itself, our representation for the dependency structure of arguments, differ- ent encoders in the biaffine model, and syntac- tic information additionally fed to the model. Our work demonstrates that dependency pars- ing for AM, an overlooked idea from the past, deserves more attention in the future.