Download Managers

I’ve used Fresh Download for years - works well, simple user interface and ad - spyware free !

Can someone please tell me why one download manager would be faster / slower than another ? Wouldn’t your internet connection be the deciding factor ?

Thanks.

PS - I remember reading a comparison of d/l managers by the author of Spybot, I think the openoffice download page has the link.