Even when connecting a LAN (Network) cable to your console, downloads still stay pretty slow so the conclusion there is that it might not have anything to do with your WiFi network or network at all. You might have noticed that when downloading content from the PSN store or Xbox Live marketplace downloads can sometimes be really slow.
