When it comes to doing things quickly, the faster your computer, the more efficient your tasks. Your computer will run faster if you don’t have wallpaper loaded.
Adding a wallpaper image on top of everything else your computer does will slow everything down a little bit. The speed may not be effected a lot, but your computer will be slower.
More important, wallpaper on your desktop is something that is rarely seen while you use your computer. You probably can’t see your wallpaper right now. There are plenty of screen savers that will load a whole list of pictures for you when your computer is idle. You will enjoy them more when you don’t have a program running right over top of them.