Yes, it affects your tone, especially if you're using a cheap bass (like me).
I've had to screw my pickups to the lowest height that I can (I'm afraid I'll break something if I screw it any tighter), and I still have to have pretty high action for the E and G strings because MY PICKUP IS THE WRONG SIZE (bloody Samick, but it's cheap, so..). If I lower the action the bass just sounds SUPER crappy.
Luckily I like high action.
But I still want it a little bit lower la :?