You want 4 to 6volts max. Current depends on what type motors you're working with. C and D cans should be fine with 5 or so amps, cobalt motor need a lot more, 7 to 10 amps should do. If you are thinking about using some commercially available unit designed for car audio use, which it appears you are, they usually do not work out real well, most don't drop down to a low enough voltage and are not well regulated, and can have a large ripple component (real bad for slot motors). So unless you are planning on a LAB GRADE variable power supply, stick to one of the units designed for slot racing, you will be happier in the long run. By the way, the only lab grade unit I currently know is available is over twice the price of the made for slot racing ones. Of course, you can design and build your own if you are up to the challenge, the one I've been using for the last 18 years, still works perfectly, but is a little light on amps for todays cobalt. Maybe I'll build a new one.
Improving performance one hazardous material at a time.