OK, so you have a project and now you want to power it off of a battery, how do you choose the best setup?

The two easiest cases are the extremes:

• Is your project very power-hungry? Projectors, large sound systems, and motorized projects all draw on the order of amps of current! You'll want to go with lantern cells (one-time use) or lead acid batteries (rechargable). If you are planning to be somewhat 'abusive' to the battery (heavy-usage, running it down all the way) you may want to look at "marine deep cycle" batteries.
• Is your project super-small, like an inch on each side? You're going to have to go with a lithium coin cell (one time use) or little lithium-polymer cells like the ones used for tiny RC planes.

Here are some other very popular cases:

• Do you need to make a lot of these things? Go with inexpensive alkaline batteries in popular sizes.
• Need to be user-servicable? 9V or AA size batteries are universal!
• ~5V input necessary? 3 Alkaline (4.5V) or 4 NiMH cells (4.8V) will get you pretty close - check your circuit to see if it'll run at these slightly lower voltages
  
• Making a 'rechargeable battery pack'? Use a battery holder from your local hobby/electronics/repair shop and stick with NiMH batteries, then recharge them with a high quality charger.
• Want to replace alkalines with rechargeables? Test to make sure that the lower voltage won't make the device unhappy.
• Need to stack batteries? Remember to stack batteries only if they have matching C and Ah capability, if you stack a 9V and a AA to make 10.5V the 9V will drain in 1/10th the time leaving you with 1.5V.
• Want your rechargable batteries to last a long time? Use a high-quality charger that has sensors to maintain proper charging and trickle charging. A cheap charger will kill off your cells.