First, decide what you'll be using for external storage. I picked a 1 terabyte USB hard drive from Seagate, because I had it laying around as a spare backup drive. You can probably find the equivalent on Amazon for $60-70.
The Raspberry Pi doesn't have enough power to run most external hard drives or SSDs, so unless you have a drive with its own power supply, you'll probably need a powered USB hub.
I picked up a j5create JUH340 4-port hub a while ago at a nearby Best Buy for about $25, and so far haven't had any problems with it. Adafruit offers a USB 2.0 powered hub which has been tested with the Pi and should work well. You'll want to make sure you get something that supports USB 2.0.
It won't offer all the benefits of a "real" drive, but if you've got a decent-sized USB thumb drive available, you can use it instead. This probably won't require a powered hub.
Unfortunately, no matter what you do, the Pi has to boot off an SD (microSD, in the case of the B+/Pi 2) card. Assuming you already have a Pi up and running, your existing card should work just fine, but you might want to use a spare just in case.
The card should have a standard Raspbian install already present. I started with Adafruit's 4GB SD card with a pre-installed Raspbian image, but you should be able to use just about any card with an existing install.
If needed, you can follow our guide on preparing an SD card for the Pi.
You can certainly set up the SD card and external drive from another machine, but we'll do everything here from the command line on the Pi. You shouldn't need another computer as long as your Pi already boots into Raspbian.
