You can easily install an open source toolchain to get started developing right on the Chumby Hacker board!
Step one is to download the following file http://files.chumby.com/hacks/falconwing_toolchain.sh (20M) and save it onto any USB key. Then insert that USB into one of the CHB's USB ports.
Then cd into /mnt/storage and run the ./falconwing_toolchain.sh script to install the toolchain.
Please save a backup of the falconwing_toolchain.sh file somewhere so you dont end up having to re-download it (Chumby is kindly hosting this file but its very large!)