Gemma

Remove two of the opposite legs from the button.

Attach the button to the back of the Gemma and solder.

Beginning with the power and ground wires this time, twist them together, attach to the Gemma, and solder.

The power wires connect to Vout. 

The ground wires connect to GND. Note that one of the legs for the button is also on the Gemma's GND pad. 

Test the whole thing! Give the button both short and long presses to see the modes.