Identifying specific colors with a color sensor is not as simple as pointing the sensor at a color patch and taking a reading. There are many factors such as distance, field of view and ambient lighting conditions that can affect readings. Adafruit forum member systembolaget has developed a system capable of quickly and reliably identifying many different colors and has put together an excellent tutorial on the subject.
See the video and github repository link below:
