It means the message didn't successfully send. Perhaps it was an MMS pic message that was sent while no data was enabled and never attempted to send again. You can long press most messages to find out its details, delete it, or resend.
The caution shows on one of my contacts, but when I open that contact's text messages I don't see the caution sign, how would I find the text that didn't go through so I can get rid of it?
Only one of my contacts have the caution sign, but when I open to text her, I don't see the caution sign, how would I find out which text didn't go through and delete it?
I know on Message+ once you are in the contacts text screen, swipe from the far right of the screen to the left, and that will bring up time stamps, and the caution sign on the text that did not send. You might have to scroll up until you find the message that did not send. Once you find it, press and hold on that message until "delete message" pops up.