Of course, the naive description as many heard it (which includes Newton having an apple fall on his head) is not true. There also does not exist any known source where Newton discusses anything about apples, and how they relate to his thoughts on gravitation.
However, there are multiple secondary sources, providing accounts of a related 'apple incident', some event involving an apple falling down to Earth, which helped Newton in his thought process which would eventually lead him to the law of universal gravitation. It is, of course, difficult to investigate the veracity of such anecdotes, but it's the best I can do. The first account is by William Stukeley, who wrote the following:
we went into the garden, & drank thea under the shade of some appletrees; only he, & my self. amidst other discourse, he told me, he was just in the same situation, as when formerly, the notion of gravitation came into his mind. "why should that apple always descend perpendicularly to the ground," thought he to himself; occasion'd by the fall of an apple, as he sat in a contemplative mood. "why should it not go sideways, or upwards? but constantly to the earths center? assuredly, the reason is, that the earth draws it. there must be a drawing power in matter. & the sum of the drawing power in the matter of the earth must be in the earths center, not in any side of the earth. therefore dos this apple fall perpendicularly, or toward the center. if matter thus draws matter; it must be in proportion of its quantity. therefore the apple draws the earth, as well as the earth draws the apple.
Newton's assistant John Conduitt also mentions an apple in a story about Newton:
In the year 1666 he retired again from Cambridge to his mother in Lincolnshire. Whilst he was pensively meandering in a garden it came into his thought that the power of gravity (which brought an apple from a tree to the ground) was not limited to a certain distance from earth, but that this power must extend much further than was usually thought. Why not as high as the Moon said he to himself & if so, that must influence her motion & perhaps retain her in her orbit, whereupon he fell a calculating what would be the effect of that supposition but being absent from books & taking the common estimate in use among Geographers & our sea men before Norwood had measured the earth, that 60 English miles were contained in one degree of latitude his computation did not agree with his Theory & inclined him then to entertain a notion that together with the power of gravity there might be a mixture of that force which the moon would have if it was carried along in a vortex, but when the Tract of Picard of the measure of the earth came out shewing that a degree was about 69.5 English miles. He began his calculation a new & found it perfectly agreable to his Theory
EDIT 17-12-2014 in response to fdb's comment: The original sources for the quotes are found here (first quote, check out the page titled 'William Stukeley') and here (second quote). Indeed, Wikipedia also reproduces these quotes, but that does not detract anything for my answer, in my opinion.