Who knows, and who knows if it even happened. This anecdote is repeated almost verbatim after Gibbs's “Crackpots” Who Were Right II published in Prespacetime Journal, not exactly a standard of historical scholarship. Other sources omit the colorful phrase, see e.g. MacTutor on Apery. Gibbs gives no explanations:
"In 1978 he presented a lecture on his proof at the Journées Arithmétiques de Marseille which was
greeted with doubt and disbelief. Each step he wrote on the blackboard appeared to be a
remarkable identity that his audience considered unlikely to be true. When someone asked him
“where do these identities come from?” he replied “They grow in my garden.” obviuosly [sic!] this did not
boost anyone’s confidence.
Nevertheless, a few mathematicians recognised that there was something significant in the
proposed proof. They checked the identities numerically and found that they did indeed seem to
hold. It was not long before the full validity of Apéry’s work was confirmed and the skeptics were
forced to eat their words."
Gardening metaphors are not uncommon among mathematicians, see e.g. The Secrets of Ramanujan’s Garden. Plato himself uses it in Theaetetus to describe growth of knowledge, albeit not directly applied to mathematics. One could speculate that Apery referred to some kind of platonic "garden" of mathematical curiae he was keeping, but that would just be adding to a likely fable.
