I've been told that Pythagoreans kept as a secret the incommensurability of certain quantities like the diagonal of the square. Are there other mathematical discoveries whose authors didn't want the public to learn or know about?
In antiquity Archimedes kept his method of mechanical theorems about areas and volumes secret because, under Plato's influence, using lowly mechanics in glorious geometry was frowned upon at the time. The method is revealed in his letter to Eratosthenes, "for some things that first became clear to me by means of mechanics were afterwards demonstrated by means of geometry". In On Spirals he mentions sending false "theorems" to Alexandrians to prevent them from passing his as their own, "so that those who claim to discover everything, but produce no proofs of the same, may be confuted as having pretended to discover the impossible".
In 16th century del Ferro and Tartaglia kept their solutions to the cubic secret, also for priority reasons and to win current at the time mathematical duels that conveyed status and money. In fact, Tartaglia rediscovered the solution independently, after learning that del Ferro knew it, and then generalized it, which allowed him to beat Fiore, del Ferro's student, in one of those duels. It was knowing that the solution exists that gave the determination to find it, he wrote.
Many 17th century authors would cipher their results to claim priority and prevent others from taking advantage of them before they could. Newton famously sent an anagram to Leibniz, which expressed the idea of solving differential equations using calculus.
With the advancement of publish or perish these sorts of practices went away. Publishing is relatively quick and people usually only hold off until the publication, if that.
But other reasons came up. Babbage (the inventor of computers) discovered a statistical method for breaking Vigenere ciphers in 1846. It was kept secret because British intelligence was using it to read ciphered diplomatic correspondence, see Mr. Babbage's Secret. The method was rediscovered and published by Kasiski in 1863.
The story repeated itself in the 20th century. The first public key cryptosystem was developed by Cocks in 1973, also for British intelligence. It was only declassified in 1997. But already in 1977 Rivest, Shamir and Adleman published an equivalent now named after them, RSA.
Hamming discovered his error-correcting codes in 1948, but did not publish until 1950 because he was waiting for the patent to be approved. By that time, Golay rediscovered them and published, which led to a priority dispute, see Thompson, From Error-Correcting Codes Through Sphere Packings to Simple Groups.
A different type of secrecy emerged in the academia, especially within so-called schools, usually created and led by a single prominent mathematician. It concerns not so much results as intuitions, hunches, promising problems and conjectures, heuristics, informal tricks, etc., that are not so readily sharable in published form. Those, as Arnold put it "like parables, are only explained to the own disciples in private".
In this vein, there was a bizarre episode involving Kolmogorov. At his seminar in 1960 he explained his so-called $n^2$ conjecture, then standing for several years. A week later Karatsuba, a young walk-in, disproved it and told him. After reporting the finding Kolmogorov shut down the seminar. Two years later he published the result and credited it to Karatsuba, but without not only his permission but even knowledge, and bundled with results of his own student, see Was Kolmogorov enraged after learning about the Karatsuba multiplication algorithm? One possible explanation is that Kolmogorov originally misjudged the nature of the work in progress he was sharing. It was more immediately doable than he thought, and, per Arnold's maxim, only to be "explained to the own disciples in private".
To summarize, secrecy does not work for long, the results are soon rediscovered by others.
Sometimes, secret information has been deposited with a trusted organization.
In 1915, the Académie des Sciences in Paris gave the topic for its 1918 Grand Prix. The prize would be awarded for a study of iteration from a global point of view.
During the later half of 1917 Gaston Julia deposited his results in sealed envelopes with the Académie des Sciences.
Pierre Fatou, on the other hand, published an announcement of his results in the December 1917 part of Comptes Rendus. It later became evident that they had discovered very similar results. Julia requested that his sealed envelopes be opened: they proved that he had the priority. As a result, Fatou did not enter for the Grand Prix and it was awarded to Julia.
Whether Julia or Fatou deserves the credit of having priority matters little since their work was certainly totally independent.
Another possible example is Rejewski's foundational work on Enigma, encompassing rather deep group-theoretic results. Unsurprisingly, cryptography leans towards secrecy.
On a different line there's Gauss' work on non-Euclidean geometries. He wrote about his decision "not to allow it to become known during my lifetime. Most people have not the insight to understand our conclusions and I have encountered only a few who received with any particular interest what I communicated to them. In order to understand these things, one must first have a keen perception of what is needed, and upon this point the majority are quite confused. On the other hand it was my plan to put all down on paper eventually ... ."
Pythagoras discovered the scale on which all music is based.
He also discovered that this scale fits into the pentagram, which symbolizes a mathematical law found through out nature, also known as the fibonacci sequence, or the "Golden Section".
Recent artists recaptured this secret, you can find the proportions of the golden ratio when you measure the facial proportions of the mona lisa for example.
As for secrecy, it is more about preservation.
The moment you define something it becomes a double-edged blade, because that thing is now debatable by a mass public, which may change the meaning of what you defined entirely.
True knowledge is pre-symbol.
It is not so much a secret as a result of people seeing the symbols they come up with also exist "out there" and from that both of these can conclude they both reached the same conclusions and they both know it.
A lot of conspiracy theories and misunderstandings arise out of this, when people think the prevalence of the pyramid with the eye is evidence of an organized group.
But that is not how it happens at all, it happens in an entirely opposite way.