Who among professional mathematicians are also known as fiction writers? I know Omar Khayyam (11-12 century), and two more recent ones: Sofya Kovalevskaya and Michele Audin.
For the purposes of this question, let us define a "mathematician" as a person who has at least a few papers reviewed in Zentralblatt.
Charles Lutwidge Dodgson, better known as Lewis Carroll, wrote some well-known fiction books.
In case you consider Marvin Minsky a mathematician (16 papers in Zentralblatt), he coauthored with Harry Harrison the science fiction novel The Turing Option (by the way, a pretty bad book).
Like Quenau, also Jacques Roubaud was a member of the Oulipo group and a prolific author of fiction and poetry. He passes the Zentralblatt test, as it lists two entries: Monades et descente, with Jean Benabou, and La notion d’associativité rélative.
Roberto Vacca (1927) is a researcher in logic and number theory (six entries in Zentralblatt), but also a well-known (in Italy) science popularizer. He debuted as a writer of science fiction and political fiction in 1963 with the novel Il robot e il minotauro (The robot and the minotaur), followed by other twelve books. These novels were not very successful and fame as a writer came only in 1971 with the essay (not fiction) on the apocalyptic theme Il medioevo prossimo venturo (edited in English as The Coming Dark Age The Middle Ages), considered a classic of futurology .
Felix Hausdorff published, under the pen name Paul Mongré, several essays, some poetry and a play, but no fiction.
Fred Hoyle was an Astronomer, but he passes the Zentralblat test. He wrote several novels, most notably The Black Cloud.
In addition to The Black Cloud, I read a thriller he wrote with his son, Geoffrey, called The Westminister Disaster. Probably dated, but I enjoyed it at the time.
My answer copied from another forum:
Mathematician Eric Temple Bell
President of the MAA, 1931-32
Author of the book Men of Mathematics
Bell numbers, Bell polynomials
etc.
Also was a successful science fiction author under the pseudonym John Taine.
See the fascinating book
Reid, Constance, The Search for E.T. Bell, MAA spectrum, The Mathematical Association of America, 1993
Rudy Rucker is a mathematician, computer scientist, and SF writer. He wrote the Ware Tetrology (Software, Wetware, Freeware, Realware)
Since I'm fluent in Russian, I did some googling for russian ones.
Sofya Kovalevskaya was not simply a published mathematician, but a first woman who obtained what can be classified as a doctoral degree in math. Aside from links to her publications, her Wiki page has a link to a novel she authored.
Roman Mikhaylov. His Wiki page (no English version, sorry) has links to both his scientific publications - and he also happens to be a member of Russian Academy of Sciences, so he's clearly a "someone" - and his books and - with him seeming being a polymath - his theater and movie works as well.
Not sure if this would count as a "mathematician", as he was primarily a mathematics teacher and not sure if he published any actual math research - but he did advanced math studies as part of his education career; Alexandr Volkov (who wrote a Russian re-do of "Wizard of Oz" called "The wizard of emerald City" and then 5 sequels that were original works since he hated Baum's sequels)
Aleksandr Solzhenitsyn. Didn't publish, but when he was sent to labour camps where at least part of the time was spent in "sharashka" type research institute, where his math/physics talents were utilized (less relevant, his second wife was an actual mathematician).
Mikhail Lomonosov was a famous 18th century russian polymath (Moscow State University is named after him); who mostly worked in sciences but also - as was true of people like him - a mathematician. He also happened to be very literary inclined and a poet.
Elena Sergeevna Ventsel (aka Irina Grekova) - hat tip to comment by @Roger Vadim
If you expand this to other sciences, I can probably rattle off 10 more easily from memory, including at least half the main figures in Soviet SciFi (astronomer Strugatsky, engineer Zamyatin - who literally stated he chose engineering as "the most math-y specialty he could", paleontologist Efremov, biophysicist Nick Perumov, biologist Eskov).
Speaking of apocryphal fan-fiction, since I brought up the last two people famuos for "Tolkien was wrong" ones based in Middle Earth, one of the most famous and controversial Harry Potter fan-fiction works was written by AI researcher Eliezer Yudkowsky, and honestly I'm inclined to count the latter as "math".
If you consider Bertrand Russell to be a mathematician, he wrote two volumes of short stories near the end of his life but they were met with ''baffled silence'' even by his most ardent fans (in the words of his biographer Ray Monk).
Eric Temple Bell (known for Bell polynomials) wrote poetry and science fiction novels under the pseudonym John Taine. His science fiction novels are known for their interesting ideas but as a writer he is considered to be extremely lacking in style, tone, sensitivity and characterization (plus anything else one might expect from a good fiction writer).
Ian Stewart has also written fantasy fiction books with Terry Pratchett.
Norbert Wiener, one of the founders of modern control theory and information theory, wrote a science-fiction novel named The Tempter. The hero in the book was based on the real-life story of physicist Oliver Heaviside, how he came up with the transmission line equations and solved the problem of dispersion via inductive loading, and how his work was later patented by the AT&T in an ethically-dubious manner.
It was also a terrible book.
In 1959 a curious book appeared, a novel of all things, written by Norbert Wiener, the "father of cybernetics." All the names were changed, but it is in fact a fictionalization of Heaviside's adult life as told through the events of the Campbell-Pupin loading coil episode (which is altered to be a "stolen" invention in feedback control). The book is perfectly awful as a novel (some of the dialogue is funny enough to be read aloud at a party), but most technical people would probably enjoy it because Wiener brings a lot of technical details into the story. Heaviside was tremendously admired by Wiener (Pupin was not), and the book was Wiener's attempt at justice for his late hero. The best one can say about it, perhaps, is that Wiener meant well.
- Oliver Heaviside: the life, work, and times of an electrical genius of the Victorian age (2002), Paul J. Nahin
In addition, the Wikipedia article showed that he also wrote some short science fiction stories under the name "W. Norbert" for MIT's student newspaper, Tech Engineering News, including The Brain (anthologized in Groff Conklin's Crossroads in Time) and The Miracle of the Broom Closet (reprinted the same year in The Magazine of Fantasy & Science Fiction).
Raymond Quenau (1903-1976), a French writer and mathematician.
Even if he wasn't a professional mathematician, he dealt with mathematics and published writings of mathematics and about mathematics.
It seems to me that he passed the Zentralblatt test:
https://zbmath.org/?q=raymond+queneau
He studied mathematics at university, and during all his life he continued to cultivate his passion for mathematics, also participating to seminars of the most important mathematicians in Paris. In particular he participated to the meetings of Bourbaki in the fifties, and had a frendship and professional collaboration with Geoge Kreisel, collaborating to the book Elements de logique mathématique, published by Kreisel and Krivine in 1968.
He made research about number theory. The results of his work about integers were presented at the Académie des sciences in Paris on April 1968. And it was subsequently published in the Journal of Combinatory Theory, commented by the famous mathematician Giancarlo Rota.
Those and other information about Quenau mathematician and writer can be found in
https://www.amazon.it/Mathematical-Lives-Protagonists-Twentieth-Century/dp/3642136052
But, above all, Raymond Queneau is famous as writer and poet.
He was a prominent figure in the French literature of the twentieth century, in particular for experimental literature.
He founded, together with his friend Perec, the Oulipo (Ouvroir de Litérature Potentielle), an experimental writing group, where the use of mathematical structures in literary endeavours was discussed.
His most well known novels are Zazié dans le metro (1959), Les fleuers bleues (1965) , and he is famous for his Exercises de style (1947).
$\;$
In addition, we cannot forget Martin Gardner, who wrote novels and short stories. He is so famous that needs no introduction:
https://en.wikipedia.org/wiki/Martin_Gardner
He passed the Zentralblatt test:
Gregory Benford is primarily known as a physicist, but does meet the Zentralblatt criterion. He is the author of Timescape and many dozens of other speculative fiction stories.
Jacques Dixmier published L'Aurore des Dieux, a collection of science-fiction stories. Its forward was written by Alain Connes.
And Johannes Kepler published a novel called Somniun.
Vernor Vinge was primarily a computer scientist but he has two citations on zbMath. Vernor Vinge
A query to Wikidata yields 31 individuals listed as "novelist" and "mathematician", 29 of whom have a dedicated article in Wikipedia (26 in English language Wikipedia). I think some of them have a degree in Mathematics but could fail the Zentralblatt test.
The 29 people are Thomas Carlyle, Ronald Ross, Lewis Carroll, Simon Newcomb, Sofia Kovalévskaia, Raymond Queneau, Vernor Vinge, Larry Niven, Ian Stewart, Eric Temple Bell, Rudy Rucker, Leo Perutz, Henry Adams, Gérald Tenenbaum, Charles Sheffield, Herbert Yardley, Manil Suri, Julio Cesar de Mello e Souza, David Bensoussan, Benedict Freedman, Carin Gerhardsen, Chandler Davis, Malka Elisheva Schaps, Sarah Mason, Hajime Amagi, Leila Schneps, Emily Tanimura, Vladislav Šak and Emily Coddington, some of which have already been mentioned in previous answers. Links to their Wikipedia articles and Wikidata items can be found in the linked query.
Chandler Davis https://en.wikipedia.org/wiki/Chandler_Davis wrote some SF stories as well as poetry.
Victor Snaith wrote The Yukiad.
To add to people mentioned by others:
This is not technically an answer to your question, but Greg Egan is a professional author who also does mathematics recreationally. They have a MO profile and have definitely done some novel mathematics.
On a side note, I really love their book "Permutation City" which plays with the idea of isomorphic copies of one's mind living in other structures of the Universe, ensuring a kind of "isomorphic immortality".
Manil Suri (61 papers in Zentralblatt) is a mathematician at the University of Maryland who has written the books:
The Death of Vishnu (2001),
The Age of Shiva (2008),
The City of Devi (2012).
Peter Schattschneider is an experimental physics and a retired professor at the Technical University of Vienna. He also writes science fiction: https://en.wikipedia.org/wiki/Peter_Schattschneider
