I make this answer tentatively since I cannot find the document, an autobiographical chapter by Lettvin I read some years ago which discussed among other things this situation. Lettvin had a whole book available online, but searching I can't find it.
Basically, Lettvin claimed that Pitts and McCulloch were not sufficiently sympathetic to socialism, so far Wiener was concerned. (Apparently Wiener, explains Lettvin, travelled with his wife to Mexico to visit Trotsky.) Lettvin wrote that the dislike Wiener's wife had for McCulloch was the last straw in a strained relationship, compounded by other misunderstandings.
Furthermore, McCulloch was not particularly bohemian in any way, so far as Alex Andrew, a younger member of the group Wiener split, recalls in an article in Kybernetes. He and his wife were of very wealthy families, Andrew recalls, although their wealth was strongly decreased over time. Andrew does not say anything about any Bohemian behaviour of McCulloch.
Lettvin argued: Pitts was the only one in the group who was dependent on Wiener for funding and help in academia after Pitts and McCulloch left Nicholas Rashevsky at Chicago to work with Wiener. After Wiener broke off relations with the McCulloch group, Pitts was put under a lot of stress.