As a student in applied maths, I can remember being told that in the 1940s there were early attempts of parallel computing not using any machine but only human calculation power.
I believe the story ran that students were sitting in a classroom and exchanging piece of papers between neighbours. I also believe I was told it was led by Kolmogorov. I have no idea whether this has any true basis and couldn't find a lead on the web.