On my opinion, there are two important reasons:

a) German government did not press the matter. Nobody convinced it that the thing is plausible and important. (In the US there was a lobby who could convince the US president in this. Szilard, V. Bush and Einstein played an important role). 

b) I think even if they wanted, they could not do it. They simply had no enough resources for a project of such scale. I mean resources of all kinds, including human, industrial, economic etc. Look how many top scientists were employed in the Manhattan project. And how many of them were immigrants from Germany, Austria, Italy. There were also problems with all other resources, and bombing played some role too. Look at the Oak Ridge uranium enrichment plant. Where could the Germans build anything comparable? And even if built this could not be kept secret, and would be certainly bombed.