Are there any examples in the history of mathematics of a mathematical proof that was found by accident, in the sence that in the effort of proving it, ending up proving something intuitively different? Of course, I do not mean trying to prove that a proposition holds and proving that indeed it does not. Are there any branch of mathematics born as a result of such an "accident"?
Squaring the circle is the age-old attempt to construct a square the same size as a circle, using a compass and straightedge, in a finite number of steps. It was finally proven impossible in 1882.
However, Menaechmus, in his study of the issue, devised the conic sections, beginning a whole branch of mathematics.
One can argue that most parts of mainstream mathematics are borne "by accident". That is we start with a very particular but still important example, then generalizations are discovered and a large theory emerges completely unexpected by the person who started all that. For example Max Dehn studied tessellations of hyperbolic plane and discovered an algorithm to solve the word problem in surface groups. This eventually lead to the small cancelation theory, solutions of bounded Burnside problem and many other group theoretic problems, Gromov hyperbolic groups and the whole geometric group theory.
Perhaps something closer to the question.
in the effort of proving it, ending up proving something intuitively different
One can count here numerous attempts to prove well known open conjectures which resulted in proving weaker and seemingly different statements. For example, Tao's "approximation" of Collatz problem is interesting by itself but is a result of an attempt to solve the problem, Moore's theorem about Folner sets in the R.Thompson group $F$ is a result of his attempt to resolve the amenability question for this group.