These operations arose from the study of quaternions see e.g. Thomson and Tait's Treatise on Natural Philosophy, that should probably have information on the sort of math. Stokes's theorem originated with Thomson (Lord Kelvin) around 1850 and in there the expression for curl appears. The history of quaternions is certainly interesting, starting with Hamilton in 1843. Evidently, the significance of a sum and difference of partial derivatives was recognized for its utility.
PLAUSIBLE INTUITION
There is very good reason to use these operators, strictly from an analytical perspective. The question of how to define a derivative in multivariate calculus motivates us to consider first a two-dimensional vector field case. In such a case, the change in direction can be described by taking derivatives of the $x-y$ components of the field with respect to various directions.
Let $\mathbf{A}$ be the vector field with two components, $A_x$ and $A_y$, each functions of the position $(x,y)$. Then the total change can first be considered by the change in $A_x$ with $x$ and the change in $A_y$ with $y$, derivatives in the direction of the components, and second by the change in $A_x$ with $y$ and $A_y$ with $x$. It is relatively straightforward to show that we should sum the first two to maintain their useful properties, $\frac{\partial A_x}{\partial x} + \frac{\partial A_y}{\partial y}$, and we should take the difference of the cross derivatives for their useful properties, $\frac{\partial A_y}{\partial x} - \frac{\partial A_x}{\partial y}$. These become our two most useful derivatives, which are the divergence and curl respectively.
Extending this to three dimensions, we can readily adapt the divergence derivative by simply adding another partial derivative term. The cross derivative terms, however, require careful handling, as there are now two cross derivatives for each component, for a total of 9 partial derivatives. To extend the curl derivative, we consider the 2D curl derivatives in a given plane, for the three principle planes ($xy$, $xz$, and $yz$), and we allow each curl in a plane to be a component in a vector. Thus we lose no information. The vector component we should use would be the component normal to the plane.
In the language of vectors, this curl operator would be:
$$ \left (\frac{\partial A_z}{\partial y} - \frac{\partial A_y}{\partial z}\right )\mathbf{i} + \left (\frac{\partial A_x}{\partial z} - \frac{\partial A_z}{\partial x}\right )\mathbf{j} + \left (\frac{\partial A_y}{\partial x} - \frac{\partial A_x}{\partial y}\right )\mathbf{k} $$
HISTORICAL DEVELOPMENT
A modern explanation with vectors is one thing, but historical development is of most concern for us here. One could arguably draw a connection directly from Stokes's theorem to the introduction of curl. From (Katz 1979), there were some hints of Stokes's theorem leading to its appearance in the Smith's Prize Exam, but the curl operation is made quite clear looking at the problem itself.
If X,Y,Z be functions of the rectangular coordinates $x,y,z$, $dS$ an element of any limited surface, $l,m,n$ the cosines of the inclinations of the normal at $dS$ to the axes, $ds$ an element of the boundary line, shew that
$$\iint l \left (\frac{\partial Z}{\partial y} - \frac{\partial Y}{\partial z}\right ) + m\left (\frac{\partial X}{\partial z} - \frac{\partial Z}{\partial x}\right ) + n \left (\frac{\partial Y}{\partial x} - \frac{\partial X}{\partial y}\right ) dS = \int \left (X \frac{dx}{ds} + Y \frac{dy}{ds} + Z \frac{dz}{ds} \right )$$
... the single integral being taken all around the perimeter of the surface.
Apparently, the left hand side had already appeared in a few earlier works of Stokes, although I'm not aware of any earlier history of the expression. As for the introduction of divergence, I'm sure the sum of derivatives often appeared in the calculus of quaternions, so it might take more work to track the origin of this expression.
In general, a complete differential is a differential form which is the differential of some scalar function. Today, we would say the differential is exact. This idea comes very naturally from potential theory, developed by Green, Hamilton, Gauss, and others. It was known that the condition for e.g. a quaternion differential,
$$ \mathbf{A} = A_x dx + A_y dy + A_z dz $$
to be complete is that
$$ \left (\frac{\partial A_y}{\partial z} - \frac{\partial A_z}{\partial y}\right ) = \left (\frac{\partial A_x}{\partial z} - \frac{\partial A_z}{\partial x}\right ) = \left (\frac{\partial A_x}{\partial y} - \frac{\partial A_y}{\partial x}\right ) = 0$$
And so we are left to wonder about the behavior of quaternion fields for which this is not true, and we ask whether there is sufficient information to uniquely characterize a quaternion field using both the differential form and the cross derivatives above.
This leads us to an idea introduced in 1858 by Helmholtz that quaternion fields can be decomposed into their curl and divergence expressions which we have given above. Helmholtz's work On Integrals of the Hydrodynamical Equations, which Express Vortex-Motion (1858) presents similar quantities to divergence and curl in the context of fluid dynamics and vortices.
As for the notations and notions of divergence and curl, the introduction of the symbol $\nabla$ certainly brought something to the issue. This was, in fact, introduced by Hamilton, and was used greatly by Maxwell and Tait among others. Maxwell in particular was one to attach physical meaning to divergence and curl, though his conventions were slightly different from ours today. He used the convergence which is the negative divergence, and he called curl the rotation. These are plainly laid out in sec. 25 (p.30) of Volume 1 of his treatise.
Selected References
Katz, Victor J. The History of Stokes' Theorem. 1979.
Helmholtz, H. On Integrals of the Hydrodynamical Equations, which Express Vortex-Motion. 1858.
Maxwell, James C. Treatise on Electricity and Magnetism, volume 1. 1873.
Tait. On Green's Theorem. 1872.
Thomson and Tait. Treatise on Natural Philosophy. 1879.
Further Reading
Crowe, Michael J. A History of Vector Analysis. 1967.
P.S. I am not very familiar with quaternions, so some of the stuff above where I say "quaternion field" etc I'm projecting vector field concepts to quaternions. I'm sure this would be outrageous to someone more familiar, so edits in this way (and others) are welcome.
