There's two notions of the formalisations of integers that I know of. The better known tradition follows Dedekind and Peano, culminating in Peanos axioms (this actually was developed under the impetus of then newly developing set theory. However, it turns out that it's possible to lift Peanos axioms into the context of topos theory, a rival to set theory whose underlying logic is intuitionistic rather than classical - in that context it's called the Natural Numbers Object).

The less well-known tradition, but older one, is one instigated by Gauss: his Gaussian integers. The modern interpretation is constructing a ring of integers for every number field.