As the title already says, I'm interested in the number of scientific articles published per year, preferably going back to the times of Newton or even Galilei (who is generally considered to be the founder of modern science). I already found a few resources (ourworldindata.org, dblp.org, wordsrated.com, this paper), but none of them are exactly what I'm looking for. Their data typically only cover a relatively short timespan, or don't include recent years.
6
-
$\begingroup$ "scientific articles published" is a time-dependent concept. $\endgroup$– kimchi loverCommented Jan 17 at 15:11
-
$\begingroup$ What do you mean by that? $\endgroup$– mapfCommented Jan 17 at 15:14
-
2$\begingroup$ There was a time when there were no scientific journals (for instance, during Galileo's life), and hence no published scientific articles. Ideas we disseminated thru books and thru letters. Nowadays scientific ideas are also communicated by Arxiv articles as well as by journal articles. So the meaning of article counts changes over the centuries. As it does by field, as well. $\endgroup$– kimchi loverCommented Jan 17 at 15:39
-
$\begingroup$ Thanks for elaborating. That's fair of course, I should have used a better word. I guess "literature" is more fitting. $\endgroup$– mapfCommented Jan 19 at 10:56
-
1$\begingroup$ My understanding is that some of the impetus for the formation of the Royal Society in 1660 was a response to the (then) overwhelming number of scientific papers being published. Members were urged to revert to publishing books as an antidote. en.wikipedia.org/wiki/Royal_Society (Sorry, I don't have a direct reference for this claim.) $\endgroup$– DJohnsonCommented Jan 24 at 0:02
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
Caveats
As far as I can tell from my research, due to the complexity of the task, there is currently no single platform that has a virtually comprehensive data base of all the scientific literature published over the last few centuries. I guess this is mostly for three reasons that are also intertwined (these are my conclusions and may be wrong):
- For one—especially regarding more recent publications due to the rise of the internet—,there is an ever increasing number of (online) journals and platforms where such literature can be published, and it's very difficult to keep track of it all, and also cross-reference it with archives (such as arxiv.org) that also publish pre-prints etc.
- Secondly—especially regarding old literature from times before publication standards were established (such as papers, journals, theses)—, it's difficult to categorize some such literature as scientific.
- Thirdly—also especially regarding older literature—, some literature is published in languages other than English (the currently universal language of science), and some even in more than one. So it's not only difficult to find all the works written in other languages, but also to cross-reference them.
Lastly, while I think it should be possible to reasonably select which formats of science literature to consider (e.g. only articles, theses, and similar), I wonder how to handle different versions of essentially the same work, for example in case of (text) books. If they are fundamentally upgraded, then that may be a reason to count them more than once, but especially with the modern inflation of re-releasing the same book every year "in a new and updated version" that only had its chapters moved around and a word changed (to make money off of helpless students no less), I don't think they should count. Similarly, I'm also not sure about how to handle errata, or maybe even letters. And I wonder if, like with novels, there are already nonsene textbooks and articles out there that were entirely written by AI, and if we will soon see an inflation of such "literature" as well. I certainly wouldn't count it.
Resources
Anyways, here are the best (non-comprehensive) resources that I could find so far:
- openalex.org: a huge, freely accessible data base of scientific literature, that goes back several hundreds of years. It differentiates between the following publication types: article, book-chapter, dissertation, book, dataset, paratext, reference-entry, report, peer-review, editorial, erratum, grant, letter, standard (I have no idea), other. It also allows you to easily download the number of publications per year.
- dimensions.ai: very similar to OpenAlex, but requires the creation of a user account, and includes fewer publication types: article, chapter, proceeding, preprint, monograph, edited book.
- webofscience.com: has been around for much longer than OpenAlex or dimensions.ai, but only goes back to 1900. To my surprise, it also contains significantly fewer entries than either of the other two, even though it supports a myriad of different publication types: article, art exhibit review, bibliography, biographical-item, book, book chapter, book review, correction, dance performance review (!!!), data paper, database review, early access, editorial material, excerpt, expression of concern, fiction, creative prose, film review, hardware review, item withdrawal, letter, meeting, meeting abstract, meeting summary, music performance review, music score, music score review, news item, poetry, proceedings paper, publication with expression of concern, record review, reprint, retracted publication, retraction, review, script, software review, theater review, tv review/radio review, withdrawn publication. (Update note: previously, I said that the Web of Science contained more entries than the previous two data bases, but I looked at the wrong data and was mistaken.)
- Then there is also the data provided by the World Bank (which is also what ourworldindata.org and wordsrated.com rely on), but as far as I can tell, their data only goes back to 1996, and is actually provided by the National Science Foundation (NSF). It also only includes scientific and technical journal articles, so it is very limited in that regard as well. And lastly, unlike the other three, it is not a data base of the underlying literature, but an accumulation of statistical data. So I would not recommend it.
Comparison
Below I compare the different resources in a plot. As you can see, OpenAlex has by far the most entries (in total and per year), and also covers the longest time period. So of the four resources, it seems to be the most comprehensible data base, but keeping in mind the caveats mentioned above, I don't know if it's also the most reliable (i.e., maybe some works have more than one entry, e.g. due to different versions/languages etc., and data sets should probably also not be counted).
