Yes, it was derived from the looks of voltaic batteries. More generally, the early symbols for circuit diagrams (cell, resistor, capacitor, etc.) were based on the physical appearance of the devices they represented. This lasted until the 1950-s, when the advent of the transistor changed the scene, see Icons and Electronics by Jones-Imhotep for the fascinating story of how it happened. I would also like to thank the author, who was kind enough to help me locate the primary sources cited below.
The Cyclopedia of Telephony and Telegraphy, Vol. 1 (1919), available on Gutenberg, describes the interpretation of the battery symbol, even less removed in the visual appearance from the batteries, shown in the OP, than the modern schematization:
"The conventional symbol for a cell, either of the primary or the secondary type, consists of a long thin line and a short heavy line side by side and parallel. A battery is represented by a number of pairs of such lines, as in Fig. 67. The two lines of each pair are supposed to represent the two electrodes of a cell. Where any significance is to be placed on the polarity of the cell or battery the long thin line is supposed to represent the positively charged plate and the short thick line the negatively charged plate. The number of pairs may indicate the number of cells in the battery. Frequently, however, a few pairs of such lines are employed merely for the purpose of indicating a battery without regard to its polarity or its number of cells. In Fig. 67 the representation at A is that of a battery of a number of cells connected in parallel; that at B of a battery with the cells connected in series; and that at C of a battery with one of its poles grounded."
An even earlier Le Téléphone, le Microphone et le Phonographe (1878) displays circuit diagrams for a phonograph with the familiar battery symbol explicitly referred to as "batterie voltaïque".
