I assume you mean besides the obvious point that one involves water and the other involves electricity.
The most fundamental difference between batteries and pumps is their mechanism of moving what they move.
Pumps generally either push the water (an Archimedes screw is an example of this kind of pump) or suck the water by reducing the air pressure in front of it (most real-world pumps use this method).
Batteries move electrons in a fundamentally different way. They are designed so that a chemical reaction draws electrons from the cathode to the anode, so that the cathode contains fewer electrons and the anode contains more. (Usually this results in the cathode being positively charged and the anode being negatively charged, but that actually depends on what sort of battery you're dealing with and how it is being used.)
Then, once the battery is connected to a circuit, the electrons try to restore electrostatic equilibrium, resulting in them being pulled through the circuit from the anode to the cathode. (Confusingly, the "current" is said to be going the opposite direction, from the cathode to the anode, because current is a flow of positive charge, while electrons are negative. Alas, we are stuck with this strange convention.)
A pump that sucks is more similar to a battery than a pump that pushes water directly, because both involve creating an imbalance and then using the equilibration force to our advantage. But even then, a pump creates an imbalance in the air in order to move water, while a battery creates an imbalance in the electrons in order to move charge.
A pump that pushes water is almost nothing like a battery, as such pumps actually use mechanical motion to push and can even be used to carry solid objects in the same fashion. This is why we need to be careful if we try to use analogies like "batteries are electron pumps."