Historically speaking, battering rams were used almost exclusively to bring down gates (sometimes walls). They are now used to bring down pretty much any structure in game which makes them very unrealistic. Yes, I get it, gameplay... I'm just clarifying something about how compromises in historicity for gameplay's sake can actually negatively affect gameplay in some respects (rampaging rams).
Buildings were brought down by infantry units, not battering rams. Infantry units would set buildings on fire, or pull down supporting columns with ropes or hack through supporting walls, or even disassemble structures block by block, by hand, using only simple tools.
Rams are actually very easy to destroy or capture outright... Any unaccompanied ram caught out in the open by enemy infantry should realistically be captured or destroyed with relative ease. Rams were never used in melee combat, and rams killing units or horses is super-awkward and annoying. Rams were not hermetically sealed from the outside world, making the people operating it invulnerable or something. Quite the opposite... You would NOT want to be one of the dudes they assign to the ram (on account of the high chance of death). Rams were normally built on the spot, and usually only 1, or just a few at the time, accompanied by a battalion, but not armies of battering rams going in solo. This is obviously not how ancient warfare worked.
Ancient battles can easily last several hours or even days, and sieges can last weeks/months/years... If you translate those times to in-game time, where battles can last a few minutes, battering rams should go down in a matter of seconds if not properly protected.
I'm not saying that everything about battering rams in game needs to change, I'm just saying that history can't be used as a justification for current battering ram-mechanics, because it's so far off from how they were actually used.