The most immediate benefit (and arguably the most important too) of the ele is that you are able to start collecting res away from a CC without spending the initial 100 wood. Very few players ever make a worker ele and instead just rely on the initial ele plus storehouses that they build. So doing this would be a major nerf.
Maybe other civs could acquire Worker Elephants as mercenaries. Then they can choose whether to get Worker Elephants or use the resources for other things e.g. upgrades. I don't know what a fair cost would be, maybe something like 40 food and 40 metal (I tried to get a similar proportion as for archer mercenaries).
I'm really new, so I don't know if this makes any sense; was just a thought. And I see that this wouldn't achieve the goal of making the civs more distinguishable, which would be in my opinion the much more interesting approach.