I always took the 'hand weapon and shield only, in order to get the 6+ ward save' as a sign that the unit is using a phalanx type formation, with their shields high and their weapons jabbing over them. In other words, they're not fighting normally, and would not be able to use fancy or strange weapons effectively that way.
Despite what we see in movies, shields aren't very useful in personal combat. They were only really used in phalanx formations, jousting, and to protect against arrows. If the combat became man to man, shields were usually dropped, so you could attack more effectively.
I like the random terrain chart because it keeps people from having arguments if someone wants an odd terrain piece, like a river. Anything that keeps arguments to a minimum, and the game moving is alright in my book.
My only complaint with the new edition is the fact that cavalry don't really have a way to negate steadfast bonuses without the cav unit being absolutely huge (and thus very expensive). Cavalry can still inflict a lot of casualties, especially if they have a higher initiative than their opponent, but with enemy blocks of troops having steadfast, and very likely a BSB nearby, breaking them is nearly impossible on their own.
All in all, cavalry is now mainly for warmachine hunting and for tipping 'infantry vs infantry' close combats in your favor, and if that's the case, you might as well use chariots instead, for the impact hits.