Warmachine /Hordes is not that expensive to start no. The best way is probably to go for one of the 2 starter boxes (either Khador+Menoth or Circle+Legion) but there are battle boxes for each fraction.
There is 1 rulebook to the game, the Hordes one is called Primal MkII and the Warmachine one is called Prime MkII. You will need the one you have for your side of the game.
Pick a faction you like the models from and then go for the book for that version of the game. If you go for one of the 4 startbox factions get that box instead since it contains a rulebook.
Regarding expansions they contain a few entries for all armies of that half of the game, they are not needed to have but can be fun to read. EVERY entry you buy comes with a rule card that has the rules the model(s) have, so there is no need to get the expansion books.
Tournaments generally have 2 or 3 lists required, this means that you cant at default end up with classic DE flying paperboatvenomcannonlist vs Ork green horde or similar. That doesnt mean that there are no bad matchups, there definitely are, but you can mitigate that better since you have more then 1 list to pick from.
The game is by far better balanced then GWs games, it might not seem that way instantly but you will start to realise that after a while.
Regarding "several options for army types" this is where HordeMachine differs hugely from GWs games.
Changing your caster(read general) will give you new abilities, spells, skills and incentives to use certain units. Changing the caster and a few things in the army will do a lot for how a army behaves on the table.
This naturally leads to one wanting to have more different things but thats another story.
The game itself is encouraging creative play, using your own models for targets is not uncommon, and has 3 different win conditions and no fixed game length (so no 5-7 turns). If you like Blood Bowl you should love HordeMachine since the game works the same way. You do one thing at a time, evaluate result and then goes for the next one