From playing Black Templars, I can tell you, not every space marine force needs to be mechanized. I could take maybe one tank and a drop pod, and the rest of my army could slog it out, and I'd still have a good chance of winning (BT move towards the enemy when they're getting shot up).
I don't play BT anymore. Nor do I play DE anymore, or SoB (though I've played each enough to have learned a few tricks about them).
That said, I play Imperial Fist (vanilla, with almonds), and while I like to field a rhino, a razorback, a predator, a vindicator and a land raider in my bigger battles, I don't have to.
I've fought mech once with an all-infantry army of goldenrods. The key is to take meltaguns and missile launchers, then combat squad every squad (tactical, assault, devastator or veteran). And I sometimes take an Inquisitor just so I can have a continuous orbital bombardment for area-denial. :biggrin:
The point is, you utilize the weapons at your disposal. Most armies have some form of long-range anti-armor (missiles, dark lances, rail guns, etc) and can provide decent cover fire for their troops as they move towards objectives (tau carbines, sniper rifles, splinter/shuriken cannons). Some races don't have as much in long range anti-tank (Tyranids) but can tie down most armor until the big nasties arrive (carnifex, tyrant, etc).
My current list for non-mechanized space marines (the theory being, my force drop-podded in before the battle, and the strike cruiser was either destroyed or drawn off into battle before the thunderhawks could bring down the transports) includes an assault squad with 2 plasma pistols and a meltabomb/fist sergeant, a vanguard vet squad with two thunderhammers, two meltabombs and a relic blade (str6 still gets most transports) and two devastator squads, each 10 strong, each with 2 LC and 2 ML. I combat squad the assault squad and each of the dev squads, which gives me almost the entire battlefield covered by heavy weapons (4x4 table with deployment zones).
The only thing I've ever had to worry about with that list is the Imperial Guard's howitzers. Earthshakers make me want to take Land Raiders and Terminators.
Anywho. Hope this helped you get a feel for an army list idea that can counter mechanized.