Playing "down" to your competition is one way to improve your gaming skills, but it's no where near as effective as playing someone who is equal or above your skill "level."
Correct, but as I said in my initial post, this is me talking about situations where finding such players is either difficult or impossible. I didn't say this was the best way to improve, just that it was one way of doing so.
That being said the real problem with playing "down" to your competition is your competition never tends to play "up," for various reasons.
That depends. Several of the people at my club have become markedly better over the last year, mainly due to the influence of me and my friends. Others, of course, will never improve because they refuse to take advice, or because being good at the game is not important to them, which is their choice.
Most players that play 40k (or any game) "competitively" will have access to much more resources then a "casual" player would have, especially if they have been playing for an extended period of time. Playing "down" becomes easy when you use units that were good in a previous edition of a codex but now aren't so great any more, while playing "up" may require several new units along with new strategies to use them. While you can proxy units for a while, this may become an issue after an extended period of time.
And this was my point in doing all of this gradually, over a period of many weeks. Once you find someone willing to get on board with improving their army then you can help them plan purchases or give them advice on what to avoid.
If this happens it means that the "casual" player is expecting the "competitive" player to cater to their needs to make the game more enjoyable for them. Why must one type of player, who carries a stigma in the community for their play style, be forced to play a certain way, while the complaining players can continue playing how they want?
Uh, I think you missed the majority of my point. It's not about one player forcing his style on another, it's about compromise. Unless you like mashing casual armies into the ground with your GT Winning tourney list (and are, by extension, a complete tool) then no-one enjoys a completely one-sided affair, to the point where those players may never play each other again. I know that there are people at my club who avoid playing with me, and there are people I avoid playing against in turn, simply because I literally can't lower my army to the point where it would be an even fight. I would have to deliberately play badly to even give the illusion of a fair fight - and that is what I am trying to avoid.
Remember this is all in the context of someone who enjoys playing competitively, but who lacks a decent number of similar people to play with. It could equally apply to the only person in a gaming club who builds fluffy armies facing down 20 people with min/max netlists, and the same principles would apply. The fluffy player tries to improve, the competitive players try to have fun with units they would never normally take.
The only problem I see is that by banning any unit from the standard codex, you're basically gimping that army instead of learning how to deal with it. I'll agree that it may not reflect on your generalship as a whole but it is better to learn to deal with the unknown or unexpected than not.
Just banning a unit doesn't necessarily gimp a codex. For example in Magic: The Gathering, my local playgroup has always heavily frowned on Land Destruction decks. That doesn't mean that we've gimped Red or Black, it just means that as a group, we do not find Land Destruction fun, and although we could build decks to counter it, we prefer to just remove it from the equation when we're having social games. In tournaments, of course we'll play against it (and if we think it's a likely choice to come up, we may start practicing against it as well) but there are still plenty of red decks you can build that don't involve Stone Rain (or whatever).
Likewise agreeing not to use the Doom doesn't mean the Nid dex is gimped (the majority of competitive lists don't even take him) it just means that if your local scene has less Mech than normal, and likes plasma a bit more than melta as a result, amicably banning the Doom prevents games that are not fun for one or both players.