Every codex can potentially be competitive, but two armies definitely have it the hardest: Tau and Necron. It's almost impossible to make a list from these armies that can stand up to the best that the newer, more competitive codexes can offer.
Daemons and Tyranids both have a bad reputation, but both can do very well. Daemons won the West Coast Ard Boyz last year, and Tyranids has done well in some of the bigger tournaments. The reason they both have a bad rep is that they can be challenging to play, as they have odd rules.
You forgot the Chaos Marines and Orks, 123birds, which are both either tier 2 or 3, depending upon your point of view.
Chaos Marines still have a lot of VERY effective units. Berserkers, Plague Marines, Daemon Princes, Obliterators, Terminators, Land Raiders, and Rhinos with havok launchers are all rock solid. In the right list, Defilers, Dreads, Vindicators, Sorcerers, and Noise Marines also work very well. However, there are indeed a lot of very mediocre chaos units, and there's nothing in the fast attack section worth taking, aside from maybe a small expendable unit of raptors. This can leave Chaos Marines with few options.
A horde of Orks marching alongside a big mek w/KFF, with a line of Killa Kans in front and lootas in the back makes for a very difficult army to beat, even if your army is tier 1. However, Dawn of War missions are particularly difficult for this list, and if it's also the mission with only 2 objectives (one in each deployment zone), the match will almost always end in a draw. The Orks don't lose much, but they draw a lot, so it depends on your point of view how good they are.
Here's how I view the tiers:
Top: Space Wolves, Imperial Guard, Blood Angels, Grey Knights
Middle: Chaos Marines, Dark Eldar, Orks, Eldar, Witch Hunters, Dark Angels, Black Templar, Chaos Daemons, Tyranids
Low: Necron, Tau
That's basically all you need to know. Yes, there's a lot in the middle, but those armies are really more or less equal.
As for what makes a competitive list, it's having enough of everything to beat any army. You need enough anti-horde shooting (like flamers), anti-vehicle shooting (like meltas), long range shooting (like auto-cannons), and close combat power (like powerfists).
The first casualty of war is neither innocence nor truth.
The first casualty of war is always subtlety.