To be fair, there were a lot more Xenos empires, some of which were pretty powerful (the Orks being the most pervasive) during the Great Crusade, so I wouldn't say they "had it easy."
Tau, Tyranids, and Necrons? The Imperium wins, easy. Not only is it the golden age of humanity, where the Imperium is completely united, but the Legions are still intact, so no one bats an eye at 10,000 marines marching together, the Primarchs are still around, and so is the Emperor. At this point in time, nothing stands a chance.