OP states my thoughts pretty much perfectly. HH novels that focus on primarchs are still fantastic. Short story compilations are also great.
But full sized novels about pretty much anything else in the 40k universe are becoming intolerably boring. Part of that is the nature of the universe. As it says on the tin "in the grim darkness of the far future, there is only war". You average SM novel takes that to an extreme and ignores interesting plot, character development, etc.
The only thing that would get me really ramped up about 40k (as opposed to 30k) again would be if they actually advanced the overall plot. Do something with the emperor. Maybe bring back some primarchs as a precursor. Subplot about the purged primarchs. Something. Anything. They'll always have the tabletop but if they don't find something interesting to do after the HH novels are done, they run the risk of losing non-gamers due to the repetitiveness of everything.
Yeah I would love to see some progression, but we have all been told, never gonna happen. Anyway the Dark Eldar and Eldar fluff is pretty cool. That keeps me going.
Although another problem, anytime I read anything that doesn't include SM or Guants Ghosts, I am instantly like, don't give two shits!