If there is regression going on, I think it's part and parcel of the concept of technological monopoly imposed on the Imperium by the Mechanicus, coupled with the challenges involved in keeping a galaxy-wide empire in sync with all its parts.
Sometimes it takes a month to travel between certain planets. Sometimes it takes a year--or a decade. Sometimes the bureaucracy of the Administratum or the Mechanicus is so bogged down that a century might pass before an emergency is successfully reported to the appropriate authorities.
So, at some point, Forge World Lucius had some sort of emergency that led to their losing the ability to manufacture Baneblade super-heavy tanks. Was that technology lost to the Imperium as a whole? No. Other Forge Worlds continue to manufacture them. In that space of the time, there had to be some reason why other Forge Worlds did not get Lucius that data. According to "Imperial Armour", that's because the Adeptus Mechanicus are very picky about what Forge Worlds built what. Probably out of a desire to ensure that high-value knowledge is not gained by enemy forces, only a few Forge Worlds (likely the most well-defended ones) can build Baneblades.
In the interim, though, a Magos developed the Macharius-class Super Heavy. I imagine this was done because Lucius had obligations to produce X amounts of equipment for Imperial Guard units fighting wars.
The development and approval process of the Macharius took at least 200 years (per the same book). Why did it take 200 years? Who knows. It's not stated. The Mechanicus can be so plotting and cold-hearted that they might have stalled the permission just to see if the Macharius (which was being vetted at the same time) would serve as a cheaper, quicker-to-build alternative. Or it might have been rival Mechanicus factions sabotaging each others' efforts for two centuries (one for the Baneblade, the other for the Macharius). Or the wait (for granting the Baneblade rights) might have been a form of punishment (for losing the knowledge to begin with).
At any rate, by that time, Lucius had received permission to manufacture the Baneblade once more (meaning, the requisite data was sent to them). The Macharius was still seen as a worthy product, though, especially as demand for the standard Baneblade outstripped Lucius' supply of those super-heavies. It ended up serving as a viable alternative.
So, was there regression here? Temporarily, and even then it was more of a lateral move, or, if you will, a temporary setback, rather than a true regression. Are there cases of full-on regression? Yeah... but I would argue that, when the novels state...
"Forget the power of technology and science, for so much has been forgotten, never to be re-learned..."
... that they're referring to things lost and forgotten before the Imperium came about.
Or, when Battlefleet Gothic states that ...
"The superior technologies used to construct the multiple lance turrets on the Apocalypse have now all but been forgotten and so the small number of examples of this ship can never be replaced."
... I think this is basically along the lines of what the fluff states about Baneblades. Meaning, if there are only X number of Forgeworlds trusted with manufacturing Baneblades, than the number of Forgeworlds/dockyards worthy of building an Imperial Battleship had to be minute indeed. Of those, I doubt any one makes more than one class of Battleship, and it would probably be a pretty monumental effort to convert dockyards to produce technologically more complex, superior warships and lance arrays. In this case, I think what we're really talking about is a cost vs. benefit issue.
Are there cases where knowledge is lost... as in, jet-bikes? Sure--even if I'm pretty that, should the Master of the Ravenwing be cool enough to turn in his jetbike, the Mechanicus could figure it out inside a few centuries.
More often than not, though, I'd argue that the complexity of the Imperium, which magnifies logistical problems exponentially; the veneration of technology, which lengthens the process of vetting and approving new products and processes; and the state of constant war, which shifts the priority to production of proven, dependable products versus research of new, superior things; are the real reasons you're seeing stagnation (as opposed to actual regression).
Cheers,
P.