It is not that it does not work, and as I explain in a later article, it is that when we are fixing time, we are fixing to an unknown functionality and quality, and probably that is not a good call. Of course, you can make deadlines as predictable as you want, but that means that you are sacrificing other things. Also, if you nail all estimates, even with good quality, probably you are wasting resources by not automating those parts that can be automated.

What you ask about to make it more "engineering" and less art, that is precisely what DORA metrics are about. With the Accelerate book in hand, they become the first real tangible results about all pseudoscience around software development. The Accelerate way of defining success and organizing software development, is no longer of just satisfying some deadlines, rather than delivering the maximum value. But that requires such a big shift in mindset that I have compared it to two different psychological trends.

So, at the end, the question is, what do you want? Predictable deliveries or value?

EDIT: I found the video of Dave Farley in which he opposes to predictability, he firmly exposes predictability as a very ineffective way of building software.