If such a cause exists, of course, then naturalism is false, so naturalists, understandably chary about accepting the conclusion that their metaphysics is false, sometimes take refuge in the argument that the universe is infinitely old, past eternal, or beginning-less. The post linked to above offered scientific reasons for rejecting this argument, but there are philosophical reasons as well.
One of these is that an actually infinite set of any physical entities, whether they be moments, or atoms, or whatever, is probably impossible. Philosopher William Lane Craig explains in the following short video some of the paradoxes that arise in an infinite series of entities and why such a series is highly implausible: Moreover, even if the universe were in fact infinitely old it still could never have arrived at the present moment.
Kirk Durston in an article at Evolution News and Views explains why:
The evidence from science points to a beginning for the universe. Some atheists, understanding the possible theological implications of a beginning, prefer to set aside science and assert that the past is infinite either in terms of the number of years this universe has existed, or in terms of a fantasized infinite series of universes in a multiverse....In other words, if the universe is infinitely old then, if you began counting back from the present moment, you could never count back to a starting point. No matter if you counted forever you would never reach a first moment of the universe. This means, however, that neither could you count forward from infinity past to the present moment. If the universe extends infinitely into the past and contains an infinity of past moments then no matter how many of those moments tick by the present moment would never arrive.
In the real world, an infinite past means that if you were to set the current year as t = 0 and count back into the past, there would never be an end to your counting, for there is no year in the past that was the "beginning." No matter how long you counted, you would still have an infinite number of years ahead of you to count and, if you were to look back at the set of years you have already counted, it would always be finite.
Put differently, in order for a series of moments to arrive at the present there has to be a starting point, but if the past is eternal then the necessary starting point keeps receding further and further into the past and in fact does not exist at all. If there's no initial moment then there's no second moment, and if no second then no third, and so on, and if all this is so, then there is no present moment either. But obviously there is a present moment, so it would seem that the assumption of a past eternal universe is false.
Durston goes into more detail than this, but the implication is clear. If the universe is not past eternal then it had a beginning. And if it had a beginning it had a cause. And any cause of the universe must have the properties listed above, all of which is to say that a finite universe is strong evidence that theism is true.