A post at Uncommon Descent has a graphic that illustrates quite nicely why many philosophers believe that the universe cannot be infinitely old:
The major reason for thinking the earth is infinitely old is a metaphysical one, not a scientific one. It's the desire to avoid a beginning to the universe. And the reason why some wish to avoid a beginning to the universe is because a beginning implies a transcendent cause and that's starting to sound a lot like Genesis 1:1 and that's just unacceptable.
There's a principle most philosophers accept called the Principle of Sufficient Reason (PSR). This principle states that every effect must have a cause adequate to account for the effect. If this is so then a universe that had a beginning must have had a cause adequate to account for the universe we see.
A cause of the universe would have to be extraordinarily powerful and intelligent. It would have to transcend space and time (which are part of the fabric of the cosmos), and, since the universe has generated personal beings like us it's reasonable to assume that the cause of this is itself personal.
Of course one could deny all this by denying the PSR, but that seems to me to be a pretty steep intellectual price to pay to avoid having to acknowledge that there is a Creator. It would, in effect, destroy science which is based upon the PSR.