Philosopher Edward Feser, a Thomist scholar, writes a piece at First Things in which he exposes Krauss' philosophical naiveté and lack of understanding of the things about which he writes.
Krauss wants to argue, according to Feser, that the answer to the question of what caused the universe can't be God because then we can ask the question what caused God. This argument, however, Feser dispatches in a couple of paragraphs by showing how the classical Aristotelians, the Neo-Platonists and Gottfried Leibniz long ago showed that the ultimate cause of the universe cannot itself be something that requires a cause.
Leibniz, for instance, argues that God is not a contingent being whose existence is somehow caused by something else. If he were he would be part of the contingent world and not God at all. God is ab defino a necessary being which is the ultimate cause of all contingent entities. Apart from the existence of this being no contingent things (things which could possibly not be and which depend upon something else for their existence) could exist.
Feser then addresses Krauss' main point:
Krauss’ aim is to answer the question “Why is there something rather than nothing?” without resorting to God — and also without bothering to study what previous thinkers of genius have said about the matter. Like Richard Dawkins, Stephen Hawking, Leonard Mlodinow, and Peter Atkins, Krauss evidently thinks that actually knowing something about philosophy and theology is no prerequisite for pontificating on these subjects.The universe exists and the question is, why? What caused it to exist? Whatever the ultimate cause that cause itself must be something whose existence is not contingent and therefore necessary. That is, it not only must exist, it cannot not exist and cannot be caused by anything else to exist. Such a being, if not the God of traditional theism, is something very much like him.
Nor is it merely the traditional theological answer to the question at hand that Krauss does not understand. Krauss doesn’t understand the question itself. There is a lot of farcical chin-pulling in the book over various “possible candidates for nothingness” and “what ‘nothing’ might actually comprise,” along with an earnest insistence that any “definition” of nothingness must ultimately be “based on empirical evidence” and that “‘nothing’ is every bit as physical as ‘something’”— as if “nothingness” were a highly unusual kind of stuff that is more difficult to observe or measure than other things are.
Of course, “nothing” is not any kind of thing in the first place but merely the absence of anything. Consider all the true statements there are about what exists: “Trees exist,” “Quarks exist,” “Smugly ill-informed physicists exist,” and so forth. To ask why there is something rather than nothing is just to ask why it isn’t the case that all of these statements are false. There is nothing terribly mysterious about the question, however controversial the traditional answer.
For a detailed treatment of this issue read Paul Herrick's reply to atheist philosopher Keith Parsons here.