To say that within a given space there are infinitely many possible states of affairs is not to say that literally anything can happen in that space. It only means that everything possible may happen. One might rebut that given an infinite number of possibilities, nothing is impossible. However, this is not correct; a system can still be governed by laws which preclude some facts, while still permitting an infinite number of states of affairs.
For example: Let us grant that language is a purely biological phenomenon, and all linguistic expressions occur in finite physical systems (i.e. systems constituted by material organisms, responding to natural environments by communicating with spoken words, signs, or writing). Many linguists believe that languages are infinitely generative—that is, within any language composed of a finite number of words and grammatical rules, an infinite number of grammatical sentences can be stated. That is, generativity posits that there are finite physical systems—like the brain/body of a fluent language speaker, or a legal pad and pencil wielded by that speaker—which can enter into an infinite number of configurations, corresponding to the infinity of possible sentences expressible and interpretable by a speaker/hearer.
However, some material configurations are impossible. For example, brain-states or pencil marks on a legal pad cannot embody a correct formula for squaring the circle, because squaring the circle is mathematically impossible. That some configurations are forbidden to a system does not necessarily mean that the possible states for that system to be in is finite. Not every infinite set contains every number—there are different, non-overlapping infinities. (e.g. There are more composite numbers than there are prime numbers, but there are infinite quantities of both.)
The set of conceivable states of affairs, which includes worlds with circle-squaring formulae, may very well be larger than the set of possible states of affairs, which contains no worlds with circle-squaring formulae. However, that does not mean the second set is any less infinite (if it is indeed infinite).