Now, it doesn't need to be motion of matter or energy present, you can remove all the matter and energy and spacetime will still not freeze over because of these virtual particles. It's not enough to say that matter or energy present in spacetime, has to move because of the uncertainty principle: because even if you remove all the visible matter and energy, you still find energy left. It's a simple, experimental truth that the vacuum will not freeze due to the Brownian motion of particles, or the zero point influence of the ground state fields.