I find it interesting that you tell us we have no way of knowing, and then proceed to tell us what we have no way of knowing. Forgive me if I assume that you're either a troll or a crackpot.
Why can't it be denied? What if everything we experience (or better said what I experience) is all a creation made by my mind? Sort of like a dream, how would you know? Perhaps the whole universe and everything that I've ever experienced in my life is my own invention and thus malleable by my mind. In said example, the universe is not physical, but an illusion made to exist in.
I had ideas like this too. Please Register or Log in to view the hidden image! Maybe all matter is virtual and we experience reality as if we are avatars in this virtual reality.
The matrix effect, it's possible and we have to way of knowing... I loved that movie so much when it came out