Welcome.
From a young age, I regarded this question of existence - why does anything exist? - with deep curiosity.
Yeah, me too. I've always had a strong sense of the... contingency... of everything. Not just our own lives and all the physical objects around us, but space, time and the more abstract regularities that we call 'physical laws' as well.
I think that the question: 'Why is there something rather than nothing' is the ultimate and fundamental question of ontology.
It's possible that science has something to offer us here, in terms of our current understanding of the laws of the physical universe, and yet to my mind scientific explanations fall short of answering the underlying philosophical question.
I agree. Science is very good at explaining why things are this way rather than that way, given preexisting conditions and the more abstract and general principles of nature. But it's not very helpful in telling us why nature exists in the first place and why it has any principles at all. Employing science to answer the fundmental ontological question is almost certainly going to involve us in circular reasoning.
Frankly, I don't know of any way of answering the fundamental question of existence. Attempts to address it seem to involve us in circularity, infinite regress or some kind of contradiction.
Short of finding an answer to the question "Why does anything exist?" I have found it more fruitful to give it - the reason or explanation for material existence - a name.
This is the point in your post where I'm going to start disagreeing with you. Naming the unknown answer to the universal question is going to be misleading by its very nature.
For one thing, it assumes that there IS an "answer". But one of the possibilities is that this might turn out to be what philosophers call a "pseudoproblem". In other words, we might be misconceiving the issues somehow, and confusing ourselves into thinking that there's a question that needs answering when there really isn't.
The name itself is unimportant, but giving it a name at least enables further constructive consideration.
If we must name it, I prefer something like "the unknown explanation for being itself, for why existence exists in the first place". That's cumbersome, but it sums up the issue while remaining clear that we don't actually have a clue about the answer.
I call it "Reforme". What then can I say about Reforme? Reforme is just a name, a label, which I give to something I do not understand. Those of a religious persuasion might equate Reforme with God. Personally I have no truck with theology or religion of any kind. And yet, despite having no idea what Reforme is, or what the nature of Reforme might be, I regard Reforme with wonder and delight, verging on awe. For there is scarcely anything more amazing, wonderful and awe-inspiring than the magnificent reality of the physical universe.
Though you deny your own religiosity, that sounds like a religious statement to me. You're free to feel that way (I kind of feel that way myself sometimes, when I'm confronted with the cosmic mysteries) but I think that you are getting far out in front of the evidence and are kind of reintroducing the old 'God' concept under your new name.