"Primary Matter" to my recollection was physical matter, but divorced from the soul or other divine spark that made it, in its true essence, what it is. As a crude example, imagine a chair. Whatever chair you imagined is really just a potential chair, it is the "form" of a chair, but without primary matter that gives it physical reality.
Imagine that form is a real thing and that it is only because the form exists that chairs--any chair--can exist. This is how many of the ancients imagined the world (and is key to understanding things like, alchemy, for example and why they thought that only four Empedoclean elements were needed to make every substance that exists...because earth air fire and water were physical only--primary matter--and there was no end to the forms they could take.)
Now imagine taking a physical chair, but extracting from it that "essence" or "form" of "chairness". Not merely smashing the chair to bits, mind you, But simply separating from the crude physical matter the underlying essence of what it is to be a chair. The formless physical matter that would be left behind is the primary matter of that (former) chair.
Edit: I should add, in case there is confusion...generally, to be "primary matter" the physical matter in question would have to be devoid of all form. So imagine a wooden chair, and take the "chairness" out of it...what you are left with is not primary matter because it's still "wood". If you extract the "woodness" (the essential intangible essence of what it means to be "wood") from it, and any other form the remains may hold, the those remains are "primary matter." (Now, why "wood" was a form and the "earth" "air" "fire" and "water" were not, I have never understood. There is a discussion of it by Paracelcus, but I couldn't follow what distinction we was trying to make.)