Because Earth is not a perfect sphere and the Sun and Moon act on it.
"The precession of the equinoxes is caused by the gravitational forces of the
Sun and the
Moon, and to a lesser extent other bodies, on the Earth. It was first explained by Sir
Isaac Newton.
[19]
Axial precession is similar to the precession of a spinning top. In both cases, the applied force is due to gravity. For a spinning top, this force tends to be almost parallel to the rotation axis. For the Earth, however, the applied forces of the Sun and the Moon are nearly perpendicular to the axis of rotation.
The Earth is not a perfect sphere but an
oblate spheroid, with an equatorial diameter about 43 kilometers larger than its polar diameter. Because of the Earth's
axial tilt, during most of the year the half of this bulge that is closest to the Sun is off-center, either to the north or to the south, and the far half is off-center on the opposite side. The gravitational pull on the closer half is stronger, since gravity decreases with distance, so this creates a small torque on the Earth as the Sun pulls harder on one side of the Earth than the other. The axis of this torque is roughly perpendicular to the axis of the Earth's rotation so the axis of rotation
precesses. If the Earth were a perfect sphere, there would be no precession."
https://en.wikipedia.org/wiki/Axial_precession#Cause
Or don't you "buy that" either?