No. That's exactly the right idea, but it doesn't depend specifically on light being photons. The same argument applies to light waves.
This was exactly the premise behind the Michelson-Morley experiment. Looking at things in the reference frame of the interferometer itself, the effect should theoretically look as if the speed of light varies, depending on the motion of the interferometer through the "ether".
The result of that experiment, of course, was that no variation in the speed of light was observed: a null result. The conclusion is that there's no "ether". Moreover, the result can be explained using Einstein's theory of relativity.