(Definition) A reference frame in which the law of inertia holds is called an inertial frame.
The law of inertia states that the velocity of a particle (= point-like rigid object) may only change if there is force acting upon the particle.
Note: velocity is speed together with its direction; i.e. it's a vector.
Intuitively, this means that if all forces acting upon a particle were "switched off", it would continue to move at the constant velocity it has reached until then. Thus, there is a physical reason to think that not all reference frames are "equal", but some of them are "special", namely the inertial frames.
(Theorem) Given any one inertial frame, exactly those reference frames are inertial frames which are moving at a constant velocity relative to it.
Note: a practical way of switching off forces is to net them out vectorially.