Most folk waste time in one way or another at work , the difference is smokers are targetted because of social stigma (and the ease in proving they are not working).
Besides they might stay in the office later to finish work off, they may skip lunch... There are numerous times when we all work extra time and it is not noted. And I've seen non-smokers take 10 minutes to go to the bog
Some folk are lazy other aren't so.