Your most likely correct and have a mite infestation, it could be a few other things though so I would be getting a magnifying glass and having a good look at the underside of the leaves, mites are not all that fast and should be easy to spot if they are there in any real numbers, look for red or brown specks that move! While you are there keep your eye out for webs (a mite symptom) and any other little crawlies that may be hiding under the leaves during daylight. My best advice would be to just nuke whatever it is early, before the plants get too big and start to fill out (also gives you plenty of time to try a second dose before budding start). "Eco neem" or a similar product should be enough to get rid of pretty much all the likely candidates, just be thorough and make sure you soak every inch of the plants (stems and leaves, top and bottom).