You should sweep the entire area, just as you would do if you were mopping your kitchen or bathroom.
Also, make sure to sweep or dust your baseboards before cleaning them. This will help ensure that there is no dirt or dust stuck to them.
A clean paint brush works well if you don’t own a handheld broom. It may actually work better than a handheld broom.