If when you first put it on you say to yourself, "It fits a little tight", then that is what you want. When I bought my helmet recently I though for sure I needed an XL based on my measurment...tried on some XL and they were very comfortable but as they wore in they would have been too big. The L has broken in over the last 3 months and is just about perfect now. You don't want your helmet sliding around on you when your turning your head at high speeds or even more so in a fall.
In short, if it feels a little loose in some areas get the next smaller size and let your head/sweat break in the padding to mold around your head.
<!--EDIT|Over_Easy
Reason for Edit: None given...|1155825454 -->