Take some rough grit sandpaper (100 or less) and sand the rough areas down, then use higher grits (150-600) until you get it as smooth as you want it. This worked for my helmet. I'd recommend wearing a dust mask and eye protection as FG dust is a very nasty thing to inhale, and you definately don't want to get it in your eyes.
I'd have to say Sandpaper as well, even though a dremal is a great tool, it's just that one slip of the dremal & there goes a precious piece cut right down your FG helmet. Just take the time & sand'er down... good luck -