it just sounds like the inside of the helmet is lumpy. with most slush cast stuff, the wall thickness is pretty uneven. you need to sand the inside down until the thickness is consistent across the whole opening.
any helmet i've made is like that... i spend about 10 minutes with a dremel smoothing the inside of the visor opening.