since alminum is so soft, it does tend to clog grinding wheels, your almost better off with a metal type bit like an end mill. Your best bet is to drill a series of holes and to file out the sections that link them to make the oval hole ... makin a barrel extention? 
Sometimes the use of a JIG will help, if you can drill a hole into a flat piece of metal and then clamp that to the tube your trying to drill the hole into, the jig will align the hole and prevent the drill bit from wandering. and if you can make the oval hole in a piece of metal, then you can realign it onto the tube when your filing, and you will end up with a better hole that isn't crooked, or mishapen.
