JJS might have the answer, but, it could also be a bunk countersink. Some of the cheaper ones can chatter pretty bad, running slow or fast.
Is it a multi-fluted CS or a single fluted ?
Generally, for larger holes, I prefer a single fluted CS.
For my lanyard tubes though I have a couple of good quality 3/8" (major diameter) multi-fluted CS's that have been working really nice for me. I use them in a DeWalt VS rechargeable drill, running fairly slow, and they cut a very clean countersink.
After CS 'ing I have an 1/8" brass split pin that I run in a #30 Foredom handpiece. I slip a strip of 320 grit SC paper into the pin, roll the strip around it, insert it into my lanyard tube and polish the ID.
First the CS, then the ID polish makes for a nicely finished lanyard tube.
2thumbs