Jennifer,
There are a couple reasons that your constellation plot could be circling.
1) The noise introduced by the physical channel causes the
demodulator to lose carrier locking. If this is the case,
switching to a lower M-ary should fix the problem. This sounds a
lot like what you are describing. How have you verified that
there is not enough noise in the channel for this to be the case?
2) If you're only hitting a small number of the symbols on the
map, the receiver can have a hard time locking to the actual generated
phase, and the same result can occur. This is called a
false-lock, and is quite common in higher M-ary modulation
schemes. The solution to this would be ensure generation of all
symbols by padding your data with a PN Sequence of a high order
(N>13). To test this out, try sending just a PN Sequence
instead of user-defined data. If the problem goes away, then try
reintroducing your data padded with a PN Sequence to hit all the
symbols.
Best of luck, Jennifer! Let me know how this goes.
Travis W