Actually ignore my question. My transfer speeds are ~6 microseconds/byte (not milliseconds!). This is still a little slower than I would like, but not totally unreasonable! Note the vertical axis in the included plot (pdf) is in ms, so the linear coefficient in the fit equation is 13 microseconds/(16 bit word).