Thanks for that link.
I fixed my problem, but now I'm wondering why what I did fixed it. Sampling the channels in the other order fixed it, but I don't see any reason why it should. Both of the input devices have identical resistive and capacitive loads, the signals only differ in magnitude by about 50%, and I have the same gain settings for both channels. Is there any other sort of asymmetry that would cause that fix to work? I don't like the idea of using this without really understanding why it's working.