Assuming that there isn't a memory leak in the activex object you're accessing (a very real possibility), you are probibly not closing a reference--though there could be something about how you are doing the conversion that is causing the problem. If you post your code I could get a better idea.
In the mean time, you might want to consider getting into the habit of routinely closing any references you open. Sometimes a close isn't required, but attempting to close a reference when the close isn't needed doesn't hurt anything.
Note that if you read a property that returns a reference (or an array of references), or invoke a method that returns a reference, those references definately need to be explicitly closed.
Mike...