Speed up listbox

[Shako]

Hi

I created a program which takes a txt file and parses it to a list box.

Im using a listbox so that I can get that line text.

I have a problem when that txt file is very large e.g.. (140MB - 1000000 Lines) it takes about 40mins to parse the file.

Is there a way I can speed it up ?

Thanks

Shako

[ebalci]

Hi Shako,

I can suggest using a textbox and using the SetCtrlAttribute(... , ... , ATTR_CTRL_VAL, ...) function instead of SetCtrlVal.

At the end of your loop call ProcessDrawEvents to refresh the screen.

Otherwise, by using SetCtrlVal in the loop, the screen is repainted after each write and this can be significant for your case.

Let us know if it works.

[RobertoBozzolo]

Ehm, Shako, beside listbox or textbox performance, I can see another problem in your application...

Are you *sure* you want your user to deal with a one-million-lines archive in a single control?? The most experienced user will loose itself with such a huge volume of data!!!

[Shako]

I tried what you said ebalci, there is no difference in speed, compared to using InsertListItem() and InsertTextBoxLine().

Im still not sure why it is slow.

But i done a test, i read the file line by line and saved it to another file.

The result where just about a minute or two faster.

But it still takes about 30 - 35 minutes to do this.

 Roberto Bozzolo

The files information has to be inserted no matter how long.

The files content is a log of what the user has done e.g key press, selected values, screen display values etc.

Which my software emulates.

[RobertoBozzolo · Accepted solution]

Shako, screen access is notoriously one of the most severe bottleneck in program activity, and file I/O is close to it. Given this, I can suggest you some alternatives that may help you: they are implemented in the simple attached project which you may use to experiment on your data file.

The first improvement is to hide the listbox / textbox while you are updating it: this way screen access is limited to the single operation of making the control visible at the end of process. This trick will work only if there is no call to ProcessDrawEvents ()  or ProcessSystemEvents () during the process, either explicitly issued (e.g. in an concurrent timer callback running) or implicitly exiting from a callback.

Another improvement is to read the file in binary mode in large chunks, adding data to the textbox as they are read.

The third improvement, limited by available system memory, is to read the whole file in a single pass.

On a 10mb text file I have on my machine results are as follows:

• binary read in 4095-bytes chunks with the control visible: 210 sec
• Same as above with control hidden: 120 sec
• Single-pass binary read (textbox visible): 0.34 sec
• Same as above with control hidden: 0.18 sec

In case of limited system memory you can tailor this mechanism to read the file in 3-4 chunks: I suppose you will get a very good performance!

It is to be noted, though, that I am not performing any operation on data read: no line splitting, no parsing... nothing at all. Even single tabs embedded in your data, if any, are not honoured by the textbox. That is to say: switching to a listbox is not a feasible option, as it requires you at least to split data into separate lines.

[Shako]

Thanks.

Reading the file into a chunk of memory works brilliantly.

Dont know why i didnt think of that.

The only problem i have now is that i used the list box so that when the use clicks on a line i could get the text for that line and emulate it.

Is it possible to get text from a line that the user clicks on ?

[RobertoBozzolo]

You should be able to obtain it by using:

GetCtrlAttribute (..., ..., ATTR_TEXT_SELECTION_START, &offset);

GetTextBoxLineIndexFromOffset (..., ..., offset, &lineIdx);

GetTextBoxLine (..., ..., lineIdx, buffer);

[Shako]

Thanks for all the help.