Excel DataPlugin crashes with heavy charts in xlsm file

[alek00]

Hi all, 

I have a data plugin for importing ".xlsm" files whit several worksheets. The dataplugin works fine in most of the situations, and I'm able to extract the information I need by processing the required worksheets (the ones not needed, are skipped). 

However, the excel files to be processed include sometimes ¿heavy? charts (15 signals, 15mil datapoints each) in some of the not processed sheets. When this happens, the dataplugin is not able to even load the file, and crashes 

127 20/11/2023 10:54:03 Error:
Cannot load the file "C:\myexcel.xlsm" with the loader "ImportToDiadem".

More information:
"Error in <ImportToDiadem.vbsd> (line: 20):
Could not open file: C:\myexcel.xlsm"

I have tried to use two file opening options:

1. configure the dataplugin as Filename reader only and then:
   
   Sub ReadStore(Filename)
   Dim Workbook
   [line 20] Set Workbook = OpenSpreadsheet(Filename)
   etc...
2. configure the dataplugin as spreadsheet reader and then: 

Sub ReadStore(Workbook)
etc

In the second case, I simply get a "Cannot load the file" error without any line reference.

I have seen that the issue gets solved when eliminating from the charts included in the ".xlsm" some of the signals or some of the datapoints. Moreover, it also gets solved when the big chart is divided in smaller ones (ie, instead of having a chart with 15 signals, having 3 charts with 5 signals each). 

I tend to think that the issue is maybe related with how the OpenXML defines the charts, but this is just a wild guess, I'm not familiar with the deeps of how Diadem opens excel files. The issue is the same when using .xlsm or .xlsx extensions. 

Do you know if there is any kind of limitation for my use case that I'm not considering? Any ideas on how to solve it? 

I'm using Diadem 2021 Service Pack 1. 

Thanks for your help! Any ideas are welcome. 🙂

[alek00]

Hi again, 

To keep things simple, attached you can find a: 

-A quick example of a dataplugin for reading xlsx (ChartIssue.vbsd, included in the zip file). 

Sub ReadStore(FileName) 

  'Create a File object to process the file containing the data. 

    Dim Workbook, Sheet, Block, Grp, Name, NewChannel, i

    Set Workbook = OpenSpreadsheet(Filename)

    Set Block = Workbook.Sheets(1).GetCellBlock(2, 1)

    Set Grp = Root.ChannelGroups.Add("mytest")

    For i = 1 To 2

      Name = Workbook.Sheets(1).GetCellValue(1, i)

      Set NewChannel = Block.Channels.Add(Name) 

      Call Grp.Channels.AddDirectAccessChannel(NewChannel)

    Next

End Sub

-OK.xlsx file (successful import) 

-NOK.xlsx file (failed import)

The only difference between the two xlsx files is the length of the dataseries in the chart, one has 102637 rows (x and y values) and works fine; the other has 102638 (just one more!) rows (x and y values) and causes the dataplugin to crash.