Maps and Config Files

[drjdpowell]

I agree, for the classic INI file, because such a file is very, very simple in structure, with simple independent option organized in groups.  But you're starting to break those rules with your map thing.

[paul_cardinale]

---

@drjdpowell wrote:

I agree, for the classic INI file, because such a file is very, very simple in structure, with simple independent option organized in groups.  But you're starting to break those rules with your map thing.

---

On the outside, I have the simplicity of .ini files.  On the inside I have maps.  No rules are being broken.

[paul_cardinale]

---

@drjdpowell wrote:

---

@paul_cardinale wrote:
I consider that a bad compromise.  Readability is unpleasant.  Try explaining to a non-tech how to change a setting in a JSON file.  

---

Seems very readable to me.  An example:

{
  "Module_DB file":"",
  "SQLite busy timeout (ms)":10000,
  "Analysis Configuration":[],
  "Column Widths":{
    "AnalysisName":82,
    "Camera":65,
    "Colour":46,
    "ImageIndex":21,
    "InValid":42,
    "Pattern":52,
    "TestTime":131,
    "unitID":111
  },
  "Overlay Options":{
    "ID":true,
    "Alignment":{
      "Alignment Markers":"Input and Output",
      "Edge Detection":false,
      "Circular IG":false
    }
  }
}

---

@paul_cardinale wrote:
And from a programming standpoint, the code wouldn’t be as clean as using a binary file.

---

There is no way an OOP-based config system is going to be as clean to code as using a good JSON library.  

---

Yes there is.  My system is very very clean.

[Jay14159265]

---

@paul_cardinale wrote:

I started using maps (finally).  Needing to initialize them from .ini files, I made this.

---

Thanks for posting this, good code idea. After playing with the polymorphic VIs I ended up dealing with the raw strings over doing the type conversion at the time of the read. Just load it all into memory and type convert as needed. I don't want to take the hit on so many reads when the system is starting up so just convert the entire INI file to a MAP and grab info as needed. 

 

[wiebe@CARYA]

---

@Jay14159265 wrote:

---

@paul_cardinale wrote:

I started using maps (finally).  Needing to initialize them from .ini files, I made this.

---

Thanks for posting this, good code idea. After playing with the polymorphic VIs I ended up dealing with the raw strings over doing the type conversion at the time of the read. Just load it all into memory and type convert as needed. I don't want to take the hit on so many reads when the system is starting up so just convert the entire INI file to a MAP and grab info as needed. 

---

You literally force each and every key to be read at startup.

So that actually does take the entire hit when starting up.

Not that it's a bad thing to do, it just doesn't match the reason why you do it. Your access after loading might be faster (is it?), at the expense of the hit at startup. Not putting everything in a map will speed up loading, at the expense of access for each read\write.

A compromise would be add keys to the map when read\written and if it's not in the map when read, read it from the ini and put it in the map. Then you'd have fast startup and fast key access, except for the first key access.

Most performance when using the config file library is probably string/type conversion. Storing strings in a maps won't solve that. Unless the configs get big, then there might be sting array searches slowing things down.

[Jay14159265]

---

wiebe@CARYA wrote:

---

@Jay14159265 wrote:

---

@paul_cardinale wrote:

I started using maps (finally).  Needing to initialize them from .ini files, I made this.

---

Thanks for posting this, good code idea. After playing with the polymorphic VIs I ended up dealing with the raw strings over doing the type conversion at the time of the read. Just load it all into memory and type convert as needed. I don't want to take the hit on so many reads when the system is starting up so just convert the entire INI file to a MAP and grab info as needed. 

---

You literally force each and every key to be read at startup.

So that actually does take the entire hit when starting up.

Not that it's a bad thing to do, it just doesn't match the reason why you do it. Your access after loading might be faster (is it?), at the expense of the hit at startup. Not putting everything in a map will speed up loading, at the expense of access for each read\write.

A compromise would be add keys to the map when read\written and if it's not in the map when read, read it from the ini and put it in the map. Then you'd have fast startup and fast key access, except for the first key access.

Most performance when using the config file library is probably string/type conversion. Storing strings in a maps won't solve that. Unless the configs get big, then there might be sting array searches slowing things down.

---

Thanks for the insight. There is soemthing maybe I don't understand about the Config File Lib. I'm of the impression that each time you open - get key - close using the Config File lib you are, underneath the hood, opening a file, parsing it, then closing it. So if you need to load many settings you will be doing many open - read - close file I/O operations. Is this not the case? It would seem that if this is the case it would be better to take the open - read - close file I/O operation hit once and then operate off an object in memory. 

A few years back I ran into an issue using the LabVIEW XML library to read in settings. It turned out that it was quite inefficient to the point that it was much faster to simply read the XML file in as text and parse it using my own XML library. After that I figured the Config File lib probably suffers the same or similar inefficiency though I did not test this. 

[wiebe@CARYA]

---

@Jay14159265 wrote:

---

wiebe@CARYA wrote:

---

@Jay14159265 wrote:

---

@paul_cardinale wrote:

I started using maps (finally).  Needing to initialize them from .ini files, I made this.

---

Thanks for posting this, good code idea. After playing with the polymorphic VIs I ended up dealing with the raw strings over doing the type conversion at the time of the read. Just load it all into memory and type convert as needed. I don't want to take the hit on so many reads when the system is starting up so just convert the entire INI file to a MAP and grab info as needed. 

---

You literally force each and every key to be read at startup.

So that actually does take the entire hit when starting up.

Not that it's a bad thing to do, it just doesn't match the reason why you do it. Your access after loading might be faster (is it?), at the expense of the hit at startup. Not putting everything in a map will speed up loading, at the expense of access for each read\write.

A compromise would be add keys to the map when read\written and if it's not in the map when read, read it from the ini and put it in the map. Then you'd have fast startup and fast key access, except for the first key access.

Most performance when using the config file library is probably string/type conversion. Storing strings in a maps won't solve that. Unless the configs get big, then there might be sting array searches slowing things down.

---

Thanks for the insight. There is soemthing maybe I don't understand about the Config File Lib. I'm of the impression that each time you open - get key - close using the Config File lib you are, underneath the hood, opening a file, parsing it, then closing it. So if you need to load many settings you will be doing many open - read - close file I/O operations. Is this not the case? It would seem that if this is the case it would be better to take the open - read - close file I/O operation hit once and then operate off an object in memory. 

A few years back I ran into an issue using the LabVIEW XML library to read in settings. It turned out that it was quite inefficient to the point that it was much faster to simply read the XML file in as text and parse it using my own XML library. After that I figured the Config File lib probably suffers the same or similar inefficiency though I did not test this. 

---

A, yes. If open the file each time, it will be a lot of overhead, and your solution does help for that.

Usually, you'd either

+ open once, pass the reference, and read\write using the reference. Or

+ open, convert to internal structure, close (your solution)

I modified the config lib decades ago, to use a string as reference, in stead of a reference. But as that's basically a global, I don't use that anymore. It had other features, like the ability to save without closing.. 

I now use (and developed) GitHub - Configuration File Library for LabVIEW. It is OO and so I can do funky stuff, like updating menu's (recent files) whenever I write to my 'recent files' key. This reminds me to update the online library...

[Jay14159265]

---

---

wiebe@CARYA wrote:

---

I now use (and developed) GitHub - Configuration File Library for LabVIEW. It is OO and so I can do funky stuff, like updating menu's (recent files) whenever I write to my 'recent files' key. This reminds me to update the online library...

---

Thanks for posting this, I knew someone out there made a better Config file tool : ) 

[wiebe@CARYA]

---

@Jay14159265 wrote:

---

---

wiebe@CARYA wrote:

---

I now use (and developed) GitHub - Configuration File Library for LabVIEW. It is OO and so I can do funky stuff, like updating menu's (recent files) whenever I write to my 'recent files' key. This reminds me to update the online library...

---

Thanks for posting this, I knew someone out there made a better Config file tool : ) 

---

There's a presentation about it too (not sure if the repo points to it):

(2) 2018 NIWeek Wiebe Walstra Case Study In OO Design Building Flexible INI Library - YouTube

[Yamaeda]

---

wiebe@CARYA wrote:

---

---

You literally force each and every key to be read at startup.

So that actually does take the entire hit when starting up.

Not that it's a bad thing to do, it just doesn't match the reason why you do it. Your access after loading might be faster (is it?), at the expense of the hit at startup. Not putting everything in a map will speed up loading, at the expense of access for each read\write.

A compromise would be add keys to the map when read\written and if it's not in the map when read, read it from the ini and put it in the map. Then you'd have fast startup and fast key access, except for the first key access.

---

You must read a silly amount of keys for that to be a significant impact i'd say.