[SystemLink TDM] Enabling Data Filter causes errors in Analysis Automation

[JKSH]

This is a summary of https://forums.ni.com/t5/SystemLink/TDM-Server-quot-Connection-to-the-Datasource-lt-XYZ-gt-localhost... -- it got quite lengthy and contains red herrings.

Main Issue (SystemLink 2020 R2 + SystemLink TDM 20.1)

If Data Filter is enabled, then the following Analysis Automation Tasks will fail with "Connection to the Datasource: <XYZ>@localhost could not be established. Access denied.":

• Manual Tasks run with "Preconfigured Data Source"
• Triggered Tasks
• Scheduled Tasks

However, Manual Tasks run with "Selected elements in Data Cart" will still succeed. This is despite the fact that the Data Cart items come from the same DataFinder instance that failed before.

Workarounds

The error can be avoided if:

• I disable the Data Filter.
• (Manual Tasks only) I select an ASAM-ODS server instead of a DataFinder instance as the Data Source.

Sanity Checks

My DataFinder instance is definitely unblocked and accessible from other parts of SystemLink:

For more details on how the Analysis Automation was set up, see my steps and screenshots at https://forums.ni.com/t5/SystemLink/TDM-Server-quot-Connection-to-the-Datasource-lt-XYZ-gt-localhost...

Questions

1. Is this a bug in the Analysis Add-On, or have I done something wrong?
2. If it's a bug, will a patch be released for it?
3. If it's not a bug, how do I get the Data Filter to work with my Analysis Automation Tasks?

[JKSH]

Ping?

Is anyone else having these issues on SystemLink 2020 R2 + SystemLink TDM 20.1, or is it just me?

[dmurthy]

Similar problem I am facing but just that the analysis script does not trigger automatically but works manually.

Just curious, are your scripts using datafinders generated before 2020 working?

[JKSH]

---

@dmurthy wrote:

Just curious, are your scripts using datafinders generated before 2020 working?

---

Thanks for the lead. I currently don't have a pre-2020 DataFinder installed so I'll try to get one set up to test it out. Will have to wait till the new year though as our office is closed for the next 2 weeks.

To clarify, did you get any error messages when you tried to trigger an analysis script? (See my screenshots above)

[JKSH]

---

@dmurthy wrote:

Just curious, are your scripts using datafinders generated before 2020 working?

---

I ran tests based on your report at https://forums.ni.com/t5/DIAdem/Diadem-2020-Analysis-script-does-not-trigger-automatically/m-p/41091...

1. On a fresh new VM, I installed SystemLink 19.6 + TDM 19.6. Script (Triggered Task with Data Filtering enabled) works fine.
2. I used NIPM to upgrade to SystemLink 20.1 + TDM 20.1. The same script still works fine.
3. I deleted that VM and started a fresh new one. I installed SystemLink 20.1 + TDM 20.1 directly (without doing any upgrades), and configured the server the same way as I did in Step #1. The same script that worked in Step #1 no longer works. It reports that it can't connect to the DataFinder instance ("Connection to the Datasource: MyTestDF@localhost could not be established. Access denied.")

For Step #3, the script that errored had Data Filtering enabled. If I modify the script to disable Data Filtering, it works fine.

This looks like a bug in the core functionality of a very expensive product. Is nobody at NI willing to investigate this...?

[Brad_Turpin · Accepted solution]

Hi JKSH, dmurthy,

First of all, I apologize that it has taken me so long to come up to speed on the SystemLink 2020 R2 feature that I suspect is behind this sudden feature dropout you've seen.  I really should have been able figure out this symptom you reported (I believe) months ago.  Mea culpa.

SystemLink 2020 R2 introduced a new comprehensive permission gating scheme based on Workpaces and Roles, as configured in the new "Security" widget of SystemLink.  In order to be able to connect to any DataFinder, the user logged into SystemLink needs to be assigned to the correct Workspace and have the "Data Maintainer" role or equivalent.  For the purpose of getting a previous DataFinder working after upgrading to SystemLink 2020 R2, I advise staying with the "Default" Workspace everywhere.  Can you please check in the new "Security" section if the user you logged into SystemLink with (browser log in) has been assigned access to the "Default" Workspace with a "Data Maintainer" role?  This is NOT intuitive.  If you don't specifically navigate to to the Security widget and intentionally add the Workspace and role to your user, you will suddenly stop being able to connect to any DataFinder with that user.  This is true for either interactive connections (DIAdem, Data Navigation web client) or programmatic connections (Analysis Server task, web service call that connects to a DataFinder).

Please let us know what the status is of the user account you use to log into SystemLink.

Brad Turpin

Principal Technical Support Engineer

NI

[JKSH]

Hi Brad,

Thank you for continuing to follow up in such great detail. SystemLink is a rapidly-evolving product so I understand that there's always new things to learn! No apologies needed 🙂

You are right: If I stick to the Default workspace assign the "Data Maintainer" role to the SystemLink TDM User Account, then the error described in my original post above goes away. Your colleague, Eva, gave similar advice a few days earlier.

Eva also mentioned that the permissions setup process is improved in SystemLink 2020 R4. This sounds promising, but I'm currently commissioning the system on 2020 R2; I'll hold off on any major upgrades until everything else is working well.

---

@Brad_Turpin wrote:

If you don't specifically navigate to to the Security widget and intentionally add the Workspace and role to your user, you will suddenly stop being able to connect to any DataFinder with that user.  This is true for either interactive connections (DIAdem, Data Navigation web client) or programmatic connections (Analysis Server task, web service call that connects to a DataFinder).

---

Really? I understand that accessing DataFinder via web services is protected by the Security widget settings, but I'm pretty sure that access via DIAdem is not.

My SystemLink TDM User Account is a local Windows admin account. The SystemLink Server is hosted on the same machine as this account. If I log into Windows as the SystemLink TDM User user, I can use DIAdem's Navigator to browse and perform search queries on that DataFinder instance (even before the permissions were assigned via the Security widget). Is this the intended behaviour?

---

This is NOT intuitive.

---

Agreed.

Due to several simultaneous factors, I struggled to connect the dots and figure out what the error message and various people were trying to tell me:

• As mentioned above, the SystemLink TDM User Account can use DIAdem's Navigator perform search queries on that particular DataFinder without any permissions. That's why I was so convinced that the account already had "DataFinder access rights".
  
  
• The SystemLink TDM User Account does not need the permission "Analysis Automation and Execution > Execute tasks" to execute any tasks. Thus, it didn't cross my mind that it needs the permission "Data Indexing and Navigation > Execute queries
  • I would've thought that "Execute tasks" is a more important permission than "Execute queries" here
    
    
• No permissions are needed at all to execute a Manual Task with Data Filtering when using an ASAM-ODS data source. Thus, I didn't expect to need any permissions to execute the same Task when switching to a DataFinder data source.
  • On that note: Would NI be adding access control for ASAM-ODS in the future?
    
    
• Since the server starts a Triggered Task when it detects a new file, I presumed that the server already knows which file it is and that the server would tell the Task which file to act on. Thus, it did not cross my mind that the Task would still need to execute a DataFinder query.
  • On that note: What is the purpose for query execution by a Triggered Task?
    
    
• Different terminology is used to describe the same thing:
  • The server's error message was "Connection to the Datasource... could not be established. Access denied."
  • Another NI employee told me via email support, "the SystemLink TDM user... needs to have DataFinder access rights."
  • The Security widget calls it "Data Indexing and Navigation > Execute queries"

Knowing what I know now, I can see that everyone has been trying to point me to point me in a particular direction. However, it just wasn't obvious that "Permission to execute queries" == "Permission to connect to data source" == "DataFinder access rights". Newcomers might not make the connection between "DataFinder" and "Data Indexing and Navigation" either.

If NI standardized the terminology and improved the unintuitive bits, that would really lower the barrier for us users.

Thanks for reading. I hope that sharing the sources of my confusion can help other users in the future.

[EvaW]

Hi JKSH,

thanks for your feedback.

Regarding the execution of the query assigned to a task if a triggered task is executed:

The DataFinder triggers the Analysis Automation instance for every new/changed file. And the query for a triggered task allows you to determine whether a task is executed for the file or not. E.g. you want the task to be executed files with author=abc and another task to be executed for files with author=xyz.

Have a nice day,

Eva

[JKSH]

---

@EvaW wrote:

The DataFinder triggers the Analysis Automation instance for every new/changed file.

Ah... that's where my confusion came from. I assumed that the Analysis Automation was triggered after the filter criteria was checked. That makes sense now, thanks.