Supported Report output formats from Jupyter notebooks?

[Phil.Lundeen]

Is there any documentation on the supported "type" values for Jupyter notebook outputs that can be represented in the Reports views? I see a couple examples in https://www.ni.com/docs/en-US/bundle/systemlink/page/creating-a-new-jupyter-notebook.html which suggest we can have types of "data_frame" and "scalar" (although I can't for the life of me get "scalar" results to do anything useful).

And when the "type" is "data_frame", I see examples that do and do not include a "format" field (e.g. "format": "XY"). But I can't find any source documentation that defines all supported types and formats anywhere.

I have a jupyter notebook that generates an HTML table of data, including links to specific test monitor results in one of the columns, and I'd like to be able to make that HTML table available from the Reports view, rather than forcing users to go in a run the Jupyter Notebook directly. So I'm looking for any available information and/or examples for how I might do that.

[Phil.Lundeen]

Ideally, what I'd like to see is a schema file for the "result" object that is passed from a Jupyter notebook to the Reports view.

[T-REX$]

Hi Phil,

I think this needs to make it into the online documentation, but the best definition of the SystemLink report format that I know is shown in the 'Download Source' content from a report.

The notebook that you download has the following information in markdown:

The result format for a SystemLink report consists of a list of output objects as defined below:
- `type`: The type of the output. Accepted values are 'data_frame' and 'scalar'.
- `id`: Corresponds to the id specified in the 'output' metadata. Used for returning multiple outputs with the 'V2' report format.
- `data`: A dict representing the 'data_frame' type output data.
    - `columns`: A list of dicts containing the names and data type for each column in the dataframe.
    - `values`: A list of lists containing the dataframe values. The sublists are ordered according to the 'columns' configuration.
- `value`: The value returned for the 'scalar' output type.
- `config`: The configurations for the given output.
    - `title`: The output title.
    - `graph`: The graph configurations.
        - `axis_labels`: The x-axis label and y-axis label.
        - `plots`: A list of plots to display mapped from the dataframe's columns, along with configuration options.
            - `x`: The dataframe column corresponding to the x-axis values.
            - `y`: The dataframe column corresponding to the y-axis values.
            - `style`: The plot's style. Accepted values are ['LINE', 'BAR', 'SCATTER'].
            - `color`: The plot's color. Accepted formats are ['blue', '#0000ff', 'rbg(0,0,255)'].
            - `label`: The plot's name, to be shown in a plot legend. 
            - `secondary_y`: Whether or not to display this plot on a second y-axis.
            - `group_by`: A list of columns in the dataframe on which to group data, e.g. to color individual points.
        - `orientation`: 'HORIZONTAL' or 'VERTICAL'.
        - `stacked`: Whether or not to display the plots stacked on top of each other.

---

The reports are directly tied to a graph visualization, so I don't know of any option in the reports section to show an HTML table or other type of visualization here.

[Phil.Lundeen]

Thanks, @T-REX$.

That embedded markdown is helpful, but I don't think it's the whole story. In the help documentation, there is an example that looks like this:

result.append({
    'type': 'data_frame',
    'id': 'data_frame_output', 
    data': [{
        'format': 'XY',
        'x': ['2018-11-17T00:00:00', '2018-11-18T00:00:00', ...],
        'y': [94.0, 89.9, ...]
    }],
    'config': {
        'title': 'Title',
        'graph': {
            'axis_labels': ['x-axis-label', 'y-axis-label'],
            'tick_labels': [{'x': 0, 'label': 'tick label 0', ... }],
            'orientation': 'VERTICAL',
            'plot_style': ['SCATTER'],
            'plot_color': ['blue']
        }
    }
})

 so I think there are more formatting options that that sample markdown points out.

And I definitely agree that the online documentation needs a lot of work.

[T-REX$]

One way that you could achieve your goal of not 'forcing users to go in and run the Jupyter Notebook directly' might be to use grafana dashboards. The output of notebooks can also be used as a datasource for panels on a grafana dashboard.

[T-REX$]

Just wanted to follow-up with some more information on the discrepancy between what's listed in the markdown I posted and what's in the online help.

After talking with one of the developers a bit, it looks like the example shown in the help is from one of the older v1 reports, and the markdown I shared is from a v2 report. While I don't have the full list of configurations for a v1 report output, the v2 configuration I posted above should be complete to my knowledge.

I'm filing a documentation bug to clear up the confusion and to list the new configuration options for v2 notebooks there.