NIPM: Allow feeds to use .nipkg files that are hosted on GitHub releases, Amazon Web Services, etc.

[JKSH]

GitHub Releases is a place where we can host downloadable packages for our projects.

NI is already using it:

https://github.com/ni/niveristand-scan-engine-ethercat-custom-device/releases/

Each package is given a nice clear URL like
https://github.com/ni/niveristand-scan-engine-ethercat-custom-device/releases/download/v19.0.0/ni-sc...

How nice it would be if my NIPM feed could point to GitHub-hosted packages! This way, I don't need to maintain my own file server. I created a custom feed to try it out; this is what my Packages.gz contains:

Architecture: windows_all
CompatibilityVersion: 190006
Description: Provides support for the NI Scan Engine and EtherCAT custom device for NI VeriStand 2018.
DisplayName: NI Scan Engine and EtherCAT for VeriStand 2018
DisplayVersion: 19.0.0
Eula: eula-ni-standard
Filename: https://github.com/ni/niveristand-scan-engine-ethercat-custom-device/releases/download/v19.0.0/ni-scan-engine-veristand-2018-support_19.0.0.11_windows_all.nipkg
Homepage: http://www.ni.com
LanguageSupport: en
MD5sum: be9d301d98465ac3157fbb8b6cee370b
Maintainer: National Instruments <support@ni.com>
Package: ni-scan-engine-veristand-2018-support
Plugin: file
Priority: standard
Section: Add-Ons
Size: 38198876
UserVisible: yes
Version: 19.0.0.11

Alas, although NIPM accepted the feed, it did not allow the package to be downloaded ("Redirection to another URL is forbidden"):

It turns out that the nice URL above actually redirects to an Amazon cloud storage location, and the final URL is like

https://github-production-release-asset-2e65be.s3.amazonaws.com/150638172/d0544200-8d25-11e9-9a52-58...

Could the restriction on redirections be lifted somehow? Perhaps there could be an optional property in the feed to specify that a particular Filename is allowed to redirect elsewhere. Perhaps the feed creator can nominate one or more "whitelisted" domains for a particular Filename's redirection.

(originally posted in the NIPM forum)

[pawhan11]

Great idea

[CL_eisenwr]

FYI, NI now has created a NIPM Idea Exchange

[CL_eisenwr]

I just found this page today. Does that mean this has been implemented? Or must the feed and package be at the same location?

[JKSH]

---

@CL_eisenwr wrote:

I just found this page today. Does that mean this has been implemented? Or must the feed and package be at the same location?

---

The feed and package don't have to be in the same location. NIPM has supported this for years (possibly since the first release).

However, NIPM requires the package URL to be static; redirections are not allowed.

What does "redirection" mean? Well, if you send a HTTP request to get a file from this URL:

https://github.com/ni/niveristand-scan-engine-ethercat-custom-device/releases/download/v19.0.0/ni-scan-engine-veristand-2018-support_19.0.0.11_windows_all.nipkg

The server at github.com tells you that you can to load the file from somewhere else. In other words, it redirects you to a different URL like:

https://github-production-release-asset-2e65be.s3.amazonaws.com/150638172/d0544200-8d25-11e9-9a52-585a4992b432?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190701%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190701T040716Z&X-Amz-Expires=300&X-Amz-Signature=c830c14d99beb20df55c4fbabb756d3e179953224b70c1fc7cc021cf74bca9f0&X-Amz-SignedHeaders=host&actor_id=2548865&response-content-disposition=attachment%3B%20filename%3Dni-scan-engine-veristand-2018-support_19.0.0.11_windows_all.nipkg&response-content-type=application%2Foctet-stream

NIPM refuses to download the package from the redirected location.

This idea is asking for NIPM to allow (some) redirections, so that we can host packages on GitHub Releases. This has not yet been implemented.

[CL_eisenwr]

JKSH,

Thanks for the clarification. +1 to this particularly with forth coming community edition and hopefully more people wanting to share code.

[Schliepe]

It would also be great if it could be done from BitBucket, Google Drive, OneDrive.  All kinds of interesting places we all might have access to where we could put things for our use or others.

[JoGra]

@APena

Is there an update on this idea?

[Scott_Richardson]

NI has released Package Manager 2023 Q3 which allows feed URLs to redirect when download package from hosted web services, such as GitHub, and OneDrive.