Muliple USRP RIO 2943R, synchronization and phase lock problem

[brooksprumo]

What are your start triggers configured as? 

When using the PPS Trig signal-based synchronization, all synchronization signals are muxed over a single physical wire. Because of this, you should only have either Rx or Tx synchronization enabled. Here's a better explanation:

1. Configure start trigger on USRPS
   • For the Rx USRPs,start trigger = Immediate
   • For the Tx USRPs, start trigger = Rx Start Trigger
2. Synchronize 
   1. Disable the sync trigger on the Rx side for *ALL USRPs*
      • You'll need to modify your code to do this. I recommend looping through all 8 USRPs and disabling trigger sync on both Rx and Tx to be safe.
   2. call the niUsrpRio Sync VI
      • This is the green VI you're already calling
   3. Enable trigger synchronization on only the Rx side for *ALL USRPs*

With this, all USRPs will receive a synchronized Rx Start Trigger, and the Tx side of the four USRPs will pick it up. Does that make sense? If not, I can try to mock up a VI.

[True_Boombo]

Hi Brooks,

Your suggestion is always very constructive. 

Here I have one quick question about your suggestion:

As wrote:

Configure start trigger on USRPS

• For the Rx USRPs,start trigger = Immediate
• For the Tx USRPs, start trigger = Rx Start Trigger

In the test labelled red, if I set start trigger = Rx Start Trigger, it means I need to enable transmitting and receiving function within same USRP box, I think I cannot enable TX by using Rx Start trigger from another USRP box.

In this case, there will be transmitting signal and receiving signal in one USRP box, which introduces self-interfere on Rx radio from Tx radio. 

That's why I try to configure 4 USRPs as Tx while the other 4 as Rx - to avoid self-interference.

Anyway, I will try to make change according to your suggestion, and test first. Will ask for help if I cannot achieve the goal.

Thank you very much 

Cheers,

Bo

[brooksprumo]

---

@True_Boombo wrote:

Configure start trigger on USRPS

• For the Rx USRPs,start trigger = Immediate
• For the Tx USRPs, start trigger = Rx Start Trigger

In the test labelled red, if I set start trigger = Rx Start Trigger, it means I need to enable transmitting and receiving function within same USRP box, I think I cannot enable TX by using Rx Start trigger from another USRP box.

In this case, there will be transmitting signal and receiving signal in one USRP box, which introduces self-interfere on Rx radio from Tx radio. 

---

You don't need to receive on the four USRPs you plan to only transmit on. Setting the Tx USRPs' start trigger to Rx Start Trigger is purely a trigger issue and an artifact of only have a single physical trigger between the USRPs. For example, if you look at the FPGA VI for the AUX I/O signal-based synchronization, you'll see that the Rx trigger is on a different physical wire than the Tx trigger; this prevents the triggers from coliding. The configuration I suggested above it to work around the physical wiring limitations.

[True_Boombo]

Hi Brooks, I got your idea. I also checked AUX FPGA VI. it use AUX9 and AUX10 for Tx and Rx trigger respectively. 

I am changing the code now, will update the testing result later

[True_Boombo]

Hi Brooks,

I made modification according to your suggestion, If my understanding is right, the modification is quote straightaway as show in following pictures:

The testing result: I ran the host VI. the VI can run, but the on the receving channels, I can only receive some noise via these receving channels (as on the transmitting side, I sent the CW sine wave as you already see in the previous post).

Did I do anything wrong on the sync configuration?

Cheers,

Bo

[True_Boombo]

Hi Brooks,

I think there may be something not right in the first modification, I think following modification should be right:

But, actually with above configuration, there is still noise on the receiving channels. 

Cheers,

Bo

[brooksprumo]

I think you should keep version 1, as I think you need to enable Rx Sync on all 8 USRPs. I'll have to investigate this more tomorrow. By the way, what version of the USRP driver are you using? There are some trigger bugs that happen if you're trying to trigger Rx and Tx from eachother--which is what you're doing.

[True_Boombo]

Hi Brooks,

I am using USRP Driver 15.5. It is a beta version driver. I applied to use it from NI as I am trying to using UBX160 for wider bandwith when I design the system'.

I did some more tests this morning:

For all tests, I was using the version 1 "configure synchronization" modification as you suggested. I tested 2 different top level VI changes:

1. after "confugure synchronization", i have one "niUsrpRio Session (out)" handle which includes 8 USRP. I feed this handle to both receiving loop and transmitting loop, as shown below

 The testing result of this method: The receving channels only receiving noise . seems like the Tx channel did not transmitting signals

2. I separate the "niUsrpRio Session (out)" (8 USRP) to "Rx niUsrpRio Session" (4 USRP) and "Tx niUsrpRio Session" (4 USRP), and then feed them to Rx loop and Tx loop respectively, As shown below:

The testing result of this method: We can observe signals from all rececing channels. show phase lock and time sync when the IQ rate is lower (below 40MS/s), when go over 60MS/s, it shows same behavior as I described in video.

As described in this document http://zone.ni.com/reference/en-XX/help/373380D-01/usrphelp/synchronization/

I feel like the the trigger should be already aligned by the FPGA. 

I also confused about which sync method we are exactly using. At begining , I think the sync method I am using is FPGA self-synchronization. When I check the "configure synchronization"  subVI, I notice we are using a block called "Host Driven" (The green subVI you mentioned before). 

Cheers,

Bo

[brooksprumo · Accepted solution]

15.5, OK.

The signal-based synchronization Sample Project uses the host-driven synchronization VIs by default. You can switch that, but you'd have to switch the VIs on the FPGA too, and then recompile. I feel the synchronization VIs are working in your design, and I would not recommend changing them now.

It may be worth trying out the multi-device Time-based synchronization Sample Project. There is a built-in "Multi Device Tx and Rx Streaming" host VI already. The Time-based synchronization does not have triggering dependencies between the Tx and Rx sides, which can result in more robust triggering. Make sure to set your OctoClock's switch to Internal instead of External so that the OctoClock generates a PPS.

With your current code, I'm not sure why at 60 MS/s the received waveforms seem to move around and at 40 MS/s they do not. It might be the Start and Acquire/Generate VIs, but I'm not sure what to look for next.

Here's what I think you can try now:

1. Use your current design and try out one Rx and one Tx USRP. If there were PCIe throughput bottlenecks, this should elimate that. If this observed behavior is *different* with two USRPs instead of eight USRPs, then it might be a PCIe bandwidth issue.
2. Try the multi-device Tx and Rx SPr. Again, try with just two USRPs first. You can start off with both USRPs transmitting and receiving, which will amount to running to top-level host VI without any modifications. After that, you can split the USRP RIO sessions--Rx USRPs to the Rx loop, and Tx USRPs to the Tx loop--and see if that works.

Hopefully we can figure something out with these two things. Let me know how it goes!

[True_Boombo]

Hi Brooks,

Thanks for your reply.

I made some quick responses on "Multiple Device Time-based synchronization Sample Project" as I had tried this example before I move to trigger sync.

1. I tried "single device time based Tx" and "single device time based Rx" in separate VIs, sharing same 10Mref clk and PPS. The result: phased locked and time synced

2. I tried "single device time based Tx" and "multiple device time based Rx" in separate VIs, sharing same 10Mref clk and PPS. The receiving channels show similar result as we discussed.

3. I tried "Multiple Device Time-based synchronization Sample Project", I think this example does not fit my application. As I mentioned before, this project has to configure the Tx and Rx in same USRP box, If you check the Rx config panel of this example project, you may notice that it is using Tx start trigger to trig Tx channels. I need to change the configuration before I test this example

For the PXI bottle problem

When I design this system, I included 2 CPS 8910 and 2 PXI 8384, one set for Tx stream, another set for Rx stream, to avoid the PXI interface throughput limit. Each PXI 8384 is connected with CPS 8910 with X9 MAX cable.

I tried to run 1Tx usrp and 1Rx usrp with my current hardware and software setting, I found I can see good sync and phase lock alright at 60MS/s, but if I increase to 80MS/s, the problem appears again.   

Cheers,

Bo