Advanced Search

I tested now and get same behavior as you. This was some small news for me. I thought it only started reconnecting if it had already been in use already by LinuxCNC.

The test was like:
1. Pendant off, plug dongle into computer. No disconnects for 5 minutes.
2. Turn pendant on. Disconnects start with around 20 s cycle.
3. Turn pendant off. It looks like dongle stays disconnected. Maybe just bad/good luck with the timing.

I already made some first steps for a new driver. The initial target of that was to have reliable polling. I already abandoned it and started thinking more about the firmware. But maybe I should give it another shot. I'll try to have a look tomorrow if I can clean it up just slightly and post it here. It's simple enough to play around with and it should be able to provide stable polling.

If you just have the dongle and pendant attached and switched on do you get resets every 20 sec?
I mean, without linuxcnc.

I took the dongle and pendant to a bit newer N150 PC. Plugged it in. Ok, was discovered.
Switched on the pendant and it started with the 20 sec disconnects/reconnects.
Switched off the pendant with the button and it stopped. There is a clear relation.
It can be related to polling, I guess.
Searched for a simple program to regularly poll a usb device.
Best I found was "usbreset" but it didn't change anything.

Interesting that you MRx says it is stable. Advice was to test with USB 2.0 and I have looked
and it isn't easy to find a PCI-E USB 2.0 board I can get in a few days. My motherboard
Gigabyte Z170N-WIFI does have an internal header for USB 2.0/1.1 and I should get a cable
and test I guess.

I had/have very similar issues as you mention earlier today Hakan. The after the pendant disconnected once, it kept doing so every 20 s or so, just like for you. And you see the LIBUSB_ERROR_NO_DEVICE because the dongle has disconnected. However, if you again start LinuxCNC (maybe need to try a couple of times to get the timing right), the pendant will resume working just fine and the disconnects stop for the time being.

As I have said, I believe the reason for the disconnects is that the dongle is unhappy with the current situation and resets itself. And "unhappy with the current situation" is most likely because the host has not been polling its interrupt endpoint for some time (maybe half a second or so).

I attach here a logic analyser screenshot, capture file as well as wireshark capture from a "forced" disconnect event. For this capture, I had the dongle disconnected, LinuxCNC (sim) running a program and in the background waiting for the pendant. I then started all captures, connected the pendant, pressed the Step button (and maybe some others) on the pendant and jogged the wheel. This caused the Usb::setupAsyncTransfer assert failed.

In the screenshot you can see that from approximately 2950 ms (corresponding to wheel jogging), there are no packets until ~3600 ms (except for SOF, which I have turned off on purpose). And at 3750 ms, there is a reset on the USB bus, corresponding to the dongle "disconnecting" and the assert failed. This reset is as far as I'm concerned generated by the dongle itself, because it was unhappy with the conditions.
 

If I zoom in a little bit on the disconnect/reset event, that becomes more clear. Here I have turned the SOF packets back on, and we see that they are sent every 1 ms as expected. And then suddenly the D+ signal drops to zero, indicating a bus reset (either from the device or the host). And I see no reason why the host should generate a reset in this situation, so I say its generated by the dongle. This is consistent with repeated disconnects every 20 s already discussed.
 

The .sr file can be opened with PulseView and the .pcapng with Wireshark, in case anybody wants to have a look.

At least for me the pendant is anything but solid. And I do think the dongle firmware is mainly to blame. That being said, it could very well be that different drivers/hardware could mask some of the issues I experience. If only I knew what causes (all) the disconnects, I'm sure we could figure out some ways to fix it.

Under normal operation there's no disconnect of the XHC pendant. It's solid. If you power it off it will disconnect.

I heard from others that they had EMI problems with USB and the XHC pendant in the past, I'm going to operate it with a large CNC in the near future so I will see if it will hit me too.
I think you have to fix a problem on the USB layer, maybe use another system which has a native USB 2.0/ehci port. The XHCI USB ports can cause issues depending on the hardware.

By the way you can easily convert an WHB04B to 6 axis by removing a stopper with a knife which is inside the rotary switch. 6 axis are wired up even on the 4 axis versions.

Had a session with a lot of mode shifts, auto programs, manual and mdi and not being careful when mode shifting.
Worked great, it felt like there was no problem with the xhc.
I had checked autosuspend which was set to 2. Set it to -1 and the disconnect/connects in syslog stopped.

Finished after a few hours. Checked syslog and now the xhc is just laying there doing nothing on the milling machine.
The faint light is on, that's all. And the syslog fills with new disconnect/connects. Every 10-20 seconds it happens.
autosuspend was already set to -1.
Went to the milling machine and turned off the pendant. New disconnects/connects stopped coming in syslog.
Didn't touch the usb dongle during the whole day. The pendant causes the disconnects? When idle?
 

Disconnects might be caused by multiple reasons.

a) poor xhci support for that system - try to connect the pendant to a USB 2.0 port
b) check if USB autosuspend is disabled (as root)
cat >/sys/module/usbcore/parameters/autosuspend
echo -1 >/sys/module/usbcore/parameters/autosuspend
c) if connected to a USB hub - don't do that, connect it directly
d) EMI issues, check if there's a pattern that can trigger this, check if a relay is able to disconnect it somehow; I had HDMI displays resetting due to issues with relays a decade ago (eg. missing flyback diodes)

I worked on the XHC pendant some years ago to fix some issues, I never had disconnect issues with it.

Had event logging on today. It failed again (loop =10).
libusb_submit_transfer failed with LIBUSB_ERROR_NO_ DEVICE (-4) 
github.com/tenderlove/libusb/blob/763471...libusb/libusb.h#L750
github.com/tenderlove/libusb/blob/763471...13/libusb/io.c#L1288

 

Syslog is absolutely full of usb disconnects of the xhc device. Every 16-17 second the device disconnects and reconnects.
Must be the xhc component that does this? It only happens when linuxcnc is running.

 

If I look in github.com/tenderlove/libusb/blob/master...usb/os/linux_usbfs.c and search for LIBUSB_ERROR_NO_DEVICE,
they are there.
Why are all the disconnects/reconnects there? Wouldn't surprise me if that can cause something.
 

The problem is that the dongle resets itself. Basically the same as pulling it out of the USB port and reinserting it.

In the situation where you try to jog with the pendant while running a program, this is caused by a too long time during which the xhc-whb04b-6 component does not poll the dongle. This we "fixed" with the loops=10.

In the other situations that happens "randomly" after some time, I do not know what the reason is.

What did you find as being the problem? You probably already wrote it, can you do it again?
What is the way to fix?

I'm pretty sure it's easiest to just add a printf("%d\n", r); between the libusb_transfer_call and the assert statement. And I'm also fairly sure I did that at one point.

And then you can just match the terminal ouput against the libusb_error enum:
github.com/libusb/libusb/blob/master/libusb/libusb.h#L1282

The docs for libusb_submit_transfer also has a section about the errors it can return and a little bit about what they mean:
libusb.sourceforge.io/api-1.0/group__lib...7dad2fee4b27962848ce

I wonder what "r" is - it doesn't say.
I'll some printouts.
One can possibly dig deeper into the layers and see exactly where it fails.
One level down:
github.com/libusb/libusb/blob/dcd30cec9a...ff/libusb/io.c#L1486

The crashing doesn't respond well to provocations. I can generally get it to crash when switching modes,
but it doesn't crash when I want it to crash.

Okay, thank you. I'm fairly certain that the reason for that message is that the pendant (dongle) has reset itself. But I don't think we know why it did so in this case. It could be something that the xhc-whb04b-6 component is doing wrong (different from the mode change delay). But it could also be that the dongle firmware is just that unstable.

I don't have any real good way to troubleshoot this. For the mode change timeout, I could capture the USB signals with a logic analyzer. But that is not feasible with my equipment if I need to wait for maybe one hour for the issue to occur. Wireshark is a possibility, but it does not show the low level packets...

I think the github issue is dead so better discuss here.

It's always the same error:github.com/LinuxCNC/linuxcnc/blob/ffc393...whb04b-6/usb.cc#L465
Don't have an actual log available, pretty sure it's that line

Yeah, at least for me the loops = 10 was not the solution to everything, so not surprised the situation is the same for you. As far as I have seen, this loops = 10 would only help during a LinuxCNC mode change (auto/manual/mdi/teleop/joint).

I was not aware of that issue, so that you for bringing that up. I will write a comment there (if I can), because I think what we have discussed here is a partial solution.

I would be interesting in knowing the error message (if any) when it fails, i.e. console ouput. This I can also "generate" myself, but right now I'm not using the machine.

Used the mpg today, sadly it crashed again. Loops = 10.

I actually opened an issue on github now over two years ago github.com/LinuxCNC/linuxcnc/issues/2237
No resolution. I never made a serious attempt to understand the code structure of the component.
Just looked at what was going on around the triggered assert() and made some more or less random changes.

Where is the blocking call? I'll see if I can do some ... tests.