Unexpected realtime delay error message
- thefabricator03
- Offline
- Platinum Member
Less
More
- Posts: 1130
- Thank you received: 533
04 Jul 2019 06:29 - 04 Jul 2019 09:32 #138593
by thefabricator03
Unexpected realtime delay error message was created by thefabricator03
Hi Guys,
I am getting the following error messages on my plasma machine when the machine is halfway through a long cut,
Unexpected realtime delay on task 0 with period 1000000,
This message will only display once per session,
Run latency test and resolve before continuing,
hm2/hm2_7i76e.0 Smart Serial port 0;
DoLt not cleared from previous servo thread. Servo thread rate probably too fast. This message will not be repeated, but the hm2_7i76e.0sserial.0.fault-count;pin will indicate if this is happening frequently.
hm2/hm2_7i76e.0 error finishing read! iter=283342
After I get these errors the machine will continue to give folllowing errors,
I am running Linux mint LMDE RT kernel I downloaded with Synaptic manager.
Any idea what is going on?
I am getting the following error messages on my plasma machine when the machine is halfway through a long cut,
Unexpected realtime delay on task 0 with period 1000000,
This message will only display once per session,
Run latency test and resolve before continuing,
hm2/hm2_7i76e.0 Smart Serial port 0;
DoLt not cleared from previous servo thread. Servo thread rate probably too fast. This message will not be repeated, but the hm2_7i76e.0sserial.0.fault-count;pin will indicate if this is happening frequently.
hm2/hm2_7i76e.0 error finishing read! iter=283342
After I get these errors the machine will continue to give folllowing errors,
I am running Linux mint LMDE RT kernel I downloaded with Synaptic manager.
Any idea what is going on?
Last edit: 04 Jul 2019 09:32 by thefabricator03.
Please Log in or Create an account to join the conversation.
- pl7i92
- Offline
- Platinum Member
Less
More
- Posts: 1875
- Thank you received: 354
04 Jul 2019 07:05 #138597
by pl7i92
Replied by pl7i92 on topic Unexpected realtime delay error message
hi
what is your latency of the pc
and how is the watchdog initalizised
hm2_7i92.0.watchdog.timeout_ns 10000000
i think there is some zeros missing
can you post your ini and hal if posible
so we can check all entries
what is your latency of the pc
and how is the watchdog initalizised
hm2_7i92.0.watchdog.timeout_ns 10000000
i think there is some zeros missing
can you post your ini and hal if posible
so we can check all entries
The following user(s) said Thank You: thefabricator03
Please Log in or Create an account to join the conversation.
- thefabricator03
- Offline
- Platinum Member
Less
More
- Posts: 1130
- Thank you received: 533
04 Jul 2019 07:22 #138603
by thefabricator03
Replied by thefabricator03 on topic Unexpected realtime delay error message
Please Log in or Create an account to join the conversation.
- tommylight
- Away
- Moderator
Less
More
- Posts: 19460
- Thank you received: 6529
04 Jul 2019 11:49 #138614
by tommylight
Replied by tommylight on topic Unexpected realtime delay error message
That night be caused by the intel ethernet
Search the forum for coalescence, that might help. Again, assuming you have and intel board since the latency looks fine and having excursions on the servo thread. Seen that a lot on 2 Dell E6510 and a Lenovo T420S i use for testing.
Search the forum for coalescence, that might help. Again, assuming you have and intel board since the latency looks fine and having excursions on the servo thread. Seen that a lot on 2 Dell E6510 and a Lenovo T420S i use for testing.
The following user(s) said Thank You: thefabricator03
Please Log in or Create an account to join the conversation.
- PCW
- Away
- Moderator
Less
More
- Posts: 17940
- Thank you received: 4811
04 Jul 2019 13:10 #138623
by PCW
Replied by PCW on topic Unexpected realtime delay error message
This could be due to a couple of things, it could be a Ethernet latency issue
or it could be you have completely lost contact with the card due to
1. Power supply problems at the 7I76E end
2. A problem with the 7I76E,host or cable
When your system is running normally for some time (no errors) run this command from a terminal:
halcmd show param *.tmax
This (and the clock speed of your CPU) will indicate if its a marginal timing issue
Also if you get an error, check dmesg for any link up/down events:
dmesg | tail -50
Also posting your hal/ini files might give a clue
or it could be you have completely lost contact with the card due to
1. Power supply problems at the 7I76E end
2. A problem with the 7I76E,host or cable
When your system is running normally for some time (no errors) run this command from a terminal:
halcmd show param *.tmax
This (and the clock speed of your CPU) will indicate if its a marginal timing issue
Also if you get an error, check dmesg for any link up/down events:
dmesg | tail -50
Also posting your hal/ini files might give a clue
The following user(s) said Thank You: tommylight, mkardasi, thefabricator03
Please Log in or Create an account to join the conversation.
- thefabricator03
- Offline
- Platinum Member
Less
More
- Posts: 1130
- Thank you received: 533
04 Jul 2019 21:35 #138646
by thefabricator03
I posted them in the third post.
Replied by thefabricator03 on topic Unexpected realtime delay error message
Also posting your hal/ini files might give a clue
I posted them in the third post.
Please Log in or Create an account to join the conversation.
- PCW
- Away
- Moderator
Less
More
- Posts: 17940
- Thank you received: 4811
04 Jul 2019 22:59 - 04 Jul 2019 23:00 #138651
by PCW
Replied by PCW on topic Unexpected realtime delay error message
So the hal/ini files look OK.
To determine whats happening, the timing information
(and your CPU clock speed since the tmax numbers are in units of CPU clocks )
would be helpful.
also noting if the link was lost (via dmesg) would help
To determine whats happening, the timing information
(and your CPU clock speed since the tmax numbers are in units of CPU clocks )
would be helpful.
also noting if the link was lost (via dmesg) would help
Last edit: 04 Jul 2019 23:00 by PCW.
Please Log in or Create an account to join the conversation.
- thefabricator03
- Offline
- Platinum Member
Less
More
- Posts: 1130
- Thank you received: 533
05 Jul 2019 01:31 - 05 Jul 2019 01:32 #138658
by thefabricator03
Replied by thefabricator03 on topic Unexpected realtime delay error message
Ok there are the values of halcmd,
plasma@plasma:~/linuxcnc-plasmac$ halcmd show param *.tmax
Parameters:
Owner Type Dir Value Name
39 s32 RW 6755 debounce.0.tmax
30 s32 RW 0 hm2_7i76e.0.read-request.tmax
30 s32 RW 1190409 hm2_7i76e.0.read.tmax
30 s32 RW 102827 hm2_7i76e.0.write.tmax
23 s32 RW 6919 motion-command-handler.tmax
23 s32 RW 83173 motion-controller.tmax
33 s32 RW 4446 pid.s.do-pid-calcs.tmax
33 s32 RW 9905 pid.x.do-pid-calcs.tmax
33 s32 RW 3391 pid.y.do-pid-calcs.tmax
33 s32 RW 5521 pid.y2.do-pid-calcs.tmax
33 s32 RW 5490 pid.z.do-pid-calcs.tmax
36 s32 RW 18479 plasmac.tmax
24 s32 RW 1350822 servo-thread.tmax
And the values of dmesg after the error occurs.
plasma@plasma:~/linuxcnc-plasmac$ dmesg | tail -50
[11094.897955] sd 6:0:0:0: [sdc] Mode Sense: 23 00 00 00
[11094.900832] sd 6:0:0:0: [sdc] No Caching mode page found
[11094.900833] sd 6:0:0:0: [sdc] Assuming drive cache: write through
[11094.920588] sdc: sdc1
[11094.933333] sd 6:0:0:0: [sdc] Attached SCSI removable disk
[11095.354215] FAT-fs (sdc1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[15266.093096] perf: interrupt took too long (3939 > 3937), lowering kernel.perf_event_max_sample_rate to 50750
[15934.868745] intel_powerclamp: Start idle injection to reduce power
[15938.872943] intel_powerclamp: Stop forced idle injection
[16311.257054] intel_powerclamp: Start idle injection to reduce power
[16315.261257] intel_powerclamp: Stop forced idle injection
[16827.650355] intel_powerclamp: Start idle injection to reduce power
[16831.654614] intel_powerclamp: Stop forced idle injection
[17485.613726] usb 2-1.7: USB disconnect, device number 9
[18110.231756] intel_powerclamp: Start idle injection to reduce power
[18114.234807] intel_powerclamp: Stop forced idle injection
[22335.969886] e1000e: eno1 NIC Link is Down
[22350.805129] usb 2-1.7: new high-speed USB device number 10 using ehci-pci
[22350.942743] usb 2-1.7: New USB device found, idVendor=18a5, idProduct=0302
[22350.942746] usb 2-1.7: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[22350.942747] usb 2-1.7: Product: STORE N GO
[22350.942748] usb 2-1.7: Manufacturer: Verbatim
[22350.942749] usb 2-1.7: SerialNumber: 900093C1F5275B15
[22350.943026] usb-storage 2-1.7:1.0: USB Mass Storage device detected
[22350.943085] scsi host6: usb-storage 2-1.7:1.0
[22351.956745] scsi 6:0:0:0: Direct-Access Verbatim STORE N GO 5.00 PQ: 0 ANSI: 6
[22351.969304] sd 6:0:0:0: Attached scsi generic sg3 type 0
[22354.486338] sd 6:0:0:0: [sdc] 30322688 512-byte logical blocks: (15.5 GB/14.5 GiB)
[22354.490326] sd 6:0:0:0: [sdc] Write Protect is off
[22354.490327] sd 6:0:0:0: [sdc] Mode Sense: 23 00 00 00
[22354.493201] sd 6:0:0:0: [sdc] No Caching mode page found
[22354.493202] sd 6:0:0:0: [sdc] Assuming drive cache: write through
[22354.512956] sdc: sdc1
[22354.525827] sd 6:0:0:0: [sdc] Attached SCSI removable disk
[22354.942591] FAT-fs (sdc1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[22797.808328] usb 2-1.7: USB disconnect, device number 10
[87975.423069] usb 2-1.8: USB disconnect, device number 8
[87975.820331] usb 2-1.8: new low-speed USB device number 11 using ehci-pci
[87975.934182] usb 2-1.8: New USB device found, idVendor=1a2c, idProduct=2124
[87975.934184] usb 2-1.8: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[87975.934186] usb 2-1.8: Product: USB Keyboard
[87975.934187] usb 2-1.8: Manufacturer: SEM
[87975.936925] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.8/2-1.8:1.0/0003:1A2C:2124.0008/input/input20
[87975.996485] hid-generic 0003:1A2C:2124.0008: input,hidraw1: USB HID v1.10 Keyboard [SEM USB Keyboard] on usb-0000:00:1d.0-1.8/input0
[87975.999633] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.8/2-1.8:1.1/0003:1A2C:2124.0009/input/input21
[87976.056422] hid-generic 0003:1A2C:2124.0009: input,hidraw2: USB HID v1.10 Device [SEM USB Keyboard] on usb-0000:00:1d.0-1.8/input1
[87979.194425] e1000e: eno1 NIC Link is Up 100 Mbps Full Duplex, Flow Control: Rx/Tx
[87979.194429] e1000e 0000:00:19.0 eno1: 10/100 speed: disabling TSO
[88288.356358] intel_powerclamp: Start idle injection to reduce power
[88292.357979] intel_powerclamp: Stop forced idle injection
plasma@plasma:~/linuxcnc-plasmac$
plasma@plasma:~/linuxcnc-plasmac$ halcmd show param *.tmax
Parameters:
Owner Type Dir Value Name
39 s32 RW 6755 debounce.0.tmax
30 s32 RW 0 hm2_7i76e.0.read-request.tmax
30 s32 RW 1190409 hm2_7i76e.0.read.tmax
30 s32 RW 102827 hm2_7i76e.0.write.tmax
23 s32 RW 6919 motion-command-handler.tmax
23 s32 RW 83173 motion-controller.tmax
33 s32 RW 4446 pid.s.do-pid-calcs.tmax
33 s32 RW 9905 pid.x.do-pid-calcs.tmax
33 s32 RW 3391 pid.y.do-pid-calcs.tmax
33 s32 RW 5521 pid.y2.do-pid-calcs.tmax
33 s32 RW 5490 pid.z.do-pid-calcs.tmax
36 s32 RW 18479 plasmac.tmax
24 s32 RW 1350822 servo-thread.tmax
And the values of dmesg after the error occurs.
plasma@plasma:~/linuxcnc-plasmac$ dmesg | tail -50
[11094.897955] sd 6:0:0:0: [sdc] Mode Sense: 23 00 00 00
[11094.900832] sd 6:0:0:0: [sdc] No Caching mode page found
[11094.900833] sd 6:0:0:0: [sdc] Assuming drive cache: write through
[11094.920588] sdc: sdc1
[11094.933333] sd 6:0:0:0: [sdc] Attached SCSI removable disk
[11095.354215] FAT-fs (sdc1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[15266.093096] perf: interrupt took too long (3939 > 3937), lowering kernel.perf_event_max_sample_rate to 50750
[15934.868745] intel_powerclamp: Start idle injection to reduce power
[15938.872943] intel_powerclamp: Stop forced idle injection
[16311.257054] intel_powerclamp: Start idle injection to reduce power
[16315.261257] intel_powerclamp: Stop forced idle injection
[16827.650355] intel_powerclamp: Start idle injection to reduce power
[16831.654614] intel_powerclamp: Stop forced idle injection
[17485.613726] usb 2-1.7: USB disconnect, device number 9
[18110.231756] intel_powerclamp: Start idle injection to reduce power
[18114.234807] intel_powerclamp: Stop forced idle injection
[22335.969886] e1000e: eno1 NIC Link is Down
[22350.805129] usb 2-1.7: new high-speed USB device number 10 using ehci-pci
[22350.942743] usb 2-1.7: New USB device found, idVendor=18a5, idProduct=0302
[22350.942746] usb 2-1.7: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[22350.942747] usb 2-1.7: Product: STORE N GO
[22350.942748] usb 2-1.7: Manufacturer: Verbatim
[22350.942749] usb 2-1.7: SerialNumber: 900093C1F5275B15
[22350.943026] usb-storage 2-1.7:1.0: USB Mass Storage device detected
[22350.943085] scsi host6: usb-storage 2-1.7:1.0
[22351.956745] scsi 6:0:0:0: Direct-Access Verbatim STORE N GO 5.00 PQ: 0 ANSI: 6
[22351.969304] sd 6:0:0:0: Attached scsi generic sg3 type 0
[22354.486338] sd 6:0:0:0: [sdc] 30322688 512-byte logical blocks: (15.5 GB/14.5 GiB)
[22354.490326] sd 6:0:0:0: [sdc] Write Protect is off
[22354.490327] sd 6:0:0:0: [sdc] Mode Sense: 23 00 00 00
[22354.493201] sd 6:0:0:0: [sdc] No Caching mode page found
[22354.493202] sd 6:0:0:0: [sdc] Assuming drive cache: write through
[22354.512956] sdc: sdc1
[22354.525827] sd 6:0:0:0: [sdc] Attached SCSI removable disk
[22354.942591] FAT-fs (sdc1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[22797.808328] usb 2-1.7: USB disconnect, device number 10
[87975.423069] usb 2-1.8: USB disconnect, device number 8
[87975.820331] usb 2-1.8: new low-speed USB device number 11 using ehci-pci
[87975.934182] usb 2-1.8: New USB device found, idVendor=1a2c, idProduct=2124
[87975.934184] usb 2-1.8: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[87975.934186] usb 2-1.8: Product: USB Keyboard
[87975.934187] usb 2-1.8: Manufacturer: SEM
[87975.936925] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.8/2-1.8:1.0/0003:1A2C:2124.0008/input/input20
[87975.996485] hid-generic 0003:1A2C:2124.0008: input,hidraw1: USB HID v1.10 Keyboard [SEM USB Keyboard] on usb-0000:00:1d.0-1.8/input0
[87975.999633] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.8/2-1.8:1.1/0003:1A2C:2124.0009/input/input21
[87976.056422] hid-generic 0003:1A2C:2124.0009: input,hidraw2: USB HID v1.10 Device [SEM USB Keyboard] on usb-0000:00:1d.0-1.8/input1
[87979.194425] e1000e: eno1 NIC Link is Up 100 Mbps Full Duplex, Flow Control: Rx/Tx
[87979.194429] e1000e 0000:00:19.0 eno1: 10/100 speed: disabling TSO
[88288.356358] intel_powerclamp: Start idle injection to reduce power
[88292.357979] intel_powerclamp: Stop forced idle injection
plasma@plasma:~/linuxcnc-plasmac$
Last edit: 05 Jul 2019 01:32 by thefabricator03.
Please Log in or Create an account to join the conversation.
- thefabricator03
- Offline
- Platinum Member
Less
More
- Posts: 1130
- Thank you received: 533
05 Jul 2019 02:07 #138659
by thefabricator03
Replied by thefabricator03 on topic Unexpected realtime delay error message
Attachments:
Please Log in or Create an account to join the conversation.
- rodw
- Away
- Platinum Member
Less
More
- Posts: 10780
- Thank you received: 3551
05 Jul 2019 02:26 #138660
by rodw
Replied by rodw on topic Unexpected realtime delay error message
I'm no expert with Linux but that looks like a HDD hardware fault or a corrupt diskThat would do it. maybe run fsck as suggested and see what happens
FAT-fs (sdc1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
Please Log in or Create an account to join the conversation.
Time to create page: 0.174 seconds