Jump to content




Updating a wired network can cause a peripheral deadlock


21 replies to this topic

#1 RichardG867

  • Members
  • 196 posts

Posted 26 May 2013 - 06:46 PM

Observed mainly in peripherals which use tick callbacks.
Java stack information for the threads listed above:
===================================================
"Coroutine-29":
at dan200.computer.shared.TileEntityCable.callMethodRemote(TileEntityCable.java:432)
- waiting to lock <0x00000000c64a2ce0> (a java.util.HashMap)
at dan200.computer.shared.TileEntityCable.callMethod(TileEntityCable.java:246)
at dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.call(PeripheralAPI.java:115)
- locked <0x00000000c9077a70> (a dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper)
at dan200.computer.core.apis.PeripheralAPI.callMethod(PeripheralAPI.java:478)
at dan200.computer.core.LuaJLuaMachine$2.invoke(LuaJLuaMachine.java:304)
- locked <0x00000000bf2caac0> (a dan200.computer.core.LuaJLuaMachine$2)
at org.luaj.vm2.lib.VarArgFunction.onInvoke(Unknown Source)
at org.luaj.vm2.TailcallVarargs.eval(Unknown Source)
at org.luaj.vm2.TailcallVarargs.arg1(Unknown Source)
at org.luaj.vm2.LuaClosure.call(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
at org.luaj.vm2.lib.BaseLib.pcall(Unknown Source)
at org.luaj.vm2.lib.BaseLib$BaseLibV.invoke(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.TailcallVarargs.eval(Unknown Source)
at org.luaj.vm2.TailcallVarargs.arg1(Unknown Source)
at org.luaj.vm2.LuaClosure.call(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
at org.luaj.vm2.lib.BaseLib.pcall(Unknown Source)
at org.luaj.vm2.lib.BaseLib$BaseLibV.invoke(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.call(Unknown Source)
at org.luaj.vm2.LuaClosure.execute(Unknown Source)
at org.luaj.vm2.LuaClosure.onInvoke(Unknown Source)
at org.luaj.vm2.LuaClosure.invoke(Unknown Source)
at org.luaj.vm2.LuaThread$State.run(Unknown Source)
- locked <0x00000000bf2cc050> (a org.luaj.vm2.LuaThread$State)
at java.lang.Thread.run(Unknown Source)
"Server thread":
at dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.queueEvent(PeripheralAPI.java:214)
- waiting to lock <0x00000000c9077a70> (a dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper)
at dan200.computer.shared.TileEntityCable$RemotePeripheralWrapper.detach(TileEntityCable.java:559)
at dan200.computer.shared.TileEntityCable.detachPeripheral(TileEntityCable.java:398)
at dan200.computer.shared.TileEntityCable.findPeripherals(TileEntityCable.java:662)
- locked <0x00000000c64a2ce0> (a java.util.HashMap)
at dan200.computer.shared.TileEntityCable.func_70316_g(TileEntityCable.java:278)
- locked <0x00000000c64a2ce0> (a java.util.HashMap)
at net.minecraft.world.World.func_72939_s(World.java:2199)
at net.minecraft.world.WorldServer.func_72939_s(WorldServer.java:546)
at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:652)
at net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:571)
at net.minecraft.server.integrated.IntegratedServer.func_71217_p(IntegratedServer.java:171)
at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:469)
at net.minecraft.server.ThreadMinecraftServer.run(SourceFile:573)
Found 1 deadlock.



#2 MudkipTheEpic

  • Members
  • 639 posts
  • LocationWhere you'd least expect it.

Posted 31 May 2013 - 12:19 PM

I got this error too! Thanks for posting it. This needs more attention.

#3 Cloudy

    Ex-Developer

  • Members
  • 2,543 posts

Posted 31 May 2013 - 12:42 PM

How did you get it? What peripherals?

#4 RebelNode

  • Members
  • 6 posts

Posted 04 June 2013 - 04:17 PM

Here's two stack traces that are related: http://pastebin.com/EnW4Ud9b and http://pastebin.com/yEtP0mV8 .

I had a computer with a wired modem connected with network cable to another wired modem connected to an advanced computer. Also a monitor on the adv computer, but the wired modems and cables seem to be the problem. CC version 1.53.

#5 dwi

  • New Members
  • 1 posts

Posted 04 June 2013 - 05:13 PM

[SEVERE] Current Thread: Server thread
[SEVERE]    PID: 14 | Suspended: false | Native: false | State: BLOCKED
[SEVERE]    Thread is waiting on monitor(s):
[SEVERE]	    Locked on:dan200.computer.shared.TileEntityCable.findPeripherals(TileEntityCable.java:683)
[SEVERE]	    Locked on:dan200.computer.shared.TileEntityCable.func_70316_g(TileEntityCable.java:281)
[SEVERE]    Stack:
[SEVERE]	    dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.queueEvent(PeripheralAPI.java:214)
[SEVERE]	    dan200.computer.shared.TileEntityCable$RemotePeripheralWrapper.attach(TileEntityCable.java:556)
[SEVERE]	    dan200.computer.shared.TileEntityCable.attachPeripheral(TileEntityCable.java:391)
[SEVERE]	    dan200.computer.shared.TileEntityCable.findPeripherals(TileEntityCable.java:683)
[SEVERE]	    dan200.computer.shared.TileEntityCable.func_70316_g(TileEntityCable.java:281)
[SEVERE]	    net.minecraft.world.World.func_72939_s(World.java:2278)
[SEVERE]	    net.minecraft.world.WorldServer.func_72939_s(WorldServer.java:786)
[SEVERE]	    net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:832)
[SEVERE]	    net.minecraft.server.dedicated.DedicatedServer.func_71190_q(DedicatedServer.java:320)
[SEVERE]	    net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:728)
[SEVERE]	    net.minecraft.server.MinecraftServer.run(MinecraftServer.java:612)
[SEVERE]	    net.minecraft.server.ThreadMinecraftServer.run(SourceFile:573)

[SEVERE] Current Thread: Thread-4883
[SEVERE]    PID: 5270 | Suspended: false | Native: false | State: BLOCKED
[SEVERE]    Thread is waiting on monitor(s):
[SEVERE]	    Locked on:dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.attach(PeripheralAPI.java:82)
[SEVERE]	    Locked on:dan200.computer.core.apis.PeripheralAPI.startup(PeripheralAPI.java:336)
[SEVERE]	    Locked on:dan200.computer.core.Computer$1.execute(Computer.java:879)
[SEVERE]    Stack:
[SEVERE]	    dan200.computer.shared.TileEntityCable.attach(TileEntityCable.java:357)
[SEVERE]	    dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.attach(PeripheralAPI.java:82)
[SEVERE]	    dan200.computer.core.apis.PeripheralAPI.startup(PeripheralAPI.java:336)
[SEVERE]	    dan200.computer.core.Computer.initLua(Computer.java:770)
[SEVERE]	    dan200.computer.core.Computer.access$1200(Computer.java:33)
[SEVERE]	    dan200.computer.core.Computer$1.execute(Computer.java:879)
[SEVERE]	    dan200.computer.core.ComputerThread$1$1.run(ComputerThread.java:117)
[SEVERE]	    java.lang.Thread.run(Thread.java:722)

I think it was caused by one computer and multiple advanced monitors wired together. Computer was running this script http://pastebin.com/nriptbm8

#6 Cloudy

    Ex-Developer

  • Members
  • 2,543 posts

Posted 05 June 2013 - 06:44 AM

Unless you can reproduce on forge without MCPC+, I'm not willing to consider this as a problem. Bukkit is not supported here in the slightest.

#7 micle546

  • Members
  • 3 posts

Posted 07 June 2013 - 08:30 AM

Believe this to be related to the OP, No bukkit, code deadlock using wired modems. In SSP all CC blocks and items lock up. In SMP, the Server stops responding, and kicks all active players with a 'Read timed out' error. Since the game didn't actually crash, I have no crash reports.

I have been able to reproduce the bug in a fresh SMP install with latest versions of listed mods.

Modlist:
CC 1.5.3
MiscPeripherals
Buildcraft
MCForge

Recorded the process I used to reproduce the bug, will upload soon. Used this script from Pastebin: U6r5ApXA

#8 Cloudy

    Ex-Developer

  • Members
  • 2,543 posts

Posted 07 June 2013 - 12:02 PM

What peripherals were attached to said cables?

#9 micle546

  • Members
  • 3 posts

Posted 07 June 2013 - 09:30 PM

An advanced computer hooked up to an advanced monitor and 8 gate readers

#10 MudkipTheEpic

  • Members
  • 639 posts
  • LocationWhere you'd least expect it.

Posted 07 June 2013 - 10:06 PM

View Postmicle546, on 07 June 2013 - 09:30 PM, said:

An advanced computer hooked up to an advanced monitor and 8 gate readers

Try with just vanilla CC peripherals.

#11 OniBait

  • Members
  • 8 posts

Posted 24 August 2013 - 11:11 AM

This is a deadlock in ComputerCraft code and will occur on vanilla forge. Since it is a deadlock that occurs based on certain timings, it may or may not be easy to reproduce. Best way I can think of trying to make it occur is to run a program in one of the scripts above continually polling for peripherals to attach, while adding network cables that have peripherals already attached to them (not sure if the exact peripheral makes a difference in this case).

Example: Computer ---&gt; Cable ---&gt; Break ---&gt; Cable --&gt; Peripheral (keep fixing and breaking the break in the cable)

Looks like this deadlock will occur if you have anything calling the PeripheralAPI while blocks are being placed and ticked on the same CC network.

How to fix it:
My suggestion would be to use java.util.concurrent.locks.Lock and tryLock() instead of the synchronized statements in TileEntityCable's methods -- i'm pretty sure that the logic that is there, it'd be "fine" to skip trying to attach the peripheral in that scenario since you'd end up either attaching it in the other thread or else try attaching it the next tick anyway.


As for the deadlock:
This is where it occurring. Below is an example of the issue from CC 1.53 -- so line numbers will be different for 1.56 (where the issue still exists)

(note: this is from one of several examples I've seen of this -- the exact PeripheralAPI calls can vary though)

Server Thread:
TileEntityCable.updateEntity() calls findPeripherals() both of which gets a lock on m_peripheralsByName it then goes down the call chain and tries to acquire a lock on PeripheralAPI.PeripheralWrapper (via the synchronized method queueEvent()) and deadlocks due to the other thread.

Computer Thread:
The CC thread is calling Computer.InitLua() which calls PeripheralAPI.startup() which acquires the lock on PeripheralAPI.PeripheralWrapper (via the synchronized method attach())and the tries to acquire a lock on m_peripheralsByName in the TileEntityCable.attach() method and deadlocks due to the server thread above.


Here are the stack traces -- I've highlighted which methods specifically are the source of the deadlock:


Current Thread: Thread-169
PID: 287 | Suspended: false | Native: false | State: BLOCKED
Thread is waiting on monitor(s):
Locked on:dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.attach(PeripheralAPI.java:82) &lt;-- Lock reference #1
Locked on:dan200.computer.core.apis.PeripheralAPI.startup(PeripheralAPI.java:336)
Locked on:dan200.computer.core.Computer$1.execute(Computer.java:879)
Stack:
dan200.computer.shared.TileEntityCable.attach(TileEntityCable.java:357) &lt;-- Deadlocked due to #2 (and #3) below
dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.attach(PeripheralAPI.java:82)
dan200.computer.core.apis.PeripheralAPI.startup(PeripheralAPI.java:336)
dan200.computer.core.Computer.initLua(Computer.java:770)
dan200.computer.core.Computer.access$1200(Computer.java:33)
dan200.computer.core.Computer$1.execute(Computer.java:879)
dan200.computer.core.ComputerThread$1$1.run(ComputerThread.java:117)
java.lang.Thread.run(Thread.java:722)


Current Thread: Server thread
PID: 15 | Suspended: false | Native: false | State: BLOCKED
Thread is waiting on monitor(s):
Locked on:dan200.computer.shared.TileEntityCable.findPeripherals(TileEntityCable.java:683) &lt;-- Lock Reference #2
Locked on:dan200.computer.shared.TileEntityCable.func_70316_g(TileEntityCable.java:281) &lt;-- Lock Reference #3
Stack:
dan200.computer.core.apis.PeripheralAPI$PeripheralWrapper.queueEvent(PeripheralAPI.java:214) &lt;-- Deadlocked due to #1 above
dan200.computer.shared.TileEntityCable$RemotePeripheralWrapper.attach(TileEntityCable.java:556)
dan200.computer.shared.TileEntityCable.attachPeripheral(TileEntityCable.java:391)
dan200.computer.shared.TileEntityCable.findPeripherals(TileEntityCable.java:683)
dan200.computer.shared.TileEntityCable.func_70316_g(TileEntityCable.java:281)
net.minecraft.world.World.func_72939_s(World.java:2362)
net.minecraft.world.WorldServer.func_72939_s(WorldServer.java:803)
net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:820)
net.minecraft.server.dedicated.DedicatedServer.func_71190_q(DedicatedServer.java:320)
net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:716)
net.minecraft.server.MinecraftServer.run(MinecraftServer.java:600)
net.minecraft.server.ThreadMinecraftServer.run(SourceFile:573)

#12 OniBait

  • Members
  • 8 posts

Posted 24 August 2013 - 04:02 PM

Also just saw this other thread -- exact same issue and looks like is easier to reproduce.

http://www.computerc...hread-deadlock/

#13 Foone

  • Members
  • 25 posts

Posted 26 August 2013 - 12:51 PM

I've been having the same problem every 1-3 days on my 1.6.2 server.

http://pastebin.com/hvhNVhEv

My server isn't CC-only, though. I've got OpenPeripherals loaded and it seems to increase the frequency of the deadlocks. I looked at the relevant code in OpenP and I don't think it's responsible for the deadlock, but it is making them happen more often simply by increasing the number of things which are valid peripherals and therefore the number of attach/detach events.

The most annoying thing about this deadlock is that it prevents the server from exiting properly, so it has to be kill -9'd and it causes every turtle that was in motion to lose its identity: no tool, no upgrade, no name (and therefore no files).

#14 Cloudy

    Ex-Developer

  • Members
  • 2,543 posts

Posted 26 August 2013 - 07:50 PM

OpenP does some silly stuff - the same stuff MiscPeripherals did. That's why it deadlocks. This won't be a problem as of the 1.6.2 version as there is no need for OpenP to do the thing it does!

#15 Foone

  • Members
  • 25 posts

Posted 27 August 2013 - 12:05 AM

Sorry if I wasn't clear, this was in the 1.6.2 version. (I had it lock up again today, it definitely seems to be more frequent with more turtles+peripherals)

I'll try to reproduce on 1.6.2 + only CC.

#16 Cloudy

    Ex-Developer

  • Members
  • 2,543 posts

Posted 27 August 2013 - 05:52 PM

Please try and repeat it without MiscP/OpenP.

#17 OniBait

  • Members
  • 8 posts

Posted 27 August 2013 - 11:05 PM

Cloudy -- this post is the exact same issue: http://www.computerc...hread-deadlock/

You can solve this fairly easily by using a ReentrantLock in the java.util.concurrent namespace instead of the synchronized(m_peripheralsByName){} blocks. (I went into detail as to what the issue was in my post above)

Lock lock = new ReentrantLock(); // at the class level

// In your method body in place of your synchronized() statements
if (lock.tryLock()) {
	// do whatever
}

Since you are doing this inside of a tick, the worst case scenario here is you don't get the lock this tick, but you get it the next one. (which is much better than a deadlock)

#18 Foone

  • Members
  • 25 posts

Posted 27 August 2013 - 11:05 PM

Done. This is a fresh 1.6.2 install + latest forge + CC 1.56. It took about 30 minutes, then it locked up and turtles/computers/disk drives couldn't be clicked on, either left or right click (so you couldn't break them). Vanilla blocks worked normally.
Turtles froze in place.

I had two turtles, a disk drive, and a computer. The turtles were both moving up/down in/out so they'd constantly generate attach/detaches.

http://pastebin.com/B0KP6300

#19 Foone

  • Members
  • 25 posts

Posted 30 August 2013 - 01:32 PM

I had two server crashes yesterday, and I had a new behavior on moving turtles. Normally any moving turtles just get reset back to toolless/upgradeless/ID-less/fileless, but now I'm getting duplications.

Two brand new turtles standing where a turtle was moving when the server crashed. This is a very frustrating bug as it only becomes more frequent the more turtles you have.

EDIT: I just had two crashes in a row: it crashed, and after I kill-9'd it and was replacing erased turtles, it crashed once more. This is getting really frustrating

#20 Flow86

  • Members
  • 3 posts

Posted 23 September 2013 - 02:36 AM

Hi, I have the same problem here since today, worked like a charm for months now. Don't know what changed tonight.

Only have (vanilla cc) modems attached to computers. still getting the deadlock now :/





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users