Jump to content




Turtle mining swarm

turtle wireless

147 replies to this topic

#1 joebodo

  • Members
  • 69 posts

Posted 01 February 2014 - 09:19 PM

Turtle Mining Swarm (v1.2a)


(A really bad video I made during testing)

Test with 25 miners


Features
  • Nearly infinite autonomous mining.
  • Miners live off the land (they consume coal as needed).
  • Keeps the area it is mining chunk loaded (only 2 chunks loaded at a time if using spot-loaders).
  • Picks up where if left off if the server restarts.
Status Viewer
View progress on a computer/turtle, monitors, or terminal glasses display.
Attached Image: status-turtle.jpg Attached Image: status-monitors.jpg Attached Image: status-glasses.jpg

Installation Instructions
Step 1:
v1.2a - stable
pastebin get 1ucZM1hu setup
v1.3a - newest
pastebin get pa8LvVWe setup

Step 2:
setup
Select Mining Boss or Mine Worker (only 1 turtle needs to be setup as a boss)

Step 3:
Add fuel to the turtles. Note that the mine workers only need this done once as they will refuel themselves when needed.

Operating Instructions
Place down the "boss' in the bottom left corner of a chunk (press F9 to see chunk boundaries). His back and left side should be against the chunk boundaries.

Boss inventory example:
Posted Image

Slot 1: Add at least 2 chunk loaders (spot loaders are recommended)
Slot 2: Add ender chests (1 for each miner)

In the following slots, add the resources that the miner should ignore. I put in the following in this order: 64 stone, 64 dirt, 64 gravel, 64 cobblestone. Note that each miner will be given 1 of these resources to use for comparison while mining. I included cobblestone as the miner will use any of these resources to fill in any blocks it has mined.

The remainder of the slots can be filled with turtles (do not leave any empty spaces between the items). At least 1 miner is required :)

Additional Notes
Spoiler

Change log
Spoiler

Attached Thumbnails

  • Attached Image: boss.jpg

Edited by joebodo, 19 March 2014 - 11:22 PM.


#2 HeffeD

  • Members
  • 61 posts

Posted 02 February 2014 - 03:07 PM

I'm still in the process of testing, but this appears to be nearly the perfect mining script. Very impressive! It's incredibly easy to use.

I really like the fact that the turtles attempt to backfill, so you end up with extremely few 'junk' blocks from the mining process. In 3 chunks, I have only had 1 piece of dirt and 1 piece of cobblestone returned. B)

I have a question though. I'm not very advanced with Lua yet, so I don't understand a lot of your code. I've been trying to determine what your script does when a turtle encounters a dungeon chest, or a storage minecart in a mineshaft. It appears that it just breaks them and doesn't attempt to suck up the loot?

Also, coal usage is excessive. After mining 3 chunks, I have zero coal returned, and some turtles strolling around with 7k+ fuel! (One with 10k :o ) A turtle can go a long way on 7k fuel... Perhaps allow them to carry another enderchest for fuel and they can just top off when they need to? Give them a buffer of 300 or something, then they can either use coal they've mined to hit the buffer, or in case they haven't found any, place the enderchest and grab a few pieces of coal.

I guess you can probably tell that my base depends on coal for fuel? ;) (Liquifacted coal from Thermal Expansion to be precise...)

Another comment that I feel could improve the script would be to allow the user to set boundaries. I think it's great that you can start the mining process and basically just let it go until it hits the far lands, but I tend to like to mine certain areas. Like extreme hills biomes. So it would be great if you could tell it you wanted to mine a 4x4 chunk area, for example. In order to keep the chunks loaded, (using spot loaders) you could serpentine through the area. For example, mine the set number of chunks forward, then turn to the right at the boundary, and head back towards the chunk you started on. using that method you could always keep the current chunk loaded, as well as chunk that's not quite finished mining yet.

Anyway, great work on this! :)

#3 joebodo

  • Members
  • 69 posts

Posted 02 February 2014 - 03:26 PM

Thanks for your response. I'm glad you like it.

Currently, the turtle does not try to suck up any items that may be broken (carts/chests) - I'll take a look and see if I can add this.

Once the turtle reaches 25,000 in fuel, it will stop refueling and will start returning coal. You can change this value by searching for 25000 and adjusting to what you feel is ok. If the turtle drops below 1024 though, the boss will pick him up.

I do want to implement a few configurable options like starting/ending depth, etc. but I wanted to get something out instead of messing with it forever :D. Ultimately, it will process chunks in a circular manner (I have the code for it, just haven't finished testing). This is probably another thing that should be configurable.

FYI, once you run the launch script once, it extracts all the files into the swarm directory. You can then just run the program from the swarm directory instead of having it extract each time. If you run it this way, you can open mineWorker and change line 60 to have the fuel level you want (5k is reasonable).

#4 HeffeD

  • Members
  • 61 posts

Posted 02 February 2014 - 04:22 PM

I did see the 25,000 value, but I assumed I was misunderstanding your code because I couldn't imagine that such a high number would be the fuel buffer. Currently my turtles are mining their fifth chunk and the most fuel any of them have mined is 12k. It's going to take them a while to hit 25,000. When I start running this outside of my testing world, I'll definitely change the buffer value.

Yes, circular mining would work well. It's also probably easier to code than serpentining because you'd only be turning one direction throughout the mining process.

Since I'm testing, out of curiosity I used WorldEdit to remove the garbage blocks from the three chunks I'd mined earlier and found that for some reason, there are a few valuable blocks left behind. Most of them are right along the chunk boundary, but there are a few instances where the blocks are a block away from the boundary, and in two instances, three blocks away. There are no leftover blocks at all passed three blocks from the boundary. I guess this is a side effect of your code keeping the turtles inside the chunk boundary? (Which I like, by the way. The turtles should definitely stay inside the chunk you've told them to mine.

Using WorldEdit to tell me the block distribution remaining in the three chunks after removing the garbage blocks, this is what was left behind:
2 - Certus Quartz. (Applied Energistics)
3 - Obsidian.
4 - Redstone Ore.
14 - Iron Ore.
14 - Various ore from Thermal Expansion. (World edit doesn't know what those are, so they're all lumped together)
22 - Coal.

Obviously, so few remaing isn't a huge deal, but I thought it was interesting that blocks were only missed near the chunk boundary. Luckily none of those were diamonds. ;)

#5 HeffeD

  • Members
  • 61 posts

Posted 02 February 2014 - 04:46 PM

Just had an odd thing happen.

I wanted to test resuming after a restart. The boss went over and apparently called 4 of the 5 miners to come up out of their holes, then it came back to its place by the spot loaders and stopped. Since the turtles are just paused, I looked at the boss to see what it said.

It shows turtles 2,3, and 4 ready for pickup, and says it's releasing the miner, but then threw an error.

parallel:22: turtle:22: Expected number

The boss' inventory is completely full of the fourth turtle at the surface's inventory plus the initial garbage blocks. Is it possible the boss tried to pick up the turtle after taking its inventory, then not having any space to fit the turtle, the program errored? (I haven't tried this before, so I don't know the expected procedure of a resume)

Edit: I wanted to find out if this was due to a full inventory on the boss' part, so I reset everything and almost immediately restarted. The boss called three turtles up and picked one of them up. Now it says that it's releasing the miner, but it's trying to release it where one of the turtles at the surface is. So it's currently stuck in a loop trying to place the turtle where one is currently sitting. It apparently tries, path finds around a little bit, then tries again.

Edited by HeffeD, 02 February 2014 - 05:08 PM.


#6 joebodo

  • Members
  • 69 posts

Posted 02 February 2014 - 06:21 PM

I just reproduced that problem - I'll try and fix and should be able to post another version tonight.

Edit: I updated the OP with the new pastebin code. I fixed the error that you encountered (along with some other minor issues) and reduced the refuel limit to 7500. I'll get to the other things you mentioned shortly.

Edited by joebodo, 03 February 2014 - 01:39 AM.


#7 HeffeD

  • Members
  • 61 posts

Posted 03 February 2014 - 03:21 AM

View Postjoebodo, on 02 February 2014 - 06:21 PM, said:

Edit: I updated the OP with the new pastebin code. I fixed the error that you encountered (along with some other minor issues) and reduced the refuel limit to 7500. I'll get to the other things you mentioned shortly.

Thanks! :)

I'm just about to go to bed, so I'll try some resume tests tomorrow. Thanks again for sharing this script.

#8 joebodo

  • Members
  • 69 posts

Posted 04 February 2014 - 05:20 PM

View PostHeffeD, on 02 February 2014 - 03:07 PM, said:

... . I've been trying to determine what your script does when a turtle encounters a dungeon chest, or a storage minecart in a mineshaft. It appears that it just breaks them and doesn't attempt to suck up the loot?

The latest version (0.3) will now attempt to suck up any items after digging. It will also suck up anything that falls to the bottom of the bore location. There were a few other minor changes for some problems that I have found.

#9 HeffeD

  • Members
  • 61 posts

Posted 05 February 2014 - 02:46 PM

There are still some issues with resuming. I let the turtles mine a hole all the way to bedrock, then I let them get started on the next hole and exited minecraft.

Upon restarting, the mining boss went to rescue the first turtle and threw an error:
miningBoss:589: attempt to index ? (a nil value)

Edited by HeffeD, 05 February 2014 - 02:49 PM.


#10 Buho

  • Members
  • 110 posts

Posted 06 February 2014 - 08:22 AM

Quote

I did see the 25,000 value, but I assumed I was misunderstanding your code because I couldn't imagine that such a high number would be the fuel buffer.
I don't send my turtles out on a mining operation if they have less than 100,000. :P Probably overkill, but a normal mining run can burn 20,000 fuel and I want all the coal (I pre-fuel with lava).

#11 HeffeD

  • Members
  • 61 posts

Posted 06 February 2014 - 02:09 PM

View PostBuho, on 06 February 2014 - 08:22 AM, said:

I don't send my turtles out on a mining operation if they have less than 100,000. :P Probably overkill, but a normal mining run can burn 20,000 fuel and I want all the coal (I pre-fuel with lava).

Yeah, that's a tad overkill... I want all the coal as well, but if a turtle is only using what it actually needs, they find more than enough coal to fuel themselves as well as enough excess to power my base.

#12 joebodo

  • Members
  • 69 posts

Posted 06 February 2014 - 08:50 PM

View PostHeffeD, on 05 February 2014 - 02:46 PM, said:

Upon restarting, the mining boss went to rescue the first turtle and threw an error:
miningBoss:589: attempt to index ? (a nil value)

Should be okay now (OP updated), I shuffled some things around and it seems to restart fine. I tested restarts about 10 times without fail (should have tested this more originally - sorry!).

#13 Slackratchet

  • Members
  • 10 posts

Posted 06 February 2014 - 11:54 PM

That is an impressive application of power. Nice work.

#14 joebodo

  • Members
  • 69 posts

Posted 07 February 2014 - 08:25 AM

View PostSlackratchet, on 06 February 2014 - 11:54 PM, said:

That is an impressive application of power. Nice work.

Thanks - it was a bit of effort.

#15 HeffeD

  • Members
  • 61 posts

Posted 07 February 2014 - 06:21 PM

View Postjoebodo, on 06 February 2014 - 08:50 PM, said:

Should be okay now (OP updated), I shuffled some things around and it seems to restart fine. I tested restarts about 10 times without fail (should have tested this more originally - sorry!).

Don't worry about it. With such a complex bit of code, there are bound to be hiccups. :)

I'll try out the new version later. I'm currently sending some turtles rampaging across a test world to see how stable things are for extended durations. They're currently almost 700 meters away from their starting point, and have been going for a little under 20 hours.

I only have 5 miners at the moment, because through some previous stress tests with 10 miners, it appears that the mining boss can get overwhelmed eventually. I've had two instances previously where my framerate absolutely tanked. 0-1fps, and my Java client was monopolizing my CPU, so it was tough to get anything to actually respond. I actually did get to the mining boss both times and it wasn't reporting anything odd, although the first time it happened, one of the spot loaders was missing. I have no idea what happened to it because it was nowhere along the trail the miners left. I have a feeling that it somehow disappeared due to the extremely low framerate. Perhaps getting broken by the mining boss, but never actually showing up in the turtles inventory or something...

Exiting the client and restarting improved the framerates, but the mining boss was stuck at the message that it was going to resume in 5 seconds. Any attempts to restart the mining met with the same results, so I'd just start testing in a new world.

So far only using 5 miners, the test has run much longer than either of the tests with 10 miners and I've had no issues with framerate.

Very nice job with this! As I said earlier, it's nearly the perfect mining script. B)

Edit: Corrected Typo. 10 hours should have read 20. We're nearing 24 hours now.

The suck command is working. One of the turtles has picked up some chest loot.

Edited by HeffeD, 07 February 2014 - 09:38 PM.


#16 joebodo

  • Members
  • 69 posts

Posted 08 February 2014 - 01:12 PM

HeffeD - BTW, thanks so much for the in-depth testing. A new version (v0.5) is now included in the OP.

View PostHeffeD, on 07 February 2014 - 06:21 PM, said:

I only have 5 miners at the moment, because through some previous stress tests with 10 miners, it appears that the mining boss can get overwhelmed eventually.
I have had this happen once before - I thought it was just minecraft freaking out. Next time I encounter this, I'll try and find the root cause.

View PostHeffeD, on 07 February 2014 - 06:21 PM, said:

The suck command is working. One of the turtles has picked up some chest loot.
Excellent - I don't think it's perfect though as some of the loot gets scattered. Not sure the best approach as I don't want to attempt to chase down all the drops.

View PostHeffeD, on 07 February 2014 - 06:21 PM, said:

anything odd, although the first time it happened, one of the spot loaders was missing
I'm not really sure how this could happen as well - but... I improved the code involved and, hopefully, this won't happen again.

Since you seem so dead set on testing ( :D), I've included some logging options. You can use -l wireless or -l file (as in miningBoss -l wireless) to see what's going on internally. The file option will write to a mine.log file. I also included a logMonitor program with the pastebin. All you need to do is connect a wireless turtle or computer to some monitors and run the logMonitor program (and start the miningBoss as 'miningBoss -l wireless'). I need to improve this logging a bit so that I can use it to troubleshoot problems in the future. So, for right now, it's only there if you are interested in seeing what's going on.

Oh - I also improved the movement code a bit (less turns needed now). This wasn't really a problem, but it bugged me :D. And finally, I haven't forgotten about your other requests/issues, I just want to get this really stable before diving into those. Thanks again.

#17 HeffeD

  • Members
  • 61 posts

Posted 10 February 2014 - 11:51 AM

View Postjoebodo, on 08 February 2014 - 01:12 PM, said:

Excellent - I don't think it's perfect though as some of the loot gets scattered. Not sure the best approach as I don't want to attempt to chase down all the drops.

No, you definitely don't want to bother with chasing down the drops. I'm not sure of the best solution.

I know that this script:
http://www.computerc...tle-ore-quarry/
Allows you to put a chest in one of the slots and uses that as a comparison block. If a chest is found, it sucks the items out before breaking the chest. I don't think this method will work for storage minecarts though.

This script:
http://www.computerc...ant-version-20/
Will optionally use the OpenPeripheral mod ( http://www.openblocks.info/ ) to recognize the chest as an inventory and will suck the items out. Again, the storage carts don't seem to work with this method and carts are simply broken, with some of the items being picked up by turtle.suck.

I'm thinking that if storage minecarts aren't capable of being detected like chests, it probably makes the most sense to simply break both of them and suck up what you can, rather than using a different method on both items.

View Postjoebodo, on 08 February 2014 - 01:12 PM, said:

Since you seem so dead set on testing ( :D), I've included some logging options. You can use -l wireless or -l file (as in miningBoss -l wireless) to see what's going on internally. The file option will write to a mine.log file. I also included a logMonitor program with the pastebin. All you need to do is connect a wireless turtle or computer to some monitors and run the logMonitor program (and start the miningBoss as 'miningBoss -l wireless'). I need to improve this logging a bit so that I can use it to troubleshoot problems in the future. So, for right now, it's only there if you are interested in seeing what's going on.

Great! I'm currently trying the log to file option. :)

So far, I haven't had any issues resuming with the latest version. B)

#18 Buho

  • Members
  • 110 posts

Posted 11 February 2014 - 08:38 AM

Quote

I only have 5 miners at the moment, because through some previous stress tests with 10 miners, it appears that the mining boss can get overwhelmed eventually. I've had two instances previously where my framerate absolutely tanked. 0-1fps, and my Java client was monopolizing my CPU, so it was tough to get anything to actually respond.
I have 16 turtles in the swarm I wrote on my two year old rig and I often get 20fps on the shortest visibility when they're all running (2fps on max render distance). I'm not sure if its the complexity of the code or the sheer number of movements the turtles are doing per second. I've thought about rewriting my program to be simpler just to see but it's tough to be motivated to rewrite when there might be no payoff! Just wanted to pop in and say you're not the only one, and it's not just this program.

#19 HeffeD

  • Members
  • 61 posts

Posted 11 February 2014 - 12:07 PM

View PostBuho, on 11 February 2014 - 08:38 AM, said:

I have 16 turtles in the swarm I wrote on my two year old rig and I often get 20fps on the shortest visibility when they're all running (2fps on max render distance). I'm not sure if its the complexity of the code or the sheer number of movements the turtles are doing per second. I've thought about rewriting my program to be simpler just to see but it's tough to be motivated to rewrite when there might be no payoff! Just wanted to pop in and say you're not the only one, and it's not just this program.

Thanks for your input, Buho.

However, If the movement of the turtles was the issue, I would expect it to be a consistant low framerate. However, the issue I was reporting was outside the norm. I had normal framerates as the turtles were working, but after an extended timeframe of running the script, all of a sudden the framerate dropped to unusability.

Restarting the client solved the framerate issues, but the script would no longer function. Breaking and replacing the old turtles did not help. (I've seen a script break all of the computers in a world more than once...) The only fix was starting a new world.

Edit: Removed nested quote.

Edit 2: I've had turtles disappear on two separate occaisions. I suspect that it happened during a resume, (I just saw that things appeared to be working fine and didn't check the mining boss status until later...) but it hasn't happened during any of the resumes that I've watched... :blink:

I also just had a failed resume using 10 mining turtles. The mining boss had called 8 out of 10 turtles up, when the script just stopped running. When this happened, the hourglass cursor icon popped up, and the framerate tanked and the client became unresponsibe. The miners were still sending updates, but the mining boss was frozen. No errors, and nothing odd showing up on the log file. Attempts to restart mining were unsuccesful. I'll start a new test world later.

Edited by HeffeD, 11 February 2014 - 04:02 PM.


#20 Buho

  • Members
  • 110 posts

Posted 12 February 2014 - 08:26 AM

HeffeD, that actually sounds similar to my experience. I haven't paid that close attention, but it does seem I have fine framerate for a while before it slows down. Jumping into the nether and back brings the FPS back, even after they've stopped, I think. I'll keep a closer eye on it, maybe it should be reported in bugs.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users