Jump to content




[MC1.12][Fork] CC: Tweaked


49 replies to this topic

#41 SquidDev

  • Members
  • 1,352 posts
  • LocationDoes anyone put something serious here?

Posted 05 November 2018 - 07:43 AM

View PostSquidDev, on 25 October 2018 - 11:20 AM, said:

stdin is just a wrapper for read, though apparently is somewhat broken.
Just to note that CC:T 1.80pr1.10 (I need a better naming scheme) has been released, which fixes several bugs introduced by the filesystem rework. Amongst the changes, there's a fix for io.stdin (and thus io.read) not working as expected.

#42 IndustrialLemon

  • Members
  • 61 posts

Posted 10 November 2018 - 01:40 AM

Hey sir, quick question. Are there any plans to add some or all of the features prevalent in the computeacube fork? I really enjoy the regex api and the fact that pocket computers act like maps, but also like the changes that come along with CC:tweaked.

#43 SquidDev

  • Members
  • 1,352 posts
  • LocationDoes anyone put something serious here?

Posted 10 November 2018 - 01:49 AM

View PostIndustrialLemon, on 10 November 2018 - 01:40 AM, said:

Are there any plans to add some or all of the features prevalent in the computeacube fork? I really enjoy the regex api and the fact that pocket computers act like maps.
Pocket computers acting like maps is also present in CC:Tweaked (as well as printed pages having similar functionality). Interestingly enough, Computeacube's implementation of this comes from CC:Tweaked's predecessor.

As far as Computeacube's additional APIs go, I've no plans for adding them to CC:Tweaked. While PCREs are more powerful than Lua's patterns, introducing an API for them is a little grotty. It's another non-standard way of manipulating strings, and doesn't integrate as cleanly with the rest of the language.

Edited by SquidDev, 10 November 2018 - 01:49 AM.


#44 IndustrialLemon

  • Members
  • 61 posts

Posted 10 November 2018 - 02:16 AM

Oh so lua has a way of accomplishing the same thing? I've no problem with that so long as I can pick at words from documents and possibly webpages.

EDIT: Also I'm sure there's a better place to report bugs but while I've got you. I just experienced a bug that was so hard to describe. I have a GPS system that runs off of a disk/startup dir and every time I would edit the disk/startup.lua file and save it would save the file with clips of the programming strewn about at the bottom. Every save it was worse. So, in the beginning, it was just a piece of, 'shell.run("gps host "..x, y, z)'. But a couple saves later and it was more like this.
  shell.run("gps host "..x, y, z)
end
t"..x, y, z)
end
end

t"..x, y, z)
en
end
x, y, z

end


Keep in mind everything after the first 'end' was deleted everytime each save before the next.

Edited by IndustrialLemon, 10 November 2018 - 02:21 AM.


#45 SquidDev

  • Members
  • 1,352 posts
  • LocationDoes anyone put something serious here?

Posted 10 November 2018 - 10:36 PM

View PostIndustrialLemon, on 10 November 2018 - 02:16 AM, said:

EDIT: Also I'm sure there's a better place to report bugs but while I've got you. I just experienced a bug that was so hard to describe. I have a GPS system that runs off of a disk/startup dir and every time I would edit the disk/startup.lua file and save it would save the file with clips of the programming strewn about at the bottom.
Thanks for the report. I'm working on a fix, for the time being it might be worth downgrading to 1.80pr1.8 - the last couple of versions introduced an experimental rewrite of filesystems, and we're still working out a few kinks.

#46 SquidDev

  • Members
  • 1,352 posts
  • LocationDoes anyone put something serious here?

Posted 24 November 2018 - 02:32 PM

I've just pushed another CC:Tweaked release, which fixes the remaining issues with the file system rewrite (well, the ones I know about), as well as a couple of other bug fixes, and performance/stability improvements.

As far as interesting changes go (you can see the full changelog on GitHub)
  • Rename all tile entities to have the correct computercraft: prefix. Existing tile entities should be mapped via a data fixer, but please tell me if you encounter issues.
  • Fix files not being truncated when opened for a write.
  • .read* methods no longer fail on malformed unicode. Malformed input is replaced with a fake character (this is what CC did before).
  • Fix numerous issues caused when wireless modems were attached to wireless networks.
There's really not much I can say about this (aside from how embarrassing a couple of the bugs were), so I'm instead going to chunter about some of the backstory to one of the changes:

In preparation for this release, we updated SwitchCraft to use a development build of CC:T, just to see how stable it was. Unfortunately, the answer came all too soon, as SwitchCraft crawled to a halt.

Due to some other changes, some wireless modem code was now being called far more frequently than before. The function in question acquired a lock, added of removed something from a collection, and released the lock. However, this function would be called both on the main thread, and by several computers off-thread. The main thread would spent an awful lot of time waiting for other threads to finish, and so lagging the sever.

While we were able to fix this quite quickly (it turned out the lock wasn't really needed), we started having other issues - somehow, Java had created far too many threads, and so was unable to execute critical tasks. We didn't previously keep track of how many threads the JVM was using, but after adding that, we were able to see there was a bit of a problem.

While the number of "live" threads was pretty steady, the total number of threads ever created was growing. Rapidly. In fact, the JVM was creating, and then promptly destroying, 50 threads per second. The obvious solution here was to pool the threads: instead of creating and destroying them, we can take a thread from the pool, use it to run some Lua, and then return it once finished. This means you only need to create new threads when the pool is empty.

A couple of modifications to Cobalt later, and we had a graph which looked like this. The total number of threads looks far more healthy, but the number of live threads never goes down! The thread pool was set to evict threads which hadn't been used for a while, so something else was going on. It turns out, computers were not correctly cleaning up their coroutines when they shut down. We had a couple of computers which were stuck in boot loops, which only exacerbated the problem. Thankfully this was trivial to fix, leaving us with a graph which looks much more sane.

Ideally in the future, we would be able to move Cobalt to use one thread for all coroutines, which would substantially reduce the number of threads used (of the server's ~1.2k threads, ~1.1k are created by ComputerCraft), however that is still a way away. This'll have to do for now...

I'm sure there was a point for me writing this massive wall of text, but it eludes me. I hope somebody maybe finds this a little bit interesting :).

#47 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,065 posts
  • LocationTasmania (AU)

Posted 25 November 2018 - 02:16 AM

That's a lot more threads than I was expecting to see. About how many computers is SwitchCraft running on average?

#48 SquidDev

  • Members
  • 1,352 posts
  • LocationDoes anyone put something serious here?

Posted 25 November 2018 - 09:35 AM

View PostBomb Bloke, on 25 November 2018 - 02:16 AM, said:

That's a lot more threads than I was expecting to see. About how many computers is SwitchCraft running on average?
There's about 220 computers on right now. At a bare minimum, they'll be running 3-4 coroutines (the main coroutine, rednet.run, the shell, and maybe also multishell). There's a couple of computers with over 20 coroutines on the go, so not quite sure what they're doing.

#49 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,065 posts
  • LocationTasmania (AU)

Posted 25 November 2018 - 10:51 PM

A couple hundred! I wonder if an automatic shut down timer should be implemented somewhere - there'd need to be a setting to disable it for individual computers (which the gps/repeat scripts should probably toggle automatically), but I suspect that in most cases, if a computer/turtle hasn't received an event in ~five minutes (half an hour? an hour?), it really doesn't need to be running at all. rednet.run seems a good place to slip one in, if it were to be done Lua-side, as that's already being resumed for all event types anyway.

#50 SquidDev

  • Members
  • 1,352 posts
  • LocationDoes anyone put something serious here?

Posted 26 November 2018 - 10:57 AM

View PostBomb Bloke, on 25 November 2018 - 10:51 PM, said:

But I suspect that in most cases, if a computer/turtle hasn't received an event in ~five minutes (half an hour? an hour?), it really doesn't need to be running at all.
Interestingly, we already track when computers receive events, and so have a plot of what computers have done something in the last minute*. From the looks of it, about half of them are doing something, though how important that something is a whole different matter. There's a couple of things to note though:
  • There are a reasonable number of computers which don't do anything, but need to respond to events when they get them. For instance, SC has a lot of shops, which need to respond to monitor touches.
  • CC:T has a command to shutdown all (or some) servers on a computer. We've used this in the past when the server is utterly choking, but you find the number of "on" computers is pretty much at the normal level within a few hours.
  • I'd argue that punishing computers not doing anything is a little counter-productive. Obviously every computer is a bit of a memory (and thread) drain, but computers doing more work than required is definitely more of an issue than computers just sitting doing nothing at all.
It'd kinda be interesting to do a "tab wrangler" type program, randomly shutting down computers if there's more than a specific number running. I'm not sure it'd be effective, but definitely interesting.

*I've just realised that SwitchCraft has just become one giant data-gathering operation for CC now. Oh well.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users