Jump to content




[MC1.12][Fork] CC: Tweaked


70 replies to this topic

#41 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 05 November 2018 - 07:43 AM

View PostSquidDev, on 25 October 2018 - 11:20 AM, said:

stdin is just a wrapper for read, though apparently is somewhat broken.
Just to note that CC:T 1.80pr1.10 (I need a better naming scheme) has been released, which fixes several bugs introduced by the filesystem rework. Amongst the changes, there's a fix for io.stdin (and thus io.read) not working as expected.

#42 IndustrialLemon

  • Members
  • 61 posts

Posted 10 November 2018 - 01:40 AM

Hey sir, quick question. Are there any plans to add some or all of the features prevalent in the computeacube fork? I really enjoy the regex api and the fact that pocket computers act like maps, but also like the changes that come along with CC:tweaked.

#43 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 10 November 2018 - 01:49 AM

View PostIndustrialLemon, on 10 November 2018 - 01:40 AM, said:

Are there any plans to add some or all of the features prevalent in the computeacube fork? I really enjoy the regex api and the fact that pocket computers act like maps.
Pocket computers acting like maps is also present in CC:Tweaked (as well as printed pages having similar functionality). Interestingly enough, Computeacube's implementation of this comes from CC:Tweaked's predecessor.

As far as Computeacube's additional APIs go, I've no plans for adding them to CC:Tweaked. While PCREs are more powerful than Lua's patterns, introducing an API for them is a little grotty. It's another non-standard way of manipulating strings, and doesn't integrate as cleanly with the rest of the language.

Edited by SquidDev, 10 November 2018 - 01:49 AM.


#44 IndustrialLemon

  • Members
  • 61 posts

Posted 10 November 2018 - 02:16 AM

Oh so lua has a way of accomplishing the same thing? I've no problem with that so long as I can pick at words from documents and possibly webpages.

EDIT: Also I'm sure there's a better place to report bugs but while I've got you. I just experienced a bug that was so hard to describe. I have a GPS system that runs off of a disk/startup dir and every time I would edit the disk/startup.lua file and save it would save the file with clips of the programming strewn about at the bottom. Every save it was worse. So, in the beginning, it was just a piece of, 'shell.run("gps host "..x, y, z)'. But a couple saves later and it was more like this.
  shell.run("gps host "..x, y, z)
end
t"..x, y, z)
end
end

t"..x, y, z)
en
end
x, y, z

end


Keep in mind everything after the first 'end' was deleted everytime each save before the next.

Edited by IndustrialLemon, 10 November 2018 - 02:21 AM.


#45 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 10 November 2018 - 10:36 PM

View PostIndustrialLemon, on 10 November 2018 - 02:16 AM, said:

EDIT: Also I'm sure there's a better place to report bugs but while I've got you. I just experienced a bug that was so hard to describe. I have a GPS system that runs off of a disk/startup dir and every time I would edit the disk/startup.lua file and save it would save the file with clips of the programming strewn about at the bottom.
Thanks for the report. I'm working on a fix, for the time being it might be worth downgrading to 1.80pr1.8 - the last couple of versions introduced an experimental rewrite of filesystems, and we're still working out a few kinks.

#46 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 24 November 2018 - 02:32 PM

I've just pushed another CC:Tweaked release, which fixes the remaining issues with the file system rewrite (well, the ones I know about), as well as a couple of other bug fixes, and performance/stability improvements.

As far as interesting changes go (you can see the full changelog on GitHub)
  • Rename all tile entities to have the correct computercraft: prefix. Existing tile entities should be mapped via a data fixer, but please tell me if you encounter issues.
  • Fix files not being truncated when opened for a write.
  • .read* methods no longer fail on malformed unicode. Malformed input is replaced with a fake character (this is what CC did before).
  • Fix numerous issues caused when wireless modems were attached to wireless networks.
There's really not much I can say about this (aside from how embarrassing a couple of the bugs were), so I'm instead going to chunter about some of the backstory to one of the changes:

In preparation for this release, we updated SwitchCraft to use a development build of CC:T, just to see how stable it was. Unfortunately, the answer came all too soon, as SwitchCraft crawled to a halt.

Due to some other changes, some wireless modem code was now being called far more frequently than before. The function in question acquired a lock, added of removed something from a collection, and released the lock. However, this function would be called both on the main thread, and by several computers off-thread. The main thread would spent an awful lot of time waiting for other threads to finish, and so lagging the sever.

While we were able to fix this quite quickly (it turned out the lock wasn't really needed), we started having other issues - somehow, Java had created far too many threads, and so was unable to execute critical tasks. We didn't previously keep track of how many threads the JVM was using, but after adding that, we were able to see there was a bit of a problem.

While the number of "live" threads was pretty steady, the total number of threads ever created was growing. Rapidly. In fact, the JVM was creating, and then promptly destroying, 50 threads per second. The obvious solution here was to pool the threads: instead of creating and destroying them, we can take a thread from the pool, use it to run some Lua, and then return it once finished. This means you only need to create new threads when the pool is empty.

A couple of modifications to Cobalt later, and we had a graph which looked like this. The total number of threads looks far more healthy, but the number of live threads never goes down! The thread pool was set to evict threads which hadn't been used for a while, so something else was going on. It turns out, computers were not correctly cleaning up their coroutines when they shut down. We had a couple of computers which were stuck in boot loops, which only exacerbated the problem. Thankfully this was trivial to fix, leaving us with a graph which looks much more sane.

Ideally in the future, we would be able to move Cobalt to use one thread for all coroutines, which would substantially reduce the number of threads used (of the server's ~1.2k threads, ~1.1k are created by ComputerCraft), however that is still a way away. This'll have to do for now...

I'm sure there was a point for me writing this massive wall of text, but it eludes me. I hope somebody maybe finds this a little bit interesting :).

#47 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 25 November 2018 - 02:16 AM

That's a lot more threads than I was expecting to see. About how many computers is SwitchCraft running on average?

#48 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 25 November 2018 - 09:35 AM

View PostBomb Bloke, on 25 November 2018 - 02:16 AM, said:

That's a lot more threads than I was expecting to see. About how many computers is SwitchCraft running on average?
There's about 220 computers on right now. At a bare minimum, they'll be running 3-4 coroutines (the main coroutine, rednet.run, the shell, and maybe also multishell). There's a couple of computers with over 20 coroutines on the go, so not quite sure what they're doing.

#49 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 25 November 2018 - 10:51 PM

A couple hundred! I wonder if an automatic shut down timer should be implemented somewhere - there'd need to be a setting to disable it for individual computers (which the gps/repeat scripts should probably toggle automatically), but I suspect that in most cases, if a computer/turtle hasn't received an event in ~five minutes (half an hour? an hour?), it really doesn't need to be running at all. rednet.run seems a good place to slip one in, if it were to be done Lua-side, as that's already being resumed for all event types anyway.

#50 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 26 November 2018 - 10:57 AM

View PostBomb Bloke, on 25 November 2018 - 10:51 PM, said:

But I suspect that in most cases, if a computer/turtle hasn't received an event in ~five minutes (half an hour? an hour?), it really doesn't need to be running at all.
Interestingly, we already track when computers receive events, and so have a plot of what computers have done something in the last minute*. From the looks of it, about half of them are doing something, though how important that something is a whole different matter. There's a couple of things to note though:
  • There are a reasonable number of computers which don't do anything, but need to respond to events when they get them. For instance, SC has a lot of shops, which need to respond to monitor touches.
  • CC:T has a command to shutdown all (or some) servers on a computer. We've used this in the past when the server is utterly choking, but you find the number of "on" computers is pretty much at the normal level within a few hours.
  • I'd argue that punishing computers not doing anything is a little counter-productive. Obviously every computer is a bit of a memory (and thread) drain, but computers doing more work than required is definitely more of an issue than computers just sitting doing nothing at all.
It'd kinda be interesting to do a "tab wrangler" type program, randomly shutting down computers if there's more than a specific number running. I'm not sure it'd be effective, but definitely interesting.

*I've just realised that SwitchCraft has just become one giant data-gathering operation for CC now. Oh well.

#51 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 20 December 2018 - 06:02 PM

I've just pushed a new release of CC:T. It's mostly bug fixes, but there's some very minor improvements too:

Changes from the last CC:Tweaked release Minor fixes
  • Fix InventoryUtil ignoring the stack limit when extracting items
  • Fix computers not receiving redstone inputs sent through another block.
  • Fix JEI responding to key-presses when within a computer or turtle's inventory.
On a very unrelated note, I've been playing around with running CC:T on the 1.14 snapshots using a new modding toolchain called Fabric. Rest assured, I'm not going to leave 1.13 or Forge behind, but if you fancy messing around with CC:T (albeit a rather buggy and incomplete version), on Minecraft's bleeding edge, do check it out!

#52 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 21 December 2018 - 08:50 AM

There's a 1.14 already? Oy vey.

#53 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 11 January 2019 - 09:57 PM

It's a new year, and thus must be time for a new CC:Tweaked release! I can't say it's an exciting release, but there's been a lot of internal changes, so I'd like to get them out before the changeset gets too crazy.

The big change this release is that I've (finally) rewritten the http API to entirely use Netty. While Netty has been used for websockets since they were first introduced, we've always depended on the Java standard library to make http requests. While this is convenient, it does end up being rather limiting in what we can do with it. The switch to netty offers several small, but critical advantages:
  • Support http to https redirects: While the http API has always handled redirects on the same protocol, it wouldn't allow redirecting between protocols. This means websites which automatically upgrade you to https will function as expected.
  • Allow using the PATCH request method: We've allowed using most other methods for a while now (GET, POST, PUT, DELETE, etc...), but not PATCH. This meant you had to use custom headers to interact with some REST APIs.
  • Less thread usage: Previously, every HTTP request would be run on a separate thread, which was a little inefficient. The asynchronous nature of netty means that ComputerCraft should rarely go above 8 threads, how ever many concurrent HTTP requests you are making.
I honestly didn't mean to write that much, but on the bright side, it now makes it look like there were actual changes this release! Even if they didn't impact anyone.

Just a couple of other things which changed this release:
  • The config file has had a bit of a spring cleaning: Related properties should be grouped into child categories (http, turtle, peripheral), which makes the whole file a little easier to navigate. This should be automatically upgraded when you load the mod, but let me know if you experience issues.
  • I've also begun work on implementing the long-discussed HTTP limits, as described in this issue. I've tested them a fair bit, but again if you experience issues (or regressions with the netty changes), do report them so they can be fixed ASAP!
  • There's also been an awful lot of cleanup going on throughout the main CC codebase. This issue documents it a little more, but we're pretty much just replacing our home-grown implementations with ones built into vanilla or Forge. It means we can delete an awful lot of code (always a plus), and hopefully make things more consistent too.
There's a couple more things I've not mentioned here in the release notes, if that's your sort of thing.


#54 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 21 January 2019 - 05:30 PM

It's a bit soon to the last one, but there's just been another CC:Tweaked release. Not a lot has changed, but there's a couple of useful bug fixes and one (not very) exciting optimisation, so I thought it'd be better to get it into the wild sooner rather than later.

Changelog
  • Allow seeking within ROM files. I've wanted this since the initial implementation, but there were some unexpected problems. I've finally got round to implementing this (though it possibly is of dubious use).
  • Make several tile entities (modems, cables, and monitors) non-ticking, substantially reducing their overhead, On SwitchCraft this reduced tick time by about 10ms, though this is probably more than most people will see due to the Sponge overhead and the rather large number of monitors and cables.
    That said, it's still a pretty large change, so let me know if you experience issues related to any of the peripheral blocks.
Minor fixes
  • Fix cables not rendering the breaking steps
  • Try to prevent /computercraft_copy showing up in auto-complete.
  • Fix several memory leaks and other issues with ROM mounts.
  • Fix not being able to craft upgraded turtles or pocket computers when Astral Sorcery was installed.


#55 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 24 January 2019 - 09:44 PM

Posted Image

Spoilers! Yeah, I can't draw numbers very well.

Edited by SquidDev, 24 January 2019 - 09:45 PM.


#56 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 24 January 2019 - 10:39 PM

Oh hey, you added turtles! :D

;)

#57 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 21 February 2019 - 10:22 PM

Posted Image

Yeah, I'm bad at screenshots.

Anyway, I've just pushed an alpha of CC:Tweaked for Forge 1.13.2. A couple of things to note:
  • Forge is still pretty unstable (for instance, the current build, .45 is broken - you'll have to use .44). As a result, there's currently some features we can't implement and bugs we can't fix (there's some container desync, turtle upgrades don't always render. This should be resolved over the coming weeks.
  • This is not compatible with previous worlds: the save format has changed a lot, and there's nothing I can do to load worlds created in 1.12 or before.
  • We're still an alpha, and Minecraft internals have changed a lot - there will be bugs, and there are still changes to be made.
If you do have issues, questions, or just would like to see the work done, check out the pull request!

It's probably worth mentioning at this point that 1.12 will remain in development for a wee while yet - I'm expecting it to take a while before 1.13 has become "mainstream" (and 1.14 will probably be out before then).

#58 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 23 February 2019 - 11:02 AM

There's just been another CC:Tweaked release! Probably the most obvious change here is that we're no longer using the "1.80pr1.x" versioning scheme - it's now just 1.81.0. There's a little bit more of an explanation on the issue, but it basically boils down to it being unreasonable to tie ourselves to CC versions, especially given there's not been any changes to the repo for almost a year.

Now, onto more exciting things! Like with most CC:T releases, there's been a fair few changes, but you're unlikely to notice any of them. The "big" change is that Cobalt got a lot of long-needed attention, meaning it's even more compliant to standard Lua, and a wee bit less resource intensive in places. Many thanks to SwitchCraft yet again for helping test things.

Also, many thanks to Lignum and WIlma456 for their PRs - one of the sad things about CC:T is that it has had much less community involvement than the original repo, so it's always nice to have other contributors! Anyway, the changelog:

Changes from the last CC:Tweaked release
  • Handle connection errors on websockets (Devilholk)
  • Make require a little more consistent with PUC Lua, passing the required name to modules and improving error messages.
  • Track how long each turtle action takes within the profiling tools
  • Bump Cobalt version
    • Coroutines are no longer backed by threads, reducing overhead of coroutines.
    • Maximum stack depth is much larger (2^16 rather than 2^8)
    • Stack is no longer unwound when an unhandled error occurs, meaning debug.traceback can be used on dead coroutines.
  • Reduce jar size by reducing how many extra files we bundle.
  • Add term.nativePaletteColo(u)r (Lignum)
  • Split colours.rgb8 into colours.packRGB and colours.unpackRGB (Lignum)
  • Printers now only accept paper and ink, rather than any item
  • Allow scrolling through the multishell tab bar, when lots of programs are running. (Wilma456)
Minor fixes
  • Fix modems not being advanced when they should be
  • Fix direction of some peripheral blocks not being set
  • Strip \r from .readLine on binary handles.
  • Websockets handle pings correctly
  • Fix turtle peripherals becoming desynced on chunk unload.
  • /computercraft tables are now truncated correctly


#59 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 25 February 2019 - 09:46 AM

View PostSquidDev, on 23 February 2019 - 11:02 AM, said:

The "big" change is that Cobalt got a lot of long-needed attention, meaning it's even more compliant to standard Lua

Well THAT sounds ominous...

#60 SquidDev

    Frickin' laser beams

  • Members
  • 1,409 posts
  • LocationDoes anyone put something serious here?

Posted 25 February 2019 - 10:45 AM

View PostBomb Bloke, on 25 February 2019 - 09:46 AM, said:

Well THAT sounds ominous...
At this point it's just really minor things about when/how debug hooks are called, how the stack is unwound, etc... There shouldn't be any noticeable behaviour changes. The other changes are far more likely to cause issues.

Edited by SquidDev, 25 February 2019 - 01:04 PM.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users