7 replies to this topic

[viluon]

vBenchmark

---

The ultimate benchmarking tool for ComputerCraft

Hello community! I haven't posted any project for quite a long time so I've decided to finally release a semi-complete project of mine called vBenchmark (viluon's benchmark), a modular benchmarking solution for ComputerCraft 1.6+ and 1.7.

Screenshots

vBenchmark during a test

The usage screen

Portable PC support is being worked on

Benchmark tests

As of version 0.2.3, vBenchmark is able to perform the following tests:

• Optimized write test
  
• Unoptimized write test
  
• Color swap/rainbow test
  
• Line clear speed test
  
• Clear speed test
  
• Buffer scroll test
  
• Anti-aliasing test
  
• SHA256 hash test
  
• AES Encryption perf. test
  
• AES Decryption perf. test
  
• Table entry creation
  
• Table entry deletion
  
• Table entry access
  
• Diffie-Hellman key exchange
  
• Simple compilation
  
• Medium compilation

I am working on new tests to better evaluate the performance of your platform. ComputerCraft 1.7 benchmarks are being worked on.

Compatibility

ComputerCraft versions and devices
So far, vBenchmark has been tested on CC 1.6x and 1.7pr20. It should work for screens of computers, turtles and monitors (with text scale >=1). Support for portable devices is being worked on (the GUI has a wrong layout but the controls work).

ComputerCraft emulators
vBenchmark has been successfully tested with CCEmuRedux.

Usage

The vBenchmark usage can be obtained via

vBenchmark.lua help

How to interpret the results?
The results of the benchmark are measured in calls over time. The higher the resultant number, the better your platform does and the more times the test finished.
I want to get more precise results
Try to increase the limit for one test.

vBenchmark.lua -maxtime 5 -maxcalls 50000

will set the maximum time limit a test can take to 5 seconds and the maximum number of calls to 50,000.
I don't need <test>
If you only want to benchmark a specific part of your platform (such as see how good it is at graphics), you can use the testonly option and specify a list of test codes that should be benchmarked.

vBenchmark.lua -testonly ow

will benchmark only using the Optimised write test. Keep in mind that you can retrieve the list of test codes using

vBenchmark.lua -list --stop

Another option to use when defining a custom-built benchmark is -exclude. Given a list of test codes, -exclude will remove these tests from the next benchmark. Use as follows

vBenchmark.lua -exclude ow

will benchmark with all available tests except Optimised write test.
How should "lists" be formatted?
Lists given to options should be comma separated and include no space characters, e.g. "hsh,aese,aesd," would make a valid list whilst "hsh, aese, aesd," would not. Do not forget to include a comma at the end of the list.
Updating
To update, simply run

vBenchmark.lua update

Keep in mind that vBenchmark only updates to released versions, i.e. updating an experimental version will install the latest release.

Credits

Thanks to Xenthera for anti-aliasing code and Anavrins for Diffie-Hellman Key Exchange. AES implementation by SquidDev. Thanks to all programmers who contributed to the LuaSrcDiet project. This project also uses SHA256 implementation by GravityScore, edited for safe yielding. vBenchmark was inspired by 1lann's benchmark script and was released because I couldn't stand the feature lack of this program (no offense).

News

Update News
10. 6. 2015 The experimental 0.2.5 was released. It mainly focuses on code improvements (more information in the Version history).

10. 6. 2015 I am getting ready to switch to a modular approach which allows test packages to be downloaded on demand. This will greatly reduce the size of the core and will allow for much faster maintenance and development. The community can also contribute new tests. Moreover, vBenchmark will have the feature to benchmark a program.

8. 6. 2015 The experimental version 0.2.4 was released. 3 new benchmark tests were added and the source code is now undergoing a revamp which includes renaming of variables and overall improving readability.

Download

---

Grab the latest release from Pastebin.

vBenchmark minified

pastebin get M4RK9hep vBenchmark.lua

vBenchmark source code

pastebin get nCEk57MH vBenchmark.lua

vBenchmark experimental version

pastebin get NGJhBGm9 vBenchmark.lua

Warning: vBenchmark is beta software. It could and will cause issues. I am open to constructive criticism and will welcome any bug reports.

Version history

0.2.5 experimental
+code revamp continues, new GUI code improvements
+test marks now display after math.floor()ing to improve readability (output file still includes full versions)
+test marks now align to the right
+localized common APIs and functions for faster execution and better minifying
+improved dependency download process, downloads all APIs in a single run

0.2.4 experimental
+started code revamp work, change from space to tab indenting, started renaming variables
+minor comment improvements
*3 new tests, term.blit test, text file read and write test

0.2.3 build 151
First public release

<0.2.3
Development versions only released to a few CC Pro's.

License

vBenchmark is licensed under the MIT License.

The MIT License (MIT)

Copyright (c) 2015 Andrew Kvapil

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Feel free to post your benchmark results!

This is great! How can I help you?
An upvote () and a nice comment/bug report/test result is the best you can do

Edited by viluon, 10 June 2015 - 04:23 PM.

[Creator]

That all seems very nice and advanced. However, who is we?

[Dahknee]

Creator, on 08 June 2015 - 02:11 PM, said:

That all seems very nice and advanced. However, who is we?

I have learnt, viluon taught me , Although now I know what it does, it is actually pretty cool especially for comparing emulators!

Edited by DannySMc, 08 June 2015 - 04:46 PM.

[SquidDev]

DannySMc, on 08 June 2015 - 02:22 PM, said:

Although I do not see a use for this? What would you use it for? Sorry for my ignorance, I just don't understand.

I know this has been used to profile LuaJC and I will probably use it if I ever get round to finishing my emulator. Basically this can be used to find what version, modifications, emulators, etc... run the quickest and in which regions - for instance CCEmuRedux might have faster drawing but LuaJC has faster computation, etc...

viluon, on 08 June 2015 - 02:01 PM, said:

AES and SHA256 implementations by SquidDev.

The SHA256 implementation isn't mine - well I hope so or otherwise I'm writing code without realising it.

Edited by SquidDev, 08 June 2015 - 02:35 PM.

[supernicejohn]

This seems very cool and scientific! Saw one possible bug though, otherwise great!
http://postimg.org/image/c5natuzlb/ <-- At table entry deletion it seems to say that 2809.524 is worse than itself, is it rounded to 3 after comparison or does it check only of one is greater otherwise false?
Nice work! To all of the people under credits aswell!

[Creator]

This is amazing. The GUI is nice and is seems very scientific.

[TheMrIron2]

Very impressive, I can now compare CC emulators' speed to a real CC computer's speed, which proves very useful. I could probably also see how badly server lag affects computers, although I haven't tried that yet.

[LoganDark]

Thanks! I never thought I would come across a useful benchmarking tool until I found this beauty. May I ask, who made the ASCII art?