Jump to content




Checkpoint - A work around for persistence

utility lua api

7 replies to this topic

#1 Lupus590

  • Members
  • 1,962 posts
  • LocationUK

Posted 21 March 2018 - 01:10 AM

Checkpoint helps your program get back to where it was by providing checkpoints to continue from, combine this with some intelligent "where were we" logic at the start of each checkpoint and saving of essential data to disk (functionality not provided by Checkpoint) and your program will be able to continue as if the world didn't stop existing :blink:.

Download it with pastebin get mNMJpZG2

Checkpoint is also on GitHub!

Using Checkpoint
Here is an example file (the one I used to test this program, it uses all of the functions provided by Checkpoint).
local checkpoint = dofile(shell.dir().."/checkpoint.lua") --# Checkpoint can also be loaded with require and os.loadAPI

-- DEMO FLAGS

local doTerminateTest = true --# boolean

local doCheckpointRemoveTest = false --# boolean

local doErrorInCallbackTest = false --# boolean

local useCheckpointErrorTrace = false --# boolean

local checkpointFileName = nil --# nil or string, nil for default, string should be absolute path

-- END OF DEMO FLAGS

local args = {...}


--# Checkpoint works using callbacks, we define a bunch here with inventive names like t, c and d
local function t(...)
  print(table.concat({...}, " ")) --# just test code
  args = {"test"} --# notice that this doesn't effect anything, although
  checkpoint.reach("second") --# this tells Checkpoint what to run next, checkpoint.reach returns whatever the callback returns
end

local function c(...)
  print(table.concat({...}, " "))
 
  print("Queuing terminate event, rerun program for next part of test")
  if doTerminateTest then
    os.queueEvent("terminate")
  end
  checkpoint.reach("third")
end

local function d()
  print("Terminate?")
  sleep(0.001) --# catch that terminate if we are on first run, second run won't have a terminate
  print("I'll take that as no")
 
  if doErrorInCallbackTest then
    error("doErrorInCallbackTest")
  end
 
  if doCheckpointRemoveTest then
    checkpoint.remove("third") --# removing checkpoints is more of a debug thing to make sure that your program is flowing in the right direction
    print("removing third checkpoint, prepare for error")

	 checkpoint.reach("third")
  end
  return "return test"
end

--# here we define our checkpoints under the following format:
--#   label - this is the name of the checkpoint and what your program will need to use to refer to it
--#   callback - this is the function which gets called when checkpoint.reach is given the corresponding label
--#   callback args - these are the values passed to the callback when it gets called

checkpoint.add("start", t, 1, unpack(args))

checkpoint.add("second", c, 2, unpack(args))

checkpoint.add("third", d)


local r = checkpoint.run("start", checkpointFileName, useCheckpointErrorTrace) --# identifies if your program needs to continue or starts from given label if it doesn't

print(tostring(r)) --# checkpoint.run returns whatever the last checkpoint callback does



And here is the output of the above:
Posted Image
GIF made with RecGif

Order of Execution
It is advised to not have statements after a call to checkpoint.reach as they are not garanties to be compleated at least once. However this may be useful if the callback needs to do some cleanup which would not be needed to be done if the program closed and restarted at the checkpoint.

Given a program like this one:
local checkpoint = require "checkpoint"

local function c1() --# first callback
  print("1")
  checkpoint.reach("second")
  --# danger zone
  print("2")
end
checkpoint.add("first", c1)

local function c2() --# second callback
  print("3")
end
checkpoint.add("second", c2)

checkpoint.run("first")
The output would be:
1
2
3
But if the program gets interupted at the danger zone, and then restarted, the (effective) output would be:
1
3

Future Improvements
  • Better errors, particularly with tracebacks. If it's the program which is causing the error then I would like the error to message to blame it correctly. If anyone knows how to do this I would appreciate the help.
  • Rewrite: see if I can avoid "function chaining", idea: checkpoint.reach sets a nextCheckpoint value and returns instead of calling the next callback, when checkpoint.run sees this it runs that next checkpoint after setting to nil, if the next one is nil then assume program end This may help improve errors or at lease make implementing improvements to the error reporting easier. I'll look into this when it's not 1:23 AM

Edited by Lupus590, 23 April 2018 - 08:40 PM.


#2 apemanzilla

  • Members
  • 1,421 posts

Posted 21 March 2018 - 01:13 AM

View PostLupus590, on 21 March 2018 - 01:10 AM, said:

Better errors, perticually with tracebacks. If it's the program which is cauing the error then I would like the error to message to blame it correctly. If anyone knows how to do this I would appreciate the help.

You can use trace for that

#3 Lupus590

  • Members
  • 1,962 posts
  • LocationUK

Posted 21 March 2018 - 01:18 AM

View Postapemanzilla, on 21 March 2018 - 01:13 AM, said:

View PostLupus590, on 21 March 2018 - 01:10 AM, said:

Better errors, perticually with tracebacks. If it's the program which is cauing the error then I would like the error to message to blame it correctly. If anyone knows how to do this I would appreciate the help.

You can use trace for that

I've been using the error tracing built into mbs and it doesn't blame the callbacks when I think it should.

Edit: trace gives the same error trace

Edited by Lupus590, 21 March 2018 - 01:20 AM.


#4 Incinirate

  • Members
  • 216 posts
  • LocationDanktown, Memeland

Posted 21 March 2018 - 01:35 AM

Seeing this:
-- Not to bothered that this doesn't work
-- setmetatable(checkpoint, {__call = checkpoint.reach}) -- allow checkpoints to be reached via checkpoint(label) as well as checkpoint.reach(label)

The reason that doesn't work is because the __call metamethod does not directly pass the arguments it was called with, it passes the table too, the identity looks like this:
__call = function(self, ...) end

So if you want it to work change it to something like:
setmetatable(checkpoint, {__call = function(_, ...) checkpoint.reach(...) end})


#5 Lupus590

  • Members
  • 1,962 posts
  • LocationUK

Posted 21 March 2018 - 01:43 AM

View PostIncinirate, on 21 March 2018 - 01:35 AM, said:

setmetatable(checkpoint, {__call = function(_, ...) checkpoint.reach(...) end})

I ended up realising that and decided I didn't want to have to fiddle with the error levels to get it to report properly, I then found out that error levels were borked anyways. That and I decided that checkpoint.reach looks nicer too.

#6 apemanzilla

  • Members
  • 1,421 posts

Posted 21 March 2018 - 02:03 AM

View PostLupus590, on 21 March 2018 - 01:18 AM, said:

View Postapemanzilla, on 21 March 2018 - 01:13 AM, said:

View PostLupus590, on 21 March 2018 - 01:10 AM, said:

Better errors, perticually with tracebacks. If it's the program which is cauing the error then I would like the error to message to blame it correctly. If anyone knows how to do this I would appreciate the help.

You can use trace for that

I've been using the error tracing built into mbs and it doesn't blame the callbacks when I think it should.

Edit: trace gives the same error trace

You can copy the buildStackTrace function from the source of trace and then use that with xpcall (see line 69) inside the checkpoint functions. That should work and properly blame the callbacks. You might also have to tweak the contents of the table (e.g. lines 26 and 27 of trace) to generate the full trace without duplicates properly.

#7 Lupus590

  • Members
  • 1,962 posts
  • LocationUK

Posted 21 March 2018 - 06:54 PM

Updated paste so that Checkpoint doesn't function chain, this does change how programs act when using Checkpoint in particular ways.

Given a program like this one:
local checkpoint = require "checkpoint"

local function c1() --# first callback
  print("1")
  checkpoint.reach("second")
  print("2")
end
checkpoint.add("first", c1)

local function c2() --# second callback
  print("3")
end
checkpoint.add("second", c2)

checkpoint.run("first")

In the old version of Checkpoint, the order of execution would output:
1
3
2

In the new version:
1
2
3

Edited by Lupus590, 21 March 2018 - 07:19 PM.


#8 Lupus590

  • Members
  • 1,962 posts
  • LocationUK

Posted 23 April 2018 - 08:44 PM

OP Edit: Link to GitHub Added, Demo code and Gif Updated.

Code update: We now have error tracebacks! We even tell you what checkpoints were run (since they won't appear in the traceback).





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users