6 replies to this topic

[KingofGamesYami]

So I've been messing around with github's API to create an installer / updater, and I found this:

https://developer.gi...s/#get-contents

Which has been very helpful so far. I can now easily download everything in a repo, but I'm wanting to check a file before downloading, to see if it is already the latest version.

I think I can use the sha key to do this somehow, but I'm not sure how. I heard on IRC you can, but they could be mistaken. From Stack Overflow I got that it's the hash of the blob, but I don't know what a blob is or how I could check against it.

{
  "type": "file",
  "encoding": "base64",
  "size": 5362,
  "name": "README.md",
  "path": "README.md",
  "content": "encoded content ...",
  "sha": "3d21ec53a331a6f037a91c368710b99387d012c1", --#this
  "url": "https://api.github.com/repos/octokit/octokit.rb/contents/README.md",
  "git_url": "https://api.github.com/repos/octokit/octokit.rb/git/blobs/3d21ec53a331a6f037a91c368710b99387d012c1",
  "html_url": "https://github.com/octokit/octokit.rb/blob/master/README.md",
  "download_url": "https://raw.githubusercontent.com/octokit/octokit.rb/master/README.md",
  "_links": {
    "git": "https://api.github.com/repos/octokit/octokit.rb/git/blobs/3d21ec53a331a6f037a91c368710b99387d012c1",
    "self": "https://api.github.com/repos/octokit/octokit.rb/contents/README.md",
    "html": "https://github.com/octokit/octokit.rb/blob/master/README.md"
  }
}

[Lyqyd]

I use the sha github provides for each file in Packman. I end up just storing the sha from the last time the file was downloaded to compare against the results returned by github in the future. Reading a string out of a file is much less computationally intensive than trying to calculate the hash again!

[KingofGamesYami]

That's what I was thinking of doing. Thanks!

Would it be worth storing it as a comment on top of the program, vs in a separate file? Or rather, is there any difference in speed reading the first line of a 1000 line file vs 1 line file?

[Lyqyd]

The one line file would likely be marginally quicker, but I'm not sure how much of a difference it would make in practical terms--you're also using data from a website. With Packman, I store the sha hashes for each file in a single file per installed package, which may be wise to do if you're going to be using this technique to keep multiple files up to date. An "installed-versions" file or similar such would do it.

[KingofGamesYami]

I thought about creating a new topic, but this thread "inspired" the code I wrote so...

I'm having troubles.

Spoiler

--this is an improved installer which will remove the need to track file paths
--eventually it will allow updates without redownloading the entire repository
--author: KingofGamesYami


--loading json
local json = {}

do
        local env = {} --weird meta enviorment stuffs
        setmetatable( env, {__index = _G} )
        local response = http.get( "https://raw.githubusercontent.com/lupus590/CC-Hive/New-installer/src/Shared/json.lua" )
        if not response then
                error( "Could not get json.lua", 0 )
        end
        local func, err = loadstring( response.readAll(), "json" )
        setfenv( func, env )
        if func then
                local ok, err = pcall( func )
                if not ok then
                        error( err, 0 )
                end
        else
                error( err, 0 )
        end
        for k, v in pairs( env ) do
                json[ k ] = v
        end
end

local dirsToGet = { {url="https://api.github.com/repos/lupus590/CC-Hive/contents/src/Shared", path="Shared" } }

if turtle then --setting up places to scan
        table.insert( dirsToGet, {url="https://api.github.com/repos/lupus590/CC-Hive/contents/src/Turtle", path="Turtle"} )
else
        table.insert( dirsToGet, {url="https://api.github.com/repos/lupus590/CC-Hive/contents/src/Client", path="Client"} )
end

local filesToGet = {} --a table of files we need to download
local isRunning = true

local installed = {}

local function scanDir( str ) --#a function for scanning a directory and allocating it's contents to the above tables
        local f = json.decode( str )
        for k, v in pairs( f ) do
                if v["type"] == "dir" then
                        table.insert( dirsToGet, {url = v.url, path = v.path:match( "src/(.+)" ) } )
                elseif v["type"] == "file" and v.name:match("%.(.-)$") == "lua" then
                        table.insert( filesToGet, {url = v.download_url, path = v.path:match( "src/(.+)%.lua" ) } )
                        installed[ v.path ] = v.sha
                end
        end
end

local function getFile() --a function that downloads a single file at a time
        while isRunning or #filesToGet > 0 do
                if #filesToGet > 0 then
                        local toGet = table.remove( filesToGet, 1 )
                        local r
                        repeat
                                print( "Downloading " .. toGet.path )
                                r = http.get( toGet.url )
                        until r
                        print( "Downloaded " .. toGet.path )
                        local file = fs.open( toGet.path, "w" )
                        file.write( r.readAll() )
                        file.close()
                else
                        sleep( 0.1 )
                end
        end
end

local function getDir() --a function that makes directories
        while(#dirsToGet > 0)do
                local sPath = table.remove( dirsToGet, 1 )
                local response
                repeat
                        response = http.get( sPath.url )
                until response
                fs.makeDir( sPath.path )
                scanDir( response.readAll() )
                response.close()
        end
        isRunning = false
end

parallel.waitForAll( getDir, getFile, getFile, getFile ) --runs 3 getFiles, 1 getDirs at once!
print( "Done!" )
local file = fs.open( "installed" )
file.write( textutils.serialize( installed ) )
file.close()

The code works perfectly without the last three lines, however when they are added this line:

local file = fs.open( "installed" )

errors with

Expected string, string

Which I find really, really aggravating. Obviously fs.open does not expect two strings. Neither does file.write, nor textutils.serialize. Yet without those lines, no errors occur.

[Lyqyd]

Well, actually, fs.open does expect two strings. How else would it know whether you want the file opened in read mode, write mode, append mode, or one of their binary counterpart modes? It appears that you may want to use "w", for write mode, for the second argument.

[KingofGamesYami]

Did I really?.. Oh I did .. *hides under a rock*