←  Programs

ComputerCraft | Programmable Computers for Minecraft

»

gitget - Version 2 release!

apemanzilla's Photo apemanzilla 07 Mar 2014

GitGet - Version 2

It's been quite a while since I've released a program, so I decided to update gitget. Version 2 is done!

Posted Image

Gitget is a simple, lightweight utility that allows you to easily download entire GitHub repositories, or make installers that install programs from GitHub easily. It can be run directly from the command line, or you can easily edit the program into an installer by changing some variables.


Usage

A lot of stuff has been changed since the original version and gitget is much better in general now. Basic command-line usage remains the same:

Posted Image

Running the program without arguments will display the usage as seen above.

User is the username of the GitHub account hosting the repository.

Repo is the repository name.

Branch is the branch to download. If not specified it will download the master branch.

Path is the local path to save the data to. By default it will be the root folder of the computer.

Note that gitget will overwrite files/folders as necessary.


Making an installer

It is very simple to edit gitget into a basic installer. If you open up the file, starting at line 9 you will see several variables and comments. You can edit these as necessary, then run the program with no arguments and it will download the repository you specified, optionally running code before/after the download as well.


Changes

There have been many changes since the original version:

By default gitget will now download files asynchronously. This results in HUGE speed benefits.

In asynchronous mode, if gitget fails to download a file it will retry instead of simply erroring.

In asynchronous mode, instead of spamming folder/file names, it now shows a nicely formatted progress bar as the files are downloaded.

Fixed a bug with files with spaces in their name.

Simpler preset values - easier to understand when creating an automatic installer/downloader by editing gitget.


Download

You can download gitget here or run this on a computer:

pastebin get W5ZkVYSi gitget


If you have any problems or suggestions, please post! I will update this with more features or fixes if you guys have any.


Edited by apemanzilla, 27 March 2015 - 01:53 AM.
Quote

apemanzilla's Photo apemanzilla 07 Mar 2014

Updated to fix a minor bug. Any feedback?
Quote

apemanzilla's Photo apemanzilla 18 Mar 2014

Updated - the program is faster and uses far less GitHub API calls now.
Quote

Csstform's Photo Csstform 07 Apr 2014

I'm going to try this until I get my own built. Thanks!
Quote

apemanzilla's Photo apemanzilla 07 Apr 2014

View PostCsstform, on 07 April 2014 - 12:03 PM, said:

I'm going to try this until I get my own built. Thanks!
Someone other than me actually uses it :o
Quote

lukasdragon's Photo lukasdragon 12 Apr 2014

May I use this in my new program called RemoteCon?
Quote

Agoldfish's Photo Agoldfish 12 Apr 2014

View PostApemanzilla, on 07 March 2014 - 02:58 AM, said:

You can distribute the resultant program all you want, just remember to give me credit for the downloader somewhere in there ;)

View Postlukasdragon, on 12 April 2014 - 03:15 AM, said:

May I use this in my new program called RemoteCon?
Yep.
Edited by Agoldfish, 12 April 2014 - 03:32 PM.
Quote

apemanzilla's Photo apemanzilla 13 Apr 2014

View Postlukasdragon, on 12 April 2014 - 03:15 AM, said:

May I use this in my new program called RemoteCon?
Absolutely, just add my name somewhere :P
Quote

Blue's Photo Blue 01 May 2014

you could add an optional UI.
Quote

apemanzilla's Photo apemanzilla 02 May 2014

View PostGlass Systems, on 01 May 2014 - 02:19 PM, said:

you could add an optional UI.
This was designed to be simple and easy to use, as well as modular. I feel like a GUI would just slow it down.
Quote

Engineer's Photo Engineer 04 May 2014

View Postapemanzilla, on 02 May 2014 - 01:06 PM, said:

View PostGlass Systems, on 01 May 2014 - 02:19 PM, said:

you could add an optional UI.
This was designed to be simple and easy to use, as well as modular. I feel like a GUI would just slow it down.
That's why it's supposed to be optional xD
Quote

civilwargeeky's Photo civilwargeeky 28 May 2014

Would it be possible to download from a certain folder or maybe files from a repo? I have a repo that has a bunch of lightweight programs, and I really only want one (or at least the ones in the lua folder). If there is an easy way to do this, that would be awesome.
Quote

SquidDev's Photo SquidDev 28 May 2014

I'm impressed it only uses 1 API call. I thought you had to pass the SHA hash to get the tree instead (goes and edits installers).

Why don't you do requests asynchronously though? My installer for SEE and the OneOS installer both do this. I don't know if it is any quicker but it probably should be. I notice you don't check for http errors:

local function download(url, file)
	    save(http.get(url).readAll(),file)
end

I've found that when downloading an entire repo, at least one file will fail at some time (it may just be my internet though). I guess you could just queue these failed files for another download until they have been tried 3 or so times.

I hope I don't sound too critical, I do think this code is pretty clever but those are just the things I would change.
Quote

apemanzilla's Photo apemanzilla 28 May 2014

View PostSquidDev, on 28 May 2014 - 12:08 PM, said:

I'm impressed it only uses 1 API call. I thought you had to pass the SHA hash to get the tree instead (goes and edits installers).

Why don't you do requests asynchronously though? My installer for SEE and the OneOS installer both do this. I don't know if it is any quicker but it probably should be. I notice you don't check for http errors:

local function download(url, file)
	    save(http.get(url).readAll(),file)
end

I've found that when downloading an entire repo, at least one file will fail at some time (it may just be my internet though). I guess you could just queue these failed files for another download until they have been tried 3 or so times.

I hope I don't sound too critical, I do think this code is pretty clever but those are just the things I would change.
Asynchronous requests might be faster as I could send them all in a single batch and then wait for the events for each, but I don't feel like speed is really a problem unless you're downloading OneOS or something else large.

This code was mostly written for my use to download my APIs, but if people start using it I may "professionalize" the code with things like asynchronous requests and better error handling.

What problems were you having with your repo? Often ones I've seen occur when you're out of API calls, or the computer is full. I haven't seen an API call usage error since the update that changed it from one call per folder to one call period, however.

I also intend to add support for git submodules at some point so you can import your APIs without hard-copying the files from another repository.
Edited by apemanzilla, 28 May 2014 - 12:49 PM.
Quote

SquidDev's Photo SquidDev 28 May 2014

View Postapemanzilla, on 28 May 2014 - 12:45 PM, said:

What problems were you having with your repo? Often ones I've seen occur when you're out of API calls, or the computer is full. I haven't seen an API call usage error since the update that changed it from one call per folder to one call period, however.

It wasn't with this script. Sometimes when downloading a lot of files a couple would 'get lost' and return a http_failure. Retrying each file normally sorted this. My computer's internet is pretty rubbish though so that is probably part of the reason.
Quote

apemanzilla's Photo apemanzilla 28 May 2014

View PostSquidDev, on 28 May 2014 - 01:13 PM, said:

View Postapemanzilla, on 28 May 2014 - 12:45 PM, said:

What problems were you having with your repo? Often ones I've seen occur when you're out of API calls, or the computer is full. I haven't seen an API call usage error since the update that changed it from one call per folder to one call period, however.

It wasn't with this script. Sometimes when downloading a lot of files a couple would 'get lost' and return a http_failure. Retrying each file normally sorted this. My computer's internet is pretty rubbish though so that is probably part of the reason.
Make's sense. It could also be a spam prevention measure or similar by GitHub.
Quote

lebalusch's Photo lebalusch 28 May 2014

Great idea I am already seeing the limitations of Pastebin one file at a time. This would be great tool for distributing your whole program library to new pc's in world.

Diamonds to you :-)
Quote

apemanzilla's Photo apemanzilla 28 May 2014

View Postlebalusch, on 28 May 2014 - 04:00 PM, said:

Great idea I am already seeing the limitations of Pastebin one file at a time.  This would be great tool for distributing your whole program library to new pc's in world.

Diamonds to you :-)

Thanks. I may update it in the future with more features due to the recent suggestions. Stay tuned!
Quote

apemanzilla's Photo apemanzilla 29 May 2014

Alright, beginning work on the next version, which will hopefully have these features:
  • Asynchronous mode (If unstable, will be optional and usable via argument or preset mode)
  • Submodule support
  • MAYBE a gui? Maybe.
  • Automatic ignoring of certain files (.gitignore, other internal files) (custom ones available for preset mode)
  • Better preset mode. It sux atm.

Edited by apemanzilla, 29 May 2014 - 06:41 PM.
Quote

Lua.is.the.best's Photo Lua.is.the.best 07 Jun 2014

I.. sense.. AWESOMENESS!!
Now I can finally create an installer without creating a function called get(repo, file) and then calling it like a hundred times!!
Now, all I have to do is:
local username = "aepic999"
local repo = "MyProject"
function hte()
if fs.exists(gitget)
shell.run("gitget", username, repo)
else
print("GitGet application does not exist. Downloading...")
shell.run("pastebin", "get", "6aMMzdwd", "gitget")
shell.run("gitget", username, repo)
end
echo("Installed.")
end
if http then
print("HTTP enabled. You can continue.")
hte()
else
print("HTTP not enabled. App will not continue.")
end
while my other version that used get() was long, and I gave up when I reached the API folder.
EDIT: Put a measure in there to make sure it doesn't still run the code when HTTP API is off.
EDIT 2: Put the hte() function directly after the variables so the if wouldn't get confused, nor the function.
EDIT 3: And just to note, if you want to make this your own, change "username" and "repo", but KEEP THEM STRINGS! If you don't, the shell.run() calls I use will fail.
EDITSAURUS: You can hard-code the strings (hardcoded: not in variable or dependent on variables).
Edited by Lua.is.the.best, 07 June 2014 - 01:57 AM.
Quote