Jump to content




Howl - Lua build system


38 replies to this topic

#1 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 13 December 2014 - 12:37 PM

9 months ago I wrote a program called PPI, which gave the ability to combine multiple files into one file. I've been using this for a while, and hand massive issues with it. Recently I've been rethinking this, and how it could be improved.

The result of this is Howl, a Lua build system. Howl is based mostly on Grunt and Rake. It uses 'tasks' , which can depend on other tasks.

Features Planned
I'm not sure how realistic these are, do say what else you would like below (or on Github). Usage
Firstly you need to download Howl:

pastebin run LYAxmSby get 703e2f46ce68c2ca158673ff0ec4208c/Howl.min.lua Howl

Then you need to define a Howlfile, more on that later.
Then all you need to run is /Howl [task name]. Every directory from the current directory to the root will be checked for the presence of a Howlfile (Howlfile or Howlfile.lua). This will be executed.

Why?
Spoiler

Howlfiles
Spoiler

Screenshots
Spoiler

Contributing
I'd love ideas, suggestions, feedback, or code! Howl is on GitHub, so do send me a pull request, or start an issue.

Edited by SquidDev, 12 April 2016 - 08:07 PM.


#2 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 20 December 2014 - 07:36 PM

Just pushed an update. This adds support for:
  • Oeed's Compilr. This packages all files into one with an emulated file system, perfect for Bedrock apps.
  • Bootstrap script. Instead of rebuilding your file every time, you can create a bootstrap script. This runs loadfiles in the right order to resolve dependencies.
  • Runs in normal Lua. It will now run in any normal installation of Lua with Penlight installed. This is pretty much standard on most installations of Lua I've found, but you can get it from LuaRocks. Colo(u)rs won't work in DOS terminals, sorry.


#3 ardera

  • Members
  • 503 posts
  • LocationGermany

Posted 07 January 2015 - 05:17 PM

Great, works perfectly!

#4 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 12 January 2015 - 07:14 PM

ardera and I have been working hard (thanks a bunch). Version 0.1 has been released! This adds support for:
  • Source verification on combination
  • Traceback and line mapping support
  • Finalizer for errors
  • Busted test framework support
You can see Howl in action on my AES project. I use Howl to both build and test the project, which works wonderfully.

#5 micmou

  • Members
  • 54 posts

Posted 13 January 2015 - 07:06 AM

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

Edited by micmou, 13 January 2015 - 07:13 AM.


#6 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 13 January 2015 - 07:44 AM

View Postmicmou, on 13 January 2015 - 07:06 AM, said:

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

I guess it would be possible. This is mostly designed for development, so working in an environment where you can control the max computer storage (emulator or in Minecraft via config). I'm not sure how it would work though, I'll have a think about it though.

#7 micmou

  • Members
  • 54 posts

Posted 13 January 2015 - 08:23 PM

It could also be used to increase security and realism as well many cool possibilities for servers with A CC economy and security I know is non existent with CC just a cooler and more challenging system

Edited by micmou, 13 January 2015 - 08:41 PM.


#8 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 14 January 2015 - 09:30 AM

View Postmicmou, on 13 January 2015 - 07:06 AM, said:

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

Been thinking about this. Building on a networked computer could mean two things:

Deployment
Build on one computer, then automatically send a file to several computers. For instance if you have several computers that run the same program, you could send the file over rednet. This could be defined with something like:
Tasks:Task "deploy"(function()
  -- Read a file and build an object
  local handle = fs.open(File "mycode.lua", "r")
  local file = textutils.serialize({path = "mycode.lua" contents = handle.readAll()})
  handle.close()
  -- For every computer under protocol "mysystem"
  for _, id in pairs({rednet.lookup("mysystem")}) do
    -- Send the file
    rednet.send(id, file)
  end
end)
  :Requires "mycode.lua"

Then you could have on every computer a process that hosts itself on "mysystem" and listens for messages like that. This functionality could be built into the script itself, though I don't recommend that for deployment.

Running Howl deploy would build "mycode.lua" and send it to all computers.

Multiple "build servers"
I'm still not sure of the purpose of this.

View Postmicmou, on 13 January 2015 - 08:23 PM, said:

It could also be used to increase security and realism as well many cool possibilities for servers with A CC economy and security I know is non existent with CC just a cooler and more challenging system
Realism yes, I don't understand what this has to do with security though. Sorry, slightly confused at the moment.

#9 micmou

  • Members
  • 54 posts

Posted 15 January 2015 - 04:26 AM

View PostSquidDev, on 14 January 2015 - 09:30 AM, said:

View Postmicmou, on 13 January 2015 - 07:06 AM, said:

Would it be at all possible to build from networked compuetrs? To somehow bypass the file size limit on computers? I am not too familiar with CC just got back into it after a long time so don't be mad if this is a face palm question.

Been thinking about this. Building on a networked computer could mean two things:

Deployment
Build on one computer, then automatically send a file to several computers. For instance if you have several computers that run the same program, you could send the file over rednet. This could be defined with something like:
Tasks:Task "deploy"(function()
  -- Read a file and build an object
  local handle = fs.open(File "mycode.lua", "r")
  local file = textutils.serialize({path = "mycode.lua" contents = handle.readAll()})
  handle.close()
  -- For every computer under protocol "mysystem"
  for _, id in pairs({rednet.lookup("mysystem")}) do
	-- Send the file
	rednet.send(id, file)
  end
end)
  :Requires "mycode.lua"

Then you could have on every computer a process that hosts itself on "mysystem" and listens for messages like that. This functionality could be built into the script itself, though I don't recommend that for deployment.

Running Howl deploy would build "mycode.lua" and send it to all computers.

Multiple "build servers"
I'm still not sure of the purpose of this.

View Postmicmou, on 13 January 2015 - 08:23 PM, said:

It could also be used to increase security and realism as well many cool possibilities for servers with A CC economy and security I know is non existent with CC just a cooler and more challenging system
Realism yes, I don't understand what this has to do with security though. Sorry, slightly confused at the moment.

Well many really good security systems for example the one I built at my school query a remote server for like the files to challenge but that is more like databases maybe a community built project as a training thing to demonstrates the concept of networking like in house software companies commonly uses modular building systems to develop software I build networking hooks into main project as a module not likely in CC but still cool to demonstrate the idea easily

EDIT: Sorry If I am hard to understand BTW it is an issue I am working on.

Edited by micmou, 15 January 2015 - 04:28 AM.


#10 ardera

  • Members
  • 503 posts
  • LocationGermany

Posted 16 January 2015 - 06:50 PM

I think he means that he wants to use multiple CCPC's as filesystem resources.

#11 FUNCTION MAN!

  • Members
  • 292 posts

Posted 16 January 2015 - 06:59 PM

View Postardera, on 16 January 2015 - 06:50 PM, said:

I think he means that he wants to use multiple CCPC's as filesystem resources.

IMO that shouldn't be implemented in the core but as a task

#12 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 16 January 2015 - 07:37 PM

View Postardera, on 16 January 2015 - 06:50 PM, said:

I think he means that he wants to use multiple CCPC's as filesystem resources.

I think you are probably right. I'm guessing what micmou is describing is checking if a file has changed on a remote server and rebuilding it if so. File watching is hard enough without using networked computers (editing files outside CC means you can't even add hooks to the fs API).

View PostDr. Poof, on 16 January 2015 - 06:59 PM, said:

IMO that shouldn't be implemented in the core but as a task

micmou, if you really need this I suggest writing something to do this, and executing it with dofile. Howl is currently 60kB, I don't really want to be adding elements to the core without justification. If you do end up implementing it, I would be very interested to see the result.

#13 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 03 February 2016 - 09:16 PM

Ouch: I've left this for a long time. However it isn't dead - I'm working on it on and off. I've just pushed a new build which adds support for Lua style require statements.

You can add this to your Howlfile:
local files = Files()
  :Add "wild:*.lua"	 -- Use files, wildcards or patterns
  :Remove "test"	   -- Exclude files, some such as .git are removed by default
  :Startup "main.lua" -- Defaults to startup
Tasks:RequireAs(files, "build", "build/build.lua")
  :Link() -- Link to the files instead, similar to the bootstrap task

The produced script will allow individual files to require others, instead of manually defining dependencies.

Other changes include dependency tasks depending on produced files and Lua 5.2 support in the combiner.

#14 Lupus590

  • Members
  • 2,029 posts
  • LocationUK

Posted 04 February 2016 - 11:52 AM

Any news on an LDoc clone/port? It will make my life a lot easier for developing Hive.

Edited by Lupus590, 04 February 2016 - 11:53 AM.


#15 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 04 February 2016 - 12:10 PM

View PostLupus590, on 04 February 2016 - 11:52 AM, said:

Any news on an LDoc clone/port? It will make my life a lot easier for developing Hive.

I'm currently working on it. At the moment though I'd recommend running it from normal Lua instead.

#16 Lupus590

  • Members
  • 2,029 posts
  • LocationUK

Posted 04 February 2016 - 12:18 PM

Got a format for comments so I don't have to add/modify them latter? or is that part not finalised yet?

#17 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 04 February 2016 - 12:39 PM

View PostLupus590, on 04 February 2016 - 12:18 PM, said:

Got a format for comments so I don't have to add/modify them latter? or is that part not finalised yet?

I'll just use LDoc's. Howl uses something pretty similar already so feel free to look around in the repo. The general format will be something like:

--- Dumps an object
-- @param object The object to dump
-- @tparam[opt=true] boolean meta Print metatables too
-- @tparam[opt=""] string indent Starting indent level
-- @treturn string The dumped string
local function dump(object, meta, indent)
end

Porting is a big task though due to the massive number of dependencies on Penlight.

#18 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 11 April 2016 - 03:38 PM

Just released another update! Howl now supports uploading to Gists. This is really useful as it means you can upload your built code from the command line, making sharing your code even easier!

Tasks:gist "upload" (function(spec)
  spec:summary "My awesome project"
  spec:gist "703e2f46ce68c2ca158673ff0ec4208c"
  spec:from "build" {
    include = { "Howl.lua" }
  }
end) :Requires { "build/Howl.lua" }

I'd love to do this for pastebin, but sadly it isn't supported :(.

#19 minizbot2012

  • Members
  • 122 posts
  • LocationPalm Bay, Florida

Posted 12 April 2016 - 01:04 AM

Awesome to see gist support in other programs :) (I know its not my API, but, still) I also noticed that you are using tokens, I probably should convert my API over to tokens as well (I don't think user / pass is appropriate for an API for these types of things).

#20 Lupus590

  • Members
  • 2,029 posts
  • LocationUK

Posted 22 August 2016 - 05:06 PM

Bug report, check your github. Also, I posted you a suggestion regarding user experience.

Edited by Lupus590, 22 August 2016 - 05:07 PM.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users