Jump to content




OCL [Operating System Compatibility Layer] 0.4

utility api lua

59 replies to this topic

Poll: OCL (21 member(s) have cast votes)

What is your opinion of the OCL? - Tell me why in the comments!

You cannot see the results of the poll until you have voted. Please login and cast your vote to see the results of this poll.
Vote

#21 minizbot2012

  • Members
  • 122 posts
  • LocationPalm Bay, Florida

Posted 11 July 2012 - 08:14 PM

if parentShell == nil then
-- Run the startup from the ROM first
local sRomStartup = shell.resolveProgram( "/rom/startup" )
if sRomStartup then
  shell.run( sRomStartup )
end

-- Then run the user created startup, from the disks or the root
local sUserStartup = shell.resolveProgram( "/startup" )
for n,sSide in pairs( redstone.getSides() ) do
  if disk.isPresent( sSide ) and disk.hasData( sSide ) then
   local sDiskStartup = shell.resolveProgram( fs.combine(disk.getMountPath( sSide ), "startup") )
   if sDiskStartup then
	sUserStartup = sDiskStartup
	break
   end
  end
end

if sUserStartup then
  shell.run( sUserStartup )
end
end
that is what i have in the shell file, i am going to try a CC re-install
EDIT:
after reinstalling computercraft it started to work!!, it was apparently a conflict with the events api.

#22 toxicwolf

  • Members
  • 201 posts
  • LocationUnited Kingdom

Posted 11 July 2012 - 08:28 PM

View Postminizbot2012, on 11 July 2012 - 08:14 PM, said:

EDIT:
after reinstalling computercraft it started to work!!, it was apparently a conflict with the events api.
Hmm.. Could you link me to this events api, so I could try and find what might have caused the conflict?

#23 minizbot2012

  • Members
  • 122 posts
  • LocationPalm Bay, Florida

Posted 11 July 2012 - 08:46 PM

http://www.computerc...075-events-api/ is the one i was using. (front page of apis and utilities section of the program library)

#24 toxicwolf

  • Members
  • 201 posts
  • LocationUnited Kingdom

Posted 11 July 2012 - 09:48 PM

View Postminizbot2012, on 11 July 2012 - 08:46 PM, said:

http://www.computerc...075-events-api/ is the one i was using. (front page of apis and utilities section of the program library)
Okay, thanks for that, I will take a look into the issue. https://github.com/t...lf/OCL/issues/3

EDIT: Okay, I reproduced the problem, and as far as I can tell, it actually seems to be purely an Events API bug, with nothing to do with the OCL. I'll go ahead and close the GitHub issue for now.
I've let the creator know about the issue, so hopefully something can be resolved, as it does look like a very good resource.

Edited by toxicwolf, 12 July 2012 - 08:42 PM.


#25 toxicwolf

  • Members
  • 201 posts
  • LocationUnited Kingdom

Posted 13 July 2012 - 03:04 AM

View PostMysticT, on 11 July 2012 - 01:30 PM, said:

View Posttoxicwolf, on 11 July 2012 - 08:21 AM, said:

Okay, thanks B)/> I will see about implementing some kind of multi-boot system into the OCL, or something :)/>
The multi-boot is actually in another file, wich is placed as boot/loader. It checks for files with .startup extension, and shows a menu to choose wich one to boot.
In the .startup file you put all the code to start your os, load every api (since there's no apis loaded, wich is better to make an os) and start the shell (GUI or text-based). This can be changed, but we should make a system that every os follows, so you can have all of them B)/>
Take a look at today's commits on the GitHub page for the OCL (links are above in OP), and see if you think of what I've done. I hope it was a step or three in the right direction (after chucking the custom shell code out of the window etc) B)/>

I also changed the write function to something of my own creation, and while it is probably inferior, it does the job well enough to warrant the change. It is also a lot smaller than the original.

#26 kazagistar

  • Members
  • 365 posts

Posted 13 July 2012 - 05:31 PM

Unless I am misunderstanding the source code I see, I feel like OCL is trying to add too much functionality, which might be incompatible with some operating systems. What EXACTLY is the stated purpose of this program? I tend to come from the unix-y school of multiple small utilities functioning independently, and I feel that you are adding too many features in one monolithic component. Here is what I feel would be most useful for me, each as a separate concern in some way.

(1) Parallel boot loader, that can be used to run a custom bios. Allows the user to select which OS they want to run.

(2) A standard compatibility library between operating systems.

In my opinion, these two should be separate, to the point of being essentially separate projects, so that if someone does not want OCL, they can just use the bootloader, and if someone does not want to use the boot-loader, they can still run an OCL operating system without any problems. I propose that bios.lua look into "/rom/boot/" and "/boot/" which will contain replacement bios files for each possible OS, and run the selected one. Other features can be added, but I really don't think it should be forcing the programmers to conform to the OCL specifications if they don't want to for whatever reason. If you don't, people might be forced to run your system in a compatibility layer just to run their system properly which would be a silly state of affairs.

As for OCL itself, it makes a lot of assumptions about the nature of your OS which might not hold true. For me, this is particularly relevant, because the way I am designing my OS breaks a lot of those assumptions.

(1) What if an operating system does not load it's programs and such from the hard drive, but rather from some other source, like HTTP or rednet? In other words, I might want to, as part of booting my OS, load my shell/programs/libraries from a specific remote location, and then use loadstring to parse them directly into functions, so the system is at no point stored on the local computer? For example, on a multiplayer server, you might be worried your turtles could be captured by "the enemy", and so you run the OS in such a way that if the turtle is captured, there is no way to recover your code. I don't see how I can sensibly implement OCL.hasCustomShell() on such a system. Maybe this is not really the concern of OCL.

(2) The ID and read/write functions are cool.

(3) I don't understand getRoot(). What is the use case for this? All the use cases that I can think of are covered by the other directory-getting functions.

(4) The other path getting functions are neat, and a good idea, but they share the same problem as #1. I really don't see a fix for this that does not increase complexity by a significant amount; I guess the only way to create a "remote" filesystem would probably be to modify "fs" significantly and allow special paths that link to virtual files.

(5) I really like the abstraction on the idea of "apps". It is clean, simple, and allows programs to be written for many systems at once. I feel like this can and should be the core of this library. Here is how I would recommend expanding this: make it possible to query the system for what apps can be run, and then run apps directly. Thus, someone wants to write a multi-OS custom shell, they can easily see what apps are visible, and run one, irrelevent of how the system is assembled. Or, if you wish to deploy a system as multiple separate apps, you can call one from the other, without having to deal with any of the implementation details.

(6) Centralized logging system is an awesome, amazing idea. Not sure what the purpose of getLogFile is though... what if the OS does not actually write the logged information to a file, but just keeps it in memory? Or sends it to be stored on another computer somewhere? What multi-OS program needs to be able to read a log file? If you have user-based file permissions, it is very possible that it wont be able to open the file anyway.

(7) Unless I missed it, there is no removeFromStack(). Thus, if you addToStack, you will be permanently one off. Am I wrong? Also, I don't see the point of this exactly. If the OS wishes to track what programs you have run, it can override the various methods of calling a program to do so. What is the use case for this exactly?

(8) I have no idea how exit program possibly works.

(9) OCL.run() does not allow arguments to be passed into the called program. This is a major, major flaw, in my opinion.

This kind of API really is a very cool boon to authors of operating systems and software. However, I feel that the focus is too scattered, and forces certain ideas of what an OS should and shouldn't do which prevents it from reaching it's full potential. The problem I feel is that there is a disconnect between the natural lua way of viewing a program as a function, and the way this program view it, as a file. In lua, there is no real difference between a program file and a function, save the fact that one is stored on the disk. You load em up into a function, pass in parameters, and get return results: this is how it SHOULD be, how I structure my code, how I structure my OS. My "run" functions can take a path or a function: both work the same.

One other thing: what kind of functionality are you assuming is availible to the program? Are you assuming that all the standard libraries exist unchanged? Because I certainly have done my fair share of tinkering with libraries. What if "fs" is modified or removed? The program will silently fail. This should at the very least be made explicit, but here is another proposal: turn this into a more generic "dependancy manager". The interface would let you view what programs and libraries are availible, at what versions. You could load the libraries you need (if they are not loaded) via a standard interface, or run installed apps.

Then, separately, you would define a number of "library interfaces". These could include a read-write interface, a logging interface, a "standard filesystem locations" interface, etc. An OS can implement whichever ones in wants to, and the user can require whichever libraries and programs they want to, and the system would determine if the program can or cannot run in the given environment. The program would look like this:
requireLib("OCL.term", 1.0)
requireLib("OCL.log", 1.0)
requireLib("OCL.paths", 1.0)
requireLib("someNetworkingSystem", 0.32)
requireApp("programCore") -- these could have versions too in theory
requireApp("someDatabase")
OCL.log("Program started")
OCL.term.nPrint("This program is going to call another")
local result = runApp("programCore")
someNetworkingSystem.send(result)
runApp("database","store",result)
-- etc
requireLib would make sure the library exists at the correct version. It might load the library from a file if needed or whatever else the OS writer decides. If it cannot load a correct version of the library into the current global namespace for whatever reason, it will throw an error. Thus, the program writer will be assured that all the correct functionality exists, and the user will be informed with a helpful error message infoming them exactly what is missing from their system.

PS: Could you please license it? Linking or distributing non-licensed copyrighted code is illegal, so it would be helpful if you tossed in some kind of disclaimer or whatever allowing usage. I am a big fan of WTFPL, but whatever you prefer.

#27 toxicwolf

  • Members
  • 201 posts
  • LocationUnited Kingdom

Posted 13 July 2012 - 06:12 PM

kazagistar, thanks you for your very insightful (albeit very long B)/> ) post. To be quite honest, some of the things you mention I have not thought about or didn't really know about - I'm quite the amateur.
Thanks for pointing out some things that I missed, such as for the run function. I agree with the points you bring up though, you have a clear idea on a much better way for the OCL to work.
I will try to rebuild the system, but I could probably do with help. After all, I do want this to be a community run project, so that it can be as good and as useful as possible.

For now, I'll be thoroughly re-reading your post and try to see what I can come up with.

PS: Thanks for the heads up on the license thing, I really didn't know about that! :)/>

#28 kazagistar

  • Members
  • 365 posts

Posted 13 July 2012 - 06:51 PM

I will very gladly start submitting patches to your github project as soon as the licensing is set up. The main reasons I haven't started writing code already are the following.
  • Since the idea was yours, I wouldn't want to just start hacking away at it in a way wouldn't like at all.
  • If you didn't like my alternative version, we would end up with competing standards, which would weaken both. I would rather have a standard that I feel is flawed but functional then have two competing standards that conflict.
  • I have a tendency to over-abstract, which can be really really bad for something like a voluntary interface standard. If the minimum requirements are easy to understand, then it is more likely that an OS author or program writer will work to conform to them. So it is better for me to work with others.
Code, like every written work, is copyrighted automatically under the full possible protections of the law, whether you write a copyright statement or not. In theory, code could belong under a "procedure", which is something that cannot be copyrighted, but alas, that is not how the courts decided many decades ago. Thus you have the same legal protections for code as you would for a song, book, or picture. Of course, in a community like this, it is very very unlikely that anyone will ever sue (that would be silly) but the principle still stands. That is why there exists a range of licenses for code. If you just give someone code, they cannot legally redistribute it, even if they modify it, in the same way as they cannot redistribute a mp3, but if you can grant them that permission, and you can attach terms to that permission. For example, they cannot sell it commercially. Or they have to give the author due credit. Or they have to give you a cut of all the profits. Or they have to distribute any modified copies with the source code, under the same license. As I said, I prefer the WTFPL because I honestly don't care what people do with my code, who they sell it to, or if they claim to have made it.

#29 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 13 July 2012 - 07:18 PM

I agree with kazagistar, that's why I said my os won't be compatible (with the current version), cause it would be to hard to adapt my os design to that api.
And what I said was to have a boot loader independant of the api, so you can use one without the other.
I would start writing code now, but I think it would be better to agree on the way it will work before starting to code it.

#30 kazagistar

  • Members
  • 365 posts

Posted 13 July 2012 - 08:15 PM

Here is some ideas on how I think the basic bootloader should work:

First, all defined variables and functions should be local. The it should pass on to the custom loaded bios the same global state that was passed to it, unmodified.

There should be three folders to stick the custom bios: "/rom/bios/", "/bios/" and "/disk/bios/". These should be declared as constants at the top of the program, so it is easy to modify these path names when installing the custom loader.

There should be some way for a user to set a default and effectively disable it, even if they are on SMP and don't have the ability to modify the ROM. I recommend having a file called "default" that, if it exists, is run without prompting. If there are multiple default files, it will select in this order: "/rom/bios", "/bios/", "/disk/bios". (Server owners should have final say or be able to disable it in this way if they want, and the local system needs to be searched before disks, so people can make hack-proof systems.)

The bootloader will then assemble a list of all the bios files in each location by path.

If there are no items in any bios folder, the bootloader will display an error, explaining that you will need a disk to unbrick the computer.

If there is only one option, the bootloader will silently start that bios. This way, a server owner can put the craftos bootloader into "/rom/bios", and everything will work exactly like it normally does, until a user adds more systems.

If there is more then one option, the bootloader will prompt the user to select an option, and then run that option.

Comments or ideas?

#31 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 13 July 2012 - 08:47 PM

View Postkazagistar, on 13 July 2012 - 08:15 PM, said:

First, all defined variables and functions should be local. The it should pass on to the custom loaded bios the same global state that was passed to it, unmodified.
Of course, the bios shouldn't modify anything, just load the boot files.

View Postkazagistar, on 13 July 2012 - 08:15 PM, said:

There should be three folders to stick the custom bios: "/rom/bios/", "/bios/" and "/disk/bios/". These should be declared as constants at the top of the program, so it is easy to modify these path names when installing the custom loader.
How could I never thought of booting from disk, really needed. Just one thing, It would be better to use "boot" instead of "bios", since the bios is what searches the files and loads them, and the files are used to boot the os. Just a naming thing, not really important.

View Postkazagistar, on 13 July 2012 - 08:15 PM, said:

There should be some way for a user to set a default and effectively disable it, even if they are on SMP and don't have the ability to modify the ROM. I recommend having a file called "default" that, if it exists, is run without prompting. If there are multiple default files, it will select in this order: "/rom/bios", "/bios/", "/disk/bios". (Server owners should have final say or be able to disable it in this way if they want, and the local system needs to be searched before disks, so people can make hack-proof systems.)
Good idea, so you can make some kind of single-boot to load just one os. It helps with computers that need to startup without user input, like servers, gps towers, etc.

View Postkazagistar, on 13 July 2012 - 08:15 PM, said:

The bootloader will then assemble a list of all the bios files in each location by path.

If there are no items in any bios folder, the bootloader will display an error, explaining that you will need a disk to unbrick the computer.

If there is only one option, the bootloader will silently start that bios. This way, a server owner can put the craftos bootloader into "/rom/bios", and everything will work exactly like it normally does, until a user adds more systems.

If there is more then one option, the bootloader will prompt the user to select an option, and then run that option.
Nothing to add here.
Ok, I'll start writing something and post what I get.

#32 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 13 July 2012 - 09:40 PM

Ok, here's what I have so far, tell me what you think about it:
Spoiler
Warning: not tested.

#33 kazagistar

  • Members
  • 365 posts

Posted 13 July 2012 - 10:01 PM

About the naming, I said "bios" because that is what the file was originally called that the files in that folder replace, but boot is a good idea. If they are defined near the top of the file, it is easy for a server owner or whatever to modify them to their needs. I just read through some of Mysticos, and you seem to have a lot of this sort of thing in there already with the multi-boot.

Now, for the other part, there is a lot I am still unclear myself about how it should be done. Here is my current idea: the entire standard interface for this consists of just one function, "require(...)", in the global scope (or maybe some other scope, like OS?) You pass to it a set of strings, each one representing to the OS what "requirement sets" you wish to demand, that look somewhat like a path. Here is an example of what might look like:
require("std:turtle", "std:term", "ocl:log(1.0)", "ocl:paths(1.0)", "ocl:io(0.7)", "myos:networking(3.2)")
The only requirement to be minimally OCL compliant that you have a function that when called, tells you if it cannot provide any of the given parameters, and throws a helpful error telling you what is missing, and maybe what you can do to fix it. Now, I am not sure if this is too confusing or not: the names in the strings are defined as separate "interfaces", that you can be compliant with, but they don't have to all be in a single table. For example, you could have an interface which guarantees specific functions in multiple tables, or that guarantees functions directly in the global namespace, etc. Now, most of the standard interfaces should be well behaved, but this allows custom interface authors some wiggle room. Also note, none of the interfaces have to actually exist before the function is called. It is perfectly viable for an OS to load the particular needed interface directly in require.

(BTW, I tried to keep it simple, at just one function, but tell me if this seems over the top.)

The rest of the OCL standard would be a set of these interfaces. There would be an "std:(whatever)" for a full, functioning implementation of each standard API that comes with craftos (except maybe OS?), and just calling "std" would ensure all of them. I am not sure where the global functions would go, or if we even want to deal with that (what if you cannot access "print()" or "getfenv()" or even "ipairs")? The alternative is to just roll that into a OCL library. There would also be the OCL standard libraries. Here is a few ideas for those (names are placeholders):
OCL:log = OCL.log(text)
OCL:io = OCL.nWrite(...) OCL.read(...) etc
OCL:app = OCL.run(app, arguments) OCL.deployApp(?) OCL.removeApp(?) etc
OCL:base = OCL.shutdown() OCL.restart() OCL.version() OCL.info() etc

The number one thing I am worried about is how fine grained these should be, for permissions use cases.

#34 kazagistar

  • Members
  • 365 posts

Posted 13 July 2012 - 10:31 PM

I can't test it yet, cause I am on my netbook at the moment, which is unable run minecraft, but at a glance it seems ok. The test would be to copy bios.lua to /rom/boot and see if it works.

Hmm, why did you chose to search for os "by extention"? In what ways is this better then just declaring the "boot" folder to be reserved for OS bootloaders and leave it at that? If users or OS authors want store other files, they can put them elsewhere, I don’t see a case in which this might be a problem. Also there is issue #5 in the bugtracker already which relates to fitting more options on the screen, and being able to use number keys instead of arrows again might be nice, but these are minor details that can wait indefinitely.

Anyways, I am going to head home so I can start slinging some code... hopefully my coding style is not too weird. If only there was some way I could run computercraft without minecraft (/me stares meaningfully at mysticT).

#35 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 13 July 2012 - 10:46 PM

I like the require function idea. Each os can implement it the way they want, checking if it's already loaded or loading it from a predefined folder or something.
For global functions, it's better to have some defined in the ocl api, and maybe add a function that loads them to the global scope, so you can use them like always (you could assign them yourself, but having a function that does it would be nice).

View Postkazagistar, on 13 July 2012 - 10:31 PM, said:

Hmm, why did you chose to search for os "by extention"? In what ways is this better then just declaring the "boot" folder to be reserved for OS bootloaders and leave it at that? If users or OS authors want store other files, they can put them elsewhere, I don’t see a case in which this might be a problem. Also there is issue #5 in the bugtracker already which relates to fitting more options on the screen, and being able to use number keys instead of arrows again might be nice, but these are minor details that can wait indefinitely.
Well, I copied most of the code from the boot loader I already had, and I used extensions because I needed to add more files that shouldn't be seen as a boot file.
It's not really needed, since you can simply put those files in a folder, so it can be removed. (I don't like extensions, but it was the easier solution I found at that time)
I also thought about the menu being limited, but it shouldn't be a problem for now (there isn't that much oses yet). Anyway, it wouldn't be too hard to implement it, but let's focus on more important things now.

View Postkazagistar, on 13 July 2012 - 10:31 PM, said:

Anyways, I am going to head home so I can start slinging some code... hopefully my coding style is not too weird. If only there was some way I could run computercraft without minecraft (/me stares meaningfully at mysticT).
:P/>
CCEmu won't be ready in some time, I stoped development for now, to focus on the os. The funny thing is that I started working on it because it was really annoying to have to open minecraft each time I wanted to test something, and I thought it would help develop the os :)/>
There's not too much left to do, I need to make the fs api and I can start using it. Then adding redstone and peripherals support shouldn't be to hard, and I already have some ideas on how to do it.
Anyway, this is kinda off-topic, so I better stop.

#36 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 14 July 2012 - 12:55 AM

Ok, new version:
Spoiler
Added the scrollable menu, so you can have as many boot files as you want. Also removed the need for file extensions.
I tested it and it works, but there might be bugs anyway.
If there's only one boot file, it loads it and you don't even notice the loader was there :P/>

#37 kazagistar

  • Members
  • 365 posts

Posted 14 July 2012 - 04:04 AM

I talked to the owner of the server I play on a bit, and he had a different philosophy for this, along more minimalist lines
-- Bios file name
local biosName = ".bios"
-- a function to try booting a bios, returns true if successful
local function tryBoot(path)
    local file = fs.open(path..biosName, "r")
    if file then
	    local func, err = loadstring(file.readAll())
	    file.close()
	    if func then
		    return pcall(func)
	    else
		    return false
	    end
    end
    return false
end
-- search the root, the drives, and the rom for a valid working bios file
if tryBoot("") then return end
for _,side in ipairs({"top","bottom","left","right","front","back"}) do
    if peripheral.getType(side) == "drive" then
	    local path = peripheral.call(side, "getMountPath")
	    if path then
		    if tryBoot(path.."/") then return end
	    end
    end
end
if tryBoot("rom/") then return end
-- print error and wait for keystroke
term.write("No bios file found, please use a valid boot disk")
while coroutine.yield() ~= "key" do end
(Note, I made this, and it is probably still buggy as well).

The idea is, if you want to create a multi-boot selector, why should we force one on you? Why not let that be selected by the user as well? Thus, this program would load a multi-boot selector, which would in turn load a bios, but you could select a different one if you like. Higher customizability.

Honestly, I prefer your approach more, but just having this as an option is nice, for people to decide what they want more.

BTW, I don't see the "default" thing built in to your system. Any plans on that front?

#38 toxicwolf

  • Members
  • 201 posts
  • LocationUnited Kingdom

Posted 14 July 2012 - 10:21 AM

I'm loving the ideas you guys are throwing around, especially the interface and require function idea. I'm wondering what use I'll actually be to this project now, since my knowledge on how this all should work is very limited...I do however seriously appreciate the effort you are putting in!

#39 MysticT

    Lua Wizard

  • Members
  • 1,597 posts

Posted 14 July 2012 - 04:52 PM

View Postkazagistar, on 14 July 2012 - 04:04 AM, said:

The idea is, if you want to create a multi-boot selector, why should we force one on you? Why not let that be selected by the user as well? Thus, this program would load a multi-boot selector, which would in turn load a bios, but you could select a different one if you like. Higher customizability.
Well, that was my original idea. If you look at my os bios, it looks for a boot file or directory in the computer's root folder, and loads it, if there isn't one it loads CraftOS. Then, there's two boot files, one for multi-boot and one for single boot of my os.
We could make it that way, and provide a multi-boot loader for those who want it, so you can use it if you don't want to make one yourself.

View Postkazagistar, on 14 July 2012 - 04:04 AM, said:

BTW, I don't see the "default" thing built in to your system. Any plans on that front?
I wasn't sure on how to do it. Should it look for an specific file name in the boot list? Or first check if the file exists and load it without checking for other boot files?

EDIT:
Ok, here's a version without the multi-boot:
Spoiler
I'm not sure on how to display if there's an error on the boot files. Currently, if all the boot files have an error, it will just say "No boot files could be loaded", but won't say what the error was.

Edited by MysticT, 14 July 2012 - 10:38 PM.


#40 Lyqyd

    Lua Liquidator

  • Moderators
  • 8,462 posts

Posted 14 July 2012 - 10:06 PM

@toxicwolf - Fix submitted for #7 on github.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users