Jump to content




'Augmented' Reality with OpenPeripherals


19 replies to this topic

#1 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 22 May 2016 - 01:37 PM

Because I can't focus on anything for more than 2-3 days, I present to you: OpenPeripherals augmented reality:

Posted Image

Slightly longer video

If you can't tell what is happening: that red point and box are being translated so they are always over the sensor. This uses OpenPeripheral's terminal glasses to draw the screen and its sensor to get where I am looking and my position. I'm then converting that information into a matrix and projecting the point to it renders in the correct location. This could easily be extended to draw lines between points or entire objects in world.

However, there are several many caveats:
  • High latency: getting player position takes a tick, then sending to the client takes another tick so there is a large delay between user movement and what is show.
  • No Z buffer/Z-sorting: so objects closer to you may not be drawn in front of objects further away.
  • Things aren't perfectly positioned when further away.
If people are interested in the code, it is on GitHub.

Edited by SquidDev, 24 May 2016 - 01:37 PM.


#2 Bomb Bloke

    Hobbyist Coder

  • Moderators
  • 7,099 posts
  • LocationTasmania (AU)

Posted 22 May 2016 - 02:46 PM

I'd say most all of your "caveats" are really just "unimplemented features that need moar code", with the exception of the lag... which isn't that bad. Has a sort of charm to it, really.

Puts me in mind of the cyber interface from System Shock, in particular how it acts before you turn the help system off.

#3 InDieTasten

  • Members
  • 357 posts
  • LocationGermany

Posted 22 May 2016 - 04:27 PM

This is also depending on FOV, isn't it? Since the world rendering is depending on that, and the grid system of terminal glasses aren't, right?

EDIT: May as well be one of the bigger sources why it's inaccurate for longer distances

Edited by InDieTasten, 22 May 2016 - 04:30 PM.


#4 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 22 May 2016 - 04:31 PM

View PostBomb Bloke, on 22 May 2016 - 02:46 PM, said:

I'd say most all of your "caveats" are really just "unimplemented features that need moar code", with the exception of the lag... which isn't that bad. Has a sort of charm to it, really.
True. "Current issues" might be a better title.

View PostInDieTasten, on 22 May 2016 - 04:27 PM, said:

This is also depending on FOV, isn't it? Since the world rendering is depending on that, and the grid system of terminal glasses aren't, right?
This depends on FOV, render distance and screen size (though I doubt render distance has an effect). However there are config options for all of these so it should be easy to tweak.

Edited by SquidDev, 22 May 2016 - 04:31 PM.


#5 LDDestroier

  • Members
  • 1,095 posts
  • LocationACDC Town

Posted 22 May 2016 - 07:43 PM

Neato. Make it so it renders a computercraft terminal onto a giant wooden board!

#6 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 22 May 2016 - 08:07 PM

I've integrated my clipping algorithm into this. Lines now work, and things don't render behind you!
Posted Image
It doesn't work for filled triangles yet (something is broken somewhere).

Edited by SquidDev, 22 May 2016 - 08:07 PM.


#7 Creator

    Mad Dash Victor

  • Members
  • 2,168 posts
  • LocationYou will never find me, muhahahahahaha

Posted 22 May 2016 - 08:24 PM

View PostLDDestroier, on 22 May 2016 - 07:43 PM, said:

Neato. Make it so it renders a computercraft terminal onto a giant wooden board!

That'd be awesome. Seriously, do it.

#8 Vex

  • Members
  • 6 posts

Posted 23 May 2016 - 06:19 AM

This is definitely a very interesting concept and shows potential, keep it up!

#9 Waitdev_

  • Members
  • 432 posts
  • LocationWherever you want me to be

Posted 23 May 2016 - 11:29 AM

This is so cool! It's a shame though about the delay :/

View PostLDDestroier, on 22 May 2016 - 07:43 PM, said:

Neato. Make it so it renders a computercraft terminal onto a giant wooden board!
And yeah this would be awesome!

#10 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 24 May 2016 - 01:47 PM

I hacked together a horrible clipping algorithm for triangles. Which means all shapes now work!

Posted Image

There are some issues with triangles vanishing for no reason though not sure why. Fixed! I'm sorting triangles by their distance from the player though again this isn't perfect. Do not the nice animated triangle by the computer though. Well, I think it looks nice anyway :).

View PostLDDestroier, on 22 May 2016 - 07:43 PM, said:

Neato. Make it so it renders a computercraft terminal onto a giant wooden board!
It would be cool, but I'm not sure it is worth the effort. Also you'd have to split each character into pixels and then draw them: there are just under 1000 characters on a normal screen. given the number of pixels per character you're looking at 10k polygons: I'm not sure this engine can handle it. There are probably ways to optimise it so there are less polygons but...

Edited by SquidDev, 25 May 2016 - 01:54 PM.


#11 Selim

  • Members
  • 189 posts
  • LocationHiding in Online Storage

Posted 24 May 2016 - 02:29 PM

Will it work with normal text? If I wanted to have a string hover a location, could I do it with just the addText function rather than drawing each pixel?

#12 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 25 May 2016 - 01:53 PM

I fixed the clipping algorithm (again) so everything is even less buggy. I wrote a silly little "game" where you pick up the floating cube.

Posted Image

It looks better when your playing it. Sort of.

View PostSelim, on 24 May 2016 - 02:29 PM, said:

Will it work with normal text? If I wanted to have a string hover a location, could I do it with just the addText function rather than drawing each pixel?
Yeah. It would be kinda cool to use this to add overlays with labels to tiles and entities to give additional information (such as entity health).

Edited by SquidDev, 25 May 2016 - 01:54 PM.


#13 Dragon53535

  • Members
  • 972 posts
  • LocationIn the Matrix

Posted 25 May 2016 - 02:46 PM

For your 'game' perhaps try adding an arrow on screen when the cube isn't on screen showing you where it is.

#14 HDeffo

  • Members
  • 214 posts

Posted 25 May 2016 - 05:07 PM

I had a similar program to this on another language/game and I remember being able to ignore screen size and convert the 2D space to 3D space regardless of graphic settings in a small quick math problem....Honestly no clue if it will port over 100% to minecraft but assuming as long as players keep the same FOV it should work I assume. Give me a minute to download the things I need just to recover this old code


Here you go like I said I don't know how well this will port over but it only needs the FOV modifier instead of screen size (it just continues to follow off screen)

float FOV = 1.7320508075688774;
vector OffScreen = <-1000.0, -1000.0, -1000.0>;
vector Region2HUD(vector objectPos, vector cameraPos, rotation cameraRot)
{
    objectPos = (objectPos - cameraPos)* (ZERO_ROTATION / cameraRot);
    objectPos = <-objectPos.y, objectPos.z, objectPos.x>;
    float xHUD = (objectPos.x * FOV) / objectPos.z;
    if (xHUD > -3.0 && xHUD < 3.0)
    {
	    float yHUD = (objectPos.y * FOV) / objectPos.z;
	    if (yHUD > -1.0 && yHUD < 1.0)
	    {
		    float zHUD = (objectPos.z - 2) / objectPos.z;
		    if (zHUD > -1.0 && zHUD < 1.0)
			    return <0.0, -xHUD / 2.0, yHUD / 2.0>;
	    }
    }
    return OffScreen;
}

Edited by HDeffo, 25 May 2016 - 05:27 PM.


#15 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 01 October 2016 - 10:17 AM

Sorry for the necro. I saw this on my project list and decided to do some polishing up. Instead I added another program: I present to you furnace cam!

Posted Image

It doesn't do anything especially fancy: just renders a progress bar of each furnace. I thought it was quite cool having dynamic information about in game items such as objects. Techinically this could be expanded over a whole base. Another thing I'd like to see is applying this to entities: you could provide a health display of nearby mobs.

The source can be found on GitHub.

Edit
I re-implemented Neat using this.
Posted Image
It doesn't change the colour of the bar when it gets low, nor does it render potion effects. But hey, it looks quite good.

Edited by SquidDev, 01 October 2016 - 12:00 PM.


#16 meeko011

  • Members
  • 31 posts

Posted 18 March 2017 - 12:03 PM

This is too good to not necro. As far as the lag is concerned, couldn't you do something similar to what multiplayer games do to make up for the delay? Also, is there a forcefield peripheral that can generate/use another mod to generate invisible forcefields? If so, then you could easily combine the two for some awesome functionality. Another thing I noticed is that your examples, and the functionality possibilities, are very similar to the pneumatic helmet from PneumaticCraft. This is fantastic, as the pneumatic helmet has no ComputerCraft support, so a CC-supported replacement is something I've always wanted. So, for instance, you could create ghost blueprints just using your eyes and then use drones to clear them out.

In combination with Plethora's neural interface and kinetic module, you could turn a second player (second account?) into your personal Terminator with a full readout.

#17 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 18 March 2017 - 01:24 PM

View Postmeeko011, on 18 March 2017 - 12:03 PM, said:

This is too good to not necro.
Thanks!

View Postmeeko011, on 18 March 2017 - 12:03 PM, said:

As far as the lag is concerned, couldn't you do something similar to what multiplayer games do to make up for the delay?
Sadly I doubt this would help. In multiplayer games, the rendering is done by the client, with the server sending updates about position and what not. Here, everything is done by the server: user input and rendering, then sent off to the client.You might be able to guess where the user will look next and use that instead, but then you'd get a weird swaying motion if they stop moving suddenly. I guess I could try it...

View Postmeeko011, on 18 March 2017 - 12:03 PM, said:

Also, is there a forcefield peripheral that can generate/use another mod to generate invisible forcefields? If so, then you could easily combine the two for some awesome functionality.
Not that I'm aware of. It would probably be possible to add compat with MFFS to Plethora, I'll look into it.

View Postmeeko011, on 18 March 2017 - 12:03 PM, said:

So, for instance, you could create ghost blueprints just using your eyes and then use drones to clear them out.
There's an OpenComputers mod which adds this functionality. I'm planning on adding something similar to Plethora.

Edited by SquidDev, 18 March 2017 - 01:24 PM.


#18 Dave-ee Jones

  • Members
  • 456 posts
  • LocationVan Diemen's Land

Posted 27 March 2017 - 04:27 AM

This is quite neat, augmented virtual reality :P

What you could do is make the server a command computer so then you can add special effects to the augmentation, say, if you touched an augmented object that would usually be harmful you can damage yourself with a command. E.g. spawn a harming potion on yourself or something like that.

EVEN BETTER!

Augmented mobs - Skeleton! An augmented skeleton object which can shoot real arrows at you (using the spawn entity - arrow command). Hardest part would be working out the direction and speed of the arrow to hit you...A fireball might be easier. :P

But the ideas are there. You could do all kinds of things with commands stitched into this...*Looking through the commands list in awe of what you could do* :OO

#19 AyScorch

  • Members
  • 10 posts

Posted 17 June 2018 - 01:10 PM

/USE PLETHORAS GLASSES/

#20 SquidDev

    Frickin' laser beams | Resident Necromancer

  • Members
  • 1,427 posts
  • LocationDoes anyone put something serious here?

Posted 17 June 2018 - 06:24 PM

View PostAyScorch, on 17 June 2018 - 01:10 PM, said:

/USE PLETHORAS GLASSES/
Plethora didn't even exist when I started this (first commit was June 14th 2016), let alone including the overlay glasses! It should be pretty trivial to port this to Plethora, though native 3D support is on the roadmap for Plethora (though has been for several years :/).





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users