neonascent

IndieGames just posted a recently released iPhone game which features “holographic 3D”, where the perspective of the rendered 3d view changes as the phone moves around. With the inclusion of a red/green anaglyph view mode, this is a really engaging 3D display system.

These 2D displays - which recognise viewing angle and render accordingly - have some important advantages over more exotic 3D visualisation systems; they are cheap, require very little software/hardware investment, and are less likely to lead to motion sickness. Just how little hardware investment is needed is demonstrated in this project video by Johnny Lee (via Hack-A-Day), showing his “simple, but incredibly effective VR head tracker”:

 
 
And this seems like a good time to mention a 3D photography project I worked on from a while back. I’ve already posted the results, but have a good write-up of the setup that I’ve not published. The background to the project is that after dismantling my 30 x 6MP camera array, I naturally had a lot of cameras on my hands, all with hacked cable-release connections. I managed to reappropriate some of them into 3D cameras:

3D photography works by capturing two simultaneous views of a scene, and then recombining them for viewing. There are many different ways to display 3D photographs; the most popular among them include creating Cyan-Red Anaglyphs for viewing with coloured 3d glasses, Direct Stereo Pairs (where the two images are placed next to each other and either viewed "cross-eyed", or with a viewer), and "wiggle" stereoscopy, where the two views are animated together. This setup allows quick and easy creation of 3d images with your chosen method, and even allows recording of synced 3d video!

This 3D photo setup consists of two 6.1 megapixel cameras, a dual-camera bracket (can be used on a tripod or by hand), and a shutter release sync cable. The bracket provides a stead mount for the cameras, at an optimum distance, and allows tripod mounting. The setup also comes with two 512MB memory cards.

Direct Stereo Pairs (Cross-eye)

Red-Cyan anaglyph

"Wiggle" Stereoscopy

I’ve found that the best viewing results are produced with Direct Stereo Pairs using a viewer. Cardboard foldup 3d viewers can be purchased online very cheaply.

There is a large collection of free software for creating anaglyphs and other 3d images.

I’ve always loved how effective "Wiggle" stereoscopy is. I will try to find the time to experiment with wiggle-interpolated video, although one attempt already on YouTube doesn’t seem particularly effective.

I’ve just converted a dual camera video into a wiggle stereograph video. This is just a test with a fixed distance between the images. There are points where the subject object is closer, and the 3D effect could be improved by hand-aligning the twin frames. Overall it seems to work quite well though. Certainly possibility for an effective 3D visualization.

What do you think? Can you get a sense of the depth in the video, or does it give you a headache?

Recently I have been researching ways of tracking and visualising use of space by pedestrians. The two technologies I have been looking at at Volumetric reconstruction and Model-based figure reconstruction. The two methods are very different in their approach. Volumetric reconstruction in one respect is completely dumb; it does not try to understand what it is looking at, and instead uses the maths of projection to reconstruct space purely on what it can see. In contrast, Model-based figure tracking attempts to model the behaviour and properties of a person, in order to understand how they are moving through space. Instead of just giving raw use of space, it gives you an idea of what the bodies are actually doing, and how.

Volumetric reconstruction uses position-calibrated cameras to take the silhouettes from each camera angle. It then uses these silhouettes to reconstruct the space being used. The process is the reverse of the Left, Top, and Front views windows in a 3D modelling program; the 3D program creates these views from what it knows about the 3D model, whereas here we effectively recreate the model based on the views. Like the 3 view windows, this process needs 3 specific perspectives to recreate a scene properly. It’s main weakness is in handling occlusion of views, as it has no way of knowing about any gaps behind what it can see.

The above and below images are from a 3D simulation of a test-system using only two cameras. The cameras raw view is processed to find the difference between the constant background and the changing pixels of the pedestrian. The images on the right are the results of processing.

The reconstruction works quite well provided there is no occlusion - i.e. people walking in front of each other. In fact, as we are dealing just with Pedestrians, we could could probably get away with just one camera; the top-down view. This is because we can assume a reasonable height for the pedestrians. The top-down view is also extremely helpful because it does not feature much occlusion. Think of a birds-eye view of people walking around, and you’ll agree it’s not very often that people jump up or fly in front of each other!

As you can see, the results are not perfect, but they are a very easily achieved, and are visually interesting. This process can very easily be run in real-time, resulting in animated blobs moving about the open space.

I have been looking today at Volumetric reconstruction, which I am more familiar with having worked with it in implementing my 3D scanner. Hopefully I will get a chance to look at Model-based tracking in more depth (Reading university - where I studied - has a strong computer vision group doing this sort of thing). If you are super-keen now, you can check out a paper in the field: “ROBUST PEDESTRIAN TRACKING USING A MODEL-BASED APPROACH”.

Add-Art is a Firefox extension which replaces advertising images on web pages with art images from a curated database.

Whats Add-Art all about?
http://add-art.org/content/background

Howto Install the Add-Art Firefox Plugin
http://add-art.org/content/installation-instructions

I’ve recently installed this, and still stop occasionally while browsing to wonder why everything seems so much nicer to look at. Apart from the psychological benefits of an absence of material solicitation, pages are a lot easier to read without the flashing words.

This is a quick experiment with what’s called “wiggle” stereoscopy to enhance the perceived 3D feel of a rendered video. It doesn’t seem to make much of a difference, possibly because the camera flyover motion gives a strong sense of 3D without needing anything else. It may work very well with live action, or something with less profound movement in.

Also, interesting to play around with a wider 3D depth, and the optimal frequency of the “wiggle”.

Here is another test video. After some experimentation, 12FPS look to be the optimal framerate.

The video above shows the work of artist Pablo Valbuena; working with projective augmentation of space:

This project is focused on the temporary quality of space, investigating space-time not only as a three dimensional environment, but as space in transformation.

For this purpose two layers are produced that explore different aspects of the space-time reality. On the one hand the physical layer, which controls the real space and shapes the volumetric base that serves as support for the next level. The second level is a virtual projected layer that allows controlling the transformation and sequentiality of space-time.

The blending of both levels gives the impression of physical geometry suitable of being transformed. The orverlapping (sic) produces a three-dimensional space augmented by a transformable layer suitable to be controlled, resulting in the capacity through the installation of altering multiple dimensions of space-time.

The overall effect is to give a fantastic sense of the dynamism of volume. Mapping the physical space in this way invites one - or me at least - to explore the possibilities of that space.

I’ve been inspired to experiment (using my Augmented Reality resources) with abstracting real buildings into their volumetric footprints; removing the fascias, the textural materiality, and leaving just the space. God knows there’s a movement towards “digital buildings”, with projected faces - it should be interesting to turn the tables and privilege the walls themselves, and then perhaps begin to bring the possibility of projections and materiality back in.

On a technical note, the object-size to marker-size ratio necessary for these volume projections makes the Augmented Reality marker alignment very sensitive to error. Effectively the far edges of the object act like the needle of a gauge amplifying motion, and it’s very easy to notice when they are out of “whack”. I’ll have to tweak the matching code, or capture using a bigger marker, before I can upload some video. Alternatively, I could use edge-detection from the initial stage of markerless AR to detect the wire-frame structures outright.

I’ve recently been trying to explain to people what I’d like to do with architecture and augmented reality, and getting blank looks - i.e. failing. Most succinctly, I want to mess with existing architecture by virtually intervening with it, using a technology called Augmented Reality. Augmented Reality is

and is fantastic, because when the software is running on a mobile phone, it allows the viewer to walk around the installation and get a more spacial feel for it and its context.

Explaining ideas is always a really good way to explore/discover/flesh-out the idea in your own mind. Hence I’m thinking of either making a comic-style illustrated description, or a 3D animated one. Either medium will give me plenty of opportunity to think about the spaciality of different interventions.

I’ve created a few Adobe Photoshop CS2 Actions, to experiment with how different grey-scale filters will look on 3D rendered video content. Here are the video tests, and you can download a .ZIP of the four actions applied (the Actions should work with CS2 or above). I’d be happy to provide a video tutorial of how to convert video to image-sequence, apply actions, and put the image back into video, if anyone needs it?

• Download filter Actions

I have finally got animated VRML working, but somewhere along the line the JPEG library broke, so textures will have to wait. so I’m using a lower-quality GIF texture. Just a few more tweaks of the processing code, and I’ll feel happy to switch to the modelling and “sculptural installation” work.

The static model is a scan of myself I did a few years ago using a home-built 3D scanner. I’ve skinned it onto a skeleton and applied one of the motion-capture gestures from Max Payne 2.

Apologies for very repetitive subject of posts lately - I’ve been focussed quite strongly on this project. What you think do far? Do you have any thoughts of how to use the technology to engage with architecture? I’d love to read your comments!

This weekend I have spent many hours getting up to speed with C programming, and trying to compile library dependencies for ARToolkit. Until now, the ARToolkit was only useful for real-time processing of video using a webcam. The result the weekend’s work is that I can now load image sequences (that have been exported from video), process them, and spit them out the other end with the augmented reality overlay. I’ll be able to go out into the field and start creating architectural interventions.

So far I have only applied this to the simple box code, but I will look at the VRML tomorrow.

From We Make Money Not Art blog:

littleBits intro from ayah bdeir on Vimeo.

littleBits is a growing library of preassembled circuit boards, made easy by tiny magnets. All logic and circuitry is pre-engineered, so you can play with electronics without knowing electronics. Tiny magnets act as connectors and enforce polarity, so you can’t put things in the wrong way. And all the schematics will be shared under an opensource license so you can download, upload, suggest new bits and hopefully see them come to life.

Very cool and fun.