PCMAG.COM is reporting that YouTube has just released their automatic captioning system for public use:

The technology still isn’t perfect, but Google is also using YouTube, with its vast array of accents, languages, background noise, and other distractions to improve its technology.

But the closed-captioning also means that millions of users who are hard of hearing, deaf, or simply speak another language as their primary language can now have a greater access to YouTube videos, the company said. “A core part of YouTube’s DNA is access to content,” said Hunter Walk, the product team lead for YouTube. “From day one, that’s what we were hoping to do with video.”

It’s worth noting that as well as opening up information access for “hard of hearing, deaf, or [non-English speakers]“, it’s also another example of Google making the world more and more machine-readable. Google Books is another massive-scale example of this.

Edit: After requesting automatic transcription of one of my YouTube videos yesterday (which features pretty clear spoken English), I’ve received a “Transcription Failed” message today. No option to retry or suggestions why. I.e. this tech might not be so fantastically world-changing yet.

Renew Newcastle

This particular CitySwitch was conducted in collaboration with Renew Newcastle, which works to temporarily populate empty shopspace with galleries, teahouses, etc. to aid revitalization of the area. They work with a fascinating legal arrangement, non-threatening for property owners, which sets up a 30-day rolling “license to use”. The temporary occupier pays $20/week to cover insurance and minor upkeep. It is a framework that could possibly translate to spaces in Sydney.

   

The Teams

Team A: “Coal Tea Room” (run by Kiyoshi Kametani and Toshinori Esumi)

Created a Japanese-style Tea room in an old McDonald’s building, and incorporated a tea ceremony into the presentation.

Team B: Marking the City (run by Jun Inokuma and Kei Kaiho)

Attempting to find and emphasis a property of the city. This was the time I worked with, and we looked at the strong wind from the ocean. We went through a number of prototypes, looking at creating sound, but eventually chose the spinners. I had my doubts at times, but the final work was elegant and worked very well to transform the space.

Team C: Urban Performance (run by Takao Nishizawa, Satoru Yamashiro and Jin Hidaka)

This team was split in two to create a project proposal video. One half worked on the ideas, and the other on the visualization. The final video looked very fresh and used a visual technique of speeding up hand-drawing of overlays. The video should be up on the the Team site shortly.

Team D: Sticky Streets (run by UTS students)

The UTS team proposed using tape art and a freeze mob to alter the street (using Melbourne artist Buff Diss and SydneyFreezeMob as their references sources). Personally, I was underwhelmed by their proposal, which pointed towards copying rather than seeking a point-of-departure. As this was the most publicly visible project, it would have been nice to see something more inspiring.

Mapping

I had originally intended to create a real-time mapping system that would be visually appealing and dynamic enough to be projected on the closing day. The idea was that as visitors visited, photo-documented, and tweeted about the interventions, the map would update with their contributions. This plan was changed when it seemed unlikely that enough people would be using the keyword taxonomy, or even any of these services. The design changed to accommodate a time-line, with the use geared more towards archival and replay of the entire event. The final map will be able to grow as documentary photos are posted, and social networking chatter continues in the afterglow. I initially seeded the map via the CitySwitch Newcastle Flickr photostream, geocoding and tagging images as they were uploaded.

Proto-type map using Google Maps, with test video captured on the second day; the first video worked out very well, but it proved impossible to interview other venue owners in the short timeframe.

The current version of the map is served from a service called Dipity, which collects sources together in a number of time-space depended visualizations; including a timeline, a flipbook, and a map. This seemed perfect, however some experimentation the speed of update and limitations of Flickr source sizes means it requires more work. The Dipity Help forum contains unanswered questions older than a year: disappointingly abandoned. I will experiment with creating RSS feeds in Yahoo Pipes, that allow more than the 200 image limit.

PhotoSynth

I had a chance to do some documentation and 3D reconstruction of the interventions, and the results are initially a few PhotoSynths. I will add more as I process them, and may work on the pointclouds themselves when I get a chance. Note: to view the PhotoSynths, you will need the Silverlight browser plugin (available from the Photosynth pages).

Queens Wharf Tower

Loopspace

Recently I’ve been looking at the emerging humanitarian uses of collaborative micro-blogging, and in particular Project EPIC’s initiative to define a “folksonomy” of tweet formats to aid Haiti crisis response. I’m currently building a set of Yahoo Pipes feeds that process and locate relevant tweets onto an interactive map (I’ve been hampered by outages in Yahoo Pipes services). Yahoo Pipes is a sort of rudimentary visual programming, that will allow easy reconfiguration by non-programmers once these pipes have been set up. These tools are intended to be easily extendible and “mashable” into new uses.

I’ll post about the Yahoo Pipes work when it’s completed, but needless to say I am aware and involved in the positive potential of augmented reality and surrounding technologies.

I’ve been motivated to write about the negative potential after reading Augmented Planet’s “The Case Against Augmented Reality”. Although it gained some attention as a dissenting voice against the generally positive coverage AR gets, it was a little underwhelming, and even the comments failed (for me) to really go very deep into things. I’m effectively re-posting an annotated version of my comments on that page:
 

It’s not surprising that with any new tool there will be ways to use if maliciously. However, occasionally a technology dramatically changes the game. For example, by reducing the effort needed to get meaningful results, Data Mining has transformed our relationship to casual data tracking. While one commenter thinks the problems arise exclusively from the exploitation of social networking data, I believe the re-contextualization of available data leads to new dangers.

On the one hand Augmented Reality can similarly redefine our visualisation of data to give a wholly new experience of our environment. On the other hand, any active participation in an AR landscape is tied to geo-tagged micro-blogging (what I call Geogging) and the potential for vast, emergent pattern tracking.

Computer games offer a huge insight into possible future uses of Augmented Reality. I originally became interested in the potential of AR when playing the Parkour game Mirror’s Edge; from the first-person perspective of running around city rooftops (often pursued by police), paths of escape are indicated by colour codes. I see this provision of leveraged information as key, providing a situational awareness advantage in any sort of conflict.

Shady characters of the criminal underworld (or whichever “badguy” you’re worried about), can tag and share spatial/temporal information with each other in the same way Crackers share security vulnerability exploits and Phishers share creditcard numbers.

Consider the GPS in Grand Theft Auto; it would be fairly easy to crowd-source police car locations with an iPhone app, augment your GPS display, and orchestrate a getaway with identical “radar” information to the game – It’s something thousands of people have practised while stoned at home on their couch. *

Hiro from the book Snowcrash used his Augmented Reality to get a free car, with a pinch of social networking.

Finally, as mentioned above, the use of AR applications will fuel the shift into geo-tagged social networking/micro-blogging. Local police already monitor Facebook and Twitter to pre-empt flashmobs and illegal parties. The unintended composite power of a constellation of personal/spatial/temporal data will be enormous, and there is no barrier to access; anyone can use it.

* or alternatively use geo-coded Twitter tweets with Layar running on a phone along side the GPS.

Edit (19/2/2010): Of course good old Twitter searches can be scary too! The BBC has just reported on the creation of the site PleaseRobMe.com, showing the locations of empty houses:

"not long ago it was questionable to share your full name on the internet. We’ve gone past that point by 1000 miles."

The Dutch developers told BBC News the site was designed to prove a point about the dangers of sharing precise location information on the internet.

Recently, Bruce Sterling featured Voxopolis (below) on his blog. The project extends Conway’s Game of Life into the 3rd dimension in order to evolve a city design. It is one of six presented in the Helvepolis – Urban Design in Vitro! exhibition, showcasing the work of students at ETH in Zurick, studying Masters of Architectural Studies in Computer Aided Architectural Design:

voxopolis from Dino Rossi on Vimeo.

The ETH Masters program looks at “the use of current information technologies as an augmentation of concepts of architecture. [Exploring] new techniques and methods for design that incorporate scripting and programming languages.” As my interest in the use of computer technology in architecture lie in the areas of understanding inhabitants’ spatial practices to inform design, better engineering, and architectural modeling, I am generally quite cynical towards CAAD projects that look cool, but don’t serve actual human bodies. I have given some personal commentary to a few of the CAAD projects. I’d be curious to hear your views.

Voxopolis(above)

As pointed out, the resulting city looks “kind of weird and implausible”, but beyond looks this project upsets me in it’s approach to architecture. Where is the human body in the simulation? The scales and sizes of the blocks are pre-defined, and there is no interaction between the game rules and the rules of human habitation and scale.

The evolution does not rely on any sort of heuristics (guides to improvement), apart from the abstract rules of the Game of Life, so the evolution is purely a result of these. Also, it looks likely that the street configuration that the game is run on has been set up initially, according to the experimenter’s whim.

Attractive City

AttractiveCity – an interactive City generator from stefanie sixt on Vimeo.

It’s totally abstracted from the experience of being in the city. A city can look fantastic from 10,000 ft above, but how does that help peoples street-level lives?

Himmelstad

Himmelstadt-CAAD ETH from Hua Hao on Vimeo.

Another evolutionary system. This one involves some interaction with a user, who seems to select which designs will survive on the basis of looks. It is a funny approach to an evolutionary system, when you can prune off genes to make it look more like a conventional city and less like an orgy of algae. In all of these, the heuristics/target properties should be explicitly stated. For me, coming up with something cool looking is not enough.

Sandy City

Processing City – Sandy City (HD version) from mjchen on Vimeo.

I really like this project; they are not trying to throw a magical evolutionary algorithm at the problem, and there is no attempt to pretend the choice of archetypes are god-given.

Also, it is so refreshing to see human-eye-views of the created cities, as the viewer moves realistically through the space. Even more refreshing is the attention to sound design, as an important aspect of the architectural experience.

About Play

The computer isn’t a magic box. Just because the computer does something – something emergent – doesn’t mean that the conditions of emergence will be in any way productive of anything useful.

From (collective) experience, some architects seem to treat the CAD process this way. Imagine having a magic wand that you waved at stuff to transform it into something altogether weird-looking and different. We wouldn’t wave it around willy-nilly and put the resulting blob to use as a building, although admittedly the aesthetics of the thing would be exciting and interesting, and might get you some kudos.

I truly believe that the element of unrestricted play can be extremely, creatively productive in most endeavors, architecture included. I also recognize that these projects can be thought of as playful breaks from traditional design. My concern, however, is the assumption of the equation

something + computers = better something

in any situation.

But please let me know if you disagree; I have colleagues who suggest that a computer-based generation of options, coupled with considered selection, is a good design approach.

Edit: This discussion has been posted on Bruce Sterling’s Beyond the Beyond blog as “Generating Cities”, and responded to at “The Authors of Voxopolis speak”. My continued response is in the comments below.

The Third & The Seventh from Alex Roman on Vimeo.

The small (simulated) depth-of-focus and constant shifting focus helps with convincing the viewer it’s actually real, and it guides the eye across a catalog of gorgeous surfaces. As much as I am concerned with a shift of perception into pure site, when the camera rounds an object and focus slides languidly over it the experience is almost tactile. Perhaps it’s a synesthesial short-circuit?

I’ve been wondering about the layout of my blog recently, and whether it may need a revamp to make it a little easier and enjoyable to read. Aware of some of the sophisticated tools available as part of Google Analytics (a very helpful, free web statistics analyser) I decided to test out the effect of a new theme. For something that seems obvious to want to do, it was fairly involved. I was sure it would involve multi-variant testing in some way, but the full process is not so straightforward.

Here are some step-by-step instructions for anyone else wanting to test the effects of different themes in their wordpress blog: 

1. Have an exisiting Google Analytics account, or set one up now.

 

2. download, install, and activate wordpress Theme Switcher Reloaded. This allows wordpress to select which theme is displayed based on URL parameters.

 

3. Activate the Theme Switcher widget, take note of the URLs for each theme option, and then deactivate. You probably don’t want the selections visible to visitors, but you need to get the URL to set up multi-variant testing. The URLs will be in the format: “http://<your domain>/?wptheme=<theme name>”

 

4. set up multi-variant testing with two or more theme URLs from above. This will require pasting code into the header and footer files of your themes. All you really need is the content control code Google gives you to place in your starting page, but you can also set up a conversion page to test which theme gets the most conversions. I’m more interested in themes effect on metrics, so it is not necessary to even set up conversion code or tracking; I’m effectively just using the code to deliver a different theme to 50% of the visitors.)

 

5. Add custom variables to Google Analytics tracking code, with the name of the theme. Modify your existing Google tracking code (best located in each theme’s footer template file) to include something like:
   

   pageTracker._setCustomVar(
   1, // This custom var is set to slot #1
   “Theme”, // names this custom var
   “Wordpress Default”, // Sets the value of “Theme” to current theme
   3 // Sets the scope to page-level
   );

   between the “pageTracker._initData();” line and the “pageTracker._trackPageview();” line. Modify the theme name accordingly.

 

6. set up Advanced Segments in Google Analytics based on your custom variable values eg. “Custom Variable (Value 1) Matches exactly Green” to test the Green theme.

That’s it! Check your Analytics results in a day or so and start comparing.

The video above is taken from Rubin’s Flash Panorama Laboratory. It is the result of experiments using non-professional commercial cameras to create “the world’s most affordable video solution capable of shooting and virtually recreating all possible viewing directions of the environment”. As you can see, the custom created video player allows you to move around the field of view, and zoom in and out. You can play with it on their site.

It would be interesting to explore the sense of immersion that could be produced by tying this into a head mounted (or even just more directly mouse controlled) interface, with the possibility of recreating recorded first-person experiences a la Johnny Mnemonic.

With the announcement that Twitter will be building in support for latitude/longitude tagging, we are guaranteed a steady flood of points to build our Psycho-Social City.

The following is an example of what is being done with Twitter at the moment, without fine-grain location information:

Landscapes and Tweetscapes

My interests lie predominantly in the idea of space as a “practiced place” – our experience and understanding of a location laying beyond the geometric, and having a significant social component. This social component is read through others’ everyday, idiosyncratic, casually creative performance within a space; their spatial practices. The best example of this I can think of involves imagining you are alone in an urban park at twilight. Around a corner echoes the sounds of either breaking glass and shouting, or children laughing and playing. Consider how the audible trace of these spatial practices alters your experience of the park.

Micro-blogging combined with augmented reality hits social space from three fronts: First, it captures a trace of spatial practices that may otherwise go undocumented (You are here, doing this, seeing this…). Second, it folds back into our navigation and experience of places, and transforms the social communication of spaces (You are near, you ate here, you liked it…). Finally, it can be used (by, for example, the police) to repress unwanted spatial practices (They are there, doing this, right now.)

The potential power of this technology to build a rich, near-realtime map of a city has not gone unnoticed. While the the USGS is uses Twitter to determine the magnitude of earthquakes, advertising agencies are developing data mashups to track trends at postcode level. Twitter is certainly anticipating these uses:

“[...]Compelling context will be added to each burst of information. […] You could switch from reading the tweets of accounts you follow to reading tweets from anyone in your neighborhood or city—whether you follow them or not. It’s easy to imagine how this might be interesting at an event like a concert or even something more dramatic like an earthquake. “ (Twitter Blog)

Project Epic and Information Tools

Project Epic is a “multi-disciplinary, multi-university, multi-lingual research effort to support the information needs by members of the public during times of mass emergency“, recently launched and receiving a $2.8M grant from the US National Science Foundation. They have been monitoring the buzz of micro-blogging coming out of the recent Haitian earthquake, and are attempting to improve the clear communication of information and help requests:

“The Haitian earthquake has led to the spontaneous use of Twitter to communicate about missing people, request help of various kinds, and other disaster-related needs. [Project Epic] is promoting ‘Tweek The Tweet’, a collection of defined hashtags (‘beacons’) and related microsyntax.”

The proposed microsyntax is designed based on realworld scenarios and messages. For example, the message

Altagrace Pierre needs help at Delmas 14 House no. 14.

becomes

#haiti #name Altagrace Pierre #need help #loc Delmas 14 House no. 14.

Assuming total trust and transparency when Tweeting, such microsyntax could support a localised community support network ranging from freecycling to getting urgent medical attention (First-Aiders keeping tabs on the “#need” search results in their area). Using such a casual, vernacular communication system for something so serious has major issues though. Tweets can tell lies. Tweets can be read by the stalking school gunman. Tweets can incriminate. How will our City look if we start fearing the walls we build?

The video shows reconstructed mesh models of the city of Rome. The meshes were constructed using “Structure from Motion“, a “process of finding the three-dimensional structure by analyzing the motion of an object over time”. In this case, the motion is actually the motion of tourists around the city, capturing the same buildings from many different angles. The power of such an approach is that thousands of creative-commons images of a particular city can be retrieved with a quick flickr search.

Sameer Agarwal’s team have built on existing reconstruction technology such as Microsoft’s PhotoSynth, and PhotoTourism, part of Washington U’s Community Photo Collections project. The big improvement however, is that they have extended it to run on many computers in parallel, thus allowing it to quickly process thousands of photos:

The data set consists of 150,000 images from Flickr.com associated with the tags “Rome” or “Roma”. Matching and reconstruction took a total of 21 hours on a cluster with 496 compute cores.

Previous Systems

I have been experimenting with structure-from-motion for a while now. Since late 2008, Photosynth has allowed users to upload and process their own photosets, and there are tools available (binarymillenium exporter, Photosynth Point Cloud Exporter) for exporting the resulting point-cloud, producing results like this:

Unfortunately, I have not had very successful results with my own photosets using PhotoSynth; the algorithm has failed to match images up correctly.

Video-based structure-from-motion

After disappointing results from image based structure-from-motion, I experimented with video-based processing. The theoretical advantage of video-based processing is that there are constraints on the positions of each concurrent frame, as the camera travels through its path of motion. There are a handful of camera-tracking applications designed for video production, intended to allow overlay of computer-generated content which moves correctly with the rest of the scene. Voodoo Camera Tracker is total free, and allows exporting of calculated point-clouds. The following are the quite promising results from an initial test:

PhotoTourism code

Excitingly, Washington U. has released the updated code used for PhotoTourism at the Bundler website. According to the site, Bundler is

a structure-from-motion system for unordered image collections (for instance, images from the Internet) written in C and C++. An earlier version of this system was used in the Photo Tourism project.

It will be interesting to see how robust the calculated camera-positions/point-clouds will be from this system. I will post results as I get them.

If you’ve had any experience with any of these systems, I’d love to hear from you (via the comments) as to the outcomes!

We live in a world which privileges sight, playing on a Western philosophical heritage that scopes out and objectifies the world. While sight affords us a voyeuristic viewpoint separated from the world (could we feel so detached if we could see 360 degrees around us?), sound envelopes us.

Research in the emerging field of acoustical archaeology suggests that ancient temples were designed with sound strongly in mind, and certainly many of them – such as the Chichen Itza in Central Mexico – exhibit extraordinary acoustic qualities. However, contemporary Architecture seems to be driven by an occularcentrism; still no acoustic modelling in architectural packages, while an architecture student friend is told by her lecturer that they work exclusively in the “graphic realm”.

I recently met up with a sound professional turned researcher, Mark Ward, at a conference on interactive entertainment (IE2009). Mark acknowledged the under-appreciated nature of sound, even in cinematographic sound design, and is working to advance the body of knowledge on sound, immersion, and emotion. Of particular interest to me is Mark’s overview of Impulse Response analysis. Impulse Response effectively captures the acoustic signature of a space, by examining the way sounds echo around it. For sound production this means an arbitrary sound can be modelled very accurately as it would sound within the space. For architectural applications this allows a high-quality, cheap, and standard way of recording and archiving the acoustic qualities of a location.