A Writers Notebook
A mixed reality media installation using vision/sensor fusion
The sample images here are from the first installation, A Writers Notebook, shown in the in-conjunction exhibition to the International Symposium on Electronics Arts in July 2008. The installation shown here is “in-progress presentation of the research in real-time stable marker-less tracking techniques currently underway in the Interaction and Entertainment Research Center. This first installment places a virtual characterization of the writer Somerset Maugham in proximity with the viewer in such way that allows the viewer to move around and view from multiple angles. A video describing the mixed treality technology and installation is below:
Participants wearing head mounted display systems, witness virtual characters of various notable figures, including Somerset Maugham, Joseph Conrad, and Jean Harlow, immersed within a real world environment that is modeled on the Raffles Hotel Long Bar they once frequented. This system allows the user to experience pseudo-historical events impressed over a present day real world environment. Through the application of research in occlusion, and by embedding large mesh animated characters, this installation demonstrates the results of the technical research and the conceptual development of immersive multimodal interaction.
A Writers Notebook is a "sketch" in the development of the larger mixed reality installation, "The Long Bar." The Long Bar is an augmented and mixed reality multi-media art installation set at the famous Long Bar at the Raffles Hotel. The installation uses AR and MXR technologies to develop historical, culturally significant events into a multi-modal immersive interactive experience.
The installation is a demontration of in-progress research in real-time stable marker-less tracking techniques currently underway in the Interaction and Entertainment Research Center. This first installation places virtual tigers in proximity with the viewer in such way that allows the viewer to move around and view from multiple angles and use hand and voice signal to coax responses form the virtual animals. Through gesture and voice recognition prompts given by the viewers can be used to coax responsive interactions from these virtual animals. Inherent in this training is an underlying question as to who is really being trained in such circumstances. As we see in animal training shows in animal parks humans inevitably alter their behaviours to elicit the response from the animals. These behaviours are unnatural and belie the truth that the trainer is being trained as much as the animals.
International Symposium on Electronic Art 2008 Project Background
2009 Pensyl, W. (Russell),), Tran, C.T.Q., Lee, S.P, Jernigan, D. K. “Robust Hybrid Tracking with Life-size Avatar in Mixed Reality Environment,†13th International Conference on Human-Computer Interaction 2009,San Diego, California, USA
1. ARTOOLKIT http://www.hitl.
washington.edu/artoolkit
2. CAARLS, J., JONKER, P.P., PERSA, S. Sensor Fusionfor Augmented Reality. EUSAI 2003: 160-176
3. HOL, J.D. SCHON, T.B., GUSTAFSSON, F., SLYCKE, P.J. 2006. Sensor fusion for augmented reality. In Proceedings of 9th International Conference on Information Fusion.
4. KATO, H., BILLING-HURST, M. 1999. Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In
Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 99). October, San
Francisco, USA
5. KSATRIA GAMEWORKS. http://www.ksatria.com
MXRTOOLKIT http://sourceforge.net/projects/mxrtoolkit
6. NGUYEN, T.H.D., QUI, T.C.T., XU, K., CHEOK, A.D.,
TEO, S.L., ZHOU, Z.Y., ALLAWAARACHCHI, A., LEE, S.P., LIU, W., TEO, H.S., THANG, L.N., LI, Y., and
KATO, H. 2005. Real Time 3D Human Capture System for Mixed-Reality Art and Entertainment, IEEE Transaction
on Visualization and Computer Graphics (TVCG), 11, 6 (2005), 706 - 721.
8. SLABAUGH, G., SCHAFER, R., LEVINGSTON, M. 2001.
Optimal ray intersection for computing 3D points from Nview correspondences.
9. SONG, M., ELIAS, T., MARTINOVIC, I., MUELLERWITTIG,
W., CHAN, T.K.Y., 2004. Digital Heritage
Application as an Edutainment Tool. In Proceedings of the
2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry
10. SPARACINO, F., WREN, C., DAVENPORT, G., PENTLAND, A. Augmented performance in dance and theater. In International Dance and Technology 99, ASU, Tempe, Arizona, February 1999.
11. UEMATSU, Y., SAITO, H. 2007. Improvement of accuracy for 2D marker-based tracking using particle filter. In Proceedings of 17th International Conference on Artificial Reality and Telexistence.
12. WII REMOTE http://wiibrew.org/wiki/
Wii_Remote
13. WII YOURSELF. http://wiiyourself.gl.tter.org/