Research Projects

This page included descriptions and images of some projects I have worked on. Publications are included for each project, and are summarised on the publications page. Videos are available on the videos page.

The development of an x-ray vision system using outdoor augmented reality was the focus of my PhD research.

The system uses a mobile robotic platform to capture video information from a remote scene. The robot is built using the framework, wheels and motors of an electric wheelchair. Custom electronics were build to allow computer control of the wheels. The robot is equipped with a survey-grade GPS unit, video camera, orientation sensor and laptop computer. These components allow the robot to be remotely controlled using wireless networks, and to capture video images with accurate position and orientation information.

The video images are then used in conjunction with a 3D geometric model of the environment to build an image-based reconstruction of a remote, occluded scene. Projective texturing is used to accurately texture the model using frames from the remote video stream.

By displaying a computer-generated view of a remote scene on the users display, registered with the corresponding real-world location, allows the user the ability of x-ray vision

This image demonstrates one of the visualisations, the Edge Overlay, which renders parts of the occluding surface (in this case a brick wall) on top of the x-ray vision. This produces a much more compelling sense of depth to the images. In addition to overlaying the edges, the edges of the x-ray vision are softly blended.

User interactions are also possible with the x-ray vision system. The user's hands are tracked using the head-mounted camera. The user may drag out a rectangular shape with their two thumbs to specify a region to display the x-ray vision. This is virtually tearing a hole through a wall to see to the other side

• Avery, B., Sandor, C., and Thomas, B. Improving Spatial Perception for Augmented Reality X-Ray Vision, In IEEE Virtual Reality. pp 79-82. Louisiana, USA. 2009 pdf

• Avery, B., Thomas, B., and Piekarski, W. User Evaluation of See-Through Vision for Mobile Outdoor Augmented Reality. In 7th Int'l Symposium on Mixed and Augmented Reality. pp 69-72. Cambridge, UK. 2008 pdf

• Avery, B., Piekarski, W., and Thomas, B. Visualizing Occluded Physical Objects in Unfamiliar Outdoor Augmented Reality Environments. In 6th Int'l Symposium on Mixed and Augmented Reality. pp 285-286. Nara, Japan. 2007. pdf

I worked with Ross Smith on a short project to build an arm tracking system for our mobile outdoor AR systems using orientation sensors attached to the users' arm segments. Intersense Inertiacube 3's were attached to the forearm and upperarm with elastic straps. An additional sensor is on the AR head-set. Using kinematics, a hand location can be calculated. Custom pinch-gloves are also incorporated allowing the user to use pinch-gestures to select objects. The screenshot below shows an example application in which the user can move their tracked hand/arm (shown as a gray cylinder+sphere) to select and move virtual red balloons.

• Smith, R. Avery, B., and Thomas, B. Lightweight Hand and Arm Tracking for Mobile Augmented Reality. In 12th Int'l Sympoisum on Wearable Computers. pp 105-106. Pittsburgh, USA. 2008 pdf

I have worked on multiple separate projects constructing compact and robust outdoor wearable computer systems. All previous implementations involved large backpack frameworks with large components attached. Our goal was to take a known design and compact it as much as possible. Components were all removed from their manufacturers casings and embedded into the belt-worn case. Components were all permanently connected to one another with solder joints preventing the space taken up by bulky connectors. The NiMH batteries are separately mounted to the belt to facilitate hot-swapping etc.

The belt-worn case contains a number of components:

• 1.7Ghz Laptop motherboard,
• Survey-grade GPS processor,
• Power regulators (12 volts, 3.3 volts),
• Hard Disk Drive,
• Bluetooth module,
• USB hub,
• USB/RS-232 converter,
• Wireless video transmitter, and
• Custom micro-controller electronics.

Connected to the belt-word case is the head-mounted electronics. Mounted to the head are:

• GPS Antenna,
• Orientation sensor,
• Camera,
• Head-mounted display, and
• Speakers

The design of these wearable computer is being published as a book chapter by Springer in a book entitled Engineering Mixed Reality System in late 2009.

TEXT

• Avery, B., Piekarski, W., Warren, J., and Thomas, B. Evaluation of User Satisfaction and Learnability for Outdoor Augmented Reality Gaming. In 7th Australasian User Interface Conference. pp 17-24. 2006. pdf

• Avery, B., Thomas, B., Velikovsky, J., and Piekarski, W. Outdoor Augmented Reality on Five Dollars a Day. In 6th Australasian User Interface Conference. pp 79-88. Newcastle, NSW. 2005. pdf

TEXT

• Piekarski, W., Avery, B., Thomas, B., and Malbezin, P. Hybrid Indoor and Outdoor Tracking for Mobile 3D Mixed Reality. In 3rd Int'l Symposium on Mixed and Augmented Reality. Tokyo, Japan. 2004. pdf

• Piekarski, W., Avery, B., Thomas, B., and Malbezin, P. Integrated Head and Hand Tracking for Indoor and Outdoor Augmented Reality. In IEEE Virtual Reality. pp 11-276. 2004. pdf