Various virtual reality (VR) techniques can be combined to support virtual prototyping of digital handheld products, but conventional VR-based prototyping still suffers from lack of reality and presence in spite of a significant amount of time and eff ...
Various virtual reality (VR) techniques can be combined to support virtual prototyping of digital handheld products, but conventional VR-based prototyping still suffers from lack of reality and presence in spite of a significant amount of time and effort, and costly investment in equipments necessary to construct a decent virtual environment. Moreover, it is not easy to acquire tangible user interaction with low-cost VR devices. Recently, augmented reality (AR) approaches have been applied as alternatives for developing VP solutions to overcome the shortcomings of VR-based prototyping.
In this study, we developed a new approach to tangible AR-based interaction and simulation for virtual prototyping of digital handheld products, which does not require high-cost devices, makes the user experience product design (shape, function, user interface) with ease, and provides a feeling like manipulating products with human hands.
The virtual model of a digital handheld product consists of four components: a product model, multimedia contents data, an HMI functional model, and a finite state machine. We addressed the generation of these components and their systematic operation, and described how to construct the virtual model using them. The HMI functional model is an information model that represents the human-machine interface (HMI) related functional behavior of the product. In this study, we adopted a state transition methodology to capture the functional behavior.
By manipulating tangible objects with his or her hands in a computer-vision based AR environment, the user interact with virtual product models which are overlaid on real world image in real time. We use two types of tangible objects: one is for the product, and the other is for pointer. The product-type tangible object is used to manipulate the position and orientation of the product in an AR environment. Combined with the product-type tangible object, the pointer-type tangible object is used as a tool to create HMI events. The user creates HMI events by touching specified regions (buttons or sliders) of the product-type tangible object with the pointer-type tangible object. We proposed interaction approaches using an RP model for the product-type tangible object and using a paper model or a finger fixture for a pointer-type tangible object. Interestingly, we found that using the RP model and the finger fixture as tangible objects, the user can interact with the virtual product by touching the product-type tangible object with the tip of his or her index finger wearing the finger fixture. Additionally, we estimated a valid region of the size and the interval to realize smooth and accurate interaction in the AR environment, and proposed a solution to recover the image of real objects (i.e., hands) occluded by the image of virtual objects in the AR environment.
Based on the proposed approach, we implemented a product design evaluation system based on the proposed approach to tangible AR-based interaction and simulation, and performed a preliminary user study to investigate the usefulness of quality of the approach. From the user study, we found that it obtained highly encouraging feedback from users, and that the system can be used as a good tool for design review and evaluation of digital handheld products.