Our team has observed the autonomous construction issue in past decade (2007-2017), and designed the AutoCon, an autonomous structural component connector, over the past three years (2015-2017). Autonomous construction is a long-lasting yet critical problem in construction industry. The rising demand of automation technology has caused progressive development and application in several industries recently, such as manufacturing industry. They have suffered from issues including insufficient worker and low-quality working environment. In the current trends, people are less willing to work as handler or operator in unsafe condition, which results in insufficient worker. On the other hand, the recent rise of human right awareness also requests a higher quality of working environment. Therefore, the automation technology is a solution for these issues. For example, maneuvering robots in production line which worker can tele-operate at safe location. This also happens in construction industry, featuring a more hazardous and unsafe working environment.
Construction industry is the economy locomotive of the industry, whose competitiveness is affected by speed and quality of construction process. Take Taiwan Semiconductor Manufacturing Company (TSMC) as an example, they utilize 3D visualization and on-site vision system to improve the efficiency of new high-tech facilities construction in order to maintain the pioneer of high-tech industry. We believe that the development of autonomous construction could also affect the other industry. On the other hand, on-site safety is always the most critical problem throughout all kinds of industry. In 2015, construction industry took 19.3% of working fatal accident happened in USA and over 4 times higher than overall average in Taiwan, which makes less people want to work on construction site. National Development Council of Taiwan predicted that the construction industry is the only one having decreasing employment population between 2015 and 2025, with 2455 people leaving the industry every year. Therefore, with the development of autonomous construction, worker will not require to work in an unsafe environment, which will decrease the fatal toll and increase the willingness to work in construction industry.
Despite the huge demand of autonomous construction, the key technique is still under developing and not provided in the industry. In the past decade, the computational speed, the cost of hardware, and the artificial intelligence algorithm are improved rapidly, which provides a turning opportunity for the autonomous construction. Comparing with 1960s, the construction methods have not changed significantly even though the autonomous construction are widely discussed and improved. The current automation methods are restricted to the structural component which is not designed for autonomous construction purpose. Thus, in order to develop an autonomous construction method, we designed a new structural connector, named AutoCon, for construction industry.
Crowd Intelligence on Flow Velocity Measurement (2012-2014)
Flow velocity measurement is important for hydrology survey and disaster prevention. Recently due to the popularity of the image sensors and processors, the image-based flow velocity measurement becomes an important research direction. Particle Image Velocimetry (PIV) is one of the major examples. It compares the neighboring frames in the video clips, identifies the similar image features and calculates the velocity of the flow. However, due to the uncertainty of the features, PIV sometimes becomes very inaccurate. In this research, we would like to take the advantages of human’s perception system- with strong ability of feature identification and tracking. We developed a method called Crowd-Based Velocimetry (CBV) to allow incorporate the human perception capacity in the estimation of the flow velocity. CBV includes three main steps: (1) video preprocessing, (2) feature tracking by the crowd,  (3) velocity calculation. In the video preprocessing step, we first project the raw video, which has an inclined view angle to the view perpendicular to the water surface. Then we apply two enhancement algorithms, Canny edge detector and Eulerian Video Magnification to produce two additional video clips. Finally we segment the video clips into the still images based on the estimated range of flow velocity and the required size of data. For the second step, we developed a user interface to allow the crowd to specify similar features in neighboring images. The crowd need also express their confidence when identifying the features using the user interface. For the third step, we calculate the flow velocity from the crowd inputs and accumulate the results for further analysis. Here we calculate the velocity by dividing the time difference between the images from the distance between similar features on the two images. To validate the CBV, we used a video clip from the field. The video includes significant features on the water surface. The measurements of the flow velocity from the center of two surfaces using radar meter (Stalker SVR Pro II) is 1.73 m/s. We also used PIV to calculate the flow velocity and get the results, 0.57 m/s, which is 67% difference from the radar measurement. The velocity calculated by CBV is 1.80 m/s, which is 4% difference from the radar measurement.
A Video-Enabled Semantic Virtual Construction (2012-2014)
Good site layout planning can avoid possible conflicts of equipment and increase accessibility during a construction process. In current practice, it is difficult to re-plan a layout during an ongoing site process, since the location and quantity of on-site resources change over time. Many researchers have developed on-site cameras to facilitate dynamic space management, but this approach still suffers from fragmentation between site monitoring and planning. In this research, we aimed to rapidly retrieve geometric information from on-site cameras for dynamic site planning. To this end, we developed a method containing four steps—projection, duplication, description, and calibration—and incorporated it in a novel software tool. The first step, projection, aims to establish a projective model that maps video images to the actual site using cameras. The second step, duplication, aims to rapidly acquire the positions and dimensions of construction objects based on the video images and model their 3D geometries in a geometric virtual construction. The third step, description, aims to link the geometric models with their corresponding real objects and build a semantic virtual constriction, which incorporates not only on-site scenarios but also engineer knowledge. The fourth step, calibration, aims to improve the accuracy of the virtual construction for realistic planning. The software tool thus developed allows engineers to load a video and specify the location of each filmed object, as well as to specify the meaning of the objects. The algorithms in the software tool simultaneously create numerical models to link the images on the video with the virtual models. We used a real campus building to validate the usability of our method. Using closed circuit television (CCTV) videos retrieved from the site, the software took 123 seconds to create a corresponding virtual construction site. We compared actual measurements of the modeled objects with their virtual dimensions and found errors ranged from 0.2 to 1.2 m. The low error in the results indicates that our method is a feasible way to transform video data to a numerical virtual construction site with reasonable time and accuracy.
Rear-Screen and Kinesthetic Vision 3D Manipulator (2012-2014)
3D manipulation, comprehend and control 3D objects on computers effectively, is a long-lasting problem. Most existing control interfaces locate in front of the display. This requires manipulators to imagine that manipulated objects exist in front of the display, which is actually behind the display from visual perception. In the research, Rear-Screen and Kinesthetic Vision 3D Manipulator are developed for models manipulating on laptops to solve the above-mentioned problem. Manipulators are enabled to manipulate 3D objects directly, achieving the "Real Space Virtual Reality." Instead of the front-screen setup of a motion sensor, it tracks user's hand motions behind screens, coupling the actual interactive space with the perceived visual space. In addition, to provide dynamic perspective of objects according to manipulators' sights, heads' position are tracked, so called Kinesthetic Vision. The purpose is to gain depth perception by the "motion parallax" effect. To evaluate the performance of the proposed method, an experiment was conducted in 3 setups for comparison: front-screen, rear-screen and rear-screen with Kinesthetic Vision. In the experiment, 12 Subjects were recruited and asked to grasp and move a cube from a fixed starting location to a designated location in each trial. There were 20 designated locations scattered in interactive space. In each setup, subjects were asked to go through 5 blocks of trials, including 20 trials in a block. Moving time and distance were recorded during experiments. By repeated measured ANOVA, results show that there are significant differences in object grabbed time and moving efficiency, especially in depth direction of movements. In conclusion, the setup of Rear-Screen and Kinesthetic Vision make better performance for 3D manipulation.
ABAS: an Autonomous Beam Assembly System for Steel Structure (2012-2014)
This research focuses on a long-lasting yet critical problem in erection assembly of steel structures. In current state of practice, steel workers need to stand on the unfinished structure to assist the assembly of the structural elements manually. They need to pull the wire hanging under the rigging elements and align the bolting holes between the moving and fixed elements. This works are often performed in high places, which can be very risky. Therefore, we developed an autonomous beam assembly system (ABAS) to prevent the workers from working on the high place. The ABAS consists of four methods: rotation, alignment, bolting and unloading. The rotation method is a flywheel equipped on the top of the rigging beam aiming to rotate the beam to the assembly angle. The alignment method includes both vertical and horizontal alignments. The vertical alignment relies on a camera, a marker on the column and a light signal to align the beam altitude. The horizontal alignment relies on the special-designed shape that can smoothly guide the beam into the right position. The bolting method is used to assembly the beam. We designed an additional plug hole above each bolt hole. The bolt can plug in and slide to the bolt hole. The unloading method is used to unload the cable and the ABAS. We use a pin mechanism for beam-hook connection so it can be removed by motor easily. The system is built in a scaled-experiment construction site to validate the feasibility. The result shows that the ABAS needs less operation space and operates faster than the traditional method. In conclusion, we develop an autonomous beam assembly system that can help reduce the accidental falls of the steel beam assembly process. The ABAS fits the current erection method and can be broadly introduced to existing construction sites.
Design Decision Recorder (2012-2013)
This research is proposed and implemented cooperatively with Prof. Wolfgang Huhnt in TU Berlin. It focuses on developing a semi-automatic approach to document implicit design ideas of construction modeling processes, and on the other hand, providing right-in-time notifications when subsequent modifications take place that affect earlier design decisions. In this research, designers for construction models are requested to record their design knowledge, regulations, and limitations by attaching relationships with annotations among geometrical objects they created during modeling procedures. A graph structure of design decisions is then established, which can be used to trace necessary information by traverse algorithms. With the help of traverse mechanisms on the design decision graph, designers are able to retrieve relevant notes recorded by themselves and at the same time get notification concerning possible design conflicts if the design decision graph shows dependency loops. A Rhinoceros’ plug-in, named Design Decision Recorder, has been implemented. As a pilot study, the implementation shows great potentials in construction modeling assistance. Possible extensions to the implementation and user feedback have to be evaluated as a next step.
Accessibility Study for Pipeline Maintenance (2011-2013)
Pipeline maintenance is becoming an important issue in modern construction and building information model (BIM) research. An understanding of pipeline accessibility considerations in terms of operation and maintenance is essential for planning and management. Previous studies have highlighted the complexity of multi-pipes including mechanical, electrical and plumbing (MEP) pipelines and the importance of information visualization, but few have proposed a way to consider accessibility problems during operation and maintenance. Therefore, this study develops a systematic method to evaluate accessibility with respect to pipeline maintenance. We first divided pipeline accessibility into three categories: (1) visual accessibility—the visibility for an inspector to view; (2) approachable accessibility—the difficulty for an inspector to approach; and (3) operational accessibility—the pipeline that can be operated by the inspectors. We created mathematical models and discussed the ergonomic details about each category. We then developed a user interface, VAO Checker, in which V, A and O stand for visual, approachable and operational respectively, to display visual information about pipeline accessibility. Through instantaneous analysis, the system visualizes the accessibility of the pipelines. We visually represent the intersection and union of these three categories to illustrate the varying accessibility of pipe elements. A usability test was conducted to validate the system’s effectiveness. The results of the usability analysis show that users have higher correctness when using VAO Checker than 2D plan drawing and 3D model, and they evaluate the performance of this tool better than 2D plan drawing. Pipeline designers can benefit by using this tool to sketch a suitable traffic flow for engineers to investigate. Furthermore, the substantial amount of information saved in the layout database could be referenced for future optimization.
Using Augmented Reality in a Multi-Screen Environment for Construction Discussion (2011-2013)
Discussion is critical in identifying, predicting, and resolving the potential problems in the field of construction. This process relies heavily on oral communication with the assistance of construction drawings, schedules and other related documents. Since most construction projects include multiple working phases and involve multiple parties, it is difficult for participants to clearly grasp the whole picture of a construction site and to make accurate predictions about future activities. In this research, we proposed a visualized environment to facilitate the discussion process. It includes a stationary display called ″BIM Table″ for displaying public information and for collaboration among disciplines, and multiple mobile devices for showing private information. We employed augmented reality technologies to connect the BIM Table and the mobile devices as well as the public and private information. We named this discussion environment AR-MS (Augmented Reality and Multi-Screen) system. This system aims to reduce the complexity of discussion information while keeping necessary information available during the entire discussion process. To validate the AR-MS system, a user test with 36 participants (N=36) was conducted. The participants were required to perform the three tasks: data-finding (DF), problem prediction (PP), and decision-making (DM) using both the AR-MS system and the conventional paper-based method. We discovered that the completion time is significantly shortened using the AR-MS system in both DF and PP tasks, while the accuracy in all three tasks showed no significant difference between AR-MS system and conventional method. Based on the results, the AR-MS system can be of benefit in information searching and foreseeing potential problems on a construction site and the discussion process can be made more effective and efficient by its use.
Fall Protection Safety Training Suite (2010-2012)
This research is proposed and implemented cooperatively with Prof. Ken-Yu Lin in University of Washington. Construction has been one of the most dangerous industries, with fall being the most common type of hazards. This research is sponsored by OSHA to develop six 3D visualized and scenario-based training cases on the topic of fall protection. The cases use minimum amount of text descriptions and intend to maximize the benefits of visualization. We hope our 3D simulated training scenarios will reduce the language and literacy barriers for potential trainees, and increase trainees understanding as well as learning interests on the topic of fall protection.
iSafe: An Innovative iPad System for Construction Site Safety Audits (2010-2012)
This research is proposed and implemented cooperatively with Prof. Ken-Yu Lin in University of Washington. Occupational safety is an important issue in construction. Although technology innovations are benefiting the different areas of construction, safety, still being challenged by fatalities in the industry, has not fully taken advantage of these innovations. Besides, safety audit data collected by contractors internally have mostly been used for meeting discussions only and not been further utilized for active learning. Additionally, the paper-based audit method is not well structured and often leads to repeated efforts and inconsistency among the data records. In this research, we proposed a safety audit system, iSafe, to facilitate the safety audit process on site and to collect high-quality data records for violation pattern analysis. The system went through three stages of design and development with the participation of safety subject experts from the greater Seattle area. An iPad 2 based prototype system was developed for field use and evaluation. A questionnaire survey was administered for the initial assessment of iSafe and the system is currently undergoing a three month field evaluation phase with the participation of three general contractors. The design and development of the system has been previously reported and the paper will limit its discussion to the strategies used during prototype deployment.
Motion Planning and Coordination for Mobile Construction Machinery (2010-2012)
The aim of this research is to develop a motion planning and coordination method for commonly used construction machinery, including excavators, mobile cranes, trucks, and tractor-trailers. This method includes four steps: (1) model development, (2) C-Space (configuration space) construction, (3) path finding, and (4) machine coordination. The first step is to categorize construction machinery into three types: wheel, track, and chain, which is followed by further simulation of the behavior of the machinery using mathematical equations. The second step is to construct the C-Space for the machinery. This transfers a complex geometric problem in real-world space (i.e., Cartesian space) into an artificial space, which can simplify the following path-planning process. The third step is to compute the collision-free paths in the C-Space and transfer them into Cartesian space. The fourth step is to coordinate the paths of the machinery to avoid possible conflicts. In order to demonstrate and validate the four-step method, a computer program is implemented, which simulates and visualizes the motion of construction machinery by using a graphics engine, Microsoft XNA, and a physics engine, Nvidia PhysX. A virtual construction site is developed to test the efficiency of the developed method, with narrow working areas that only allow the passage of crawler cranes or similar machinery. A case in which two construction machines simultaneously move on a construction site and have the possibility of collision is designed and tested on the virtual site. The results of this test case indicate that the time required for the calculation of path planning and coordination of the machines is approximately one second, while the construction of the C-Space averaging 10.45 seconds for wheel and track machines, and about 45.21 seconds for chain machines. The results of this study demonstrate the successful implementation of a model and C-Space of construction machines and the generation of a collision-free path for multiple machines. This research provides an effective method for planning the operation of construction machinery, and has practical applications in construction planning.
A Lightweight Bridge Inspection System Using a Dual-cable Suspension Mechanism (2010-2012)
Visual inspection is the primary means of evaluation of the structural conditions of a bridge. Since visual inspection often requires engineers to place themselves in hazardous environments, many researchers have developed inspection vehicles and robot arms in order to increase safety and accessibility during inspections. However, the use of such heavy equipment may only be suitable for large bridges due to restrictions in working space. Therefore, this research aims to develop an innovative bridge inspection system that is portable, simple, affordable, and can be easily applied on smaller yet large number bridges. We developed a systematic approach, which includes a lightweight inspection system using a dual-cable suspension mechanism, as well as strategies to conduct inspections. The inspection system is composed of three components: the controller, the dual-cable suspension, and the inspection platform. By using a dual-cable suspension mechanism, the bridge inspector can manipulate the controller to place the inspection platform beneath the bridge and capture images with digital vision devices. Unlike robot arms or heavy vehicles, the cable-based system has the capacity to sustain much more than its own weight. This means that the total weight is greatly reduced and the cost is much lower than in existing methods. Additionally, the inspection strategies developed in this research can provide complete, efficient and flexible inspection procedures. Vertical and horizontal inspection strategies have been developed to allow sequential inspection for normal structures and multi-view inspection for special structures by manipulation of the inspection system. A prototype of the inspection system has been constructed and two field experiments were conducted to test the feasibility of the proposed method. The experimental results proved that the proposed system was able to work efficiently on a narrow bridge without affecting the traffic. The inspection system can also provide centimeter-level resolution inspection images, allowing general defects such as cracks and spalling concrete to be seen clearly. With further research, the proposed system and concept is expected to enhance overall bridge inspection efficiency.
Control of Fast Crane Operation (2011-2012)
Cranes are often a governing factor  in the efficiency of critical tasks undertaken during construction programs. The speed of crane erection can significantly influence the overall project progress. Fast crane operation may result in large sway of the hanging object, raising safety concerns. This research aimed to develop a simple control method for fast crane operation. The sway angle was limited to maintain the system’s controllability and safety. To achieve the goal, we first developed a mathematical model using single and double pendulum systems to simulate the sway of the crane cable. By analyzing the equations of motion, we found that a three-stage procedure consisting of (1) piecewise acceleration, (2) constant speed, and (3) piecewise deceleration, was able to effectively reduce the sway angle yet still maintain high-speed operation. From the numerical simulations we found that the operational time was proportional to the square root of the traveling distance, which indicated that the three-stage procedure suited fast operation, especially for longer distances. We also conducted a physical experiment using a high precision robot arm, the KUKA Robot KR 16, as a scaled crane in order to validate the predictions of the numerical experiments. The experimental results were found to be in good agreement with numerical simulations. In short, the proposed method takes advantage of pendulum motions physics. Without the addition of any sensors the operator was able to control the sway of the crane cable by following the principles of the three-stage procedure. In future, we hope to see this proposed open-loop control method integrated into current crane control mechanisms to promote crane operation that is both safe and fast. 
SimCrane 3D+ (2010-2012)
SimCrane 3D+ is a crane simulator with a new simulation approach. Users can experience more realistic operations in the simulator. In order to improves a critical drawback commonly seen in existing simulators: lack of perception of 3D depth, this research adds two major components, kinesthetic vision and stereoscopic vision to the system. To realize the kinesthetic vision, we integrated Microsoft Kinect as the motion sensor. We also derived two transformation matrices, one for the dynamic eye position captured from the motion sensor and the other one for calibrating the distortion due to the inclined view angle. The stereoscopic vision was realized by integrating NVIDIA 3D VisionTM package, which includes a 3D rendering pipeline and a pair of 3D glasses, with a 3D display. We finally developed a crane simulator, coined SimCrane 3D+, by integrating the kinesthetic vision and stereoscopic vision in a game framework based on Microsoft XNA toolset. We found that SimCrane 3D+ can process the continuous readings from the motion sensor and smoothly rendering stereoscopic views. With the enhancement of kinesthetic and stereoscopic vision, the users obtain better perception of depth and precise visibility during the operation. In the future, the research results have great potential to be used for training operators and rehearsal critical erections.
Attention-Based User Interface for Tele-Operated Crane (2009-2011)
Conference Paper: 2011(Scopus)
Recent investigations into the potentials of tele-operation have revealed that risks to human safety at the jobsite can be reduced drastically with the removal of the need for many personnel to be physically onsite. This research focused on the interface design of a tele-operated crane by using markerless augmented reality (AR) technologies and considering human attention. The attention-based interface has been designed with consideration of how the users' attention is divided into three view types: focused views, ambient views, and alert views. To verify the usability of the interface, we conducted a user test in a simulated environment using a robot arm. The results indicate that the interface prevented 57% of collisions possibility. The utility rate of the different view types was found to be correlated with general erection activities. The research results provide an ideal reference for designing user interfaces of tele-operated cranes in the future.
Zebra: an AR-Enhanced Tele-Operated Crane Prototype (2009-2011)
This research focuses on one of the major challenges in a tele-operated crane system: the user interface (UI). This UI needs to provide rich information retrieved from the fields and display them properly to enhance the operation and decision-making processes. In this research, we designed Augmented Reality enhanced (AR-enhanced) and quad-view UIs for the tele-operated crane system. The tele-operated crane system is built based on an industrial robot arm, KUKA CR16, and IP cameras as remote sensing units. A user test of the system has been conducted. The results show that AR-enhanced displaying and multiple views can significantly improve the efficiency and reduce the loading of operators comparing to transitional operation method.
Pavement Inspection Robot (2006-2011) 
Pavement inspection robot is a novel idea for obtaining the pavement profile of an existing road. The eventual goal of the ongoing project is to develop an autonomous robot which can simultaneously perform multiple inspection tasks during its motion. This robot should be able to compute the pavement condition in real-time and determine the coming inspection goals. The motion-planning module for the robot has already been developed. It allows the robot to perform inspection tasks following four motion modes, including longitudinal, transversal, random and grid modes. In the near future, multiple sensors will be integrated into the robot to test the performance of the autonomous behaviors.
A Walkthrough of Large Scale Composite Hazard (2011)
The composite hazard consisting of flood, debris flow, deep collapse, shallow collapse and landslide dams was responsible for the massive scale of the disaster in Siaolin village from the typhoon Marakotin Taiwan. This kind of disasters are usually not well understood by the general public; however they often cause huge damage due to the lack of vigilance regarding such disasters. In this research, we proposed a game-based walkthrough approach to simulate a large scale and composite hazard to promote of disaster prevention. The system uses a storytelling style combined with a walkthrough of the disaster in a virtual environment. The disaster introduction method consists of two steps: the introduction step and the simulation step. The introduction step introduces the process and details of the disaster through voice narration and reading material. Then the simulation step provides the walk through and visualization of the disaster. The proposed method provides a story telling based, interactive environment, which can motivate people to understand the disaster and help them realize the importance of disaster preparedness. Through this system, we are able to understand the whole course of the hazards, raise people’s awareness of the disaster and thereby reduce the damage caused.
SUM Framework (2007-2011)
This research presents a three-stage consulting framework of system evaluation, usability study and management plan (SUM) was proposed and implemented for a firm of this size, which included three parts: (1) System Evaluation: requirement analysis and performance evaluation of both hardware and software components of the 4D tool; (2) Usability Study: usability tests and improvement of the 4D tool; and (3) Management Plan: workflow re-engineering for the firm to be able to successfully implement and apply the 4D management tool to actual projects. To validate the proposed framework, we worked with an industrial partner to apply the SUM framework in practice. The results show that this framework can effectively identify the problems encountered during the introduction of a 4D tool, including problems in system limitation, engineers' capacities, and workflow changes.
Construction Truck Accessibility Evaluation (2009-2010)
Construction site layout planning is concerned with the existence, positioning, and timing of the temporary facilities that are used when throughout a construction project. Previous researchers have developed many approaches for dealing with this complicated problem. However, little research has been conducted on the problem of transport accessibility, which can significantly influence the operational efficiency of transportation and safety of the working environment. Therefore, we aimed to develop computational methods to visualize and simulate the transportation details at job sites. In this research, we developed an accessibility evaluation system comprising of four modules: (1) transportation module, (2) site importing module, (3) accessibility evaluation module, and (4) visualization module. The computational system was implemented using Microsoft XNA (a game platform) and Nvidia PhysX (a game engine). The realistic visualization simulations made possible using this approach can provide solid references to engineers to enhance the quality of layout planning. Using the system we developed, engineers can identify potential accessibility problems and unsafe situations early in the design and planning stages.
IDEAL Method (2009-2010)
Imagination is an important ability for humans and it needs to be cultivated. Based on our literature review on the nature and imagination, it can be concluded that imagination is a mental process characteristic of the following three key features: possibility, connectivity, and boundary-crossing. In view of these features, an imagination training model called IDEAL was developed by our research team. The IDEAL training model consists of four basic steps: initiation (I), development (DE), alternative (A) and links (L), this model is concrete and easy to operate. Our research team had applied the IDEAL model in various college courses, including general education courses, freshmen career planning and a keystone course for engineering students. 
SPACE JAM (2009-2010) 
SPACE JAM is a multi-user shared environment for inter-generational communication, based on blending the real and the virtual environments. SPACE JAM connects people of different generations without forcing them to adopt unfamiliar communication tools. It demonstrates the way in which our everyday environment can be augmented with computational capabilities while maintaining the natural interactions. The setup of SPACE JAM consists of a computer, a video camera, and a projector, which are placed in front of a physical wall. A camera captures and identifies the physical objects and messages posted on the wall, and the computer converts the information into the virtual living space such as Twitter or Facebook. Digital information can also be displayed on the wall by a projector. SPACE JAM also combines recognition technology and multimedia software to handle photo browsing. The usability tests were conducted in Apr,2010 and by elderly welfare promotion group, NTU INSIGHT Center. The tests have involved 6 test users and feedbacks were sent back to R&D department for prototype revision. 
Hand-Eye Coordination Training Software for Dementia Prevention (2009-2010) 
The hand-eye coordination training software aims to activate memory and hand-eye coordination abilities which are controlled by the frontal lobe and decline substantially as the incidence of dementia, an irreversible aging phenomenon. This software contributes to delay the incidence of dementia and also reduce the frequency of it through interesting interactive games. In the games, users are requested to follow the guidance and press the objects appearing on the screen as soon as possible in a total of 30 seconds. When finishing the game, the users will receive feedbacks (no score display) based on their performance and data will be recorded for tracking each case. The usability tests of the hand-eye coordination training software were conducted in Jul, 2010 and by elderly welfare promotion group, NTU INSIGHT Center. The usability tests have involved 10 test users. 
BIM-AR iHelmet (2008-2010) 
This research focuses on a long-lasting construction problem: onsite information retrieval. We have therefore developed a wearable device, coined the iHelmet, that can project the construction drawings and related information based on the needs of the users. This device avoids engineers carrying bulky construction drawings to the site and reduces the effort required in looking for the correct drawings to obtain the information they need. To validate the usability of the iHelmet onsite, we conducted a user test with 34 participants. We compared the efficiency and effectiveness of retrieving building information using the iHelmet with using the traditional 2D drawings approach. The results showed that the mean completion times were significantly shorter for participants using the iHelmet. The mean success rates of participants arriving at the correct answers were also significantly improved for those using the iHelmet. 
Erection Director (2006-2010) 
The Erection Director project aims to develop a numerical simulation method that can visualize realistic erection activities in operational details. We integrated physics-based animation methods which are widely used in game physics or training simulators to generate detailed erection activities in a virtual environment. A physics-based environment for simulating detailed erection activities, Erection Director, was built in this research. Because the high-fidelity visualizations can be generated in the virtual environment, in the future engineers will be able to plan the erection activities precisely before physical construction commences. 
4D Color (2007-2010) 
The 4D color research aims to develop ideal color schemes for 4D models. Toward this goal, we developed the selection, examination, and user test (SEUT) procedure, a systematic procedure to determine the ideal color schemes to present 4D models. In addition to the requirements for the construction purposes, we consider the usability of certain colors in 4D models in the SEUT procedure, too. In other words, we consider both the perceptional and the psychological capacities of users while selecting the colors in this research. 4D Color Selector and Evaluator is a website which presents our work in finding the ideal color scheme for 4D models. We currently list three color schemes and their corresponding images of 4D simulations with two industrial 4D software on the website. 
Virtual Hydraulic Experiment (2007-2010) 
Laboratory class is very important in the education of hydraulic mechanics. However, it usually requires a high expenditure on equipment purchase and maintenance. Virtual equipment that renders the equipment in a virtual world and provides an interactive interface for users can be an ideal solution to reduce costs. In this research, we focus on developing virtual equipment for a Venturi tube experiment. We employed multiple paper-prototypes to gather user feedback on the preliminary design. Following the final prototype, we then implemented the virtual equipment on computers. Interactive functions have been built into the virtual equipment. Users are able to manipulate the equipment and obtain real-time feedback as if they were using actual equipment. The results indicate that the development of the virtual equipment is technically feasible. 
Sleeper Coach (2007-2009) 
The eventual goal of sleep coach research is to develop a sleeper-centered environment to improve sleep quality for insomnia patients. Currently, the research team is focusing on developing a handheld device, Sleep Coach, to facilitate users to develop the benefits of habitual sleep. Sleep Coach includes four modules: screening, advice, training, and diary modules. In the future, we will connect the Sleep Coach device to an environmental control system to create a pervasive computing environment, which can dynamically adapt to provide the most suitable environment to enhance the users' sleep qualities. 
SimuSurvey (2006-2010) 
SimuSurvey is a computer-based simulator for the purpose of surveyor training in a computer-generated virtual environment. Five subsystems are included in SimuSurvey to support various training activities: the level simulator, theodolite simulator, accessory simulator, total station simulator, and the tangible controller. This high-fidelity simulation environment aims to also enhance learning results and enrich students' learning experience. The instrumental errors were implemented as a module and integrated with SimuSurvey. We derived the equations to simulated and visualized the causes of the errors on computers by analyzing the spatial relationship between the rotational axes in the instrument and the reference axes in the surveying environment. These instrumental errors have considering ten instrumental errors, reflecting the reality in an actual survey. Instructors may ''create'' multiple instruments with different errors to facilitate in-class explanations and other teaching activities. 
Security Robot Simulator (2006-2008) 
Security Robot Simulator (SRS) is a virtual environment in which users can simulate the behaviors of security robots. Since physics-based visualizations and interactive user interfaces are provided, security professionals can use SRS to design the security plans or validate the effectiveness of the plans. SRS includes a patrol-planning module, which can compute the most efficient patrol path for security robots in a dynamic environment. This system can also be used by security guards to visualize and evaluate security planning. 
iCrane (2003-2005) 
The major goal of the iCrane research program is to develop computational methods and tools to plan, simulate, and visualize the operation of construction cranes in virtual automation environments. The crane-specific motion planning methods and collision-detecting methods were developed to enable the crane to follow safe and efficient paths that can be used to erect the structural elements of a construction project. A crane-specific modeling method was also developed to render realistic automations that visualize the crane motions on computers. To coordinate multiple cranes working simultaneously on a construction site, an effective computational method was also developed to avoid conflicts between cranes during the erection processes.