Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> Data visualization is a technique used to explore real or simulated data by representing it in a form more suitable for comprehension. This form is usually visual since vision provides a means to perceive large quantities of spatial information quickly. However, people who are blind or visually impaired must rely on other senses to accomplish this perception. Haptic interface technology makes digital information tangible, which can provide an additional medium for data exploration and analysis. Unfortunately, the amount of information that can be perceived through a haptic interface is considerably less than that which can be perceived through vision, so a haptic environment must be enhanced to aid the comprehension of the display. This enhancement includes speech output and the addition of object properties such as friction and texture. Textures are generated which can be modified according to a characteristic or property of the object to which it is applied. For example, textures can be used as an analog to color in graphical displays to highlight variations in data. Taking all of these factors into account, methods for representing various forms of data are presented here with the goal of providing a haptic visualization system without the need for a visual component. The data forms considered include one-, two-, and three-dimensional (1-D, 2-D, and 3-D) data which can be rendered using points, lines, surfaces, or vector fields similar to traditional graphical displays. The end result is a system for the haptic display of these common data sets which is accessible for people with visual impairments. <s> BIB001 </s> Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> Mental mapping of spaces, and of the possible paths for navigating through these spaces, is essential for the development of efficient orientation and mobility skills. The work reported here is based on the assumption that the supply of appropriate spatial information through compensatory channels (conceptual and perceptual), may contribute to the blind people’s spatial performance. We developed a multi-sensory virtual environment simulating real-life spaces. This virtual environment comprises developer / teacher mode and learning mode. 1. RATIONALE The ability to navigate space independently, safely and efficiently is a combined product of motor, sensory and cognitive skills. This ability has direct influence in the individuals’ quality of life. Mental mapping of spaces, and of the possible paths for navigating through these spaces, is essential for the development of efficient orientation and mobility skills. Most of the information required for this mental mapping is visual information (Lynch, 1960). Blind people lack this crucial information, thus facing great difficulties (a) in generating efficient mental maps of spaces, and therefore (b) in navigating efficiently within these spaces. A result of this deficit in navigational capability is that many blind people become passive persons, depending on others for continuous aid (Foulke, 1971). More then 30% of the blind do not mobilize independently outdoors (Clark-Carter, Heyes & Howarth, 1986). The work reported here is based on the assumption that the supply of appropriate spatial information through compensatory sensorial channels, as an alternative to the (impaired) visual channel, may contribute to the mental mapping of spaces and consequently, to blind people’s spatial performance. Research on blind people’s mobility in known and unknown spaces (Dodds, Armstrong & Shingledecker, 1981; Golledge, Klatzky & Loomis, 1996; Ungar, Blades & Spencer, 1996), indicates that support for the acquisition of spatial mapping and orientation skills should be supplied at two main levels: perceptual and conceptual levels. At the perceptual level, the deficiency in the visual channel should be compensated with information perceived via other senses. Touch and hearing become powerful information suppliers about known as well as unknown environments. In addition, haptic information appears to be essential for appropriate spatial performance. Haptics is defined in the Webster dictionary (1993), as “of, or relating to the sense of touch”. Fritz, Way & Barner (1996) define haptics as “ tactile refers to the sense of touch, while the broader haptics encompasses touch as well as kinaesthetic information, or a sense of position, motion and force.” Haptic information is commonly supplied by the cane for low-resolution scanning of the immediate surroundings, by palms and fingers for fine recognition of objects’ form, textures, and location, and by the legs regarding surface information. The auditory channel supplies complementary information about events, the presence of other people (or machines or animals) in the environment, materials which objects are made of, or estimates of distances within a space (Hill, Rieser, Hill, Halpin & Halpin, 1993). At the conceptual level, the focus is on appropriate strategies for an efficient mapping of the space and the generation of navigation paths. Research indicates two main scanning strategies used by people: route and map strategies. Route strategies are based in linear (therefore sequential) recognition of spatial features. Map strategies, considered to be more efficient than the former, are holistic in nature, comprising multiple perspectives of the target space (Fletcher, 1980; Kitchin & Jacobson, 1997). Research shows that blind people use mainly route strategies while recognizing and navigating new spaces (Fletcher, 1980). <s> BIB002 </s> Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> The paper describes the HOMERE system: a multimodal system dedicated to visually impaired people to explore and navigate inside virtual environments. The system addresses three main applications: preparation for the visit of an existing site, training for the use of a blind cane, and ludic exploration of virtual worlds. The HOMERE system provides the user with different sensations when navigating inside a virtual world: a force feedback corresponding to the manipulation of a virtual blind cane, a thermal feedback corresponding to the simulation of a virtual sun, and an auditory feedback in spatialized conditions corresponding to the ambient atmosphere and specific events in the simulation. A visual feedback of the scene is also provided to enable sighted people to follow the navigation of the main user. HOMERE has been tested by several visually impaired people who were all confident about the potential of this prototype. <s> BIB003 </s> Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> Stimulating the small forces when objects slide over the textured surface enriches the interaction between a user and a virtual world. A major design constraint for haptic textures is the generation of a sufficiently realistic texture given hard constraints on computational costs. A haptic texture presentation method based on the 3-DOF DELTA device was proposed. The height map of the texture surface was transformed from the image data with GAUSS filters, and the texture force and friction were calculated based on the geometry data. The simulated texture force was applied to the user when he/shi exploring the virtual surface with DELTA haptic device. The preliminary psychophysical experiments were performed to evaluate the quality of presentation. <s> BIB004 </s> Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> We introduce a force-field haptic rendering method for converting visual data to haptic data. The method includes a novel framework to convert specialized 3D map models into force fields. We generate the final force fields by using structure from motion and implicit surface approximation algorithms. Visually impaired people then can learn to navigate with these maps, using off-the-shelf haptic <s> BIB005 </s> Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> Haptic data visualization is a growing research area. It conveys information using the sense of touch which can help visually impaired people or be useful when other modalities are not available. However, as haptic devices and virtual worlds exhibit many challenges, the haptic interactions developed are often simple and limited to navigation tasks, preferring other modalities to relay detailed information. One of the principal challenges of navigation with haptic devices alone, particularly single point-based force-feedback devices, is that users do not know where to explore and thus obtaining an overview is difficult. Thus, this paper presents two types of interaction technique that aim to help the user get an overview of data: 1) a haptic scatter plot, which has not been investigated to any great degree, provided by a force model and 2) a new implementation for a haptic line chart technique provided using a guided tour model. <s> BIB006 </s> Virtual Reality and Assistive Technologies: A Survey <s> Haptic VEs for visual impairments <s> To people who are visually impaired, sensory substitution requires conversion from distal visual data to proximal haptic cue. The challenge is to identify the optimum haptic cues that will make a common haptic "language", which can eventually be used to convey information about facial expressions to the individuals who are blind. Once this language has been developed, the next challenge is learning and training. We propose a dynamic haptic tool for simultaneous learning and interaction with a proposed haptic language. We call this application "Haptic mirror", an interactive haptic-based reflection of one's self which can be comparable to a visual mirror. Once the user has acquired this language, it can be used to convey the facial expressions of social peers. <s> BIB007
A haptic VE can work as an effective learning platform for the blind and visually impaired. This can also enhance social interaction capabilities of the blind through a better understanding and evaluation of the surroundings (Yasmin and Panchanathan 2016). The conventional method of teaching blind students is through raised line drawings, cutout tiles, or hardboards with complex figures, or cross-sections embossed. VEs proved to be more flexible and scalable. In a VE, a haptic line chart was initially presented by a haptic cylinder or embossed ridge to feel with force feedback haptic devices provided by 3D Systems (Yu, Ramloll, and Brewster 1999; 2000; Roberts, Franklin, and Cullinane 2002). Later, attraction forces were used to keep the user on the line  . When the Haptic Interaction Point (HIP) was close enough to the line, it was pulled towards the line. Once on the line, no forces were generated to resist the motion of the HIP along the line, but motion away from the line was opposed and a relatively large force was required while attempting to pull away from the line. This helped the blind user to find and trace the line in a tangible line chart. The blind user could haptically explore some points of interest, i.e., maximum, minimum, and inflexion points on the graph. While haptic navigation, non-speech audio demonstrated the overview of the data set by mapping MIDI notes pitch to the Y-axis values while X-axis was mapped to time. Synthesized speech was used to provide the user with exact coordinate values in the chart. Figure 13 displays a variety of haptic bars, charts, and so on. Yu et al. proposed that engraved lines work better than an embossed ridge as this prevent the user from falling off the surface while navigating in non-constrained mode (Figure 13a)  . Like line graphs, different bars or sections of a pie chart were also explored for haptic data representation as demonstrated by Figure 13b and Figure 13c respectively. Engraved bars enabled the user to stick to the bar and prevented slip-off  . In haptic bar charts, audio pitch corresponded to the Y value of the top of the bar. Haptic pie charts were built with resistive walls to form an enclosure for each segment, and a groove around the edge was used to provide an overview of the information  . In haptic pie charts, pitch value was made proportional to the length of the arc. Different frictional forces helped to haptically distinguish different lines in a line chart, different bars in a bar chart, and sections in a pie chart. Panels et al.worked on haptic-based representation of scatter plot for the blind BIB006 ). In a scatter plot, data is displayed as a collection of points to understand the relationship between the variables through the size and location of the point cloud, the direction of the relationship, and whether outliers exist. A haptic scatter plot was modeled by assigning a repulsive force to each point. While exploring the plot, a greater force was felt for larger concentrations of data points (Figure 13d) . In a haptic representation of 3D surface plot, polygons or implicit functions were used for collision detection BIB001 . In a haptic-based representation, variation in data was also represented via different surface properties, i.e., surface friction, stiffness, texture, viscosity, and so on BIB004 . The higher the surface friction, the higher resistive force is felt by the user during haptic navigation. Surface textures are usually generated by extracting data height from image processing, perturbing surface normals, creating noise in the data set, and more ( Figure 13e ). Haptic texturing was used for a representation of data variables or different regions, in the same fashion as color is used in a vision-based representation BIB001 Moustakas et al. introduced a force-field haptic rendering method of 3D maps to let the visually impaired navigate these maps using off-the-shelf haptic devices BIB005 ). Users were found to be enthusiastic about the system and later, asked for options to be able to zoom in on to specific areas of the map for further investigation. Lahav et al. concluded that blind peoples spatial performance can be enhanced if appropriate spatial information is provided through compensatory channels BIB002 . In their experiment, blind individuals were asked to haptically explore a VE that simulated real room with objects. Following the exploration task, they were asked to give a verbal description of the environment and construct a scale model of it. Haptic feedback and haptic exploration enabled and helped the blind individuals develop a highly accurate cognitive map of the environment. Parente et al. proposed Blind Audio Tactile System (BATS) where geographical information were presented with different haptics-based feedback: subtle bumps for country boundaries, large bumps for state boundaries, and a constant vibration on cities (Parente and Bishop 2003). Speech had been used for objects names and descriptions and also for directions. Non-speech auditory icons had been added to provide surrounding information: traffic sounds for cities, bird chirping for forests, and so on  . Lecuyer et al. designed a haptic and audio-based multimodal application called HOMERE to explore and recognize the environment to prepare the blind for an independent visit to an existing site in a real world (e.g., tourist site, administrative building, airport, or train station) by ludic exploration in the virtual world BIB003 . As demonstrated in Figure 14a , the user could detect obsta-cles and feel the textures on the ground or on some objects in the VE with a virtual cane. In a HOMERE system, thermal feedback corresponded to the simulation of a virtual sun while an auditory feedback corresponded to the ambient atmosphere or oral guidance from a virtual avatar. A visual feedback of the scene was also provided for the sighted user. HOMERE has been tested by several visually impaired people. All were found to be enthusiastic and confident about the potential of the developed prototype. Nikolakis et al. used 3D Systems Touch X haptic device in combination with Cybergrasp and Cyberglove to develop a customized VR set up with full five-finger haptic feedback  ). The haptic device allowed the visually impaired to explore details of the object, while the CyberGrasp helped the user perceive the general shape and size of the object (Figure 14b ). The single interaction point of the haptic device provided force feedback onto the index finger. With the help of CyberGlove, the positions of all finger tips were determined and the positional data of the fingertips were used for collision detection between the fingers and the virtual object. Next, output force for all finger tips were generated. In the application, an audio-haptic menu assisted the user to select an object. The selected object declaimed itself via audio feedback. Next, the user could explore the object haptically while listening to information about it. Yasmin et al. developed a haptic mirror to enrich the abilities, lives, and self-sufficiency of blind individuals in interpreting different facial expressions and assist in understanding non-verbal cues of interaction partners in social communication BIB007 . It was hypothesized that persistent haptic exploration of different movements of ones own facial features for different expressions in the VE would help a blind individual master the haptic language of facial expressions. Now, when others facial data would be accessed through the VE, the blind would be able to interpret others facial expressions. At first, facial data were collected via a webcam. Next, extracted features were normalized, enhanced, and haptically augmented. Figure 14c demonstrates the proof-of-concept Haptic mirror prototype in use by a blind participant. The preliminary findings convincingly supported the idea and social relevance of the described system, its necessity, and acceptability by the target users.