Abstract:

Retrieving the stylus of a pen-based device takes time and requires a second hand. Especially for short intermittent interactions many users therefore choose to use their bare fingers. Although convenient, this increases targeting times and error rates. We argue that the main reasons are the occlusion of the target by the user’s finger and ambiguity about which part of the finger defines the selection point. We propose a pointing technique we call Shift that is designed to address these issues. When the user touches the screen, Shift creates a callout showing a copy of the occluded screen area and places it in a non-occluded location. The callout also shows a pointer representing the selection point of the finger. Using this visual feedback, users guide the pointer into the target by moving their finger on the screen surface and commit the target acquisition by lifting the finger. Unlike existing techniques, Shift is only invoked when necessary over large targets no callout is created and users enjoy the full performance of an unaltered touch screen. We report the results of a user study showing that with Shift participants can select small targets with much lower error rates than an unaided touch screen and that Shift is faster than Offset Cursor for larger targets.

I.INTRODUCTION:

Many pen-based devices, such as personal digital assistants (PDAs), mobile phone-PDA hybrids, and ultra mobile personal computers (UMPCs) utilize sensing technologies that can track not only a stylus, but also touch input. This makes touch input an option when pen input is not possible, such as one-handed operation However, pen-based user interfaces often contain small dense targets, making selection with a finger

Slow and error prone. So what is it that users give up by not using the pen?

Figure 1.

(a) Small targets are occluded by a user’s finger.(b) The proposed Shift technique reveals occluded screen content in a callout displayed above the finger. This allows users to fine tune with take-off selection. (c) By adjusting the relative callout location, Shift handles targets anywhere on the screen

While fingers are somewhat less accurate than pens in terms of fine control, accuracy is not the primary reason for the high error rate. In our observation, the main reason is the ambiguous selection point created by the finger’s contact area in combination with the occlusion of the target. When selecting targets smaller than the size of the finger contact area, users start having difficulty determining whether or no they have acquired the target. Unfortunately, targets smaller than the finger’s contact area are also occluded by the finger, preventing users from seeing visual feedback. Consequently, applying a technique to enhance accuracy will not solve the problem. Manipulating control display (CD) ratio or offering insitu zooming enhance accuracy, but they do not address occlusion directly or define a clear selection point. Occlusion and selection point ambiguity can be addressed with the Offset Cursor. The Offset Cursor creates a software pointer a fixed distance above the finger’s contact point. The Offset Cursor uses take-off selection in which the target is selected at the point where the finger is lifted rather than where it first contacted the screen. This allows users to touch the screen anywhere and then drag the pointer into the target. Shift is a technique created to address the problems with using finger touch on pen-based interfaces Shift offsets the screen content to avoid all three drawbacks of Offset Cursor and leads to significantly better targeting performance.

II.WHAT IS SHIFT?

Shift technique in two scenarios.

A.Scenario-1:

(a) The user touches the screen intending to acquire a small target located near other targets. Shift determines the presence of targets small enough to be occluded by the finger (b) In order to eliminate occlusion, Shift “escalates” by creating a callout that contains a copy of the occluded screen area placed in a non-occluded location on the screen. Similar to Offset Cursor, the callout includes a pointer representing the finger contact point to eliminate selection point ambiguity (c) The user fine-tunes the pointer position

While maintaining contact with the screen; (d) once the correct position is visually verified, lifting the finger causes a brief Phosphor afterglow and completes the selection.

B.Scenario-2:

When acquiring a large target, Shift behaves differently. Occlusion is not a problem in this case, so Shift does not escalate by default. By lifting their finger immediately, the user makes the selection as if using an unaided touch screen. Shift avoids the three drawbacks of Offset Cursor: 1) Shift requires interaction overhead only when really necessary, for small targets. This conditional escalation results in a significant speed-up Conditional escalation is a property unique to Shift. If applied to Offset Cursor, users would have to perform additional movements and automatic escalation could not be determined in some cases. 2) Shift does not result in any inaccessible screen areas. While the callout’s default position is above the target, it can be placed elsewhere to prevent clipping by display edges 3) Shift behaves as touch screen users expect: it allows users to aim for the target itself. This enables walk-up scenarios. In the worst case where a user ignores the callout, Shift is no worse than a standard touch screen.

III.SPECIALIZING THE USE INTERFACE FOR FINGRES

Maximum usability can be achieved with user interfaces designed directly for the capabilities of the finger. Such user interfaces typically avoid small targets in the first place. Karlson et al. explore this strategy with AppLens and Launch Tile, two thumb-specialized designs for PDA application shells. However, creating input-specialized versions of applications is expensive and will often be difficult to do for legacy applications. Also, if both pen and finger input are used intermittently, then ideally two sets of input specialized interfaces would need to be available

IV.AVOIDING OCCLUSION

Although a touch pointer makes the offset visible, it also makes pointing a compound task: first acquire the handle, drag it to the desired location, then fine-tune and tap to make the actual selection. Another issue is that it occupies permanent space on the display, making the touch pointer less suitable for small screen devices. Some applications which create multiple views of the same content for navigation or accessibility could be repurposed to address occlusion. For example, the Microsoft Windows Magnifier Tool creates an additional view of the area around the pointer. However, this permanently occupies screen space at a fixed position, unlike Shift which uses conditional escalation to place a second view near the finger. Recently, researchers have explored using the underside of an interactive table to address occlusion. However, this approach requires specialized hardware and usability with a hand-held device was not evaluated.

V.DESIGN

In order to guide the design process, we created a model of Shift’s expected targeting performance. This model is the basis for all our hypotheses and it guided us through several rounds of pilot studies. We formulated our hypotheses within the context of Offset Cursor and unaided touch screen selection. The simplicity of unaided touch screen input makes it fast across all targets

Sizes, but there is an approximate threshold size where occlusion makes selecting smaller targets error prone. We call this the occlusion threshold. Offset Cursor avoids these problems by offering a defined selection point and avoiding occlusion. In exchange, however, users spend additional time estimating the offset distance and fine-tuning their selection position. During pilot testing we were surprised to observe that the time loss was not limited to small targets where occlusion was a problem, but affected all target sizes we tested including targets as large as 41mm. We discuss this in detail in the experiment section. We hypothesized that Shift performance should differ depending on target size. For targets smaller than the occlusion threshold, Shift should perform roughly the same as Offset Cursor since both offer improved accuracy at the expense of additional user effort. However, we did not know how long Shift’s visual reorientation step would take in comparison to Offset Cursor’s distance estimation step. For large targets, however, we expected a clear performance benefit for Shift over Offset Cursor since without escalation Shift works like an unmodified touch screen.

VI.TASK AND STIMULI.

Figure 2. Apparatus: participants acquired targets on an IPAQ 4100 PDA using their index finger.

Participants were presented with a series of individual target selection trials. Six different target sizes were used, with each target positioned a constant distance away at four different angles. Participants were instructed to acquire these targets as quickly and accurately as possible. Participants acquired targets with the index finger of their dominant hand while holding the device in their nondominant hand (Figure 2). An earlier pilot study had found similar patterns of performance between one-handed thumb targeting and two-handed finger use (except that the thumb condition showed a higher variance due to thumb ergonomic issues). We used separate conditions for Fingertip and Fingernail because our pilot studies had found that different participants preferred using their finger tip while others preferred fingernail. Figure 2, Apparatus: participants acquired targets on an IPAQ 4100 PDA using their index finger.

At the beginning of each trial, a solid red 48px (48 pixel by 48 pixel) start button was displayed along with the target, which was rendered as white with a red border (Figure 11a). The target was placed diagonally, 120px away from the start button. Targets were never placed at screen edges so all targets were reachable using Offset Cursor. Stimuli were displayed in front of a street map background for increased ecological validity. Participants selected the start button using the currently active technique condition. Once selected, the start button disappeared and the target turned solid red in figure 3(b).

Figure 3.(a) 3(b) 3(c)

Figure 3:

Experimental task stimuli: (a) red start button and white target with red border; (b) start button disappears when selected, target turns red; (c) visual feedback when over target

When the pointer was over the target, the target provided visual feedback by turning yellow (Figure 3(a)). The trial was completed when participants lifted their finger. Successful target acquisition was confirmed with a click sound; unsuccessful attempts resulted in an error sound. Since we expected the Touch condition to have high error rates with small targets, participants advanced to the next trial regardless of error. We recorded task times, errors, and all movement and escalation events.

VI.HIGH ACCURACY

SELECTION ENHANCEMENTS

As stated earlier, the purpose of Shift is to enable users to acquire targets by avoiding target occlusion, not necessarily enhancing targeting precision. Our study shows that the basic version of Shift described above allows for the acquisition of targets as small as 6 pixels (2.6 mm). Some situations, however, may require users to acquire targets smaller than that. Fortunately, Shift lends itself well to precision enhancements like zooming and control display (CD) ratio manipulation

Figure 4. Shift with zooming enhancement: (a) before finger contact; (b) after escalation with 4× magnification in callout

VII.CD ENHANCEMENT RATIO

Touch screens are typically operated with a CD ratio of 1 in which the pointer position is mapped 1:1 with the finger input position. Especially for systems that do not support a tracking state, a 1:1 mapping is important because it allows users to aim for a target. However, once the user’s finger is in contact with the screen, a pointer can be displayed providing users with visual feedback. Then, finger movement can control the pointer in a relative manner; with the pointer moving fasAs explained in the previous section, we position the callout to avoid occlusion with the finger. This is done regardless of the target’s original position. In some cases moving the finger makes the original target position no longer occluded. However, in a pilot user study, participants told us they always used the callout for visual feedback, even if they realized that the target in the main display was visibleter or slower than the finger directing it.

VIII.CONCLUSION

Shift enables the operation of a pen-based device, such as a PDA or UMPC, with fingers. Our experimental results show that Shift’s conditional escalation overcomes occlusion problems and allows users to select small targets reliably. Unlike Offset Cursor, Shift preserves the speed and simplicity of direct touch interaction for large targets. While the user study reported in this paper focused on these quantitative differences, another benefit mentioned earlier might have an even bigger impact on Shift’s deployment: By allowing users to aim for the actual target, Shift remains compatible with regular pen and touch input. This compatibility keeps the interaction consistent when switching back and forth between pen and touch. And, maybe most importantly, it makes it easy to deploy Shift in walk-up scenarios or to retrofit existing systems. We plan to study such scenarios as our future work.

REFERENCES:

[1]. Vogel, D. Balakrishnan, R. (2005). Distant freehand pointing and clicking on very large, high resolution displays.

[2]. Wigdor, D., Leigh, D., Forlines, C., Shipman, S., Barnwell, J., Balakrishnan, R., Shen, C. (2006). Under the Table Interaction.

Ø www.touch-screen.in

Ø www.touchusability.com

Ø www.extremetech.com