Study Of User Experience And Ux Evaluation

Published Date: 02 Nov 2017

Abstract - The goal of user experience analysis and evaluation methods in the software industry is to improve customer satisfaction through the utility, ease of use, and quality of interaction with applications, services and products. So far user experience deals more with evaluation of short term goals and consequently on facets relating to the initial adoption of new product designs. Nevertheless the user and the product evolve over long duration of time and thus new methods need to be found to perform UX evaluation. Today’s applications such as ubiquitous systems are more and more aware of user’s habits and the purpose of the user needs. The features of products and the context of use will affect the human’s experiences and preferences about the use of device. Thus, user experience in user-product interaction needs careful attention and can be seen as a prime area of focus.

Keywords- User experience evaluation, Product development process, Repertory Grid Technique, UX Curve Technique, User satisfaction.

I.INTRODUCTION

User Experience (UX) is a multidimensional concept and a commonly accepted definition is still in the making. As Hassenzal andTractinsky (2006) argue the concept of user experience attempts to work function beyond the limitations of HCI by bringing out relevant aspects such as beauty, fun, pleasure and personal growth that satisfy general human needs but have little key values. User experience (UX) evaluation is relatively easy with existing products that people have been using in their daily lives for several months. It is more challenging to evaluate product experiences earlier on, when they are just prototypes or even concepts on paper. Still, we need to take care that each product will allow the intended experiences before the product is on the market. The earlier we can evaluate user experience the more likely it is that a product will be successful [1].

However, there are several difficulties to overcome. The first impression is often very different from the long term user experience, but we cannot run long-term field studies with concepts that do not actually work. We cannot evaluate user experience in the real context when the system is just an idea on the paper. We cannot see how the different parts of the system will create a holistic user experience when we have an idea of one new feature only. It is hard to run iterative evaluations when the target users are on a different continent than the development. It may be difficult to agree a common goal with external stakeholders who influence the UX. Recently, HCI researchers have begun exploring the relationship between user and product Like for instance its affective qualities rather than efficiency; experiences rather than performance; fun and playability rather than error rate; and sociability and affective qualities rather than learn-ability. Thus we find that effective and powerful UX evaluating techniques our capable of evolving over long periods of time and it is imperative to find or develop appropriate and mature procedures for gathering and analyzing empirical data in relation to these new, experience and meaning-related aspects of interacting with computers. In this paper, discuss, and empirically demonstrate a techniques that could become a useful tool for dealing with the ‘new HCI’ such as the Repertory Grid Technique (RGT), UX Curve etc [2].

II. PROCESS OVERVIEW

This section talks about the approaches that we must take while adopting or creating new UX evaluation techniques. This section consists of two principal strategies: UX target settings and UX evolution.

The process commences by examining user requirements and psyche. We innovate new concepts by reflecting user data with market insights and technological opportunities. UX improvement is continuous process of identifying problems, gaps and new ideas. We do this in all phases of the process, by analyzing the UX evaluation results for concepts being developed, field feedback for existing products, and competitive trends on the market, for example. UX improvement topics are then created and followed-up via systematic analysis and applied in the development. The practical challenge is to link the available ideas to right development activities at the right time

UX improvement process is, first of all, UX management activity. Fundamentally, here we apply conceptual-analytical research and data gathering techniques. The need is to use and continuously develop methods to evaluate usability and UX in the different phases of development. After the product is released we gather feedback from the field via many controlled and uncontrolled group channels. This information can be vital in changing Business Intelligence and can be used for further improving the forthcoming products [3].

III. REQUIREMENTS FOR UX EVALUATION

We have identified a set of properties that UX evaluation methods should have in order to be applicable in industrial settings. If a method fails to fulfil these criterions, it will be hard to integrate it into the processes to be used routinely.

First, the evaluation method needs to be lightweight. In global companies, there is a requirement to make data collection from different parts of the world both physical and virtual easy. Tens and thousands of new products applications and services are flooding the market each and every year and they need to be evaluated against their respective competitor’s product to provide benchmarks. This means evaluation results are needed from the get go.

Second, the theory behind UX evaluation needs to be applicable for various types of products and prototypes. If the evaluations are tied to certain product features, we have to define that set for each product or service individually. A general set of UX elements makes it easier to evaluate different products of different categories and to utilize the same elements in different phases of Software Development Life Cycle. Although we would not find any actual problem sources with these general UX elements, they help us to see trends and to provide a point of reference to diverse solutions.

Third, we do UX evaluations in order to improve the products. We need to find out the pros and cons of the evaluated system. Some UX evaluations aim for finding out the exact emotions that users have at the moment, or whether they see a product/service/application as masculine or feminine.

Finally, we might want to relate the project incentives to outstanding user experience, not only to technological worth. This would require a fair UX evaluation procedure that produces some kind of a UX score. Fairness means that all different types of products, with different target user groups, would be on the same line.

IV. USER EXPERIENCE AND RESEARCH METHODS

Basically, user experience refers to the experience that an individual gets when he/she interacts with a product in particular environments subject to certain conditions. In practice, there are numerous different kinds of people, products and environment s that influence the experience that interaction evokes (Figure 1). The user and the product interact in the particular context of use that social and cultural factors are influencing. The user has the following aspects: values, emotions, expectations and prior experiences, among others. Also, the product has influential factors, for example, mobility and adaptivity. All these factors influence the experience that user- product interaction evokes. [4, 8, 9]

Moreover, in order to investigate user-product interaction, researchers need to determine the nature of a product. The type of the product will affect the research methods and targets. For example, user experience studies of web sites [5] emphasize visual issues whereas research of hand-held devices needs more attention on issues such as size, weight and mobility. In addition, the target use group needs to be defined before developing or testing prototypes; for instance, if the device will be put to public use and the users are not very familiar with computers, the interface should be simple and clear [3].

Figure 1.User experience forms in interaction with user and product in the particular context including social and cultural factors

There are several methods in the user experience research area that have been used for capturing experiences, for instance interviews, observation, surveys, diaries, storytelling and prototyping [6]. In long-term use, surveys, diaries [7] and storytelling have been regarded as an effective way to get information about user experience. That is because the user can express some of his/her experiences in a written form. Stories are ways to organize and remember experiences and they enable humans to communicate experiences in different situations to the particular people involved [4]. The method allows designers, clients or users to "experience it themselves" rather than just witness a demonstration of someone else’s experience. [10]

Ubiquitous environments bring new aspects to user experience research. One reason for that is that environments and systems, according to Mark Weiser’s vision [1991], should be invisible to the user; however, it should be possible to evaluate the interaction with the system.

V. USER EXPERIENCE EVALUATION TECHNIQUES

There are several techniques that can be used to evaluate user experience the current available data shows the count for these techniques stands at around 82 classified methods ranging from simple to highly complex techniques that utilize capable inbuilt algorithms and structure to evaluate UX. In this paper two of these methods RGT (Repertory Grid Technique) and UX Curve will be discussed at length. These two methods allow accumulation of data and important statistical value and offer two varied approaches to the problem of finding an effective UX evaluation methodology.

VI. REPERTORY GRID TECHNIQUE

RGT is a technique for empirically eliciting and evaluating people’s subjective experiences of interacting with technology. It may be regarded as a methodological extension of Kelly’s Personal Construct Theory [11]. While it is not necessary to fully buy into the underlying theory to use RGT in practice, Kelly argued that we make sense of our world through our own ‘construing’ of it. That is, we tend to model what we find in the world according to a number of personal constructs, which are bipolar in nature. According to Kelly, we judge for instance other people through forming construct such as Tall—Short, Light— Heavy, Handsome—Ugly, and so on. A ‘construct’ is hence a single dimension of meaning for a person allowing two phenomena to be seen as similar and thereby as different from a third [1]. Kelly suggested RGT as a structured procedure for eliciting a repertoire of these conceptual constructs and for investigating and exploring their structure and interrelations [12, 13, 14].

A repertory grid in itself is the outcome of a successful application of the technique. It is a table, a matrix, whose rows contain constructs and whose columns represent elements, i.e. the phenomena under investigation. Repertory grids also typically embody a rating system used to quantitatively relate each element in relation to the qualitative constructs. An individual repertory grid table is constructed for each subject participating in a RGT study.

First, an individual participating in an elicitation session produces his or her own constructs, i.e. what bipolar dimensions of meaning the person see as the most important ones for talking about the elements (the investigated phenomena).

Second, after having provided their own unique inputs in the form of acceptable design pattern logic/construct, the participant is then made to provide a rating value for the extent to which each aspect in the study pertains to each bipolar construct according to some kind of measurement scale. Hence in RGT methodology the keywords are constructs and elements serve as the linchpin behind every grid in this approach. The constructs are a measurement of expression as the participating individual uses them as links to explain their connection to related links constructs thus serve as joints or links that bridge the gap between connotation and familiarity in connection to the given situation’s elements.

Despite its popularity in fields such as Management Information Systems, education, psychology and in the development of security systems and knowledge integration systems the interest in it from an HCI point of view did gain prominence in the 80s with a dedicated issue based solely on the topic in the International Journal of Man-Machine Studies (Vol. 13,No. 1,1980). But since then RGT’s impact in HCI related field has been limited but not gone unnoticed [2, 15].

VII. UX Curve

UX Curve method has been developed to provide assistance to users in retrospectively reporting how and why their experience with a product or a service or an application has changed over time. In this method the candidate draws one or more curves to describe how the experience about a product has changed over time thereby noting down all and every factor that vary with respect to time. The curve drawing area is formed of a timeline and a horizontal line that divides positive and negative experiences. Additional horizontal and vertical lines can be used as more precise scales for the quality of experience and for time periods. UX Curve is a pen-and-paper method, but there is a related tool called iScale, described elsewhere in this methods set.

The final purpose was to develop a simple and effective method for evaluating long term UX. The interesting part being the identification of the chronological order of experiences as this aspect has been shown in psychological studies to not only affect users’ overall evaluations [16], but also to predict later behaviour [17].

Figure 2. A simple UX Curve

The above curve is but a simple UX curve generated to showcase the simplicity involved in generating UX curve and how it helps in evaluating UX

Figure 3 This is a more complex UX curve which is related to monitoring UX over a variety of parameters.

VIII. Conclusion

The purpose of this paper was to define how user experience can be evaluated for various needs and scenarios. In general, the capturing of user experience is quite difficult, because there are so many different factors in user -product interaction (Figure 1). For the evaluation, those factors should be clarified and a goal for the test defined in a test plan. This may help make the evaluation more systematic.

The techniques namely RGT and UX Curve illustrate the fact that constructs, graphs and elements are appropriate tools for capturing user experience. However, this study confirmed that however these methods maybe effective they do have strengths and weaknesses thus there is a need to develop more methods to be used in order to evaluate user experience. In addition to the constructs and graphs, researchers will need more efficient ways to get information about the user’s emotions and experiences, concerning for example collection and interpretation of body gestures and facial expressions. In order to collect authentic emotions, the test situation should be organized so that is as natural as possible. As part of future research, more user experience evaluations will be done for different scenarios, using different methods.