Enhancing User Experience in AI-powered Applications

The emergence of new technologies is on the rise

Technology, such as artificial intelligence, is on the rise and has a significant impact on the digital landscape. Artificial intelligence presents both exciting new opportunities and challenges for humans. As an interaction and UX designer, I saw an opportunity to learn and grow by exploring the technology that interests me. I strongly believe that UX designers should be aware of emerging technologies and new trends to ensure successful design for users. To create impactful designs, designers should have a deeper understanding of the technical aspects of AI, user behaviour, and the ethical implications of design decisions. This knowledge will empower us designers to be more adaptable and flexible in dealing with uncertainty and rapid change, ultimately leading to innovative and effective solutions.

Implementing artificial intelligence in products and services has increased significantly in recent years. AI tools, such as chatbots integrated into applications and websites, have become popular among companies to simplify work-related tasks and improve user experience. The emergence of AI-powered chatbots has significantly impacted businesses and individuals differently. As a result, there was a growing interest in understanding coexisting AI with Humans and how AI affects our lives and work.

The overarching discussion emerged around the implementation of AI in the work/design process, and considerations revolved around potential challenges within User Experience (UX) and Interaction Design (IxD). Another theme for discussion was the consequences of AI for the workforce and creation.

The selected focus for the study revolved around the impact of AI on user experience. Specifically, the research would explore the interaction between AI-powered chatbots and users and the ways of designing AI-powered applications that meet the users’ needs and preferences.

The challenge

Selecting a scientific problem within an unfamiliar domain

The challenge was to explore a new subject, emphasising in-depth investigation and problem analysis with broad appeal. The challenge wasn’t just about identifying a problem but also navigating through the academic terrain of the subject, understanding its nuances, appreciating the context, identifying gaps, and choosing the framework. The absence of prior expertise in the selected area added an element of uncertainty, but it also sparked curiosity and interest. Starting the scientific study in March, we were given two weeks to deliver a research plan, four weeks to conduct empirical research, and then to present an academic report.

My role

I worked on this independent project as an Interaction Designer, earning a Bachelor of Science degree at the Linnaeus University, Kalmar, Sweden.

Contribution

Leading research.
Literature review.
Conducting user (primary) research.
Identifying the problem areas and quality elements that impact the user experience.
Design principles were created to improve the user experience based on primary & secondary research.

Acknowledgements

I conducted this study in cooperation with Nicon, a Karlskrona-based company and developers of AI-powered applications.

My supervisor, Peter Adiels, who is a lecturer at Linnaeus University, provided me with valuable advice and guidance. It was very beneficial to discuss the scientific problem, research question, and design framework with him.

Together with fellow student Johanna Berntsson we had participatory discussions on the theme, research questions, and concept mapping for the project. We also collaborated on the choice of theoretical framework and methodology.

Timeline

Research: March-May 2023.
Presentation of the scientific report: August 2023.
Publication: September 2023. Download the complete report here.

Communication

This study, along with empirical research, involved participants situated in Sweden and was communicated in Swedish. Thus, the artefacts presented below are documented in Swedish.

Tools

Miro was used for collaborative concept mapping and affinity diagramming of the empirical result.
Google Docs was used to document the research.
Axure was used to create the final presentation of the design recommendations.

The scientific challenge

Decreasing positive perception and lack of knowledge regarding AI

Despite the majority of people acknowledging the benefits of AI, there has been a decreasing positive perception and usage of this technology (Svenska folket och AI, Svenska folkets attityder till Artificiell Intelligens 2022). This shift in attitude is marked by increased scepticism and caution towards both explicit and implicit applications of AI. The scientific challenge lies in understanding the reasons behind this shift and how AI-powered applications impact human interactions and decision-making processes. As technology becomes increasingly prevalent in users’ daily lives, there is a growing need for thoughtful and responsible design of digital services to address the new requirements and usability issues that AI creates.

According to many experts, we are only beginning to scratch the surface of what AI technology can do, and there is a limited understanding of this technology among a wider audience. Designers who specialize in human-computer interaction (HCI) and user experience (UX) play a crucial role in shaping the development of AI systems and eventually, HCAI (human-centered AI) systems. However, despite their importance, UX designers still lack the necessary knowledge and skills to fully capitalize on the potential of AI.

The approach

Explore the use of interfaces in AI-powered applications to identify factors that optimize human-centered experiences.

To address this challenge, the study aimed to contribute to the existing knowledge by focusing on the specific aspect of user experience in the interaction with AI-powered chatbots within digital interfaces. The approach involved identifying and analyzing fundamental factors that influence the use of interfaces in AI-based applications, with a particular emphasis on optimizing the user experience. The study adopted a human-centered perspective, acknowledging the importance of responsible and thoughtful design in the era of increasing AI implementation in everyday life.

The study’s purpose was to enhance understanding of the user experience aspects related to chatbot interactions and AI applications in interface design. By identifying these aspects, the goal was to formulate relevant recommendations for chatbot design that can support future efforts in developing user-friendly applications based on artificial intelligence.

Deep in the Literature

I delved into previous research within the study domain of artificial intelligence and chatbots, which included studying the background and historical development, as well as user experience and usability, aiming to cultivate a comprehensive understanding of the subject. The literature review played a crucial role in shaping a comprehensive understanding of the studied area and identifying critical aspects to consider when designing user experiences.

This process provided me with valuable insights into the subject of artificial intelligence and interacting with AI chatbots. Articles were sourced through searches on Google Scholar and Scopus databases. Selection criteria focused on finding conference publications and scientific articles published between 2019 and 2023. I also used other sources, such as research reports, library books and expert talks at the conferences.

I reviewed 49 articles and narrowed them down to 11. I then extracted quality attributes that impact the user experience when interacting with AI-powered chatbots. A set of quality attributes are documented in the table below. Following this initial phase, I defined overarching themes and pinpointed specific problem areas. This qualitative exploration aimed to unravel the nuanced dynamics between users and AI systems.

Insights

The problem areas that impact the user experience, either positively or negatively

Communication • Control • Transparency • Presentation • Accessibility

Primary Research

Speak to the people

The next step was to move from the theoretical approach of broad conceptual themes to more specific variables. Real users were involved in order conducting research associated with AI-powered chatbots and user interactions.

The study involved seven participants with varying experiences and interests in chatbot usage. A limitation of seven participants was set for the primary research to ensure manageability and maintain the quality of the study.

Five participants tested AI-powered applications through a digital interface, and in-depth interviews were conducted at the end of the study. The age limit of participants was set at 20 years because an app tested was a wine- and whiskey-tasting app, among other articles.

Additionally, two representatives of an IT company participated in the study who provided the test environment and who could influence the product outcome. In-depth interviews were conducted in order to expand knowledge in the area of AI-driven services with the help of those participants’ expertise.

An overview of the AI-powered apps that were evaluated

Whizper is a mobile application that uses AI technology to allow users to create event invitations and upload images. More features are planned for the app as it’s still in the early development phase. During this phase, a hi-fi prototype was used for research purposes. Some of the features tested included creating invitations and chatting with the bot. “Whizper” is a product developed by Nicon, a company based in Karlskrona, Sweden.

Bing Chat launched in February 2023 and it is an AI-powered language model integrated into Microsoft’s search engine. It can be accessed through the Microsoft Edge browser or the Bing application. This product enables users to receive relevant information, cite sources, and chat with the bot in a conversational format of their choice. Additionally, users can customize the tone and style of their conversations with the bot.

Research Framework

To conduct in-depth research on the selected subject and identify the factors that affect users and their interaction with AI-powered interfaces, we have selected Nielsen’s Usability Heuristics for User Interface Design for conducting a study with real users. Nielsen’s evaluation checklist is widely used and effective in the field of interaction design, particularly within the user-centered design framework. These recommendations were originally developed by Molich and Nielsen in 1990. Then Nielsen developed these recommendations to ten heuristics that can be applied across different types of interfaces, as he says. I have personally used these heuristics many times in product evaluations during my previous projects.

Furthermore, we have used Microsoft’s Guidelines for Human-AI Interaction, which I discovered during my preliminary (literature) research. The combination of these two sets proved to be beneficial as they complemented each other. The guidelines were created by experts to help designers and non-designers in developing more responsible products. The comprehensive framework of the guidelines ensures a good user experience.

The frameworks were used to develop end-user interview questions and then to analyze their responses.

Analysing research findings

Usability study findings

Result

Twelve qualitative factors that impact the user experience were identified within four problem areas: communication, human control, transparency in system functions, and visual presentation.

The study suggests recommendations considering these factors that can increase user satisfaction and contribute to future work in developing applications based on artificial intelligence. Follow the link to read the recommendations. The whole rapport of this study you can read by following this link. Both of the documents are written in Swedish.