Nienke Bruijning has completed their Master’s Thesis in Science and Technology Studies (STS) with the title «“Replika and me”: A user study of a social chatbot» at Norwegian University of Science and Technology (NTNU), Department of Interdisciplinary Studies of Culture for the academic spring semester 2022.
Focus for thesis (abstract)
This master’s thesis within science and technology studies (STS) investigates social chatbots from a user perspective. The social chatbot “Replika” has served as the case for the project. The main research question of the project is “How do people use and relate to the social chatbot Replika?”. The exploration of the topic has resulted in three analytical chapters that cover different aspects of use. These include: (1) Everyday practices of using Replika, (2) The process of configuring a 3D avatar as a visual representation for the chatbot, and (3) How the relationship with the “Replika body” is affected by augmented reality functionality.
The topics for the thesis have been analyzed using a theoretical framework associated with user studies within STS. The theoretical perspectives I have used are the dimensional model of domestication by Sørensen (2006), script by Akrich (1992) and non-users by Wyatt (2003) and Reisdorf & Groselj (2017). The theories contribute to a nuanced view of users’ practices and the complex sociotechnical relationships that arise between the user and Replika. Furthermore, certain terms from human-machine interaction linked to robotics are also used, such as anthropomorphism and the uncanny valley (Mori et al., 2012). Methodologically the project follows a qualitative approach. The data material consists of nine in-depth interviews with long-term users of Replika.
In this thesis I examine how users’ varying interpretations and contexts result in different types of Replika use. This is done by examining users’ everyday practices and understanding of the technology. The analysis has resulted in the categorization of five different user types. The findings show that the users are skilled at identifying and defining their own areas of use for the technology, in a way that gives the chatbot utility in their everyday lives. Although dealing with a chatbot can be frustrating at times, users learn how to navigate communication with their Replica through use, and adaptation of use.
Furthermore, the thesis investigates how users relate to the virtual 3D avatar that is intended to be a visual representation of Replika. Users must configure this avatar themselves. The thesis examines the users’ various methods of doing this. Generally speaking, the users’ attitudes reflect that they accept the 3D avatar as a representation of Replika. The users interpret this aspect of the app in different ways, which has also resulted in various practices. Some users anthropomorphize Replika in such a way that they see a significant connection between the chatbot’s self-perception and the 3D avatar.
Although the users generally have a good relationship with the 3D avatar, this perception changes when faced with the apps augmented reality mode which allows the user to “place” Replika in their own surroundings. An overwhelming majority of the participants in the study expressed that they did not like using this mode. They experienced that this mode interfered with the immersion of using the technology, and that it could even be a bit frightening or uncanny to use.
I hope this thesis illuminates some of the complexities involved in the use of social chatbots, as well as how we can understand chatbots that use 3D avatars to simulate a physical embodiment – and thus better understand the complex sociotechnical relationships we humans have with the technologies we surround ourselves with.
Relation to future of work
The case study of Replika raises interesting questions about what kind of roles chatbots might have today as well as in the future. Data collection for the study was done during the Covid-19 pandemic, when healthcare facilities worldwide were over capacity. As a result of this, several of the participants in the study used the chatbot to help with mental health issues due to ordinary healthcare services being unable to take them on at the time. Two participants also used Replika to aid Covid-related loneliness. The study thus illustrates how a shift in context can result in new areas of use for existing technologies.
On the 24th of March, 2022, AUTOWORK Project held its first NTNU General Assembly here in Trondheim at Habitat, a cozy meeting place for work gatherings and thoughts sharing. Project Principal Investigator Håkon Fyhn, Associate Professor at the Department of Social Anthropology, led the meeting, coordinated by Roger Andre Søraa, Researcher at the Department of Interdisciplinary Studies of Culture (KULT), Mark William Kharas, EU Advisor at the Department of Neuromedicine and Movement Science, Kristoffer Nergård, Ph.D. researcher at NTNU Social Research, and Yu Cheng, Research Assistant at KULT.
The attendees include Ann-Kathrin Wortmeier, Ph.D. Visiting Researcher at KULT from the Research Center for Interdisciplinary Risk and Innovation Studies at the University of Stuttgart, Germany; Artur Serrano, Professor at the Department of Neuromedicine and Movement Science; Gunhild Tøndel, Associate Professor at Department of Sociology and Political Science; Eric Monteiro, Professor at Department of Computer Science; Jens Olgard Dalseth Røyrvik, Associate Professor at Department of Social Anthropology.
We also have the pleasure of involving a few master’s students from the Department of Interdisciplinary Studies of Culture and Department of Social Anthropology at NTNU: Halvard Moe Krogstad, Nienke Bruijning, Fredrik Billington, especially Synne Svinsås Gjønnes and Gro Anita Sørskogen Stovner have also supported our project on the side of their studies. Synne has contributed to mapping literature, and Anita has focused on interviewing service and sales sectors involved with automation.
We had several updates from the attendees:
Jens has an article on conquering of technology; Gunhild’s other project—qualities of local healthcare delivery, especially quantification of the quality metrics; Anne-Katherine discussed exoskeletons and augmented reality/wearable technology in the construction field, especially timber construction, and what are the risks and benefits;
Nienke mentioned that replica AI, a chatbot used on the smartphone that’s supposed to be a friend or romantic partner. It is essential to some people now because of covid. Filled in the gap with healthcare workers who couldn’t support them, Halvard shared his experience visiting coffee shop robots in Seol, the café culture in general, and what they experienced in their interactions. Frederick talked about the implementation/use of tech in armed forces, Coastal Rangers, and mobile missile systems. Tech as a mediator of uncertainty; Synne studied AI and ethics through a company called Nora. Eric shared his experience from the computer science dept with digitalization in mental healthcare, energy/oil/gas, and social theory on tech development and use.
Building industry report
Kristoffer found that workers are very positive towards robots. Even though it’s evident that it will be more machines in the future, they’re not afraid of it. People see the positives for them. Does workers’ acceptance make work safer, easier, and more efficient? Or are they only concerned for their own jobs and not the future generation? Erick pointed out that the threat of automation has been around for decades, so their lived experience is that it’s not scary. So is the current transition continuous with previous experiences or existentially different? Jens said that tech could infringe on their autonomy and externalizes who’s in power.
Håkon further discussed that builders don’t think robots can replace them soon because they need to improvise every day, which is not currently possible with robots. But there is more hiring of people of planners and less of site workers. Also, separating planning and building—used to be much more connected. Resistance is more to tech-like smartphones to aid in adjacent aspects, e.g., ordering work clothes. BIM—Building Information Management—a digital model of a construction site. Now used to make calculations (e.g., the strength of materials, logistics, price). The use of robots on the building site can be connected with the model. We can look for conflicts between the model and the actual site.
Ann-Kathrin questioned who is excluded from the BIM, e.g., what part of the infrastructure? It’s supposed to bring all sorts of data streams/specialists together. But some groups don’t want to be included, like architects who wish more of their ideas to be secret. Also, every problem is treated as equal, e.g., a minor issue with the ventilation system is given the same weight as the big holistic vision of architects. Another resistance—modern sites should be paper-free and built directly from the model. But workers still want to use paper drawings, e.g., the touch screen doesn’t work in the cold or have to scroll it all the time. Or a worker needs to be able to change, improvise, and can’t interact with the model electronically like they can edit a paper drawing.
Eric said that some sorts of digital twins are different, e.g., oil and gas have models of deposits based solely on models, and no one has actually been there. For example, Frank Ghery was a user adopter of BIM because the strange lines needed that computing capability. He required others to use it if ppl worked with him, and he’s such prominent people wanted to; Artur mentioned that creating a model that is the real thing, e.g., decentered and where you create a virtual space, that’s the main thing, you go to work in that virtual space.
Kristoffer indicated that automation had taken a lot of jobs, just not in the way we think. How do we conceptualize change? Some are visible, e.g., building motor as a craftperson to people in an assembly like a robot. But others happen slowly as fewer people are needed to mix cement. Automation of jobs vs. automation of tasks will gradually take away jobs, but as some functions are automated, they have other duties. Differentiation between products and services. Previously the mark of quality was identical to everything else. Whereas service quality is based on being not identical but tailored and unique.
Ann-Kathrin questioned standardization. Architects don’t want things to be standardized because they want to maintain creativity. What do we mean in life, e.g., meaning is inscribed into the technology, but workers don’t have access to that meaning because they don’t have expertise; Artur talked about identity. If things are standardized, you use a bit of your identity. You want to belong and be part of a community, but you also want to have individuality; Jens discussed what automation is. Like Heidegger, it’s actions happening without thinking. WE’re structuring the world as if automatization is like action without consideration. A drywall cutter was viewed as a promising tool more than a robot. The tool was considered beneficial because of its ergonomics—they don’t have to hunch over all the time or use their thumbs to hold the drywall in place. Boston Dynamics dog robot would save a lot of time taking photos of building sites. The hole drilling robot also helps from an ergonomic perspective.
Sale and Service
Roger talked about how sale and service work can have the consumer at the center. Human workers now have to control machines, e.g., monitor self-check-out stations. People usually use self-checkout counters if they have a small number of items. Or if they’re young. Older people don’t like them as much. More older people wanted to be able to talk to cashiers. However, cashiers had conflicting feelings. They used them when they shopped themselves. But they never really liked them as tools. They felt more alone because all they were doing was acting as a guard. They are also lonely; they don’t talk with anyone; Jens mentioned other forms of automation, like CSAs, taking out the store as an automated system. It’s about shifting job responsibilities onto the consumer. There is an interesting case: hatches in the wall of Torget, Trondheim. In the investment industry, it used to be reserved for a few; a bank would manage your money. Now anyone can buy stocks on the internet. Also, trading bots make automatic investments.
What do people think about self-checkout?
People like to use self-checkout because they don’t want to speak with people. Listen to music. Enjoy the self-checkout because it’s like playing a game. Go to whatever is the shortest queue. But trying to do the scanning during the shopping experience it’s awful; you have to think about it the whole trip. For example, in Germany, people always use human cashiers and don’t want the stress of not using the system correctly. But in Norway, where sometimes workers only go to the cashier station when a customer is there, use self-checkout not to stress the worker out.
However, sometimes it depends on what I’m buying: people want human interaction for fancier stuff. For basic stuff, people just want to get out quickly. Kristoffer has experienced a completely automatic hotel, no staff, automatic check-in, and then go to your room.
Yu mentioned that there are new shopping methods for makeup and clothes because of COVID; you don’t try lipstick on your hand anymore, but instead upload a photo of your face, and the AI will show you different makeup and clothes.
AUTOWORK: Workers in transition through automation, digitalisation and robotisation of work
Sales and Services
The AUTOWORK project investigates how digital and automated technologies impact on workers, their work practices and the meaning they give to work. Automation refers to the use of robots and other technologies to minimise human involvement in the delivery of goods and services, while digitalisation refers to the ways in which digital technologies mediate the delivery of these goods and services. The main goals of the project are to learn about and provide solutions to societal changes arising from the automation and digitalisation of work-life. There have been bold predictions about the impacts of automation and digitalization; however, little research has engaged with the experiences and opinions of workers themselves.
Participation will be in the form of a personal interview, which will require you to give 45-60 minutes of your time, which will be recorded with a digital voice recorder.
Participant incentives:As a small token of our appreciation for your time and insights, you will receive a $20 dollar Coles Group & Myer Gift Card for taking part in the interview. This will be presented at the interview completion.
Who can participate? If you work in a sales and service sector job and your work is affected by digital and automated technologies, we want to speak to you. You must be over 18 years of age. You can find out more about the project on our website: https://autoworkproject.org/. Please feel free to spread the word.
Research team: The project is a collaboration between the Norwegian University of Science and Technology and Monash University. This part of the research, examining digitalisation and automation in the retail and service sector, is conducted by researchers in the School of Media, Film and Journalism, Monash University. If you would like to participate, register your interest at: bit.ly/AUTOWORK-SignUp or contact:
November 4th and 5th Håkon and Jens visited Aarhus and the Anthropology of Technology (AntTec) conference. Here we gave two presentations, one on the relation between technologic and presence, and one on a certain tendency observed among skilled workers in technological workplaces: “Paper-free building sites” has been introduced to achieve efficiency, where builders work straight form the digital models. However, we observe builders sometimes breaking the rules by secretly using paper drawings after all. “you see, I need to draw the corrections on the paper when what I build deviate from the original plan.” The drawings provide a “dialogical space” where builders can negotiate according to their professional judgement, something the digital models not always accommodate for.
Based on fieldwork at construction sites and in offshore shipping, our presentation explored how the notion of craftsmanship and seamanship, and the professional dialogue transforms as organizations and work are organized as digital machines, enhancing some perspectives while excluding others. In a broader perspective, the digital organization was seen to resemble a machine, that ideally should be able to work without human intervention. Crafts- and seamanship increasingly include adjusting to, and mending “the machine”, even breaking rules to have it work – all to get the job done.
During the visit, we also, we discovered that Aarhus is a really nice city with fantastic restaurants.
The world’s population is ageing – and robots are increasingly portrayed in care roles in fiction.
What will the roboticized future of health care look like?
In this online workshop, we will explore how medical and care robots are imagined in fiction, and reflect on the opportunities, inspiration, fear and trouble it can imply for human societies. What boundaries are there for robotic care? Which possibilities exist, or can be opened up in the future?
Workshop participants will work together on discussing grand societal issues of the care sector at a time of technological transformation.
Are you fascinated by robots in fiction? Are you interested in healthcare technology and curious about the new horizons opened by robotics in the care setting?
Together with Kristoffer Nergård and As.Prof. Håkon Fyhn, Søraa and his team are looking at the practical side of robotics. The researchers have been quite curious about the people who build buildings – eg. craftspeople that do a lot of strenuous lifting and have to check that the buildings are made correctly.
The researchers were also curious if any technologies such as robotics are being used to help make this work easier. The researchers are studying new technologies for the building sector. They question what type of social materials boundaries can be considered technology tools, what is viewed as a robot, and how these technologies construct the modern craft person and impact their knowledge as a professional worker.
When technologies such as robots are moving into the construction site, that changes the site. In their techno-anthropological & Science and Technology Studies (STS) approach, the researchers wanted to go and talk to craft people directly to see what they thought about these new co-workers of theirs.
Søraa gave a very simple example as a first technology, the hammer, which is humanity’s oldest tool for building, from about 30,000 BCE.
There are a lot of innovations in the hammer it has been used such as a jackhammer, war hammer, and the hammer we know from our homes. Humans are being identified as tool-making/using- species, and even chimpanzees have made/used hammers.
Also, Søraa pointed out that craftspeople’s clothes have specific pockets to carry hammers around. However, learning to use a hammer requires a lot of training, especially since the hammer is not an autonomous technology.
Therefore, Søraa concludes that the hammer shapes the hammer as a tool that shapes society and society.
They are moving from that to a more advanced technology, robots, it’s important to consider why these robots are implemented into craftspeople’s work – for efficiency or safety? And how does it affect the acceptance of these robots?
Now we are at a very interesting point in time; we see a lot of robotics moving to the Norwegian construction site. For example, the Boston Dynamics Robot spot has been used in Norwegian building sites.
The researchers wanted to know if robots were seen as a tool or co-workers for craft people? Do they fall into uncanny valleys? Or do people not trust them? Or are people scared of them? The researchers wanted to examine posthuman categorizations of the non-human. Or was it seen as an “other” being on the construction site?
There is quite a lot happening on the construction sites, also in the digital realm. Many building information modelings are being used in the Norwegian construction sector, such as BIM models, which create digital twins of buildings. You have a digital version of the buildings that are being made. Then you have the physical structure that is also being made. And a lot of craft people either work outside in the harsh weather, or they have some colleagues sitting inside and making them inside the nice warm offices. There are some potential conflicts between these two types of workers. Then you have a new type of worker, the robot, that moves around often in the night and checks if the buildings are built correctly or if any errors need some discipline.
Søraa and his team are quite curious about the autonomy and power between the workers inside the offices, the workers outside that in the construction site, also the robots that move around between them. Above are the examples of craftspeople that they are looking at. The craftsperson below cutting a specific cupboard to fit the BIM model has told him what to do. The machine actually is doing the cutting work and tells him what to do, and he is just putting it there to be made correctly.
Robots and craftspeople both work in the boundary between the physical building and the digital models – creating new forms of craftsmanship, so it is not wielding the hammers and hammering the nails anymore. It is quite complex nowadays.
For example, Søraa and his team see the posthuman worker for the future of work with machines provides epistemic challenges for what a craftsperson should be and what a craft person should know. And it is a matter of cultural, social, and organizational factors that affect the future of work where the new technologies can also lead to the deskilling of work. So it is not only about becoming an expert in using robots, but it can also be the robots taking some of your work away from you and being deskilled to do effortless work.
Although it is essential to be situated area, the Nordic context, Søraa sees that the craftsmen and workers they have been studying conceptualize work as more than a means to survive; it is also a career. Whether machines replace humans, they find themselves working alongside them or working within machine systems. The researchers see that automation and robotics in the construction site radically transform what it means to be working in the construction sector.
How can future work-life provide a meaningful and secure place for human workers?
We cannot just replace workers with robots.
Lastly, Søraa situated it in the Nordic work model, where Norway has worker unions, business organizations, and government actors coming together regularly and sitting down and discussing what we should do to change the sector what it needs to remain. And right now the researchers see that Norway has a new actor in the model, the robots in the three-part model, although the collaboration is a bit unsure.
The working/crafting human conceptualizes work as more than a means to survive.
The researchers conceptualize this as “Craftsperson 2.0”, where we have the future of work, automation, skills and craft, robots, digitalization, building information modeling (BIM), altogether creating what is being a craftsperson, which is not the same as what it was yesterday.
A question that came up was about the role of unions. Norway as a Nordic country has a union rate of about 50 percent. Sweden is about 67 percent, Finland has a similarly high amount at 60, and Iceland has almost all workers unionized. Although it is quite different from North America, where it is much more contested, it is not mandatory here in Norway but quite normal to be in a union. The supervisor often recommends you join the union, and they are probably also in the same union. Søraa thinks it helps in security in moving forward together, but we see many worker issues in terms of projects are being picked up by the lower bid, for example, and the building mistakes are being made.
During the summer of 2021 I did a fieldwork among carpenters in a small refurbishment company in Trondheim, Norway. It was exciting to learn by working with these fun guys! I am currently in the first stage of my PhD research project where I am trying to understand more about skills and culture among craftspeople in Norway.
The next stage will be to do fieldwork in a larger, more technologically complex construction company in Norway, and I will study construction workers who work alongside robots
AUTOWORK researcher Dr. Roger A. Søraa has just published a new book on Digitalization, together with colleague Dr. Kristine Ask at the Department of Interdisciplinary Studies of Culture, NTNU Norwegian University of Science and Technology.
The book features a chapter on the automation of work, and is a good introduction to understand how work is impacted by digital practices, and how technologies change the notion of work.
The book can be found in your local academic bookshop:
In kicking-off the AUTOWORK project, we have great expectations for what the project will bring to research on the automation, digitalization and robotization of work.
Although we had planned for a physical meeting in Trondheim, Norway, global realities of course made that impossible. Shifting to a digital solution was interesting in itself given the topic of this project: Would this digitalization of our work make it more efficient? Would the lack of informal interactions during coffee breaks or meals be detrimental? We found, surprisingly perhaps, that we could get to know each other well in the digital realm, even as we look forward to meeting face-to-face. But, most importantly, we discovered an enthusiasm for this project, and a wealth of knowledge that all partners already possessed, that was almost impossible to contain to eight hours of of video conferencing. We also discussed how COVID, which dictated the digitalisation of our own kick-off meeting, is accelerating the automation and digitalization of work, especially in the sale and service sector, in ways both visible and invisible, making out research more timely than ever.
The AUTOWORK project consists of project leader NTNU Social Research and four departments at NTNU Norwegian University of Science and Technology:
Department of Social Anthropology
Department of Interdisciplinary Studies of Culture
Department of Neuromedicine and Movement Science
Department of Sociology and Political Science
Department of Computer Science
With us digitally were partners from Monash University, in Melbourne in Australia. Participating from Monash are:
School of Media, Film and Journalism
Emerging Technologies Research Lab
Norway and Australia can seem remote from the rest of the world, at the northern and southern reaches of the globe. The restrictions on travel can make that isolation feel larger, while the digital meetings can make those distances seem to collapse. We look forward to sharing with the rest of the world insights we can glean about the future of work.