Keywords

1 Introduction

Humor is an essential part of our social interactions with others. We use humor in our interactions with others. We appreciate humor generated by others. We are aware of humorous events in our environment. We have a sense of humor and also we want our partner and friends to have a sense of humor. We have favorite TV comedy series, comic strips, cartoons and comedians.

We can make a distinction between intentional and unintentional humor. In the case of intentional humor we can make a distinction between spontaneous humor and prepared humor. Spontaneous humor is created real-time, based upon what is happening at that moment and what is perceived by those who are present. Spontaneous humor can be a response to events happening in a physical environment, including conversational and imagined events. Prepared humor can be a joke, an anecdote, a humorous text, a cartoon, an animation or a prank. Of course TV comedy series, stage comedies and comedy movies are also examples of media that use prepared humor.

Stand-up comedy contains both prepared and, to a lesser extent, spontaneous humor. There is interaction between the stand-up comedian and his or her audience. Timing of actions (gestures, bodily movements, facial expressions) and speech (prosody) are not only functions of the prepared humor, but also functions of the feedback (laughter, applause, gestures, bodily movements, attention, gaze, et cetera) provided by the audience.

1.1 Introducing Humor

Humor is a well-known research area in philosophy, psychology and linguistics. The role of humor and laughter was already discussed in writings of the Greek philosophers Plato and Aristotle, and the Roman orator Cicero. Well-known 18th and 19th Century English, German and French philosophers developed thoughts about what causes amusement and what makes people laugh. Wit and absurdness, intentional and unintentional humor, but also aesthetics, moral and social issues became part of their attempts to characterize ‘the comics’. Obviously, at that time rough humor and jokes were present in daily life, but they received less attention than wit, word play and other forms of upper class humor (e.g., in the literature or in theatrical performances) that required some intellectual effort. Sigmund Freud investigated jokes from a point of view of relief of tension that has been built up because of being obliged to follow social conventions in real life. In jokes ‘naughty’ issues can be addressed we don’t usually introduce in social and polite conversations.

Irony, sarcasm and, more generally, non-literal language use became research issues in 20th Century linguistics. Computational linguistics and artificial intelligent (AI) approaches to language humor followed. Victor Raskin in his book “Semantic Mechanisms of Humor” [1], based on his earlier work in the 1970s, introduced a theory of joke analysis. Refinements followed in later years. In 1989 Douglas Hofstadter and his students organized a first workshop on ‘Humor and Cognition’. In this workshop we see, apart from Raskin’s work, the first computational modelling approaches to humor. The focus in these linguistic and artificial intelligence approaches was on finding conditions that make incongruities humorous and in later years on finding knowledge representation schemes that in addition to model the structure of a joke, model the knowledge that is required to understand the joke. The latter turned out to be the main problem of designing a usable formal model of jokes and of artificial intelligence in general. Modelling all common-sense and world knowledge, reasoning about this knowledge and finding associations between concepts, and exploring different forms of instantiations of concepts turned out to be an unreachable goal.

In 1996 we introduced the term ‘computational humor’ when we organized the first International Workshop on Computational Humor [2]. In this workshop we collected approaches to verbal humor that gave rise to an expectation that an algorithmic approach to humor could be successful. This workshop was followed by two other workshops [3, 4]. In these latter workshops there was interest to investigate the relation between humor and emotion, to investigate machine learning approaches to humor, and to investigate the role of humor engineering for smart environments. These environments have interconnected embedded sensors and actuators and the question arises whether their smartness can be used to generate humorous events. For smart robots, whether they act autonomously or are controlled through the Internet (of Things), the same question can be asked.

1.2 Smart Robots and Humor

Social robots can be made to know about their environment, their interaction partners, and the tasks they have to perform. Unfortunately, knowledge that is modelled is (very) limited, restricted to a very limited domain, and robot behavior based on that limited knowledge can go wrong. But, nowadays AI is less science fiction than it used to be. We will meet such robots in our daily environments.

The first robots appeared in the literature in a theatre play [5].Footnote 1 Robots, as they appear in the literature, on stage or in movies are infamous because of their lack of sense of humor, and not being able to understand and express emotions. We can see that in the behavior of Commander Data, a robot in the Star Trek television series and feature films. Despite Data’s enormous computational abilities humor and emotions are beyond his reach.

Human-Robot Interaction is now a well-established area of research. Many research papers have addressed the role of humor in human-robot interaction research. However, research does not really address humor modelling. Rather it addresses how to provide a robot with delivery skills or how people experience being addressed by a robot that is telling jokes. Does it make the robot more human-like or do people prefer a humorous robot above a non-humorous robot are research questions that are asked.

We should mention that in human-computer interaction research there is much interest in embodied virtual agents. This is a related research area. Natural communication with embodied virtual agents require that these agents have intelligence and have affective and conversational skills, including having the correct facial expressions, gestures and bodily movements when interacting with a user. Hence, similar models can be used for embodied virtual agents and social robots. Chatbot research, where the bots are not necessarily embodied, is another research area related to social robotics research.

1.3 About This Paper

The main focus of this paper is on robots that have been designed to perform stand-up comedy. These robots need to know about how to deliver humorous content to an audience: how to tell jokes, make witty remarks, and how to adapt their joke telling and timing on audience responses. In order to do so we can learn from humor research, virtual agents research, human-robotic interaction research, speech and natural language processing research, and research on nonverbal interaction. It should look like the robot understands the humorous content and the reactions of its audience. The robotic stand-up comedian should be believable. We can ask that a performing robot is also the creator of the humorous content it is delivering. Deciding about delivering prepared ‘witty’ remarks because of audience reactions seems to be a feasible aim. Maybe not that different from what is done by a human stand-up comedian.

In the next section, Sect. 2, we discuss some background and related research. We mention some relevant humor theories, various ways a robot can be equipped with a sense of humor, joke telling by robots, aspects of nonverbal and physical humor and unintentional humor. There are different senses of humor. Social robots may require a particular type of humor sense or maybe it should depend on its task or it should adapt to the sense of humor of its user, its users, or its audience. Nevertheless, the assumption is that social and performing robots have facial expressions and can make gestures or have other means to display affect with a similar effect on their users, partners, or audiences. Section 3 is on robot actors in comedy and play. Section 4 gives a state-of-the art survey of current activities of robotic stand-up comedy. Some conclusions can be found in Sect. 5, this paper’s final section.

2 Background and Related Research

In this section we have observations on humor theories and on humorous human-human and human-robot interaction. Since in this paper we are interested in robotic stand-up comedy, there is a bias in our observations to delivering humor rather than on analyzing, understanding or creating humor. Nevertheless, we need to have some knowledge of humor in order to implement humor in the interaction skills of social and stand-up comedy robots. What makes an anecdote, a joke, a story, a product, a cartoon, an animation, or a situation humorous? We can fully script a robot to deliver a particular humorous content with appropriate speech, facial expressions and gestures. The humorous content should be in the comic script, where, when and how the humor should be delivered. The comic script needs to be transformed to a robot script that tells the robot or a backstage controller of the robot how the humorous content should be delivered using detailed instructions on robot movements, speech, timing, et cetera. In an ideal situation this translation should be done automatically. This requires understanding of the comic script, it requires models of humor and models of verbal and nonverbal behavior related to the delivery of humor. Humor research should make it possible to reduce the human effort to translate a comic script into a robot behavioral script.

But clearly, we can also ask whether we can have the robot design a comic script, for example, design a joke. Or improvise and spontaneously compose a humorous act or a witticism when an interaction and situation makes that possible. As will be clear from the next subsections, we cannot expect that to happen in the next decades. But of course, progress can be made.

2.1 Theories on Humor

As mentioned in the introduction of this paper, philosophers, psychologists and linguists have tried to identify the characteristics of humor. What makes us distinguish between humor and non-humor? Various humor theories have been introduced. Some of them emphasize the function of humor rather than trying to analyse why a particular object (event, situation, product, text, cartoon, animation, …) gives rise to amusement and, maybe, laughter.

Usually a distinction is made between the reasons between why appreciate humor (superiority considerations), the functions of humor (its positive effect on our psychological state) and cognitive state changes (perceiving and resolving incongruities). Hence, there are three complimentary viewpoints.

From the superiority point of view we experience amusement and maybe start laughing laugh because we are not the butt of a joke, a prank, or whatever humorous event. Rather we laugh about the misfortune of others. This superiority viewpoint can be found in the writings of the British philosopher Thomas Hobbes (1588–1679), but it can also be deduced those of Greek philosophers such as Plato and Aristotle. In [6] an attempt is made to explain all humor from this particular point of view. The relief perspective is usually attributed to Herbert Spencer (1820–1903) and Sigmund Freud (1856–1939). It emphasizes that due to social conventions we build up tension and humor, in particular jokes that have content we don’t dare to talk about in daily conversations, releases us from that tension. Finally, the incongruity point of view emphasizes that many situations (displayed in text or in reality) can be interpreted in different ways. Slight differences in interpretations are not humorous. Opposing interpretations can be humorous. Making the successful shift from one interpretation (a stereotypical one) to a correct one (it should be far from being stereotypical) is what leads to comic amusement. This incongruity point of view has been discussed explicitly in writings of 18th Century British philosophers and writers such as Mark Akenson (1721–1770), James Beattie (1735–1803) and more famous philosophers such as Immanuel Kant (1724–1804), Arthur Schopenhauer (1788–1860) and Henri Bergson (1859–1941). Although the superiority and the relief perspective remained, in current humor research the emphasis is on incongruity. It allows us to analyze why texts or events are humorous. Victor Raskin [1] used this perspective to build a theory of joke analysis and understanding.

A stand-up comedian is not necessarily aware of these theories. For talented comedians intuition and experience are more important than humor theories and how to adhere to them. But this is not something we can expect from a robotic stand-up comedian. A robot stand-up comedian should be equipped with computational models, maybe obtained through machine learning, that tells him or her what is humorous and what is not. It is hard to imagine a stand-up comedian who does not understand the humorous effect of the content he or she is delivering. Maybe we can nevertheless imagine a robot or virtual agent simulating such a comedian in a performance where there are jokes and maybe even witty remarks and interaction with the audience, without any understanding about the content that is delivered.

Humorous content can be scripted in such a way that its deliverance by an artificial agent leads to a believable performance. The timing of humor delivery depends on audience feedback, but this feedback can be measured and can be made input for the agent interacting with the audience and make its performance dependent on this feedback. The creation of humorous content is not that ‘easy’. A stand-up comedian can have text writers, but when attending a performance we ‘believe’ that the humor that is presented to us is coming from the comedian’s creative mind and sense of humor. Design guides for creating stand-up comedy humor or sit-comedy humor are available. Superiority and relief theory aspects can guide the intentional creation of humor. Make others ridiculous (superiority) or introduce taboo topics. But the audience has to be surprised, misled and amused by verbal and nonverbal incongruities and incongruities in physical acts, events, and situations.

2.2 Humor in Human-Robot Interaction: Global Observations

We use the term humor in a wide sense. But there are different styles of humor. For example, in [7] eight styles are distinguished and discussed: sarcasm, cynicism, satire, irony, fun, humor, nonsense and wit. In the narrow sense humor is more about having a positive attitude towards incongruities, imperfections and human shortcomings. Human-Robot Interaction research is concerned with natural, that is human-human-like, interaction between social robots and their users. There have been many investigations and attempts to translate human-human interaction characteristics into what is expected by human users in human-robot interaction. These investigations include the use of humor. Instead of aiming at comprehensive human-like humor in human-robot interaction it is also possible to accept that social robots can be given or will develop an own sense and style of humor, just as individuals, families, professions, cultures, age groups, and social classes can have different preferences for humor. Humor in human-robot interaction research usually assumes that the robot should have a friendly attitude towards its users, so, no use of sarcasm, cynicism, satire or irony.

In this section we provide a global view on research on humor in human-robot interaction. Obviously, this research assumes face-to-face interactions between human and robot. That is different from human-audience or the interaction between a comedy robot and its audience. However, humor in the form of jokes, anecdotes, and witty comments are studied in humor-robot interaction and the results of this research can be used in modelling stand-up comedian research as well.

Rather than creating or adapting humorous content, the research on humor in human-robot interaction is usually about how to tell a joke and how to, as a reaction to a joke or humorous event, to display laughter in a robot or in a virtual embodied agent. Displaying laughter and distinguishing between various kinds of laughter and reasons of laughter is part of current-day research on virtual embodied agent. They are more easy to control than physical agents such as robots.

Current research on the use of humor in human-robot interaction is very limited. We can distinguish the following approaches.

  • We can give robots artificial intelligence that makes them aware of and understand their environment. They need to understand humans. It requires human-level intelligence and a further development of interaction technology in such a way that it can replace the human senses by artificial speech and language processing, vision, taste, smell, and touch experiences for the social robot. Despite progress in modelling multi-sensory interactions and experiences, the research results don’t allow us to be optimistic about modelling human humor intelligence where all these senses and intelligence have been given a place. Obviously, nothing wrong with continuing research in these areas. Humor, until now, is not really a research topic here.

  • We can implement the telling of a joke in a social robot. We have a prepared joke, it can be annotated with instructions about intonation, speech rate and pauses, and annotations can also instruct the robot to perform the gestures and facial expressions that are needed to deliver the joke in a human-like way. Obviously, we can have research that attempts to generate these instructions automatically from a joke text. In that case we have some simulated understanding of the joke and how it should be presented by a robot or a virtual agent. This does not require all the artificial intelligence and multi-sensory perception mentioned in the preceding approach. Annotating text with instructions how to turn it into verbal and nonverbal behavior of artificial agents (virtual embodied agents, robotic agents) is the research area of intelligent virtual agents and affective computing. Results of these areas are now slowly becoming part of social robotic research. Of course, the physical embodiment of social robots introduces lots of other interesting research questions. From the point of humor we can mention that there is limited research on verbal and nonverbal performance of joke telling. In addition there is research on different ways of laughing in nonverbal speech research and how to implement them in virtual agents.

  • There is also ongoing research on whether people prefer a robot that makes humorous remarks during social or task-oriented interaction above a robot that performs its task without using task-oriented or general humor. How and when should a robotic receptionist use humor in its interactions with visitors? In this research it is often the case that humans are asked to interact with a virtual representation of a robot, rather than have real interaction with the robot that includes humor in interaction. There are findings that tell us (often in Wizard of Oz interactions) that robots that seem to display a sense of humor are more appreciated and are considered to be more human-like than robots that do not use humor in their interaction with their partners. However, the experiments that lead to such a conclusion mainly concern the use of humor in some task-oriented applications rather than daily conversations we can have with colleagues, friends or family members.

  • Less research is available on when and why a robot would like to perform a physical funny act or engage in physical humorous activity that also involves human partners. This is different from research on verbal and nonverbal behavior where we expect to communicate with a robot to have a social conversation, to have it perform a particular task in which we are interested, or ask for some support and collaboration that require physical tasks. Can we use AI to make our robots design pranks or involve them in performing (digital) pranks?

  • Finally we should mention unintentional humor. There is humor when people make errors, don’t understand technology or are absent-minded. Social robots can fail in what they are supposed to do either by shortcomings in their algorithms or by bugs. Both can have serious consequences, but it is also the case that due to shortcomings, bugs, and non-anticipated human use of such digital technology humorous situations will occur. There is intentional humor evoking smiles and laughter when we tell a joke or make a witty remark. In daily life there are many more reasons to smile and laugh, and, of course, these reasons should become topic of research.

2.3 Human-Robot Joke Telling and Conversation

We can have a look at how in research on human-robot interaction how these above mentioned issues are addressed. In the following paragraphs we survey the existing literature.

In order to investigate the effect of the use of humor in human-robot interaction it is often the case that this is done without using a real, physical robot. Online surveys or Wizard of Oz experiments are more usual. Also, the experiments are usually based on the effect that is obtained when a robot tells a joke. Hence, the investigations are usually about the use of humor by a robot in a human-robot conversation. How does the use of humor support the conversation? Does it increase or interaction enjoyment, our appreciation of how the robot performs its task? Does it enhance sociality? Moreover, is there a difference in humor appreciation when the humor is presented by a human or robot and is that also dependent on the humor style or type of joke?

The questions that we ask here are relevant both for humans interacting with physical robots and humans interacting with virtual agents in general and animated virtual agents in particular. Humor that is presented using speech requires humor-related nonverbal speech features (prosody, timing) or laughter, further embodiment requires the adding of nonverbal and humor-related interaction features such as facial expressions, gestures and whole body movements. Nowadays we have control languages and not yet fully developed models that can be used to generate appropriate nonverbal speech, nonverbal expressions and bodily movements. Implementing such features in animated virtual agents is a more doable task than it is for physical robots. Further development to physical robots also requires that other features need to be taking into account. For example, movements from body, limbs and head, three-dimensional physical appearance, proximity, and physical contact, as far as they related to delivering humorous content or receiving humorous content. Obviously, apart from the questions mentioned above we can have a focus on research dealing with the modelling and implementing the issues that are related to make the steps from text to speech, from speech to virtual and animated embodiment, and from virtual embodiment to physical embodiment. This will not be done here.

Classification of responses to a joking robot where humor was elicited through canned jokes and conversational humor appears in [8, 9]. The possible enhancement of sociality in robots using different kinds of humor (wit, dry humor, corny jokes, self-depreciation) has been the topic of the experiments reported in [10]. Whether likability depends on the humor style is also investigated in [11]. They distinguish between the effect of Schadenfreude humor versus self-irony humor, where in their investigations they have two robots where one laughs at another robot (Schadenfreude), or where a robot laughs at itself (self-irony). In [12] we have also a situation where more than one robot is involved in a humor experiment. Here the main aim is to see whether jokes are more funny when they are presented by a robot rather than presented as text. But to make it more interesting, what differences in joke rating funniness will appear when one robot tells a joke and there is one more robot present that responses with either laughter or booing? While in [12] we have a question about a possible difference between joke delivery by text or robot, in [13] there is investigation whether there is difference in appreciation when a joke is delivered by a human or robot. In [14] types of jokes (disparaging or non-disparaging) are part of these investigations in appreciation of humor in human-robot interaction. Detailed investigations on user perception of task enjoyment through different combinations of voice pitch and language cues (humor and empathy) can be found in [15].

2.4 Nonverbal and Physical Humor for Robots

Although there is a lot of research on modelling of nonverbal interaction behavior of humans and also of that of animated virtual agents (see for example the proceedings of the yearly Intelligent Virtual Agents conferences), similar efforts, making use of research in these two areas, cannot yet be found in human-robot interaction, let alone modelling intended humor in humorous human-robot interaction. A humanoid robot needs, among many other things, a model that allows it to understand human nonverbal behavior and a model that allows it to generate appropriate nonverbal behavior. An overview of verbal and nonverbal aspects of human-robot interaction is presented on [16]. Most humor research focuses on textual humor (written jokes, wordplay, wit as it appears in anecdotes, humor in longer texts), so it is no surprise to see few papers published on nonverbal aspects of humor delivery. Some observations on joke performance are discussed in [17], but they rather focus on whether humor theories should take into account the role of the performance.

Another example is [18], but here the focus is on timing of joke telling (discussing pauses and speech rate only) and the conclusion is that there is no convincing theory of timing available. Automatic selection of gestures that are appropriate to use when telling a joke or a short story is discussed in [19]. Exploring different verbal and nonverbal behaviors and study the effect on the perception of funniness is discussed in [20]. In human-human interaction entrainment (synchrony) occurs between conversational partners. This entrainment shows in the face (expressions, gaze), gestures and other bodily expressions. Mimicry is an example of this entrainment [21]. This behavioral entrainment also occurs between the teller and the responder to a joke [22]. If a robot is listening to a joke it should act in synchrony with the human joke teller. If the robot is delivering a joke, it should be aware of distortions in this entrainment and treat them as signals that the responder is not understanding the joke narrative or not interested in it, and then adapt to that situation.

Laughter has received more attention than nonverbal physical behavior. Obviously, here we don’t mean the interest of philosophers investigating the sources and the functions of laughter, rather we look at research that aims at recognizing different kinds of laughter and also the artificial production of laughter. Humanoid robots need to distinguish between different kinds of laughter (laugh classification) and need to be able to produce different kinds of laughter [23]. Detection and recognition of laughter in the context of human-robot interaction is reported in [24]. Perceived naturalness of laughter in humanoid robots is discussed in [25]. Clearly, a laughing robot should also express its laughing with changes in facial expressions, head and body movements.

When a humanoid robot can move around and can perform various tasks or acts it can become part of humorous events or be active in creating humorous events in physical space. An entertainment robot that moves around or robots that guide visitors in a museum environment or travelers in an airport can detect or comment on changes in the environment in a humorous way. Observations on the kind of global responses an entertainment robot draws in a public environment can be found in [26]. In [27] experiments with a serving robot, delivering objects to its owner, are reported. It can move around, it has whole body movements and in the experiment tasks are performed in a straightforward or humorous way. Questions that are addressed are about the effect of humor in the perceived interaction quality and in the evaluation of some other humor-related characteristics of the robot.

2.5 Unintentionally Performed Humorous Acts by Physical Robots

Robots do not have to be designed to be humorous in order to make people laugh. There are YouTube compilations of robot failures that should be considered humorous. They include robots that fall while they climbing stairs, or a robot dog that slips on a banana skin. And unlike the BBC aired Robot Wars (a robot combat competition) and the high-tech RoboCup for autonomous robotic footballers, there are also contests for idiotic and crappy robots. These contests have some rules but, for example in the ‘Bacarobo’ comedy contest, the most important thing is that their stupidity should make the audience and the judges laugh. In [28] it is mentioned that the robots, the contestants, the announcer, the judges and the audience together create a ‘clown’s theatre’. This Bacarobo contest started in 2007 and 2008 in Tokyo, but later it was organized in other countries too. The same happened with the ‘Hebocon’ contest for dummy robots in which the robots have to follow some Sumo rules. The most stupid robot wins a prize. Robots that will be laughed at because of their imperfect (and too robotic) behavior have also been designed by Simone GiertzFootnote 2. These robots are not humanoid and although their behavior evokes laughter they cannot be compared with human stand-up comedians.

3 Robots in Comedy

Robots have been welcomed as ‘actors’ in movies. It is interesting to look at robots displaying humorous behavior in movies. This is not the topic of this paper. In stage plays we can also see a development from just clumsy robots moving in a mechanical way and having no sense of humor, to robots that become intelligent and have a sense of humor. The latter is often seen as a distinguishing distinction between humans and robots. In Čapek’s play we had, as was the case in early movies, human actors impersonating robots. Robots (or androids) also appear in plays of the English playwright and director Alan Ayckbourn. An extremely clumsy automatic child-minder called NAN 300F and played by actresses appears in his comedy ‘Henceforward’ (1987). Robots from that NAN series had some teething problems such as putting a baby in a microwave oven, not realizing that the kitchen had been changed by the mother. In Ayckbourn’s comedy ‘Comic Potential’ (1998) robot JC-F31-triple 3, also known as “Jacie Triplethree” has a sense of humor and through its exposure to human behavior learns about emotions. In later years we see real robots appearing on stage, but their behavior is pre-programmed or tele-operated.

In virtual and physical worlds we can look at developments that allow going from fully hand-crafted scripted animation of agent behavior and interaction capabilities to animation that is based on the availability of models that describe non-verbal speech, facial expressions, body language and eye gaze behavior and use such models to generate appropriate behavior without detailed instructions from a script. Although such models are being developed in human-computer interaction research they are not yet used to fill in details of nonverbal behavior of robots automatically. In robot theatre acts and interactions between actors are scripted and usually a robot actor is remotely controlled. In robot theatre we can make a distinction between theatre where there is no or hardly need for the actors to be aware of the audience and theatre were the authors need to be aware of the audience responses. This latter is of course the case when there is explicit interaction with the audience (interactive theatre) or where the actors adapt their timing, their nonverbal behavior, or even their choice of next actions to the audience response.

We mention a few robotic theater projects that illustrate these points. It/I is an interactive pantomime play [29] for a human actor and a computer-controlled computer-graphics actor projected on a stage screen. The human actor is tracked by cameras and this information helps in synchronizing the joint activity in the performance. The play was performed six times for a total audience of about 500 people. Cynthia Breazael [30] exhibited an interactive theater installation at the 2002 SIGGRAPH Emerging Technologies Exhibit in San Antonio. implemented an interactive installation with a terrarium as stage. Hence, the main performer was not a humanoid robot but an anemone that, using the installation’s computer vision, was aware of audience activity and changed its behavior accordingly, for example, becoming afraid when someone came too close to the terrarium. Breazael’s observations were more general: “In the future; we may see more elaborate versions of interactive robot theatre in theme parks, museums, and storefront windows. Someday, there may even be fanciful robotic characters on Broadway performing with human actors on an intelligent stage.” A step in this direction can be found in [31]. Here the stage has already three interactive robots. The narrator has a latex face that allows the display of facial expressions, the other actors have wooden mask faces. They perform a traditional Korean Hahoe play. There is computer vision, speech synthesis and recognition that allows the audience to interact in a chatbot-like way with the actors. In [32] we find a stage performance with two human actors and a robotic desk lamp. The lamp has pre-programmed gestures and sequences that are triggered by a human operator and, together with ‘eye gaze’ movements, adapted to the timing of the human performers.

The Robot Theater Project, a collaborative research project of Oriza Hirata and Hiroshi Ishiguro, started in 2008 and is ongoing. In this project robots are used as actors in theatrical performances, among them a version of Anton Chekhov’s “Three Sisters”, where Irina (Ikumi), the youngest sister turns out to be an android. Irina’s role is played by a tele-operated Robovie robot, operated by engineers from backstage. Ishiguro’s robot Geminoid F became actress in a short play, titled “Sayônara” in 2010. In 2015 she appeared in a feature film with the same title. The Robot Theater Project is discussed in [33]. This reference also includes a detailed discussion on the issues that play a role when aiming at natural and believable human-robot interaction and in particular interaction between a robot actor and human actors on stage as perceived by an audience. This discussion includes observations on scripted dialogues, synchrony, movements and gaze, anthropomorphism, appearance and the ‘uncanny valley’.

Plays including robot actors have also been performed in the Robotic Theater of the Copernicus Science Centre in Warsaw. One of the plays, “Prince Ferrix and Princess Crystal” (see Fig. 1) had a scenario based on one of the stories in “Tales of the Robots”, a book by Polish author Stanislaw Lem. Plays were performed using the programmable RoboThespian humanoid robots. A RoboThespian robot makes gestures, has facial expressions and a speech engine. It has computer vision to analyze the audience and individual faces, and it can receive audio information from various microphones.

Fig. 1.
figure 1

Photo by A. Kozak, Copernicus Science Center

RoboThespian Actors.

Full size image

As mentioned earlier, there are applications where a robot actor or a remote controller ‘just’ has to follow a script in order to have humorous content delivered in an appropriate way. But who is responsible for the humorous content? Most humor research is concerned with the analysis of jokes. There is some research on the automatic creation of puns. There is only very preliminary research on the automatic creation of humorous events. Such research could lead to automatic or computer-assisted creation of humorous content in a comic script or the spontaneous creation of humor in a particular situation. For completeness we want to mention some research that can illustrate this point of view.

Planning comic events in which virtual characters are involved has been topic of research in various papers. Can we automatically create cartoons [34, 35], humorous storytelling [36], or Mr. Bean like sketches [37, 38]? Stand-up comedy can be considered as a limited form of comic theatre. This research can help in designing events during stand-up comedy where props, sidekicks and audience play an important role. Admittedly, there is a long way to go from a computer assisting in the human, manual, creation of a script for a stand-up comedy performance to the spontaneous and real-time creation of humorous events. In virtual or digitally enhanced real world environments virtual, robotic and ‘real’ humans have to cooperate in order to create humorous events. This cooperation can be fully controlled by the script of the desired performance or it can allow non-scripted spontaneous humorous interactions when the artificial actors (whether they are virtual or physical) have an algorithmic sense of humor. Scripts that guide virtual agents in comic activities can as well be used to guide robotic agents or a mix of virtual, robotic, and human agents.

Comedy can be designed both for virtual characters and physical robots. The underlying models for their comic behavior can be the same, but it is more easy to implement humorous nonverbal behavior and acts in a virtual web-based world than in a physical robot-inhabited real world. In addition to humorous behavior and acts in virtual and physical worlds we can have an augmented reality point of view where real-world events are augmented with virtual information and events that make them humorous. This can take the form of simple textual comments or even a virtual character that is introduced to play a role in the event. No examples of this research point of view are available yet.

4 Robots as Stand-Up Comedians

We already mentioned silent comic movie and stage play actors who display robot-like behavior or act as robots, humorous from a Bergsonian point of view. There are also stand-up comedians who have imitated robots in their voice and their movements and sometimes in their simulated failures to act human-like. Among the robot imitators we have Mike MichaelsFootnote 3 as Mr. Zed the ‘Human Robot’, David Kirk TaylorFootnote 4 as the ‘Mechanical Magician, both from the nineteen nineties, and Alex MuhangiFootnote 5 as the ‘Robot Stand-up Comedian’. But now, rather than having human stand-up comedians impersonate robots we can look at robots impersonating human stand-up comedians. We cannot yet expect to experience a robot-equivalent of Lenny Bruce or Eddie Murphy. But some research that goes into the direction of replacing them with artificial stand-up comedians is done.

In the first subsection we will look at investigations into comic duos on stage, where one or both comedians are played by robots or virtual agents. This is more an area where we can expect that the performers need to be aware of the audience responses to their acts. In the second subsection we survey some of the research projects that aim at developing humanoid robots as stand-up comedians and adapt their performance to audience responses.

4.1 Comic Stand-Up Duo Performances

Comic dialogues are performed as stage acts, in circuses between the whiteface and auguste clowns and in movies, e.g. the famous ‘Who’s On First’ skit by Abbott and Costello. Manzai is a popular form of comic dialogue in Japan. It has a long tradition in Japan, but it is also a form of comic dialogue that appears in other cultures as well and we can find it wherever a comic duo performs, whether it is in a circus, on stage, or a in movie. It is also called the ‘Double Act’, or the ‘Straight and Wise Man’ act. Usually the characters have opposite personalities in order to make the duo more comical. Similar ‘Straight and Wise Man’ acts can be performed by a ventriloquist and his puppet, a human comedian and a robot (or virtual agent), or by two robots.

In Japan the two comical characters play the roles of Boke, usually the more extrovert person who expresses superficial opinions about his experiences, and Tsukkomi who usually has disdainful comments on Boke’s stories and behavior, and sometimes hits him on the head to correct him. In addition to nonverbal speech behavior (timing, utterance speed, and intonation) there are issues of appropriate nonverbal behavior (gestures, facial expressions, body movements), coordination and personality characteristics that need to be addressed. In [39] an interactive comedy system is introduced where the human plays the role of the Boke and the system the role of Tsukkomi. In this case the system uses speech and emotion recognition (the latter from speech) and speech output and facial animation of a virtual Tsukkomi is generated. Output phrases are selected from a database.

Since then other Manzai duos have been introduced, where usually we have the Boke and the Tsukkomi represented as physical human-like robots and users as by-standers or audience [40]. Hence, rather than give the humans an interactive role, the display of the robots’ conversations should be considered as a passive social medium, comparable to watching television. In [40] two humanoid “Robovie” robots took the roles of Manzai comedians. Robots were provided with script written in a scripting language designed for this purpose. Interesting also is the work presented in [41], based on results presented in many of their previous papers, where dialogue scripts for Boke and Tsukkomi (see Fig. 2) are automatically generated from web news. Keywords suggested by the audience let’s the system retrieve a news article from the web which is then transformed into a Manzai scenario consisting of humorous dialogues and misunderstandings and performed in real time.

Fig. 2.
figure 2

(Reprinted by permission from Springer Nature [41])

Two Manzai robots

Full size image

“Kobian” is a life-sized humanoid robot with an expressional face and gestures. It can detect and recognize laughter using microphones for recording sound and computer vision for detecting laughter movements [24]. Laughter elicitation is done by providing Kobian with skits based on Manzai comedy techniques, such as funny behavior (e.g., exaggeration), funny context (e.g. do unexpected things), and funny character (e.g., imitation and self-deprecating humor) [42]. The research is meant for application in nursing environments for mood improvement.

The Yoshimoto Robotics Laboratory has as motto “Making Robots To Make You Smile.” They have various entertainment and healthcare applications for the humanoid Pepper robot, among them human-robot comic dialogue, where the human is the ‘straight man’. Pepper’s voice, timing and movements are controlled behind the stage. The humor focuses on Pepper’s own robotic characteristics and capabilities.Footnote 6

4.2 Humanoid Robots as Stand-Up Comedians

There are not that many examples where humanoid robots act as stand-alone stand-up comedians. In a Manzai context they can have a robotic or human partner and especially with a human partner, whether in a Manzai context or a context where the robot is a sidekick of a human comedian, imperfections in a robot’s behavior can be made part of the act. A script that tells the robot where, when, and what to do does not exclude the possibility that there is adaptation to an audience’s response or that there is explicit interaction with the audience, although it may be in an Eliza-like way.

Stand-up comedy is about fun. We first look at a fun application where digital technology is used to capture the nonverbal behavior of stand-up comedians and translate it to similar behavior of an avatar (embodied virtual agent) impersonating the comedian. In this 2011 Kinect Comedy Fest the Kinect device was used to capture the movements of performing stand-up comedians and translate them into those of their virtual representation. In this case there was not really a scientific aim. Nevertheless, an approach where comedians’ nonverbal behavior are captured allow the analysis of such behavior and the results are useful when generating such nonverbal behavior on an avatar (embodied virtual agent).

There are not that many examples of robots that act as stand-up comedians. In the previous sections we met robots that have been programmed to tell a joke, displaying appropriate scripted nonverbal behavior, but also attempts to generate such behavior from a joke or short story text. In research on Manzai simulations with two robot performers the audience watches the performance just as it watches TV. On stage, serving an audience, it is more natural to have at least one human performer interacting with a robotic performer. One of the Manzai partners can be robotic, a robot can play the role of a sidekick, a human performer can play the role of a puppeteer. These human-robot interactions certainly assume the presence of an audience and performers, including the robotic performer, being aware of the audience and the audience responses to the humorous interactions on stage.

In the previous section we already mentioned some research projects where audience responses have impact on the interaction behavior of human and robotic conversational partners on stage. Although presented as a robotic stand-up comedian by Heather Knight [43, 44], her robot companion is a sidekick that needs her scripted guidance to initiate a scripted witty remark or to tell a joke. However, her Nao robot does some audience tracking. The Nao robot has access to a database of pre-scripted jokes. The jokes have attributes, have they been used before, what interaction do they allow, are they appropriate, et cetera. These attributes make up an audience model that shows the audience’s appreciation of these features, using this appreciation in order to choose a next joke to tell, and updating this model with the audience’s appreciation of this next joke. Audience’s appreciation was measured using audio (amount, intensity, and length of laughter and applause) and vision. Audience members could show their approval or disapproval by raising red (not funny) or green (funny) feedback cards during the performance.

A more ‘natural’ robotic stand-up comedian was introduced in the Robot Comedy Lab [45, 46]. The RoboThespian robot comedian performed during two evenings for a live audience (see Fig. 3). Its performance was preceded by the performance of two human comedians. The robot’s gestural and gaze behavior was based on observations of stand-up performances of two professional comedians. Four performative gestures were introduced: a “Welcome” gesture, a reprise gesture, a pointing gesture and a applause elicitation gesture. The robot received input from microphones and an audience tracking system. The latter allowed the robot to track individual faces and to focus on individual audience members. Moreover, the vision system used facial expression recognition for identifying the “Happy”, “Sad”, “Angry”, and “Surprised” emotions. Audio-visual recordings allowed the study of the effect of the robot’s gaze behavior and its performative gestures on the audience responses.

Fig. 3.
figure 3

(Photo by Toby Harris)

A robotic stand-up comedian [45]

Full size image

5 Conclusions

In this paper we surveyed research attempts to have humanoid robots perform as stand-up comedians. We mentioned, sometimes implicitly, research challenges. On the one hand we can ask that the robot comedian fully simulates the behavior and funniness of a human stand-up comedian. On the other hand, human stand-up comedians have different styles and why shouldn’t be there a style appreciated by a human audience that should be called a robotic style of stand-up comedy?

Robots can perform in movies, TV series, or on-stage plays with human and other robot actors. In our examples in this paper we have scripted dialogues that allow adapting to particular audience responses or planned interactions. A few times we mentioned that results of humor research should make it possible to reduce human efforts in translating a comic script into a robot behaviour script or even to design a comic script. It has been mentioned that humor is ‘AI-complete’. This is a notion from complexity theory and theoretical computer science, meaning that when we have models (and associated algorithms) for humor understanding and generation, this is only possible when we have such models and algorithms that allow a computer to behave like a human being. In the various subsections of Sect. 2 we have made clear what the problems are that need to be addressed in order to have a computer or robot or virtual agent or smart environment to be able to process humor. In Sect. 3 we looked at instances of robots appearing in theatre performances. Section 4 introduced robots as stand-up comedians. Human support of stand-up comedy was also discussed. Human support varied from having a robot tele-operated by a backstage engineer, having a robot comedian’s scripted behavior integrated with a human performer, and having such behavior adapted by being aware of an audience response to the displayed humor or humorous interactions.

There is more to stand-up comedy than the various aspects that have been mentioned in this paper and for which attempts have been made to model them in a comic robot-audience interaction. There are numerous books on stand-up comedy with attractive titles such as “Step by Step to…”, “Zen and the Art of…”, “Get Started in…”, et cetera, and several theses have been written about stand-up comedy, for example [47], or more recently [48].

We certainly are far away from robots that have autonomous humorous behavior. But there are attempts to have them act in humorous ways, to have them display joke-telling behavior and to have them become aware of audience appreciation. Let’s conclude with mentioning that in 2017 Disney Enterprises filed a patent application for huggable humanoid robots (Patent Number: US20170095925 A1 Soft body robot for physical interaction with humans). They can be used to replace the character actors at Disneyland. Maybe that is a start to have robots acting in real world environments and intentionally creating humorous acts in interaction with an audience.