Quantifying co-creative writing experiences

Collaboration with artificial intelligence (AI) is a growing trend even in the field of creativity. This paper examines which quantitative metrics can be used to comparatively analyse human-computer co-creativity with children. To study this question, 24 schoolchildren of age 10–11 wrote a poem with three co-creative poetry writing processes: a human-computer, a human-human, and a human-human-computer process. The computational participant in the processes was an AI-based application called the Poetry Machine. The children were asked to evaluate their user experience with a 5-point Likert-type questionnaire after each writing process and a comparative questionnaire after finishing all processes. The metrics used in the evaluation were immediate fun, long-term enjoyment, creativity, self-expression, outcome satisfaction, ease of starting and finishing writing, quality of ideas and support from others, and ownership. Significant differences were found in fun, long-term enjoyment, quality of ideas, support, and ownership. The high number of statistically relevant results was enabled by exposing all participants to all writing processes, and the comparative questionnaire. The human-human-computer process was evaluated the best in long-term enjoyment and the human-computer process the weakest in support and idea quality. Creativity and ease of finishing writing turned out to be outlining metrics for the co-creative processes.