Computer Science & Robotics Research
Research at the Goldsmiths Atelier
We have been focused on preparing the Franka Emika robot for collaborative work with artists, alongside developing technologies for image generation and stylization to be used by them. On the fabrication side, we have equipped the robot for painting applications, including the design of (i) a pen-holding system similar to that used by the AxiDraw pen-plotter, allowing for easy testing of different brushes and pens, and (ii) an orientable drawing pad that supports various drawing configurations depending on the medium used. On the software side, we have implemented (i) a path-planning system that enables the robot to draw vector files (SVG) or follow custom trajectories with specific kinematic requirements, and (ii) a Python API that allows for easy interfacing with the robot over a network (e.g. through Jupyter notebooks) and for programming motions through a combination of human demonstration and scripting (e.g., picking tools, mixing paint). The system includes an interactive UI for monitoring and programming the robot. On the research side, we are currently focused on developing methods that enable the generation of smooth “calligraphic” strokes with variable width profiles. The system can be easily integrated into generative imaging pipelines (e.g., Stable Diffusion), allowing for a flexible combination of criteria that guide stroke generation, such as image similarity, semantics, and structural/stylistic constraints.
Research at the Konstanz Atelier
In Konstanz, we develop the e-David painting robot into a creative partner for artists. Starting from an arbitrary image – often a reference or motif chosen by an artist – our framework transforms it into an abstract vector representation that highlights semantically significant objects and key structures. Painting agents then fill these regions on a real canvas, while a camera-based feedback system lets the robot check and correct its own work. More about the robots and the lab: Atelier Konstanz.
Early results from 2024: over several iterations the painting system evolved from indistinct shapes to increasingly recognizable and refined paintings.
Semantic and saliency-guided abstraction
Building on our panoptic-segmentation pipeline, the abstraction now uses saliency maps to decide automatically how much detail each part of an image keeps: salient objects stay detailed, less prominent areas are simplified, and a graph-based grouping step clusters regions that belong together. The method produces a hierarchy of abstraction levels in seconds, which the robot paints as successive layers – from large color fields to fine details – much like a human painter. New infilling strategies, such as dynamic curvy strokes and fast region filling with large color palettes, give the paintings a more painterly character.
Human-like strokes – joint work with Goldsmiths
Together with Daniel Berio and Frederic Fol Leymarie (Goldsmiths) and Sylvain Calinon (Idiap), we developed methods that let image-based AI generate the kind of long, smooth and human-like strokes that a brush or robot arm can actually execute: smoothing B-splines inside a differentiable vector graphics pipeline, and the sigma-lognormal model of rapid human hand movements.
Publications
- Stroh, Paetzold, Berio, Kehlbeck, Fol Leymarie, Deussen, Faraj: Using Saliency for Semantic Image Abstractions in Robotic Painting. Computer Graphics Forum 44(7), 2025. DOI
- Berio, Stroh, Calinon, Fol Leymarie, Deussen, Shamir: Neural Image Abstraction using Long Smoothing B-Splines. ACM Transactions on Graphics 44(6), 2025. DOI
- Berio, Clivaz, Stroh, Deussen, Plamondon, Calinon: Image-driven Robot Drawing with Rapid Lognormal Movements. IEEE RO-MAN 2025. DOI
- Stroh, Paetzold, Berio, Fol Leymarie, Kehlbeck, Deussen: Robotic Painting using Semantic Image Abstraction. Expressive 2025. DOI
Visits
The Atelier in Konstanz has been a meeting point for the computer science side and artists. Daniel Berio, a computer scientist and artist on the team, and two artists who are residents of the project, Anna Mirkin and Gretta Louw, visited the atelier.
Anna visited us for a few days at the end of June 2024, and we tested our abstraction framework on AI-generated images that suited Anna’s historical narrative. We then did early tests using a simplified painting agent that fills abstract regions in the picture on the canvas by fitting constrained vertical strokes into the shapes.
During this time, the shape-filling algorithm was still in the early stages of development. However, we produced early results for the constrained setting while clearing the general framework of Anna’s Project realization.
During Daniel Berio’s Visit, we operated his optimization-based image line work on one of our robots and achieved some nice results.
Together with Gretta Louw during her visit in early July 2024, parallel with Daniel Berio’s visit, we also settled on the general abstraction framework we want to use to realize her project proposal. During her stay, we focused on improving the robot-filling technique, as at that point, the region-filling system proved lacking for the elongated and curvy strokes that would fit well with the flowery patterns of the orchids. Regardless of the limitations, we painted two of her AI-generated orchid images onto stretched linen.
What came out of the visits
The region-filling techniques developed after Gretta Louw’s visit – in particular elongated, curvy strokes for the flowery orchid patterns – became the basis of her series Artificial Organisms and Animate Machines, in which the artist and e-David paint alternately on the same canvases: Gretta Louw paints the background and the robot the orchid petals. The works were exhibited at High Tech High Touch (Yvonne Hohner Contemporary, Karlsruhe, July 2025) and Code + Paint (DAM Projects, Berlin, March–April 2026). Gretta Louw also presented the outcomes of her residency at the EACVA exhibition and symposium in London in April 2026.
The collaboration with Daniel Berio continued beyond his visit and resulted in the joint publications listed above.