CHI '23 Research Highlights

by Kent Lyons, May '23

After going to a conference, like I just did with CHI '23 this year in Hamburg, Germany, I historically put together a high level overview of some of the stuff I saw that I thought was interesting in some way. Usually this took the form of a slide deck I'd present to my team. Last year when I started Inovo Studio, I turned that instead into a blog post and got some positive feedback, so I thought I'd continue that this year. Here are some highlights from the program! If you have any questions or comments, do reach out - I'd be happy to chat!


CHI was back to normal this year. Last year in New Orleans was the first in person meeting post pandemic, and while it was great to see a lot of people, attendance was understandably lower than normal. Not the case this year! In Hamburg there were approximately 3900 people in person! (and it felt that way which was awesome) and another 7-800 online. CHI was back to it's prepandemic self. The venue was also awesome - we fit in it very well (not just some small corner of a giant venue). And as normal, the CHI program is HUGE in so many ways so you can only ever see a slice of it. As such, these are just a few of the talks I saw. There is a list equally as long of papers that looked cool, but I could get to.

And CHI after hours was also in full swing with several organizations hosting events. I made it to the Chinese, French, UCSD, KAIST, Google, German and Dutch parties. There were several more I didn't make it to! And then I also had a bit of a chance to wander through Hamburg and managed to spend a few days in Berlin as well (but those are stories for a different forum!)

And now for the highlights:

Multi-Platform Content Creation: The Configuration of Creator Ecology through Platform Prioritization, Content Synchronization, and Audience Management

This paper was a qualitative study of how creators used multiple different platforms (N=21). They found 4 different themes these participants considered in using those platforms. Not too surprisingly they saw opportunities for support across platforms (things any one platform is likely not going to do), such has how to prioritize the different platforms, dealing with content across those platforms, and managing the various audiences.


Social Dynamics of AI Support in Creative Writing

This paper reflected on some past work about why an author might turn to a computer or a peer for feedback. They had very different responses in the past for different writing support tools and wanted to better understand what was going on. So they interviewed 20 creative writers and found 3 themes their authors are considering in looking for help: 1) what writers desire help with, 2) how writers perceive potential support actors, and 3) the values writers hold. For example, the author has a model of the person they're asking help of and might pick to get different feedback from different people based on their own perceived need.

We're going to see a lot more AI tools coming with the popularity of ChatGPT and the like, and I think this paper is a good example of how understanding potential considerations in a contextualized way will result in the need and use of systems in rather different ways (even if the tech under the hood might be very similar).


Flat Panel Haptics: Embedded Electroosmotic Pumps for Scalable Shape Displays

While touch screens have taken over as the dominant form of input (capacitive touch screens on mobile devices in particualr), we have lost some of the haptic affordances of physical input devices (eg keys). This paper explored a way for a flat surface to morph and prove that type of input when needed. The particular contribution of this work is an electroosmotic pump, but it was also shown in a cool functional prototype. The pump uses transitional PCB design processes and uses a charge in a PCM via to move liquid from a reservoir to a deformable area that can be expanded on demand.



EpoMemory: Multi-state Shape Memory for Programmable Morphing Interfaces

This paper examines shape changing physical structures based on applying different levels of heat (eg temperature dependent shapes). In particular they use epoxy and the fact that differing ratios of the crosslinker and epoxy resin result in an epoxy becoming deformable at different temperatures. This work also is representative of a trend in the community to explore more of the material side of devices and how that might influence interaction. (We've had a somewhat similar discussions in the wearables community about the role of computation in ISWC publications).


Toucha11y: Making Inaccessible Public Touchscreens Accessible

This was a cool device/system targeting accessibility on public touch screens. However beyond that aspect, I thought the little robot which used a tape measure as a 1d actuator, and the use of a camera for localization on the touch screen were cool ideas.


Assisting with Fingertip Force Control by Active Bio-Acoustic Sensing and Electrical Muscle Stimulation

Sensing the fingers while not getting in the way of the fingers is a hard problem (and one I've done some work in). This paper uses bio-acoustic sensing to estimate fingertip force. In this case, they wanted a non-electrical solution since they're using electrical muscle stimulation. For the sensing, they use a piezo speaker and mic on the back of the hand and swept from 20khz to 40khz. They then used a regression model to estimate pinch force.


Understanding the Perception of Human Augmentation: A Mixed-Method Study

This paper used a very small sample size, but in 4 different countries to look at perceptions of potential human augmentations. They used some intentionally vague vignettes to elicit responses. They also did a followup with with N=500 participants by physical mail (not online).

I think this is a really interesting area to explore, but also a challenging one. People are not very good at this type of speculation as they have a hard time understanding how it might actually impact their life. A slightly different framing or value proposition might drastically shift results. Likewise, things like marketing or social proof can have huge influences. How can we learn things that will hold in a hypothetical future, unknown world? Or is that not really possible? Another confound is co-evolution of the tech and people's usage/expectations so even if we start in one place, after a few iterations we might be somewhere else entirely.

It is Okay to be Distracted: How Real-time Transcriptions Facilitate Online Meeting with Distraction

This paper looked at the challenges of distraction during online meetings and the possible role of transcripts (full, summary and keyword) as a way to help reengage with the meeting content. Some of my early research was looking at how to use wearables to augment an in person face-to-face conversation and it is an area I've recently been thinking a bit more about. Leveraging the post-pandemic world of video conferencing might be a way to investigate some related topics in an easier way and this paper is in that vein.


Physically Situated Tools for Exploring a Grain Space in Computational Machine Knitting

This paper is coming from a line of work exploring computational knitting. The primary contribution of this paper was the notion of a "grain" (somewhat analogous to wood grain). This idea is to have and leverage some intermediate property of the material (here knitting direction) as a somewhat emergent property of the material when viewed in aggregate.


MechCircuit: Augmenting Laser-Cut Objects with Integrated Electronics, Mechanical Structures and Magnets

This work explore fabricating interactive structures from laser cut 2d material (PMMA plastic). The developed a pipeline that involves laser cutting the material and use the paper protective layer of the plastic as a resit for creating traces with sprayed conductive ink. Magnets embedded in the plastic serve as both mechanical and electrical connections. The authors have a library of basic structures and software to support the design.


FlexBoard: A Flexible Breadboard for Interaction Prototyping on Curved and Deformable Surfaces

This paper created a 3d printable breadboard that can flex in 1d (both concave and convex). The idea was to support prototyping on curved surfaces. The mechanical structure is cool and they conducted technical evaluations and presented some case studies of usage.


AeroRigUI: Actuated TUIs for Spatial Interaction using Rigging Swarm Robots on Ceilings in Everyday Space

Ceiling robots! These are something I've been thinking on and off about for awhile (first when I was at Technicolor when we were brainstorming robotic uses in the home). They also explore using the ceiling robots as cable robots (like skycam for you (American) football fans). They use an off the shelf robot and modify it with a cable. And they "cheat" with a magnetic ceiling which allows the robots to drive across the ceiling - a very clever trick. I'd thought about making the robots positively buoyant (helium balloons) to achieve a similar effect yet work on more "normal" ceilings. I'd still like to build that - if you are interested (and see this!), reach out.


SkinPaper: Exploring Opportunities for Woven Paper as a Wearable Material for On-Skin Interactions

It's great to see another cool paper out of Cindy's lab exploring on-skin interfaces. This paper looked at using washi paper to create woven conformal structures on the body. They did some characterizations of the material they created as well as some day in the life usage experiments.


InStitches: Augmenting Sewing Patterns with Personalized Material-Efficient Practice

This paper was motivated by the challenge of teaching a craft or process where there are non-trivial costs in terms of time or materials in doing the needed practice. This issue applies to a wide range of things in the space of making. Here they look specifically at sewing clothes from patterns. They built an interactive system that inserted different options for practice as the user was guided through the process of making a clothing items. (The tool also helped with material layout. This was needed since the practice needed more material, but is likely a useful feature by itself to help encourage usage). They explored a couple of different practice methods and did a test with 8 people from a sewing club. Seems like a really cool area to extend both in terms of modeling what practice is needed as well as to different domains.


Polagons: Designing and Fabricating Polarized Light Mosaics with User-Defined Color-Changing Behaviors

There is an art form which use polarizers to create an image that is dynamic - changes when the polarizers are moved. Currently this is very tedious with a lot of trial and error. This project built a design tool (that goes all the way down to the physics of polarization) to make the process much easier. In the category of very cool, but I have no idea what to do with it!


LaserShoes: Low-Cost Ground Surface Detection Using Laser Speckle Imaging

There have been a few works over the years using laser speckles as a sensing mechanism. The idea is a laser is shown on a surface and the micro structure of the surface creates constructive and destructive interference resulting in a unique speckle pattern. That pattern can then be picked up with an optical sensor (this is part of the principle behind optical mice). In this paper, they explore using this phenomena with a system mounted on the user's shoe to detect different surfaces. There have been a few "shoe" papers over the years in the wearable community but I still think it's an under explored area and fun to see another paper in this space. And speckle sensing is also cool! They used a fine tuned resnet-18 model for classification.


EchoSpeech: Continuous Silent Speech Recognition on Minimally-obtrusive Eyewear Powered by Acoustic Sensing

This paper is doing lip reading from a user's own glasses using active acoustic sensing (18khz - 24.5khz). They use a set of 2 speakers and 2 mics and used resnet 18. They were able to do discreet recognition out of a vocabulary of 31 items and continuous recognition on digits. In a study with 24 participants they had a word error rate of 4.5% for discreet and 6.1% for the continuous digits.


HOOV: Hand Out-Of-View Tracking for Proprioceptive Interaction Using Inertial Sensing

This work considers how to do hand tracking when the users move their hands out of the field of view of a camera (eg for an AR headset that does inside out tracking). When the hand goes out of the field of view, it switches over to a wrist worn IMU. It uses a NN to compute position and orientation. Ground thruth training data was from an optitrack.


OmniSense: Exploring Novel Input Sensing and Interaction Techniques on Mobile Device with an Omni-Directional Camera

This paper explored using an omni direction camera on a phone as a sensor for a bunch of interaction techniques developed in the past (using a variety of sensing techniques). This was likely a lot of engineering effort to reimplement all of these techniques, but it is cool to see them be unified in this way.


IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds

This work explores detecting body pose with a few IMUs placed around the body. One of the interesting things this work did was use synthetic IMU data so they could easily test various on body sensor positions. They also did some evaluations on existing datasets and open sourced their code.


MechSense: A Design and Fabrication Pipeline for Integrating Rotary Encoders into 3D Printed Mechanisms

This paper explored creating a device that could sense its rotary movement but was fabricated using 3d printing. It uses a floating (unconnected) capacitive layer in the rotar, and the stator has 3 conductive patches. By energizing different patches, the capacitance varies with angle.


AdHocProx: Sensing Mobile, Ad-Hoc Collaborative Device Formations using Dual Ultra-Wideband Radios

Collocated multi-device interaction (Eg two tablets) has been an ongoing area of research for a good while (I did some work in this space as well). Getting stuff from one device to another is still a real pain! Getting the devices on the same network and inferring spatial proximity/orientation are still not things our commodity devices do. This work used two UWB radios to form an ad hoc network and detect location. They also use some capacitive sensing to overcome interference with the UWB radios.

This is one of those areas where it's still hard to believe we don't have a good solution widely deployed in our devices - kinda like getting a random laptop and projector to work reliably! In homogeneous environments we're starting to see solutions on the display sharing side, but full cross device interaction is still illusive.

Ceramics and Pottery

I've been doing pottery for fun for awhile now and there were a few papers in this space at CHI this year. I've thought a bit about possible research contributions in this space, and after seeing these papers, I'll need to dig deeper!

Crafting Interactive Circuits on Glazed Ceramic Ware

This paper explored adding electronic circuits onto the outside of a finished ceramics pot. They collaborated with some ceramicists and were inspired by a process whereby a "finshed" pot, one that has already been through a glaze firing, is further processed. An artist applies a stencil and sandblasts off the glaze, exposing the underlying raw ceramic creating a different finish. In this paper, they adapt this process to create traces in the glazed pot and then apply a conductive ink into the resulting exposed areas.


CoilCAM: Enabling Parametric Design for Clay 3D Printing Through an Action-Oriented Toolpath Programming System

The next paper I saw in the theme of pottery was also a collaboration with potters and in this case they took inspiration from coil building. Normally this is done by rolling out coils of clay into a thin and long cylinder. Those are then stacked and pressed together to create a form. This is a bit similar to how clay 3D printers work that can be purchased off the shelf. Instead of extruding molten plastic, they extrude clay, layer by layer. The authors of this work wanted to have a design process more familiar to potters. So instead of working on a 3D model in cad, they explored a tool to manipulate individual slices as extruded by the clay printer.


3D Printable Play-Dough: New Biodegradable Materials and Creative Possibilities for Digital Fabrication

The last paper I saw in the realm of pottery (there might have been more I didn't see) was this one by Leah Buechley. I saw the demo during the interactivity and was curious what the contributing of the paper was about. Here they are not using clay but instead home made play dough. A lot of the paper is about the process of creating the material and associated challenges, while Leah's talk was more about the material itself and it's affordances such as the ability to reuse it or do rapid prints since the layer height is so large.

Some of the cool things they did was support color gradients, layering different colors in the extrusion tube and some software support for that. They also dealt with the challenge of getting constancy right using a penetrometer. They used an off the shelf clay 3d printer (Eazao Zero ~$1000). One of the areas of future work was to use this a support material for clay to print more complex shapes (this would burn off in the kiln).


Carbon Neutrality

My old team from TRI had a couple of papers on carbon neutral decisions as they relate to vehicles.

Understanding People's Perception and Usage of Plug-in Electric Hybrids

Do people plug in their plugin hybrid cars? What challenges do they face? In the US at least, yes they do and the real challenges seem to be more on the public infrastructure side to charge away from home.


Save A Tree or 6 kg of CO2? Understanding Effective Carbon Footprint Interventions for Eco-Friendly Vehicular Choices

This work is about how to communicate CO2 usage and how that might impact end users' understanding/decisions. When CO2 usage is presented they are more likely to chose the green option, but are still price sensitive. It also seems people don't really understand CO2, and they were making relative judgments. Framing (social and valence) can also impact choices.