CHI '23 Research Highlights

CHI was back to normal this year. Last year in New Orleans was the first in person meeting post pandemic, and while it was great to see a lot of people, attendance was understandably lower than normal. Not the case this year! In Hamburg there were approximately 3900 people in person! (and it felt that way which was awesome) and another 7-800 online. CHI was back to it's prepandemic self. The venue was also awesome - we fit in it very well (not just some small corner of a giant venue). And as normal, the CHI program is HUGE in so many ways so you can only ever see a slice of it. As such, these are just a few of the talks I saw. There is a list equally as long of papers that looked cool, but I could get to.

And CHI after hours was also in full swing with several organizations hosting events. I made it to the Chinese, French, UCSD, KAIST, Google, German and Dutch parties. There were several more I didn't make it to! And then I also had a bit of a chance to wander through Hamburg and managed to spend a few days in Berlin as well (but those are stories for a different forum!)

And now for the highlights:

https://dl.acm.org/doi/10.1145/3544548.3581106

This paper was a qualitative study of how creators used multiple different platforms (N=21). They found 4 different themes these participants considered in using those platforms. Not too surprisingly they saw opportunities for support across platforms (things any one platform is likely not going to do), such has how to prioritize the different platforms, dealing with content across those platforms, and managing the various audiences.

https://dl.acm.org/doi/10.1145/3544548.3580782

This paper reflected on some past work about why an author might turn to a computer or a peer for feedback. They had very different responses in the past for different writing support tools and wanted to better understand what was going on. So they interviewed 20 creative writers and found 3 themes their authors are considering in looking for help: 1) what writers desire help with, 2) how writers perceive potential support actors, and 3) the values writers hold. For example, the author has a model of the person they're asking help of and might pick to get different feedback from different people based on their own perceived need.

We're going to see a lot more AI tools coming with the popularity of ChatGPT and the like, and I think this paper is a good example of how understanding potential considerations in a contextualized way will result in the need and use of systems in rather different ways (even if the tech under the hood might be very similar).

https://dl.acm.org/doi/10.1145/3544548.3581547

While touch screens have taken over as the dominant form of input (capacitive touch screens on mobile devices in particualr), we have lost some of the haptic affordances of physical input devices (eg keys). This paper explored a way for a flat surface to morph and prove that type of input when needed. The particular contribution of this work is an electroosmotic pump, but it was also shown in a cool functional prototype. The pump uses transitional PCB design processes and uses a charge in a PCM via to move liquid from a reservoir to a deformable area that can be expanded on demand.

https://dl.acm.org/doi/10.1145/3544548.3580638

This paper examines shape changing physical structures based on applying different levels of heat (eg temperature dependent shapes). In particular they use epoxy and the fact that differing ratios of the crosslinker and epoxy resin result in an epoxy becoming deformable at different temperatures. This work also is representative of a trend in the community to explore more of the material side of devices and how that might influence interaction. (We've had a somewhat similar discussions in the wearables community about the role of computation in ISWC publications).

https://dl.acm.org/doi/10.1145/3544548.3581254

This was a cool device/system targeting accessibility on public touch screens. However beyond that aspect, I thought the little robot which used a tape measure as a 1d actuator, and the use of a camera for localization on the touch screen were cool ideas.

https://dl.acm.org/doi/10.1145/3544548.3581192

Sensing the fingers while not getting in the way of the fingers is a hard problem (and one I've done some work in). This paper uses bio-acoustic sensing to estimate fingertip force. In this case, they wanted a non-electrical solution since they're using electrical muscle stimulation. For the sensing, they use a piezo speaker and mic on the back of the hand and swept from 20khz to 40khz. They then used a regression model to estimate pinch force.

https://dl.acm.org/doi/10.1145/3544548.3581485

This paper used a very small sample size, but in 4 different countries to look at perceptions of potential human augmentations. They used some intentionally vague vignettes to elicit responses. They also did a followup with with N=500 participants by physical mail (not online).

I think this is a really interesting area to explore, but also a challenging one. People are not very good at this type of speculation as they have a hard time understanding how it might actually impact their life. A slightly different framing or value proposition might drastically shift results. Likewise, things like marketing or social proof can have huge influences. How can we learn things that will hold in a hypothetical future, unknown world? Or is that not really possible? Another confound is co-evolution of the tech and people's usage/expectations so even if we start in one place, after a few iterations we might be somewhere else entirely.

https://dl.acm.org/doi/10.1145/3544548.3580742

This paper looked at the challenges of distraction during online meetings and the possible role of transcripts (full, summary and keyword) as a way to help reengage with the meeting content. Some of my early research was looking at how to use wearables to augment an in person face-to-face conversation and it is an area I've recently been thinking a bit more about. Leveraging the post-pandemic world of video conferencing might be a way to investigate some related topics in an easier way and this paper is in that vein.

https://dl.acm.org/doi/10.1145/3544548.3581434

This paper is coming from a line of work exploring computational knitting. The primary contribution of this paper was the notion of a "grain" (somewhat analogous to wood grain). This idea is to have and leverage some intermediate property of the material (here knitting direction) as a somewhat emergent property of the material when viewed in aggregate.

https://dl.acm.org/doi/10.1145/3544548.3581002

This work explore fabricating interactive structures from laser cut 2d material (PMMA plastic). The developed a pipeline that involves laser cutting the material and use the paper protective layer of the plastic as a resit for creating traces with sprayed conductive ink. Magnets embedded in the plastic serve as both mechanical and electrical connections. The authors have a library of basic structures and software to support the design.

https://dl.acm.org/doi/10.1145/3544548.3580748

This paper created a 3d printable breadboard that can flex in 1d (both concave and convex). The idea was to support prototyping on curved surfaces. The mechanical structure is cool and they conducted technical evaluations and presented some case studies of usage.

https://dl.acm.org/doi/10.1145/3544548.3581437

Ceiling robots! These are something I've been thinking on and off about for awhile (first when I was at Technicolor when we were brainstorming robotic uses in the home). They also explore using the ceiling robots as cable robots (like skycam for you (American) football fans). They use an off the shelf robot and modify it with a cable. And they "cheat" with a magnetic ceiling which allows the robots to drive across the ceiling - a very clever trick. I'd thought about making the robots positively buoyant (helium balloons) to achieve a similar effect yet work on more "normal" ceilings. I'd still like to build that - if you are interested (and see this!), reach out.

https://dl.acm.org/doi/10.1145/3544548.3581034

It's great to see another cool paper out of Cindy's lab exploring on-skin interfaces. This paper looked at using washi paper to create woven conformal structures on the body. They did some characterizations of the material they created as well as some day in the life usage experiments.

https://dl.acm.org/doi/10.1145/3544548.3581499

This paper was motivated by the challenge of teaching a craft or process where there are non-trivial costs in terms of time or materials in doing the needed practice. This issue applies to a wide range of things in the space of making. Here they look specifically at sewing clothes from patterns. They built an interactive system that inserted different options for practice as the user was guided through the process of making a clothing items. (The tool also helped with material layout. This was needed since the practice needed more material, but is likely a useful feature by itself to help encourage usage). They explored a couple of different practice methods and did a test with 8 people from a sewing club. Seems like a really cool area to extend both in terms of modeling what practice is needed as well as to different domains.

https://dl.acm.org/doi/10.1145/3544548.3580639

There is an art form which use polarizers to create an image that is dynamic - changes when the polarizers are moved. Currently this is very tedious with a lot of trial and error. This project built a design tool (that goes all the way down to the physics of polarization) to make the process much easier. In the category of very cool, but I have no idea what to do with it!

https://dl.acm.org/doi/10.1145/3544548.3581344

There have been a few works over the years using laser speckles as a sensing mechanism. The idea is a laser is shown on a surface and the micro structure of the surface creates constructive and destructive interference resulting in a unique speckle pattern. That pattern can then be picked up with an optical sensor (this is part of the principle behind optical mice). In this paper, they explore using this phenomena with a system mounted on the user's shoe to detect different surfaces. There have been a few "shoe" papers over the years in the wearable community but I still think it's an under explored area and fun to see another paper in this space. And speckle sensing is also cool! They used a fine tuned resnet-18 model for classification.

https://dl.acm.org/doi/10.1145/3544548.3580801

This paper is doing lip reading from a user's own glasses using active acoustic sensing (18khz - 24.5khz). They use a set of 2 speakers and 2 mics and used resnet 18. They were able to do discreet recognition out of a vocabulary of 31 items and continuous recognition on digits. In a study with 24 participants they had a word error rate of 4.5% for discreet and 6.1% for the continuous digits.

https://dl.acm.org/doi/10.1145/3544548.3581468

This work considers how to do hand tracking when the users move their hands out of the field of view of a camera (eg for an AR headset that does inside out tracking). When the hand goes out of the field of view, it switches over to a wrist worn IMU. It uses a NN to compute position and orientation. Ground thruth training data was from an optitrack.

https://dl.acm.org/doi/10.1145/3544548.3580747

This paper explored using an omni direction camera on a phone as a sensor for a bunch of interaction techniques developed in the past (using a variety of sensing techniques). This was likely a lot of engineering effort to reimplement all of these techniques, but it is cool to see them be unified in this way.

https://dl.acm.org/doi/10.1145/3544548.3581392

This work explores detecting body pose with a few IMUs placed around the body. One of the interesting things this work did was use synthetic IMU data so they could easily test various on body sensor positions. They also did some evaluations on existing datasets and open sourced their code.

https://dl.acm.org/doi/10.1145/3544548.3581361

This paper explored creating a device that could sense its rotary movement but was fabricated using 3d printing. It uses a floating (unconnected) capacitive layer in the rotar, and the stator has 3 conductive patches. By energizing different patches, the capacitance varies with angle.

https://dl.acm.org/doi/10.1145/3544548.3581300

Collocated multi-device interaction (Eg two tablets) has been an ongoing area of research for a good while (I did some work in this space as well). Getting stuff from one device to another is still a real pain! Getting the devices on the same network and inferring spatial proximity/orientation are still not things our commodity devices do. This work used two UWB radios to form an ad hoc network and detect location. They also use some capacitive sensing to overcome interference with the UWB radios.

This is one of those areas where it's still hard to believe we don't have a good solution widely deployed in our devices - kinda like getting a random laptop and projector to work reliably! In homogeneous environments we're starting to see solutions on the display sharing side, but full cross device interaction is still illusive.

I've been doing pottery for fun for awhile now and there were a few papers in this space at CHI this year. I've thought a bit about possible research contributions in this space, and after seeing these papers, I'll need to dig deeper!

My old team from TRI had a couple of papers on carbon neutral decisions as they relate to vehicles.