I have been using the new AI-image generation applications DALL-E and Midjourney to explore new forms of futuristic interfaces for a new personal project I am working on. These new AI-driven forms of image generation will be useful for sketches of different possibilities and directions when creating new art and design. Below are some examples, some I have colored with other architecture and design influences like Zaha Hadid, Dieter Rams, Syd Mead, etc…

When the COVID pandemic started and everyone was forced to shelter in place, especially in San Francisco, I started working on a project that would still allow for access to artwork at home. I wrote an application for the TV screen that would access some of the collection APIs that museums have been making available to the public. I used AirTable to store the APIs and some preferred tags related to certain genres (Japanese art at the Met for example), and curated a list of museums we would be interested in viewing at home – Cooper Hewitt Design Museum, The Met in NYC, the NASA image collection, etc…

The app is written in React and needed to resolve differences in the API data calls between the different museums, display larger versions of the images on the screen, allow a local user to pause, fast forward, or “save” an image they liked using a mobile web interface. I added some other features like coloring the matte according to a complimentary color in the artwork, and in the future I will programmatically control my receiver to launch and play the artwork at specified times.

I investigated the features of the Snap Lens studio to benchmark against some of the custom computer vision algorithms we wrote at frog for the SFMOMA project. It is impressive how performant and feature rich their computer vision systems have progressed using only RGB webcams for both facial tracking and full body tracking supported by ML. I created some custom frog snap lens filters for frog design.

Walking through the house one day in late in 2020, I came across an awesome drawing of a giraffe across three 8×11 sheets of paper by my daughter. I thought it would be a fun character to 3D model, animate and render in augmented reality using the original drawing as a marker for the AR animation.

In the Summer and Fall of 2020 we converted a bedroom into a shared office and bedroom space to better accommodate the new work from home normal. Features include a secret passage to former closet hidden behind a bookshelf, concealed piano keyboard slide mechanism and a custom space portal piece inspired by Star Wars now acting as a “skylight”.

As part of the celebration of the 50th anniversary of frog design, the firm asked frogs—both current and past—from around the globe to send their interpretations of the beloved friedolin the frog logo. My submission was in the medium of pancake art, which I have been aspiring to master.

The Interpretive Gallery we designed and developed for SFMOMA launched as part of the 2018 Magritte: The Fifth Season exhibition in 2018. After the launch we learned that the marketing department was counting visitor images shared on Instagram and other social networking sites by hand in order to understand how visitors were experiencing and sharing the space. We thought this process could be automated and build a POC using Tensorflow to count images shared at certain hashtags. We would download images shared at hash tags like #magritte and #sfmoma and were able to sort which ones were taken in the interpretive gallery with about 70% accuracy which would sort them from images shared at the hashtags from thousands of others taken around the world. We were also able to train the model to understand which of the six experiences in the space the image was take that would tell us how often the experiences were shared relative to one another.

The Eyeo Festival was a design, technology, data visualization, activist, interactive and creative code conference hosted in Minneapolis every year and brought some of the most talented artists, creators and technologists to the Twin Cities each summer. I have been in 2012, 2017 and 2018 and went to the previous incarnation called flashbelt in the mid-2000s as well. After the 2017 conference I presented about what Eyeo and creative technology was all about internally at frog and made a small data exploration into the topics and speakers from previous years to understand how the topic areas may have changed over time. The resulting data visualization is visible here. The meta data and tagging of the presentations in vimeo was somewhat inconsistent, so a future version could do keyword analysis on the presentation descriptions or the transcripts of the talks themselves to paint a more accurate picture.

In 2016 a frog colleague hosted a coffee time in celebration of Beyonce. In order to support her presentation, myself and another frog technologist created a Beyonce Hologram in honor of the event.

Unified Field was part of the developer preview program for the Leap Motion Controller and we considered using it for the Kennedy Space Center Exhibit. The official Leap Controller didn’t launch until the Summer of 2013 around the time the museum opened, so we weren’t able to use it there. Since then I created a demo by hooking it up to the Shuttle Landing Simulator I developed for Kennedy, and we set it up for our Unified Field booth at the 2013 ASTC tradeshow in Albequerque, New Mexico.

Users can advance through the interface by “tapping” their finger, originally there was a joystick with a button for this application. The shuttle flight path is controlled by moving one’s hand in the same manner an airplane would fly.