6-Course Dinner in Virtual Reality

The Story

I still remember my first brush stroke in Tiltbrush. Absolute elation. Finally, a way to show what has been in my head for all of these years! I have been painting in Tiltbrush since the Summer of 2017 when I bought my HTC Vive. Tiltbrush was the first VR app I purchased. Since then I have been exploring the medium almost daily if possible. I belong to multiple groups on facebook related to virtual reality, augmented reality, art in virtual reality and 360 films.
Chef Jenny Dorsey asked in one of the Facebook groups if anyone was painting in Tiltbrush. I responded that I was. A couple of weeks later, she contacted me to see if I would be available to create food illustrations in VR. She was interested in creating an immersive experience for a dinner she was hosting in New York at the Museum of Food and Drink.
Jenny’s idea was to show the dinner guests a 360 video of her creating her 6-course meal while she created the 6-course meal in the real world. The guests would watch the experience while they smelled the appetizing aromas of her real food. I love painting food and eating wonderful food so this seemed like the project for me.
This project began with multiple conversations, followed by emails with more details. The paintings would be rendered as a 360 video showing each brush stroke to create an animated sequence of the full 6-course dinner. I watched several videos showing Jenny preparing her courses. This was a great help since I had not seen her prepare the meals in person. She sent me detailed instructions of the meals with photo references.

Sketching

I began by sketching the dishes on paper so that I could create story boards to understand the order of the dishes. Jenny was able to see what I was thinking before I began creating in Tiltbrush.

One of the biggest challenges was creating the illustrations in the proper order so that the brush strokes would render in the proper sequence. I quickly began working directly in Tiltbrush so that Jenny could visualize the illustrations and we could adjust the style of the animations. I created each animation and had Jenny approve the final images and then began recreating the illustrations so that the final order would be correct.

The biggest challenge of this project was rendering the illustrations as 360 videos with the brush stroke sequence animated. I spent many hours researching how to create this effect. I posted on multiple forums and Facebook pages to find the answer to these technical issues. The answer finally came through the Tiltbrush forum. Tiltbrush had changed their interface to make room for other functionalities and had eliminated the “NoQuickload” command. Ultimately, a Tiltbrush/Google representative posted detailed instructions on how to render this effect. A developer friend helped me with the command line functions so I was able to set up the command line for the rendering. I had to change the .bat file by adding the “NoQuickLoad” command into the correct command line so that I could render the illustrations with the brush strokes “painting” the illustrations.
I created 3 separate animations. Each final illustration required about 5 iterations. Jenny took my illustrations and incorporated them into a final 360 video with her narrative and poetry. The final video is up for The Bridging the Divide Award with the Future of Storytelling https://futureofstorytelling.org/project/asian-in-america.