Some of my recent design projects spanning native mobile, web, TV OTT, and more.
These are some of my ongoing AR & VR concept projects. Most of these are work in progress, so I'll be posting updates often!
VR app for Google Cardboard
The app is based on the premise that a user could easily explore the inside of cars from the comfort of their own living room. Since this is a demo app, it has the basic functionality of selecting a car from a list or gallery and visualizing the inside of that car in 360˙.
I decided to keep the app simple, so limiting the gallery to six cars seemed like a good proof of concept. Allowing the user to click on a car from the main menu and port to the interor of the car so they can view the interior in full 360˙ was the main focus of the experience. Then to add interactivity, I placed buttons in key areas of the car interior where panels would pop up with small chunks of info.
ARKit (iOS & Unity)
I thought that working on a navigatable map would be a fun start to experiment with ARKit using the Unity 3d plugin. As the project goes on, I'll see what items and functionality I can try adding to the scene.
I researched available 3d map model systems and found a couple that were the most notable:
Mapbox is a mapping tool that can be integrated with Unity, iOS, and Android via their SDKs. Navigation and location services are capable, as well as visual customizations. I followed these tips on how to integrate their maps with ARKit.
WRLD is also a mapping tool that can be integrated with Unity, iOS, and Android via SDKs. The maps are interactive and customizable. Their maps have a nice out-of-the-box look with extruded buildlings and a flyover animations that are popular for games. There are great tips on ARKit integration on here: "How to use ARKit with the WRLD Unity SDK"
I was able to follow the Mapbox blog post to learn the logic about setting up terrains, but I found that using one of their sample scenes was a quicker way to get started. I selected Downtown LA as my area (via coordinates). Then I imported the Unity ARKit plugin to the scene (from the Unity Asset Store) to add in AR functionality. Making adjustments to the map tile size and y-height can make the map can feel small in scale or huge like walking through the city. After a few tries, it was working!
Next, I dropped in a car asset. My first attempt was to try adding it as a Hit item (tap screen to place). The results were pretty amusing – the car looked like a giant flying car! Then I’d tap the screen again and it would place itself on the carpet below the map. Clearly, this wasn’t exactly the solution I was looking for, but I had a Hit tap items working for future reference and it made for some laughs.
The hit tap to place a car model isn't quite right, but the map is working out nicely.
I changed my tactic and decided to make the car sit on the map surface rather than add it by a tap. I added a rigid body to the car and a box collider. And I needed to add a collider to the map as well (or else the car would fall straight through the terrain). It took a little research, but found that in the Mapbox Flat Terrian Factory, you can check the “Add Collider” checkbox to set this.
The map and car settings I used in Unity to allow collsions.
Bingo! I now had a car placed on the map terrain surface. It’s out of proportion and isn't really sitting on a street (it's stuck half inside a building here) but I can adjust that later. The important part is that the car model is now on the surface of the map.
A car placed on the surface of the map.
I decided to make the scene a little more interesting, so I made some weather elements in Blender. These low poly models were really easy to whip up. I made a couple versions of clouds and a sun and then inserted them into my scene in Unity.
One of my weather models made in Blender.
I wanted a little movement on the clouds, so I added a script to make each model spin around its y-axis using Vector3.Up
, otherwise known as Vector3(0, 1, 0)
. By making the speed a public property, I could then tinker with the speed if I wanted to. To start, I set everything to a default speed of 5f
and added the script to each of the weather elements.
Rotating weather elements in the scene.
The default directional light was creating a nice shadow onto the map, but I thought I’d try to experiment with a sun and moon cycle and see if I could make the lighting more realistic. I made new “Sun” and “Moon” objects by copying the default Directional Light objects and changing out the colors to a yellow sunlight and a dark blue for moonlight.
I created a new script called “suncycle” that would control the light movement. The sun and moon objects are positioned at 180˙ from each other, facing inward, and they move in a cycle to rotate around the plane of the map. I decided to add in a public property for the speed so I could adjust it on the fly in Unity. I added this script to both the sun and moon objects.
It needs some tweaking, but it's pretty cool watching the sunset and moonrise in LA.
Here's my result for the Part 1. I have a giant map where I could walk around the city like Godzilla. The (too large) car is anchored nicely to the map. The spinning weather models hang above the scene and cast shadows onto the map. And, lastly, the sun and moon rotate around the map to imitate a day/night cycle.
Walking around my huge map with a car and weather elements and a day/night cycle.
Coming soon: Learning to Lerp and Polylines
A game concept for Oculus Rift
In a previous life I worked in the biotech / pharmaceutical / medical device world as a researcher. Even though I'm a designer now, research is still interesting to me. It's the combination of these two worlds that sparked my inspiration to create a fun and educational replication of life in the laboratory. In the experience, a player is ported into a research lab where they're taken through the steps to carry out common lab exercises and assays. Along the way we can make it fun too! Maybe they’d run into other lab folk, like stressed out Postdocs. They could interact with kiosks around the lab that provide additional understanding or background info. Providing real-life examples for the science behind the protocols would reinforce the educational angle.
Design Start Date: October 2016
Status: Ongoing personal project (Experimental)
Role: Lead Designer and Developer
Design/Dev Tools: Unity 3d, Oculus Rift, Sketch
Pretty close to what my lab bench always looked like. (Image courtesy of Northeastern University)
I wanted to get some ideas for how I would set up the laboratory. Benches, shelving, and equipment fill the room. Researchers are conducting experiments. References for typical postures and tactile grips of pipettors and test tubes.
Some of the sights in a typical biology laboratory.
Using tissue (cell) culture in experiments seemed like a good place to start with an introductory experience. I made a quick series of sketches to map out the basic steps for growing cells in culture, how they can be plated for an assay, and how to harvest the cells and use them for a variety of assays.
A basic experiment setup with cell cultures.
I chose to put together a simple prototype of the main menu using the Facebook VR Prototyping Tool . The process is simple, I created a menu layout using the Sketch template and exported the image. Then I placed the image into the example Unity project. From there, you can open the Unity project and play or export to Rift or Gear VR. I’ll most likely make the menu experience more interactive, rather than just a flat layer of choices, but this was a good proof of concept.
A quick prototype of a main menu.
Rather than take the time to create 3d assets so early in the process, I found models of some basic laboratory items in the Unity Asset store. I can use these to work with placement and interaction scripts. I also used an existing room scene from the Asset store which already had tables, lamps, and.. er, Christmas decor (you'll see me randomly pick up and toss a Christmas tree later on, so that's why...)
Sample 3d laboratory objects.
I placed an FPSController prefab in the scene, removed the character controller, added a capusle, and moved the Main Camera as a child of the prefab. As a child of the Main Camera, I added animated cartoon hands that I downloaded from the Unity asset store. (Note that this was done before the Oculus Touch and Avatar SDK were released.)
The scene setup with FPSController and random 3d test objects.
I created a script calledPickupObject
and placed it on the FPS Controller. To test proof of concept I used keyboard controls where the player would use the “E” key on the keyboard to pick up an object and then use “E” again to drop object.
My results so far – setting up the lab scene and picking up objects. (The hands still need some work... I'm really clumsy.)
There's still a lot more to be done! I'll add in Oculus Touch and Avatar functionality (this was launched after I started the project). I'll also refine the appearance of the room and start storyboaring the tutorial experience, including how to communicate and guide the player.
Apple TV and Android TV
With a large portfolio of personalized video content, Yahoo had the opportunity to collect and bring this to users in a TV format. Content spans Yahoo Sports, Yahoo Finance, our family of brands such as HuffPost, Techcrunch, and more. The Yahoo TV app would consolidate our video content into one OTT app. Along with user's preferences, it would deliver a personalized experience to help people watch video content that is most important to them. In the future, it could also sync with Yahoo web and our native mobile Yahoo app to provide all the same features plus other content such as live NFL games. It was a rather large project, but I'll highlight some of the design decisions and processes below.
Our group identified some general people problems with watching video on an OTT device and generated solutions that we wanted to explore:
These people problems were framed by a combination of industry research, early user research, and product offerings that were unique to Yahoo.
Most streaming OTT apps are structured as thumbnails in rows of categories such as Recommended for You, My List, or Trending Comedies. Users scroll through the grid and select content to watch. Any selected video is often linked to other videos in that series that keep playing related content. This is called a Video on Demand (VOD) structure. VOD grids and rows are the most common navigation pattern among OTT apps. While users are accustomed to this hunt and peck style of decision making, it also causes friction for many people. It can take up to 18 minutes on average to find something to watch. Also, couples argue at a higher rate over what to watch (up to 40% compared to 30% on traditional cable TV).1 Really, who among us hasn't experienced this themselves? We identified this as a people problem to tackle.
Sample Video on Demand (VOD) grid style navigation (Netflix).
In contrast, the Virtual Linear Channel (VLC) is a playlist filled with pre-programmed content supports long form content. It's similar to a lean back TV experience that requires little user interaction. The playlist often starts playing in small format after launching the app and then expands to full screen.
In a Virtual Linear Channel (VLC) videos are stacked end-to-end to continuously play content (Reuters TV).
Knowing a lot about Yahoo user's preferences — like their favorite sports teams, celebrities they follow, and stock tickers they monitor — meant that there was opportunity for us provide content they enjoy without the high churn of decision making often seen in traditional VODs. Our goal was to let a user simply launch the app and start watching videos in their channel with minimal effort.
Combining both VLC and VOD models for the Yahoo TV App.
I took over the project just as the MVP version was being launched. It featured the linear channel, a channel playlist viewer, a simple topics manager (no ability to follow new topics), and a VOD grid. A preliminary user test of the experience verified some of my early thoughts — the current user experience was confusing.
When launching the app and their “channel” started playing, most people seemed confused about what had just happened. They usually missed the onboarding video because they were busy signing in to Yahoo with their mobile device to initiate the signed in experience. Improving the first time onboarding flow is on the list of designs to improve for next version of the app.
Additionally, navigating the app was very heavy. Viewing the channel sections and managing topics seemed to require a cognitive load on users. The question “Where am I?” came up again here when navigating. In my redesign, I approached this with simplification and UI improvement in mind.
Main screens that were already designed in the Yahoo TV app (MVP version).
After the user launches the app, they should be able to start watching right away with minimal interaction. A side flow should allow them to explore the VOD grid and choose a different video to watch. Accessing this functionality through the main menu made the most sense. The user should always be able to view upcoming playlist videos in a Channel Playlist tray.
Early UX exploration for navigating VLC and VOD playlists.
We took a critical look at all parts of the existing designs and identified missing pieces and potential improvements for each section of the app. I subdivided the app into: Main Menu, VOD Grid, Channel Playlist Tray, Video Player, Following Topics, Welcome, Onboarding, Sign In, and Settings.
There's quite a few pieces to the app, so I'll share the Channel Playlist as an example of the process we used across Product, Engineering, and Research teams to analyze the features, to identify pain points, incorporate user and industry research, and work to align across XFN partners on a mission.
The Channel Playlist is a tray that gives users a preview of upcoming VLC content with a quick swipe up on the trackpad. The channel is organized by categories (i.e. Finance, Sports, etc.) and within each category are the individual topics (i.e. Golden State Warriors, Amazon, etc.) Within each topic were one or more videos in a playlist.
Original Channel Playlist design needed improvement.
Users understood the concept of a playlist tray but had a difficult time grasping the content that they were seeing. It felt kind of "crude" in ways that people couldn't quite put their finger on and "not very useful" in testing. The interactions and visuals were in need of design improvement. Here's how I tackled it:
Directions we aligned on exploring as a team included the navigation between topics, adding meaningful category grouping, improving the card visuals, and addressing the lack of card/topic/video context. Some open-ended questions remained around selecting individual videos. We incorporated the people problems we initially identified, namely making videos easier to watch and showing content that means most to people.
I first approached the design with sketches. Navigating sections and layers of information was the primary direction.
Snapshot of early UX ideas to create channel sections, grouping content, and navigating a playlist tray.
Designing for the people scenarios:
"I want to skip everything and just watch my Finance news." or "I want to see a specific video about Constance Wu's new movie."
I felt strongly about exploring a version that would allow users preview every video that is coming up in their playlist and, if they chose to, they could select a specific video to watch. I made the topic categories more prominent and navigatable so people could skip between groups of topics easily.
UX explorations – navigating the Channel Playlist by categories and drilling into video thumbnails.
Designing for the people scenario:
"I want to catch up on news from last night's Warriors game."
This series focuses on selecting a topic as the main action. Categories and discrete videos were not emphasized here so that the focus in on the topic. By selecting a topic, the first video in that playlist would start.
UX explorations – navigating the Channel Playlist by topic.
Taking the positive feedback from the topics variation (Exploration 2), I revised the design with the intent on better conveying that there is video content within each topic. I created cards to represent each topic and all the videos in that topic. I used the first video's static image to fill the card's frame. I started to experiment with relevant metadata like number of videos and time stamps. Our XFN team aligned on this version.
Navigating the Channel Playlist by topic with image based cards.
Further explorations included the design of the topic cards with a progress indicator and cleaning up the look and feel of the tray. The cards needed to have a title (topic name) and metadata to help the user understand the content. I added a simplified time stamp for each category as reference point for estimating watch time - a time stamp on each topic card felt crowded and overly verbose. Each card shows the number of videos and the topic title. I opted to feature our new Yahoo Sans font in extra bold, matching the new design theme rolling out throughout the company.
Prototyping the tray had multiple elements that needed to move in response to scrolling horizontally on the remote trackpad. The card size scaled up on focus and the headline for the first video slides in from the bottom. I kept the text size constant to reduce excessive movement. When a topic category is focused, the text increases to full opacity and a small purple indicator becomes active. Clicking on a new card starts that playing videos for that category. Backing out with the Menu button or swiping down dismisses the tray and returns to the video playing in the background.
Simulated scrolling prototype for launching the Channel Playlist tray.
Another key to the linear experience algorithm is knowing the user. This comes in the form of pulling in users’ preferences from their Yahoo profile, such as their favorite sports teams from Yahoo Sports or the stock tickers they follow on Yahoo Finance. In addition, users should be able to customize what they follow by adding or removing the topics either directly from the TV app (or later, from the companion app planned for native mobile.) A combination of these implicit and explicit actions would ideally serve to create a smart VLC that represents the user’s interest. The design considerations focused on how to display topics you're following (with a star) vs not following, adding emphasis to focused topics by adding a ring, and how to best show brand logos. We decided to move forward with light UI logos, but saved the ideas for darker logos for a future version.
Topics with dark UI logos.
Topics with light UI logos.
I redesigned the flow that welcomes users when they launch the app. I wanted it to reinforce the personalized nature of the experience – it would display the Yahoo weather information for their area, and their channel lineup would immediately load and start playing first inside the card, then full screen after about 30 seconds. I incorporated the same card style topics used in the Channel Playlist.
Users are welcomed into the app to reinforce personalization.
Building on the idea of creating a personalized experience, I wanted to incorporate a photographic background image in the welcome screen that would be reflective of time of day along with the greeting, weather, and location. This felt much more visually engaging than the plain blue background that was used in the MVP version. We could have a set of photos for each time segment and cycle them for a fresh welcome image on each visit.
Morning - Afternoon - Evening - Late Night. A way to daypart the experience.
We continued to work on how to best describe the main parts of the app and encourage first time users to sign in and personalize their experience. People tended to miss the original onboarding video in testing. The video felt stark so I suggested adding in ambient background music. We had another idea that would require people to swipe between each screen so they had better opportunity to understand it or go back to read something they missed. This was still as work in progress, but this is where we left it for v1.
Sequential onboarding video describes the app and shows how to sign in to Yahoo.
While I was auditing a variety of OTT apps for both Apple TV and Fire TV, I was shocked to find that accessibility is often overlooked. Default, out of the box engineering solutions for voice readers is haphazard at best. I felt strongly about our app NOT falling into this category. The lead engineer and I, being new to a11y, spent time meeting with our company’s accessibility expert as well as experts from Amazon, to learn about best practices. I audited our Android and Fire TV apps and worked to provide written scripts to our engineering team.
Apple TV App
HuffPost is known the world over as a source of irreverant news about politics, entertainment, technology, entertainment, and more. Bringing their video content to the Apple TV gives its fans an easy way to keep up to date while relaxing in their home.
To create a unique experience for HuffPost, an effort was made to reimagine the format of traditional Apple TV apps. Scrolling through video thumbnails tends to feel very static, so I wanted make sure the first experience felt alive. Launching the app starts a video playing right away. Then, watching a video in full screen and swiping between videos felt like a great way to keep the feeling fast and fresh. Extra attention was paid to the details, so I spent most of the design process prototyping the microinteractions throughout the app. From the carousel scrolling, to the full screen swiping, to the waterfall loading of the quick nav tray – the experience should feel effortless and satisfying.
Right off the bat, we start with a short branding launch animation. A video plays in the background with people walking across a busy NYC street. The HuffPost logo animates in and out. And ambient music plays, giving the scene a a little more life. This sequence plays a functional role, giving the app a chance to load the home carousel videos, while also setting the scene for the main experience.
After the launch sequence, the main carousel loads in. When swiping through, the first video of each playlist starts playing when it snaps to center. Meanwhile, short muted preview loops play on side thumbnails, to continue with the motion theme. Clicking on a video will open it in full screen.
When watching a video in full screen mode, we implemented two options to skip to other videos in the playlist. The first, most tradititonal method, is a tray of thumbnails that can be launched with a swipe up on the trackpad. Rather than static, typical thumbnails, these play a short 5 second loop of each video. It creates a pleasing sense of movement, following the idea of making the app feel alive.
Also, when watching a video in full screen, you can swipe between videos with the trackpad using a left / right motion.
iOS and Android App
Build Studio is an interview series featuring popular stars and artists, with a live studio audience. Interviews are live-streamed from their ground-floor Manhattan studio and archived videos are available to watch on the Build website. Other website features include viewing a calendar of upcoming shows, adding shows to your calendar to get a reminder, and the option to submit questions to the interviewees.
Keeping the basic features of the website was important in the design of the iOS and Android apps, however I wanted to make sure that the native app was a more dramatic experience. To do this, the home screen features the most current videos in a full screen portrait format. If a show is currently live, it will be in the first carousel slot – up front and featured – followed by recent and upcoming videos. The archived videos are easy to peruse or save in a watchlist for later. Upcoming shows can be added to a reminder list. Users can even reserve tickets to attend the filming or submit questions to ask. The Android app was designed as a Progressive Web app which is available from the Build website on mobile.
The home screen features full screen portrait videos. If there’s a live broadcast, it will be playing right here in real time. If you scroll to the right, you’ll see other recent videos recorded at our Build Studio in NYC. Scrolling horizontally through the home screen should feel effortless with a simple, soft feel. The prototype is shown with static images to focus on the parallax effect and movement of all elements.
Since there are different types of videos, we needed to account for their unique properties. For instance, if there is a live show streaming, you could send a question to ask the guest. For an upcoming show, you can request tickets or add it to your watchlist. An archived video can be favorited or watched later.
While scrolling through recent archived videos, you can add to a watchlist or check out which guests are coming soon and add a reminder. If you set a reminder, we send a push notification when the episode starts.
For first time visitors, a tooltip shows how to long press on any thumbnail to see a quick preview clip.
Android App
Vivv is a fun and original photo app that lets users take a photo or video and isolate colors that they want to pop -- all in live mode.
Based on the popular iOS version of Vivv, I focused the design of the Android app to retain the custom look and feel while functioning like a familiar Android app.
Android App
Sage is a fun way to make predictions about politics, technology, sports and entertainment, or whatever comes to mind. Friends can see predictions within their community and cast their vote.
My main design focus was to reinterpret the iOS app, incorporating Material Design elements and true Android navigation, while maintaining the custom style and spirit of the app and brand.
iOS & Android
The Activehours iOS and Android apps enable users to cash in their worked hours and receive money in their bank account in advance of their pay day, without interest or fees.
The intensive signup and registration flows involving submission of highly secure financial information such as employment info, paychecks, bank accounts, and bank credentials were structured be clear and succinct. I wanted to ensure that the design of the app's cashing slider felt intuitive and responsive. The tipping section was designed using friendly icons to inspire a user to leave an optional tip; who could resist something as simple as buying us a burger or a beer?
OTT TV Apps for AOL Brands
Roku is a major player in the OTT Streaming TV viewing industry, so it made sense for AOL to focus on bringing its highly video-oriented brands onto this platform.
I based my design process on the idea of building a simple to navigate and fun to consume family of apps. I wanted to keep a sense of consistency between the brands, building a uniform framework and layout. However, I also wanted to ensure that we would spotlight each brand's unique features; the Autoblog app should prominently feature their photo galleries, Engadget should highlight topics, TechCrunch should allow users to easily recap special events, and for Moviefone we included a tab dedicated to autoplaying the most current trailers, just like a moviehouse experience.
Responsive Web Design
As the marketing companion to the Starlike App, I wanted the website to feel as fun and friendly as the app itself. Rather than using static images, I thought it would be fun to use a video format. Incorporating motion and activity brought a sense of life to the app. Since the app revolves around discovering friends' likes and faves on social media, using videos of people interacting with their friends was a natural fit.
We found video snippets with images and scenes that people would find relatable. I designed a simple one-page site and a running loop video with clear calls to action download buttons. A backup set of layouts for mobile devices or poor connectivity with a static image of two friends scrolling through their social feeds was also implemented.
Branding Identity
After years of making jewelry for her friends and family, Marigrace decided to start her own company. She enlisted me to craft her brand identity. It was critical that her outward facing brand should represent both her product and her personality.
Marigrace's favorite icon is the butterfly. I wanted to make sure to incorporate this into her logo. Typography was inspired by old milk bottle script fonts to reflect her bubbly personality. I chose a feminine, yet striking blue and a contrasting charcoal gray. Overall, the looks is fun and whimsical with a retro 50's look. Perky blues, butterflies, and clean lines were perfect for her.
Some recent tutorials and resources I've written.
How to get started training intelligent game environments using machine learning
Discovering tools and workflows to better visualize ideas in 36o˙.
Making skyboxes in Unity3D from a variety of different materials.
How the physics of our real world translate to VR design practices.
Working at the fringe of your skill set can be intimidating.
Learning to be a better designer by studying Dave Grohl, Buster Posey, or even… that ‘Most Interesting Man in The World’ guy?
When I left my career in science to work in design, I initially focused on graphic design. After a few years, I began to design for mobile and web applications with a speciality in UX & UI. In recent years, I have expanded to AR & VR design. I love that I get to think about analytical questions and apply creative solutions while researching new technologies and learning new skills – blending my past life with my current life.