An Automated Documentation System for Modern Business

Toki, an automated meeting assistant for managing the complexity of modern business meetings. It is a web application that incorporates smart speaker platforms such as Amazon Alexa and Google home.

Sponsored by Amazon

Responsibility: I was a part of a small team with 3 people and lead the design and strategy of the project.

Everyone participated throughout the research process. During the research phase, I took charge in creating the storyboards. After that, I came up with the wireframes and the user flow for the virtual pet concept which ended up as our final solution. Furthermore, I created the high fidelity designs for the mobile application using Sketch. Also with Invision, I created an interactive prototype of the application. 

 

Challenge.

How might we manage the complexity of modern business meetings by capturing important moments for multi-disciplinary attendees?

Background.

The meeting is a vehicle for many organizational activities, and employees spend a significant amount of time in meetings. Kevin Hoffmann, an expert in organizational science notes that “meetings should add value to participants’ lives by providing a sense of progress — problems being defined, decisions get made, priorities being prioritized, and solutions being built upon the benefit of multiple perspectives.”

Our research finds that many people are unsatisfied with their current meeting experience (cite survey). Additionally, our research shows that people most frequently participate in small meetings consisting of 3-5 people across various job titles and roles. Deeper inquiry through semi-structured interviews exposed communication gaps within multidisciplinary environments.

Introduce TOKI.

Toki can be deployed on smart speaker systems such as Alexa or Google Home. 

It utilizes speech-to-text technologies to parse conversations during meetings. Toki marks important moments as decisions by capturing trigger words and phrases in conversations such as:”can you do…”, “follow-up”, “due date” and etc.

During Meeting

Mark Decisions or Actionable Items on the Timeline

Unlike rolling transcripts that are distracting and cumbersome to read, Toki encapsulates decisions or actionable items as “moments” on a timeline for attendees to verify on the spot.

​​

  • Listen and parse conversations

 

  • Generate a “moment” when a decision is verbally confirmed 

Augment Note-taking with

Relevant Transcripts

Suggest previous transcripts that are relevant to what the user is typing

​​

Help user record the context when taking notes

Provide Private and

Public Notes Options

Empower users to share their thoughts anytime

   Post Meeting

Provide Context of

Decisions and Notes

Document all the important information of decisions

 

Save snippets of transcripts around each important moment

 

Visually represent possible connections between notes and decisions on the timeline

Provide Screen

Recordings of

Attendees’ Assets

Sharing

Screen recording when a laptop is plugged into a projector

 

Cater needs to share various types of documents 

Accommodate

Second

Thoughts about
Decisions

Allow users to push back decisions when necessary

Research.

The goal of this research was to identify and understand existing problems within the meetings in US business environments. Our research revealed that the multidisciplinary meeting environment, coupled with attendees dissatisfaction and perceived lack of productivity suggest an opportunity for improved communication among small, multidisciplinary project teams. 

Here's the link to the full research report

01

Secondary Research

The goal of this research was to identify and understand existing problems within the meetings in US business environments. Our team conducted a literature review, four expert interviews

02

User Survey

We collected broader demographic data across the country and got 204 responses. We found two themes lead to back and forth discussion during meetings:

01 Different domain knowledge and different interpretation of topics;

02 Lacking enough information to agree or disagree. 

03

User Interview

During our result from the survey, we decided to focus on multidisciplinary small to medium meetings. We were able to recruit 12 people, who were selected based on their survey answers. ( 5 designers; 2 researchers;  3 engineers and 2 product managers).We used affinity diagram to organize the qualitative data.

Insights.

The goal of this research was to identify and understand existing problems within the meetings in US business environments. Our research revealed that the multidisciplinary meeting environment, coupled with attendees dissatisfaction and perceived lack of productivity suggest an opportunity for improved communication among small, multidisciplinary project teams. 

01

Knowledge Gap

In multidisciplinary meetings, attendees find it difficult to distill and engage useful details of another discipline due to knowledge gaps of different domains.

It makes the meeting less productive to get into the very detailed domain-specific topic when meeting with people from different backgrounds.  

Participant 8

02

Personal Notes VS. Universal Notes

Attendees find taking their own (digital or manual) notes useful even if there’s always someone taking notes for everyone. 

Generally, I think [personal] notes are helpful. I think if you're in the meeting and you're paying attention, the notes will jog your memory enough for you to remember what was being said. Participant 1

03

Lose Track of Conversation

The attendees lose track of the conversation as a result of not having access to the context of the meeting. 

It’s really useful if people provide some context like, information that tells me, how we are, where we are now, where we’re trying to get to. A lot of times what we’re trying to do is just getting from one place to another. If you don’t understand how you got there then you’re missing the context.  Participant 1

04

Mismatched Behavior

A mismatch between attendees’ behavior and expectations results in a vicious circle of seeking help after meetings.

Sometimes people just don't retain everything that happened in the during the meetings and just ask you questions. I always send out notes with references after the meeting, but of course, nobody reads these emails these days.​  Participant 9

Synthetic Models

Based on the research data, we built empathy with our users and created a user journey map to represent their behaviors and cognitive process.

Context Scenario: The design team is presenting their solution of Product feature A and would like to get some feedback from engineer team, marketing team and product manager, so that they could move forward. Product manager invites all people who are relevant to this subject matter to the meeting. During the meeting, the developer team challenges their solutions, present their perspectives, and propose another direction. 

We made a journey map for each persona and overlay them together to get a better sense of what is the problem.

The diagram is the meeting experience for Keith, the product manager, Sean, the graphic designer, and Diana, the software engineer. 

Design Principle.

Ideation.

We then started ideation sessions and generated over 100 concepts. We eliminated those that did not tie back to our design principles, and sorted the remaining ones into themes

We realized we were drawn to concepts that:

Promote comprehension and minimize confusion without disrupting the conversational flow

Hence we narrowed down to 5 concepts:

01 Structured Conversation; 02 Decision Timeline; 03 Interactive Slide, 04 Collaborative Glossary; 05 Question Marker

Also, We choose to go with a laptop

LoFi User Testing.

For the first round of testing, we made five low-fidelity paper prototype. The goal was to get feedback on the concept itself rather than specific functionality. We recruited 4 designers and 1 engineer (age 25~35) in tech companies through personal connections.

WHAT WE LEARN:

01 Conversation is not linear, but more organic (P2, P3, P5)

02 Reading and organizing transcriptions of one’s own words are challenging during meetings.

03 Self-documentation can distribute accountability to each individual and thus can be useful in the future.

04 Decisions are chained and contingent. One decision made on a meeting might be changed in later meetings.

05 The development of glossary might happen within a team and not just in meetings. (P2)

06 In MDMs, free-form whiteboarding is often used to communicate across disciplines, especially at the early stage of a product cycle. Slide decks are for “town hall meetings.”

07 Interjecting with questions is valuable to some participants and nerve-racking to some others.

08 Seeing questions appear while content is being presented can be disruptive.

09 Audio recording can be useful but replaying might be time-consuming and unnecessary during meetings. Visual context isn’t as disruptive.

Storyboard of Refined Concept.

According to the feedback, we decided to go with 02 decision timeline but also minimizing the effort to document any moments.

FEATURES: 01 Capture important moments (Decision, Actionable item, Notes); 02 Generate post-meeting references; 03 Promote a shared context; 04 Testify disagreements/agreements

01 Monica walks into meeting room “Bigfoot”. She have a meeting here 2 min later

02 She opens laptop and see a pop up if she is attending the meeting XYZ becasue the system detect she walks in the meeting room Bigfoot.

03 As meeting going on, whensystem detect the user says [name], “needs to do”, it generates a pop up on the timeline as a important moments

04 Everyone in the room sees decisions and their accountability.

05 After some talking in the room, Monica wants to mark something as notes for later reference. She hits the notes button.

06 She chooses from the options generated by the system

07 After the meeting, Monica can’t rememberone decision. She clicks into the calendar and located the meeting xyz

08 She locates the decision moment, and sees 2 minutes of auto transcriptions. She can also replay the audio of that moment.

MIFI User Testing.

ROUND 2: Evaluative phase

For the second round of testing, we presented digital, clickable wireframes by Invision. The goal was to get feedback on the app's usability.

POSITIVE FEEDBACK: All the participants think providing context during or after meetings would help target users retain useful information.

ACTION:

01 Unify the experience of approaching sections (timeline, notes, and transcriptions).

02 Improve the visual communication of UI elements.

03 Minimize the efforts to make personal notes and learn context during meetings.

04 Modify the flow of accessing the previous meeting.

05 Reduce privacy concerns of auto transcription.

 

HIFI User Testing.

ROUND 3: Evaluative phase

For the third round of testing, we presented two flows of high fidelity prototype created by Flinto. The goal was to get feedback on the app's usability. One is during the meeting section, another is post meeting section.

 

DURING MEETING

01 The timeline lacks clear visual cues for users to recognize “moments” and “public notes.”

02 Users cannot tell whether the avatar photo represents a task owner or an assignee of the task at a glance.

03 Users realized the options in “Add Notes” were transcripts, yet they were unsure of some details.

 

POST MEETING

01 When asked to learn the context of a moment, users tend to interact with the items on the timeline instead of reading the transcripts.

01

 

02

 

03

 

ACTION:

01 Clarify the differences between public notes and moments. (The same with private notes) 

02 Provide more context (Make 1 or 2 more interactive elements on the timeline - for example, the screen recording feature.)

03 Disclose more part of the auto-transcription suggestion. (Make text input box narrower)

04 Think about the order of auto-transcript suggestions. (Add time stems.)

05 Differentiate the quotes and what they wrote themselves.

 

User Flow.

This user flow illustrates a typical user journey of generating moments, adding notes, and viewing each item on the timeline during or after meetings. Moments are pieces of transcripts around tasks, decisions, or actionable items. Users need to sign in with their company email address. 

 

Visual System.

View the full Design Specs

 

Reflection.

What I learn

1. Triangulate Data, figure out what user really wants

Because we are designing a new tool that is nothing like it, it is hard for people to imagine they are using it until you present the prototype. I need to carefully evaluate the data collected from the user interview to see if they truly mean it. E.g. when people says they want to take better notes, they actually want to remember the important information.

 

2. Don't be a feature Creep

Users always want more with their question "why don't you add...?". However, it is important for the designer to rigorously consider the potential feature needs. We had an abundance of feature ideas. I write down all our potential features and evaluated them based on the design principles in order to prioritize the leading set.

 

Future Steps

1. Accommodate the entire workflow within the company

  • Think about the on-boarding of Toki. How does Toki collect voice data?  How does onboarding of Toki incorporate into company's employees' data?

  • Sync with other software like Jira, Quip, Sharepoint…

 

2. Design a mobile application

  • Allow other formats of assets sharing. Based on research we learned whiteboarding and other forms of assets sharing (rather than slides) are common especially at the early stage of a product cycle.

  • Allow the user to check work while moving

© 2019 Xinbei Hu