
Silingual
Multimodality Translingual Storytelling Application

Overview
Project Type Solo Project
Timeline September 2019 - December 2019 (5 months)
Role UX Researcher & Designer
Tools Adobe XD, Miro, Otter AI, Artboard Studio, Canva
Background
Issue
In today’s society, over half of the population is bilingual (Ansaldo, Marcotte, Scherer, & Raboyeau, 2008). Yet, when bilingual children are limited to the weaker language, they may not be able to show their critical thinking (CR) and creativity (CT). Silingual creates a multi-modal and translingual learning environment for bilingual students.
Challenge
How can bi- or multi-lingual children critically and creatively use their weaker language to learn and communicate?
Process

Research #1 : Validating the issue
Interviews
To validate whether bilingual children were able to show their critical thinking and creativity in environments limited to the weaker language, I facilitated 5 semi-structured interviews with bilingual teachers and 1 interview a bilingual student from Taiwan.
I explored students’ learning behaviors and needs in English class, and was able to understand the critical questions:
Do students demonstrate critical thinking skills in English class? Why or why not?
Do students demonstrate creativity in English class? Why or why not?
Key Interview Findings
I was able to quantify my hypotheses through the three key findings:
The limitation using the weaker language may narrow the expression of ideas for bilingual children.
Bilingual students:
“I know how to express my ideas in Chinese Mandarin, but I cannot do so in completely English.”
"I would like to create my own story, but my English is not fluent enough to do so."
Bilingual Teachers:
"Students have lack of English vocabulary to express themselves."
"Students are not confident in expressing themselves in English"
Teachers seldom facilitate activities that showcase students’ creativity and critical thinking because students' English ability may not let them to do so.
Bilingual teachers:
"Because students need to have competence (vocabulary, sentence patterns, dialogues) in English first before having creativity and critical thinking."*
“I feel like there are few activities that focus on creativity and critical thinking.”
Children are limited to using the weaker language because the norm is to create an "English-only” environment in English class
Bilingual Teachers:
"Teachers feel guilty when they speak Mandarin."
"I try my best to speak in English all the time, but whenever students seem confused, I speak in Mandarin."
"I try to use English as much as possible, but switch to Mandarin when students can't understand or follow my instruction."
Summary & Reflection
I can validate that students have a desire to express and challenge issues/topics in English, but cannot do so because of English ability limitations. The teachers who design the English lesson plans acknowledge the importance in CR and CT, but are uncertain about whether CR and CT is the priority in English class. An English-only environment is encouraged even when teachers know that some students can't understand in class. This left me thinking about the questions:
Do students have to develop a level of competence in English ( speaking, listening, reading, writing abilities), before they are able to critically and creatively communicate and learn in the English?
Why isn’t CR and CT valued as the criteria of whether student have competence in the English language?
Why should English class be an English-only environment, when students have trouble understanding or seem confused in class?
Scoping the challenge
How can bilingual children whose first language is Mandarin, show creativity and critical thinking using the weaker second language English?
User profiles and stakeholders
Bilingual Children:
Elementary school kids
Speaks two or more languages
Access to the Internet (Technology used for social good)
Stakeholders:
Elementary school teachers
Parents of bilingual children
Research #2: Secondary research
Competitive Analysis
To understand what’s currently available on the market, I analyzed applications that bilingual children used to exercise their second language English. I found out strengths that I would apply, and also common weaknesses that would be solved in my design solution.

Language learning applications Lingokids, Duolingo Kids, Endless Reader, and Duolingo.
Pain points:
Monolingual or code-switching between two languages
Lack of interaction with friends or peers
All animated images, not related to the real world
Common features:
Interactive between user and interface
Parent/teacher vs. kids areas
Visual design applies vivid colors
Procedures are simple instructions
Literature Review on Translingualism
Tracing back to the reflection I got from the interviews:
Why should English class be an English-only environment, when students have trouble understanding or seem confused in class?
When I was exploring literature reviews, I looked for papers that discussed about what was more than an English-only environment:
Are there any theories that improve the weaker language in a bi- or multi-lingual environment?

The book “Translanguaging“ by Ofelia Garcia and Li Wei gave me the answer. The concept of Translingualism provided insight of how using multiple languages concurrently may encourage creativity and critical thinking rather than a monolingual (e.g. English only) environment.
According to Garcia and Wei (2014) on Translingualism :
Uses two or more languages, symbols, and gestures to communicate and learn
Students can improve their weaker language ability (English), and also promote critical thinking and creativity
Summary & Reflection
From the competitive analysis:
There are limited activities that involve creativity and critical thinking such as “Make your own cake” in LingoKids under the English-only environment.
Interaction between peers is not implemented. The visual in the apps are animated, so learners might lack the opportunity to interact with the real world.
From the literature review:
Creativity and critical thinking is a part of the English competence criteria.
Translingualism can be applied to help student enhance the weaker language, and also encourage creativity and critical thinking.
Research #3: Building personas and storymapping based on second interviews
Interviews
To learn more and identify my target users, I interviewed 4 bilingual teachers and 1 student in Taiwan to develop personas for the target users.
The main questions were:
What are the demographics of students?
What are the learning habits of students?
What are the learning goals of students?
What are the learning needs of students?
What is the use of technology for students?
Personas
Based on the interviews, I built three personas based on three grade levels: first grader, third grader, and fifth grader elementary school students The three personas where mainly differentiated by their English learning goals and use of technology.






Storymapping

Storymapping for students
Targeting Bilingual Fifth-graders
For this project, I focused on fifth grader’s interaction and screens. Joe is a student with low-English ability and low motivation in learning English. Kily is a student with high-English ability and high motivation. Chichi is a student with average English-ability and normal motivation.







Storymapping for fifth grade students
Summary & Reflection
Storyboards & Scenarios
I brainstormed solutions through storyboards…
















And wrote the scenario for the Conversation App to understand the process from problem to solution…
Bob is a fifth grade Taiwanese student in elementary school. He is learning English at school. Since the classes are in English-only, Bob often gets lost in the classes. His friends are able to discuss about the activities in class and reply to the teacher, but Bob understands nothing the teacher is talking about. Thus, Bob nearly doesn't learn anything in class. Bob doesn't like English, but he still hopes to improve his English ability. He wants to be able to participate in class. If he improves his English skills, he will be able to be more confident in English and talk to his friends in English-related homework or questions. Additionally, he hopes to be able to communicate in English when he travels abroad.
One day, when he is downloading a game from his phone, he finds a Conversation App on the App Store. He recalls one time he had heard his classmates John and Ruby talking about another classmate Fibie’s photo in some English and Mandarin. Bob thought, “ Mandarin and English? You could use Mandarin too?” It was then when he walked over to them and realized they were chatting about an app called the Conversation App.
Now that Bob see the app, he’s curious and wants to see what the context is inside. When he goes into the app, there is the look-at-the-picture and describe activity John and Ruby were talking about. He sees the instructions “ You can use Chinese or English!” He decides to give it a try. There is a boy kicking a ball, so he presses the microphone function and says “有個男孩在踢ball!(A boy is kicking a ball.)" Then, the app responded with "You did great (with symbols)!". The app also featured other answer other users said, such as " 有個boy在踢ball!". Now Bob knows the English word for "boy". Bob and his classmate Fibi connect through the app. He sees the photo that John and Ruby were previously talking about. The photo is Feibei and her dog catching a ball. Bob replies “你的dog在撿ball!"The app responded with "Awesome! (with symbols)". Bob see that John replied with "Your dog is catching the ball!", and Ruby responded "Your dog is 撿the ball!"
Bob feels that maybe he can use this app to learn some English at home. From the two practices, he learned the English vocabulary: Ball, your, and catch. He also learned the sentences "A boy is kicking a ball" and "Your dog is catching a ball!" Bob feels the app is interesting, since he can add John, Ruby, and Fibie on the app and make up sentences of pictures from his friends. Bob thinks it is possible for him to use the words he knows in English to express situations with the support of Mandarin. The app has simple instructions and functions, so it's easy to use. He downloads the app into his phone. Hopefully, he can improve his English.
User Journey
Upload pictures of their daily lives in categories of themes including Food, Sports, Earth, Feelings, Occupations, Time and more.
Choose a picture of their peers other to storytell.
Think about the actions and feelings of the character(s) in the picture and describe in their native language (Mandarin), second language (English), and emojis.
Post it on to the SiFeed.
Select on pictures and view the content made by their classmates and friends.
Comment and star (like) on the stories made by their peers.
Interaction Map
updating soon
Design Solution
Create a multimodal & translingual environment for bi- or mult-lingual students to communicate and learn using different languages and symbols.
Paper Prototype
User Testing paper prototype
The user testings lead me to the final digital prototype that looked simple and visual simultaneously:
Buttons and actions had to be clear and evident
White background color with vivid color elements, icons, and pictures
Design Defense for digital prototype
I was able to solve questions through assessing the criteria of each option.

QOC showed which options should be applied after being categorized into positive, medium, and negative assessment by criteria.
Key Function 1: Critical thinking
Silingual provides a platform "SiFeed" to showcase a variety of cultures and diversity of pictures. Users look at different pictures uploaded by peers, explore the actions and environment, and think about what the characters in the pictures are doing/feeling.
Key Function 2: Creative content
Users’ create their own unique perspective of his/her world through multimodalities, multi-languages and symbols. By selecting a picture, users can use type-in or voice input to express their ideas in words and sentences. In addition, they can also choose emoticons or emojis such as facial expressions or common objects. While users storytell their version of their peers' picture in translingualism, the weaker language English can be used and improved.
Key Function 3: Interactive platform
Users can go into each picture to see other users' interpretation of the picture. In addition, users can comment on the interpretations. Users' can increase input of weaker language, promote creativity and critical thinking through peer learning and feedback.
Reflection
Social Impacts
Nurtures children's criticality and creativity: Silingual gives children the opportunity to be free in expressing themselves.
Promotes interaction: Silingual can help children learn to interact with their peers via technology.
Potential in Education
During user testing, teachers and students gave me these feedback:
“Translingualism is a novel topic that I would like to understand more about”
“I really would like to use this in my class”
“I am interested in inviting my friends to use this product”
“Where in the app store can I download it?”
Next steps…
What languages should the instructions of the app be in?
How can children’s information be protected?
How can teachers/parents guide children in using the app?
References
Garcia, O., Wei, L. (2014). Translanguaging. Language, Bilingualism, and Education. Palgrave Macmillan.
Norman, D. A. (2013). The design of everyday things. Cambridge, MA: MIT Press.