Using Emojis with Accessibility in Mind
Article 19 explores the accessibility nuances of using emojis and gifs
Joined by Dr. Damien LaRock, we explore the nuances and accessibility of modern communication, specifically emojis and gifs. Diving into the intended use of these icons and images, the different uses of the same icon, and how a screen reader notices them.
This episode is part of a three-part series focused on marketing and digital design with accessibility in mind. Check out the other episodes:
- Article 19’s episode on Marketing with an Accessibility Mindset
- Article 19’s episode on Digital Design and Accessibility
Listen to more Article 19 Podcast Episodes
Full Transcript
Produced Introduction with music bed:
Expression is one of the most powerful tools we have. A voice, a pen, a keyboard. The real change which must give to people throughout the world where human rights must come about in the hearts of people. We must want our fellow human beings to have rights and freedoms which gives them dignity. Article 19 is the voice in the room.
Kristen:
Hello, everyone, and welcome to Article 19. This is a podcast rooted in the firm belief that access to information is a human right. Here we offer information through conversations and interviews for increasing awareness, empathy, and expertise at the intersection of technology and inclusivity. I’m Kristen Witucki and I’ll be hosting the conversation today. I’m joined by repeat contributor and friend, Dr. Damien LaRock. Hi, Damien.
Damien:
Hi Kristen.
Kristen:
Behind the mic and in front of the mic, we have Markus Goldman and Harper Yatvin. Hi, Markus. Hi, Harper.
Markus:
Hello.
Harper:
Hello.
Kristen:
They’ll be jumping in with insights on this topic as they think of them. And, now let’s get to today’s episode. Emojis. So, just to offer some context, we don’t really talk on the phone anymore. That’s not news to anybody. We don’t write letters. Now we barely email. At this moment, we send messages in short bursts. Texting, whether it’s on the text app itself, on Messenger, on WhatsApp, on Slack, is really our conversational currency. And, because tone is not always clear, we tend to use emojis to substitute for not seeing each other’s faces in real-time or body language or hearing each other’s voices over the phone. But what if we don’t all understand emojis in the same way. This has led to legal problems in Canada already. There’s an article that we will post in the show notes, but basically a farmer in Saskatchewan used the thumbs-up emoji to sign off on a contract as he had before and when the contract didn’t work out for him, he hoped to get out of it by suggesting he hadn’t really signed it. And the court did not buy that. They said you have done this in the past and therefore you have in fact signed the contract. In the article, “The Unbearable Ambiguity of Emojis”, goes into that idea that emojis are interpreted across cultures differently, across people differently, frankly. And, as we always talk about accessibility, it’s interpreted across accessibility platforms differently. What if, like me, you rely on the descriptions of emojis? So I started to think about this with one of my best friends, Damien, and that’s why I brought him on to talk about emojis. Damien and I text frequently, and I noticed he would end a couple of messages with the. I put up with that for a couple of messages. And I hope that the way I finally asked about it was describe what you mean when you say relieved face. But probably I just said “[Laughter] why’d you give me the relieved face? Why did you use the Damien?
Damien:
So, I particularly like this emoji. To me, I was not interpreting it as a face of relief, but more as a face of peaceful, calm, serenity. My emotion that I was trying to get across by sending that emoji is I’m all good with this, I’m peaceful, I’m calm.
Kristen:
So, that’s what he said and that’s a very different meaning from what the description suggests. To me relieved face is like you’re finished with this conversation psshheww or as one of my friends said, when I talked about it with her, like, “Oh, I just went to the bathroom, I’m relieved.” [Laughter]So, you know, It’s a very confusing thing and I’m actually not sure that there is any description in the list of emojis that quite captures that like calmness, serenity, peace, meditating, whatever. I love talking about this. And, then we got to thinking, what if we talk about a whole bunch of these? So, we’re basically going to focus on the smileys and people category of emojis because that’s probably the most ambiguous. So, Damien, what are some of your other favorite emojis?
Damien:
I think my most commonly used emoji is the winking face. And, I’ve realized that maybe the way that I use this face is not exactly how everybody interprets it. I tend to use this face when texting to mean “got it, like I understand you.” But I also realize that a winking face where like just the one eyelid is closed the other eye is open can also mean like we have a secret understanding here or I’m like sending you a message that’s just between me and you, but nobody else really knows. Okay, I’m also a teacher. And, there are moments in my classroom where I’ll want to send like a little secret message to a paraprofessional in my classroom and have it go over the kids heads. And so, I’ll say something in a more adult-like way that the kids wouldn’t understand. I’ll do the little one-eyed wink to my para meaning like, you got me? Right? Like, did you get my secret message without the kids really understanding what we’re talking about? But in texting, I don’t necessarily mean it to be secretive. I just mean it to be like I got you, I understand.
Kristen:
Yeah and that’s the opposite kind of instance, where the description is fairly clear and then kind of your intent could be two different things.
Damien:
And, it’s interesting because, right Kristen? The description just says winky face.
Kristen:
Yeah, winky face.
Damien:
And so there’s nothing in the description that indicates like interpretation or meaning behind that. It’s just winky face.
Kristen:
Right. That’s really fascinating. I wonder how these descriptions are written? If there’s any input from, you know, a panel? If it’s, one poor person back there trying to figure it out? You know, how would you describe this? It makes me wonder. If you can let us know Apple, that’d be great. There’s another winking face that you use a lot. It’s the one that’s described as winking face with stuck-out tongue.
Damien:
I use this one quite often and generally, I use this to mean, what I said was just silly, or what you said, I think is silly or like just kidding. It’s an expression of silliness.
Kristen:
Yeah, it took a while for me to appreciate because at first, I was like “stuck out tongue” [Laughter]could mean—well, used to mean something bad or like your kids are always told not to stick their tongues out.
Damien:
And then there’s another emoji I don’t use this one quite as often, but it’s really similar. It’s a googly-eyed face with a stuck-out tongue. :zany_face: And, that one I use if I want to say like, that’s really crazy, or like oh my gosh, that’s amazingly silly. And so, that one has one smaller eye and one bigger eye. One eye is looking down, one eye is looking up. The smile is kind of crooked and the tongue is sticking out a little bit to the side. It’s a sillier face than the previous one. I use that one to mean like really really silly.
Kristen:
So, there are a couple of aspects to the description. The description says cross-eyed grinning face with stuck-out tongue. “zany_face_emoji”. Stuck-out tongue is the same in the two descriptions. So we don’t know that the tongue is stuck out in a different way. And then cross-eyed I mean; I don’t know like cross-eyed just seems like somebody should have taken that out of circulation[laughter]. But there it is, in the emojis. Harper, Markus, do you have any other favorite emojis that you wonder how it is used there?
Markus:
Yes, I was curious about the sort of smiley face with the welled tear eyes or welled water eyes and how that came across to you.
Kristen:
So that just says smiling face holding back tears. I’m really happy they added that one to the mix because that’s a good emotion to communicate, I think. But it doesn’t say welling, it just says holding back tears.
Markus:
Another one that I like is the monocle emoji with the serious face and the monocle like they’re deep in thought or hey, let me think about that. Let me process that. Let me wrap my head around that and then I’ll get back to you is what I take from that.
Kristen:
All the description says is serious face with monocle. I’ve come across that one. And, for me, I wasn’t sure what that meant. So I like having the meaning. Now, I’ll start using it. There was another one that says thinking face :thinking_face: that is what I’ve been using for thinking.
Damien:
the one with the hand on the chin
Markus:
Yeah, that’s like the thinker statue but in emoji form.
Kristen:
Rodin, Rodin, he lives on. Anyway [laughter] yeah I love that statue. But you know it doesn’t say that in the description. It just says thinking face. So I always forget it’s the hand on the chin thing. Now I’ll remember. I just remember Rodin in that one.
Harper:
This is Harper here. Markus, you definitely beat me to the punch on the face with the crying welled eyes :smiling_face_with_tear: emoji. That was my go to. One that I always find it versatile, but also a little confusing is the happy face with the sweat bead on the side :sweat_smile:. What does that say? Because to me, it can mean a couple of things.
Kristen:
Grinning face with smiling eyes and sweat drop. So, yeah, again, like how do you interpret that?
Harper:
Yeah so, If I use it after a workout at the gym, that means like, okay you know it’s a great workout and I got sweaty and I’m happy about it. But it could also mean like “Oh I did something that I maybe shouldn’t have and now I’m in trouble.” Like if we had emojis when I was 10 years old, I would definitely use that sometimes. That’s like a second way that I interpret it.
Damien:
I think I interpret this one slightly differently. I would use this to mean like I almost experienced something bad, or I almost got in trouble, but I got out of it. So like, whew, I’m saved. To me that’s what it means, like something bad almost happened, but I’m OK, I’m safe.
Markus:
That’s so interesting that y’all see it that way. I see it like also, as it’s hot and humid out, look at my forehead, my old forehead is getting all sweaty. That’s more how or like hey that’s really funny or something along those lines. Something warm in that way is what I take from it. Fascinating how these emojis are looked at or viewed at or translated differently for each individual person. It shows you how different each person’s mind and how individual each person is.
Kristen:
For sure. For me, like when I hear that I get confused because of the sweat drop, because then like if you just have one drop of sweat, like are you really sweating that much? You know? [laughter] And I almost think that it says that because of the size of the emoji more than sweating. Sweating would make more sense, maybe in the description. Harper, have you got another one?
Harper:
Yes, I do. I just looked at an emoji that has always confused me. It is the eyes of the yellow emoji person.[no emoji] And there is mist or fog all around this little emoji person. I think of it as halfway through the Homer Simpson into the bushes meme, [Laughter] which if no one knows what that is, it’s a meme of Homer Simpson being embarrassed because— let’s say his sports team isn’t doing well and then he goes into the bushes with one sports team shirt, he comes out of the bushes with the winning sports team shirt, something like that. I’m always confused about what does that actually mean? What’s the intention of it?
Kristen:
There is no intention communicated in the description. It just says face in clouds with eyes peering out. That is left to everyone’s interpretation. That’s a cool one. I never even knew about that one.
Damien:
This is an emoji that I’ve never used. But I noticed that it’s just below like a red-faced emoji that has it’s tongue sticking out and a drop of sweat.[no emoji] Like that one looks really really hot. And then just below that one is a blue emoji[what?] that’s like frozen with icicles coming from its mouth. So, just below it is the foggy one. So, I wonder if these are just like experiencing weather emojis. I can’t say I’ve ever used the fog one. Maybe next time I’m in a big cloud of fog. I’ll send it and be like hey, guys, it’s super foggy outside. [Laughter]
Kristen:
Yeah yeah yeah The air quality alerts. Yep.
Damien:
Right Right Right Right. That would have been a good one when the smoke came down.
Markus:
I thought it was like a wise old elderly person that had like the gray hair and the gray beard and the gray mustache when I first glanced at it before I looked closer to it and saw that’s more of like a fog or a mist of sorts.
Kristen:
Oh gosh. The freezing face with icicles ,[no emoji] I’ve only used when I had COVID. And, I sent that and I was like I’m cold. [Laughter] Weather emojis. Now we’re getting into mythology, I think. Any other ones you want to hash out.
Markus:
I was curious when we were talking before, we recorded but again I want to talk about these because I think they’re pretty cool are the Heavy Metal hand Sign and the I Love You sign language sign because they’re very close. One for the metal sign your thumb is in on your two middle fingers with your pinky and your index finger up and you’re like giving somebody the Maloik. And the other one is the I Love You sign language, the I – L – Y . I love this emoji so much.
Kristen:
The first one just says hey I’m making the sign of the horns.
Markus:
Yep, the maloik.
Kristen:
And then the other one is clear. It just says I love you hand gesture and that one has a skin tone associated with it. We will get to skin tones in one moment because that’s a whole separate discussion. But before we leave just kind of chatting about you know any old signs, we have to talk about the one that introduces this episode. I picked it because it kind of communicates how I feel about emojis. But it’s the smiling pile of poop. And Damien was talking about how people have misinterpreted that one.
Damien:
Yeah, I think this is the classic example where back in the day when emojis first came out and people were just experimenting with using them, many people interpreted this one as soft serve chocolate ice cream, and it is clearly not that. Well, now we know. At first, it wasn’t clear at all. But now we know like, oh no no no, that is not ice cream. But I think a slight improvement to make this slightly more clear for anyone that’s still confused is maybe if they put like a green waft of like stinky steam coming from it or something, because right now it looks like a very happy emoji, given that it has a smiley face. And, Yeah, it’s not chocolate ice cream, it’s something completely different. [Laughter]
Kristen:
Is there one for the fist bump too?
Markus:
Yeah, there is. It’s a right-facing fist .[nothing]
Kristen:
And why is it right-facing?
Markus:
I have no idea because there’s one that’s forward-facing too [nothing] and there’s one that’s left facing.[nothing]
Kristen:
The forward-facing one says closed fist. But aren’t all fists closed?
Markus:
I would think that they would say like fist bump or something like that. Because I think everybody that I know has used that emoji. There is a fist bump emoji with two fists bumping. But I think people who have used that have used that and then maybe with the expectation that you fist bump them back. Like “Hey, I got it, you got it? You got it, I got it, good.”
Kristen:
Right right right. So the expectation is already communicated without anyone needing to respond.
Harper:
As an avid user of the horns emoji. I would be remiss in saying that the sign of the horns emoji is from my alma mater, the University of Texas at Austin. And, it’s the university’s hand symbol at you know sporting events. And that’s kind of what they go by. It’s the pointer finger and the pinky finger up and every other finger down. So it’s supposed to look like horns of the Longhorn.
Markus:
And they use it in the movie, The Breakfast Club, when they’re in detention and the teacher is like “if you mess with the bull, you’re gonna get the horns” and he totally does those horns at them.
Kristen:
So I was thinking about horns totally differently. That’s interesting, but OK. Now, I think about Longhorn cattle and like duh, why didn’t I think about that? Thank you, I appreciate that. That would fit right in there. Let’s talk about another aspect of emojis. When we think about emojis, I think about them kind of as an accessibility issue. But I think we should also think about the ways that they intersect, and of course, they would also intersect with race and gender and orientation, as well. A lot of emojis have different skin tone options. And, when you select the emojis, how do you handle the skin tone option?
Damien:
I think this is really interesting. I appreciate the fact that one can choose various shades. I think that advent to emojis along the way helped to increase representation and was a progressive move, I appreciate it. So, this is something honestly that I’ve struggled a little bit with as a white man. Because I don’t tend to use the white skins option very much. It makes me a little bit uncomfortable in the sense that like I’m pointing out my whiteness, I’m not sure. And so, the happy face emojis are all yellow-skinned. In my mind, they’re very general, like anybody can use them. They don’t necessarily say anything about race or ethnicity. They’re just a cartoon. So, those are all yellow. When I choose, for example, to use, like the thumbs-up emoji, I tend to go for the default yellow option, because I almost feel like if I change that on purpose to be a white skin tone, like, am I pointing out my privilege? I’m not really sure. It’s something I’ve questioned, and I just feel slightly like hmmmm as a white man who comes with all the privilege that’s associated with that, like do I want to point that out and an emoji? I’m not really sure. But at the same time, I appreciate that there are diverse options for people who want to highlight the diversity of skin tones. So, that’s my take. I think it’s a little complicated.
Kristen:
Oh, for sure. Yeah yeah yeah. I don’t know if that could also be interpreted as like, if we don’t use the white-skinned one or the lighter medium-skinned one maybe if we are continuing to reinforce ourselves as the default. But I don’t know there’s not an easy answer for that one. So, I have a weird conundrum with skin tones, and it’s just me. So, I’m totally blind. I am white also and I live in a multiracial household. So, my husband is African American, and my children are biracial. My oldest son who’s now 12, was about like seven, he went through his like “oh wow emojis phase, I’m going to use emojis.” And he didn’t have his own phone and he would send messages on my phone then I would send in voiceover. But he made a bunch of my emojis dark skin tone because he has darker skin. And, that was how he identified them at the time. And, in voiceover, you can adjust the skin tone, but it’s a little tricky. I’m not sure why it’s so hard, but like holding it down doesn’t always bring up those options. Sometimes it does, and sometimes it doesn’t. Maybe it’s the way I hold these down. I think my voiceover gesturing could probably use a little work. But as a result, there are skin tone emojis that are darkened, and there are other ones that aren’t. And so, the ones that are darkened, I don’t use because I can’t change them [Laughter]. And, you know, I feel like it’s wrong for me to use them. It’s just really funny. You know how it’s funny and it’s not because it should be easier for me to adjust them, but it’s not. But then you were telling me yesterday about some of the skin tone emojis that don’t look right.
Damien:
I have explored emojis a bit. And, what’s kind of interesting is some of the people emojis, you can choose a variety of different skin tones, which I appreciate. Like, if you’re particularly trying to point out that this emoji is supposed to represent me – Like, I think in that context, it’s good to have the options. Right? Like sometimes I just want to send a general message. I’m not necessarily saying like I’m associating this emoji with me, but maybe in some instances, I would. There are emojis where you can change the skin tones. But then I’ve noticed interestingly, sometimes you can’t change the hair color. So you can have an emoji that has a variety of shades from light to dark, but then the hair is always blonde. As a recommendation to emoji makers, I’d like to throw out there into the emoji verse, for those people who design them to also focus on increasing representation in that way. Because it is a little bit strange to have the option to choose a darker skin tone, but then still have blonde hair. I have noticed though other emojis like that actually is an option. You can have a variety of skin tones and hair colors, it’s just not consistent.
Kristen:
Can you give me an example of one where the hair color changes and one where it doesn’t?
Damien:
I have to take a moment to find them.
Harper:
In the meantime, I just sent an emoji that I found interesting. It’s two people doing a like can-can dance. The thing that intrigued me about it is that for nearly all if not all of the single person emojis, you can select one of the predetermined skin tones but for this group of two people, you cannot select different skin tones for either both or one of the people. They are all the quote default yellow.
Kristen:
Ohhhh. Wow. Now that will be really cool when the technology catches up to allow people to alter one or the other. But this just says dancing people wearing bunny ears and doesn’t say the can-can, which I think it should, personally.
Harper:
I’m experimenting more and I found [emojis of two people in love] and you can select the different skin tones for each of the two people.
Kristen:
Okay. So, they’re thinking about it. Yeah, it says man and woman with multiple skin tones, which obviously like that doesn’t mean anything to me, with heart between them with multiple skin tones.
Markus:
That’s good. My guess is they’re probably trying to reconfigure all of the emojis one at a time to do that. It’s probably time-consuming as H-E double hockey sticks.
Kristen:
Yeah, but I mean, it should say for instance, like “man with dark skin tone and woman with light skin tone” or whatever it is. That’d be really cool.
Damien:
I think things have improved a lot because the only one that I’m finding now that still maintains the blonde hair is the one we talked about. It’s the elf [nothing] which you can change the skin tone, but it’s always blonde. But that’s a fantasy character.
Kristen:
Here’s the interesting part from an accessibility standpoint is that any character that you send will be described with a skin tone and is not described with hair color. So, no character has any hair color at all in the descriptions. They might not even have hair for all I know. [laughter]
Damien:
So something that I’ve noticed about emojis, and skin tones is that you can change the skin tone and represent a variety of different skin colors from lighter to darker. In the past, I had noticed that that wasn’t so much of an option with hair color. You could perhaps choose a darker skin tone, but you couldn’t really change the hair color to match that. But it seems like things have gotten a lot better. And, now, instead of always having a default blonde hair color, you can change the hair color as well. So emojis are becoming more diverse in a variety of ways, both skin color and hair color. But one interesting kind of exception to that is the elf emoji.[nothing] You can have a variety of different elves with different skin tones, but they all have blond hair, which is interesting to me.
Markus:
That seems to be fantasy-based where they might be trying to stay consistent with the hair color of the original design of the idea. But I noticed also with these emojis and with the dark-skinned elves that have the blonde hair, the male and the female have significantly different hairstyles where the male has more of a mullet hairstyle that’s shorter on top and the front and then long and flowing in the back and more masculine looking whereas the female elf has a long full almost like royal princess long hair And so you notice the difference between the sex as well. But yeah, I find it interesting that they’re all in blonde.
Kristen:
Wow, that’s really cool to know. I didn’t know anything about hair color because right now the Apple emoji descriptions do not announce hair color at all. I would love for hair color to be announced and for it to be adjustable. I’m guessing it’s all a huge work in progress for the folks working on emojis. Now I know that there’s hair color involved too. Thank you.
Damien:
And, that makes sense. Given that elves are kind of a fantasy creature that their hair color is consistent in that way. Something else that I noticed that’s kind of just funny to me is the vampires.
Harper:
I noticed that too.
Damien:
Do not have diverse skin colors. They just have shades of gray.[image not described]
Markus:
But they have diverse grays. They have a lighter gray and a darker charcoal gray. Is one of them like an older vampire, one of them a younger vampire
Kristen:
That one says man vampire with light skin tone and then man vampire with dark skin tone. Now that’d be nice if I knew that they were different shades of grey in the description.
Damien:
Right, because they’re not natural skin colors. They’re vampires. Right? So they’re undead beings. But I think it’s just very interesting that that’s the only one where there are shades of grey, which I’m sure it was very purposeful.
Kristen:
The colors of the undead.
Harper:
I think this goes back to kind of a frame of reference that people have for these mythical creatures. For vampires, it’s probably Dracula at some point. And for blonde hair to elves, my frame of reference personally before I started getting into fantasy role-playing games was Lego las in Lord of the Rings. So, because he had long flowing blonde hair, I assume that for a shorthand, all the elves have long flowing blonde hair, which is an interesting discussion unto itself.
Markus:
I think even in the early images in the early Dungeons and Dragons, monster manuals and character books all the elves had long flowing hair. Yeah, they all had long hair, for sure.
Kristen:
Elves don’t give each other haircuts, I guess. [Laughter]
Damien:
Something that’s interesting about the zombies, so you can select a female zombie, a male zombie, and there are no skin tones other than green. The zombies are always green. That’s the only option. But you can also select a non-binary zombie. And, there are also non-binary options for some of the other fantasy creatures and other emojis in general, which is quite interesting. It’s interesting that emoji makers are thinking about this in the year 2023 and there’s an option if you want to, to send an emoji of a non-binary zombie.
Kristen:
I love that. And it is described so there’s it says woman zombie which I think is funny. Like shouldn’t it be a female zombie. Anyway, that one says non-binary zombie. I do wonder, like I’m not sure that the zombie is the best example to talk about this, but when something is male, female or non-binary, how is non-binary differentiated visually?
Damien:
Visually? It seems like the main difference is the hairstyle.
Kristen:
Okay. What do you think about the disability emojis since we’re talking about disabilities all the time. So, we’ve got to end with a disability emoji I think before we move out of emojis and into other categories. Do you think they look pretty accurate?
Harper:
We’re focusing on disability emojis and something that’s interesting to me is that there are only certain types of disabilities that are being covered here. People who are in wheelchairs. People with things that likely signify that they are likely blind. There’s one that might be a person with hearing aids or something like that. But something that I find interesting is that they only provide certain models of let’s say, wheelchairs, or they actually don’t have any service animals or service dogs, which I think would be an important one to include. But if there’s someone who has a certain type of wheelchair, a certain type of disability, maybe they’re flying, and they need a certain method of accommodation that’s not listed here at all. Good catch Damien with the just service dog emoji although it could be just a dog. It doesn’t say service dog on it.
Kristen:
Yeah, they identify that one as guide dog which is also limiting. Right? Because there are dogs for other purposes.
Markus:
It looks more like a service dog than a pet dog, as far as the emoji is designed as well.
Kristen:
Oh, here’s service dog There we go. So, huh? No one’s walking with one.
Damien:
I’m sitting here playing with my phone and typing in some keywords to see what comes up. I was able to access the guide dog emoji with the keyword blind. So, with that keyword, I got a cane all by itself. I got people walking with canes. And a diverse array of people with different skin tones and different hair colors, but also the guide dog.
Kristen:
Okay. That’s interesting to think about how to search for that, because when I searched for it, I just used dog. And then I got all the dog related ones including guide dog and service dog. The person gesturing to her ear and then the hearing aid . Did you use ear hearing in the search?
Markus:
I put hearing in the search, and it came up with the hand pointing at the ears.
Harper:
And there’s some motion lines in that. I looked up aid and found hearing aid or an ear with a hearing aid in it. Although it could be confused for an over the ear earbud if you don’t really know what specifically you’re looking at. I also found a crutch as well under aid which is interesting, along with the bandages and some other stuff .
Kristen:
And your point Harper about disabilities, like only some being represented is really interesting. I think that, you know, there are so many invisible disabilities. And that the symbols haven’t caught up with those could be a profound way to leave some people out. I didn’t think about that before.
Harper:
Something that I was looking at previously, something that came to my attention was I was trying to find a Disability Pride Month flag for a social media post. I couldn’t find an emoji for that. When you look up pride in the emoji search box, it comes up with the LGBTQ+ Pride Flag and the I believe it’s the trans rights pride flag . Disability Pride is not there, and I’m kind of shocked by it.
Markus:
And if you search mental health, nothing comes up at all in the emojis.
Kristen:
So we have some work identified for those who create these emojis if they haven’t figured it out already. What about other parts of the phone like when you are trying to find other pictures that aren’t covered by emojis? What are your favorites?
Damien:
For me, I tend to use emojis the most when I’m texting. But I also use GIFs every once in a while. Not so often, but occasionally and GIFs are really interesting because they’re also quite visual but incorporate movement. And, they usually have some kind of like short video clip that loops again and again. There might be some text associated with the GIF. And so, you know it’s interesting, we were just talking about emojis as a category. It seems like emojis have come a long way since they’ve started in terms of becoming more diverse and more representative but at the same time have a way to go. And, like anything in our society, we’re always trying to get better, we’re always trying to improve. GIFs are interesting because I feel like a lot of GIFs are created by just everyday people and then published. So there’s a really really wide array of all kinds of GIFs that are out there. And, we were looking at GIFs for the word confused. Kristen, I sent you a GIF and I’m going to send it to you again now. I’ll describe it visually. So, this is a GIF of a man who’s standing and kind of trying to figure out a math problem. He’s holding up his fingers, one finger then another finger, and there are floating math problems all around him. They seem like they’re particularly like advanced math problems, calculus problems. And, this is a GIF that comes up when you search for the word confused. So, if I wanted to convey the emotion of being confused, I might send this to you. There’s no text along with it. But Kristen, what comes up on your end when you get this GIF?
Kristen:
When it first came to me, it said, one image. So, that’s all it said when it first arrived. When I go back to read it again, it says includes picture and it says GIPHY via hashtag images. And, then I’m going to open the picture and see if anything happens. And, all it says is photo, crisp well-lit image. So, that doesn’t tell me anything about math problems, or confusion, or anything at all.
Damien:
So, I’m wondering about this, because first of all, who created this GIF? I wonder if this was a professional GIF developer or just a person who made it and published it. And, I’m thinking well clearly that description is not very helpful for you. And so, that’s an area of work that needs to be done is like improving the visual descriptions of GIFs, so they’re more accessible for people who are blind or visually impaired. I also wonder if just the idea of everyday people who are having fun making GIFs. For example, if like the GIF maker apps— I don’t know if they do this or not, I’m not a big GIF maker myself, but I wonder if there’s an option where as a person using a GIF maker app, if you can include a description or not. If it’s not something that’s available, I’m going to put this out there for all the GIF apps out there that make your apps like that, that should be something that is included to increase accessibility. But also, if you’re just an everyday person making a GIF, let’s raise awareness about the need to like in your process of making a GIF, put that little description in there. I’ll tell a funny story. My niece Callie came to visit last week and one of the things that we did with her is I took her to see my parents, her grandparents. And, one of the things that we did for fun is we learned how to make cappuccinos from scratch. This was like a big event for the family. Then two days later, we were hanging out in the city, and we went to Eataly, the Italian marketplace in the Flatiron District on 23rd Street, and we went to the cafe. And, in the cafe, they have a poster that says the perfect cappuccino. And, it shows the proportions of espresso to milk foam to steamed milk that you put in there. And so I was like Callie, this is perfect. Let’s take a photo in front of this poster. And we’ll send it to people in the family. So, we kind of pose in front of the poster and I was ready to do a selfie and by accident, instead of shooting a photo, I accidentally shot a video. It’s a tiny, short video. It’s only three seconds long. But when I pressed the button, and then I realized it was the video, I was like “oh it’s a video.” Callie wasn’t really ready for that, so she got a little embarrassed and put her hands in front of her face like oh, no. She said she wasn’t quite sure why she had that reaction. She just wasn’t expecting it to be a video. She was expecting it to be a photo. But then we were talking about it, and we were like Callie, this actually is a really cool little, short video. Let’s turn it into a GIF. So, we got our coffees, we sat down at the bar, and we made a little GIF. And Callie thought that this would be perfect for any time you are embarrassed about something, and you make this little face of putting your hands in front of your face. So, we put text at the top of it saying “That face that you make when…” with the intention that somebody could send this GIF and then follow it up with, you know, whatever they’re embarrassed about. Like that face that you make when you realize you accidentally farted in public or something like that. So, we created this GIF, but I didn’t see anywhere in the creation of it—it might exist, I’m not really sure. But we didn’t see anywhere where we could add, like a description for people who are blind or visually impaired or anybody who wanted to access that and not just rely on the visual information. But Kristen, I’m going to send you our GIF and can you tell us what comes up for you?
Kristen:
Yeah, I’m excited. Also, I think that’d be fun for you to do with Langston sometime to make a GIF. Yeah, send it.
Harper:
While you’re getting that, Damien, I actually have an inside scoop on the GIF that you’ve sent. This is from Rooster Teeth. It’s a media company that I’ve followed for a while. It is Greg Miller of one of their in-house brands Kinda Funny. I believe they’re in-house. And, having seen this podcast set before, those numbers, the math equations, those are not part of their regular video routine. That was either done by the company and put online or someone else a fan or a different content creator, they put those equations online. So either way someone knew what they were doing and did not put alt text in this GIF as I feel like it really needs it. Because there’s so much context that’s not there. And, one of my real pet peeves in accessibility is going on to Twitter, or other platforms, or X now, I guess and when photos and links or really any sort of links are there; GIFs. Anything like that, if there’s alt texts, a lot of times, it’s just the link to the source file, which I guess is great for giving credit. But that’s not what it’s used for. That’s not the purpose of the alt text space, it only hides the credit to the original creator or the source file. And, it doesn’t give people who need alt text the information that the alt text space is there to provide. So, it’s really a lose-lose when people may feel that it’s a sort of win-win.
Kristen:
At first, when I swiped back and forth across it, all it said was photo crisp, well-lit image and I was like, wait, that’s not what it did yesterday, when we were talking about this. Just to check it out and see what happened. And so what happens is, is when I swiped back and forth across it, it kind of changes like the phone is catching up with what it’s trying to, quote-unquote, see or read. So the first few times I swipe across it, it says crisp, well-lit image. Now when I swipe over it, it says that face you make when you and then it says dot, dot, dot, the perfect cappuccino espresso, and then it’s trying to read the milk foam ratios a little bit from the poster. So, first, it pulls your text, then it pulls the poster and so, what it actually communicates to me is the face you make you have the perfect cappuccino, not that face you make when you’re embarrassed about something. So, it’s completely a different face, I think that you would make there.
Damien:
That’s really fascinating. It’s fascinating to me that your phone, I guess this is artificial intelligence working, that it’s picking up text in the image itself, not just the caption, or the title that Callie and I put at the top of it. So it’s taking texts from a variety of places, and then it kind of comes out to you as something that’s not quite what we intended.
Kristen:
Yeah. What is the face you’d make when you have a perfect cappuccino by the way?
Damien:
I would probably use my relieved face emoji [Laughter] is calmness and serenity.
Kristen:
Well yeah. That’s great. What a full circle [Laughter] we’ve made as we come back. So, just to kind of end this with a little bit of a call to action, what would any of you recommend to emoji makers, or GIF makers or you know anybody?
Damien:
As I see it, I think emojis have been improving along the way. It looks like emojis are getting more representative, but there’s still a way to go. And we’ve identified that there probably should be more representation in the disability sphere. But it seems like the biggest thing overall is having more description so that the emojis—so this is interesting, something I was thinking about actually is we want them to just describe the picture as it’s presented with very little interpretation. Right? We want these to be low inference descriptions, so that the people who are using them are conveying the intention or the interpretation, because yeah as we saw, for example, with the horns emoji. If the horns could be interpreted as both sending bad luck to someone, but also just a symbol for the University of Texas. But if you have a really clear explicit description that just says this is what the symbol shows, then it can be used in multiple contexts. It seems to me like a piece of recommendation for emoji makers is to focus on making really good clear descriptions that are low inference. I don’t know, perhaps having something in parentheses that that says could be used for if it could be helpful to include some of that interpretation. But have that sort of separately. And then I would say in terms of the GIF, it seems like that’s an area where we need more work in terms of making the descriptions that come along with GIFs less gobbledygook. If GIF maker apps don’t already include a little section where creators can put that description on their own, that’s something that should be considered because right now, it seems like what you’re getting is quite confusing.
Kristen:
Just having a caption, like a photo caption space or a space for the alt text, and going where people can add that if they want to, is really critical. And I think there’s a lot of work to be done and it’s not necessarily all on the describing end. For instance, definitely a lot of it is on the describing end. I would also add just probably, for me, I would need to learn and maybe others do as well, or have learned already, what expressions look like. So, for instance, the thinking face—on the emoji side, I didn’t realize that it’s the hand-to-chin thing. But then I sort of wonder on the human side, what are the different expressions that constitute thinking and why? And how do we get those? It’s pretty cool. Any other thoughts? Markus or Harper, before we wrap up?
Harper:
Yeah, absolutely. I think that if we’re going to give the option for alt text in GIFs and maybe emojis, that’s a little bit more complex, just make it required, not even optional, just make it required with maybe like a quick bulleted list of guidelines. If It’s more complex, do these three steps. If it’s less complex, just say what the character and what they’re saying is. It will be tough in the beginning and by tough, it may take three minutes. But after a few weeks of doing this, it will take maybe 30 seconds, and the road will be much better off for it.
Kristen:
I love making it a requirement. That’s amazing.
Markus:
I would have to say that the question of what expressions look like is very important moving forward because as we have all seen, just in this discussion right here earlier, one emoji means three, or four different things to the four of us in this conversation at times. We all translate it differently. So that has to be taken into consideration. We have to make sure that we learn to put our feelings aside at times, because many times we misinterpret what is being said, and that leads to hurt feelings or being upset or being disgruntled or being dismayed, or whatever the case may be. And, to avoid that, hopefully, we can get into the frame of mind where we’re able to set those emotions aside when dealing with emojis, or even better yet, email and communicate more, and just use them as a side part or for emphasis.
Kristen:
And also, I think there is a little bit of fear about asking what somebody means. Maybe there doesn’t need to be any hurt feelings unless you’re sending emojis in a fight, I guess. But if you’re just chatting, you know, then it’s fine to pause and ask what that means because it can actually lead to a richer conversation. Well, thank you, all of you for indulging me on the emojis, tangent rant. I think we’ve got some fun insight for people to hear and hopefully some good work for the emoji makers, the GIF makers who are already doing great work. And hopefully, they’ve thought of some of this already. Thank you all for coming onto the pod today.
Damien:
Thank you, Kristen.
Markus:
Yeah, thank you for having us. It was a lot of fun talking emojis with everybody.
Harper:
It was. It’s such a great, like pervasive topic everyone experiences but not everyone really thinks about in their day-to-day life.
Kristen:
If you liked what you heard today and are more interested in hearing about emojis, turn on the accessibility features on your phone or computer. It’s very easy, especially on an iPhone to turn on voiceover and you can listen to them and find out what they’re like when you’re sending them to your friends who are blind or low vision. If you liked what you heard today and want to explore more about digital accessibility in general, technology or company culture, and more. Schedule a time to meet with us. You can find the whole Tamman Team at tammaninc.com. That’s T-A-M-M-A-N-I-N-C dot com. Don’t forget to sign up for our newsletter too, so you never miss an event or an insight from us. Help us spread the word about accessibility by rating our podcast five stars on Spotify, Apple or wherever you listen to your podcast. It really helps our podcast grow and reach new audiences. Also, you can hit that subscription bell icon, so you never miss an episode. If social media is more your style, you can follow us at Tamman Inc. on LinkedIn, Twitter now known as X, Instagram or Facebook and share our podcast on your favorite platform. We look forward to continuing the conversation with you. Talk again soon.
Show Notes
- The Unbearable Ambiguity of Emoji
- Article 19’s episode on Marketing with an Accessibility Mindset
- Article 19’s episode on Digital Design and Accessibility
Up Next
Podcasts
Marketing with an Accessibility Mindset
