Artificial Intelligence in Seeing AI & Be My Eyes

Speaker 1:
Shaun of the Shed, an AMI original podcast.

Shaun Preece:
Ola, and welcome back to another episode of Sean of the Shed. I am Shaun Preece, I'm sat in a garden shed, and this is the show where I talk about technology and how useful it can be to blind or visually impaired people like me. Sighted people, don't run away. Don't hit that backburn. Stay there. You might find something interesting. This could be useful to you as well, trust me. Anyway, hope you're all doing well. Now, today, I want to talk about AI, artificial intelligence. Yes, I'm jumping on that bandwagon everybody is talking about, AI at the minute. But we've got a good reason to because recently, there's been updates to two very popular apps in our community, that being Be My Eyes and Microsoft's Seeing AI, and these updates add artificial intelligence to them and, wow, it's impressive.
Now, I'm going to use that word a lot, I'm sure, in this episode, but it is. It's more than that. It's mind-blowing. It's a game changer and I know we use that term a lot, right? This is a game-changing technology, but you know what? In this case, I think it actually could be. So what is artificial intelligence? Well, basically, it just means that technology thinks. I'm going to use that word advisedly. Let's say technology acts in a more human-like way. We can interact with technology more like it's a human being. We can be more conversational and it can interact back with us in a more human-like way. We can ask questions and it understands the context of those questions, the greater meaning behind it. It can follow the thread of a conversation you have with it.
It's just really impressive. There, I've said it again. It's hard to describe, but you know what? I'm going to show you exactly its potential and why I'm so impressed with it. I'm annoying myself now. So, what is the actual use case for us? Well, in both of these apps, what we can do is take a picture using your smartphone camera or you could grab a photo off a website or social media or wherever and you send it to the AI either in Be My Eyes or Seeing AI, and you can ask questions about it, right? So far, so straightforward, really. We've got automatic image description available already and we've got alt text in various social media platforms. So far, not that impressive, but it's the amount of detail and the responses and answers that you get to the questions you ask about these images that are really, I'm not going to say impressive, really cool.
Again, let's just get to it rather than talk about it. So let's kick off with Be My Eyes. Alrighty then, if you've never heard of Be My Eyes, don't worry about it. Basically, it's an app for your smartphone available for both iPhone and Android. Although I will say, currently, at the time of recording this, the Android version does not have the AI feature, but it will be there very, very soon. So hopefully, by the time you're actually watching this, it will be, so take a look. But it's an app that allows you to connect to a real-life human being, a real person, a volunteer. And basically, it creates a video call so that volunteer can see whatever you're pointing your phone's camera at and you can ask them questions such as, "Hey, I've dropped my keys, can you see them?," Or, "Can you read this letter for me? What's the text on this packaging? Where's the door? Where am I?" Whatever it may be.
It's an amazing app and incredibly useful and I use it all the time/ and that's the core functionality of Be My Eyes. But as I said, with this new update, we now have the ability to talk. Instead of talking to a real person, we can talk to AI, artificial intelligence. But why would you want to do that? Well, there are some cases where perhaps there's some information that you don't want to be read by a real person, a personal or a private letter or whatever it may be, and in that case, AI would be the better option. And let's be honest, some cases where I just don't want to talk to someone. I just want to get a quick answer to something and if I've just rolled out of bed and I'm feeling rough, I just want to talk to AI instead of a person.
There, I've said it. So that's the reason why AI may be the better option. Now, I'm not going to go into setting up Be My eyes from the start. If you want to, you can go back to episode five of Shaun of the Shed. Open up your favorite podcast app and search for Shaun of the Shed, S-H-A-U-N. And as I said, in episode five, I walk through and demo the Be My Eyes app and connecting to a human volunteer. Also in that episode, I demo Aira as well, which is a very similar app and something that we should all have on our phones, by the way. It's an audio podcast, episode five. It's before I move to this la-di-da fancy YouTube. So as I said, search for it in your favorite podcast app. So if you haven't got it already, go to the app store and search for Be My Eyes.
And once it's installed, open it up and you'll find two options. One button saying, "I am visually impaired and require assistance." And the other one saying, "I want to be a volunteer." So that's for sighted people who want to help us out. God bless you by the way. So obviously, in my case, I double-tapped on, "I require assistance," and then it asks for your name, email address, and give it a password. And that's it. You're signed up. Once you do that, on the main screen of the Be My Eyes app, obviously, you may have to give it permission to access your camera and your microphone, et cetera, but once you've done that, on the main screen you'll find five tabs at the bottom of the screen. First one is the "get support." This is where you actually connect to a real-life human volunteer. On that screen, on the main screen above the tabs, you will find just a big "call a volunteer" button.
But another very interesting thing, and a cool thing, is underneath that you will find specialized help. Now, this gives you the opportunity to talk to companies. Now what companies are available depends on where you are in the world, but for example, I use it all the time to connect to Microsoft customer support if I'm having a problem with my computer. And the customer support agent can see whatever I'm pointing my phone at and they're really good. So definitely check that out. But anyway, the thing we're interested in is the next tab, which is the "Be My AI" tab. This obviously, see what they've done here, is where the artificial intelligence volunteer lives. Now, you can explore the other three tabs if you want. They're very straightforward, but obviously, let's just jump straight into Be My AI.
Okay, let me grab my big DJ headphones so I can hear my phone. Ah, that's better. Okay, so I have my iPhone 15 here and I'm on the Be My AI tab. So on the main screen, as I said earlier,

Speaker 3:
Take picture button.

Shaun Preece:
There's the take picture button. So, let's just test it out. I'm going to take a picture of my terribly messy desk. Here we go. double-tap.

Speaker 3:
Take picture.

Shaun Preece:
Now it immediately starts analyzing that image and in a minute, it will give us a response.

Speaker 3:
The picture shows a desk with a computer setup. The main focus is a computer monitor that is displaying a video editing software. On the screen, there is a man with short light coloured hair wearing headphones.

Shaun Preece:
Yes.

Speaker 3:
He is holding a smartphone and sitting in a room with red walls and a guitar in the background. Above the monitor, there is a webcam attached. On the desk in front of the monitor, there is a black keyboard, a harmonica lying next to the keyboard, and a microphone with a round black windscreen on the right side. There are also various cables in the USB stick plugged into the computer. The monitor is a Samsung brand.

Shaun Preece:
Is that it? I'm just going to swipe to the right, see if there's any more.

Speaker 3:
Ask more button.

Shaun Preece:
Ask more button. We'll get to that in just a second. So as you heard, how much detail was in that? That's so much better than the run-of-the-mill automatic image description or alt text. It was very, very cool. In fact, it reminded me there's a harmonica here somewhere. Oh, sorry. Anyway, there we go. Now let's say I want to ask more questions about this image. As you heard, there is a "ask more" button, so let's double-tap on that.

Speaker 3:
Be My AI, the picture ends chat button.

Shaun Preece:
So on this new screen, we can end the chat. I don't have any further questions, but let's swipe on.

Speaker 3:
Your message image. Actions available, Be My AI, the picture shows a desk with a computer set up.

Shaun Preece:
So that's the conversation we've had so far, the image that I sent and it's response. Swipe on.

Speaker 3:
Call a volunteer button.

Shaun Preece:
Now, let's say I'm not happy with the AI's response. The fantastic feature here is that I can call that real-life human volunteer from here and ask questions to them instead, which is a really nice feature. Let's swipe on.

Speaker 3:
Add picture button.

Shaun Preece:
Now, I can add more pictures and we're going to show that in just a second. Next.

Speaker 3:
Message text field.

Shaun Preece:
Okay. So here's the text message where I ask further questions. So I'm going to double-tap here.

Speaker 3:
Message text field is editing, insertion pointed start.

Shaun Preece:
How old is the man in the picture?

Speaker 3:
How old is the man in the picture, insertion point at end.

Shaun Preece:
That's a dangerous question to ask, but let's see what it says. Swipe on.

Speaker 3:
Clear text button. Send a message button.

Shaun Preece:
There we go. Let's double-tap here.

Speaker 3:
Send a message. Please wait. Be My AI, please wait. Be my AI is writing a message.

Shaun Preece:
Oh, I'm nervous.

Speaker 3:
Be My AI. It's difficult to accurately determine the age of the man in the picture on the computer screen, but he appears to be middle-aged. If you need a more precise answer, I would suggest pressing the "call a volunteer" button for further assistance.

Shaun Preece:
Very good. You know what? I'll go with that. I am 50. So as long as I live to a hundred. Very good. Okay, let's try another question. Okay, so I've typed another question in here.

Speaker 3:
Clear text message. Text field is editing. Can you describe the webcam? Insertion point at end.

Shaun Preece:
I know, not very exciting questions I'm asking, I admit, but this is the beauty of AI. It's always surprising and impressive what responses you get. So let's see what it says here.

Speaker 3:
Be My AI. Certainly. The webcam is positioned on top of the computer monitor. It is small and rectangular in shape. It appears to be black in color and has a clip at the bottom, which is likely used to attach it to the monitor. The webcam is facing forward, aligned with the center of the screen.

Shaun Preece:
The amount of detail that it gives you is, quite frankly, amazing. It's so cool. But look, that's a basic, pretty boring picture. So let's try something else. I'm going to grab a photo from my camera roll and let's see what it says about that one. Okay, so I've now opened the camera roll, the photo gallery on my phone, and let's take a look at a photo. So if I swipe.

Speaker 3:
Photo date, the 19th of October. Actions available, a brown dog with a blue bandana around its neck standing on a gray surface.

Shaun Preece:
There you go. So there's the image description that the iPhone gives me of that photo, a dog with a blue bandana, I think. So let's see what AI gives us on that one. So I'm going to double-tap on that and I'm going to find the share button. Should be at the bottom somewhere.

Speaker 3:
Toolbar, share button.

Shaun Preece:
There it is. Bottom left corner. Let's double-tap.

Speaker 3:
One photo selected. One photo selected.

Shaun Preece:
And again, let's swipe through.

Speaker 3:
Add to album button. Copy foot, add to airplay button. Describe with Be My Eyes button.

Shaun Preece:
There we go. That's the option we want. double-tap.

Speaker 3:
Please wait, Be My AI is writing a message. Be My AI, the picture features an adorable dog sitting on a carpeted floor in a room. The dog appears to be a Labrador with a light brown coat. It is wearing a cute blue knitted sweater with a white bow on the front. The dog is looking straight at the camera with a calm expression. In the background, there is a white door, a black garbage bag on the left, and a small white table with a plate on it. The room seems to be well-lit and has a cozy atmosphere.

Shaun Preece:
Wow, what a difference between the iPhone's image description and AI. It gives you so much more detail. And did you notice as well, it gives you the atmosphere of the photo? And that's something that you don't usually get, the context behind it. That is so cool. Almost too much detail saying there's a garbage back in the background. I don't think it is, but I don't know. I'll have to go and check, but that is fantastic. Now obviously, all this so far is impressive, yes, but it's also pretty straightforward. Now, I want to show you this feature of adding more photos and how useful that can be. I have with me here a takeaway menu. So I'm going to take a picture of the first page, as we did before, but then I'm going to add more photos of the following pages, and I'll show you why that is so useful. So let's get to that. Okay, so I've opened Be My Eyes again and I'm back to the main screen of Take a Picture. So let's do that with the first page.

Speaker 3:
Take picture. The picture you've shared is of a menu from a restaurant likely serving Chinese cuisine. The menu is divided into sections with pictures of some dishes on the left side and the list of dishes with prices on the right side.

Shaun Preece:
Now as I said, I want to add more pages of this menu. So let's double-tap on "ask more".

Speaker 3:
Be My AI, the picture you... End chat. Be My AI, call a volunteer button.

Shaun Preece:
And let's swipe through.

Speaker 3:
Add picture button.

Shaun Preece:
Add picture. So I'm going to double-tap on that and then I'm going to take a picture of the second page and so on. I'll be back in a second. Okay, so now I've taken a picture of the second page. So let's ask a question on those two images. Okay. And here's the question I've asked.

Speaker 3:
What chicken dishes are there?

Shaun Preece:
Okay, let's send that through

Speaker 3:
Be My AI, in the new picture you've shared, there are several chicken dishes listed on the menu under different sections. Under chef's special dishes, 42B, crispy shredded chicken and [inaudible 00:15:25] chili sauce for £7.42c. Crispy shredded chicken and [inaudible 00:15:29] soy sauce for £7.42. Crispy chicken and orange or plum sauce for £7.46.

Shaun Preece:
Okay, I'm getting hungry. But you saw the ability to add multiple images and then ask questions about all of those is really cool and it's fantastic for things like this. So that is the basics of how to use the AI feature in Be My Eyes. Now let's move on to Seeing AI. So what is Seeing AI? Again, it's an app for your smartphone. This time, I'm afraid only available for the iPhone, at least currently. So sorry, Android users. It's not the same as Be My Eyes in that it doesn't connect you with a human volunteer, but it does give you access to some really useful tools. Using Seeing AI, you can scan text in a couple of different ways. The one I use all the time is called Short Text. And all it means is that if I wave my phone in front of anything, really, if it sees any text whatsoever, it'll just shout it out in real-time.
It's very quick, very rough and ready, but it's really handy for identifying things. If you want a more accurate reading, you can use the document scanning feature, which will scan your letter, your post, or whatever it may be, and read it out to you. There's also handwriting recognition, so if you want a greeting card to be read, it can do that for you, which is very nice. There's just so many, light detection, so you can check if you've left a light bulb on or color detection to see what color things are, currency, scene detection, just so many useful tools. Now, the thing, obviously, I want to show you is the new AI feature and you will find that in the "scene" tool. Now they call these various functions Channels. Now if you don't know what I'm talking about, you've never used Seeing AI before, don't worry about it, it's incredibly easy to use.
So go to the App Store and search for Seeing AI. And once you install it, simply double-tap to open it up. Now, the first time you run the app, it will give you a little description of what each channel, as it's known, each feature does. It's very good at giving you info like that. So all you need to do is swipe through until you hear, "Channels," and then swipe up or down with one finger to go through the various channels there. The one we're interested in is called Scenes. Okay, so now, as I said, I'm on the Scene channel. Let me just double-check that.

Speaker 3:
Product, scene preview.

Shaun Preece:
Perfect. Okay. So now I'm just going to go to the very first item at the top of the screen and let's see what's there.

Speaker 3:
Menu button.

Shaun Preece:
Menu button, swipe on.

Speaker 3:
Quick help button. Take picture button.

Shaun Preece:
Ah, there we go. Keep going.

Speaker 3:
Browse photos button.

Shaun Preece:
So we actually have the ability to search through our photo gallery directly from inside Seeing AI, which is nice. Let's swipe on.

Speaker 3:
Channel, Scene preview, adjustable.

Shaun Preece:
And that's it. So let me swipe back and let's take a picture. Let's recreate what we did with the first one on Be My AI and just take a picture of my terrible messy desk.

Speaker 3:
Browse photos button. Take picture button.

Shaun Preece:
Here we go.

Speaker 3:
Processing. Cancel button. A computer screen with a microphone and a keyboard.

Shaun Preece:
Oh, well, that was very quick, but what a very small short description. But don't panic, that's not it. So let's swipe on.

Speaker 3:
Save photo button.

Shaun Preece:
Save photo, which is a nice feature.

Speaker 3:
Share button.

Shaun Preece:
Share. Keep going.

Speaker 3:
Explore photo button.

Shaun Preece:
Let's keep going.

Speaker 3:
More info button.

Shaun Preece:
More info. Now, if we double-tap on this, this will give us the AI description. So that's what we want, but I just want to give you a quick tip here. You actually don't have to swipe through all that. You can also just double-tap on that short description. Let me swipe back.

Speaker 3:
Explore photo, share button, save photo. A computer screen with a microphone and a keyboard.

Shaun Preece:
So if I double-tap here, it's exactly the same as double-tapping on the more info button, let's double-tap.

Speaker 3:
On a computer screen, there's a man who appears to be holding a phone and a microphone. In front of the screen, there's a keyboard and a black round object on a surface. To the side, there's a book and a knife. The screen displays text that reads [inaudible 00:20:10] collection tool help. Other visible text includes scenes, OAV and Samsung. There's also a camera positioned above the screen.

Shaun Preece:
I don't think that was quite as good as Be My Eyes. And it's very interesting because, actually, both Seeing AI and Be My Eyes use exactly the same artificial intelligence service that is called ChatGPT. They're sending the image to the same service but getting different responses and different descriptions. Now, which one you prefer or which one you think is the better is down to experience. And I think, to be honest, try them both because you can get different responses for the same image. It's really interesting. Now, there were a few problems with that one, I don't think I have a knife on my desk. I hope not, anyway. But other than that, again, it gives you that level of detail, which is pretty cool. Now, the only thing I haven't found is the ability to ask further questions here. It seems to be a one-question or at least a description of the image.
Now, this is very early days for Seeing AI. This, at the time of recording, has only been available for a couple of days, so it is in preview mode, as they call it, almost beta in testing. So I'm sure this will get more and more features. But again, just a great little feature to add to the already great tools available in Seeing AI. And I do like the ability to browse your photo gallery and grab an image directly from inside Seeing AI. It's very nice. So, let me put this down for a second. Take these headphones off again. Nice. Okay, so that was a very, very quick look at artificial intelligence, but I'm sure you can see from that quick demo just how useful it could be for us. I honestly have been using Be My Eyes all the time. I think it's just so cool. And that ability to drill down and ask further questions and the ability to add multiple images is something I think you'll find really interesting.
I just showed you a takeaway menu there, but you can do the same thing with documents. If you get a letter and you want to just simply ask, "Oh, what's the telephone number for so-and-so?" Or, "What was the date of my appointment?" You can ask that without having to listen to the entire document again, which is just so cool. Honestly, you need to test out these features because I think you're going to love them, especially if you've got lots of photos in your photo gallery. It's so cool to have a detailed description of them because it's something we haven't really had before. Anyway, that's it. Thank you so much for watching. I hope you enjoyed it. If you did, why not hit that "like" button? And if you want to, just take a little bit of my soul, you can also hit the "dislike" button. Why not? Please don't. If you want to see more videos like this, then tap on "subscribe". Thanks again, and I will see you next time.