Two blind e-gamers share their chats

Every week, we get on the phone with ChatGPT Pro subscribers to learn how they’re actually using the product, because they tend to push the edges of what’s possible. Today, we want you to meet Tokyo-based best friends and e-gamers Masahiro Fujimoto and Naoya Kitamura. We hope their story sparks ideas for how AI might become useful in your own field.

“For someone like me, generative AI can serve as a pair of eyes,” says Masahiro Fujimoto.

More than a decade ago, Masahiro met Naoya Kitamura in the dormitory of a Tokyo school for visually impaired children. Naoya jumped into a debate Masahiro was having about metro accessibility, and before long, they were discussing manga, music, and gaming. Today, each is usually the first person the other texts when one of them has a new idea, a technical problem, or an adventure to plan.

Masahiro and Naoya were both born blind, and technology has always shaped how they engage with the world around them, from braille at school to the screen readers and text-to-speech tools they use now. But ChatGPT represents the greatest technological shift they’ve experienced. It helps them interpret visual information and build tools around their evolving needs rather than wait for an accessible tool — or another person — to become available.

At a Japanese telecommunications company, Masahiro uses ChatGPT to assess how his planning documents will appear to sighted colleagues, including whether layouts and data presentation are visually clear and appealing. “I no longer need to bug someone as often when I want to complete a task, research something, or explore one of my crazy ideas. It’s reduced a lot of stress for me because I can just do things by myself,” says Masahiro, adding, “I do still bug Naoya to help me on projects, though.”

Naoya, an engineer, voice actor, and esports player with ePARA, uses ChatGPT and Codex to build accessibility tools, including a one-button web app that summarizes articles and PDFs. An avid gamer since the days of 16-bit sound chips, he is also using Codex to develop voice-enabled gameplay patches for mainstream titles that lack native accessibility features. “Most companies assume patches are just so that people can cheat, but what we really want is for manufacturers to understand that these add-ons are needed for accessibility,” says Naoya. “We hope this will encourage them to consider accessibility from a game’s initial development phase.”

Now their skills are coming together in a speech-based Japanese poker app for blind beginners, inspired by a boom in poker’s popularity in Masahiro’s office. Masahiro developed the concept, requirements, and interface; Naoya is building it.

This is why AI feels different from any other accessibility tool they’ve used: instead of simply helping them navigate systems designed around sight, it gives them greater independence and new ways to reshape those systems for themselves and others.

Here are eight prompts from Masahiro and Naoya (all of which have been translated from Japanese):

I’m playing Street Fighter 6, and I think I missed an element of gameplay because I can’t see.

The video I’m attaching is a match screen of my gameplay.

Around the 9-second mark, can you confirm that the character on the right activated a move called Drive Rush?

[Replay.mp4]

Why was this chat valuable? “By asking Codex for a frame-by-frame image recognition of Street Fighter gameplay, I was able to confirm that there was a slight delay in the timing I could confirm by ear, which has implications for my gameplay. I heard the move ‘Drive Rush’ at 9 seconds, which is 0.15 seconds after it appears visually in the game.”

I want to build a poker app for visually impaired people. Someone else will handle the development of the app, so let’s not go too deep into the actual programming details. For now, I mostly want to define the requirements.

Also, I do not know much about poker myself, so I would like to build this while learning the rules.

To start, here are my prerequisites:

• It should be playable on smartphones and tablets

• Visually impaired users should be able to play using VoiceOver

• It should be enjoyable for everyone, from beginners to serious players

Why was this chat valuable? “Even as a visually impaired person who had never built an app before, I was able to create a compelling development plan. Also, ChatGPT quickly looked up the field of visually accessible poker and gave me a sense of what already existed and what I could improve upon. I’m developing the strategy and deployment plan, and Naoya’s going to use Codex to build it.”

In Hearthstone Deck Tracker, a tool for analyzing Hearthstone, please create the following voice features for Battlegrounds:

1. In-game, pressing N brings up opponent information, but please have the tracker also read aloud:

a) hero power, b) trinkets, c) quests, d) rewards.

2. Please add a voice notification when another player raises their Tavern Tier.

Example: “Skycap’n Kragg has upgraded to Tier 2.”

3. Please make it possible to swap the order of minions in the Tavern with the space key. Use the same logic as swapping minions on the board.

4. When I press Ctrl+1 through 5 on the board, please mark the minion cursor as ‘on.’ If a marked minion appears in the Tavern, please play check.wav.

5. Make it possible to emote directly at my current combat opponent.

Why was this chat valuable? “I was able to use Codex to create a read-aloud patch as a support-tool plugin for one of my favorite games and use it to improve my performance. I’m still mainly testing it myself, but in the future I’m considering publishing it on GitHub so other players can use it too.”

Do tarot cards for the visually impaired exist, such as cards with braille, or cards where information can be heard through a QR code?

Are there apps where tarot can be enjoyed with iPhone VoiceOver? On a Windows PC, are there browsers or software where tarot can be enjoyed using NVDA or JAWS?

Why was this chat valuable? “I wasn’t able to find any apps or services where visually impaired people can enjoy tarot, which inspired me to create one myself. ChatGPT also helped me think through how tarot can be enjoyed without relying on sight.”

Here are my play logs for this season in Hearthstone Battlegrounds.

Please analyze my play and output each of the following:

• Winning patterns

• Losing patterns

• Room for growth

[Logs.csv]

Why was this chat valuable? “Battlegrounds is a game with a lot of repetition, which lends itself well to analysis. After analyzing my records, ChatGPT helped me realize that my results were too dependent on whether there was a clear path to a ‘strong build.’ When the game required adaptation, pivoting, or making the best of weaker options, my placements dropped. So my main lesson was ‘I need to become more flexible when the ideal strategy is not available.’ Going forward, I plan to develop a program in Codex so that I can also analyze things turn by turn.”

I want to compile a portfolio of my achievements.

I will attach a spreadsheet with my work. I would like you to either reorganize it so that it’s easy to read or give me advice on how to make it easier to read. I can’t edit graphs or images myself, so please give me detailed advice on how to arrange them in the doc.

[Work.csv]

Why was this chat valuable? “I had never made a professional portfolio before, and ChatGPT gave me advice on which elements to include and made it possible to put it together as a visually appealing, easy-to-read document.”

How many ticket gates are there at JR Sakuragicho Station? If there is more than one, please tell me which ticket gate is closest to the steep slope.

Why was this chat valuable? “At stations with multiple ticket gates or exits, ChatGPT makes it much easier to check location details than traditional search. This is especially useful when finding an accessible, convenient place to meet friends.”

Please think of a route from Japan to enter the Eurasian continent without using air travel and go as far as Cairo, Egypt. Also, because I have a visual impairment, I cannot drive a car myself. There is no time limit.

Why was this chat valuable? “In one of my favorite manga, the main character travels from Japan to Egypt, but he can only travel by land and by sea. If he flies, he’s attacked. ChatGPT is helping me plan a route to do the same, but also factoring in accessibility, current events, and safety. Even if I don’t have the money to actually take this trip right now, AI helps me dream up and make practical itineraries for trips that would have otherwise seemed impossible. Someday, when I have the money, I’m going to make this happen. And obviously Naoya’s coming too.”

For more stories and use cases from experts using ChatGPT, subscribe.

Are you a Pro subscriber using ChatGPT to help you level up your work? We’d love to meet you.