（海洋産業AIプロフェッショナル育成卓越大学院プログラム）
（Development of WISE Program to foster
AI Professionals for Marine Industries）

Researching the unexplored combination of reinforcement learning and automated driving

My research involves automated driving.
I use the game creation software Unreal Engine to create realistically impossible simulations in which vehicles are driven using machine learning.
I love computer games and had heard a lot about Unreal Engine. I wondered if I could use it to do research on machine learning and automated driving, and decided to give it a try.

I started programming when I was in high school. I was also interested in machine learning, principally the area of reinforcement learning. When I was choosing my research topic, reinforcement learning was available as a plug-in in Unreal Engine, so I decided to go where no researcher had gone before--to combine reinforcement learning with automated driving.

An interesting aspect of my research is that I am using a game engine. Though many people are researching automated driving, few are using games for simulation. My research is also unusual in that I have prepared automated driving with reinforcement learning. I am alone on the frontier, which is precisely what makes it so interesting and rewarding.

Worries distinct to unprecedented research

Nobody has ever used reinforcement learning in Unreal Engine to study automated driving. While this endeavor is rewarding, it is also quite challenging due to the lack of precedent.

A simple online search is all it takes to find answers to questions about game engines. However, when machine learning and reinforcement learning are involved, I have difficulty finding answers online. In these situations, I would first ask my professor, but I also often consulted ChatGPT. When I don't know how to do a search, or a search isn't going well, I can ask AI a question and it will retrieve the answer from the internet. These answers give me words to use in new searches on my own. I find it very useful, although because it is the internet, I have to check the authenticity of the information each time. This is a situation where I find AI very useful, depending on how it is used.

Automated driving is divided into levels according to accuracy and safety. The highest level--fully automated driving where AI does it all--has not yet been realized. If we can achieve this, it will be a great contribution to society.
Additionally, reinforcement learning may be appropriate in cases other than automated driving. The main advantage of my experiments is that they can be simulated. I can use a game engine to do what cannot be done in reality due to ethical and safety concerns. Machine learning and reinforcement learning should be applicable to many different cases.

Applying for an internship in search of knowledge that leads directly to research

Internships are one of the many appeals of the WISE Program. In August, I did a two-week internship in operations at Furuno Electric Company.

I chose this company out of the many that offered internships because their business is related to my research. Understanding what is happening around the vehicle is crucial in automated driving, but cameras alone are not enough in rainy or foggy weather. Thinking that sensors and sonar could be useful in these situations, I wanted to see a production floor for ship sensors and fish finders.

During my internship, I mainly did programming. It has been a long time since I programmed from scratch, yet since I started the internship, I have been modifying programs I created as an undergraduate. This experience allowed me to relearn and reaffirm my knowledge of Python and programming.

My internship helped me realize my dream of being a working doctor

Many people at the company were applying machine learning, so my internship was quite stimulating. I wanted to work there because it felt like an ideal place for research, and I felt comfortable.
I also hoped, if possible, to become a "working doctor" --someone who studies in a doctoral program at a graduate school while working a job. I wanted such an arrangement because I thought it would be best for me to not only delve into my research, but also to contribute to society while gaining working experience during the three-year program.

I consulted the company I interned with, and they informed me that there is a way for employees to enter doctoral programs and continue their studies after joining the company.
This internship gave me a much clearer picture of my future career path, and I intend to do everything it takes to make it a reality.