（海洋産業AIプロフェッショナル育成卓越大学院プログラム）
（Development of WISE Program to foster
AI Professionals for Marine Industries）

Using neural networks to create 3D models from video

My research involves creating 3D models from videos and photographs. The goal is to input a video or multiple photos and create 3D models based on them. It is a study of the fundamentals that underpin the technology. For example, I hope to be able to create 3D models from videos that may have been taken with smartphones.

Although existing technology can also be used to create 3D models, I hope to use my own technology to create and apply 3D models.
With existing technology, any object to be modeled in 3D must be stationary. If 3D models could be created from moving objects, however, the potential for their creation would expand.

In my research, I use image analysis-based AI technology known as neural networks. Neural networks are used by generative AI that creates things like fake videos or realistic pictures that are similar to photographs. I have always been interested in images and wanted to do research on them, so I have been studying neural networks since I was an undergraduate.

Neural networks are an interesting technology that can be used in many areas beyond images. It is only recently that the accuracy has improved, and I feel that this is a developing field that will continue to gain momentum, in part because of the attention it is receiving.

Applying approaches learned in the WISE Program to research

I enrolled in the WISE Program because my academic advisor in the laboratory told me that it would give me a chance to study AI and also interact with companies. Another appealing point was the financial support available.

In the WISE Program classes, companies send representatives to give presentations which I find interesting. I like to hear about real AI in action in the real world and apply it to my own research. It has broadened my horizons and allowed me to interact with companies. I appreciate those aspects as well.

One of the most impressive things about the classes is their approach to discussion. There are classes that start with a lecture and are followed by discussion in the next class, but the point is to identify and verify the problems to be solved before jumping into a discussion rather than immediately sharing our solutions and choosing the best ones.

There are also problems in research, and they must be solved as well, so I have found this approach useful in my own research.

Wouldn't it be fun if anyone could create 3D models with a smartphone?

I am researching underlying technology, so it is relevant to anything that involves 3D models in some way, such as the metaverse. My research could make the 3D models we have today more user-friendly and accessible to everyone. Although unlikely in the immediate future, I think it would be fun if someday anyone could create 3D models using a smartphone.

I find many aspects of research interesting, for example the technology of 3D modeling itself. If the technology to create 3D models advances to the point that the metaverse is widely available, it will have a substantial impact on society, and I also find it interesting in that sense.

The difficult part is the theoretical aspect. It is all based on mathematics, which makes it difficult to understand the theory of why things are done the way they are. However, it is rewarding and fun when I write the technology into a program and it actually works. When we don't understand something, the only thing to do is to investigate thoroughly and take time to find the answer. But as even more interesting things await beyond each solution, I find it all very motivating.

In the future, I want to continue writing programs. Whether I get a job somewhere or am still doing research, I hope to do something related to programming.