With advancement in robotic technologies, in the field of medical surgery, artificial intelligence (AI) and machine learning (ML) plays a pivotal role in medical imaging, prognosis/diagnosis, treatment assistant, and automation of repetitive subtasks of surgery. However, specifically in dental surgery, reinforcement learning (RL) has not been actively applied while it has a lot in common with robot machining where RL is widely used for fully-automated trajectory generation. Therefore, in this article, by developing an RL-based tool trajectory planner, we would like to enable robot arm to automatically perform a few subtasks in dental surgery such as tooth preparation for crown through removal of cavity. We will use CopelliaSim for simulation environments, Python/Pytorch for RL, and MATLAB for robotics. The expected result of this project would be a RL-based, collision-free, fully-automated dental tool trajectory generation algorithm.

Introduction

With the advent of an aging society and the advancement in robotic technologies, dentistry industry becomes one of the biggest markets in the field of robotics. Since various surgeries in dentistry involve high-precision and repetitive tasks, robot technologies have significant potential and ability to assist dentists or perform certain types of tasks over human practitioners. Many researches have been conducted applying the robot technologies to dentistry such as maxillofacial surgery [1-3], tooth preparation [4], and robot assistant [5]. <!–2. Also AI, but is limited in the field of diagnosis estimation assesment even though rl agent has been started in the other medical field for control(related work in dental robotics & surgical robots)

  1. among many procedure, prep tooth for crown has lots in common with robot machining and many literatures exist (related work in robot machining)–> On top of that, along with the robotics technologies, machine learning (ML) and artificial intelligence (AI) has been used in dentistry as well. The ML and AI is widely exploited in dental imaging and diagnosis: clinical diagnosis [6], dental radiography [8], and oral cancer prediction [9]. However, compared to other fields of surgical robots [10-14], reinforcement learning (RL) has not been actively applied in dentistry. For the most commonly used commercial surgical robot, da Vinci System, an open-source environment is provided for users to apply reinforcement learning algorithms for controlling the surgical arms [10-11]. While subtasks of surgery in dentistry such as implantology and tooth preparation for crown has a lot in common with the field of robot machining where RL also has been actively studied and applied, [15] to the best of my knowledge, there are not many researches being conducted in RL-based dental robot control.

dentronics

*Fig 1. Vision of possible robots and artificial intelligence service network to support future dentistry [16].

In this regard, in this article, we would like to take advantage of RL to automate subtasks in dental surgery. Specifically, we will propose an rl-based trajectory planner for a robot arm to accomplish automated tooth preparation for crown. The expected contribution of our project is summarized as follow:

  • Reinforcement Learning environment for dental surgical robot
  • 3D CAD model based fully-automatic trajectory generation
  • Guarantee of collision-free trajectory with appropriately defined reward function and collision-check in simulation The structure of the rest of the article is as follow: 1) background for tooth preparation for crown, kinematics of robotics, and reinforcement learning, 2) methodology we use for this project, 3) result of the simulation of rl-based trajectory planner, and 4) conclusion.

Background

Tooth Preparation for Crown

explanation on tooth preparation procedure for crown in dentistry

crownprep

Fig 2. Tooth Preparation Procedure for Crown. (Image source: https://www.dndentalcare.com/crowns.html) <!– Please create a folder with the name of your team id under /assets/images/, put all your images into the folder and reference the images in your main content.

You can add an image to your survey like this: YOLO

Fig 1. YOLO: An object detection method in computer vision [1].

Please cite the image if it is taken from other people’s work.–>

Kinematics of Robotics

explanation on kinematics of robotics e.g. how tool trajectory is converted into a set of joint angles of a robot arm.

YOMI

Fig 3. Yomi: commercial implantology robot arm. (Image source: https://www.neocis.com) <!– Here is an example for creating tables, including alignment syntax.

| | column 1 | column 2 | | :— | :—-: | —: | | row1 | Text | Text | | row2 | Text | Text | –>

Reinforcement Learning

explanation on our method of reinforment learning: MDP, DQN <!–

# This is a sample code block
import torch
print (torch.__version__)

Formula

Please use latex to generate formulas, such as:

\[\tilde{\mathbf{z}}^{(t)}_i = \frac{\alpha \tilde{\mathbf{z}}^{(t-1)}_i + (1-\alpha) \mathbf{z}_i}{1-\alpha^t}\]

or you can write in-text formula \(y = wx + b\).

More Markdown Syntax

You can find more Markdown syntax at this page. –>

Methodology

Interaction Environment

We will use CoppeliaSim, Python, and MATLAB for this project.

environment

Fig 4. The relation chart of the simulation environment. [15]

Reward Function

Training Process

Results

Simulation of trajectory planner and evaluate the result by comparing it with the desired 3D CAD model.

Experimental Setup

Experimental Results

Analysis and Evaluation

Conclusion

Reference

[1] Sun X, McKenzie FD, Bawab S, Li J, Yoon Y, Huang JK. “Automated dental implantation using image-guidedrobotics: registration results.” Int J Comput Assist RadiolSurg 2011;6:627–34.

[2] Zhenggang C, Qin C, Fan S, Yu D, Wu Y, Chen X. “Pilot studyof a surgical robot system for zygomatic implantplacement.” Med Eng Phys 2019.

[3] Woo S-Y, Lee S-J, Yoo J-Y, Han J-J, Hwang S-J, Huh K-H, et al. “Autonomous bone reposition around anatomical landmarkfor robot-assisted orthognathic surgery.” J Cranio-MaxillofacSurg 2017;45:1980–8.

[4] Otani T, Raigrodski A, Mancl L, Kanuma I, Rosen J. “In vitroevaluation of accuracy and precision of automated robotictooth preparation system for porcelain laminate veneers.” JProsthetic Dent 2015:114.

[5] Yuan, Fusong, and Peijun Lyu. “A preliminary study on a tooth preparation robot.” Advances in Applied Ceramics 119.5-6 (2020): 332-337.

[6] Grischke J, Johannsmeier L, Eich L, Haddadin S. “Dentronics:first concepts and pilot study of a new application domainfor collaborative robots in dental assistance.” In: 2019 international conference on robotics and automation(ICRA). 2019.

[7] Chan Y-K, Chen Y-F, Pham T, Chang W, Hsieh M-Y. Artificialintelligence in medical applications. J Healthcare Eng2018;2018:2.

[8] Chan Y-K, Chen Y-F, Pham T, Chang W, Hsieh M-Y. Artificialintelligence in medical applications. J Healthcare Eng2018;2018:2.

[9] Wang X, Yang J, Wei C, Zhou G, Wu L, Gao Q, et al. Apersonalized computational model predicts cancer risklevel of oral potentially malignant disorders and its webapplication for promotion of non-invasive screening. J OralPathol Med 2019.

[10] Richter, Florian, Ryan K. Orosco, and Michael C. Yip. “Open-sourced reinforcement learning environments for surgical robotics.” arXiv preprint arXiv:1903.02090 (2019).

[11] Xu, Jiaqi, et al. “Surrol: An open-source reinforcement learning centered and dvrk compatible platform for surgical robot learning.” 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021.

[12] Baek, Donghoon, et al. “Path planning for automation of surgery robot based on probabilistic roadmap and reinforcement learning.” 2018 15th International Conference on Ubiquitous Robots (UR). IEEE, 2018.

[13] Varier, Vignesh Manoj, et al. “Collaborative suturing: A reinforcement learning approach to automate hand-off task in suturing for surgical robots.” 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 2020.

[14] Shin, Changyeob, et al. “Autonomous tissue manipulation via surgical robot using learning based model predictive control.” 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019.

[15] Zhong, Jie, Tao Wang, and Lianglun Cheng. “Collision-free path planning for welding manipulator via hybrid algorithm of deep reinforcement learning and inverse kinematics.” Complex & Intelligent Systems 8.3 (2022): 1899-1912.

[16] Grischke, Jasmin, et al. “Dentronics: Towards robotics and artificial intelligence in dentistry.” Dental Materials 36.6 (2020): 765-778.