Orthopaedic 3D printing and simulation

Introduction

Medical simulation techniques are going to be a part of mandatory process for human care, such as medical robot, medical 3D printing, medical artificial intelligence, medical big data, medical telemedicine, health care application. Enhancing and improving the quality of medical high-resolution images, three-dimensional (3D) visualization becomes possible and students’ understanding can be improved in clinical practice and education. 3D visualization enhanced 3D materialization helps physicians to judge in a very short time in complex medical problem. Based on 3D medical image data set and simulation engine technology, virtual reality (VR) simulations, such as arthroscopy training simulation or surgery, pre-operative planning, and interactive real time rehabilitation treatment have become feasible (1). This manuscript introduces the latest medical AR VR simulation technology and 3D printing in orthopedics.

Augmented reality (AR) and VR

A surgery raining previously took place through cadaver, but now there is a new area of training called VR (2,3). VR manifests a certain environment or situation that does not exist in real life. Thus, in this situation, human senses are confounded and the individual believes they are functioning in the real world. AR is a technique that provides 3D information to observers by combining 3D VR in real life. Previously, AR and VR were primarily used in the industrial and military fields. However, development of medical image technology such as computed tomography (CT), magnetic resonance imaging (MRI) have led to the availability of high-definition images and 3D visualization in the field of medicine as well. Therefore, considerable research has been conducted on the use of AR and VR in medicine, and this application is gaining attention. Since CT and MRI scan images of patients provide hundreds of images of each axis, such as axial, sagittal, and coronal plane, it is difficult to see images intuitively, which need subsequent analysis. When surgeons set up their pre-operative care plan, they reconstruct hundreds of CT, MRI images with known anatomical knowledge, matching their imagination. This is a task that takes a lot of time and requires a lot of concentration. Sometimes it can make errors in interpreting important parts. Using 3D imaging technology, hundreds of pictures can be visualized in a very short amount of time and in simple images. This is not only very accurate, but it also minimizes the effort of the physician. 3D images are also better in resolution than other 2D images, reducing eye fatigue and confusion of cognitive awareness. Currently under development, 3D images have a resolution of 80,000 voxels (volumetric pixels), which is 10 times the current commercialized 2D images. Figure 1 shows the procedure by which 3D images are generated from medical image.

Figure 1 The process by which images from existing medical imaging device (CT, MRI) are reconstructed into 3D presets and simulators.

Medical training and simulation

Virtual arthroscopic shoulder surgery using a Leap Motion controller

Minimal invasive surgery for orthopedics, such as arthroscopy surgery, has the advantage of being cosmetic good and quick to recover, but it is technically difficult and requires a high level of training strength. Doctors must operate the endoscope in a confined space such as the shoulder, elbow, or knee, while the endoscope’s field of vision is distorted by 30 or 60 degrees. For these reasons, it takes a long time to get used to triangulation. The training of minimal invasive surgery currently being performed is learned by assisting an experienced senior doctor during resident training course or fellowship. Or, if given the opportunity, participate in cadaver workshop. On a more basic level, plastic models are sometimes used. These training methods require a great deal of time, limited training resources and no economic costs to be ignored. It also has the limitation of not being able to create specific situations and train them repeatedly. What can solve these difficulties of minimal invasive surgery training are 3D printing with tactile feedback and AR, VR simulator. Because they are not intended for real human bodies, they can be trained repeatedly in environments that are very similar to the actual surgical situation without causing ethical problems and neurovascular damage. This method can be used to arthroscopy surgery faster than a conventional method. There are no resource limitations with the use of 3D VR, augmented reality control simulators. Anyone from a student to an orthopedic specialist can set up a desired situation without time constraints and practice indefinitely until they reach the desired stage. If it is not just visible but also tactile feedback added, it can be a better training tool and is already in use (4). There are several factors that lead to the transition from the conventional orthopedic training methods to 3D printing models and VR, AR simulators (5). Surgery is a rigorous process of spending hours in the operating room and learning as an apprentice. Farnworth et al. compared the difference between orthopaedic faculty and resident when operating an anterior cruciate ligament reconstruction torn using arthroscopy. The results showed that residents take more than 40 minutes to operate than faculty (6). In terms of the cost of resident at training institutions, USD $47,970 was spent per year, excluding all other costs and only considering the time aspects in the operating room (7). Using VR and AR arthroscopy simulator, they can acquire skillful technique in a faster time. Thus, the costs incurred at the aforementioned training institute can also be reduced. Other studies on medical training methods using AR and VR showed that the visual and learning effects of both technologies were excellent (8,9). We propose a virtual shoulder arthroscopic surgery model using a Leap Motion device (Leap Motion Inc., San Francisco, CA, USA) that can capture the motion of a hand in 3D form and enable motion control (Figures 2 and 3). VR simulators for arthroscopic surgery have two screens. The first screen shows, a 3D image of a ruptured rotator cuff requiring arthroscopy, and an instrument required for surgery, while the second screen shows the patient’s shoulder to be operated on. When the operator holds the arthroscope and the instrument with both hands, the Leap Motion device recognizes the movement of both sets of fingers, and the instrument moves in the direction of the surgeon on the second screen according to the movement of the finger and hand. On the first screen, the shoulder of the patient displaying the main arthroscopic portal is visible. When the surgeon uses the trocar to pierce the insertion port and enter the arthroscope, this can be visualized on the second screen (Figure 4). On scrolling through the first screen, the inside of the shoulder joint can be viewed. Scrolling through the second screen enables a change in the direction of the scope, so that the area to be viewed can be displayed on the screen. After the site to be operated is located, an electric shaver is placed on the lesion and operated. The second screen shows the debridement of the surrounding tissues. The purpose of this simulation is to allow untrained surgeons to practice the surgical technique. Because it deals with arthroscopic and arthroscopic instruments, the simulation can be performed like a direct operation and can be repeatedly performed (10). This allows doctors to learn surgical techniques in a stable, safe, and skilled manner. This technique can also be used as a method of rehabilitation in patients, such as amputated limb or nerve paralysis (Figure 5).

Figure 2 Leap motion device.

Figure 3 Schematic image of touch surface in the air.

Figure 4 The shoulder arthroscopic surgery simulator. This can be used in medical surgery training.

Figure 5 3D real-time interactive rehabilitation treatment simulation. This can be used in self healthcare. (A) A patient with a right-sided forearm amputation. The reconstructed image is shown through virtual reality. (B) The virtual reality shows the normal left arm and the imaginary right arm. These techniques allow both hands to interact in real time and facilitate rehabilitation.

Simulation surgery and pre-operative planning

The most important aspect of arthroplasty surgery is to restore the anatomy and biomechanics of the original joint. Choosing the best size and design prosthesis for patients is an important factor. If small size humeral head is used in shoulder arthroplasty, it can cause instability, and if large size humeral head, it can impinge with the tissues of surrounding joints such as rotator cuff. These abnormal biomechanics changes result in wear and instability on the articular surface, resulting in failure of the operation and a reduction in the life span of the artificial joint (11). Therefore, pre-operative planning and proper templating are very important. Pre-operative planning using VR is more accurate and more flexible than the traditional methods. It is possible to determine the humerus cutting level and the corresponding size, location, and direction of the implant (Figure 6). Further, in fracture surgeries, it is also possible to determine the appropriate location of the implant, and restore fractured fragments (Figure 7). Through these 3D images and VR pre-operative planning, trial and error and damage to the patient’s normal tissue can be reduced during the actual surgical procedure. And more accurate and shorter surgery times can speed patient recovery and expect good surgical results. Visualizing these comprehensive 3D images prior to surgery to obtain patient operation site information is highly useful compared to conventional CT, MRI images.

Figure 6 Pre-operative planning using virtual reality simulation. (A) Three-dimensional images of humeral prosthesis that are actually used for shoulder arthroplasty. (B,C) Accurate templating prior to surgery is possible compared to the reconstructed 3D humerus image from actual patient data.

Figure 7 The humeral fracture implant surgery simulation based on Gyro sensor interface.

Image guided surgery

It is possible to perform surgery safely and accurately if the inner area around the part of the area and the actual area of the surgery can be demonstrated, using AR. In general, compared to traditional surgery, minimally invasive surgery helps reduce patient’s suffering and improves functional recovery, and the aesthetic aspect to a greater extend. However, because the surgical area is generally small in the case of minimally invasive surgery, it is also vulnerable to damage to vital structures such as nerves and blood vessels, which are located deep in the body’s core. With AR, the patient’s data can be matched to the surgical field, thus addressing aforementioned issues. For example, an operation for hip joint or acetabular fracture is a very dangerous and complex surgery in an orthopedic field. The pelvis is also more complex in shape than long tubular bone and has individual variations. In addition, the acetabulum has important internal organs such as bladder uterus and complex neurovascular structures to lower leg drive close to the bony surface. For this reason, anatomical approaches to exposing the fracture are limited and require a lot of experience (12). If the screw or Kirschner wire to fix the fracture is inadvertently misplaced, it can be inserted into the joint, causing cartilage damage, nerve or blood vessels to paralyze or cause massive bleeding. This results in a very dangerous complication (13). As occasion demands, due to these difficulties, it may take more than 10 minutes to insert an only single K-wire (14). AR can be used to solve these issues, because the insertion point and path of the appropriate K-wire can be visualized through 3D images (Figure 8). A study using AR, reported that this reduced the operation time and the number of X-ray images required and the accumulated area dose and surgical task load during the operation (15).

Figure 8 Augmented reality guided surgery.

3D printing

3D printing is integrated into medicine and is widely used in many fields, especially in orthopedic areas. 3D images and 3D printers are inseparable and are ideal business partners. Until now, even though orthopedic prosthesis has been produced in large quantities at a rough standard, the development of 3D printers allows them to produce prosthesis that are suitable for the individual’s morphologic characteristics. Although not yet popular, these technologies could be a source of new revenue for the implant manufacturer. The production of a dummy model using 3D printing from already captured CT, MRI data is already realized. Other surgery departments are also working on 3D printers, but the majority of the field is the orthopedic field. They are 30.70%, 8.33%, 2.19% and 1.75% respectively in the order of knee, hip, shoulder, and hands (Figure 9) (16). Through 3D printing and virtual surgery simulation, it is possible to use patient-specific surgical implants and guiders. Ultimately, simplifying the process reduce the operation time and alleviates patient suffering. For example, in forearm rotational osteotomy, it is difficult to determine the bone cutting level and amount to be cut. However, using VR to decide on these factors, and 3D printers to produce appropriate guides and metal plates, can reduce the difficulties encountered during surgery (Figure 10). In patients with bone defects, the opposite image can be used to produce a mold that fits the patient perfectly (Figure 11).

Figure 9 Overview of papers per specific field.

Figure 10 Three-dimensional printing and simulation from CT.

Figure 11 3D model for cement spacer mold. It is possible to use the patient-specific 3D printing mold. These 3D printing mold for cement spacer can be used in medical surgery.

Conclusions

The development of CT, MRI has contributed to a breakthrough in the diagnosis and treatment of diseases. Now the use of 3D images reconstructed to understand these complex images at once is emerging as a new clinical research area. It will give clinicians an intuitive understanding of the information and help them plan for treatment. There are also disadvantages of increased costs and the application of some untested techniques to clinicians. But, 3D printing and simulation technique can reduce operation time and surgical complication, improved clinical outcome, and minimized unnecessary radiation exposure. Evolving high-definition medical 3D technology and real-time VR engine technology combined with the medical knowledge of medical professionals will result in strong clinical synergies.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Annals of Joint for the series “Medical 3D Application in Orthopaedics”. The article has undergone external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/aoj.2018.07.06). The series “Medical 3D Application in Orthopaedics” was commissioned by the editorial office without any funding or sponsorship. YLM served as the unpaid Guest Editor of the series and serves as an unpaid editorial board member of Annals of Joint from Aug 2017 to Jul 2019. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Rose AS, Kimbell JS, Webster CE, et al. Multi-material 3D models for temporal bone surgical simulation. Ann Otol Rhinol Laryngol 2015;124:528-36. [Crossref] [PubMed]
2. Akhtar KS, Chen A, Standfield NJ, et al. The role of simulation in developing surgical skills. Curr Rev Musculoskelet Med 2014;7:155-60. [Crossref] [PubMed]
3. Pedowitz RA, Marsh LJ. Motor skills training in orthopaedic surgery: a paradigm shift toward a simulation-based educational curriculum. J Am Acad Orthop Surg 2012;20:407-9. [Crossref] [PubMed]
4. Lim W, Moon YL VR. AR Simulation and 3D Printing for Shoulder and Elbow Practice. Journal of The Institute of Electronics and Information Engineers 2016;53:175-9. [Crossref]
5. Mabrey JD, Reinig KD, Cannon WD. Virtual reality in orthopaedics: is it a reality? Clin Orthop Relat Res 2010;468:2586-91. [Crossref] [PubMed]
6. Farnworth LR, Lemay DE, Wooldridge T, et al. A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthop J 2001;21:31-5. [PubMed]
7. Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177:28-32. [Crossref] [PubMed]
8. Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 2016;30:4174-83. [Crossref] [PubMed]
9. Kang BH, Kim JS, Kim HW. Study for Operation Teaching Machine Using 3D Virtual Reality System. Journal of Digital Contents Society 2016;17:287-93. [Crossref]
10. Bak S, Lee C, Shin H. Edutainment content production utilizing the leap motion device. Proceedings of The Korea Contents Associations, 2015, No.5.
11. Cho SW, Jharia TK, Moon YL, et al. Three-dimensional templating arthroplasty of the humeral head. Surg Radiol Anat 2013;35:685-8. [Crossref] [PubMed]
12. Shen F, Chen B, Guo Q, et al. Augmented reality patient-specific reconstruction plate design for pelvic and acetabular fracture surgery. Int J Comput Assist Radiol Surg 2013;8:169-79. [Crossref] [PubMed]
13. Guy P, Al-Otaibi M, Harvey EJ, et al. The 'safe zone' for extra-articular screw placement during intra-pelvic acetabular surgery. J Orthop Trauma 2010;24:279-83. [Crossref] [PubMed]
14. Starr AJ, Reinert CM, Jones AL. Percutaneous fixation of the columns of the acetabulum: a new technique. J Orthop Trauma 1998;12:51-8. [Crossref] [PubMed]
15. Fotouhi J, Fuerst B, Lee SC, et al. editors. Interventional 3d augmented reality for orthopedic and trauma surgery. 16th annual meeting of the international society for computer assisted orthopedic surgery (CAOS); 2016.
16. Tack P, Victor J, Gemmel P, et al. 3D-printing techniques in a medical setting: a systematic literature review. Biomed Eng Online 2016;15:115. [Crossref] [PubMed]