AlgoSurg: AR-Based Bone Tracking

Project Name: AR-Based Bone Tracking

Partner: AlgoSurg

Sector: Healthcare Technology

Category: Augmented Reality (AR), Medical Imaging

Description: This project aims to develop a prototype for AR glasses (Microsoft HoloLens 2/Vuzix M400) using Unity-3D, for augmented reality-based bone tracking. Users will wear AR glasses, allowing them to view a physical bone model with an attached optical marker. Through the AR glasses, users must see a virtual 3D model of the same bone, morphed onto the physical bone. As the physical bone moves (with the marker attached), the virtual 3D bone model should move in sync, providing a real-time augmented view. This technology has potential applications in medical education, surgery planning, and other medical fields.

The project is in the planning phase, with many of the necessary algorithms in place. The team is prepared with the required methods and algorithms, and development is scheduled to start in October, with the goal of delivering a functional prototype by mid-December.

The project will be divided into two phases, each expected to be completed in roughly a month. The project's progress and stipend will be per completion of phase. Candidates are also permitted to publish papers/videos related to the work with them being authors, mentioning AlgoSurg Inc. as a partner.

Roles: 2 Development Positions

Stipend: Paid project (join community for more details)

Project Duration: 2 months

Tasks/Deliverables: The primary focus of this project is to integrate and implement algorithms to develop an augmented reality environment in Unity 3D. The key tasks include:

1. Using dual infrared cameras to detect and track optical markers attached to a physical bone model (Related paper - https://www.degruyter.com/document/doi/10.1515/cdbme-2020-0027/html)
2. Alignment of a pre-created 3D virtual model of the bone with the collected point cloud, hence morphing the model onto the physical bone in real time
3. Development of a fully functional Unity 3D AR app, exported as Android or Windows
    

To enroll for the project, you must complete the assignment below. Follow the submission instructions below, and submit a drive link for your simulation video and code.

Assignment Deadline: 11:59 PM, 14th October, 2023

Time to complete: 3 hours

Problem Statement

This assignment will involve getting familiar with basic 3D simulation in Unity 3D, along with exploring the fundamental concepts the project will later use.

1. Unity3D - Setup a virtual 3D world in Unity 3D environment
2. Rendering - Upload and Render a 3D object in the 3D world (link to 3D object in .stl format is provided below) with a webcam feed as the background
3. Simple UI - Buttons in the 3D world to switch the 3D bone display on/off and/or to view three orthographic views
4. Brownie points (not compulsory) - Interaction with the 3D bone using hand gestures

Link to .stl file: https://drive.google.com/file/d/1r3HQBNC03GkEiajZ3lcigREUztBkYvXK/view?usp=drive_link

Submission Guidelines: You must submit a GitHub repository link to your code (ensuring it's public), as well as a video of your simulation. The submission link is- https://forms.gle/zQRUkBw4Grwfh2YN6

For any general queries, join the ProSpace WhatsApp group- https://chat.whatsapp.com/E09qtrcuShp1uf2w82LCsa

For assignment queries, contact:

Email: 210050001@iitb.ac.in, 210040139@iitb.ac.in

Phone: 9987361968, 9324865787