Surgical devices segmentation and tracking

The overarching goal of this thesis project was to develop an approach for the segmentation and the tracking of the surgical devices used during cardiac angioplasty, and in particular, the thin guidewires used to position the other surgical tools, such as stents. During the surgery, these tools are pushed inside the cardiac arteries under real-time X-ray guidance (a.k.a fluoroscopic imaging). These images, taken at very low doses to avoid damaging patient tissues, usually contain a significant amount of noise.

I addressed the segmentation and the tracking tasks separately. For the segmentation, which is particularly challenging because of the presence of clutter, the low signal to noise ratio and the presence of multiple structures in the images, I finally combined several layers of signal processing, clustering, and optimization methods known for their robustness. For the tracking, I developed a hybrid approach achieving better performances by combining geometric and iconic clues.

PhD Thesis
Curvilinear Structures Segmentation and Tracking in Interventional Imaging 2013 manuscript slides

Guidewires Segmentation

After several experiments, I decided to segment the guidewires in four steps: Two approaches were investigated for the guidewire detection. First, I measured the performance of a boosted classifier combining different image features, including image gradients and image Hessian at different resolutions. Then, I adopted a line detector based on steerable filters, which was improved by tensorial voting. This latter solution was preferred in my latter publications because of its robustness concerning image contrast, and for its computational efficiency.







[A]






[B]






[C]






[D]
Guidewires segmentation. [A] fluoroscopic image [B] guidewire detector output [C] extracted segments [D] final segmentation.

associated publications


Guidewires Tracking

For the tracking, I developed a B-spline snake. The guidewires were represented using a cubic B-spline, and the control points of this B-spline were displaced from frame to frame to follow the deformations of the guidewires. These displacements are particularly difficult to track because of the large heart motion, which is an order of magnitude larger than the guidewires.

The optimal snake tracker displacements were determined by solving a Markov Random Field. The most important contribution of this work resides in the combination of geometric potentials constraining the global shape of the guidewire with iconic potentials integrated along the B-spline snake tracker. The iconic potentials were significantly improved by exploiting guidewire orientation estimation provided by the steerable filters developed for their segmentation.


complete model
$E_{MRF}=\mu_{iconic} \left(\sum_{k\in{\{i,j\}}}{V_{k}^{iconic}(l_k)}+\sum_{k,r\in{\{i,j\}}}{V_{k,r}^{iconic}(l_k,l_r)}\right)+$
$ ~~~~~~~~~~\mu_{landmarks} \left(\sum_{k\in{\{i,j\}}}{V_{k}^{landmarks}(l_k)}+\sum_{k,r\in{\{i,j\}}}{V_{k,r}^{landmarks}(l_k,l_r)}\right)+$
$ ~~~~~~~~~~\mu_{consistency} \left(\sum_{k\in{\{i,j\}}}{V_{k}^{consistency}(l_k)}+\sum_{k,r\in{\{i,j\}}}{V_{k,r}^{consistency}(l_k,l_r)}\right)+$
$ ~~~~~~~~~~\mu_{coupling} \left(\sum_{k\in{\{i,j\}}}{V_{k}^{coupling}(l_k)}+\sum_{k,r\in{\{i,j\}}}{V_{k,r}^{coupling}(l_k,l_r)}\right)+$
$ ~~~~~~~~~~\mu_{rigidity} \left(\sum_{k\in{\{i,j\}}}{V_{k}^{rigidity}(l_k)}+\sum_{k,r\in{\{i,j\}}}{V_{k,r}^{rigidity}(l_k,l_r)}\right)$
The MRF-based snake tracker combines iconic costs diminishing when the tracker covers pixels likely to belong to the guidewire, matching costs for landmarks extracted along the guidewire, dynamic consistency to reduce tracker drift, a coupling between iconic and landmark/geometric terms, and a tracker rigidity.

iconic term     
landmark matching
dynamic consistency

penalizes local deforma- tions w.r.t. model $T$ updated by exponential forgetting:

$T(s) \leftarrow \frac{memory-1}{memory}T(s)+\frac{1}{memory}C(s)$
coupling     
rigidity     

associated publication