Image Stitching System With Scanning Microscopy for Histopathological Applications

Abstract

Histopathological analysis of biopsy or surgical specimen is a common clinical practice for diagnostic purposes. Essentially, the process involves slicing the biopsy or surgical sample into very thin slices, placing them on glass slides and viewing them under microscopes. Predominantly, the placement, positioning, and view control is done manually by the pathologists in most of the clinics and hospitals because of which the diagnosis remains heavily dependents upon the experience and performance of the pathologist. Moreover, the slide scanning relies predominantly on the slide placement accuracy. A misaligned slide will create misaligned images which can either miss out information or have blank artifacts due to image frame placement methodology. In this paper, a simple 'add-on' system has been presented that can be used to scan single slide with moderate speed and produces the image on a Virtual reality headset to provide the submerged feeling. Most importantly, it utilizes advanced image stitching algorithms to align the frames from the captured video stream of the slide to produce a very accurate image with a very large size. The stitching is done using the standard feature-based algorithms which have been modified in this work by incorporating affine blending maps to combine the features into final image. It has been found that the image stitching algorithm provides the stitched image with less than 2% error for the given test images. Research Methodology: The proposed approach utilizes a standard clinical microscope with 10x, 40x, and 100x magnifications. The microscope has x-y movable stage which is moved by the pathologist using two vertical knobs on the right side. Another knob on the left side is used for fine focusing. The two knobs on the right are modified in this work by placement of two stepper motors on the knobs and making them fixed to a frame structure so that the knobs can be moved easily and the whole support structure moves with the stage. Since the stage is controlled by the system in two directions only, the focus must be adjusted initially by the user manually using the left side knob until a clear vision of the tissue slide is reached. The two stepper motors are driven by standard driver units and are controlled through an Arduino board. Slide image capture is done by a digital camera fitted with an Amscope adapter so that it could be attached to microscope's eyepiece. The camera is interfaced to the PC using USB link and provides Continuous Video when the scanning starts. Although, the same camera can be used for taking still images rather than video, the mechanical delays in stopping the scanner, shutter operation, and image storage rendered this method less practical than using videos. The high level diagram for the system is shown in Fig. 1. After obtaining the video frames, the designed image stitching (IS) algorithm is applied on them. The process of IS involves a series of steps which were applied on the consecutive images, such that one of the image is taken as a reference image (RI) whereas the other one is termed as the current image (CI). The resultant stitched image will be RI for the next consecutive image and then the whole stitching process is applied. The process remain continue for each set until a final stitched image has been obtained from them. Simulation Results In order to perform autonomous clinical analysis on the slide images, it is highly desirable to visualize all slide together (as WSI) especially when the shape and size of the particle is of great importance. The designed algorithm has been applied to Various Images (Frames) acquired from the video of the slide scan and the resultant overall WSI is shown in Fig. 2. Note that the black region shown in the Fig. 2 is produced due to the over-done y-displacement. As can be seen, the stitching is done quite smoothly in spite of the motorized shifts in the scan. In order to show the capability of alignment, the y- axis displacement was over-done by slipping more steps through the worm gear. The algorithm was able to stitch the x segments separately and the y-axis was stitched subsequently. In order to quantify the performance of the presented algorithm, another test was performed using one of the colorectal cancer image from Glas Database. One full image (401x521) was divide into 36 sub-images with 25% overlap in rows and columns between the adjacent sub images. After stitching, the resultant and the original images were compared to highlight the errors as shown in Fig. 3. The grayscale values of the error image were summed together to obtain the total error gray values (TE). This number was divided by the total number of pixels in the image to calculate the percentage error (PE). It was found that the PE for the shown image was 1.16% and was found to be less than 2% for most of the images. Figure 8 shows the test images and results from various steps in the algorithm.