Google's SMILY
Source: Google

The spotting and diagnosis of cancer can be quite challenging, even for medical professionals. That’s why Google has just introduced Google’s SMILY, which is like a reverse image search for cancerous cells. This is a good move on the part of Google as the Company is facing scandals these days.

Part of the diagnostic process includes examining tissue samples under a microscope and looking for different signals that indicate one or another form of cancer. This can be a long and lengthy procedure because every type of cancer is unique. Therefore, a person looking for cancerous cells shouldn’t only look at the patient’s cells. He should also compare them to known cancerous cells so that he knows exactly what he is looking for.

The best method to match similar images is to use Machine learning objects. In Google’s reverse image search, you put in the picture, and the computer finds a visually identical image. This technique has also been used in medicine, where the computer highlights the areas of X-ray that has different patterns, which it recognizes.

It might sound good, but the complexity of cancer makes it much more complicated than simple image recognition between the two samples. For example, one cancer may be from the pancreas, the other may be from the liver, and the two will be completely different. Still, their images could be the same. Even a doctor’s “intuition” wouldn’t help in such a scenario.

Google was already aware of all these limitations beforehand. Therefore, after constant thought and hard work, Google introduced Google’s SMILY, which stands for Similar Medical Images like Yours. It is a reverse image search specifically designed for cancer diagnosis.

The operation of Google’s SMILY is quite simple. A user first puts a new sample into the system. The high-resolution image of the dyed section of tissue is then laid out. This is the standard method used for a long time. Once the image is in the tool, the doctor scrutinizes it, by zooming in and out. Once he finds something of interest, he can draw a box around it. After that, Google’s SMILY will compare this image to the entire models of the Cancer Genome Atlas, which is a massive database of cancer samples.

Similar images pop up, and the researcher can easily use them. This is quite useful. However, the researchers who built SMILY were aware that doctors need more details to find proof of cancer. Therefore, visual image similarity isn’t the only thing that matters. Many other features also interest the doctors, like the specific type of cells and their proportions.

A researcher states, “Users needed refined search results on a case-to-case basis to find out exactly what there were looking for. The solution for refinement was in “iterative diagnosis.” This process involves generating hypotheses and then finding data to test the hypotheses and then making alternative hypotheses. It became clear to us that to meet the user needs, we needed a different approach.”

To improve the user experience, the team at Google, added three extra tools into Google’s SMILY. Firstly, the user can select a single image within the area, and the system will only focus on that particular image and ignore all the other distracting features. Secondly, the user can select one image that intrigues him, and the system will give more closely related images. This helps the user go down a rabbit hole and test different things at the same time. This process is similar to the iterative diagnosis.

Thirdly, the system understands when the search result contains certain features, like fused glands and tumor precursors. Therefore, if any doctor isn’t sure that a specific feature was present, he can choose it to confirm. He can go through all the options and come close to the diagnosis.

Some doctors tested the tool and appreciated it greatly. They said that it is a better alternative to the currently used traditional interface method. These results seem promising, but it is only the beginning. You can read the papers describing SMILY here.

Many tech companies are making gadgets and tools to revolutionize the medical field. Let us hope Google’s SMILY turns out to be a success.