Blog, Books & News Related

The Semantics of Images

More and more frequently, clients (or potential clients) ask me if our semantic search engine is also able to search for images. The first few times, I was surprised and did not understand exactly why they were asking this. Now however, my answer is immediate, and it is: no, it is not able to because images contain little text, therefore making it difficult to apply semantic technology 🙂


With time, I have also come to understand that the question is almost spontaneous because people understand the difficulty in semantic text analysis. So, they think that if we have been able to solve this complex problem, then maybe we are also able to take on the problem of image recognition. Unfortunately, the only thing these two problems have in common is their extreme complexity and nothing else.


I find this subject quite intriguing and today, when I read that Google’s Labs had released an instrument to search for images similar to an image selected by the user, I couldn’t resist and immediately tested it out.

The results were interesting, but they demonstrate that ” the  almighty Google” is quite far from having solved this problem. They have employed several clever algorithms (for example, if you search for “French fries” and you select one of the proposed images, a good result is obtained), but it is clear that they are mostly based on predominant colors and on some macro elements (even the text on the page which contains the image seems to be of great influence), without any understanding of what the image actually contains. To prove it, try these examples).

The thing works pretty good with some images, with others, not at all (and often, the results are quite humorous, as in this case and in this one). Seeing as though it’s  free (at least for now :-), if you want to search for images which are similar to one that interests you… then, you can try it out and hope to be lucky.


Share On