Not even your monitor is safe from OCR

by Ilya Evdokimov | Dec 18, 2009 | OCR

OCR technology is not widely used to extract text's from user's active screens

I’ve talked about various uses of OCR that are non-conventional: anti-virus, CAPTCHA ( thought this does not work ), and now it’s time for a new one. Screen scraping. OCR technology is not widely used to extract text’s from user’s active screens, and the predominate use has been of the sneaky kind. However I suspect that screen scraping will become more popular for data validation, user identification similar to CAPTCHA, user automation, and even extreme content management. I myself have used screen scraping to convert an on-line address book from one email account to an importable format for another email account where the initial account did not have the option for export!

Essentially what screen scraping does is take a screenshot of the active window, or entire current session and reads the text in it with OCR. Although screenshot resolution is very-low, 96 dpi, the text contained in it is what is called “pixel perfect”, and does not accompany the distortions, dithering, and splotches that can appear in scans. This makes reading the text itself relatively easy, the hard part is getting to the text.

Look at your screen now. It’s probably filled with various graphics, and text everywhere. For screen scraping you cannot consider any traditional document analysis to discern where text is and what text is valuable, this has to happen after the fact. The most successful screen scraping is that which is focused on one particular portion of the screen. The next biggest challenge in screen scraping that is continuous, is the rate a screen changes. For example if you are typing a document, as I am now, you may scroll up and down very rapidly at times. Deciding when and where to capture data in an active screen can be tricky.

It may be hard for you to image why screen scraping is useful. Especially you techies who realize that the text on the screen is in digital format already somewhere. Where screen scraping is extremely valuable is when your application has to obtain data from another application. Developing connectors between applications can be very time consuming, and often a major waste of time. You have to learn the other products API, if they come out with a new version you now have to support it. But with screen scraping you can write one way to get data off the screen of ANY active application window, search for the relevant content, and presto, you never have to do it again. In the areas of enterprise content management, and conversion from a legacy system to a new, screen scraping using OCR can be the most amazing tool.

Chris Riley – Sr. Solutions Architect