Multilingual OCR Activities and Interest at ScanSoft<BR>
     ----------------------------------------------------<BR>
<BR>
                   Yang HE and Ben WITTNER<BR>
<BR>
	ScanSoft, Inc., 9 Centenniel Dr, Peabody, MA 01960  USA<BR>
	          {benw,yangh}@scansoft.com<BR>
<BR>
<BR>
   At ScanSoft, Inc., formerly Xerox Imaging Systems, we develop and <BR>
market a commercial OCR software called TextBridge. It currently<BR>
supports 56 different languages. They are organized in 6 groups - <BR>
American/European, Baltic, Central European, Cyrillic, Greek, and <BR>
Turkish. The current system can load one language group at a time <BR>
and recognize all languages in that group on the same page. <BR>
<BR>
  We have collected large ground truth data sets for our own development<BR>
purpose. The ground truth files are in either Latin 1 Codepage or <BR>
Unicode format. It has some mark-ups for certain format information.<BR>
But for accuracy evaluation only, the mark-ups are filtered out. <BR>
<BR>
   We have developed two different string matchers. One is 8-bit<BR>
codepage based. It tries to align OCR output lines with ground truth<BR>
lines and count both character and word errors. The other is Unicode<BR>
based that aligns the whole page of OCR output to the ground truth as<BR>
a single string. It can count character errors only at this point, <BR>
and we have not tested it with many languages yet. For our daily <BR>
development purpose we now convert the Unicode ground truth to its<BR>
corresponding codepage and use the first matcher.<BR>
<BR>
   We are interested in all the topics listed in the workshop's<BR>
"Technical Focus". In addition, through the process of developing<BR>
and using our data/tools, we would like to learn other people's<BR>
opinion and work on how to handle the following specific issues:<BR>
<BR>
* Issues associated with a Unicode based matcher that can check <BR>
  character and word errors across all languages.<BR>
* Punctuation "normalization", i.e., how to treat punctuation<BR>
  of the same functionality but with different shape and/or code<BR>
  as the same in different languages.<BR>
* Same words with and without diacritics. For example, "resume".<BR>
* Unusual text line orientations and running directions.<BR>
* Non-unique conversion between Big5 and GB code in Traditional<BR>
  and Simplified Chinese. Can conversion be unique via Unicode?<BR>
  Non-unique Unicode/Codepage conversions in other languages?<BR>
* Furigana (the lowered and smaller Japanese characters for<BR>
  pronunciation purpose).<BR>
<BR>
<BR>