Changes

Jump to navigation Jump to search
no edit summary
Line 47: Line 47:  
** Perfect quality
 
** Perfect quality
 
** Needs cropping and/or rotation to select the ingredients list
 
** Needs cropping and/or rotation to select the ingredients list
 +
 +
=== Steps for ingredients lists ===
 +
 +
==== Picture taking ====
 +
 +
* Taken with mobile app, uploaded to OFF server
 +
 +
==== Ingredients list cropping ====
 +
 +
* Done on mobile app just after picture taking
 +
** Cropping may be very inaccurate
 +
* Or done on web site at a later time, possibly by another user
 +
** Cropping slightly easier than on mobile
 +
 +
==== OCR ====
 +
 +
* Launched after cropping, done by the server which calls Google Cloud Vision
 +
* Cloud Vision returns a JSON object which is stored on the server
 +
 +
==== Ingredients list cutting ====
 +
 +
* The image sent by the OCR can also contain other text content
 +
** Things that are not ingredients
 +
** Ingredients in other languages
 +
** The word "Ingredients:"
 +
* Current solution
 +
** Hardcoded regular expressions
 +
* Other possible solutions
 +
** Language identification to remove other languages
 +
* Metrics
 +
** False negatives (words before or after the ingredients list that should have been removed)
 +
** False positives (words that were removed but are part of the ingredients and should have been kept)
 +
*** It is very important to have as few false positives as possible as it destructs data
 +
* Test and training sets
 +
** Only a few adhoc tests run during builds
 +
** Test sets needs to be created
 +
 +
==== Validation and/or correction by users ====
 +
 +
* Current solution:
 +
** Users on the app or the web site are shown the OCR result
 +
** OCR result is not applied if not validated by the user
 +
** but users tend to validate lists without changes even if there are errors, especially on mobile
 +
* Other possible solutions
 +
** Use the result of ingredient analysis to show users ingredients that were not recognized
 +
** Show spell suggestions
 +
 +
==== Spell correction ====
 +
 +
* Current solution:
 +
** Currently only done during ingredients analysis, not during ingredients extraction
 +
** Very simple (and slow) implementation of Peter Norvig algorithm
 +
* Other possible solutions
 +
** Spell checkers trained on ingredients
 +
*** Elastic search spellchecker
 +
*** Simspell
 +
* Metrics
 +
** Recall and precision
 +
* Test and training sets
 +
** Language models can be build with lists of ingredients from OFF
 +
*** e.g. including only ingredients lists from producers, or lists for which we have a very high ingredients recognition rate
 +
** Test sets need to be created
 +
*** Run spellcheckers on actual ingredients lists from OFF, review corrections

Navigation menu