Difference between revisions of "GSOC/2023 ideas list"

From Open Food Facts wiki
Jump to navigation Jump to search
m
 
(3 intermediate revisions by 2 users not shown)
Line 2: Line 2:
  
 
= Server-side =
 
= Server-side =
 
 
<span id="a-new-api-simplify-and-unify-the-open-food-facts-api-used-by-200-apps"></span>
 
<span id="a-new-api-simplify-and-unify-the-open-food-facts-api-used-by-200-apps"></span>
 
=== A New API: Simplify and unify the Open Food Facts API used by 200+ apps ===
 
=== A New API: Simplify and unify the Open Food Facts API used by 200+ apps ===
 
 
<span id="description"></span>
 
<span id="description"></span>
 
==== Description ====
 
==== Description ====
  
Open Food Facts provides a free and open API to read, search and write products data and photos. This API powers more than 200 apps developed by 3rd parties, and is also used by many tools like our AI platform Robotoff and our power contribution tool Hunger Games. Our API is unfortunately very difficult to understand and use as it evolved organically in the last 10 years. It's not 1 API in fact, it's several APIs to do different things, and we have multiple versions of those APIs, each with a different state of features and documentation. This project aims to bring &quot;A New API&quot; to our many reusers. An API that is simple, unified, easy to understand and use, well documented (with OpenAPI) and well tested.
+
Open Food Facts provides a free and open API to read, search and write products data and photos. This API powers more than 200 apps developed by 3rd parties, and is also used by many tools like our AI platform Robotoff and our power contribution tool Hunger Games. Our API is unfortunately very difficult to understand and use as it evolved organically in the last 10 years. It's not 1 API in fact, it's several APIs to do different things, and we have multiple versions of those APIs, each with a different state of features and documentation. This project aims to bring &quot;A New API&quot; to our many reusers. An API that is simple, unified, easy to understand and use, well documented (with OpenAPI) and well tested.<span id="expected-outcomes"></span>
 
 
<span id="expected-outcomes"></span>
 
 
==== Expected outcomes ====
 
==== Expected outcomes ====
  
Line 18: Line 14:
 
This project is about revisiting one by one all existing APIs to:
 
This project is about revisiting one by one all existing APIs to:
 
# Decide which feature should be added, changed (or maybe even removed)
 
# Decide which feature should be added, changed (or maybe even removed)
# Define the API for each feature (e.g. method, path, parameters, query and response headers and bodies etc.) and document it using OpenAPI.<br />
+
# Define the API for each feature (e.g. method, path, parameters, query and response headers and bodies etc.) and document it using OpenAPI. (This step is critical to ensure that the API will be both useful and used, and will need to be done in coordination with existing users of the API.)
This step is critical to ensure that the API will be both useful and used, and will need to be done in coordination with existing users of the API.
 
 
# Implement the API for the feature and add integration tests for it
 
# Implement the API for the feature and add integration tests for it
  
 
APIs that needs to be converted to API v3:
 
APIs that needs to be converted to API v3:
  
<ul>
 
 
* Product image upload, selection and crop
 
* Product image upload, selection and crop
 
* Full support for product write (should be done incrementally)
 
* Full support for product write (should be done incrementally)
Line 30: Line 24:
 
* Taxonomy read
 
* Taxonomy read
 
* User creation and profile edition
 
* User creation and profile edition
* Authentification
+
* Authentification<span id="project-information"></span>
 
 
<span id="project-information"></span>
 
 
==== Project information ====
 
==== Project information ====
 
+
* repository: [https://github.com/openfoodfacts/openfoodfacts-server <u>https://github.com/openfoodfacts/openfoodfacts-server</u>]
<ul>
+
* Slack channels: #product-opener
<li><blockquote><p>repository: [https://github.com/openfoodfacts/openfoodfacts-server <u>https://github.com/openfoodfacts/openfoodfacts-server</u>]</p></blockquote></li>
+
* Potential mentors: Stéphane, Alex
<li><blockquote><p>Slack channels: #product-opener</p></blockquote></li>
+
* Project duration: 350 hours
<li><blockquote><p>Potential mentors: Stéphane, Alex</p></blockquote></li>
+
* Skills required: Perl
<li><blockquote><p>Project duration: 350 hours</p></blockquote></li>
+
* Difficulty rating: Medium
<li><blockquote><p>Skills required: Perl</p></blockquote></li>
 
<li><blockquote><p>Difficulty rating: Medium</p></blockquote></li></ul>
 
  
 
<span id="bring-taxonomy-editor-to-a-fully-usable-tool"></span>
 
<span id="bring-taxonomy-editor-to-a-fully-usable-tool"></span>
 
=== Bring Taxonomy editor to a fully usable tool ===
 
=== Bring Taxonomy editor to a fully usable tool ===
 
 
<span id="description-1"></span>
 
<span id="description-1"></span>
 
==== Description ====
 
==== Description ====
Line 55: Line 44:
 
While effective for the application, this format is quite cumbersome to edit for contributors.
 
While effective for the application, this format is quite cumbersome to edit for contributors.
  
Taxonomy editor was started during last Google Summer of code. Thanks to the work of Aadarsh, we got a first demonstration of the product, and it’s been in continuous development since then. But a boost would be needed to make it a really usable project.
+
Taxonomy editor was started during last Google Summer of code. Thanks to the work of Aadarsh, we got a first demonstration of the product, and it’s been in continuous development since then. But a boost would be needed to make it a really usable project.<span id="expected-outcomes-1"></span>
 
 
<span id="expected-outcomes-1"></span>
 
 
==== Expected outcomes ====
 
==== Expected outcomes ====
  
Line 64: Line 51:
 
From there, according to the pace of the projects many more features could be added which would be of great benefit: embarked assistance integrating tools (eg. searching / linking to wikidata, or wiktionary), integration of the products database (eg. seeing which entries need to be added or translated first), navigation in the taxonomy.
 
From there, according to the pace of the projects many more features could be added which would be of great benefit: embarked assistance integrating tools (eg. searching / linking to wikidata, or wiktionary), integration of the products database (eg. seeing which entries need to be added or translated first), navigation in the taxonomy.
  
An important aspect: as this tool is meant to live a long time quality and maintainability of the project is important.
+
An important aspect: as this tool is meant to live a long time quality and maintainability of the project is important.<span id="project-information-1"></span>
 
 
<span id="project-information-1"></span>
 
 
==== Project information ====
 
==== Project information ====
 
+
* repository: [https://github.com/openfoodfacts/taxonomy-editor/ <u>https://github.com/openfoodfacts/taxonomy-editor/</u>]
<ul>
+
* Slack channels: #taxonomy-editor
<li><blockquote><p>repository: [https://github.com/openfoodfacts/taxonomy-editor/ <u>https://github.com/openfoodfacts/taxonomy-editor/</u>]</p></blockquote></li>
+
* Potential mentors: Daniel, Alex
<li><blockquote><p>Slack channels: #taxonomy-editor</p></blockquote></li>
+
* Project duration: 350 hours
<li><blockquote><p>Potential mentors: Daniel, Alex</p></blockquote></li>
+
* Skills required: Reactjs / Python / Neo4j
<li><blockquote><p>Project duration: 350 hours</p></blockquote></li>
+
* Difficulty rating: Medium<span id="improve-our-producer-platform-to-the-next-level"></span>
<li><blockquote><p>Skills required: Reactjs / Python / Neo4j</p></blockquote></li>
 
<li><blockquote><p>Difficulty rating: Medium</p></blockquote></li></ul>
 
 
 
<span id="section"></span>
 
===  ===
 
 
 
<span id="section-1"></span>
 
=== <br />
 
===
 
 
 
<span id="improve-our-producer-platform-to-the-next-level"></span>
 
 
=== Improve our producer platform to the next level ===
 
=== Improve our producer platform to the next level ===
  
Line 94: Line 68:
 
Thanks to the support of the national French public health agency Santé Publique France, the platform is already alive and working. Though it is a bit rough for our users that are not tech savvy, also there are a lot of features which could help adoption and lessen the support needed to operate it.
 
Thanks to the support of the national French public health agency Santé Publique France, the platform is already alive and working. Though it is a bit rough for our users that are not tech savvy, also there are a lot of features which could help adoption and lessen the support needed to operate it.
  
Technically the producer platform uses the same code base as the main website.
+
Technically the producer platform uses the same code base as the main website.<span id="expected-outcomes-2"></span>
 
 
<span id="expected-outcomes-2"></span>
 
 
==== Expected outcomes ====
 
==== Expected outcomes ====
  
 
The project will bring many new features to the producer platform. Here are some of the current priorities: add more inline help, make UX more convenient, support advanced updates patterns on fields, add a workflow to product edition, connect to our internal CRM (allow automating many tasks), collect more statistics on usage, support more fields and format for data import, more robust import/publishing workflows and add more tests.
 
The project will bring many new features to the producer platform. Here are some of the current priorities: add more inline help, make UX more convenient, support advanced updates patterns on fields, add a workflow to product edition, connect to our internal CRM (allow automating many tasks), collect more statistics on usage, support more fields and format for data import, more robust import/publishing workflows and add more tests.
  
Extended goals include adding rules to help producers make products with better Eco-score, advanced features to make editing products easier (like picture upload).
+
Extended goals include adding rules to help producers make products with better Eco-score, advanced features to make editing products easier (like picture upload).<span id="project-information-2"></span>
 
 
<span id="project-information-2"></span>
 
 
==== Project information ====
 
==== Project information ====
 +
* presentation of the platform: https://world.pro.openfoodfacts.org/
 +
* repository: https://github.com/openfoodfacts/openfoodfacts-server/
 +
* Slack channels: #productopener
 +
* Potential mentors: Stephane, Alex
 +
* Project duration: 350 hours
 +
* Skills required: Perl
 +
* Difficulty rating: Medium
  
<ul>
 
<li><blockquote><p>presentation of the platform: https://world.pro.openfoodfacts.org/</p></blockquote></li>
 
<li><blockquote><p>repository: https://github.com/openfoodfacts/openfoodfacts-server/</p></blockquote></li>
 
<li><blockquote><p>Slack channels: #productopener</p></blockquote></li>
 
<li><blockquote><p>Potential mentors: Stephane, Alex</p></blockquote></li>
 
<li><blockquote><p>Project duration: 350 hours</p></blockquote></li>
 
<li><blockquote><p>Skills required: Perl</p></blockquote></li>
 
<li><blockquote><p>Difficulty rating: Medium</p></blockquote></li></ul>
 
 
<span id="section-2"></span>
 
===  ===
 
 
<span id="section-3"></span>
 
=== <br />
 
===
 
 
<span id="data-quality-moderation-hub"></span>
 
 
=== Data Quality &amp; Moderation Hub ===
 
=== Data Quality &amp; Moderation Hub ===
 
 
<span id="description-3"></span>
 
<span id="description-3"></span>
 
==== Description ====
 
==== Description ====
Line 130: Line 89:
 
Open Food Facts infrastructure is currently focused on reflecting the current state of the database, which is its primary goal. However, it would be beneficial to introduce a queue/backlog system to coordinate the efforts of users in data cleaning and moderation.
 
Open Food Facts infrastructure is currently focused on reflecting the current state of the database, which is its primary goal. However, it would be beneficial to introduce a queue/backlog system to coordinate the efforts of users in data cleaning and moderation.
  
The system currently provides some level of coordination through the facets, indicating missing nutritional data and providing a list of products that need completion. However, this system lacks flexibility.
+
The system currently provides some level of coordination through the facets, indicating missing nutritional data and providing a list of products that need completion. However, this system lacks flexibility.<span id="expected-outcomes-3"></span>
 
 
<span id="expected-outcomes-3"></span>
 
 
==== Expected outcomes ====
 
==== Expected outcomes ====
  
Line 140: Line 97:
  
 
<ol style="list-style-type: decimal;">
 
<ol style="list-style-type: decimal;">
<li><blockquote><p>select nutritional table image</p></blockquote></li>
+
* select nutritional table image
<li><blockquote><p>crop nutritional table image</p></blockquote></li>
+
* crop nutritional table image
<li><blockquote><p>fill nutriments</p></blockquote></li></ol>
+
* fill nutriments</ol>
  
 
The MVP would include a back-end with an API for moderation, providing a table with information such as barcode, image_id, type_of_problem, author (including Robotoff), and moderation status (has_been_moderated). The front-end would allow users to delete pictures and mark elements as moderated.
 
The MVP would include a back-end with an API for moderation, providing a table with information such as barcode, image_id, type_of_problem, author (including Robotoff), and moderation status (has_been_moderated). The front-end would allow users to delete pictures and mark elements as moderated.
  
 
The next steps would involve supporting questions from users and allowing moderators to answer them. This would include:
 
The next steps would involve supporting questions from users and allowing moderators to answer them. This would include:
 +
* Extending the database to save questions
 +
* Send emails with the answer from moderation interface
 +
* Add buttons in main website to send questions<span id="project-information-3"></span>
 +
==== Project information ====
 +
* repository: https://github.com/openfoodfacts/hunger-games/
 +
* Slack channels: #hunger-games #robotoff #data-quality
 +
* Potential mentors: Raphael, Charles, Pierre, Alex
 +
* Project duration: 350 hours
 +
* Skills required: PostgreSQL, Python, Frontend (React/ReactNative or Flutter)
 +
* Difficulty rating: Medium<span id="impact-metrics-for-open-food-facts"></span>
 +
=== Impact and contribution metrics for Open Food Facts ===
 +
 +
==== Description ====
 +
Open Food Facts is the hub of a lot of interactions: visitors, various re-use, contributions, data import, machine learning predictions and human validation, moderation, etc. We currently have different systems interacting together, but we strive at having precise and reliable real time indicators. We would like to improve metrics (specially on contributions) to be able to measure our global impact as well as news coverage or new features impacts but also to fuel other important features like gamification or moderation.
 +
 +
==== Expected Outcomes ====
 +
The project is a bit of a patchwork of elements to assemble together but a very clear goal:
  
<ul>
+
* fix and upgrade openfoodfacts-events project to be really usable, it will be a very useful data source. Also extends its use in the existing tools, if relevant.
<li><blockquote><p>Extending the database to save questions</p></blockquote></li>
+
* make openfoodfacts-server logs more significant events to either influxdb or to openfoodfacts-events
<li><blockquote><p>Send emails with the answer from moderation interface</p></blockquote></li>
+
* write and run scripts to import data from previous years (as much as possible)
<li><blockquote><p>Add buttons in main website to send questions</p></blockquote></li></ul>
+
* build flexible and informative dashboards for the project
 +
* (stretch goal 1) bootstrap the gamification
 +
* (stretch goal 2) build a personal contributor dashboard
  
<span id="project-information-3"></span>
 
 
==== Project information ====
 
==== Project information ====
 +
As the project involves several competences you will need to be a fast learner able to quickly crawl an existing project and documentations. Also you have to be motivated by the ultimate goal of all those metrics to be able to grasp the righ way of doing them.
  
<ul>
+
* main repository: https://github.com/openfoodfacts/openfoodfacts-events
<li><blockquote><p>repository: https://github.com/openfoodfacts/hunger-games/</p></blockquote></li>
+
* secondary repositories: https://github.com/openfoodfacts/openfoodfacts-server https://github.com/openfoodfacts/openfoodfacts-infrastructure/tree/develop/docker/metrics 
<li><blockquote><p>Slack channels: #hunger-games #robotoff #data-quality</p></blockquote></li>
+
* Slack channels: #productopener #gamification
<li><blockquote><p>Potential mentors: Raphael, Charles, Pierre, Alex</p></blockquote></li>
+
* Potential mentors: Vova, Alex, Stephane, Pierre
<li><blockquote><p>Project duration: 350 hours</p></blockquote></li>
+
* Project duration: 350 hours
<li><blockquote><p>Skills required: PostgreSQL, Python, Frontend (React/ReactNative or Flutter)</p></blockquote></li>
+
* Skills required (some might be learned): PostgreSQL, Python, InfluxDB,  Grafana, Perl
<li><blockquote><p>Difficulty rating: Medium</p></blockquote></li></ul>
+
* Difficulty rating: Easy
 
 
<span id="impact-metrics-for-open-food-facts"></span>
 
== Impact metrics for Open Food Facts ==
 
Description coming soon
 
 
<span id="mobile-side"></span>
 
<span id="mobile-side"></span>
 
= Mobile-side =
 
= Mobile-side =
 
Coming soon
 
Coming soon
 +
 +
[[Category:GSOC]]

Latest revision as of 11:48, 20 January 2024

Ideas for GSOC 2023 will be posted here. There are just ideas, and are non limitative.

Server-side

A New API: Simplify and unify the Open Food Facts API used by 200+ apps

Description

Open Food Facts provides a free and open API to read, search and write products data and photos. This API powers more than 200 apps developed by 3rd parties, and is also used by many tools like our AI platform Robotoff and our power contribution tool Hunger Games. Our API is unfortunately very difficult to understand and use as it evolved organically in the last 10 years. It's not 1 API in fact, it's several APIs to do different things, and we have multiple versions of those APIs, each with a different state of features and documentation. This project aims to bring "A New API" to our many reusers. An API that is simple, unified, easy to understand and use, well documented (with OpenAPI) and well tested.

Expected outcomes

We started to build a new API v3 to provide a clean, generic and unified way to query all API features of Open Food Facts. This new API v3 is currently available for only a very small subsets of our existing APIs: product read, a very small subset of product write, and taxonomy suggestions.

This project is about revisiting one by one all existing APIs to:

  1. Decide which feature should be added, changed (or maybe even removed)
  2. Define the API for each feature (e.g. method, path, parameters, query and response headers and bodies etc.) and document it using OpenAPI. (This step is critical to ensure that the API will be both useful and used, and will need to be done in coordination with existing users of the API.)
  3. Implement the API for the feature and add integration tests for it

APIs that needs to be converted to API v3:

  • Product image upload, selection and crop
  • Full support for product write (should be done incrementally)
  • Search
  • Taxonomy read
  • User creation and profile edition
  • Authentification

Project information

Bring Taxonomy editor to a fully usable tool

Description

Taxonomies are at the heart of Open Food Facts in many aspects. It helps identify components (ingredients, labels, brands,…) and link them to useful properties, at the base of nutri-score, eco-score, allergens identification and some other properties.

Each taxonomy is a DAG (directed acyclic graph) where leaves have one or more parents. Currently the taxonomy is in a raw text file in our repository: https://github.com/openfoodfacts/openfoodfacts-server/tree/main/taxonomies.

While effective for the application, this format is quite cumbersome to edit for contributors.

Taxonomy editor was started during last Google Summer of code. Thanks to the work of Aadarsh, we got a first demonstration of the product, and it’s been in continuous development since then. But a boost would be needed to make it a really usable project.

Expected outcomes

Minimum feature set to be added would be: good inline documentation to on board users, an ergonomic UI, searching for potential issues in the taxonomy, tracking modification to enable undoing changes and comparison to original taxonomy, helping find entries to translate. The idea is to enable a wide public of contributors to participate in adding translation or simple items (like ingredients) while ensuring quality and helping moderators validate changes. Another important technical feature will be to augment significantly test coverage (at least on backend part).

From there, according to the pace of the projects many more features could be added which would be of great benefit: embarked assistance integrating tools (eg. searching / linking to wikidata, or wiktionary), integration of the products database (eg. seeing which entries need to be added or translated first), navigation in the taxonomy.

An important aspect: as this tool is meant to live a long time quality and maintainability of the project is important.

Project information

Improve our producer platform to the next level

Description

One lesser known but very important aspect of Open Food Facts is that, not only do we encourage citizens to contribute to the database (crowdfunding) but we also propose producers to do so. The incentive for them is to ensure they provide up to date and accurate information, that their products are well represented (eg. with studio photography). They also get hints on data quality to make their product better, which is an important goal to Open Food Facts.

Thanks to the support of the national French public health agency Santé Publique France, the platform is already alive and working. Though it is a bit rough for our users that are not tech savvy, also there are a lot of features which could help adoption and lessen the support needed to operate it.

Technically the producer platform uses the same code base as the main website.

Expected outcomes

The project will bring many new features to the producer platform. Here are some of the current priorities: add more inline help, make UX more convenient, support advanced updates patterns on fields, add a workflow to product edition, connect to our internal CRM (allow automating many tasks), collect more statistics on usage, support more fields and format for data import, more robust import/publishing workflows and add more tests.

Extended goals include adding rules to help producers make products with better Eco-score, advanced features to make editing products easier (like picture upload).

Project information

Data Quality & Moderation Hub

Description

Open Food Facts infrastructure is currently focused on reflecting the current state of the database, which is its primary goal. However, it would be beneficial to introduce a queue/backlog system to coordinate the efforts of users in data cleaning and moderation.

The system currently provides some level of coordination through the facets, indicating missing nutritional data and providing a list of products that need completion. However, this system lacks flexibility.

Expected outcomes

Currently, robots send messages to Slack channels that the admins need to clean. A queue of remaining problems to moderate would simplify the workload for the team and make it easier to track progress. It would also allow users to request moderation or ask questions.

In a perfect world, users would be able to create their own tasks, such as "extract nutritional table," and associate barcodes with each step.

    • select nutritional table image
    • crop nutritional table image
    • fill nutriments

The MVP would include a back-end with an API for moderation, providing a table with information such as barcode, image_id, type_of_problem, author (including Robotoff), and moderation status (has_been_moderated). The front-end would allow users to delete pictures and mark elements as moderated. The next steps would involve supporting questions from users and allowing moderators to answer them. This would include:

  • Extending the database to save questions
  • Send emails with the answer from moderation interface
  • Add buttons in main website to send questions

Project information

  • repository: https://github.com/openfoodfacts/hunger-games/
  • Slack channels: #hunger-games #robotoff #data-quality
  • Potential mentors: Raphael, Charles, Pierre, Alex
  • Project duration: 350 hours
  • Skills required: PostgreSQL, Python, Frontend (React/ReactNative or Flutter)
  • Difficulty rating: Medium

Impact and contribution metrics for Open Food Facts

Description

Open Food Facts is the hub of a lot of interactions: visitors, various re-use, contributions, data import, machine learning predictions and human validation, moderation, etc. We currently have different systems interacting together, but we strive at having precise and reliable real time indicators. We would like to improve metrics (specially on contributions) to be able to measure our global impact as well as news coverage or new features impacts but also to fuel other important features like gamification or moderation.

Expected Outcomes

The project is a bit of a patchwork of elements to assemble together but a very clear goal:

  • fix and upgrade openfoodfacts-events project to be really usable, it will be a very useful data source. Also extends its use in the existing tools, if relevant.
  • make openfoodfacts-server logs more significant events to either influxdb or to openfoodfacts-events
  • write and run scripts to import data from previous years (as much as possible)
  • build flexible and informative dashboards for the project
  • (stretch goal 1) bootstrap the gamification
  • (stretch goal 2) build a personal contributor dashboard

Project information

As the project involves several competences you will need to be a fast learner able to quickly crawl an existing project and documentations. Also you have to be motivated by the ultimate goal of all those metrics to be able to grasp the righ way of doing them.

Mobile-side

Coming soon