At the end of 2022, we would like to thank all of our users who leave their feedback ⭐️ for their contribution to the Mapflow AI-mapping all over the World. Eventually, we got to the conclusion to review the feedback on the regular basis and start summarizing it to track Mapflow progress.
Shortly, we added this feedback option to Mapflow Web App nearly a year ago — allowing users to rate the imagery processing results by simply scoring them from 1 to 5. Having said that most of our commercial users are not supposed to work through the Web App as they are usually used to working with some specialized software or a plugin and getting it powered by Mapflow API like Mapflow for QGIS.
Therefore the statistics we have is a bit biased to the feedback provided by Mapflow Web users only.
Feedback review
1. Most of the user ratings are either 5 or 1. Does it speak for the extreme perceptions of AI? By some opinions in our team it’s just more common for people to express their satisfaction or disappointment in the results while the intermediate score assessment requires more detailed review and comperison.
We’ve got 575 feedback raitings in 2022. The average score is ~3 which keeps steady. In the meanwhile the particular score of 3 shares only 15% of all feedback. The next model chart indicates the avg score by each model.
Our champions are “buildings” and “roads” 💪 (“buildings” is the most important for us due to its implementation into commercial solutions like 3D buildings for Radio frequency planning maps). The “dense-buildings” model which is applicable to specific construction textures tends to be good enough when appropriately applied. The rest is the matter of the applicability of so called “regional” models to the large number of contexts while the visual patterns and conditions vary substantially.
2. We were interested in conducting some internal research by reviewing the main issues (1-star scores mostly) as well as the overall relevance of the user scores.
We started reviewing the feedback and came up with the the following classification for the revised processings:
- bad result
- bad input
- non-relevant model
- non-relevant score
- bad result but the new version of model is available
By reviewing 1-score feedback we got the estimates that ~63% is actually bad results, however, 25% are either non-relevant model or non-relevant score.
5-score feedback is more homogenious, 85% seem to be the relevant score.
Mapflow rating by countries
Thanks to the feedback we could rate Mapflow performance by countries.
Of course, the representation of these ratings depends on the number of scores – China, India and Thailand come out on top by the number of reviews. However the avg score in these countries varies:
The main conclusion we came to by reviewing the feedback:
❗️The Mapflow users’ feedback is appeared to be the useful communication channel even if limited to the Web App. We won’t leave the free users’ feedback unresponded and will do our best to suggest the solution in case of the issues.
Having the feedback and the total number / dynamics of processings by countries through the use of different Mapflow applications (Web, QGIS, and API) we are driven to target our efforts to improve the technology and develop the more confident strategy where it doesn’t succeed as much as possible.
The new releases to come in the beginning of the next year — new version of the Building segmentation model, new UAV / Forest model for powerline monitoring projects, User data catalogue in Mapflow Web and Mapflow-QGIS and more — stay tuned and many thanks for your feedback!
