Open Access
Research Article
Patentometric review on automated plant phenotyping
Shrikrishna Kolharshrikrishna.kolhar@vpkbiet.orgSymbiosis Institute of Technology (SIT) Affiliated to Symbiosis International (Deemed University) Pune Maharashtra India; Vidya Pratishthan’s Kamalnayan Bajaj Institute of Engineering and Technology (VPKBIET) Baramati Maharashtra IndiaSymbiosis Institute of Technology (SIT) Affiliated to Symbiosis International (Deemed University) Pune Maharashtra IndiaView full profile → , *Jayant JagtapCorresponding authorjayant.jagtap@sitpune.edu.inDepartment of Electronics and Telecommunications Symbiosis Institute of Technology (SIT) Affiliated to Symbiosis International (Deemed University) Pune Maharashtra IndiaSymbiosis Institute of Technology (SIT) Symbiosis International (Deemed University) (SIU) LavalePune, Maharashtra, 412115, IndiaView full profile → , Amit Kumar Tiwarimeettiwari@gmail.comDepartment of Intellectual Property Symbiosis Centre for Research and Innovation Symbiosis International (Deemed University) Lavale Pune 412115 Maharashtra IndiaSymbiosis Institute of Technology (SIT) Symbiosis International (Deemed University) (SIU) LavalePune, Maharashtra, 412115, IndiaView full profile →
* Corresponding author · click or hover a name for details
- Published Online:
- 01 Jun 2023
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- CJSIM-2022-0047
- Pages:
- 119–140
Abstract
Plant phenotyping involves the measurement of observable traits of plants. Plant traits like leaf area, leaf count, leaf surface temperature, chlorophyll content, plant growth rate, emergence time of leaves, and reproductive organs depend on the interaction of its genotype with the environment. Plant phenotyping serves to analyze biotic and abiotic stresses on plants, select crop varieties resilient to the surrounding environment, and improve crop yield. Recent advancements in imaging technologies help expedite the growth of automatic, non-invasive, and efficient plant phenotyping systems. These plant phenotyping systems involve using different imaging techniques like visible imaging, hyperspectral imaging, chlorophyll fluorescence imaging (CFIM), thermal imaging to record, monitor, and analyze plant phenotypes using images. In the last few years, researchers have been working on developing image processing, computer vision, machine learning, and deep learning approaches for the accurate and precise analysis of plant images. Therefore, this paper reviews and presents insights about the research reported through patents in the area of automatic plant phenotyping. This review report uses patent databases like Espacenet, Lens, and Google Patents to search, review and analyze patent documents. The paper presents a patentometric analysis of all 67 patent documents available till date focusing on automatic image-based plant phenotyping. The review provides a summary and analysis of outstanding patents in terms of qualitative and quantitative patent indices. This article provides a comprehensive global patent study to aid researchers and scientists develop more efficient plant phenotyping algorithms, devices, and systems.
Keywords
References
[1] Arthur, D. E., & Michael, T. J., System, method and/or computer readable medium for growing plants in an autonomous green house, Issue WO 2019/222860 A1 (2019). https://lens.org/088-020-584-337-767.[2] Azimi, S., Kaur, T., & Gandhi, T. K., A deep learning approach to measure stress level in plants due to Nitrogen deficiency. Measurement: Journal of the International Measurement Confederation (2020). https://doi.org/10.1016/j.measurement.2020.108650.[3] Chad, D. A., Hod, L., J, N. R., A, G. M., Tyr, W.-H., Ethan, S., Jason, Y., & Siyuan, C., Methods and systems for pattern characteristic detection, Issue US 10747999 B2 (2020). https://lens.org/140-273-552-977-355.[4] Chris, M., Deep learning systems and methods for use in computer vision, Issue WO 2018/112514 A1 (2018). https://lens.org/161-534-963-040-625.[5] Das Choudhury, S., Samal, A., & Awada, T., Leveraging image analysis for high-throughput plant phenotyping. In Frontiers in Plant Science (2019). https://doi.org/10.3389/ fpls.2019.00508.[6] Dellen, B., Scharr, H., & Torras, C., Growth signatures of rosette plants from time-lapse video. IEEE/ACM Transactions on Computational Biology and Bioinformatics (2015). https://doi.org/10.1109/TCBB.2015.2404810.[7] Deshpande, N., Ahmed, S., & Khode, A., Web based Targeted Advertising: A Study based on Patent Information. Procedia Economics and Finance, 11 (2014). https://doi. org/10.1016/s2212-5671(14)00218-4.[8] Deshpande, N., Ahmed, S., & Khode, A., Business intelligence through patinformatics: A study of energy efficient data centres using patent data. Journal of Intelligence Studies in Business, 6(3) (2016). https://doi.org/10.37380/jisib.v6i3.193.[9] Ernst, H., & Omland, N., The Patent Asset Index - A new approach to benchmark patent portfolios. World Patent Information, 33(1) (2011). https://doi.org/10.1016/j. wpi.2010.08.008.[10] Espacenet. Espacenet – search results. In Espacenet (2020).[11] Falk, K. G., Jubery, T. Z., Mirnezami, S. V., Parmley, K. A., Sarkar, S., Singh, A., Ganapathysubramanian, B., & Singh, A. K., Computer vision and machine learning enabled soybean root phenotyping pipeline. Plant Methods (2020). https://doi.org/10.1186/ s13007-019-0550-5.[12] Falk, N., & Train, K., Patent valuation with forecasts of forward citations. Journal of Business Valuation and Economic Loss Analysis, 12(1) (2017). https://doi.org/10.1515/ jbvela-2016-0002.[13] Ghule, B., Laad, M., & Tiwari, A. K., Poly-4-methyl-1-pentene a dielectric material: Patent landscape. In Journal of Energy Storage, Vol. 36 (2021). https://doi.org/10.1016/j. est.2021.102335.[14] Guoxiang, S., Xiaochan, W., Yongqian, D., Yingying, L. I. U., Yongbo, L. I., Yongnian, Z., Yuhua, L. I., & Baohua, Z., High-flux greenhouse plant phenotype measurement system based on Kinect autonomous calibration, Issue CN 108895964 A (2018). https://lens. org/159-342-863-162-459.[15] Hamuda, E., Glavin, M., & Jones, E., A survey of image processing techniques for plant extraction and segmentation in the field. In Computers and Electronics in Agriculture (2016). https://doi.org/10.1016/j.compag.2016.04.024[16] Hiroaki, S., Davy, N., Bert, D. B., Luc, V. G., Marc, P., & Nico, C., Method and system for training a neural network to be used for semantic instance segmentation, Issue WO 2019/015785 A1 (2019). https://lens.org/072-110-972-143-559.[17] Huichun, Z., Meng, Z., Hongping, Z., Jiaqiang, Z., & Liming, B., Self-propelled nursery stock trunk phenotypic information acquisition system and acquisition method thereof, Issue CN 111561873 A (2020). https://lens.org/035-284-087-561-596.[18] Isaak, W., Richard, H., & Daniel, I., Mobile 3D imaging system and method, Issue WO 2020/168313 A1 (2020). https://lens.org/092-987-681-236-444.[19] Jian, G., R Trent, N., M Kyle, L., T Lloyd, C., & F Peter, A., Ecosystem for determining plant-microbe interactions, Issue US 10787639 B2 (2020). https://lens.org/064-711-531- 210-406.[20] Kamp, R. L., & Matthew, C., Method for automatic phenotype measurement and selection, Issue US 9658201 B2 (2017). https://lens.org/008-831-407-291-486. [21] Kamp, R. L., & Matthew, C., Plant treatment based on morphological and physiological measurements, Issue US 2017/0219711 A1 (2017). https://lens.org/185-289-845-309- 501.[22] Lee, S. H., Chan, C. S., Mayo, S. J., & Remagnino, P., How deep learning extracts and learns leaf features for plant classification. Pattern Recognition (2017). https://doi. org/10.1016/j.patcog.2017.05.015.[23] Li Qiongyan, Gao Yunpeng, & Weng Yuchen. An automatic detection method of wheat tiller number based on RGB image, Issue CN 107993243 B (2020). https://lens.org/139- 597-264-356-648.[24] Liang, D., Huawei, J., & Maneesha, A., High-throughput large-scale plant phenotyping instrumentation, Issue US 10470379 B1 (2019). https://lens.org/125-044-622-758-770.[25] Lin, Z., & Guo, W., Sorghum panicle detection and counting using unmanned aerial system images and deep learning. Frontiers in Plant Science, 11 (2020). https://doi. org/10.3389/fpls.2020.534853.[26] Liu, Y., Cen, C., Che, Y., Ke, R., Ma, Y., & Ma, Y., Detection of Maize Tassels from UAV RGB imagery with faster R-CNN. Remote Sensing (2020). https://doi.org/10.3390/ rs12020338.[27] Long, Q., Ruoling, D., Xu, M., Yu, J., Xunan, H., Chuang, L., Ming, T., & Qian, J., Rapid detection method for rice effective ears, Issue CN 111462058 A (2020). https://lens. org/181-974-450-056-845.[28] Mastrodimos, N., Lentzou, D., Templalexis, C., Tsitsigiannis, D. I., & Xanthopoulos, G., Development of thermography methodology for early diagnosis of fungal infection in table grapes: The case of Aspergillus carbonarius. Computers and Electronics in Agriculture, 165 (2019). https://doi.org/10.1016/j.compag.2019.104972.[29] Mishra, P., Lohumi, S., Ahmad Khan, H., & Nordon, A., Close-range hyperspectral imaging of whole plants for digital phenotyping: Recent applications and illumination correctionapproaches. In Computers and Electronics in Agriculture, Vol. 178 (2020). https:// doi.org/10.1016/j.compag.2020.105780.[30] Mochida, K., Koda, S., Inoue, K., Hirayama, T., Tanaka, S., Nishii, R., & Melgani, F., Computer vision-based phenotyping for improvement of plant productivity: A machine learning perspective. In GigaScience (2018). https://doi.org/10.1093/gigascience/giy153.[31] Moehrle, M. G., Walter, L., Bergmann, I., Bobe, S., & Skrzipale, S., Patinformatics as a business process: A guideline through patent research tasks and tools. In World Patent Information, Vol. 32, Issue 4 (2010). https://doi.org/10.1016/j.wpi.2009.11.003.[32] Noruzi, A., & Abdekhoda, M., Google patents: The global patent search engine. Webology, 11(1) (2014).[33] Osmat, J. et al., CRISPR - The Lens - Free & open patent and scholarly search. Lens.Org (2019).[34] Pandey, P. N., Saini, N., Sapre, N., Kulkarni, D. A., & Tiwari, D. A. K., Prioritising breast cancer theranostics: A current medical longing in oncology. In Cancer Treatment and Research Communications, Vol. 29 (2021). https://doi.org/10.1016/j.ctarc.2021.100465.[35] Pavicic, M., Overmyer, K., Rehman, A. U., Jones, P., Jacobson, D., & Himanen, K., Image- based methods to score fungal pathogen symptom progression and severity in excised arabidopsis leaves. Plants, 10(1) (2021). https://doi.org/10.3390/plants10010158.[36] Pengcheng, N., Hui, Z., Chengyong, C., & Yong, H., Three-dimensional full-automatic online phenotype high-flux detection platform for crops, Issue CN 110089313 A (2019). https://lens.org/048-966-801-954-275.[37] Pieruschka, R., & Schurr, U., Plant phenotyping: past, present, and future. Plant Phenomics (2019). https://doi.org/10.34133/2019/7507131.[38] Praveen Kumar, J., & Domnic, S., Image based leaf segmentation and counting in rosette plants. Information Processing in Agriculture (2019). https://doi.org/10.1016/j. inpa.2018.09.005.[39] Qingbin, Z.-J., Automatic-scanning plant phenotype analysis system, Issue CN 109032212 A (2018). https://lens.org/136-431-495-352-098.[40] Qiongyan, L., Yunpeng, G., & Yuchen, W., Method for automatically detecting wheat tiller number based on RGB images, Issue CN 107993243 A (2018). https://lens.org/010- 707-383-288-963.[41] R Nicholas, G., & J John M, D., AI-powered autonomous plant-growth optimization system that automatically adjusts input variables to yield desired harvest traits, Issue US 10803312 B2 (2020). https://lens.org/092-017-115-160-230.[42] R Steve, T., Brant, B., Kyle, G., Heath, R., & Tim, A., Autonomous integrated farming system, Issue US 10149422 B2 (2018). https://lens.org/061-661-935-547-767.[43] Raturi, M. K., Sahoo, P. K., Mukherjee, S., & Tiwari, A. K., Patinformatics – An emerging scientific discipline. SSRN Electronic Journal (2012). https://doi.org/10.2139/ ssrn.1566067.[44] Richard, S. X. R., Jurgen, F., & Anthony, P., Method and system for characterising plant phenotype, Issue WO 2013/082648 A1 (2013). https://lens.org/065-288-260-249-632.[45] Roitsch, T., Cabrera-Bosquet, L., Fournier, A., Ghamkhar, K., Jiménez-Berni, J., Pinto, F., & Ober, E. S., Review: New sensors and data-driven approaches—A path to next generation phenomics. In Plant Science (2019). https://doi.org/10.1016/j.plantsci.2019.01.011.[46] Sun Guoxiang, Wang Xiaoyan, Ding Yongqian, Liu Yingying, Li Yongbo, Zhang Yongnian, Li Yuhua, & Zhang Baohua. (2020). A high-throughput greenhouse plant phenotype measurement system based on Kinect self-calibration (Issue CN 108895964 B). https://lens.org/114-534-552-776-238.[47] Trippe, A. J., Patinformatics: Tasks to tools. World Patent Information, 25(3) (2003). https://doi.org/10.1016/S0172-2190(03)00079-6.[48] Uchiyama, H., Sakurai, S., Mishima, M., Arita, D., Okayasu, T., Shimada, A., & Taniguchi, R. I., An easy-to-setup 3D phenotyping platform for KOMATSUNA dataset. Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW (2017).https://doi.org/10.1109/ICCVW.2017.239.[49] Vincent, C. L., Singh, V., Chakraborty, K., & Gopalakrishnan, A. Patent data mining in fisheries sector: An analysis using Questel-Orbit and Espacenet. World Patent Information, 51 (2017). https://doi.org/10.1016/j.wpi.2017.11.004.[50] William, R., Matthew, B., David, B., Elliott, G., & Zhiqiang, Y., Augmented-human field inspection tools for automated phenotyping systems and agronomy tools, Issue US 10638667 B2 (2020). https://lens.org/054-875-263-814-032.[51] WIPO., Guide to the international patent classification. World Intelelctual Property Organization (2018).[52] Zhang Huichun, Zhang Meng, Zhou Hongping, Zheng Qiangqiang, & Bian Liming. Selfpropelled seedling tree trunk phenotype information collection system and collection method, Issue CN 111561873 B (2021). https://lens.org/178-172-146-444-007.
Views: 292Downloads: 5Citations: 0
