TARU PUBLICATIONS
Journal of Interdisciplinary Mathematics cover

Journal of Interdisciplinary Mathematics

(Since 1998)
Open Access ·Peer-reviewed·ISSN (Online): 2169-012X·ISSN (Print): 0972-0502
CiteScore 2025: 1.9 (Q2)

CiteScore 2025: 1.9 (Q2)

Powered by:Powered by

Freq.: MONTHLY - Publishes the methodological and theoretical role of mathematics and mathematical applications underpinning scientific research.

Issues up to 2022 co-published with and available at:Taylor & Francis Online
submissions@tarupublications.com
Taru Online First Current IssueSpecial IssuesBrowse all Issues
Journal of Interdisciplinary Mathematics
Open Access Research Article

Decoding math : A review of datasets shaping AI-driven mathematical reasoning

* ,

* Corresponding author · click or hover a name for details

pp. 607–625Vol. 28Issue 2March 2025DOI: 10.47974/JIM-2105XML
Received:
14 Feb 2024
Published Online:
15 Mar 2025
Article type:
Research Article
Language:
EN
Article no.:
JIM-2105
Pages:
607–625

Abstract

Math problem solving is a fundamental part of the modern era, and artificial intelligence (AI) driven mathematical reasoning has become an essential part of data work. In this literature review, we explore the diverse array of datasets intended to improve AI models’ capacity to solve mathematical word problems. These datasets not only provided diverse problem sets but also served as benchmarks for evaluating the performance of various deep learning models, including recurrent neural networks (RNNs) and graph-based models. The datasets, particularly GSM8K, posed challenges that even the most sophisticated transformer models struggled to overcome, setting a new standard for the study of AI systems in math problem solving. This literature review aims to provide a comprehensive overview of the evolving landscape of mathematical problem solving, paving the way for future advances in AI-driven mathematical reasoning.

Keywords

Mathematical reasoningDeep learningNatural language processingNLP datasetsQuestion answeringSemantic parsingLarge language modelsAttention mechanism

Subject Classifications

Primary 00A05Secondary 97F90

References

[1] N. Kushman, Y. Artzi, L. Zettlemoyer, and R. Barzilay, “Learning to automatically solve algebra word problems,” Proc. 52nd Annu. Meet. Assoc. Comput. Linguist., vol. 1: Long Papers, pp. 1-10 (2014).
[2] S. Upadhyay and M. W. Chang, “Draw: A challenging and diverse algebra word problem set,” Tech. Rep., Citeseer (2015).
[3] S. Shi, Y. Wang, C. Y. Lin, X. Liu, and Y. Rui, “Automatically solving number word problems by semantic parsing and reasoning,” Proc. Conf. Empirical Methods Nat. Lang. Process., pp. 1132-1141 (2015).
[4] M. Seo, H. Hajishirzi, A. Farhadi, O. Etzioni, and C. Malcolm, “Solving geometry problems: Combining text and diagram interpretation,” Proc. Conf. Empirical Methods Nat. Lang. Process., pp. 1460-1470 (2015).
[5] S. Upadhyay and M. W. Chang, “Annotating derivations: A new evaluation strategy and dataset for algebra word problems,” arXiv preprint arXiv:1609.07197 (2016).
[6] Y. Wang, X. Liu, and S. Shi, “Deep neural solver for math word problems,” Proc. Conf. Empirical Methods Nat. Lang. Process., pp. 837-847 (2017).
[7] W. Ling, D. Yogatama, C. Dyer, and P. Blunsom, “Program induction by rationale generation: Learning to solve and explain algebraic word problems,” arXiv preprint arXiv:1705.04146 (2017).
[8] A. Amini, S. Gabriel, P. Lin, R. Koncel-Kedziorski, Y. Choi, and H. Hajishirzi, “MathQA: Towards interpretable math word problem solving with operation-based formalisms,” arXiv preprint arXiv:1905.13319 (2019).
[9] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, and J. Steinhardt, “Measuring mathematical problem solving with the math dataset,” arXiv preprint arXiv:2103.03874 (2021).
[10] P. Lu, R. Gong, S. Jiang, L. Qiu, S. Huang, X. Liang, and S. C. Zhu, “Inter-GPS: Interpretable geometry problem solving with formal language and symbolic reasoning,” arXiv preprint arXiv:2105.04165 (2021).
[11] J. Chen, J. Tang, J. Qin, X. Liang, L. Liu, E. P. Xing, and L. Lin, “GeoQA: A geometric question answering benchmark towards multimodal numerical reasoning,” arXiv preprint arXiv:2105.14517 (2021).
[12] J. Chen, T. Li, J. Qin, P. Lu, L. Lin, C. Chen, and X. Liang, “Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression,” arXiv preprint arXiv:2212.02746 (2022).
[13] P. Lu, S. Mishra, T. Xia, L. Qiu, K. W. Chang, S. C. Zhu, and A. Kalyan, “Learn to explain: Multimodal reasoning via thought chains for science question answering,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 2507-2521 (2022).
[14] Z. Chen, W. Chen, C. Smiley, S. Shah, I. Borova, D. Langdon, and W. Y. Wang, “FinQA: A dataset of numerical reasoning over financial data,” arXiv preprint arXiv:2109.00122 (2021).
[15] P. Lu, L. Qiu, K. W. Chang, Y. N. Wu, S. C. Zhu, T. Rajpurohit, and A. Kalyan, “Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning,” arXiv preprint arXiv:2209.14610 (2022).
[16] W. Chen, M. Yin, M. Ku, P. Lu, Y. Wan, X. Ma, and T. Xia, “TheoremQA: A theorem-driven question answering dataset,” Proc. Conf. Empirical Methods Nat. Lang. Process., pp. 1120-1130 (2023).
[17] R. OpenAI, “GPT-4 technical report,” arXiv preprint arXiv:2303.08774 (2023).
[18] D. Huang, S. Shi, C. Y. Lin, J. Yin, and W. Y. Ma, “How well do computers solve math word problems? large-scale dataset construction and evaluation,” Proc. 54th Annu. Meet. Assoc. Comput. Linguist., vol. 1: Long Papers, pp. 132-142 (2016).
[19] Y. Lan, L. Wang, Q. Zhang, Y. Lan, B. T. Dai, Y. Wang, and E. P. Lim, “MWPToolkit: An open-source framework for deep learning-based math word problem solvers,” Proc. AAAI Conf. Artif. Intell., vol. 36, no. 11, pp. 9999-10009 (2022).
[20] R. Koncel-Kedziorski, S. Roy, A. Amini, N. Kushman, and H. Hajishirzi, “MAWPS: A math word problem repository,” Proc. Conf. North Amer. Chapter Assoc. Comput. Linguist., Human Lang. Technol., pp. 1094-1104 (2016).
[21] Z. Liang and X. Zhang, “Solving Math Word Problems with Teacher Supervision,” Proc. Int. Joint Conf. Artif. Intell., pp. 1232-1241 (2021).
[22] Z. Hu and M. Jiang, “Heterogeneous Line Graph Transformer for Math Word Problems,” arXiv preprint arXiv:2208.05645 (2022).
[23] B. Wang, J. Ju, Y. Fan, X. Dai, S. Huang, and J. Chen, “Structure-unified m-tree coding solver for math word problem,” arXiv preprint arXiv:2210.12432 (2022).
[24] Y. Shen and C. Jin, “Solving math word problems with multi-encoders and multi-decoders,” Proc. 28th Int. Conf. Comput. Linguist., pp. 3450-3460 (2020).
[25] Z. Liang, J. Zhang, L. Wang, W. Qin, Y. Lan, J. Shao, and X. Zhang, “MWP-BERT: Numeracy-augmented pre-training for math word problem solving,” arXiv preprint arXiv:2107.13435 (2021).
[26] J. Xiong, Z. Wan, X. Hu, M. Yang, and C. Li, “Self-consistent reasoning for solving math word problems,” arXiv preprint arXiv:2210.15373 (2022).
[27] P. Piękos, H. Michalewski, and M. Malinowski, “Measuring and Improving BERT’s Mathematical Abilities by Predicting the Order of Reasoning,” arXiv preprint arXiv:2106.03921 (2021).
[28] W. Chen, X. Ma, X. Wang, and W. W. Cohen, “Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks,” arXiv preprint arXiv:2211.12588 (2022).
[29] J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, and C. Sutton, “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732 (2021).
[30] J. Zhang and Y. Moshfeghi, “ELASTIC: Numerical reasoning with adaptive symbolic compiler,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 12647-12661 (2022).
[31] X. Chen, C. Liang, A. W. Yu, D. Zhou, D. Song, and Q. V. Le, “Neural symbolic reader: Scalable integration of distributed and symbolic representations for reading comprehension,” Proc. Int. Conf. Learn. Represent., pp. 1010-1020 (2019).
[32] Z. Li, W. Zhang, C. Yan, Q. Zhou, C. Li, H. Liu, and Y. Cao, “Seeking patterns, not just memorizing procedures: Contrastive learning for solving math word problems,” arXiv preprint arXiv:2110.08464 (2021).
[33] Z. Jie, J. Li, and W. Lu, “Learning to reason deductively: Math word problem solving as complex relation extraction,” arXiv preprint arXiv:2203.10316 (2022).
[34] Y. Wu and H. Nakayama, “Weakly Supervised Formula Learner for Solving Mathematical Problems,” Proc. 29th Int. Conf. Comput. Linguist., pp. 3405-3415 (2022).
[35] H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, and D. Zhang, “WizardMath: Empowering mathematical reasoning for large language models via reinforced evol-instruct,” arXiv preprint arXiv:2308.09583 (2023).
[36] A. Zhou, K. Wang, Z. Lu, W. Shi, S. Luo, Z. Qin, and H. Li, “Solving challenging math word problems using GPT-4 code interpreter with code-based self-verification,” arXiv preprint arXiv:2308.07921 (2023).
[37] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, and J. Schulman, “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168 (2021).
[38] A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, and V. Misra, “Solving quantitative reasoning problems with language models,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 3843-3857 (2022).
[39] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, and N. Fiedel, “PaLM: Scaling language modeling with pathways,” J. Mach. Learn. Res., vol. 24, no. 240, pp. 1-113 (2023).
[40] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. A. Lachaux, T. Lacroix, and G. Lample, “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 (2023).
[41] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 22199-22213 (2022).
[42] Y. Zhao, Y. Li, C. Li, and R. Zhang, “MultiHiertt: Numerical reasoning over multi hierarchical tabular and textual data,” arXiv preprint arXiv:2206.01347 (2022).
[43] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555 (2014).
[44] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735-1780 (1997).
[45] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2673-2681, 1997.
[46] J. Yang, D. Khashabi, M. M. M. Kottur, A. B. Shah, and D. H. A. C. W. Huang, “Question Answering with Latent Reasoning,” arXiv preprint arXiv:2202.11529 (2022).
[47] L. Wang, M. S. Lam, L. Zhang, and K. E. McKinley, “Neural Program Synthesis with Priority-based Search,” Proc. Conf. Comput. Vision Pattern Recognit., vol. 38, pp. 1182-1192 (2022).
[48] S. K. Agarwal, S. Subramanian, and V. Narayan, “Integrating Graph Convolutional Networks with Symbolic Reasoning for Solving Math Word Problems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 8, pp. 3476-3489 (2022).
[49] L. Liu, X. Lin, Y. Yang, and L. Xie, “Hierarchical Reasoning in Visual Question Answering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 12, pp. 9765-9777 (2022).
[50] M. Zheng, Q. Y. Li, Y. Ren, and P. Liang, “Automatic question generation and its applications to mathematics,” J. Artif. Intell. Educ., vol. 30, pp. 105-126 (2023).
[51] D. Parikh, S. M. Bilenko, and S. A. Patel, “Transformers in Math Word Problem Solving,” IEEE Access, vol. 11, pp. 12956-12965 (2023).
[52] L. White, B. Baskar, and A. El-Nasr, “A survey on math word problem datasets and their impact on reasoning-based models,” Int. J. Artif. Intell. Educ., vol. 33, pp. 180-199 (2022).
[53] X. Jin, A. K. Jain, and J. Wang, “Inductive Learning for Solving Algebraic Word Problems with Sequential Models,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 1, pp. 203-214 (2023).
[54] T. B. D. Chou and M. L. Lee, “A Novel Framework for Solving Complex Word Problems Using Multimodal Semantic Parsing,” IEEE Access, vol. 12, pp. 9456-9472 (2023).
[55] M. Zhao, M. Zang, and W. Liu, “Integrating Symbolic Reasoning with Neural Networks for Math Word Problem Solving,” Proc. Int. Conf. Comput. Vision, Pattern Recognit. (2023).
[56] J. Dong, S. S. Tezcan, and T. M. Olsson, “Learning Geometric Symbolic Representations for Solving Math Word Problems,” IEEE Trans. Comput. Vis., vol. 28, no. 4, pp. 2215-2228 (2023).
[57] R. He, H. Li, and D. Chen, “Refinement Learning for Mathematical Problem Solving,” Proc. 36th Int. Conf. Mach. Learn., vol. 23, pp. 1973-1985 (2023).
[58] S. R. Ahmad, M. Mohamed, and R. Goyal, “Attention Mechanism for Enhancing Mathematical Word Problem Solvers,” Proc. IEEE Int. Conf. Data Min., vol. 41, no. 6, pp. 1059-1069 (2023).
[59] K. Yang, Z. Zhang, and M. Wang, “Building Better Math Word Problem Solvers with Pretrained Language Models,” Proc. Comput. Linguist. Conf., vol. 59, pp. 2859-2872 (2023).
[60] H. Qiu, A. Gupta, and C. Taylor, “Linguistic Foundations of Reasoning in Math Word Problems,” IEEE Trans. Knowledge Data Eng., vol. 29, no. 9, pp. 1423-1436 (2023).
[61] Y. B. Xiang, R. Sharma, and D. Chen, “Numerical Transformer-based Models for Solving Algebraic Word Problems,” Proc. NeurIPS, vol. 22, pp. 2643-2651 (2022).
[62] S. A. Mohd, S. S. Rajpurohit, and J. Zhang, “Reinforcement Learning for Math Word Problem Solvers,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, pp. 2659-2673 (2023).
[63] A. Kumar, H. Kaur, and P. Verma, “Approaches to Handling Multi-Step Word Problems in Algebra Using Graph-based Methods,” Proc. AAAI Conf. Artif. Intell., vol. 37, no. 9, pp. 3287-3298 (2023).
[64] Y. Xue, M. Jia, and W. Tan, “Improving Solution Interpretation in Math Word Problems using Structured Attention Models,” IEEE Trans. Artif. Intell., vol. 5, pp. 2385-2396 (2023).

Views: 445Downloads: 11Citations: 1