Open Access
Research Article
A large-scale optimization method based on gradient approximation and hybrid grouping
*Haiyan LiuCorresponding authorhyliu83@126.comDepartment of Software Engineering School of Computer Science and Technology Xi’an University of Posts and TelecommunicationsXi’an, Shaanxi, 710121, ChinaView full profile → , Yi Chengcyjy2000@163.comDepartment of Software Engineering School of Computer Science and Technology Xi’an University of Posts and TelecommunicationsXi’an, Shaanxi, 710121, ChinaView full profile → , Siyu Gaogsy0408150240@163.comDepartment of Software Engineering School of Computer Science and Technology Xi’an University of Posts and TelecommunicationsXi’an, Shaanxi, 710121, ChinaView full profile → , Hanzhe Yang490007952@qq.comDepartment of Software Engineering School of Computer Science and Technology Xi’an University of Posts and TelecommunicationsXi’an, Shaanxi, 710121, ChinaView full profile →
* Corresponding author · click or hover a name for details
- Received:
- 01 Nov 2024
- Published Online:
- 01 Jun 2026
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- JIOS-2005
- Pages:
- 2165–2191
Abstract
With advances in science and technology, optimization problems increasingly involve numerous decision variables, posing significant challenges for optimization algorithms. These problems are often mentioned as LSGO problems (large-scale global optimization) and have received considerable attention. LSGO problem is very challenging to solve due to the high dimensions, countless local optimal solutions, and huge computational burden. In this paper, a gradient approximation and hybrid grouping method (GAHG) is proposed to address such challenges. Firstly, we design an adaptive gradient approximation based search algorithm to make a quick search in a reduced search region and find preliminary good solutions. During this stage, important information about the decision variables will be stored for future usage. Further more, we introduce a hybrid grouping strategy designed to efficiently and reasonably decompose the large-scale fully non-separable problems. The hybrid grouping method consists of five decomposing strategies to assign potentially highly related variables into the same group. The performance of GAHG is evaluated on widely used large-scale benchmark suite CEC’2010 and CEC’2013 large-scale test problems. Experimental results and statistical analysis are made, which indicate the proposed GAHG is effective and efficient in solving large-scale problems.
Keywords
Subject Classifications
90C06 (Large-scale problems in mathematical programming)
References
[1] C. Wang and J. Gao, “High-dimensional waveform inversion with cooperative coevolutionary differential evolution algorithm,” IEEE Geoscience and Remote Sensing Letters, vol. 9, no. 2, pp. 297-301 (2012).[2] S. Mahdavi, S. Rahnamayan, and M. E. Shiri, “Multilevel framework for large-scale global optimization,” Soft Computing, vol. 21, pp. 4111-4140 (2017).[3] M. Sam, A. DAriano, F. Corman, and D. Pacciarelli, “Metaheuristics for efficient aircraft scheduling and re-routing at busy terminal control areas,” Transportation Research Part C: Emerging Technologies, vol. 80, pp. 485-511 (2017).[4] V. J. Kadam and S. M. J. and, “Performance analysis of hyperparameter optimization methods for ensemble learning with small and medium sized medical datasets,” Journal of Discrete Mathematical Sciences and Cryptography, vol. 23, no. 1, pp. 115-123 (2020).[5] M. A. Potter and K. A. D. Jong, “Cooperative coevolution: An architecture for evolving coadapted subcomponents,” Evolutionary Computation, vol. 8, no. 1, pp. 1-29 (2000).[6] R. Salomon, “Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. a survey of some theoretical and practical aspects of genetic algorithms,” Biosystems, vol. 39, no. 3, pp. 263-278 (1996).[7] M. A. Potter and K. A. De Jong, “A cooperative coevolutionary approach to function optimization,” in Parallel Problem Solving from Nature—PPSN III (Y. Davidor, H.-P. Schwefel, and R. Männer, eds.), (Berlin, Heidelberg), pp. 249-257, Springer Berlin Heidelberg (1994).[8] Z. Yang, K. Tang, and X. Yao, “Large scale evolutionary optimization using cooperative coevolution,” Information Sciences, vol. 178, no. 15, pp. 2985-2999, Nature Inspired Problem-Solving (2008).[9] M. Omidvar, X. Li, and X. Yao, “Cooperative co-evolution with delta grouping for large scale non-separable function optimization,” Proceedings of the IEEE Congress on Evolutionary Computation (CEC2010) ; Conference date: 23-07-2010 (July 2010).[10] Z. Yang, K. Tang, and X. Yao, “Multilevel cooperative coevolution for large scale optimization,” in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), pp. 1663-1670 (2008).[11] M. N. Omidvar, X. Li, Y. Mei, and X. Yao, “Cooperative co-evolution with differential grouping for large scale optimization,” IEEE Transactions on Evolutionary Computation, vol. 18, pp. 378-393 (JUN 2014).[12] Y. Sun, M. Kirley, and S. K. Halgamuge, “Extended differential grouping for large scale global optimization with direct and indirect variable interactions,” Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (2015).[13] M. N. Omidvar, M. Yang, Y. Mei, X. Li, and X. Yao, “Dg2: A faster and more accurate differential grouping for large-scale black-box optimization,” IEEE Transactions on Evolutionary Computation, vol. 21, no. 6, pp. 929-942 (2017).[14] Y. Sun, M. Kirley, and S. K. Halgamuge, “A recursive decomposition method for large scale continuous optimization,” IEEE Transactions on Evolutionary Computation, vol. 22, no. 5, pp. 647-661 (2018).[15] Y. Mei, M. N. Omidvar, X. Li, and X. Yao, “A competitive divide-and-conquer algorithm for unconstrained large-scale black-box optimization,” ACM Trans. Math. Softw., vol. 42 (June 2016).[16] M. Yang, A. Zhou, C. Li, and X. Yao, “An efficient recursive differential grouping for large-scale continuous problems,” IEEE Transactions on Evolutionary Computation, vol. 25, no. 1, pp. 159-171 (2021).[17] H.-b. Xu, F. Li, and H. Shen, “A three-level recursive differential grouping method for large-scale continuous optimization,” IEEE Access, vol. 141946–141957, pp. 8 (2020).[18] X. Liu, F. Li, Y. Xu, and H. Shen, “A new three-level recursive differential grouping method for large-scale optimization,” in 2022 34th Chinese Control and Decision Conference (CCDC), pp. 236-241 (2022).[19] L. Zheng, G. Xu, and W. Chen, “Three stages recursive differential grouping for large-scale global optimization,” IEEE ACCESS, vol. 11, pp. 109734-109746 (2023).[20] Y. Sun and H. Yue, “An improved decomposition method for large-scale global optimization: bidirectional-detection differential grouping,” Applied Intelligence, vol. 52, pp. 11569-11591 (AUG 2022).[21] J. Ma, F. Chang, and X. Yu, “Large-scale evolutionary optimization approach based on decision space decomposition,” Frontiers in Energy Research, vol. 10, JUN 9 (2022).[22] A. Kumar, S. Das, and R. Mallipeddi, “An efficient differential grouping algorithm for large-scale global optimization,” IEEE Transactions on Evolutionary Computation, vol. 28, no. 1, pp. 32-46 (2024).[23] J.-Y. Li, Z.-H. Zhan, K. C. Tan, and J. Zhang, “Dual differential grouping: A more general decomposition method for large-scale optimization,” IEEE Transactions on Cybernetics, vol. 53, no. 6, pp. 3624-3638 (2023).[24] R. Zhong, E. Zhang, and M. Munetomo, “Cooperative coevolutionary surrogate ensemble-assisted differential evolution with efficient dual differential grouping for large-scale expensive optimization problems,” Complex & Intelligent Systems, vol. 10, pp. 2129-2149, 4 (2024).[25] X.-M. Hu, F.-L. He, W.-N. Chen, and J. Zhang, “Cooperation coevolution with fast interdependency identification for large scale optimization,” Information Sciences, vol. 381, pp. 142-160 (2017).[26] Y. Wang, H. Liu, F. Wei, T. Zong, and X. Li, “Cooperative Coevolution with Formula-Based Variable Grouping for Large-Scale Global Optimization,” Evolutionary Computation, vol. 26, pp. 569-596, 12 (2018).[27] H. Liu, Y. Wang, L. Liu, and X. Li, “A two phase hybrid algorithm with a new decomposition method for large scale optimization,” Integrated Computer-Aided Engineering, vol. 25, pp. 1-19, 04 (2018).[28] H. Liu, Y. Wang, and N. Fan, “A hybrid deep grouping algorithm for large scale global optimization,” IEEE Transactions on Evolutionary Computation, vol. 24, no. 6, pp. 1112-1124 (2020).[29] H. Liu, Y. Cheng, S. Xue, and S. Tuo, “A space-reduction based three-phase approach for large-scale optimization,” APPLIED SOFT COMPUTING, vol. 144 (SEP 2023).[30] M. Yang, M. N. Omidvar, C. Li, X. Li, Z. Cai, B. Kazimipour, and X. Yao, “Efficient resource allocation in cooperative co-evolution for large-scale global optimization,” IEEE Transactions on Evolutionary Computation, vol. 21, no. 4, pp. 493-505 (2017).[31] M. Yang, A. Zhou, C. Li, J. Guan, and X. Yan, “Ccfr2: A more efficient cooperative co-evolutionary framework for large-scale global optimization,” Information Sciences, vol. 512, pp. 64-79 (2020).[32] M. Yang, A. Zhou, X. Lu, Z. Cai, C. Li, and J. Guan, “Ccfr3: A cooperative co-evolution with efficient resource allocation for large-scale global optimization,” Expert Systems with Applications, vol. 203, p. 117397 (2022).[33] M. Yang, J. Gao, A. Zhou, C. Li, and X. Yao, “Contribution-based cooperative co-evolution with adaptive population diversity for large-scale global optimization [research frontier],” IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE, vol. 18, pp. 56-68 (AUG 2023).[34] R. Jarray, M. A. Dhaifallah, H. Rezk, and S. Bouallègue, “Parallel cooperative coevolutionary grey wolf optimizer for path planning problem of unmanned aerial vehicles,” Sensors (Basel, Switzerland), vol. 22 (2022).[35] A. Kelkawi, M. El-Abd, and I. Ahmad, “Gpu-based cooperative coevolution for large-scale global optimization,” Neural Computing and Applications, vol. 35, pp. 4621-4642 (Feb. 2023). Publisher Copyright: 2022, The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature.[36] L.-Y. Tseng and C. Chen, “Multiple trajectory search for large scale global optimization,” in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), pp. 3052-3059 (2008).[37] D. Molina, M. Lozano, and F. Herrera, “Ma-sw-chains: Memetic algorithm based on local search chains for large scale continuous global optimization,” in IEEE congress on evolutionary computation, pp. 1-8, IEEE (2010).[38] A. LaTorre, S. Muelas, and J.-M. Peña, “Large scale global optimization: Experimental results with mos-based hybrid algorithms,” in 2013 IEEE congress on evolutionary computation, pp. 2742-2749, IEEE (2013).[39] R. Cheng and Y. Jin, “A competitive swarm optimizer for large scale optimization,” IEEE Transactions on Cybernetics, vol. 45, no. 2, pp. 191-204 (2015).[40] Q. Yang, W.-N. Chen, T. Gu, H. Zhang, J. D. Deng, Y. Li, and J. Zhang, “Segment-based predominant learning swarm optimizer for large-scale optimization,” IEEE Transactions on Cybernetics, vol. 47, no. 9, pp. 2896-2910 (2017).[41] F. Wang, X. Wang, and S. Sun, “A reinforcement learning level-based particle swarm optimization algorithm for large-scale optimization,” Inf. Sci., vol. 602, p. 298-312 (jul 2022).[42] H. Deng, L. Peng, H. Zhang, B. Yang, and Z. Chen, “Ranking-based biased learning swarm optimizer for large-scale optimization,” Inf. Sci., vol. 493, p. 120-137 (Aug 2019).[43] M. Wang and M. Jiang, “Combing gibbs-sampling with adaptive particle swarm for large scale global optimization,” in 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI), pp. 856-860 (2018).[44] Y. Liu and Guiyan Hu, “A modified particle swarm optimization for large-scale numerical optimizations and engineering design problems,” Journal of Intelligent Manufacturing, vol. 30, no. 6 (2019).[45] Z.-J. Wang, Z.-H. Zhan, S. Kwong, H. Jin, and J. Zhang, “Adaptive granularity learning distributed particle swarm optimization for large-scale optimization,” IEEE Transactions on Cybernetics, vol. 51, no. 3, pp. 1175-1188 (2021).[46] X. Zhang and Q. Lin, “Three-learning strategy particle swarm algorithm for global optimization problems,” Information Sciences, vol. 593, pp. 289-313 (2022).[47] D. Li, L. Wang, W. Guo, M. Zhang, B. Hu, and Q. Wu, “A particle swarm optimizer with dynamic balance of convergence and diversity for large-scale optimization,” Applied Soft Computing, vol. 132, p. 109852 (2023).[48] S. Liu, Z.-J. Wang, Y.-G. Wang, S. Kwong, and J. Zhang, “Bi-directional learning particle swarm optimization for large-scale optimization,” Applied Soft Computing, vol. 149, p. 110990 (2023).[49] W.-Y. Fu, “Accelerated high-dimensional global optimization: A particle swarm optimizer incorporating homogeneous learning and autophagy mechanisms,” Information Sciences, vol. 648, p. 119573 (2023).[50] C. Huang, X. Zhou, X. Ran, Y. Liu, W. Deng, and W. Deng, “Co-evolutionary competitive swarm optimizer with three-phase for large-scale complex optimization problem,” Information Sciences, vol. 619, pp. 2-18 (2023).[51] M. Ester, H. P. Kriegel, J. Sander, and X. Xiaowei, “A density-based algorithm for discovering clusters in large spatial databases with noise,” 12 1996.[52] J. H. Ward Jr, “Hierarchical grouping to optimize an objective function,” Journal of the American statistical association, vol. 58, no. 301, pp. 236-244 (1963).[53] Z. Yang, K. Tang, and X. Yao, “Self-adaptive differential evolution with neighborhood search,” in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), pp. 1110-1116 (2008).[54] K. Tang, X. Li, P. N. Suganthan, Z. Yang, and T. Weise, “Benchmark functions for the cec’2010 special session and competition on large-scale,” (2009).[55] K. Tang, X. Yao, P. N. Suganthan, C. MacNish, Y. Ping Chen, C. M. Chen, and Z. Yang, “Benchmark functions for the cec’2013 special session and competition on large-scale global optimization,” (2008).
Views: 54Downloads: 20Citations: 0
