A security aware discrete mathematics framework for Nash-equilibrium based incentive design against selfish mining and validator collusion
Satpal Singh Kushwahasatpal.singh@jaipur.manipal.eduDepartment of Computer Science and EngineeringManipal University JaipurJaipur, Rajasthan, 303007, IndiaView full profile → , Pradeep Guptagupta.pradeep85@gmail.comDepartment of Computer Science and EngineeringAjay Kumar Garg Engineering CollegeGhaziabad, Uttar Pradesh, 201015, IndiaView full profile → , Lipika Goellipika.bose@gmail.comDepartment of Computer Science and Engineering (AIML)Sridevi Women’s Engineering CollegeHyderabad, Telangana, 500075, IndiaView full profile → , Y. Lakshmi Prasannaprasanna.yeluri@gmail.comDepartment of Computer Science and Information TechnologyKoneru Lakshmaiah Education FoundationHyderabad, Telangana, 500043, IndiaView full profile → , *Sonam GuptaCorresponding authorguptasonam6@gmail.comDepartment of Computer Science and EngineeringAjay Kumar Garg Engineering CollegeGhaziabad, Uttar Pradesh, 201015, IndiaView full profile →
* Corresponding author · click or hover a name for details
- Received:
- 01 Jan 2026
- Published Online:
- 31 Aug 2026
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- JIOS-2240
- Pages:
- 2987–2995
Abstract
Blockchain networks depend on incentive mechanisms to ensure that rational miners/validators follow the prescribed consensus protocol. However, strategic behavior such as selfish mining, validator collusion, and reward manipulation can degrade security, decentralization, and fairness. This paper proposes a game-theoretic framework to model and analyze incentive compatibility in blockchain systems. We formalize the interaction among participants as non-cooperative and coalition games, where agents choose between honest participation, selfish deviation, or collusive coordination. A mathematical payoff model is developed incorporating reward distribution, operational costs, detection probabilities, and penalty/slashing functions. Nash equilibrium conditions are derived to characterize when honest participation becomes a stable equilibrium. We further introduce a fairness-aware reward shaping mechanism that reduces the profitability of selfish strategies while discouraging collusion through coalition-dependent penalties. The framework provides explicit equilibrium constraints and design insights for secure and fair incentive mechanisms in both Proof-of-Work and Proof-of-Stake blockchain networks.
Keywords
Subject Classifications
References
[1] V. P. Singh, S. S. Biswas, B. Alankar, and S. Tanweer, “Cryptographic modeling and discrete structures for intrusion detection in Ethereum smart contracts,” J. Discrete Math. Sci. Cryptogr., vol. 28, no. 8, pp. 3039–3048 (2025).
[2] A. Dadhich, B. Keswani, and D. Goyal, “Comparative analysis of a novel smart contract-based hybrid access control model for blockchain-enabled secure IoT home automation systems,” J. Discrete Math. Sci. Cryptogr., vol. 28, no. 7, pp. 2875–2888 (2025).
[3] V. Ramachandran, H. M. A. Ghanimi, B. Marapelli, N. Kaur, M. R. Laxmi, R. K. Bommisetti, S. Sengan, and P. Dadheech, “An enterprise blockchain model: A reliable cryptography-based cyber-physical systems for securing user data,” J. Discrete Math. Sci. Cryptogr., vol. 28, no. 5-B, pp. 2103–2114 (2025).
[4] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” White Paper (2008). [Online]. Available: https://bitcoin.org/bitcoin.pdf
[5] I. Eyal and E. G. Sirer, “Majority is not enough: Bitcoin mining is vulnerable,” Commun. ACM, vol. 61, no. 7, pp. 95–102 (Jul. 2018).
[6] A. Sapirshtein, Y. Sompolinsky, and A. Zohar, “Optimal selfish mining strategies in Bitcoin,” in Proc. Int. Conf. Financial Cryptography and Data Security (FC), Berlin, Germany, pp. 515–532 (Feb. 2016).
[7] K. Nayak, S. Kumar, A. Miller, and E. Shi, “Stubborn mining: Generalizing selfish mining and combining with an eclipse attack,” in Proc. IEEE Eur. Symp. Security Privacy (EuroS&P), Saarbrücken, Germany, pp. 305–320 (Mar. 2016).
[8] G. O. Karame, E. Androulaki, M. Roeschlin, A. Gervais, and S. Čapkun, “Misbehavior in Bitcoin: A study of double-spending and accountability,” ACM Trans. Inf. Syst. Secur., vol. 18, no. 1, pp. 1–32 (2015).
[9] Y. Sompolinsky and A. Zohar, “Secure high-rate transaction processing in Bitcoin,” in Proc. Int. Conf. Financial Cryptography and Data Security (FC), Berlin, Germany, pp. 507–527 (Jan. 2015).
[10] A. Kiayias, A. Russell, B. David, and R. Oliynykov, “Ouroboros: A provably secure proof-of-stake blockchain protocol,” in Proc. Annu. Int. Cryptology Conf. (CRYPTO), Cham, Switzerland, pp. 357–388 (Jul. 2017).
[11] S. Mssassi and A. Abou El Kalam, “Game theory-based incentive design for mitigating malicious behavior in blockchain networks,” J. Sensor Actuator Netw., vol. 13, no. 1, Art. no. 7 (2024).
[12] W. Tang, E. Liu, W. Ni, X. Qu, B. Huang, K. Li, D. Niyato, and A. Jamalipour “Game-theoretic incentive mechanism for blockchain-based federated learning,” IEEE Trans. Mobile Comput., early access (2025), doi: 10.1109/TMC.2025.3567355.s
