Dual-branch SC wireless systems with HQAM for beyond 5G over eta-mu fading channels

In this study, the performance of hexagonal quadrature amplitude modulation (HQAM) for a dual-branch selection combining (SC) receiver for beyond fifth-generation (B5G) is analyzed in detail. For general applicability, the links are modeled with η-μ fading channels. In this context, analytical expressions of error probability (EP) are derived in terms of the Chernoff, Chiani, Prony (2 and 3 terms), and Trapezoidal approximations for Gaussian Q-function. For the high signal to noise ratio (SNR) analysis, asymptotic EP expression is also derived by using the Chernoff approximation. Further, for the comparison, the analytical EP expressions with rectangular QAM and quadrature phase-shift keying modulation schemes are obtained by using the Chiani approximation. Moreover, a comprehensive work of various modulation schemes is presented and the effects of fading parameters, type of Gaussian Q-function approximations, and average SNR values of first and second links are highlighted on the EP performance. Finally, to show the accuracy of the proposed analytical derivations, some results for the exact simulations and numerical results are demonstrated.