The main purpose of this research is to figure out the most effective determinants that play a vital role in enhancing the effectiveness of the internal Shari'ah audit in the Islamic banking industry. This paper reviews the existing literature to build comprehensive knowledge that would assist in determining the main factors that impact on the effectiveness of Shari'ah audit in Islamic banks. This research proposes a conceptual framework of factors that impact on Shari'ah audit effectiveness in IBs based on previously published studies. The proposed framework includes external and internal factors as well as internal Shari'ah audit structure, process and requirements. First, the regulators need to provide a detailed framework for Shari'ah audit which covers the main requirements for effective Shari'ah governance. Second, Islamic financial institutions (IFIs) need to pay more attention to following the Shari'ah audit process in order to achieve the objective of effective Shari'ah governance. Finally, the dearth of empirical research on the role and effectiveness of Shari'ah audit in Islamic banking highlights the need to develop an appropriate methodology to enhance the study of the effectiveness of Shari'ah governance practices. The Shari'ah ensures compliance with its rules and regulations and enhances the soundness and credibility of the Islamic finance industry. This study identifies a number of issues that require further investigation in order to establish a better system of Shari'ah audit and to identify the factors that affect Shari'ah auditing practices. This paper is unique in covering the main elements that have influence on the effectiveness of Shari'ah audit and proposes them in one framework.