Regulatory authorities have limited the voting rights of controlling shareholders in the appointment and dismissal of audit committee members to increase audit committee independence involved in the determination of audit fees. However, previous studies have overlooked whether such restrictions on voting rights of controlling shareholders have a positive effect on audit fees. Therefore, it was necessary to check whether the positive influence of the audit committee on audit quality was diluted by the excess influence of the controlling shareholders. The purpose of this study is to verify the effect of an audit committee on audit fees, and then to examine how this relationship is changed by the excess influence of controlling shareholders. The results of this study are as follows. First, a compulsory audit committee and a voluntary audit committee are positively associated with audit fees. These results indicate that the audit committee prefers high-quality audit services than a full-time auditor. Second, the influence over controlling shareholders (limited voting rights) is negatively associated with audit fees. This result means that even if a certain amount of voting rights is restricted in the controlling shareholder's equity ratio, the controlling shareholder can still exercise the influence of reducing audit fees. Third, the positive relationship between the audit committee and audit fees was changed to a negative relationship due to the influence over the controlling shareholder (voting rights limit, management ownership). These results indicate that the positive effect of the audit committee on audit quality can weaken due to the excess influence of controlling shareholders. This study suggests that various institutional improvements are needed to check the excessive influence of controlling shareholders on audit quality despite the restrictions on voting rights.