With the rapid development of augmented reality (AR) technology, the combination of reality and virtual learning material finally came true. This study adopts AR technology to enrich learning environment and students' learning experience at the same time. Furthermore, to passively avoid students being lost while they were learning in a real environment and to actively channel students into ideal learning topic, this study proposes an indoor positioning algorithm which is able to calculate the current position of the learner and further enables learners to find out where he/she is and which learning subjects he/she will learn. The experiment is carried out to demonstrate the accuracy of the proposed indoor positioning algorithm. The results show that the proposed algorithm has a higher accuracy and lower error. Meanwhile, the algorithm is able to eliminate the anomalous RSSI signals, which is the main reason for improving the positioning accuracy. The maximum error is improved by 51.22%. From the application aspect, we elaborate the implications of this study which present the substantial contributions and educational values of the system. We expect that the smart learning can be made via the system establishment in near future.