Effectively identifying an airport from satellite and aerial imagery is a challenging task. Traditional methods mainly focus on the use of multiple features for the detection of runways and some also adapt knowledge of airports, but the results are unsatisfactory and the usage limited. A new method is proposed to recognize airports from high-resolution optical images. This method involves the analysis of the saliency distribution and the use of fuzzy rule-based classification. First, a number of images with and without airports are segmented into multiple scales to obtain a saliency distribution map that best highlights the saliency distinction between airports and other objects. Then, on the basis of the segmentation result and the structural information of airports, we analyze the segmentation result to extract and represent the semantic information of each image via the bag-of-visual-words (BOVW) model. The image correlation degree is combined with the BOVW model and fractal dimension calculation to make a more complete description of the airports and to carry out preliminary classification. Finally, the support vector machine (SVM) is adopted for detailed classification to classify the remaining imagery. The experiment shows that the proposed method achieves a precision of 89.47% and a recall of 90.67% and performs better than other state of the art methods on precision and recall.