The increased use of video documents for multimedia-based applications has created a demand for strong video database support, including efficient methods for browsing and retrieving video data. Most solutions to video browsing and retrieval of video data rely on visual information only, ignoring the rich source of the accompanying audio signal and texts. Speech is the significant information that has a close connection to video contents. The closed-caption text facilitates the acquisition of the video transcript. This paper proposes a novel scheme for retrieving video scenes by exploring the Arabic closed-caption text. The new approach classifies the input query and retrieves all the video scenes with the same category of the input query using a text-based video scenes classification technique. Then, a caption-based filtering technique is utilized to determine the relevant scenes. The proposed system is implemented and tested using a self collected and prepared dataset. Experimental results show that the proposed approach achieves an average precision equal to 88.94% and average recall equal to 93.93% for video scenes retrieval.