Construction robots can efficiently carry out many decoration processes in indoor and outdoor environments. During the wall decoration process, wall bulges of different lengths and orientations appear within the working area. To successfully accomplish the wall putty bulge repair process, the construction robot needs to identify the bulge position within the working space. We proposed a deep learning model for regressing the wall putty bulge terminal points position in spatial coordinates. In this study, a hybrid visual attention mechanism that captures both channel and spatial discriminative features is proposed. Attention mechanisms boost the model's feature learning capability by answering questions about what and where to pay attention. The proposed hybrid visual attention module was integrated into the EfficientNet, a cutting-edge mobile deep convolutional neural network (DCNN). Experimental results demonstrated that the EfficientNet based on the proposed hybrid attention module outperformed the plain EfficientNet, which is based solely on channel attention.