Proximal humeral fractures are common injuries, especially in youngsters and the elderly, and usually correspond to 5–6% of all fractures. There has been a general increase in upper extremity fractures in children. Detecting PHF is the time-consuming process of manual diagnosis by professionals using X-ray images. In this paper, a novel deep learning-based BON-VNET model is proposed for the detection of fracture in bones. Initially, the input CT image is pre-processed utilizing adaptive bilateral filter. Then, the pre-processed images are segmented using V-Net segmentation model. Afterward the segmented images are fed as an input the HOG based feature extraction phase for extracting the relevant features. Finally, the machine learning (ML) based ANFIS approach is employed for the classification of PHF images. From the trail result, the proposed BON-VNET model achieves 99.77% of accuracy rate, which is more than that of the conventional DL networks. Proposed BON-VNET model exhibits an overall accuracy improvement of 13.83%, 8.82%, and 16.95% compared to InceptionV3 Near classification and SVM, respectively.