Lung cancer is among the world's leading causes of death and its accurate and early diagnosis is critical for effective treatment planning. However, lung cancer often presents subtle early symptoms, making early detection challenging, and its diagnosis relies heavily on imaging techniques that can sometimes miss small or irregular nodules. To overcome these challenges a novel MLSIA-LC has been proposed deep learning-based Multi-Level Self and Inter-Attention (MLSIA) Network for the classification of lung cancer subtypes using CT and PET scan images. Then the segmentation of lung nodules using the Attention U-Net model. Advanced texture patterns, namely Directional Hexagonal Inter Mixed Pattern (DHIMP) and Directional Hexagonal Inter-Intra Mixed Pattern (DHIIMP), are extracted from segmented regions to capture unique spatial features from both CT and PET images. These patterns are fed into a parallel network architecture comprising three pathways, each with two levels of Multi-Level Self-Attention (MLSA) networks and Inter-Attention layers, followed by dropout mechanisms to enhance feature. The experimental results of MLSIA network achieves higher classification performance compared to existing methods, particularly with the DHIIMP texture pattern. The proposed model achieved an accuracy of 92.1% and an F1 score of 90%, outperform effectively leveraging inter- and intra-pattern information