Background: Interpreting variant effects from human DNA is fundamental to precision medicine. While large Transformer-based genomic language models have delivered strong performance on coding regions, their quadratic complexity with respect to input length makes them inefficient for non-coding variant effect prediction, where relevant context may span thousands of base pairs. Evidence from large language models (LLMs) used in natural language processing suggests that layer-wise pruning can reduce model size and complexity while enhancing performance. Building on this, layer-wise pruning was applied systematically to two popular genomic LLMs - DNABERT-2 and the Nucleotide Transformer - to assess the role of individual layers and create more efficient models. Layer ablation revealed significant differences in the importance of individual layers, with the removal of some layers leading to marked performance loss, and others leading to performance improvement. Furthermore, after fine-tuning, pruned versions of the models achieved accuracy and AUROC comparable to the original models, while requiring substantially less training time and memory.