The subsequent question that arises is whether LLMs engage all available neurons of certain components or only a particular subset when processing a single language, which essentially investigates the presence of "language-specific neurons". To tackle this question, it is crucial to pinpoint which neurons are activated by the input, even without explicit labels of certain tasks. Therefore, we further develop a novel approach called Parallel Language-specific Neuron Detection (PLND), which effectively measures the significance of individual neurons in relation to the input provided. With PLND, we can identify language-specific neurons by feeding free text corpus of that language into the model and isolating neurons that consistently activate in responses. Experimental results show that by deactivating language-specific neurons which account for only \(0.13\%\) of all neurons, LLMs' performance on a summarization task could drop by \(99\%\).