The most fundamental reason is real volume of knowledge of an interviewer. The problem is that only few can test outside their narrow field of expertise (in other words, test wider than they are actually proficient in). Unconsciousness about actual limits of their knowledge, and misunderstanding about these limits within a team — all these outlines the sky line of the recruitment process.
Narrow vision of overall responsibilities is the next reason to fail. ~75% of engineers focus on their specific niche technology and consider it as a core of their knowledge, meanwhile standing opposite to the necessity/influence of background knowledge and skills. Obviously, 90% of their interview questions are focused on this specific niche. And this way the team might maturate merely within this specific niche of knowledge... and in such a narrow vision though.
Almost every interviewer is afraid to fail. People often treat an interviewing process as a battle of COMPETEnce. This is a good motivation for testing the things they are confident in. Well, that's good, but what about walking the ways they know not so perfectly well? This carbonizes the weaknesses of the team.
What about merely the number of questions you ask a candidate during a standard one-hour interview? The common practice says that it rarely exceeds 30 and usually comes with all administrative and common questions like `how do you do` or `what do you think about the weather today`. Notably, these 30 questions give you a foundation to make a hiring decision.
Every professional direction has nearly 15 areas of knowledge a candidate should be acquainted with. Every such area has 3-5 sub-directions usually, that should be at least superficially tested. Beyond professional knowledge, we should also remember about so-called `soft-skills`, domain knowledge, experience story, introductory and general management questions, and so on.
Without a well-elaborated structure of interviews (and questionaries), you can’t even pretend to be objective in comparing candidates. Knowledge is an infinitely multidimensional area and inspecting it you are always close to comparing red with long. Thus your hiring process will be Brownian and will never be enhanced.
Almost every interviewer confuses their general impression with an actual level of knowledge and skills of a candidate. Merely some people can translate their intuition to clear reasoning, and less yet can explain them to others. This problem deepens by the absence of retrospectives and detailed feedbacks culture.
Few people understand that they have to sell the company to a candidate. Yet fewer understand that the way they interview a candidate outlines 90% of the time the one interacts with the company (this hour or two will strongly infuence their final decision). Yet less can sell and arrange an interesting and engaging communication in their area of expertise.
Recruitment is recognized as one of the most stressful jobs in a modern business environment and only a few like doing this part of their job. Negative attitude worsens the atmosphere of interviews and slows down the recruitment process by continuous delays in every communication with candidates.
Absence of soft-skills structure and understanding of what they actually are. This forces interviewers to wonder permanently what people company really needs. This way they become unconsciously taking themselves as a face-mold and allow themselves just up and like someone or not. Therefore, there is no neither rhyme nor reason and thus we get a skill-less team.
Uncontrollable segregation of duties and lack of understanding about the entire process, areas of responsibilities, and strengths/weaknesses of everybody concerned. It so happened that most employees expect to be managed, that someone with a higher position controls everything and knows what to do. Meanwhile, a manager usually thinks an HR specialist manages all the stuff. And an HR specialist (rarely understands knowledge area or real responsibilities on a position opened) thinks everything is under control of an interviewing expert. This is a broken communication loop, and thus a vicious circle of buck-passing and recruitment process degradation.
And the last thing to get is that a manager’s assessment capabilities are limited merely to other people feedback gathering. This way blank spots of an interviewer become blank spots of the entire company.
P.S. You might also like to read about the Johari Window Model — a simple analytical tool that they commonly use for blind-spots uncovering.