Blog Post 6: A Mini “Critical Review”

Introduction

The article “Technical interview performance is kind of arbitrary. Here’s the data.” provided extensive insight into the issues with today’s job evaluation methods. The methodology presented in the article is straightforward; it involves gauging the applicant’s technical abilities on a specific platform known as interviewing.io, which allows the participant to attempt different technical interviews. The results are then graphed to view the standard deviation between technical interviews. The author highlights that looking at such data is pivotal, as companies do not want to accept under-qualified applicants and further explained that due to extensive resources, companies have cheaper hiring. This in turn guarantees a few good applicants with the major drawback being that the damage from the subpar applicants can be irreversible to the company. The author of the article proceeds to analyze the probability of failing an interview based on the applicant’s technical score. This is done via dividing applicants based on their scores, then within the cohorts determining the likelihood of failing. After repeating the process 10,000 times the data is then graphed and presented.

Critiques

The author’s methodology was well thought out and allowed for insightful conclusions; that being said, it was not without flaws.

First Critique

The first issue in the methodology is that different companies lack a standardized method of interviewing. This is inherent when based on specific interviewers and companies. The volatility can be chalked up to high expectations rather than the interview itself having an issue. To elaborate, if an individual was to take an interview at a startup, the difficulty would be different than interviewing for Apple. The expectation also differs, as a 3/4 on a technical interview for a startup could translate to a 2/4 for a large corporation such as Apple. This is usually due to a high number of applicants and overall higher expectations. The conclusions would be impacted as this would make the standard deviation seem higher but the causality is apparent only upon further inspection.

Second Critique

The second issue present within the methodology is that since the author(s) did not attempt to establish a control group and rather just made observations, this would qualify as an observational study. The meaning behind that is that the writers can not make any conclusions as there is no effort to design an experiment to test the hypothesis. For it to be an experimental study, the author would need to make a group and individually test the skills in the group; then proceed to make them take part in technical interviews on the platform and contrast the data. Only then would the author have designed an experiment and as such can draw conclusions such as “interviewing is doomed”.

Conclusion

In conclusion, the author collected data to draw insightful conclusions but would benefit from designing an experiment to confirm his hypothesis as well as analyze the data for possible causality. The article gave the audience a good amount of information regarding the issues with modern job evaluation methods and teaches them to be mindful in the future when applying for a job.