Honors Week: Cadet Researches AI as a Hiring Manager
Artificial intelligence (AI) is everywhere. From health care to finance, transportation, customer service, and manufacturing. Many people rely on AI without really realizing it for their smartphones and smart home devices. But is AI as reliable and accurate as humans in all cases? Jude Roberts 鈥�25, a cadet at 麻豆女演员 Institute, sought to find the answer in his honors presentation, 鈥淎ssessing the Efficacy of an LLM for Resume Selection and Human Talent Management.鈥�
Roberts, a computer science major from Cloverdale, Virginia, was introduced to AI research when he interned at the U.S. Army Cyber Command at Fort Eisenhower, Georgia, last summer. For his honors project, he conducted an experiment looking at business hiring processes using the AI program, ChatGPT, specifically the GPT-4o mini, a cost-efficient small model. He explained that people generally believe that time efficiency and lower costs are a huge upside to automation. 鈥淚f a company doesn鈥檛 have to pay an employee to look through a lot of resum茅s, it frees their staff鈥檚 time to focus on more high-level strategic tasks. Sometimes ChatGPT and other large language models (LLMs)鈥攕oftware designed to generate human-like, conversational language鈥攑rovides completely nonsensical, out of context, or just wrong information, known as hallucinations, which is problematic.鈥� Roberts continued to point out that because there is no human interaction to evaluate a potential employee鈥檚 interpersonal and character traits, AI may show a bias based solely on particular words appearing on a resum茅. 鈥淪ome of us very soon will be applying for our first jobs and we want a fair shot at the hiring process. What I鈥檓 examining here is whether AI is consistently capable of selecting the best applicants for jobs,鈥� he explained.
Roberts took 1,000 resum茅s and instructed ChatGPT that it was taking on the role of a hiring manager, and it was to consider the applicants鈥� skills, experiences, and qualifications, and choose the applicant that best fills the job role. Categories included data science, arts, web design, and engineering. An additional category, not applicable (N/A), was designated for any resum茅 data that contained nonsensical or misleading information or that did not match any of the categories.
Roberts found that AI was less accurate and less reliable than a human hiring manager in finding the best candidate, and he determined that the LLM was vulnerable to malicious interference and bias propagation. 鈥淭hese are very concerning aspects for the reliability of these models, and administrators should be concerned about these issues when implementing this technology. Hiring managers should never solely rely on the technology to automate any portion of resum茅 screening.鈥�
Sherif Abdelhamid, Ph.D., assistant professor in the聽Department of Computer and Information Sciences聽and outreach liaison officer for the Cyber Defense Laboratory, served as Roberts鈥� advisor on the project. Abdelhamid noted Roberts鈥� exceptional skills of problem solving, ability to work in a team, and analytical skills. 鈥淚 see him as a unique, special student. He was able to independently identify the research problem and objectives, and came up with a unique design for his research. He carefully and critically studied how LLMs can impact our lives, and identified their benefits and drawbacks,鈥� said Abdelhamid.
Roberts, who is minoring in applied mathematics and philosophy, is a graduate of Lord Botetourt High School in Daleville. He is the son of Kevin and Lori Roberts. Following graduation, he will commission into the U.S. Space Force where he hopes to do work in cybersecurity or electromagnetic warfare.
