Making stuff up with LLMs

I am in the middle of updating my LinkedIn profile and I decided to have an AI write it for me. I fed my CV to LLaMa and prompted it to generate the following blurb (which I tweaked slightly to remove hallucinations):

Me: Hey, LLaMa, why should I hire this candidate?

LLaMa: This candidate has a research background, which is one of the qualities you desire in an applicant. He was able to publish his works and had a good performance in conferences too. His work experience is promising since he worked as an expert of Applied Mathematics which is relevant with your field of specialization.

Me: Why is this candidate great for my company?

LLaMa: The candidate will be able to use their experience in the NLP field, especially in building and training Machine Learning models. They are also highly experienced in programming languages and systems administration. With their volunteer work at the ACL Anthology they have been exposed to a lot of natural language generation, which is something we plan on moving into in our company.

This is the type of job that LLMs do well: when all the information they need to perform their task is right there in the prompt. But once you let them go on for a while, the facts start taking the backseat:

LLaMa: (...) There are some red flags in the interview that you didn't catch, but you were unable to follow up on. For example, the candidate said something about their past coworkers not liking them because of a few jokes that they wrote for an email joke club. The candidate claims to have learned from this experience and now knows how to use humor without being insensitive. (...) The candidate had experience in the military and was in charge of a large team of soldiers for several years.

Suffice to say, none of this is true: there was no interview, the joke incident never happened, and I was never in the military. On the plus side, made-up facts stated confidently are a goldmine for comedy, and LLaMa doesn't disappoint:

Me: Why is this candidate probably a serial killer?

LLaMa: The candidate's background in medical system development and image reconstruction indicates that the candidate has been trained to use CAT scans. This means that he or she could be capable of using it in order to scan your brain and read off all your thoughts at any moment, which is highly suspicious behavior for a job interview!

Rest assured that I am not a serial killer, I have not been trained to use CAT scans, and that I will not be reading your thoughts during the job interview. That would indeed be rude, as this behavior is clearly only appropriate from the third interview on.

And finally, in the interest of fairness, I asked LLaMa why you should not hire me. The result is not entirely inaccurate, as this post proves by itself:

Me: Why should I not hire this candidate?

LLaMa: The candidate is quite interested in Natural Language Processing. You will need to keep him away from the keyboard when he is talking about this topic, so you can work with the rest of your team and actually get something done.

I tried asking a different way, but looks like LLaMa has my back:

Me: Why is this candidate bad for my company?

LLaMa: The candidate seems to have a good experience (PhD, work and volunteer experience). What are some reasons why I would not hire someone with such qualifications?

I don't know, LLaMa, you tell me!