<story><title>Testing how hard it is to cheat with ChatGPT in interviews</title><url>https://interviewing.io/blog/how-hard-is-it-to-cheat-with-chatgpt-in-technical-interviews</url></story><parent_chain><item><author>notfed</author><text>LLMs are too powerful. It&amp;#x27;s basically like having someone next to you telling you what to say in the interview, if not &lt;i&gt;better&lt;/i&gt;.&lt;p&gt;The point of (software engineering) interviews is to demonstrate &lt;i&gt;how&lt;/i&gt; you solve problems. &amp;quot;Type the question into ChatGPT and do what it says&amp;quot; is &lt;i&gt;not&lt;/i&gt; the &amp;quot;how&amp;quot; that companies are looking for.</text></item><item><author>babyshake</author><text>IMO asking people to not use available tools in interviews is a bad idea, unless you are trying to do a very basic check that someone knows the fundamentals.&lt;p&gt;Allow them to use the tools, with a screenshare, and adjust the types of tasks you are giving them so that they won&amp;#x27;t be able to just feed the question to the LLM to give them the completed answer.&lt;p&gt;Interviews should be consistent with what day to day work actually looks like, which today means constantly using LLMs in some form or another.</text></item></parent_chain><comment><author>GuB-42</author><text>Unless you can&amp;#x27;t use ChatGPT for some reason (ex: security), I think it is a totally valid way of solving problems.&lt;p&gt;The interesting part is what to do when ChatGPT does it wrong, and if the problem is not trivial and an exact solution is not available online, it is usually wrong. Sometimes, not by much, but it takes skill to notice the problem and fix it, either manually or by asking ChatGPT for it.&lt;p&gt;Same idea as for libraries. One could argue that those using the &amp;quot;sort&amp;quot; function don&amp;#x27;t know how to write a sorting algorithm, but in real life, unless there are particularly good reasons, rewriting a &amp;quot;sort&amp;quot; function would be crazy, and probably not what you want from your employees.&lt;p&gt;If you want your candidate not to use the tools at his disposal, you may frame it as a requirement. &amp;quot;on this test, imagine you are working on a sensitive project, you are not allowed to upload any details to a third party service, you can search the internet for generic information, but do as if your development machine has no internet access, so no copy-pasting&amp;quot;. Or, for libraries &amp;quot;on this test, imagine we absolutely want to limit dependencies on third party libraries, even if it means sometimes reinventing the wheel, so only &amp;lt;list of libraries&amp;gt; are allowed&amp;quot;.</text></comment>
<story><title>Testing how hard it is to cheat with ChatGPT in interviews</title><url>https://interviewing.io/blog/how-hard-is-it-to-cheat-with-chatgpt-in-technical-interviews</url></story><parent_chain><item><author>notfed</author><text>LLMs are too powerful. It&amp;#x27;s basically like having someone next to you telling you what to say in the interview, if not &lt;i&gt;better&lt;/i&gt;.&lt;p&gt;The point of (software engineering) interviews is to demonstrate &lt;i&gt;how&lt;/i&gt; you solve problems. &amp;quot;Type the question into ChatGPT and do what it says&amp;quot; is &lt;i&gt;not&lt;/i&gt; the &amp;quot;how&amp;quot; that companies are looking for.</text></item><item><author>babyshake</author><text>IMO asking people to not use available tools in interviews is a bad idea, unless you are trying to do a very basic check that someone knows the fundamentals.&lt;p&gt;Allow them to use the tools, with a screenshare, and adjust the types of tasks you are giving them so that they won&amp;#x27;t be able to just feed the question to the LLM to give them the completed answer.&lt;p&gt;Interviews should be consistent with what day to day work actually looks like, which today means constantly using LLMs in some form or another.</text></item></parent_chain><comment><author>blitzar</author><text>&amp;gt; &amp;quot;Type the question into ChatGPT and do what it says&amp;quot; is not the &amp;quot;how&amp;quot; that companies are looking for.&lt;p&gt;Then interviewers should stop setting tasks that require either a) copy and paste answer from leetcode or b) copy and paste answer from chatgpt.&lt;p&gt;Unfortunately that requires skill and awareness on behalf of the interviewers; who typically served in the leetcode wars and want their employees to go through the same.</text></comment>