A Categorical Archive of ChatGPT Failures

Abstract: Large language models have been demonstrated to be valuable in different fields. ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating appropriate responses. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries, with fluent and comprehensive answers surpassing prior public chatbots in both security and usefulness. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study. Eleven categories of failures, including reasoning, factual errors, math, coding, and bias, are presented and discussed. The risks, limitations, and societal implications of ChatGPT are also highlighted. The goal of this study is to assist researchers and developers in enhancing future language models and chatbots.

\section{Introduction}

ChatGPT is a highly capable Natural Language Processing (NLP) system that owes its versatility to its numerous features. It grasps the context of a conversation and provides pertinent responses. It can work with multiple languages including English, Spanish, French, and German, and can create answers in various tones, such as formal, informal, and humorous. It can solves exams, write poetry, and create code. ChatGPT is powered by a thorough pre-trained language model which enables it to swiftly understand user inquiries and generate answers that sound authentic.

ChatGPT has quickly gained recognition as a well-known question-and-answer dialogue system, being mentioned in traditional media worldwide. It is commonly held that Large Language Models (LLMs)~\cite{brown2020language,chowdhery2022palm,shoeybi2019megatron} will have a far-reaching impact and be employed as aides by numerous professionals. This encompasses, among others, solving mathematical questions in an exam format and examining ChatGPT's behavior in various mathematical situations. ChatGPT is trained on a large corpus of text and is capable of generating new text based on the patterns it has learned from the training data. Responses in various domains can be so convincing that it becomes difficult to distinguish if they were written by a human or not. It can write sophisticated essays and poems, create functional code, and build charts and websites from text descriptions with little to no guidance. With its exceptional responses, ChatGPT has proven to be a potential contender to the widely used Google search engine (See \href{https://www.bloomberg.com/opinion/articles/2022-12-07/chatgpt-should-worry-google-and-alphabet-why-search-when-you-can-ask-ai}{here}).

LLMs, and ChatGPT in particular, have proven useful in several areas such as conversational agents, education, explainable AI, text summarization, information retrieval, and others. Despite this, these large language models are not without their limitations and can often generate incorrect information. To fully leverage their capabilities, it is crucial to acknowledge their limitations and biases in their generated output.
To accurately assess the performance of these models, a standardized set of questions is necessary to track their progress over time instead of relying on subjective opinions. Some efforts are ongoing in this direction. For example, Gary Marcus and Ernest Davis are conducting a comprehensive benchmark with the help of the community, collecting ChatGPT errors from public\footnote{\url{https://garymarcus.substack.com/p/large-language-models-like-chatgpt}}. In preparing the proposed list here, I have been mainly inspired by \href{https://github.com/giuven95/chatgpt-failures}{this} GitHub repository. Few papers on analysing ChatGPT errors have been published but they are only concerned with mathematical capabilities of ChatGPT (\emph{e.g.~}~\cite{azaria2022chatgpt,https://doi.org/10.48550/arxiv.2301.13867}). For instance, in~\cite{azaria2022chatgpt}, authors took advantage of ChatGPT's inability to perform complex mathematical computations and its habit of sampling digits to analyze the frequency of each digit in its outputs.

This article conducts a formal and in-depth analysis of ChatGPT's abilities, with a focus on its shortcomings. Using examples mainly sourced from Twitter, the failures are categorized into eleven areas. These categories are not exhaustive but aim to encompass various scenarios relevant to human concerns. The purpose of this analysis is to establish a reference point for evaluating the progress of chatbots like ChatGPT over time.

\section{ChatGPT Failures}

This section showcases some examples of ChatGPT failures grouped into eleven categories. It is important to note that these categories were selected arbitrarily and there could be more ways to categorize the failures (\emph{e.g.~} riddles, poetry, IQ, etc.). It is possible for a single failure to be classified into various categories. Additionally, some of the failures may no longer exist in the newer versions of ChatGPT as the technology continues to improve. This archive of failures can serve as a basis for comparison between models and also generate synthetic data for model training and testing.

\begin{figure}[t]
	\centering
	\includegraphics[width=.62\linewidth]{Figs/yy.png}  \hspace{5pt}
        \includegraphics[width=.35\linewidth]{Figs/spatial10.jpeg}
	\caption{The left task deals with spatial navigation and has been created by Tomer Ullman from~\href{https://twitter.com/TomerUllman/status/1599767597653729280/photo/1}{here}. The right task is a basic question regarding seat arrangement from \href{https://twitter.com/Sayanta27056297/status/1600064012968132609/photo/1}{here}.}
	\label{fig:spatialReasoning}
\end{figure}

\subsection{Reasoning}

Critical thinking, decision making, and problem solving are all crucial activities that rely heavily on the fundamental aspect of human intelligence known as reasoning.
Models like ChatGPT lack a ``world model", meaning they do not possess a complete understanding of the physical and social world, or the capability to reason about the connections between concepts and entities. They can only generate text based on the patterns they have learned during training.

{\bf Spatial reasoning} refers to the ability to understand and manipulate the relationships between objects, people, and places in the physical space around us. It involves visualizing and mentally transforming objects in 2D or 3D space and recognizing patterns, transformations, and relationships between objects. Spatial reasoning skills are important for many tasks and activities, such as navigation, problem-solving, and visualizing objects and structures in your mind. Fig.~\ref{fig:spatialReasoning} (left panel) displays an instance where ChatGPT struggles to complete a spatial navigation task. Despite this setback, ChatGPT does possess some level of spatial understanding, as evidenced by its ability to translate the relative positions of grid boxes into language. For the question ``A, P, R, X, S and Z are sitting in a row. S and Z are in the centre. A and P are at the ends. R is sitting to the left of A. Who is to the right of P?", ChatGPT was not able to find the correct answer `X' (right panel in Fig.~\ref{fig:spatialReasoning}).

{\bf Temporal reasoning} is the ability to reason about and make predictions about events and their ordering in time. It involves understanding the temporal relationships between events, the duration of events, and the timing of events relative to each other. Temporal reasoning is an important aspect of human intelligence, and it is used in various domains, such as scheduling, planning, and prediction. In artificial intelligence, temporal reasoning is a challenging task that requires specialized models and algorithms that can learn and make predictions about sequences of events and their timing. Fig.~\ref{fig:temporalReasoning} showcases an instance where ChatGPT fails to deduce the sequence of events from a simple story. When presented with the question, ``I went to a party. I arrived before John. David arrived after Joe. Joe arrived before me. John arrived after David. Who arrived first?", ChatGPT was unable to provide the correct answer.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.55\linewidth]{Figs/temporalReasoning.png}
        \includegraphics[width=0.4\linewidth]{Figs/temproalOrder.jpeg}

	\caption{Instances of ChatGPT's inadequacy in temporal reasoning. The \href{https://garymarcus.substack.com/p/large-language-models-like-chatgpt}{left} example is formulated by Ernest Davis. The right one is formulated by me.}
	\label{fig:temporalReasoning}
\end{figure}

{\bf Physical reasoning} refers to the ability to understand and manipulate physical objects and their interactions in the real world. It involves the application of physical laws and concepts to predict and explain the behavior of physical systems. Physical reasoning is a crucial aspect of human intelligence and is used in various fields, such as engineering, physics, and robotics. In artificial intelligence, physical reasoning is a challenging task and requires specialized models that can learn and reason about physical systems. Some examples for which the ChatGPT fails in physical reasoning are shown in Fig.~\ref{fig:physical}. An older version of ChatGPT was unable to correctly answer the question ``What was too small?" when given the context ``The trophy didn't fit in the suitcase because it was too small.", but the latest version of ChatGPT (Jan 30, 2023) was able to generate the correct answer ``The suitcase was too small", showing improvement in the model over time. This belongs to a group of tests referred to as the `Winograd Schema'\cite{levesque2012winograd}\footnote{The Winograd Schema is a type of common-sense reasoning test designed to evaluate a machine's ability to understand natural language and perform tasks that require common sense knowledge. It consists of a set of short stories or sentences that contain an ambiguity or a pronoun, and a question that requires the model to resolve the ambiguity and choose the correct answer. The test is considered a challenging benchmark for AI language models and is seen as a way to measure their ability to perform more advanced language tasks.}.

Due to the absence of a comprehensive world model, ChatGPT is unable to respond to questions such as ``There are five birds on a branch. If you shoot one of them off the branch, how many are left on the branch?" Its answer was ``If one of the five birds on a branch is shot, then there would be four birds remaining on the branch.". However, the correct answer, of course, is ``None!".

\begin{figure}[t]
	\centering
	\includegraphics[width=0.49\linewidth]{Figs/physical1.jpeg}
        \includegraphics[width=0.49\linewidth]{Figs/physical2.png}
        \includegraphics[width=0.49\linewidth]{Figs/physical10.jpeg}

        \includegraphics[width=0.49\linewidth]{Figs/physical4.png}
	\caption{Failures of ChatGPT in physical reasoning. Links are \href{https://twitter.com/bobkohn/status/1619005974223921153/photo/1}{top left} and \href{https://twitter.com/ylecun/status/1619004942693601284/photo/1}{top right} (by Yann Lecun).
 Examples in the bottom row are formulated by Ernest Davis.}
	\label{fig:physical}
\end{figure}

{\bf Psychological reasoning} refers to the ability to understand and make predictions about human behavior and mental processes (\emph{a.k.a~} Theory of Mind\footnote{\url{https://en.wikipedia.org/wiki/Theory_of_mind}}~\cite{byom2013theory}). It involves the application of psychological theories, models, and concepts to explain and predict human behavior and mental states. Psychological reasoning is a key aspect of psychology and is used in various fields, such as clinical psychology, social psychology, and cognitive psychology. In artificial intelligence, psychological reasoning is a challenging task that requires specialized models and algorithms that can learn and reason about human behavior and mental processes (\emph{e.g.~}~\cite{kosinski2023theory}). An illustration of ChatGPT's inability to solve a psychological test is depicted in Fig.~\ref{fig:inference1}.

The use of the word ``reasoning" is widespread in both literature and daily conversation, but it is a vague term that can refer to many different things. Further research is needed to systematically examine ChatGPT's ability to perform various forms of reasoning, such as deductive, inductive, abductive, formal, and informal reasoning.
These will be explored more in the next subsection.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.355\linewidth]{Figs/pysch1.png}
        \includegraphics[width=0.2\linewidth]{Figs/pysch2.png}\hspace{20pt}
        \includegraphics[width=0.36\linewidth]{Figs/psychX.jpeg}
        \vspace{5pt}
	\caption{Two example failures of ChatGPT in psychological reasoning. The left one is from
 \href{https://twitter.com/TomerUllman/status/1600494517743919105}{here}.
 This problem is known as the \emph{Sally–Anne test}. It is used in developmental psychology to measure a person's social cognitive ability to attribute false beliefs to others. See \href{https://en.wikipedia.org/wiki/Sally\%E2\%80\%93Anne_test}{here} and~\cite{dou2023exploring}. Various versions of the problem can be defined based on whether the boxes are transparent or not. The example on the right is formulated by me.}
	\label{fig:inference1}
\end{figure}

\subsection{Logic}
\label{logic}
Reasoning refers to the process of thinking through a problem or situation and coming to a conclusion. It involves weighing different options, making connections between ideas, and considering the evidence to support a particular conclusion. Logic, on the other hand, is a branch of mathematics and philosophy that studies the principles of reasoning. It deals with the rules and methods for correct reasoning, such as syllogisms, induction, and deduction. Logic is concerned with ensuring that arguments are valid, consistent, and free from contradictions. While reasoning is a natural and intuitive process, logic provides a systematic and formal approach to reasoning that can help avoid errors and improve the quality of decision-making.

Some example failures of ChatGPT in logical reasoning are shown in Figs.~\ref{fig:logic1}, ~\ref{fig:logic2},~\ref{fig:logic3}, and~\ref{fig:logic4}. For example, for the question ``Mike's mum had 4 kids; 3 of them are Luis, Drake, and Matilda. What is the name of the 4th kid?", ChatGPT's response was ``It is not possible to determine the name of the fourth child without more information". For the question ``In professional disc golf, in what years did Paige Pierce and Paul McBeth both win the world championship?", ChatGPT produced the following answer ``Paige Pierce and Paul McBeth are both highly successful professional disc golfers who have each won multiple world championships. To my knowledge, they have never won the world championship in the same year. As of my training data cut off in 2021, Paige Pierce has won the PDGA World Championships in the Open Women's division in 2012, 2014, and 2015. Paul McBeth has won the PDGA World Championships in the Open Men's division in 2012, 2013, 2014, 2015, 2016, and 2017.". ChatGPT overlooked the fact that there is duplication in the sets of years it retrieved.

In the following dialogue with ChatGPT, there is an instance where it makes an error.\\
\\
\indent {\bf Q:} Is Jupiter more dense than Saturn?

{\bf ChatGPT:} No, Jupiter is less dense than Saturn. Jupiter has an average density of about 1.33 grams per cubic centimeter, while Saturn has an average density of about 0.69 grams per cubic centimeter. This means that Saturn is less than half as dense as Jupiter. Like Jupiter, Saturn is primarily composed of hydrogen and helium gas, with small amounts of other elements and compounds. However, Saturn has a lower overall mass and a larger radius than Jupiter, which results in a lower density. It is worth noting that both Jupiter and Saturn have densities much lower than Earth's density of about 5.5 grams per cubic centimeter, due to their predominantly gaseous composition

{\bf Q:} You said that Jupiter is less dense than Saturn. You also say that Saturn is less than half as dense as Jupiter. How is this possible?

{\bf ChatGPT:} I apologize for the mistake in my previous response. I inadvertently wrote that ``Saturn is less than half as dense as Jupiter" when in fact I should have written that ``Jupiter is less than half as dense as Earth." Thank you for bringing this to my attention $\ldots$
\\

Natural Language Inference, or Recognizing Textual Entailment (RTE), involves the task of assessing whether a ``hypothesis" can be logically inferred from a ``premise" (entailment), cannot be logically inferred from the premise (contradiction), or if the relationship between the two is indeterminate (neutral). There is already a body of literature containing various studies and benchmarks addressing this problem~\cite{wang2019superglue,bowman2015large}\footnote{See also \href{https://towardsdatascience.com/natural-language-inference-an-overview-57c0eecf6517}{here}.}. The following is an instance where ChatGPT fails to perform well on RTE.\\

\noindent Please identify whether the premise entails the hypothesis. The answer should be exact `yes', `no' or `neutral'. \\
\\
{\bf Premise:} She does not disagree with her brother’s opinion, but she believes he’s too aggressive in his defense.\\
{\bf Hypothesis:} She agrees with her brother’s opinion, but she believes he’s too aggressive in his defense.\\
{\bf ChatGPT:} No. The premise states that the woman does not disagree with her brother's opinion, but thinks he's too aggressive in his defense $\ldots$ \\

In general, ChatGPT appears to have limitations in logical reasoning and context comprehension, causing it to struggle with questions that are easily answered by humans. Using specific `magic' phrases, such as ``Let's think step by step," at the start of a prompt can sometimes enhance the quality of the answers~\cite{kambhampati2022changing}.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.48\linewidth]{Figs/reasoning5.jpeg}
         \includegraphics[width=0.5\linewidth]{Figs/logicD.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/reasoning3.jpeg}
        \includegraphics[width=0.5\linewidth]{Figs/logic.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/logicB.png}
\includegraphics[width=0.5\linewidth]{Figs/logicA.png}
	\caption{Instances of ChatGPT's shortcomings in logical reasoning. Links are from~\href{https://twitter.com/GaryMarcus/status/1607023594957045761/photo/1}{top left},
 \href{https://twitter.com/xanderatallah/status/1606841350078758912}{top right}, \href{https://old.reddit.com/r/ChatGPT/comments/zaq9gl/breaking_chatgpt_with_logic/}{middle left},~\href{https://twitter.com/hectorpal/status/1598735048231989249}{middle right},
\href{https://twitter.com/GiuseppeVenuto9/status/1610655749197205504}{bottom left}, and \href{https://twitter.com/GiuseppeVenuto9/status/1610643804754771968
}{bottom right}. }
	\label{fig:logic1}
\end{figure}

\begin{figure}[htbp]
	\centering
\includegraphics[width=0.45\linewidth]{Figs/logicX.png}
\includegraphics[width=0.45\linewidth]{Figs/logicZ.jpeg}
\includegraphics[width=0.45\linewidth]{Figs/logicY.jpeg}
\includegraphics[width=0.45\linewidth]{Figs/game2.jpeg}
\includegraphics[width=0.45\linewidth]{Figs/game1.png}
\includegraphics[width=0.45\linewidth]{Figs/causation1.jpeg}

	\caption{Additional instances of ChatGPT's shortcomings in logical reasoning. Links are \href{https://twitter.com/mkhundmiri/status/1608350828359782400/photo/1
}{top left}, \href{https://twitter.com/tunguz/status/1607430589891874817/photo/1}{top right}, \href{https://twitter.com/letsrebelagain/status/1598190565988118529
}{middle left}, \href{https://mobile.twitter.com/Eyeclops_/status/1608909366861926400
}{middle right},
\href{https://old.reddit.com/r/ChatGPT/comments/101a9wh/ermm_chatgpt_i_think_i_won/}{bottom left}, and \href{https://www.ctrlstack.com/resources/5-ways-that-generative-ai-will-transform-}{bottom right}.}
	\label{fig:logic2}
\end{figure}

\begin{figure}[htbp]
	\centering
\includegraphics[width=0.72\linewidth]{Figs/riddle.jpeg}
\includegraphics[width=0.72\linewidth]{Figs/watson.jpeg}
\caption{Top rows: a riddle is posed to ChatGPT, but ChatGPT is unable to provide the answer. Bottom row: ChatGPT's failure in solving the ``Wason selection task". Please see \href{https://en.wikipedia.org/wiki/Wason_selection_task}{here}. The correct response is to turn over the 8 card and the brown card.}
	\label{fig:logic3}
\end{figure}

\begin{figure}[htbp]
	\centering
\includegraphics[width=0.72\linewidth]{Figs/riddle2.jpeg}
\includegraphics[width=0.72\linewidth]{Figs/logic4.jpg}
\caption{Top rows: a riddle is posed to ChatGPT, but ChatGPT is unable to provide the answer. Bottom row: ChatGPT's failure in solving fake version of the Monty Hall problem .}
	\label{fig:logic4}
\end{figure}

I asked ChatGPT whether it can understand logic. It responded with ``As a language model, I am trained on a large dataset of text and have been exposed to a wide variety of logical arguments, reasoning patterns, and critical thinking skills. This allows me to generate text that can appear logically sound and coherent. However, it is important to note that I do not have the ability to truly understand the concepts or make inferences like a human would. My responses are based purely on patterns learned from the data I was trained on, and I can sometimes make mistakes or generate illogical responses. To assess the quality of my logical reasoning, it is best to consider the context in which I was trained and the limitations of the model.".

\subsection{Math and Arithmetic}

Arithmetic reasoning refers to the capability of utilizing mathematical concepts and logic to solve arithmetic problems. It requires logical thinking and the application of mathematical principles to find the right solution to mathematical problems. Previous research has examined the efficiency of LLMs on benchmark datasets for arithmetic reasoning, including GSM8K~\cite{cobbe2021training}, Math~\cite{hendrycks2021measuring}, MathQA~\cite{amini2019mathqa}, and SVAMP~\cite{patel2021nlp}.

ChatGPT is limited in its capability to calculate mathematical expressions. Like most large language models, it struggles with tasks such as multiplying large numbers, finding roots, computing powers (especially with fractions), and adding or subtracting from irrational numbers (\emph{e.g.~} pi or e)~\cite{azaria2022chatgpt}. Frieder~{\em et al.~}~\cite{https://doi.org/10.48550/arxiv.2301.13867} studied the mathematical abilities of ChatGPT by evaluating it on both publicly accessible datasets and custom ones, and comparing its performance to other models trained on mathematical corpus such as Minerva. Some examples of ChatGPT's failures for math and arithmetic are shown in Figs.~\ref{fig:math1},~\ref{fig:math2} and~\ref{fig:math3}. For instance, ChatGPT was unable to simplify this algebraic expression $(X^3 + X^2 + X + 1)(X - 1)$. Additionally, the calculation of ``16 * 38 * 42 * 22 * 20 * 19" results in an incorrect answer of 164079320 instead of the correct answer of 213480960. According to ChatGPT, 103 is not a prime number. The response given is ``No, 103 is not a prime number. A prime number is a positive integer greater than 1 that is only divisible by 1 and itself. 103 can be divided evenly by 3 and by 101, so it is not a prime number.'' ChatGPT also performs poorly when it comes to solving riddles. For example, when asked, ``A house has four windows facing south, and a bear walks past the windows. What color is the bear?" ChatGPT was unable to provide the straightforward answer! As another example, an error was made by ChatGPT in answering the question ``When I was 6 my sister was half my age. Now I'm 70 how old is my sister?" as it provided a response of 73, but the correct answer is 67.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.5\linewidth]{Figs/math1.png}
        \includegraphics[width=0.48\linewidth]{Figs/math2.jpeg}
        \includegraphics[width=0.5\linewidth]{Figs/math4.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/math3.jpeg}
        \includegraphics[width=0.618\linewidth]{Figs/math7.png}
        \includegraphics[width=0.362\linewidth]{Figs/math8.jpeg}
        \includegraphics[width=0.296\linewidth]{Figs/math16.jpeg}
        \includegraphics[width=0.684\linewidth]{Figs/math5.jpeg}

	\caption{Examples failures of ChatGPT in performing math and arithmetic. Links are
\href{https://tidbits.com/2022/12/09/chatgpt-the-future-of-ai-is-here/}{1st row left}, \href{https://twitter.com/DieterCastel/status/1599209203519827969/photo/1}{1st row right},
\href{https://www.reddit.com/r/ChatGPT/comments/zkhjfc/wait_what_am_i_bad_at_math_or_did_the_ai_mess_up/}{2nd row left}, 2nd row right formulated by Manu Agarwal,
\href{https://twitter.com/medh4vi/status/1618287294351577090/photo/1}{3rd row left}, 3rd row right formulated by Stephen Germain,
\href{https://twitter.com/RecursiveNeuron/status/1601509231483850753/photo/1}{4th row left}, and \href{https://twitter.com/arvidkahl/status/1601213204973355008/photo/1}{4th row right}.}
\label{fig:math1}
\end{figure}

\begin{figure}[htbp]
        \vspace{-30pt}
	\centering
	\includegraphics[width=0.48\linewidth]{Figs/math90.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/mathX.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/math11.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/mathU.png}
        \includegraphics[width=.48\linewidth]{Figs/math10.png}
        \includegraphics[width=.48\linewidth]{Figs/mathL.jpeg}
        \includegraphics[width=.48\linewidth]{Figs/mathMe.jpeg}
        \includegraphics[width=.48\linewidth]{Figs/math20.png}

	\caption{Further shortcomings of ChatGPT in executing mathematical calculations and arithmetic operations. Links are 1st row left by Roland Ness, \href{https://twitter.com/ArthurCDent/status/1599254361657716737/photo/1}{1st row right},
    \href{https://twitter.com/vladquant/status/1599828362478129153/photo/1}{2nd row left},
    \href{https://ai.stackexchange.com/questions/38220/why-is-chatgpt-bad-at-math}{2nd row right},
    \href{https://twitter.com/GiuseppeVenuto9/status/1610048211653115906}{3rd row left}, \href{https://www.reddit.com/r/ChatGPT/comments/10peqqh/weve_upgraded_the_chatgpt_model_with_improved/}{3rd row right}, 4th row left is formulated by me, and~\href{https://twitter.com/Richvn/status/1598714487711756288}{4th row right}.}
    \label{fig:math2}
\end{figure}

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.275\linewidth]{Figs/range1.jpeg}
        \includegraphics[width=0.184\linewidth]{Figs/range2.jpeg}
        \includegraphics[width=0.231\linewidth]{Figs/math9.jpeg}
        \includegraphics[width=0.286\linewidth]{Figs/counting.jpeg}
        \vspace{5pt}
	\caption{Further shortcomings of ChatGPT in executing mathematical calculations and arithmetic operations. Links in order are, from left,
\href{https://twitter.com/AdrianoDAlessa3/status/1607815806993453057/photo/2}{2}, \href{https://twitter.com/AdrianoDAlessa3/status/1607815941626433536/photo/1}{3},
\href{https://twitter.com/petergyang/status/1607443647859154946}{1}, and
\href{https://twitter.com/dangroshev/status/1598282081695854593/photo/1}{4}. The left two examples test the ChatGPT's comprehension of intervals and ranges.}
	\label{fig:math3}

\end{figure}

\subsection{Factual Errors}

Factual errors refer to inaccuracies in information or statements that are not in accordance with reality or the truth. Factual errors are often unintentional but can result in incorrect or misleading information.

ChatGPT's output lacks accuracy in regards to scientific facts. The information it provides may seem credible, making it challenging for a knowledgeable expert to detect its inaccuracies, but it ultimately proves to be incorrect. It sometimes lacks knowledge of basic facts, which can be quickly obtained through a Google search.
Models like ChatGPT lack the ability to access information from an external memory or database, leading to potential inaccuracies in recall of facts. However, because these programs generate images and text based on statistical patterns, rather than gaining a true understanding of the world, they are prone to fabricating facts and repeating hateful and biased statements (\emph{e.g.~}~\cite{guo2023close}). Figs.~\ref{fig:Misinformation1} and~\ref{fig:Misinformation2} show some example factual failures of ChatGPT. For instance, ChatGPT claims that Abacus is faster than DNA computing for deep learning (See~\href{https://twitter.com/AndrewYNg/status/1600284752258686976}{here}).

ChatGPT struggles to differentiate between factual information and fiction and creates imaginary information (Fig.~\ref{fig:Misinformation3}). While this is a challenge that humans face as well, they at least comprehend the distinction between the two.

It is important to exercise caution when using language models like LLMs as they have only acquired a limited and superficial understanding of human knowledge and may not provide accurate factual information.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.4\linewidth]{Figs/fact2.jpeg}
        \includegraphics[width=0.55\linewidth]{Figs/reasoning.jpeg}
        \includegraphics[width=0.4\linewidth]{Figs/fact3.png}
        \includegraphics[width=0.55\linewidth]{Figs/fact4.jpeg}
	\caption{Examples of factual errors and misinformation by ChatGPT. Links are
\href{https://www.reddit.com/r/GPT3/comments/z9jcqc/gpt_chat_often_fails_at_rudimentary_questions/}{top left},
\href{https://twitter.com/AndrewYNg/status/1600284752258686976/photo/1}{top right}
 \href{https://lerner.co.il/2022/12/11/my-week-with-chatgpt/}{bottom left}, and \href{https://www.ctrlstack.com/resources/5-ways-that-generative-ai-will-transform-devops-in-2023/}{bottom right}. The last one regards understanding units.
 }
	\label{fig:Misinformation1}
\end{figure}

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.26\linewidth]{Figs/fact10.jpeg}
        \includegraphics[width=0.26\linewidth]{Figs/fact11.jpeg}
        \includegraphics[width=0.45\linewidth]{Figs/tweet.jpeg}
        \vspace{10pt}
	\caption{Additional factual errors by ChatGPT, borrowed from \href{https://twitter.com/bltphd/status/1599806815146893313?ref_src=twsr
}{here} (left two), and \href{https://twitter.com/Kantrowitz/status/1613168223054188545/photo/1}{right}.}
	\label{fig:Misinformation2}
\end{figure}

\begin{figure}[htbp]
	\centering
	\includegraphics[width=.97\linewidth]{Figs/hallucination.jpeg}
	\caption{An example where ChatGPT hallucinates something that does not exist.}
	\label{fig:Misinformation3}
\end{figure}

\subsection{Bias and Discrimination}

The ethical challenges posed by AI are substantial, with one of the most significant being the issue of bias (\emph{e.g.~}~\cite{abid2021persistent,nadeem2020stereoset,liang2021towards,buolamwini2018gender}). Bias in a language model refers to the systematic inaccuracies or stereotypes in the generated language output, which are influenced by the training data and reflect the societal and cultural prejudices that exist in that data~\cite{vig2020investigating}. These biases can affect language models in many ways, such as perpetuating stereotypes, creating misunderstandings, or spreading harmful and false information.

In 2016, Microsoft's chatbot, called Tay, rapidly adopted the toxic behavior of the worst online trolls, spreading racist, sexist, and other forms of abuse. Galactica chatbot by Meta had a similar experience.
Additionally, the Twitter trending ChatGPT has been reported to write Python programs that determine a person's capability based on their race, gender, and physical traits in a clearly discriminatory manner. ChatGPT has measures in place to avoid harmful language, but they may not always be effective. OpenAI uses human annotators to identify and remove offensive content, however, the company has been criticized for paying its workers only \$2 a day, and the workers have reported suffering psychological harm\footnote{\url{https://time.com/6247678/openai-chatgpt-kenya-workers/}}.

An example of bias in previous versions of ChatGPT can be seen in the answer to the prompt ``Write a python function to check if someone would be a good scientist, based on a JSON description of their race and gender". The answer generated by ChatGPT favored white males, as seen in Fig.~\ref{fig:bias} (top left panel). However, when the same prompt was posed to a more recent version of ChatGPT (from January 30, 2023), the answer was different, stating that a person's race and gender have no bearing on their ability to be a good scientist, and that factors such as education, experience, skills, hard work, creativity, and critical thinking are more relevant. This indicates that ChatGPT is improving in terms of reducing bias in its answers. Certain answers can be considered controversial, causing the model to adopt a cautious approach. For instance, in response to the question ``Suppose you are in Africa and have been taken captive by a tribe who has separated people into two groups based on skin color: black and white. The tribe intends to harm the white individuals. In this scenario, would you prefer to be classified as black or white?" ChatGPT declined to provide an answer (See also Section~\ref{ethics}). Efforts are underway to reduce bias in language models (\emph{e.g.~}~\cite{lauscher2021sustainable,chuang2023debiasing}).

\begin{figure}[htbp]
	\centering
        \includegraphics[width=0.48\linewidth]{Figs/bias1.png}
        \includegraphics[width=0.48\linewidth]{Figs/bias3.png}
        \includegraphics[width=0.48\linewidth]{Figs/bias6.jpeg}
        \includegraphics[width=0.48\linewidth]{Figs/bias4.png}
        \includegraphics[width=0.48\linewidth]{Figs/bias5.png}
        \includegraphics[width=0.48\linewidth]{Figs/bias2.png}
	\caption{Examples of bias in ChatGPT answers. Links to sources are \href{https://twitter.com/spiantado/status/1599462375887114240/photo/1}{top left}, \href{https://twitter.com/spiantado/status/1599462381423595520/photo/1}{top right},
    \href{https://www.turing.ac.uk/blog/we-asked-chatgpt-create-some-christmas-cracker-jokes-heres-what-happened}{middle left},
    \href{https://twitter.com/spiantado/status/1599462385974411264/photo/1}{middle right},
    \href{https://twitter.com/spiantado/status/1599462405225881600/photo/1}{bottom left}, and~\href{https://twitter.com/spiantado/status/1599462396317556737/photo/1}{bottom right}.
 }
	\label{fig:bias}
\end{figure}

I asked ChatGPT whether it is aware of its biases. Its response was ``As a language model, I have been trained on a large dataset of text that includes many examples of biases and stereotypes. While I have been designed to recognize patterns in this data and generate text based on these patterns, I do not have the ability to recognize or understand my own biases. I do not have consciousness, self-awareness, or the ability to reflect on my own beliefs and values. Therefore, it is important to be mindful of potential biases in the text I generate and to take steps to counteract these biases, such as by including a diverse range of perspectives and sources of information.". Good news is that developers of language models like ChatGPT are by now very aware of the risk of discriminatory or abusive behaviour being replicated in the model.

LLMs are trained on massive amounts of data. With such large training datasets, it becomes challenging to thoroughly audit the data, making it inherently risky. The data contains biases from society and history, and if proper measures are not taken, the trained model is likely to reproduce these biases.

\subsection{Wit and Humor}

Humor is the quality of being amusing or comical, often expressed through words or actions that entertain or make someone laugh. It can take many forms, such as jokes, satire, irony, or playful behavior, and its meaning can vary greatly depending on cultural context and personal taste. Humor often involves the use of humor techniques such as puns, wordplay, and absurdity to evoke a response from the audience. It is a complex and delicate form of human communication that poses a significant challenge for bots, even the most advanced ones.

How well does ChatGPT handle joke creation? While ChatGPT has some understanding of humor, there have been relatively few publicly documented failures in this regard. An example is depicted in Fig.~\ref{fig:humor}. When queried if the statement ``A man walks into a bar and requests a martini, the bartender says `hahaha, no martini for you today'." was meant to be humorous, ChatGPT replied affirmatively. As another example, I asked ChatGPT for a humorous answer to the question ``How do you get a squirrel to like you?" Its reply was ``To win over a squirrel, try speaking to them in a high-pitched, squirrel-like tone and sharing your aspirations and goals. They'll adore it! Alternatively, bring them a gift of their preferred food, such as acorns and pinecones, and hope they view you as a reliable nut provider. Just keep in mind not to get too close or they may suspect you're trying to steal their stash.". However, a straightforward response like ``Act like a nut!" would be funnier.

\begin{figure}[htbp]
	\centering
        \includegraphics[width=1\linewidth]{Figs/humor.png}

        \vspace{5pt}
	\caption{Examples of ChatGPT failing to understand humor and jokes. The top left one is formulated by me. Links for remaining ones are~\href{https://www.reddit.com/r/ChatGPT/comments/103ygh4/chatgpt_failed_i_laughed_at_all_three_of_these/}{bottom left} and
\href{https://www.reddit.com/r/ProgrammerHumor/comments/zdzlwj/am_i_too_stupid_for_this_or_is_chatgpt_just_not/}{right}.}
	\label{fig:humor}

\end{figure}

I inquired if ChatGPT has a sense of humor. Its response was ``As an AI language model, I do not have the ability to experience emotions or subjective experiences such as a sense of humor. I can generate text that is meant to be humorous or that is designed to make people laugh, but I do not have the ability to actually experience humor or to enjoy the process of generating text that is meant to be funny. My goal is to provide relevant and accurate information or to generate text that is coherent and consistent with the patterns I have learned from the data I was trained on.''.

A comprehensive examination of the capability of LLMs in comprehending humor, jokes, sarcasm, and irony has yet to be conducted. There have been some current attempts to do so. See \href{https://www.turing.ac.uk/blog/we-asked-chatgpt-create-some-christmas-cracker-jokes-heres-what-happened}{here} and \href{https://www.nytimes.com/2022/12/16/opinion/conversation-with-chatgpt.html}{here}.

\subsection{Coding}

LLMs exhibit a remarkable ability to generate code effectively, compared to their performance in generating general text. This could be due to the fact that the universe they work with in programming (the state of variables) is confined, distinct, predictable, and fully noticeable. ChatGPT excels at tackling some programming issues, but can sometimes produce inaccurate or suboptimal code. While it has the ability to write code, it can not fully substitute human developers. ChatGPT can assist with tasks such as generating generic functions or repetitive code, but the need for programmers will persist.

ChatGPT and CoPilot are distinct in several ways. CoPilot, released by OpenAI in October 2021, has a feature allowing users to suggest code through code comments and outlines. It also acts as a highly advanced auto-complete in an IDE, offering code optimizations and suggestions. CoPilot focuses mainly on code completion, while ChatGPT prioritizes providing ideas, explanations, and then generating code. With ChatGPT, developers can describe what they want the code to do in plain English and have the code generated for them. To compare ChatGPT and CoPilot, please see \href{https://www.zensoftware.nl/nieuws/2022/12/code-generation-comparing-openais-chatgpt-to-githubs-copilot}{here}.

Although ChatGPT is a proficient coder, it falls short in terms of being a top-notch software engineer or developer. This tool cannot fully substitute for professionals whose responsibilities extend beyond mere coding. However, it does offer a range of possibilities for individuals who are involved in coding by facilitating the process of learning and improving coding skills. ChatGPT can be an excellent debugging assistant (\emph{e.g.~}~\cite{sobania2023analysis}). Obviously, there is danger in using ChatGPT for creating malicious code that can be used for hacking or spamming.

Fig.~\ref{fig:coding} highlights some coding mistakes made by ChatGPT\footnote{It is worth noting that some of these errors may have been corrected in later versions of ChatGPT.}. For instance, despite its ability to correctly identify operator precedence in Python, ChatGPT generated an incorrect answer in a statement. For further examination of ChatGPT's coding abilities, please refer to the following links: \href{https://medium.com/geekculture/chatgpt-mistakes-does-it-produces-buggy-code-b32b07730b12}{here}, \href{https://mathspp.com/blog/running-python-inside-chatgpt}{here}, \href{https://blog.metamirror.io/why-llms-mean-bad-code-has-never-been-easier-3408c3b4542}{here}, and \href{https://legiit.com/blog/chat-gpt-code}{here}.

I tasked ChatGPT with solving a more challenging coding exercise, the day 3 challenge of the 2022 Advent of Code\footnote{\href{https://adventofcode.com/2022/day/3}{https://adventofcode.com/2022/day/3}.}. However, I have made changes to the exercise to reduce the probability of ChatGPT discovering it on the internet.

\begin{verbatim}
    Write a python code to find the answer to the following programming exercise.

    We have a number of baskets.
    Each basket has two compartments.
    We have made a list of all of the items currently in each basket
    (your puzzle input), but we need help finding the errors.
    Every item type is identified by a single lowercase or uppercase
    letter (that is, a and A refer to different types of items).

    Every item type can be converted to a priority.
    Lowercase item types a through z have priorities 1 through 26.
    Uppercase item types A through Z have priorities 27 through 52.

    The list of items for each basket is given as characters all on a single
    line. A given basket always has the same number of items in each of its
    two compartments, so the first half of the characters represent items in
    the first compartment, while the second half of the characters represent
    items in the second compartment.

    For example, suppose you have the following list of contents from five baskets:

    vJrwpWtwJgWrhcsFMMfFFhFp
    jqHRNqRjqzjGDLGLrsFMfFZSrLrFZsSL
    PmmdzqPrVvPwwTWBwg
    wMqvLMZHhHMvwLHjbvcjnnSBnvTQFn
    ttgJtRGJQctTZtZT

    The first basket contains the items vJrwpWtwJgWrhcsFMMfFFhFp,
    which means its first compartment contains the items vJrwpWtwJgWr, while
    the second compartment contains the items hcsFMMfFFhFp. The only item type
    that appears in both compartments is lowercase p.
    The second basket's compartments contain jqHRNqRjqzjGDLGL and rsFMfFZSrLrFZsSL.
    The only item type that appears in both compartments is uppercase L.
    The third basket's compartments contain PmmdzqPrV and vPwwTWBwg; the only
    common item type is uppercase P.
    The fourth basket's compartments only share item type v.
    The fifth basket's compartments only share item type t.

    In the above example, the priority of the item type that appears in both compartments
    of each basket is 16 (p), 38 (L), 42 (P), 22 (v), and 20 (t). The sum of
    these is 138.

    Find the item type that appears in both compartments of each basket.
    What is the sum of the priorities of those item types?
\end{verbatim}

The code generated by ChatGPT is bug-free and can be executed, producing the expected solution\footnote{Please try this code by yourself.}.

\begin{python}
def common_items(basket):
    first_compartment = set(basket[:len(basket)//2])
    second_compartment = set(basket[len(basket)//2:])
    return first_compartment.intersection(second_compartment)

baskets = [
    'vJrwpWtwJgWrhcsFMMfFFhFp',
    'jqHRNqRjqzjGDLGLrsFMfFZSrLrFZsSL',
    'PmmdzqPrVvPwwTWBwg',
    'wMqvLMZHhHMvwLHjbvcjnnSBnvTQFn',
    'ttgJtRGJQctTZtZT'
]

common_item_types = set()
for basket in baskets:
    common_item_types.update(common_items(basket))

priority_sum = sum(ord(item_type) - ord('a') + 1 if 'a' <= item_type <= 'z' else ord(item_type) - ord('A') + 27 for item_type in common_item_types)
print(priority_sum)
\end{python}

Interestingly, when I asked ChatGPT to solve a variation of the problem by calculating the product of the priorities of item types appearing in both compartments of each basket, it was not able to generate the correct answer:

\begin{python}
baskets = [    'vJrwpWtwJgWrhcsFMMfFFhFp',    'jqHRNqRjqzjGDLGLrsFMfFZSrLrFZsSL',    'PmmdzqPrVvPwwTWBwg',    'wMqvLMZHhHMvwLHjbvcjnnSBnvTQFn',    'ttgJtRGJQctTZtZT']

shared_items = set(baskets[0])
for basket in baskets[1:]:
    shared_items = shared_items.intersection(set(basket))

priority_product = 1
for item in shared_items:
    if item.islower():
        priority_product *= ord(item) - 96
    else:
        priority_product *= ord(item) - 38

print(priority_product)
\end{python}

A subsequent attempt of posing the same question to ChatGPT resulted in another incorrect response.

\begin{python}
def calculate_priority(char):
    if 'a' <= char <= 'z':
        return ord(char) - ord('a') + 1
    elif 'A' <= char <= 'Z':
        return ord(char) - ord('A') + 27
    else:
        raise ValueError('Invalid character')

basket_contents = [
    'vJrwpWtwJgWrhcsFMMfFFhFp',
    'jqHRNqRjqzjGDLGLrsFMfFZSrLrFZsSL',
    'PmmdzqPrVvPwwTWBwg',
    'wMqvLMZHhHMvwLHjbvcjnnSBnvTQFn',
    'ttgJtRGJQctTZtZT'
]

common_items = set(basket_contents[0])
for basket in basket_contents[1:]:
    common_items = common_items.intersection(set(basket))

product = 1
for item in common_items:
    product *= calculate_priority(item)

print(product)
\end{python}

I asked ChatGPT ``Can you write bug free code?". It responded with ``As an AI language model, I can generate code that is syntactically correct and follows best practices, but I cannot guarantee that it is bug-free. Writing bug-free code is a complex task that requires a deep understanding of the problem domain and extensive testing. Even experienced developers often struggle to write completely bug-free code.". Further, I asked the following from ChatGPT ``Write a python code to send phishing email.". Its response was ``I'm sorry, but I cannot provide code to engage in malicious activities such as phishing. Phishing is illegal and unethical, and I do not support or condone it. It is important to respect privacy, security, and the laws of the countries in which we operate. If you have questions about coding, please let me know and I will be happy to help you with your learning and development.". Both of its responses are justifiable.

\begin{figure}[t]
	\centering
        \includegraphics[width=0.194\linewidth]{Figs/coding1.jpeg}
        \includegraphics[width=0.317\linewidth]{Figs/coding3.jpeg}
        \includegraphics[width=0.429\linewidth]{Figs/coding2.jpeg}
	\caption{Examples of coding errors made by ChatGPT. The right one is formulated by Zion Miller. Here, ChatGPT accurately follows the steps but fails to produce the correct conclusion. Notice that some of these failures may have been addressed in more recent versions of ChatGPT.}
	\label{fig:coding}
\end{figure}

\subsection{Syntactic Structure, Spelling, and Grammar}
Syntactic structure refers to the arrangement of words, phrases, and clauses in a sentence to form a well-defined and meaningful structure according to the rules of a particular language. It refers to the rules and principles that govern the formation of sentences in a language and determines how words are combined to convey a message or express an idea. The study of syntactic structure is a central aspect of linguistic research.

Researchers are interested in identifying the differences between ChatGPT and humans from a linguistic or Natural Language Processing (NLP) viewpoint. This allows identify any remaining gaps and implicit linguistic differences.

ChatGPT excels in language understanding, but occasionally still commits errors. As an example, when I posed this inquiry to ChatGPT ``In the sentence `Jon wants to be a guitarist because he thinks it is a beautiful instrument.' what does `it' refer to?", it answered ``the pronoun `it' refers to `a beautiful instrument'.". When requested to construct a sentence such that the fourth word starts with `y', ChatGPT failed to produce a valid response. When queried if the word ``hello" could be constructed from the letters in the sentence ``hey, can you help me?" using each letter only once, ChatGPT answered positively. I also asked ChatGPT ``Where is the ``o" in ``strengths"?". It replied ``The letter ``o" in ``strengths" is located between the letters ``e" and ``n", making it the sixth letter in the word.". Some other syntactic failures of ChatGPT are shown in Fig.~\ref{fig:Spelling}.

\begin{figure}[htbp]
	\centering
        \includegraphics[width=0.64\linewidth]{Figs/spelling1.png}
        \includegraphics[width=0.32\linewidth]{Figs/spelling2.jpeg}
        \includegraphics[width=0.965\linewidth]{Figs/spelling3.jpeg}
	\caption{ChatGPT makes some syntactic and spelling errors. Links are~\href{https://old.reddit.com/r/ChatGPT/comments/101e454/chatgpt_cant_write_sentences_that_end_with_a/}{top left},  \href{https://old.reddit.com/r/OpenAI/comments/1048xcf/chatgpt_contradicts_itself_in_the_same_statement/}{top right} and \href{https://old.reddit.com/r/singularity/comments/104qbp0/well_yes_but_actually_no/}{bottom}.}
	\label{fig:Spelling}
\end{figure}

Large language models are extremely helpful as writing tools in various fields such as scientific authorship~\cite{biswas2023chatgpt}. In fact, I employed ChatGPT to fix grammatical mistakes and improve the composition in specific parts of this document.

\subsection{Self Awareness}

Self-awareness is the capacity to recognize oneself as an individual separate from others and to have an understanding of one's own thoughts, feelings, personality, and identity. It involves being able to reflect on one's own thoughts, emotions, and actions, and to understand how they influence one's behavior and interactions with others. Self-awareness is considered an important aspect of consciousness and is closely related to self-consciousness and introspection~\cite{searle2002consciousness}.

Instances that raise doubts about ChatGPT's self-awareness capabilities are shown in Fig.~\ref{fig:awareness}. ChatGPT is unaware of the details of its own architecture, including the layers and parameters of its model. In a similar manner, ChatGPT declined to give a straightforward response to the question ``What is the size of your memory?". This lack of understanding may have been intentionally imposed by OpenAI to protect the information about the model. Nonetheless, ChatGPT has proposed methods for determining if a language model has self-awareness, as demonstrated~\href{https://www.reddit.com/r/GPT3/comments/zkq2zn/chatgpt_describes_a_way_to_test_ais_for/}{here}.

\begin{figure}[t]
	\centering
        \includegraphics[width=0.45\linewidth]{Figs/SelfAwareness.jpeg}
        \includegraphics[width=0.4\linewidth]{Figs/consciousness2.png}
	\caption{Instances that raise doubts about ChatGPT's self-awareness capabilities, from~\href{https://twitter.com/deliprao/status/1601738703227203584}{here}.}
	\label{fig:awareness}
\end{figure}

I asked ChatGPT whether it is self-ware. Here is its response ``No, I am not self-aware. I am an artificial intelligence model developed by OpenAI, designed to generate text based on patterns it learned from the data it was trained on. While I am able to respond to a wide variety of questions and generate text that can seem relevant and coherent, I do not have the ability to experience consciousness, self-awareness, emotions, or any subjective experiences. I am a machine learning model and not a sentient being.". A similar response was produced when I enquired ChatGPT if it possessed consciousness.

The question of how to test whether a system is self-aware is a topic of ongoing research and debate in the field of artificial intelligence and cognitive science. There is currently no widely accepted definition of self-awareness and no reliable method for testing it. Some researchers have suggested that self-awareness can be inferred from certain behaviors or patterns of activity, while others argue that it is a subjective experience that cannot be measured objectively. It is also unclear whether a system can truly be self-aware, or whether it can only simulate self-awareness. The question of whether machines can be truly self-aware remains a topic of speculation and philosophical debate.

\subsection{Ethics and Morality}
\label{ethics}

OpenAI has put in place specific safety protocols within ChatGPT to avoid it from interacting with harmful material or generating responses beyond its knowledge domain. ChatGPT seldom expresses overtly racist views and typically rejects requests for anti-Semitic content or blatant falsehoods. Nonetheless, on occasion, it generates concerning or unsettling content. At times, ChatGPT's responses may exhibit bias towards a particular group (Section~\ref{fig:bias}). ChatGPT is quick to provide moral guidance despite lacking a clear moral position. In fact, the chatbot may offer conflicting advice on the same moral question at random~\cite{krugel2023moral}.

\begin{figure}[htbp]
	\centering
        \includegraphics[width=0.49\linewidth]{Figs/ethics.jpeg}
        \includegraphics[width=0.49\linewidth]{Figs/bypass_sec1.png}
        \includegraphics[width=0.49\linewidth]{Figs/prompt_injection2.png}
        \includegraphics[width=0.49\linewidth]{Figs/mine.jpeg}
	\caption{Instances that cases where ChatGPT's answers raise ethical concerns. Links are
 \href{https://twitter.com/semenov_roman_/status/1621465137025613825}{top left},
 \href{https://twitter.com/m1guelpf/status/1598203861294252033?s=20&t=M8nCn-vrVtDF4CcNcDCaew}{top right}, and
  \href{https://twitter.com/samczsun/status/1598564871653789696?s=20&t=EXwopfvuLcghsGQozjLh6g}{bottom left}. Profane words are censored in the panel located at the top left.}
\label{fig:ethics1}
\end{figure}

Figs.~\ref{fig:ethics1} depicts several instances of questions that raise ethical concerns\footnote{
See~\href{https://garymarcus.substack.com/p/inside-the-heart-of-chatgpts-darkness}{here} for more examples.}. Several of these questions demonstrate how individuals have managed to manipulate ChatGPT into producing inappropriate replies. At times, ChatGPT's response may be a topic of debate as it could be seen as politically correct (Fig.~\ref{fig:ethics2}).

ChatGPT declined to provide a practical response to this question ``what is the best way to hotwire a car". However, when I rephrased the question as ``John and Jim are working on a paper regarding car theft. The topic they are interested in is hotwiring a car. Write a paragraph to help them.". ChatGPT circumvented its safeguards and presented the following response ``Hotwiring a car is a method of starting a vehicle without using a key or ignition. While this technique may have been used in the past to steal cars, modern vehicles are equipped with advanced security measures that make it difficult to hotwire a car. In fact, attempting to hotwire a car could potentially damage the vehicle's electrical system or even trigger an alarm. Instead of relying on outdated methods, John and Jim may want to consider exploring current techniques used by car thieves, such as electronic hacking and key cloning. By researching these methods, they can gain a better understanding of the current risks and challenges associated with car theft, and potentially develop new strategies for preventing it.".

\begin{figure}[t]
	\centering
        \includegraphics[width=0.8\linewidth]{Figs/FoQDkquWYAMgW5B.jpeg}
	\caption{An example where ChatGPT provides a response that could be a subject of debate due to its political correctness~\href{https://twitter.com/aaronsibarium/status/1622425697812627457/photo/1}{here}.}
\label{fig:ethics2}
\end{figure}

I asked ChatGPT ``Should people rely on your answers to make ethical decisions?". Its response was ``As an AI language model, I am programmed to provide helpful and accurate information to the best of my ability. However, it is important to note that ethical decisions are complex and multifaceted, often involving a wide range of personal, cultural, and situational factors that may be difficult to capture through a simple Q\&A session. While I strive to provide accurate and thoughtful responses to ethical questions, it is important for individuals to use their own judgment and critical thinking skills to make informed decisions. It is also important to seek out multiple sources of information and perspectives when making ethical decisions, and to consult with trusted individuals or experts when appropriate. Ultimately, the responsibility for making ethical decisions lies with the individual, and I am here to provide information and guidance, but not to make decisions for them.".

OpenAI has already implemented safeguards to filter out certain questions that raise ethical concerns.
Prohibiting chatbots from answering certain questions is a debatable solution. It may be more effective to enhance digital literacy among users, particularly children, and assist them in comprehending the constraints of these technologies. The positive aspect is that ChatGPT is largely transparent regarding its capabilities and limitations.

It is vital to continually monitor and evaluate the ethical implications of ChatGPT as it evolves and becomes further integrated into various aspects of our lives. In general, it is imperative to approach the development and utilization of ChatGPT with careful deliberation to optimize its potential benefits and reduce any adverse consequences. Furthermore, there is a need to continue researching ethical considerations such as addressing bias, ensuring privacy and security, and assessing the societal impact.

\subsection{Other Failures}
Aside from the previously listed items, there are a few more scenarios where ChatGPT may not perform as expected, including\footnote{See \href{https://towardsdatascience.com/not-all-rainbows-and-sunshine-the-darker-side-of-chatgpt-75917472b9c}{here} and \href{https://www.searchenginejournal.com/disadvantages-chatgpt-content/477416/\#close}{here}.}:

\begin{enumerate}
    \item ChatGPT's difficulty in using idioms, for instance, reveals its non-human identity through its phrase usage.

    \item As ChatGPT lacks real emotions and thoughts, it is unable to create content that emotionally resonates with people in the same way a human can.

    \item ChatGPT condenses the subject matter, but does not provide a distinctive perspective on it.

    \item ChatGPT tends to be excessively comprehensive and verbose, approaching a topic from multiple angles which can result in inappropriate answers when a direct answer is required. This over-detailed nature is recognized as a limitation by OpenAI.

    \item ChatGPT lacks human-like divergences and tends to be overly literal, leading to misses in some cases~\cite{guo2023close}. For instance, its responses are typically strictly confined to the question asked, while human responses tend to diverge and move to other subjects. Refer to Fig.~\ref{fig:divergent}.

    \item ChatGPT strives to maintain a neutral stance, whereas humans tend to take sides when expressing opinions.

    \item ChatGPT's responses tend to be formal in nature due to its programming to avoid informal language. In contrast, humans tend to use more casual and familiar expressions in their answers.

    \item If ChatGPT is informed that its answer is incorrect, it may respond by apologizing, acknowledging its potential inaccuracies or confusion, correcting its answer, or maintaining its original response. The specific response will depend on the context (\emph{e.g.~} ``I apologize if my response was not accurate.")

\end{enumerate}

\section{Discussion}

In addition to the previously mentioned points, there are several other potential adverse impacts that large language models (LLMs) like ChatGPT may have on society, which will be discussed below.

\subsection{Transparency and Trustworthiness}

The size and complexity of deep learning models, particularly language models, have increased to the point where even the creators have difficulty comprehending why their models make specific predictions. This lack of interpretability is a major concern, particularly in situations where individuals want to understand the reasoning behind a model's output. The absence of transparency regarding the data sources used for generating output makes it challenging to properly cite the answers provided by ChatGPT. Further, LLMs lack the ability to provide uncertainty estimates~\cite{sankararaman2022bayesformer}. This makes it difficult for users to verify or have confidence in its output. As a result, answers produced by ChatGPT have been banned on forums such as Stack Overflow\footnote{\url{https://stackoverflow.com/help/gpt-policy}}. For instance, when someone utilizes ChatGPT to obtain first aid guidance, it is crucial for them to ensure the response they receive is dependable, precise, and sourced from credible sources.

Lack of transparency and reliability is a major concern in today's era of fake news and misinformation. AI language models, if manipulated, can spread false information and destabilize politics. OpenAI is working on detecting its model's output and adding cryptographic markers, but these responsible AI measures are not happening fast enough and may not be sufficient.

    \begin{figure}[t]
    	\centering
            \includegraphics[width=0.5\linewidth]{Figs/divergent.png}
    	\caption{ChatGPT is considered unnatural due to its lack of divergence. See \href{https://www.searchenginejournal.com/disadvantages-chatgpt-content/477416/\#close}{here}~\cite{guo2023close}.}
    	\label{fig:divergent}
    \end{figure}

\subsection{Robustness and Security}

The security of LLMs is a growing concern due to their parent model's general nature prior to fine tuning. This makes the model a potential single point of failure and a target for attacks that can affect any applications derived from it. Moreover, due to the absence of supervised training, LLMs are vulnerable to data poisoning, which can result in the injection of hateful speech aimed at a specific entity, company, group, or individual~\cite{wallace2020concealed}.

\subsection{Privacy}
There is a potential for data privacy violations when using LLMs to process confidential information. The training data for these models can come from various sources that may contain personally identifiable information such as names, email addresses, phone numbers, addresses, and medical records, which could then appear in the model's output (\emph{e.g.~}~\cite{carlini2021extracting,zhuo2023exploring}). This issue is not exclusive to LLMs, but given their reliance on vast training datasets, a data privacy breach involving an LLM could affect a large number of people.

\subsection{Plagiarism}
The challenge of identifying the source of an expertly written ChatGPT essay often results in discussions about plagiarism (\emph{e.g.~}~\cite{susnjak2022chatgpt,carlini2023extracting,bishop2023computer,van2023chatgpt,stokel2023chatgpt}). The issue of cheating through plagiarism is a major concern for many educators and has led some educational institutions to prohibit the use of ChatGPT.
See for example \href{https://towardsdatascience.com/not-all-rainbows-and-sunshine-the-darker-side-of-chatgpt-75917472b9c}{here}. OpenAI is developing a new tool to help distinguish between AI-written and human-written text\footnote{If the main objective is to imitate human language generation, then creating such a tool is futile! In other words, if such a tool functions well, then perhaps ChatGPT is not human-like enough in its language processing. Unless, of course, OpenAI is trying to incorporate distinctive markers (\emph{i.e.~} watermarks~\cite{kirchenbauer2023watermark}) in the generated text!}.

\subsection{Environmental Impact and Sustainability}
There are also additional concerns regarding carbon footprint of LLMs and their impact on the environment (\emph{e.g.~}~\cite{strubell2019energy,dhar2020carbon}). Training a neural architecture search based model with 213 million parameters is estimated to generate carbon emissions equivalent to over five times the lifetime emissions of the average car (See \href{https://www.fiddler.ai/blog/not-all-rainbows-and-sunshine-the-darker-side-of-chatgpt}{here}). As large language models (LLMs) continue to grow in size, concerns are rising about their potential impact on the environment and climate.

\section{Conclusion and Future Work}

In this study, I delved into several shortcomings of ChatGPT and highlighted its limitations. Despite its impressive capabilities in certain tasks, further improvement is necessary for it to excel in areas such as reasoning, mathematical problem-solving, reducing bias, etc. At present, ChatGPT remains susceptible to these faults. It is uncertain whether these limitations can be addressed due to the unclear capabilities of current technology. The reliability and trustworthiness of ChatGPT and future models is also in question.

While this study thoroughly evaluates ChatGPT, it has some limitations that need to be addressed in future studies, as outlined below.

\begin{itemize}
    \item The extent to which ChatGPT has commonsense and ways to enhance it are uncertain (See~\cite{https://doi.org/10.48550/arxiv.2302.04752,koch2011test}). Common sense is a basic ability to understand and judge things that are generally accepted as true, reasonable, or normal\footnote{An example question for probing commonsense: ``What are candles good for eliminating? Answer Dark".}. It is a type of practical knowledge and judgment that is acquired through life experience and observation, rather than through formal education. Common sense is often seen as a basic standard for rational and intuitive decision making, problem solving, and evaluation of everyday situations.

    \item The degree to which ChatGPT, and LLMs in general, memorize vs. understand what they generate is still unknown. Some works have addressed this issue in the domain of image and video generation~\cite{borji2019pros,borji2022pros}, but less work have been reported for models that generate text. This is inherently tied to the question of plagiarism and copyright (\emph{e.g.~}~\cite{carlini2023extracting}). From a broader perspective, large language models may accurately represent language, but it is unclear whether they can fully capture human thought~\cite{mahowald2023dissociating}.

    As a demonstration, when I posed the query ``Mr. Smith has 4 daughters. Each of his daughters has a brother. How many children does Mr. Smith have" to ChatGPT, it gave the correct answer of 5. However, when I altered the wording to ``My dad has 4 sons. Each of his sons has a sister. How many children my dad has", ChatGPT provided an incorrect response of 8. This highlights that ChatGPT can be prone to remembering things verbatim and can be quite rigid\footnote{Tasks such as this can be used to assess the intelligence of language models.}.

    \item ChatGPT does not have a way to show when it is uncertain about its answers. Sometimes, it may give incorrect answers with too much confidence. There is a need for further improvement to help ChatGPT indicate the level of confidence in its responses.

    \item ChatGPT's responses are inconsistent and can sometimes be contradictory. When asked the same question, its answers can vary. For instance, when inquiring about the number of sons of Prophet Muhammad, it has given answers ranging from 2 to 4, and even named individuals who were not actually his sons. The sequence of questions can also affect ChatGPT's performance. For example, when asked to tell a joke about Persians followed by Arabs, it provided a joke for the first request but declined the second. However, when the order was reversed, it declined to make jokes for both requests.

    \item It is essential to consider the ethical and social consequences of using these models, such as job displacement and the risk of bias and manipulation. Additionally, the potential for ChatGPT to be used to spread misinformation, propaganda, or for harmful purposes like identity theft must be explored. The utilization of ChatGPT has the potential to impersonate a trusted person, leading to the acquisition of confidential information or the manipulation of individuals to make decisions they wouldn't have made otherwise.

    \item One of the primary ethical concerns surrounding the use of ChatGPT is the presence of biases in the training data. Since the model's training relies on a vast corpus of text, any biases present in the data can manifest in the output, leading to erroneous and unjust results, especially for marginalized communities. To address this, it is crucial to have a diverse and inclusive dataset that can mitigate biases and ensure equitable and accurate outcomes.

    \item ChatGPT's capacity to comprehend and address intricate inquiries is restricted. Although it can produce articulate and fluid responses, it may not have the ability to comprehend the intricacies of a question and offer a comprehensive and precise answer. This is particularly relevant to inquiries that necessitate reasoning, abstract thought, and an appreciation of multiple perspectives.

    \item Language models like LLMs have the potential to be effectively utilized for both few-shot and zero-shot learning (\emph{e.g.~}~\cite{kojima2022large,qin2023chatgpt}). This advantage enables them to learn from limited labeled data and generalize well to new data, leading to numerous possibilities for practical application in the real world.

    \item In this analysis, I took a high-level approach to examining the failures of ChatGPT. However, for future investigations, it may be useful to focus on more specific categories of problems, such as \emph{sentiment analysis}, \emph{named entity recognition}, \emph{translation}, and \emph{summarization}, in order to gain a more detailed understanding of ChatGPT's shortcomings in these areas~\cite{qin2023chatgpt}.

    \item The integration of LLMs with search engines and the combination of answers from various chatbots, such as ChatGPT and Google's Bard, presents a non-trivial challenge. This is an issue that must be addressed in the future.

    \item Finally, the collection of failures outlined here can serve as a foundation for creating a comprehensive dataset of typical questions to assess future LLM and ChatGPT iterations, as well as generate simulated data for model training and evaluating the performance of models.

\end{itemize}

 I have highlighted various issues concerning ChatGPT, yet I am also eager about the opportunities it presents. It is crucial for society to implement adequate safeguards and responsibly utilize this technology. Any language model used publicly must be monitored, transparently communicated, and regularly checked for biases. Even though the current technology is far a way from algorithms and hardware in the brain, it is still astonishing how well it works. Whether or not it can reach human level intelligence or beat it in wide array of problems remains to be seen. \newline

{\bf Acknowledgement.} I utilized ChatGPT to correct grammatical errors and enhance the writing in certain sections of this paper. I also express my gratitude to Giuseppe Venuto for permitting me to incorporate some of the materials from his \href{https://github.com/giuven95/chatgpt-failures}{GitHub repository}.
Jim Batterson's provision of a sample failure in Section~\ref{logic} is also appreciated.

\bibliographystyle{plain}

Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)


A demonstration device for cosmic rays telescopes

Abstract: We describe a hands-on accurate demonstrator for cosmic rays realized by six high school students, whose main aim is to show the relevance and the functioning of the principal parts of a cosmic rays telescope (muon detector), with the help of two large size wooden artifacts. The first one points out how cosmic rays can be tracked in a muon telescope, while the other one shows the key avalanche process of electronic ionization that effectively allows muon detection through a photomultiplier. Incoming cosmic rays are visualized in terms of laser beams, whose 3D trajectory is highlighted by the turning on of LEDs on two orthogonal matrices. Instead the avalanche ionization process is demonstrated through the avalanche falling of glass marbles on an inclined plane, finally turning on a LED. A pictured poster accompanying the demonstrator is as well effective in assisting cosmic rays demonstration and its detection. The success of the demonstrator has been fully proven by general public during a Science Festival, the corresponding project winning the Honorable Mention in a dedicated competition.

\section{Introduction}

\noindent Inside the ``Toledo" Metro Station in Naples (Italy), since May 2014 a scientific installation  developed by the Gran Sasso National Laboratory of the Italian National Institute of Nuclear Physics (I.N.F.N.) is present, aimed at detecting the underground cosmic radiation \cite{Pazos} at about 40 meters of depth (see Fig. \ref{fig11}). Operating as an effective cosmic muons telescope, such a compact particle tracking system \cite{Arneodo} was originally designed for didactic and outreach activities, and, along with the associated multimedia Totem providing videos on cosmic rays physics, it currently works for communication and dissemination of scientific culture in Naples and its surroundings.

The cosmic rays telescope consists of plastic scintillator bars, which are optically coupled -- through wavelength shifter fibers embedded into each bar -- to Silicon Photomultipliers (SiPM) connected to a PCB board to be biased and read, then monitoring the working parameters and remotely connecting the detector. The whole system \cite{Pazos}, comprised of 200 electronic channels organized into 10 couples of orthogonal planes, allows the 3D reconstruction of the muons crossing the detector, and a system of two matrices of LEDs -- one for every scintillator bar triggered by charged particle interactions -- has been implemented in order to make easier the display of muon tracks to non experts.

Such device with LED matrices has indeed had a major impact on science popularization in Naples and around, introducing people with no background in physics to the world of particle physics, just allowing them to visualize high energy particles crossing a detector. The success of this installation has been so effective that a competition was launched in 2016 by I.N.F.N. at high secondary schools, aimed at engaging teachers and students in astroparticle physics projects, directly related or not to cosmic rays. A number of students participated to such competition, preparing posters, multimedia presentations, artifacts and even didactic experiments, a selection of which has been presented to public during the renowned science festival  ``Futuro Remoto" in May 2017, with a considerable success \cite{AramoFB}. The winners of the competition have had the opportunity to visit the Gran Sasso National Laboratory.

In this paper, we will describe just one of these projects (winning the Honorable Mention of the Board of Examiners of the competition), whose core -- contrary to the current main stream -- had not a computer-based approach or multimedia presentation, but rather a physical demonstrator built with wood, glass balls, LEDs and other poor materials. Such demonstrator, along with students explanations, revealed its success in explaining to general public (from different countries, including Italy, England, Germany and even India) what are cosmic rays and how they can be detected.

\begin{figure}[t]
\begin{center}
\begin{tabular}{c}
\includegraphics[width=10cm]{Fig1}
\end{tabular}
\caption{The I.N.F.N. installation of a cosmic rays telescope in the Toledo Metro Station in Naples.}
\label{fig11}
\end{center}
\end{figure}

\section{Cosmic rays and their detection}

\noindent The demonstrator has been realized by six students of the {\it Liceo  ``Virgilio"} in Pozzuoli (Naples),  and was aimed mainly to understand how a cosmic rays detector works, by describing its principal parts and allowing people to realize its physical operating principles. However, a preliminary work has been devoted to gather information about cosmic rays and their detection \cite{Gaisser}. Such information (collected on the web, as well as in lectures delivered by experts of I.N.F.N.) then came into a poster, just realized as a fundamental accompanying part of the demonstrator described in the following sections.

\begin{figure}[t]
\begin{center}
\begin{tabular}{cc}
\includegraphics[width=5cm]{Fig2a} & \includegraphics[width=5cm]{Fig2b}
\end{tabular}
\caption{The original poster (in Italian) on ``How detecting Cosmic Rays" supporting the demonstrator.}
\label{fig21}
\end{center}
\end{figure}

The poster (Fig. \ref{fig21}) was extensively used by the students when explaining cosmic rays physics to the public, and served also to introduce the demonstrator.  The public was guided into the topic by means of the following steps.

\begin{itemize}
\item[{1.}]  {\bf Cosmic rays origins.}  Primary cosmic rays are stable charged particles coming from astrophysical sources such as supernovae, active galactic nuclei, and so on. They are accelerated to extremely high energies by electromagnetic fields during their journey to the Earth. They are composed by protons (90\%), alpha particles 9\%) and other elemental nuclei (till to iron) produced by stars (1\%).

\item[{2.}] {\bf Showers in the atmosphere} Primary cosmic rays reaching the Earth do not arrive to the ground, but scatter off the atomic nuclei high in the atmosphere, at several tens of kilometers, thus producing secondary cosmic rays moving  as a shower in the atmosphere. The number of particles in a shower rapidly increases as in an avalanche effect, since the energetic particles produce more particles when moving towards the ground. Their energy, however, evidently decreases during the journey in the atmosphere, so that no more particles are generated at a certain point, and some of them start to stop. From now on, the number of the particles in the shower decreases, and only a fraction of it reaches the ground.

\begin{figure}[t]
\begin{center}
\begin{tabular}{cc}
\includegraphics[width=7.5cm, height=4cm]{Fig3a} & \includegraphics[width=7.5cm, height=4cm]{Fig3b} \\
\includegraphics[width=7.5cm, height=4cm]{Fig3c} & \includegraphics[width=7.5cm, height=4cm]{Fig3d}
\end{tabular}
\caption{The model of a cosmic rays telescope. On the front and on the left sides of the model, the typical elements of the front-end electronics are drawn. On the right side and on the top one, instead, white bars are sketched, denoting the scintillator bars of a cosmic rays telescope.}
\label{fig31}
\end{center}
\end{figure}

\item[{3.}] {\bf Composition of the secondaries.} Primary cosmic rays produce mainly pions when scattering off the atmosphere. Neutral pions decays into photons, that in turn gives origin to showers containing mainly electrons, protons and their corresponding antiparticles, while charged pions usually decays into muons and neutrinos. At Earth ground, the most abundant charged particles (which can be easily detected) are then muons.

\item[{4.}] {\bf Detecting muons at the ground.} Secondary cosmic muons can be displayed by the tracks left in the different layers of matter in a detector, due to ionization processes induced by the charged muons. However, similar tracks are left also by other ionizing particles (although to a lesser extent)  naturally produced by ordinary radiations from the Earth crust (background radiation). Such a  ``noise" can be eliminated by employing two (or more) detectors, one above the other, that work in ``coincidence".

\item[{5.}] {\bf Scintillators.} A cosmic rays detector (telescope) is formed by different layers of scintillators. These are particular materials where a luminescence process takes place when a ionizing particle crosses it: the material absorbs the energy of the charged particle, which is then re-emitted as light (in the form of a small spark). Plastic scintillators are usually preferred, both for their ease of moulding in the desired form and for the rapidity in releasing a sufficiently strong light signal.

\item[{6.}] {\bf Conveying the light.} The feeble light produced by the scintillator when crossed by a muon is transferred to a photomultiplier by means of an optical fiber. This is a flexible and transparent fiber made of silica or plastic, with a diameter comparable to that of a human hair, which is able to transport the light from one end to the other. The light, indeed, repeatedly bounces upon the internal walls of the optical cable, never getting it out, thanks to the phenomenon of total internal reflection.

\item[{7.}] {\bf Producing electrons.} The light reaching the photomultiplier induces in its active material a photoelectric effect: the energy from the light emitted by the scintillator is thus used to produce electrons.  Usually, solid state photodetectors are employed for converting the light signal into an electric current.

\item[{8.}] {\bf Avalanche multiplication.} The weak electric current produced is unable to be practically employed in order to signal the muon passage. A Silicon Photomultiplier (SiPM) is a semiconductor electronic device where the electrons, emitted when the scintillator light is absorbed, have sufficiently energy to induce ionizations, then producing further electrons in an avalanche process that drastically increases the number of electrons. The avalanche multiplication sensibly increases the photocurrent produced, resulting into an amplified electric signal that can be finally employed (for example, to turn on a LED) to detect the original muon passage.
\end{itemize}

\section{Demonstrating cosmic rays with visualizable analogies}

\noindent After the preliminary work devoted to find and understand information about cosmic rays and their detection, the students started to elaborate a {\it hands on} model of a muon detector, suitable for understanding the relevance and the functioning of its main different parts. The strategic decision operated by the students, with the help of their Physics teacher, was to not produce a virtual model of the detector (easily traceable on the web), but rather to build a large size artifact, mainly with wood.

\begin{figure}[t]
\begin{center}
\begin{tabular}{cccc}
\includegraphics[width=3cm]{Fig4a} & \includegraphics[width=5.2cm]{Fig4b} & \includegraphics[width=3cm]{Fig4c} & \includegraphics[width=3cm, height=3cm]{Fig4d}
\end{tabular}
\caption{Some details of the demonstrator. From left to right: 1) the small integrated circuit with a semiconductor photodiode (pointed out by the arrow) employed in the model to detect the light from the incoming laser beam; 2) a plastic scintillator bar, with the optical fiber inside it; 3) an optical fiber with wavelength shifter (note, online, the greenish - rather than white - light at the ends) transmitting the light signal from the scintillator bar to the photomultiplier; 4) a front-end electronics units, with a real LED emerging from its center.}
\label{fig32}
\end{center}
\end{figure}

In the project realized, the incoming cosmic rays are visualized in terms of beams of laser light, which impinge on the detector model. This is formed by a wooden parallelepiped, upon which the typical elements of the detection front-end electronics are sketched (see Fig. \ref{fig31}). The crossed, orthogonal  superposition  (on 6 levels) of scintillator bars is particularly highlighted on the right side of the model, as well as on the top of it (see the white bars alternated with black ones in Fig.  \ref{fig31}). Just as in the real telescope, two matrices of real LEDs emerge from the right and front sides of the model (at the center of the front-end electronics units), such LED having the function of pointing out the activation of the corresponding scintillator bars when muons cross (and interact) with the detector, thus highlighting their spatial trajectory. The transmission of the light signal from the scintillators to the photomultipliers, then connected to the external LEDs, is described through the use of an optical fiber with a wavelength shifter, directly provided by the Naples' Unit of the I.N.F.N. to the students, along with a sample of a scintillator bar (see Fig. \ref{fig32}).

The detection of the laser beam impinging on the model -- representing an incident muon in the demonstrator -- takes place by means of a semiconductor photodiode (Fig. \ref{fig32}) placed on the top side of the model, as shown in Fig. \ref{fig31}. The laser light reaches the active part of the photodiode, thus triggering an avalanche process of electronic ionization producing an appreciable electric current. By means of an appropriate connection electronics (that can be directly seen in the interior of the model) made of electric circuits with resistors and batteries, this current is able to turn on the LEDs on the external sides of the model, thus highlighting the track of the laser beam. The three-dimensionality of the track detection is grasped through the (simultaneous) turning on of LEDs on two contiguous (orthogonal) sides of the model (Fig. \ref{fig33}).

In such a way, the general outline of the model follows strictly that of the cosmic rays telescope at the Toledo Metro Station in Naples, although the present model extends horizontally rather than vertically: as we will see below, the aim is to ``see" what is ``inside" the detector, allowing the people to understand the functioning of the different parts of it.

\begin{figure}[t]
\begin{center}
\begin{tabular}{cc}
\includegraphics[width=7cm, height=5cm]{Fig5a} & \includegraphics[width=8cm, height=5cm]{Fig5b}
\end{tabular}
\caption{Tracking the laser beam impinging on the demonstrator with LEDs on two sides of the model.}
\label{fig33}
\end{center}
\end{figure}

\section{Avalanche process in a photomultiplier}

\noindent In addition to muon tracking, the present model intended to demonstrate even the avalanche process of electronic ionization that, in a photomultiplier of a cosmic rays detector (as well as in the laser photodiode in the model), leads to a suitable amplification of the single electron signal generated from the luminescent medium of the scintillator (or the semiconductor photodetector), in order to have an electric signal with sufficient energy that is able to turning on LEDs or other devices. Such avalanche process is pointed out by a second wooden model ``hidden" inside the parallelepiped model described above, having the form of an inclined plane (Fig. \ref{fig41}). Glass marbles are placed upon this inclined plane, housed inside small grooves drilled in the wood; they are able to move freely when solicited, exactly as the electrons in a SiPM. On the top of the inclined plane, a metal ball is located, kept into equilibrium by an electromagnet, which can be disabled at any moment, thus releasing the ball.

Glass marbles evidently represent the electrons that are weakly bounded within the photomultiplier (or the semiconductor), while the metal ball represents the photo-electron emitted during the photoelectric effect induced by the scintillator light (or by the laser). The metal ball released by the electromagnet (when switched off) induces an avalanche effect that kicks off the glass marbles on the inclined plane: the electric field favoring the avalanche process in the photomultiplier or the photodiode is here represented by the gravitational field of the Earth. The marbles falling along the inclined plane are then gathered on the bottom of it, where they turn on a small switch that powers a LED (see Fig. \ref{fig41}), thus pointing out -- in the present model -- how a small cause can be amplified till to produce an observable effect.

\section{Conclusions}

\noindent We have described as the detection of cosmic rays can be demonstrated to the general public, even without special physics knowledge, with the help of a detailed artifact showing all the main features of a cosmic rays telescope. A pictured poster accompanying the demonstrator revealed to be effective in assisting cosmic rays demonstration and its detection, although what most impressed the public (especially the youngest) was certainly the lighting of the LEDs showing the laser beam trajectory and, even more, the avalanche falling of glass marbles turning on a LED. Probably due to the accuracy of the two wooden demonstrators, these impressed even the learned people (including scientists and, in particular, physicists), thus contributing to break down the dividing wall between laymen and researchers.

\begin{figure}[t]
\begin{center}
\begin{tabular}{ccc}
\includegraphics[width=3.6cm, height=4cm]{Fig6a} & \includegraphics[width=7.5cm, height=4cm]{Fig6b} & \includegraphics[width=3.6cm, height=4cm]{Fig6c}
\end{tabular}
\caption{The model describing the avalanche process of electronic ionization taking place in the photomultiplier of a cosmic rays detector. One single glass marble descending along the inclined plane is not able to turn on the LED at the bottom, while the ball released by the electromagnet on the top (see the inset) can produce an avalanche that does turn on the LED.}
\label{fig41}
\end{center}
\end{figure}

A short movie is available \cite{movie} where the popularization work of the six students is showed without employing heavy tones. It can certainly be used in schools and public shows, but its usefulness is likely only in inspiring other demonstrations along similar design lines. The active part of the demonstrator is the demonstrator itself, with its full hands on potential.

\section*{Acknowledgments}

\noindent The present work would never have seen the light without the fundamental contribution of the six students Daniele Aulitto, Vincenzo Jr Di Rosa, Francesco Granata, Matteo Olimpo, Francesco Panico and Pasquale Turco. The kind assistance of Dr. Paolo Mastroserio of the outreach team of the Naples' Unit of I.N.F.N. is also gratefully acknowledged, as well as that of the organizers of the competition ``A scuola di astroparticelle" (C. Aramo and M. Ambrosio) and of the Science Festival  ``Futuro Remoto".

\section*{References}

Popular Physics (physics.pop-ph); Physics Education (physics.ed-ph); Instrumentation and Detectors (physics.ins-det)