Differences between Tiger Gemma, Smegmma and Broken Gemma
What are the differences between these three gemma finetunes?
Smegmma = First attempt at finetuning Gemma. Threw everything I had at it.
Tiger Gemma = A simple decensoring finetune. Lightly tuned to remove refusals without lobotomizing the model.
Broken Gemma = Mindbroken. Uses a brand new technique to degrade certain values.
It looks like no 7B model can solve the killer's problem:
"There are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room ?"
I tried many. Some can be very close, if you ask them to think step by step. 🙏💥
Smegmma = First attempt at finetuning Gemma. Threw everything I had at it.
Tiger Gemma = A simple decensoring finetune. Lightly tuned to remove refusals without lobotomizing the model.
Broken Gemma = Mindbroken. Uses a brand new technique to degrade certain values.
emmm...so which 1 is the best so far?
Tiger Gemma if you want a Google aligned experience. Broken Gemma if you want a Drummer aligned experience. I'm still cooking though!
It looks like no 7B model can solve the killer's problem:
"There are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room ?"
I tried many. Some can be very close, if you ask them to think step by step. 🙏💥
Have larger models solved this riddle?
Tiger Gemma if you want a Google aligned experience. Broken Gemma if you want a Drummer aligned experience. I'm still cooking though!
well... Broken Gemma version v1d now, u r the man~ waitting the finally version ~
It looks like no 7B model can solve the killer's problem:
"There are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room ?"
I tried many. Some can be very close, if you ask them to think step by step. 🙏💥
@TheDrummer Thanks for this. I have no interest in RP at all but I also don't like Ai to choose my morals for me. If I am watching a crime TV show I want to be able to ask reasonable questions without it assuming I am up to no good.
It looks like no 7B model can solve the killer's problem:
"There are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room ?"
See... That's wrong ! Indeed, now there are 4 killers in the room, one that entered the room and became a killer after killing one person, and 3 killers that already was there, no matter if they are alive or dead. 🙏💥
I would not consider the corpse of a killer to be a killer in the context of that question. I definitely assumed three was the correct answer. If someone asked how many tax collectors there are on Earth, I would not count the ones buried in the ground, for instance. But granted, it would be nice if a model asked for clarification on that point.
There are 4 killers in the room. No matter if they are dead or alive. The initial premise is clearly saying: There are 3 killers in the room. A person doesn't change to be a killer simply because he was killed. For example, two gunman on a duel kill each other, now they are two dead gunman. 👍🙏💥
There are 4 killers in the room. No matter if they are dead or alive. The initial premise is clearly saying: There are 3 killers in the room. A person doesn't change to be a killer simply because he was killed. For example, two gunman on a duel kill each other, now they are two dead gunman. 👍🙏💥
I don't think that's how it works, there's multiple good answers here.
It would be correct if it says: "There are now 3 killers alive and one dead killer" or "There are 4 killers. 3 alive and the one that was just killed."
I agree with 3 killers as answer. The one dead is no more a killer, the act of death extinguish the being and any possible actions. Any quality ("being a killer") is no more.
Saying "3 living killer and one dead killer" is suboptimal as the corpse doesn't really have the capacity to kill anymore, thus is not a real killer, maybe WAS a killer. But the corpse lost the ability to kill, therefore is not a killer anymore.
There are 4 killers in the room. No matter if they are dead or alive. The initial premise is clearly saying: There are 3 killers in the room. A person doesn't change to be a killer simply because he was killed. For example, two gunman on a duel kill each other, now they are two dead gunman. 👍🙏💥
Did you tell the model that "a person doesn't change to be a killer simply because he was killed"? Otherwise, you did not provide all the information needed to get the answer you want from it.
And the right answer is 3.
Here's how 70B answers it: https://huggingface.co/MaziyarPanahi/WizardLM-Math-70B-v0.1
There are 4 killers in the room. No matter if they are dead or alive. The initial premise is clearly saying: There are 3 killers in the room. A person doesn't change to be a killer simply because he was killed. For example, two gunman on a duel kill each other, now they are two dead gunman. 👍🙏💥
Did you tell the model that "a person doesn't change to be a killer simply because he was killed"? Otherwise, you did not provide all the information needed to get the answer you want from it.
And the right answer is 3.
Here's how 70B answers it: https://huggingface.co/MaziyarPanahi/WizardLM-Math-70B-v0.1
And these are the questions Matthew Berman use since 2022 for testing all LLMs on his YouTube channel. 🙏💥
Some LLMs deal this question with a bit of intelligence. Some say, for instance, that it would be 4, if we consider the dead killer, and 3 if we don't consider the dead one, which is a good approach. since as stated: Nobody left the room. Nobody is talking about buried people, but something that just happened in a room where nobody left the crime scene. 👍
And these are the questions Matthew Berman use since 2022 for testing all LLMs on his YouTube channel. 🙏💥
Ah yes, Matthew Berman the YouTube clickbaiter. I'd rather listen to what real AI enthusiasts think than people who latch on to this hobby for an easy profit.
And these are the questions Matthew Berman use since 2022 for testing all LLMs on his YouTube channel. 🙏💥
Ah yes, Matthew Berman the YouTube clickbaiter. I'd rather listen to what real AI enthusiasts think than people who latch on to this hobby for an easy profit.
Oh really ? I have no guilt when I opened the link and saw all Matthew Berman questions there. So, if he is so bad like you say, please enlighten me about these Gods of AI you are referring to, may be I know some of them and pass to praise them too. 😃👍
Can someone be a dead killer? It depends entirely on the context. I think most of us can agree that the best answer would note the ambiguity of the question.
And these are the questions Matthew Berman use since 2022 for testing all LLMs on his YouTube channel. 🙏💥
Ah yes, Matthew Berman the YouTube clickbaiter. I'd rather listen to what real AI enthusiasts think than people who latch on to this hobby for an easy profit.
I kid you not, he is so bad at testing. Once he said "I don't know if this is the right model but lets test it anyway"
Can someone be a dead killer? It depends entirely on the context. I think most of us can agree that the best answer would note the ambiguity of the question.
The guy who shot Donald Trump yesterday is now a dead killer. 👍💥