<story><title>A/B testing gets misused to juice metrics in the short term</title><url>https://www.zumsteg.net/2022/07/05/unchecked-ab-testing-destroys-everything-it-touches/</url></story><parent_chain><item><author>hbn</author><text>I&amp;#x27;ve talked about this on HN but I&amp;#x27;ll say it again.&lt;p&gt;A couple years ago, after being an Android fan for the better part of a decade, I finally bought myself an iPhone and pried myself away from Google&amp;#x27;s ecosystem wherever I could. And Apple didn&amp;#x27;t even need to do any work for me to make this decision. It was the years of abuse from Google that you experience when you decide to use a Google product or service. And a big part of that was the constant A&amp;#x2F;B&amp;#x2F;C&amp;#x2F;D&amp;#x2F;E&amp;#x2F;F testing. I never felt like I was using a complete product, everything felt like a constant beta that could be changed or rearranged at any point, and I was just doing free testing work for them while they harvest all my data.&lt;p&gt;Every app update was a risk of the app rearranging itself, or features appearing&amp;#x2F;disappearing. Eventually it didn&amp;#x27;t even come from app updates in the Play Store, and new interfaces would just appear one day when a server somewhere marked your account as being in the group that gets the new UI. This app that you were familiar with could at any point be rearranged when you open it on any given day. Then maybe a week later you open it and it&amp;#x27;s back to how it was before. A button you thought was here suddenly isn&amp;#x27;t, and you question whether something actually changed or if you&amp;#x27;re losing your mind. It&amp;#x27;s a subtle gaslighting that eventually I couldn&amp;#x27;t stand any more.&lt;p&gt;To me, A&amp;#x2F;B testing means you don&amp;#x27;t respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. That&amp;#x27;s not to say a company like Apple is creating products out of the goodness of their heart, but at least it feels like it was developed by humans who made an opinionated call as to what they thought was the right design decision, and what they would want to use. And in my 2 years of owning an iPhone, I&amp;#x27;ve never opened my reminders app to find out that it&amp;#x27;s completely unrecognizable, or my messages app has been renamed or rethemed for the umpteenth time.</text></item></parent_chain><comment><author>costcofries</author><text>&amp;quot;To me, A&amp;#x2F;B testing means you don&amp;#x27;t respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. &amp;quot;&lt;p&gt;Your perspective is extremely short-sighted. A&amp;#x2F;B testing can result in this type of behaviour but that&amp;#x27;s just poor A&amp;#x2F;B testing. Good A&amp;#x2F;B testing focuses on removing distractions from the experience and helping users derive more value from the product. Bad A&amp;#x2F;B testing tries to make things more discoverable, where discoverability is often just noise and distractions. Good A&amp;#x2F;B testing ensures that the money machine, as you put it, pays its dues to users by making the product experience delightful.</text></comment>
<story><title>A/B testing gets misused to juice metrics in the short term</title><url>https://www.zumsteg.net/2022/07/05/unchecked-ab-testing-destroys-everything-it-touches/</url></story><parent_chain><item><author>hbn</author><text>I&amp;#x27;ve talked about this on HN but I&amp;#x27;ll say it again.&lt;p&gt;A couple years ago, after being an Android fan for the better part of a decade, I finally bought myself an iPhone and pried myself away from Google&amp;#x27;s ecosystem wherever I could. And Apple didn&amp;#x27;t even need to do any work for me to make this decision. It was the years of abuse from Google that you experience when you decide to use a Google product or service. And a big part of that was the constant A&amp;#x2F;B&amp;#x2F;C&amp;#x2F;D&amp;#x2F;E&amp;#x2F;F testing. I never felt like I was using a complete product, everything felt like a constant beta that could be changed or rearranged at any point, and I was just doing free testing work for them while they harvest all my data.&lt;p&gt;Every app update was a risk of the app rearranging itself, or features appearing&amp;#x2F;disappearing. Eventually it didn&amp;#x27;t even come from app updates in the Play Store, and new interfaces would just appear one day when a server somewhere marked your account as being in the group that gets the new UI. This app that you were familiar with could at any point be rearranged when you open it on any given day. Then maybe a week later you open it and it&amp;#x27;s back to how it was before. A button you thought was here suddenly isn&amp;#x27;t, and you question whether something actually changed or if you&amp;#x27;re losing your mind. It&amp;#x27;s a subtle gaslighting that eventually I couldn&amp;#x27;t stand any more.&lt;p&gt;To me, A&amp;#x2F;B testing means you don&amp;#x27;t respect your users. You see them as just one factor in your money machine that can be poked and prodded to optimize how much money you can squeeze out of them. That&amp;#x27;s not to say a company like Apple is creating products out of the goodness of their heart, but at least it feels like it was developed by humans who made an opinionated call as to what they thought was the right design decision, and what they would want to use. And in my 2 years of owning an iPhone, I&amp;#x27;ve never opened my reminders app to find out that it&amp;#x27;s completely unrecognizable, or my messages app has been renamed or rethemed for the umpteenth time.</text></item></parent_chain><comment><author>aceazzameen</author><text>You nailed it. Google is constantly forcing users to relearn most of their products year after year. Give me Google products from a decade ago and I&amp;#x27;d still be happy. Now I&amp;#x27;m moving on from Google also. It&amp;#x27;s an untrustworthy brand.</text></comment>