One more bump before Black Friday . <| sep |> you ' re right , Im an idiot lol <| sep |> The American Akita should be called Akita Ken and the Japanese Akita should be called Akita Ryu . <| sep |> I still listen to the soundtrack few months after playing . Beautiful . <| sep |> One claim he makes is Google ' s translation engine is ' trained ' on corpora ranging from news sources to Wikipedia . The bare description of each corpus is the only indication of the context from which it arises . From such scanty information it would be difficult to infer the appropriateness or inappropriateness of a word such as ' shitloads '. A human speaks maybe 25 , 000 words per day , and maybe hears much more , only a fraction of which are actually processed . So let ' s say that 50 , 000 words are either generated or heard each day by the average adult human . Over a span of 10 years , that amounts to 182 , 500 , 000 words . Yet , the corpora used to train productionscale translation models is orders of magnitude larger . Even if you account for the other kinds of information that a person is subjected to on a daily basis , it still pales in comparison . Granted , these neural net models aren ' t as efficient at learning language as humans but there is no reason to think they won ' t do much better than they do now . Computer languages , on the other hand , have contextfree grammars . The young Dutchman , however , lacked the social experience with English to grasp the subtler rules that shape the native speaker ' s diction , tone and structure . And neural nets don ' t learn just contextfree grammars , either . They are capable of learning phrase structure grammars , which are capable of representing TuringComplete computation . Now , the training algorithms may not be able to learn every such grammar , due to complexity constraints , and maybe also model constraints but existing ones can learn more than just contextfree grammars . The problem , as with all previous attempts to create artificial intelligence AI going back to my student days at MIT , is that intelligence is incredibly complex . To be intelligent is not merely to be capable of inferring logically from rules or statistically from regularities . Before that , one has to know which rules are applicable , an art requiring awareness of sensitivity to situation . Programmers are very clever , but they are not yet clever enough to anticipate the vast variety of contexts from which meaning emerges . Hence even the best algorithms will miss things and as Henry James put it , the ideal translator must be a person ' on whom nothing is lost '. For all we know , logic and statistical inference go all the way down that that is all the brain does . Also , it ' s not really accurate to say that neural machine translation works based on statistical pattern matching . The way it actually works is that statistics is used to train a model but once trained , there is no statistics anymore what remains is a program , encoded in a neural net . The program doesn ' t contain a little table of probabilities , or compute regressions or anything like that . A way to think of training a model is to first fit a regression line to a set of data points . Once that line is obtained , you can throw away all the data used to produce it you just need the slope and yintercept . That line encodes a little program . It says that given a certain value x , the expected value of y should be ax b . Now , there are priors that set the outlines of the model . One prior in the above is that a line is a good fit for the data . Maybe a parabola would be a better model to use , for example . But there ' s no reason to believe that the priors for producing a strong machine translation system need be that complex maybe just a deep LSTM network of some kind , trained on lots of data , can do a pretty good job , and even handle multisentence context . As I ' ve said before , progress in AI seems to be progressing in the following way right now somebody hears about a Deep Learning approach to a problem , and says , It can ' t possibly solve that ! And then it does . And then they say , Ok , well , but it doesn ' t solve this aspect of the problem . We are nowhere near to solving that . And then it does . And then , Ok , but that ' s not really the real problem here . Here ' s another problem , and I ' m sure it will never solve that . You have to basically have a humanlevel AI to solve that one like winning at Jeopardy ! And it does , and still doesn ' t reach humanlevel AI in achieving the goal . A good example is driverless cars First , it was claimed they could never work , as it requires common sense to make left turns . After they won the Urban Grand Challenge , it was claimed that the approach would never work in the real world , since you need complex maps . There are shadows on the road , and some roads don ' t have lane markers . Then neural netbased selfdriving cars showed an aptitude at handling shadows and roads without lane markers . Skeptics then pounced on the long tail you might get them to drive ok on these reasonably wellbehaved situations without maps but they will never be able to achieve level 4 or level 5 autonomy . Then NVIDIA and TESLA released videos of their latest work on training neural nets to drive and TESLA has announced they will have a crosscountry 100 % automated drive from California to Times Square in 2017 . And what will happen if Elon Musk succeeds in 2017 with his demo ? Skeptics will probably double down well , but what about Trolley Problems ? Who do you decide to save in an accident ? What about how human aggressiveness is a part of driving ? you have to act like you mean business , and driverless cars will never be able to imitate that , as it requires social understanding .