foreman-installer on wheezy fails

Question: I'm trying to install foreman on a Debian Wheezy machine. When I follow the guide on theforeman.org, I get the following error from the installer:
<code>root@server:/tmp# foreman-installer
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead.
 Removing mount files: /etc/puppet/files does not exist or is not a directory
 /Stage[main]/Foreman::Plugin::Setup/Foreman::Plugin[setup]/Package[ruby-foreman-setup]/ensure: change from absent to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install ruby-foreman-setup' returned 100: Reading package lists...
Installing             Done                                               [100%] [....................................................]
  Something went wrong! Check the log for ERROR-level output
  The full log is at /var/log/foreman-installer/foreman-installer.log
</code>
The /var/log/foreman-installer/foreman-installer.log file doesn't show anything more useful:
<code># grep ERROR /var/log/foreman-installer/foreman-installer.log
[ERROR 2014-05-27 11:35:11 main]  Removing mount files: /etc/puppet/files does not exist or is not a directory
[ERROR 2014-05-27 11:35:33 main]  /Stage[main]/Foreman::Plugin::Setup/Foreman::Plugin[setup]/Package[ruby-foreman-setup]/ensure: change from absent to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install ruby-foreman-setup' returned 100: Reading package lists...
</code>
When I run the apt-get command with debugging on I get this:
<code># DEBUG=1 /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install ruby-foreman-setup
Reading package lists...
Building dependency tree...
Reading state information...
ruby-foreman-setup is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up ruby-foreman-setup (2.0.4) ...
+ . /usr/share/debconf/confmodule
+ [ !  ]
+ PERL_DL_NONLAZY=1
+ export PERL_DL_NONLAZY
+ [  ]
+ exec /usr/share/debconf/frontend /var/lib/dpkg/info/ruby-foreman-setup.postinst configure
+ . /usr/share/debconf/confmodule
+ [ ! 1 ]
+ [ -z  ]
+ exec
+ [  ]
+ exec
+ DEBCONF_REDIR=1
+ export DEBCONF_REDIR
+ LOGFILE=/var/log/foreman-install.log
+ PLUGIN=foreman_bootdisk
+ trap db_stop EXIT
+ cd /usr/share/foreman
+ [ ! -z 1 ]
+ bundle show foreman_bootdisk
+ bundle install
Fetching gem metadata from https://rubygems.org/
Query List: ["rails", "json", "rest-client", "audited-activerecord", "will_paginate", "ancestry", "scoped_search", "net-ldap", "uuidtools", "apipie-rails", "rabl", "oauth", "deep_cloneable", "foreigner", "safemode", "ruby_parser", "fog", "fog-core", "unf", "foreman_bootdisk", "pg", "rbvmomi", "fast_gettext", "gettext_i18n_rails", "gettext_i18n_rails_js", "i18n_data", "foreman_setup"]
Query Gemcutter Dependency Endpoint API: rails json rest-client audited-activerecord will_paginate ancestry scoped_search net-ldap uuidtools apipie-rails rabl oauth deep_cloneable foreigner safemode ruby_parser fog fog-core unf foreman_bootdisk pg rbvmomi fast_gettext gettext_i18n_rails gettext_i18n_rails_js i18n_data foreman_setup
Fetching from: https://rubygems.org/api/v1/dependencies?gems=rails,json,rest-client,audited-activerecord,will_paginate,ancestry,scoped_search,net-ldap,uuidtools,apipie-rails,rabl,oauth,deep_cloneable,foreigner,safemode,ruby_parser,fog,fog-core,unf,foreman_bootdisk,pg,rbvmomi,fast_gettext,gettext_i18n_rails,gettext_i18n_rails_js,i18n_data,foreman_setup
HTTP Success
Query List: ["activerecord", "audited", "jeweler", "appraisal", "sqlite3", "fog-brightbox", "nokogiri", "fog-json", "net-ssh", "net-scp", "mime-types", "multi_json", "formatador", "excon", "builder", "ruby-hmac", "unicode", "named-parameters", "google-api-client", "deface", "gettext", "po_to_json", "activesupport", "hoe", "newgem", "actionpack", "rack", "rake-compiler", "bundler", "railties", "actionmailer", "activeresource", "sprockets-rails", "activemodel", "actionview", "rake", "actionwebservice", "trollop", "cdb-full", "tokyocabinet", "ffi", "netrc", "rdoc", "sexp_processor", "ParseTree", "ruby2ruby", "unf_ext", "rspec", "launchy"]
Query Gemcutter Dependency Endpoint API: activerecord audited jeweler appraisal sqlite3 fog-brightbox nokogiri fog-json net-ssh net-scp mime-types multi_json formatador excon builder ruby-hmac unicode named-parameters google-api-client deface gettext po_to_json activesupport hoe newgem actionpack rack rake-compiler bundler railties actionmailer activeresource sprockets-rails activemodel actionview rake actionwebservice trollop cdb-full tokyocabinet ffi netrc rdoc sexp_processor ParseTree ruby2ruby unf_ext rspec launchy
Fetching from: https://rubygems.org/api/v1/dependencies?gems=activerecord,audited,jeweler,appraisal,sqlite3,fog-brightbox,nokogiri,fog-json,net-ssh,net-scp,mime-types,multi_json,formatador,excon,builder,ruby-hmac,unicode,named-parameters,google-api-client,deface,gettext,po_to_json,activesupport,hoe,newgem,actionpack,rack,rake-compiler,bundler,railties,actionmailer,activeresource,sprockets-rails,activemodel,actionview,rake,actionwebservice,trollop,cdb-full,tokyocabinet,ffi,netrc,rdoc,sexp_processor,ParseTree,ruby2ruby,unf_ext,rspec,launchy
HTTP Success
Query List: ["RubyInline", "SexpProcessor", "erubis", "archive-tar-minitar", "rcov", "sprockets", "thor", "aruba", "RedCloth", "syntax", "rubigen", "cucumber", "retriable", "jwt", "extlib", "faraday", "autoparse", "addressable", "signet", "httpadapter", "sinatra", "liquid", "colorize", "spoon", "configuration", "rspec-mocks", "rspec-expectations", "rspec-core", "diff-lcs", "spicycode-rcov", "test-spec", "camping", "fcgi", "memcache-client", "mongrel", "ruby-openid", "thin", "text", "locale", "levenshtein", "minitest", "rubyforge", "gemcutter", "mini_portile", "racc", "tenderlove-frex", "rexical", "weakling", "highline", "github_api", "git", "thoughtbot-shoulda", "mhennemeyer-output_catcher", "rr", "mocha", "redgreen", "peterwald-git", "schacon-git", "bones", "needle", "jruby-pageant", "echoe", "rails-observers", "mail", "text-format", "i18n", "bcrypt-ruby", "tzinfo", "arel", "activerecord-deprecated_finders", "journey", "rack-test", "rack-cache", "rack-mount", "rack-ssl", "thread_safe"]
Query Gemcutter Dependency Endpoint API: RubyInline SexpProcessor erubis archive-tar-minitar rcov sprockets thor aruba RedCloth syntax rubigen cucumber retriable jwt extlib faraday autoparse addressable signet httpadapter sinatra liquid colorize spoon configuration rspec-mocks rspec-expectations rspec-core diff-lcs spicycode-rcov test-spec camping fcgi memcache-client mongrel ruby-openid thin text locale levenshtein minitest rubyforge gemcutter mini_portile racc tenderlove-frex rexical weakling highline github_api git thoughtbot-shoulda mhennemeyer-output_catcher rr mocha redgreen peterwald-git schacon-git bones needle jruby-pageant echoe rails-observers mail text-format i18n bcrypt-ruby tzinfo arel activerecord-deprecated_finders journey rack-test rack-cache rack-mount rack-ssl thread_safe
Fetching from: https://rubygems.org/api/v1/dependencies?gems=RubyInline,SexpProcessor,erubis,archive-tar-minitar,rcov,sprockets,thor,aruba,RedCloth,syntax,rubigen,cucumber,retriable,jwt,extlib,faraday,autoparse,addressable,signet,httpadapter,sinatra,liquid,colorize,spoon,configuration,rspec-mocks,rspec-expectations,rspec-core,diff-lcs,spicycode-rcov,test-spec,camping,fcgi,memcache-client,mongrel,ruby-openid,thin,text,locale,levenshtein,minitest,rubyforge,gemcutter,mini_portile,racc,tenderlove-frex,rexical,weakling,highline,github_api,git,thoughtbot-shoulda,mhennemeyer-output_catcher,rr,mocha,redgreen,peterwald-git,schacon-git,bones,needle,jruby-pageant,echoe,rails-observers,mail,text-format,i18n,bcrypt-ruby,tzinfo,arel,activerecord-deprecated_finders,journey,rack-test,rack-cache,rack-mount,rack-ssl,thread_safe
HTTP Success
Query List: ["text-hyphen", "ZenTest", "termios", "metaclass", "english", "rspec-support", "multipart-post", "tlsmail", "treetop", "daemons", "fastthread", "gem_plugin", "cgi_multipart_eof_fix", "rack-protection", "tilt", "shotgun", "backports", "multimap", "multi_test", "gherkin", "term-ansicolor", "polyglot", "json_pure", "hike", "bcrypt", "ruby-yadis", "childprocess", "background_process", "bcat", "rdiscount", "flexmock", "allison", "abstract", "markaby", "metaid", "mab", "descendants_tracker", "oauth2", "hashie", "eventmachine", "eventmachine-le", "http_parser.rb", "preforker", "loquacious", "little-plugger", "bones-git", "bones-extras", "atomic"]
Query Gemcutter Dependency Endpoint API: text-hyphen ZenTest termios metaclass english rspec-support multipart-post tlsmail treetop daemons fastthread gem_plugin cgi_multipart_eof_fix rack-protection tilt shotgun backports multimap multi_test gherkin term-ansicolor polyglot json_pure hike bcrypt ruby-yadis childprocess background_process bcat rdiscount flexmock allison abstract markaby metaid mab descendants_tracker oauth2 hashie eventmachine eventmachine-le http_parser.rb preforker loquacious little-plugger bones-git bones-extras atomic
Fetching from: https://rubygems.org/api/v1/dependencies?gems=text-hyphen,ZenTest,termios,metaclass,english,rspec-support,multipart-post,tlsmail,treetop,daemons,fastthread,gem_plugin,cgi_multipart_eof_fix,rack-protection,tilt,shotgun,backports,multimap,multi_test,gherkin,term-ansicolor,polyglot,json_pure,hike,bcrypt,ruby-yadis,childprocess,background_process,bcat,rdiscount,flexmock,allison,abstract,markaby,metaid,mab,descendants_tracker,oauth2,hashie,eventmachine,eventmachine-le,http_parser.rb,preforker,loquacious,little-plugger,bones-git,bones-extras,atomic
HTTP Success
Query List: ["tins", "spruz", "facets", "language", "escape_utils", "bones-rcov", "bones-rubyforge", "bones-rspec", "bones-zentest", "multi_xml", "httpauth", "faraday-middleware"]
Query Gemcutter Dependency Endpoint API: tins spruz facets language escape_utils bones-rcov bones-rubyforge bones-rspec bones-zentest multi_xml httpauth faraday-middleware
Fetching from: https://rubygems.org/api/v1/dependencies?gems=tins,spruz,facets,language,escape_utils,bones-rcov,bones-rubyforge,bones-rspec,bones-zentest,multi_xml,httpauth,faraday-middleware
HTTP Success
Query List: []
Unmet Dependencies: ["faraday-middleware", "spicycode-rcov", "tenderlove-frex", "mhennemeyer-output_catcher", "peterwald-git", "schacon-git"]
Fetching gem metadata from https://rubygems.org/
Query List: ["faraday-middleware", "spicycode-rcov", "tenderlove-frex", "mhennemeyer-output_catcher", "peterwald-git", "schacon-git"]
Query Gemcutter Dependency Endpoint API: faraday-middleware spicycode-rcov tenderlove-frex mhennemeyer-output_catcher peterwald-git schacon-git
Fetching from: https://rubygems.org/api/v1/dependencies?gems=faraday-middleware,spicycode-rcov,tenderlove-frex,mhennemeyer-output_catcher,peterwald-git,schacon-git
HTTP Success
Query List: []
Bundler could not find compatible versions for gem "nokogiri":
  In Gemfile:
    foreman_setup (= 2.0.4) ruby depends on
      nokogiri (~> 1.4.3) ruby

    rbvmomi (~> 1.6.0) ruby depends on
      nokogiri (1.6.1)

/usr/lib/ruby/vendor_ruby/bundler/resolver.rb:129:in `block in resolve'
/usr/lib/ruby/vendor_ruby/bundler/resolver.rb:127:in `catch'
/usr/lib/ruby/vendor_ruby/bundler/resolver.rb:127:in `resolve'
/usr/lib/ruby/vendor_ruby/bundler/definition.rb:192:in `resolve'
/usr/lib/ruby/vendor_ruby/bundler/definition.rb:127:in `specs'
/usr/lib/ruby/vendor_ruby/bundler/definition.rb:122:in `resolve_remotely!'
/usr/lib/ruby/vendor_ruby/bundler/installer.rb:48:in `run'
/usr/lib/ruby/vendor_ruby/bundler/installer.rb:12:in `install'
/usr/lib/ruby/vendor_ruby/bundler/cli.rb:220:in `install'
/usr/lib/ruby/vendor_ruby/bundler/vendor/thor/task.rb:22:in `run'
/usr/lib/ruby/vendor_ruby/bundler/vendor/thor/invocation.rb:118:in `invoke_task'
/usr/lib/ruby/vendor_ruby/bundler/vendor/thor.rb:263:in `dispatch'
/usr/lib/ruby/vendor_ruby/bundler/vendor/thor/base.rb:386:in `start'
/usr/bin/bundle:13:in `<main>'
+ [ -f /usr/share/foreman/config/database.yml ]
+ [ ! -z 1 ]
+ /usr/sbin/foreman-rake db:migrate
Could not find gem 'foreman_setup (= 2.0.4) ruby' in the gems available on this machine.
Run `bundle install` to install missing gems.
+ true
+ chown -Rf foreman:foreman /usr/share/foreman
+ exit 0
+ db_stop
+ echo STOP
</code>
I'm not that familiar with ruby. Any idea howto fix this, so that the installer can complete the foreman install?
Comment: do you have puppet installed, if so where? And is your puppet without a master?
Comment: the foreman-installer took care of the puppet install. So yes, there is a plain puppet install on the system. I just followed: http://theforeman.org/manuals/1.5/quickstart_guide.html
Answer: At the end, this turned out to be a proxy problem. After setting the enviroment variables http_proxy and https_proxy everything went fine. Up to this point I only had apt configured to use the proxy.

foreman-installer on wheezy fails


jQuery text animate help. Stop at last word

Question: I found this code and it works great! But
how do I stop it when it reaches the last word?
<code>(function(){

    // List your words here:
    var words = [
        'awesome',
        'incredible',
        'cool',
        'fantastic'
        ], i = 0;

    setInterval(function(){
        $('#changerificwordspanid').fadeOut(function(){
            $(this).html(words[i=(i+1)%words.length]).fadeIn();
        });
       // 2 seconds
    }, 2000);

})();
</code>
Answer: before you start the code for the animation check if there is a word in the array and if not stop the interval.
<code>(function(){

// List your words here:
var words = [
    'awesome',
    'incredible',
    'cool',
    'fantastic'
    ], i = 0;

var interval = setInterval(function(){
    if(i+1 < words.length) {
        $('#changerificwordspanid').fadeOut(function(){
            $(this).html(words[i=(i+1)%words.length]).fadeIn();
        });
    } else {
        window.clearIntervall(interval);
    }
   // 2 seconds
}, 2000);
</code>
})();
clear interval can be found here:
http://de.selfhtml.org/javascript/objekte/window.htm#clear_interval
Answer: You can avoid to interval by do this, It's will be continue adding word smoothly.
<code>(function(){

// List your words here:
var words = [
    'awesome',
    'incredible',
    'cool',
    'fantastic'
    ], i = 0;

(function loop(){
    $('#changerificwordspanid').fadeOut(function(){
        // send the loop function to fadeIn, It will exec when fadeIn complete
        $(this).html(words.shift()).fadeIn(loop);
    });
})();

})();
</code>
If you also need 2 sec delay, after each fadeIn effect completed. do this
<code>(function(){

// List your words here:
var words = [
    'awesome',
    'incredible',
    'cool',
    'fantastic'
    ], i = 0;

(function loop(){
    $('#changerificwordspanid').fadeOut(function(){
        // settimeout for 2 sec before called loop
        $(this).html(words.shift()).fadeIn(function(){
           setTimeout(loop, 2000);
        });
    });
})();

})();
</code>


Illegal access: this web application instance has been stopped already. Could not load org.apache.log4j.spi.NOPLoggerRepository

Question: I am using Netbeans 6.7 with Tomcat 6.0 and jasper reports for generating reports in a web application. I always get this exception when I run my application. can someone please help me out?
The exception is
INFO: Illegal access: this web application instance has been stopped already.  Could not load org.apache.log4j.spi.NOPLoggerRepository.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1566)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
        at org.apache.log4j.LogManager.getLoggerRepository(LogManager.java:175)
        at org.apache.log4j.LogManager.getLogger(LogManager.java:199)
        at org.apache.log4j.Logger.getLogger(Logger.java:105)
        at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:283)
        at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:193)
        at org.hibernate.connection.DriverManagerConnectionProvider.close(DriverManagerConnectionProvider.java:147)
        at org.hibernate.connection.DriverManagerConnectionProvider.finalize(DriverManagerConnectionProvider.java:142)
        at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
        at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
        at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)

Comment: http://stackoverflow.com/questions/how-to-ask
Answer: It could be due to a memory problem, I had this message and also a
java.lang.OutOfMemoryError: Java heap space, adjusting startup parameters to:
-XX:PermSize=512M -Xms1024m -Xmx4096m solved the problem the problem.
Answer: I found I would get a similar error occasionally while running my JSP web application with Netbeans on Tomcat and always while debugging it. In my case it was "could not load Java.lang.math or Oracle.lnx.lib".  A coworker had me undeploy the web application instances from tomcat and it solved the problem.  To undeploy a webapplication instance, go to the Services Tab > Servers and then double click on Apache Tomcat or TomEE and after running the application once since opening netbeans,  a Web Applications Folder should appear.  If you double click on that, you should see instances of your application appear.  You should be able to right click them and then click undeploy (you may have to right click them and hit stop first).  Finally, you should stop and then start tomcat before building and running your application.
Hope this helps.


Standard error of a proportion on a 2 x 2 contingency table

Question: Why on a 2 x 2 table such as

where $p = \frac{a}{a+b}$
the standard error of $p$ is
$$\left(\frac{(a+c)(b+d)}{(a+b+c+d)^2(a+b)
}\right)^{1/2}$$
?
Answer: Under the null hypothesis of no difference, the common rate estimator is $p_c=(a+c)/N$, where $N=a+b+c+d$. For $p=\frac a {a+b}$, the variance estimate is $\frac {p_c(1-p_c)}{a+b}= \frac{(a+c)(b+d)}{(a+b+c+d)^2(a+b)}$ because $Var(\frac XN) = \frac{\pi (1-\pi)}N $for binomial distribution $X\sim B(n,\pi)$. The square root of the estimated variance is called standard error.
Comment: The 'common rate estimator' is the "probability of Success" in the binomial with the column $a+c$ being the Success column? And $p=\frac{a}{a+b}$ is the probability of Success among the first row, for example, corresponding to subjects treated with a specific drug?
Comment: Yes to both questions.


Which class to define as positive in unbalanced classification

Question: In a classification task, why do we usually choose the minority class as "positive" case for response variable? For example, if there are 1000, 9000 for class A and B respectively, we usually define class A as 1 and class B as 0, thus the response variable can have average as 0.1 instead of 0.9. If the response variable has average like 0.6 or 0.7, will there be any issues for the classification algorithm? Do we need to choose the class with average 0.4 or 0.3 as positive class? Thanks in advance!
Answer: It does not matter. It is done that way because it would be something like disease detection where most people don't have the disease, but that's what you want to catch. However, all of the math works out fine either way. If you made the heathy class the $1$s, your model would be outputting the probability of being healthy instead of the probability of having the disease. The English phrasing of those two might carry different connotations, but the mathematical logic is the same.
Comment: Statistics has a different take on class imbalance than you might see many other places, however.
https://stats.stackexchange.com/questions/357466/are-unbalanced-datasets-problematic-and-how-does-oversampling-purport-to-he
https://www.fharrell.com/post/class-damage/
https://www.fharrell.com/post/classification/
https://stats.stackexchange.com/a/359936/247274
https://stats.stackexchange.com/questions/464636/proper-scoring-rule-when-there-is-a-decision-to-make-e-g-spam-vs-ham-email
https://twitter.com/f2harrell/status/1062424969366462473?lang=en
Comment: Will there be any numerical approximation issue in the computing process?
Comment: There's always the possibility of numerical issues when you do math on a computer.
Answer: @Dave's answer is that all classification algorithms are symmetrical in regard to which class is called positive and which is called negative, or which is assigned to 0 and which to 1. This is probably true in almost all cases, but maybe one. It is true for decision trees and ensembles of decision trees since the classes are treated as categorical variables, as is for KNN, naive Bayes and so on. The two families of classifier algorithms that threat the classes as numbers, and therefore may be non-symmetrical in relation to the values 0 and 1 are logistic regression and SVM, but one can verify that the formulas are symmetrical, and thus no problem there.
But there is one case which I think may not be symmetrical regarding 2nd order aspects. I am not 100% sure and would appreciate if more knowledgeable people would chip in this discussion. Consider a neural network (MLP for example) optimizing log loss.
$$ log loss = \sum [y_i \log p_i + (1-y_i) \log (1-p_i)] $$
$y_i$ is the correct output, $p_i$ is the predicted probability.
The formula is symmetric by exchanging the y_i from 0 to 1 and vice versa, BUT the optimization being computed by the neural network is not (this is the 2nd order aspect). Output with $y_i = 0$ do not contribute to the log loss and thus do not contribute to the gradient and this it is not used in the gradient descent of the neural network. Therefore, by using the majority class 0 one is not using most of the data to optimize the MLP! The minimum is the same but a) if the gradient descent will reach that minimum, b)  if it will the convergence rate to that minimum and c) the solution the MLP will settle given that it does not reach the global minimum are not the same whether one uses 0 or 1 for the majority class.
Am I right here?
Comment: This is incorrect. Switching the labels means that the "other" of the $y\log$ and $(1-y)\log$ terms drops to zero, but the reason for having both is to turn one on while leaving one off, specifically so both categories are considered. // Since this really is a distinct question, perhaps you can delete this and post as a question. If you follow this comment, you might even consider writing a self-answer.


Erro durante Excução da página - Xamarin

Question: Sou novato em programação mobile, por isso queria um help para tratar um erro que parece que não foi definido pela IDE (Visual Studio).
O caso é que tenho uma tela a qual queria inserir aquele carrossel com cards de promoção (algo semelhante à página inicial da store do google). Eis que durante os testes, o VS não acusa erro no código mas na compilação aparece o problema.
Eu coloquei uma imagem com print do erro. Obrigado a todos!

XAML :
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="JetSolutionsApp.Views.Aba.Parceiros">

        <StackLayout>
        <CarouselView x:Name="slideTopo">
            <CarouselView.ItemTemplate>
                <DataTemplate>

                </DataTemplate>
            </CarouselView.ItemTemplate>
        </CarouselView>
    </StackLayout>
</ContentPage>

Code :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using Xamarin.Forms;
using Xamarin.Forms.Xaml;

namespace JetSolutionsApp.Views.Aba
{
    [XamlCompilation(XamlCompilationOptions.Compile)]
    public partial class Parceiros : ContentPage
    {
        public Parceiros()
        {
            InitializeComponent();

              var nomesCarousel = new List<string>
              {
                  "jose"," maria", "joaquim", "joão"
              };
            slideTopo.ItemsSource = nomesCarousel;

        }
    }
}

Este é o erro:
Comment: Tente usar ``
Answer: Experimente alterar o XAML do CarouselView para o seguinte:
<CarouselView x:Name="slideTopo" ItemsSource="{Binding .}">
    <CarouselView.ItemTemplate>
        <DataTemplate>

        </DataTemplate>
    </CarouselView.ItemTemplate>
</CarouselView>

Se não funcionar a solução acima, podemos tentar de outra forma:
C#
namespace JetSolutionsApp.Views.Aba
{
    internal List<string> Nomes { get; set; }

    [XamlCompilation(XamlCompilationOptions.Compile)]
    public partial class Parceiros : ContentPage
    {
        public Parceiros()
        {
            InitializeComponent();

            Nomes = new List<string>()
            {
                "josé",
                "maria",
                "joaquim",
                "joão"
            };

            slideTopo.ItemsSource = Nomes;
        }
    }
}

XAML
<CarouselView x:Name="slideTopo" ItemsSource="{Binding Nomes}">
    <CarouselView.ItemTemplate>
        <DataTemplate>

        </DataTemplate>
    </CarouselView.ItemTemplate>
</CarouselView>

Comment: Boa noite João, eu experimentei a alteração que me sugeriu mas o erro permanece. Eu procurei por erros de codificação mas não ha nada, apenas esta falha.
Comment: Resposta editada.


Why can't I estimate the Fundamental Matrix from a coplanar set of points?

Question: I am learning how to estimate the Fundamental Matrix via the 8-point algorithm:
From N > 8 corresponding points that satisfy $xFx' = 0$, we need to solve the LS problem:
$kron(x,x')f = 0$
Where $kron(x,x')$ is a N by 9 matrix made of the elements of the corresponding points, and $f$ is the matrix $F$ as a column vector.
However, I came a cross a statement that if the set of N corresponding points is coplanar, or lying on the same 3D plane, then the LS problem has no solution.
Can someone please explain why the fundamental matrix can't be estimated from a coplanar set of points? Both an algebraic and geometric explanation will be welcome.
Answer: In the book, Three-dimensional computer vision by Olivier Faugeras (Page 271) is explained why you can't have coplanar points.
Basically it's because the matrix $kron(x,x^\prime)$ has to be of rank more or equal to 8 to compute the inverse of the eight first columns of $kron(x,x^\prime)$ and recover the essential matrix (as 8x1 vector) as
$$ f=-F_{33}(U_8)^{-1}C_9 $$
Where $U_8$ represent the $n\times8$ matrix formed by the first eight columns of $kron(x,x^\prime)$ and $C_9$ is the ninth colimn vector of $kron(x,x^\prime)$.


Can several magnetic heads in a cylinder of one disk read or write simultaneously?

Question: For example, a disk has 8 surfaces, each has one magnetic head. If all but one heads disappeared, the disk can reach 10MB. What will the speed if using all heads?
Answer: Theoretically you should get 8*10MB/s=80MB. But this is kind of abstract speed because you need to have 8 information management blocks to get info from each head (not sure but IMHO there is only one and it switch between heads). And if your information is not organized based on cylinders and fill one track you will have you heads back and forth to combine info from different areas.
Answer: While there may be many read/write heads in a hard drive, only one of them can be active at any time.
One reason for that is that there is only one amplification/decoding circuit. In theory you could add more, but a more subtle reason preventing that is that HDDs use embedded servo information for the linear actuator.
The embedded servo aligns the single active head to its current track. Since there's only one actuator, only that one head is aligned. It's not possible to align all heads simultaneously due to manufacturing tolerances, thermal expansion and small deviations from perfectly circular tracks.
So, regardless of the number of heads, the sequential read/write speed of a rotating disk drive is ultimately defined by its recording density and rotational speed. (Unless there really are multiple actuators like in Exos 2X drives, or you are referring to ancient drives using stepper motors or dedicated servo heads.)
Comment: [Hard Drive Technology Timeline](https://datarecovery.com/rd/history-of-hard-drive-technologies/#:~:text=1965) indicates that all the heads were used simultaneously in the old technology. (That's also how I remember it working from when I worked with disks in the 1970s.)
Comment: In the 1970s and 80s with stepper technology that might have been true. But that''s more a topic for [retrocomputing.se].
Comment: I wonder how long it will be before *all* moving-part-storage becomes retro.


passing up headers from auth_request

Question: I have an authorization module which is called whenever a request is made to a private endpoint.
The module parses the token from the Authorization header, and:

if it's invalid, it returns 401
if it's valid, it returns 200
if it's valid but is about to expire in X minutes, it generates a new token and returns that one in the Authorization header.

"profile" is one of the private endpoints, and it's configured this way:
location /profile {
  auth_request /jwtverify;

  auth_request_set  $authorization $upstream_http_authorization;
  proxy_set_header  authorization $authorization;
  proxy_pass        http://private-profile:80;
}

jwtverify is configured this way:
location = /jwtverify {
  internal;
  proxy_pass              http://auth-module:8080/auth/verify;
  proxy_pass_request_body off;
  proxy_set_header        Content-Length "";
  proxy_set_header        X-Original-URI $request_uri;
  auth_request_set        $http_authorization $upstream_http_authorization;
}

Now, everything works except for requirement no. 3: if the auth module sets the Authorization header, the client never receives it.
The flow should be like this:

Client makes the request
auth-module intercepts the request and, if valid, the proxy passes it to the private service
When the response is sent, headers set by auth-module should be kept and sent to the client

I think I didn't understand properly how to combine auth_request_set, proxy_set_header, auth_request_set, it might also be that they aren't correct for this scenario.
Is there a way to accomplish this in NGINX?
Answer: Ok, I was able to do that with the help of the headers_more module.
The full configuration is:
location = /jwtverify {
  internal;
  proxy_pass              http://auth-module:8080/auth/verify;
  proxy_pass_request_body off;
  proxy_set_header        Content-Length "";
  proxy_set_header        X-Original-URI $request_uri;
}

location /profile {
  auth_request /jwtverify;

  # this gets called right after auth_request returns.
  # it reads http "authorization" header from upstream (= auth_request)
  # and sets it to the variable $auth_header
  # https://stackoverflow.com/a/31485557/1759845
  auth_request_set $auth_header $upstream_http_authorization;

  # this gets called right before sending response to client.
  # it adds the previously set variable (= "authorization"
  # header from auth_request) to the response
  more_set_headers "Authorization: $auth_header";

  proxy_pass        http://private-profile:80;
}

Comment: Was the blockage simply that you're trying to use the standard `Authorization` header to return the new token? If you used a private header, could this use `add_header` instead of `more_set_headers`?
Comment: @TBBle I honestly don't know. It would be a limitation though, as this specific header needs to be the standard `Authorization` header, so even if that was the issue I'd still be using `more_set_headers`
Comment: Thank you. I ask because I have a similar use-case, but am free to use a custom header for the return channel, while not being as-free to add non-standard modules to the system (in this case to the Kubernetes NGINX Ingress distribution). For anyone else in my situation, I found [a bug report on that distribution](https://github.com/kubernetes/ingress-nginx/issues/3716) that indicates it would work for `add_header` but other functionality such as `if` would not function correctly due to the operation of `proxy_pass`.
Answer: This is how I was able to solve this without a custom module:
location /auth {
  internal;
  proxy_pass http://localhost:3000/auth;
  proxy_pass_request_body off;
  proxy_set_header Content-Length "";
  proxy_set_header X-Original-URI $request_uri;
  proxy_set_header X-Original-Remote-Addr $remote_addr;
  proxy_set_header X-Original-Host $host;
}

location /protected {
  auth_request /auth;
  auth_request_set $authentication_id $sent_http_x_authentication_id;
  proxy_pass http://localhost:3000/protected;
  proxy_set_header X-Original-URI $request_uri;
  proxy_set_header X-Original-Remote-Addr $remote_addr;
  proxy_set_header X-Original-Host $host;
  proxy_set_header X-Authentication-Id: $authentication_id;
}

Have your /auth endpoint include a response header. Mine sets X-Authentication-Id
Use auth_request_set to set a variable based on the response header
Use the variable to set the header as part of the /protected request
Comment: Does this only add the header if the `auth_request` subrequest responds positively (200)? If you get a 403 then the header is not added, right?
Comment: your `/auth` endpoint is in control of that. If you don't want it, don't add it before you return a response.
Comment: This is the better answer IMHO.

passing up headers from auth_request


Value of differential equation at non existent points in the original function.

Question: Lets say I have a function.
$$y^2+y=x^3$$...
Thus, the differential eqn will be..
$$y' = \frac{3x^2}{2y+1}$$
What does the differential equation give at the points that doesnt lie in the original function?
Comment: @cineel In this case you have two branches for the "function". You can still find the derivative for each branch. Or if you want a single valued function, use $x$ as a function of $y$, and $y'=\frac1{dx/dy}$
Comment: $y^2+y=x^3$ has 2 values of y for some x, so y is not a function of x.
Answer: Generally speaking, the equation $f(y)=g(x)$ leads to multiple branches, where if both $f(\cdot )$ and $g(\cdot )$ are differentiable, then a derivation of $y$ w.r.t. $x$ can be defined as $y'f'(y)=g'(x)$. The latter derivative makes sense as long as the points lie on the branches of $f(y)=g(x)$ and is insignificant elsewhere. Also note that for example in your case, $y$ can be defined as a function of $x$ on each branch ($y=-1/2+\sqrt{x^3+1/4}$ on one branch and $y=-1/2-\sqrt{x^3+1/4}$ on another), and whether or not this function can be explicitly found, $y$ in $y'=\frac{3x^2}{2y+1}$ should be replaced by some $h(x)$, for which $$y'=\frac{3x^2}{2h(x)+1}.$$


ASP.NET MVC - Mock a Form Request

Question: I'm just learning ASP.NET MVC and I'm trying to create a mock form request for a unit test.
I'm using RhinoMocks.
I have looked at the following websites but cannot get these to work.
http://blog.maartenballiauw.be/post/2008/03/19/ASPNET-MVC-Testing-issues-Q-and-A.aspx
Update:
Controller Code:
    ///
    /// Creates a new entry
    ///
    /// <returns></returns>
    [AcceptVerbs(HttpVerbs.Post)]
    public ActionResult Create([Bind()]Person person)
    {
        if (Request.Form["DateOfBirth"].ToString() == "")
        {
            TempData["message"] = "Please select a date of Birth";
            ViewData["DateOfBirth"] = Request.Form["DateOfBirth"].ToString();

            MvcValidationAdapter.TransferValidationMessagesTo(ViewData.ModelState, person.ValidationMessages);
            return View();
        }
        else
        {

        if (person.IsValid())
        {
            person.DateOfBirth = Convert.ToDateTime(Request.Form["DateOfBirth"]);

            personRepository.SaveOrUpdate(person);
            TempData["message"] = person.Firstname + " was successfully added";
            return RedirectToAction("Create", "OrderDetails", new { id = person.ID });
        }
        else
        {

            ViewData["DateOfBirth"] = Request.Form["DateOfBirth"].ToString();

            MvcValidationAdapter.TransferValidationMessagesTo(ViewData.ModelState, person.ValidationMessages);
            return View();
        }

        }

    }

Answer: If you change the action method to have a FormCollection as the final parameter you can then pass in a FormCollection instance that contains all your values. The MVC framework will automatically pass in the values from the form within that parameter when running live.
public ActionResult MyMethod(FormCollection form)
{
    // in testing you will pass in a populated FormCollection object
    // at runtime the framework will populate the form parameter with
    // the contents of the posted form
}

Here is a reasonable example of it being used.
Edit
Have you tried this:
    ///
    /// Creates a new entry
    ///
    /// <returns></returns>
    [AcceptVerbs(HttpVerbs.Post)]
    public ActionResult Create([Bind()]Person person, FormCollection form)
    {
        if (form["DateOfBirth"].ToString() == "")
        {
            TempData["message"] = "Please select a date of Birth";
            ViewData["DateOfBirth"] = form["DateOfBirth"].ToString();

            MvcValidationAdapter.TransferValidationMessagesTo(
                ViewData.ModelState, person.ValidationMessages);
            return View();
        }
        else
        {

        if (person.IsValid())
        {
            person.DateOfBirth = Convert.ToDateTime(form["DateOfBirth"]);

            personRepository.SaveOrUpdate(person);
            TempData["message"] =
                person.Firstname + " was successfully added";
            return RedirectToAction(
                "Create", "OrderDetails", new { id = person.ID });
        }
        else
        {

            ViewData["DateOfBirth"] = form["DateOfBirth"].ToString();

            MvcValidationAdapter.TransferValidationMessagesTo(
                ViewData.ModelState, person.ValidationMessages);
            return View();
        }

        }

    }

Comment: Thanks, I still cannot get this to work. Is there any other examples?
Comment: Any chance of editing your question with some code? The action method and your test code perhaps?
Answer: You can also mock the form and I suggest you take a look at http://mvccontrib.codeplex.com/:
var form = new NameValueCollection();
form.Add("publish", "true");
_controller.Request.Stub(x => x.Form).IgnoreArguments().Return(form);
Comment: how did you initialize _controller?
Answer: Unless you're testing MVC itself, shouldn't you be mainly testing that the controller's action does the right thing with the arguments passed by the framework?
You can probably mock more indirect form access via:
controller.ActionInvoker.InvokeAction(ctx);

where ctx is a ControllerContext, with the form data etc. Here's an example using rhino to provide the context (MoQ also shown).

ASP.NET MVC - Mock a Form Request


Limit of several variables

Question: What would be the limit of the given multi-variable functions?
$$1. \lim_{(x,y) \to (0,0)} x \cdot sin \left(\frac{1}{y}\right)$$
$$2. \lim_{(x,y) \to (0,1)} \frac{e^{x}-y}{xy}$$
I tried to solve with x=0 and y=0 but in this this method didn't work. Is there any other way??
Thanks for the help.
Comment: Have you tried looking at the Taylor expansions of the functions?
Comment: hhhmm... that might work.. i will look at it...
Answer: For
$$1. \lim_{(x,y) \to (0,0)} x \cdot sin \left(\frac{1}{y}\right)$$,
$ |x\sin\frac{1}{y}|\leq |x|\rightarrow 0$ as $x\rightarrow 0$.
Answer: For
$$2. \lim_{(x,y) \to (0,0)} \frac{e^{x}-y}{xy}$$
The limit does not exist. Take for example $(\frac{1}{n}, \frac{1}{n})$ and $(\frac{-1}{n}, \frac{1}{n})$  and get that their limits are $+\infty$ and $-\infty$
Comment: i mistyped before.. sorry for that but y $\rightarrow$ 1
Answer: By applying  the taylor expansion of $e^x=1+x+x^2/2+...$, we have $\frac{e^x-y}{xy}=\frac{1+x+x^2/2+...-y}{xy}$ which has the same limit as $\frac{x}{xy}$, namely $1$ as $(x,y)\rightarrow 0$.  The limit is 1.

Limit of several variables


Hausdorff measure of the middle third Cantor set and Compactness

Question: In the proof of the Hausdorff dimension of the middle third cantor set I cannot understand why we need the following underlined statement. I cannot understand why we need only consider closed intervals? Moreover, I am not sure in the construction why we would include/exclude the end points of the intervals.
Answer: In this proof $|U|$ is the diameter of $U$. It isn't hard to see that for any $\epsilon > 0$ there and any set $U$ with finite diameter there is an open interval $V$ with the property that $U \subset V$ and $|V| = (1 + \epsilon)|U|$. To see this, observe that if $U$ has finite diameter then $U$ is bounded, $U \subset [\inf U, \sup U]$, and $|U| = \sup U - \inf U$. Define $V = (\inf U - \delta,\sup U + \delta)$ where $\delta > 0$ is chosen so that $|V| = |U| + 2\delta  = (1 + \epsilon)|U|$.
Let $\{U_i\}$ be a cover of $F$ by sets with finite diameter. Let $\epsilon > 0$ be given and select the $\{V_i\}$ accordingly. Since $F$ is compact there exists an index $I$ so that $\{V_i\}_{i=1}^I$ is a cover of $F$. If in this instance you can verify that $\sum_{i=1}^I |V_i|^s \ge 3^{-s}$ you will obtain
$$
(1+\epsilon)\sum_i |U_i|^s =  \sum_i |V_i|^s \ge  \sum_i |V_i|^s \ge 3^{-s}
$$
for the arbitrary cover. Then let $\epsilon \to 0^+$.
Comment: 1) Cover $F$ by arbitrary closed intervals $\{U_i\}$.
2) Define $V_i$ by enlarging $U_i$ accordingly.
3) $\{V_i\}$ is an open cover of $F$ so it reduces to a finite subcover $\{V_i\}_{i=1}^N$.
4) $\{\overline{V_i}\}_{i=1}^N$ is a finite cover of $F$ by closed intervals.
5) $\displaystyle \sum_{i=1}^N |\overline{V_i}|^s \ge 3^{-s}$.
Comment: I dont understand your second from last sentence starting with "If in this instance...."
Comment: Your underlined text includes the phrase "we need only verify (2.14) if $\{U_i\}$ is a finite collection of closed subintervals of $[0,1]$". If you can prove the inequality for the finite collection $\{\overline U_i\}$ the argument I sketched shows how to obtain the general inequality
Comment: Sorry could you clarify how this creates a set of $\underline{closed \ intervals}$?
Comment: Why do we need to verify that $\sum_{i=1}^I |V_i|^s \ge 3^{-s}$? Arent we verifying that $\displaystyle \sum_{i=1}^N |\overline{V_i}|^s \ge 3^{-s}$ ? Surely if we could verify that $\sum_{i=1}^I |V_i|^s \ge 3^{-s}$ holds we could just say that closed set would be slightly bigger


Why does MemoryStream not offer a constructor taking a "long" capacity?

Question: According to .NET MemoryStream - Should I set the capacity?, it's preferred to set the initial capacity of a MemoryStream if you know it beforehand.
In my code, I'm receiving (from a third party library) a Stream object. This Stream is wrapped around a GetObjectResponse class.
Since this class will be disposed after the transaction is closed, I need to copy the received stream so I can continue using it afterwards.
The Stream class exposes a Length property to determine the length of the stream, so as an initial step to begin the copying process I started writing:
Stream destination = new MemoryStream(response.ResponseStream.Length);

But the compiler throws an error, stating that MemoryStream does not take a long parameter, instead it takes an int.
Why would they do this?
If MemoryStream extends from Stream, this means that it can have a long Length. If it can have a long size, why can't it take a long initial capactiy?
I'm trying to avoid initializing the stream with no capacity, because I want to take advantage of the fact that I know how big the stream is going to be, but the constructor seems to be constraining this wish.
How can I approach this and why does MemoryStream cannot take a long capacity?
Answer: MemoryStream does not support more than about 2GB of data right now. This is a current limitation. It does not have to exist in principle.
This is simply a case of a feature not implemented (yet).
Implementing a bigger stream is difficult on the current CLR which only supports arrays with up to about int.MaxValue elements.
Comment: It in principle has to exist.  You can't index the array returned by its GetBuffer() and ToArray() members otherwise, array indices must be less than int.MaxValue.
Comment: Ah, that's true. The limitation is locked into the API surface. Maybe a future CLR fixes the array length limitation, though.


Showing $\lim_{(x,y) \to (0,0)} \sin (xy) / xy = 1$

Question: How could we show that $$\lim_{(x, y) \rightarrow (0, 0)} \frac{\sin xy}{xy}=1$$
??
Could you give me some hints ??
EDIT:
Could we show it as followed??
$$\lim_{(x,y) \rightarrow (0,0)} \frac{\sin xy}{xy}$$
$$ x \rightarrow 0, y \rightarrow 0$$
$$\xi=xy \rightarrow 0$$
$$\frac{\sin xy}{xy}=\frac{\sin \xi }{\xi}\overset{ \xi \rightarrow 0 }{\rightarrow } 1$$
Is this correct?? Could I improve something??
Comment: Do you know that $\lim_{x \to 0}\frac{\sin x}{x} = 1$?
Comment: Hint: Use a small neighborhood of $(0,0)$ to show that as $(x,y)$ tends to $(0,0)$, $xy$ tends to zero.
Comment: Yes, I know... @Arthur
Comment: Could we use this to show that $$\lim_{(x, y) \rightarrow (0, 0)} \frac{\sin xy}{xy}=1$$ ?? How?? @Arthur
Comment: Try to show that both $x^2+y^2$ and $x+y$ approach zero.
Comment: I have added my initial post... Could you take a look and tell me if we show it in that way??
Comment: There are more questions marks than dollar signs in this post. I've edited the title to make it meaningful.
Answer: If you know the Taylor series for sin(z) then you can expand it for $z=x \cdot y$. Then divide by $x \cdot y$
Comment: @MaryStar Certainly is the easiest, assuming you know the expansion. Otherwise you need to do the multivariate analog of the squeeze theorem.
Comment: Is using Taylor series the only way to show it?? @Zach466920
Comment: are you saying that $\lim_{x,y) \to (0,0) xy = 0,$  and $\lim_{h \to 0} \frac{\sin h}{h}$ are not sufficient to conclude that the limit is $1?$
Comment: @abel Ed0's answer works as well.
Comment: I have added my initial post... Could you take a look and tell me if we show it in that way??
Answer: Show first that  $\underset{x\to0}{\lim}\underset{y\to0}{\lim} \dfrac{\sin xy}{xy}$ is $1$, then show that  $\underset{y\to0}{\lim}\underset{x\to0}{\lim} \dfrac{\sin xy}{xy}$ is $1$, then show that if $x$ and $y$ are functions that depend on a parameter $t$ where $\underset{t\to t_{0}}{\lim} x(t) = \underset{t\to t_{0}}{\lim} y(t) = 0$ then  $\underset{t\to t_{0}}{\lim} \dfrac{\sin (x(t)y(t))}{x(t)y(t)} = 1$ for any kind of parametrization.

first you have to know that $$\lim_{x\to 0}\dfrac{\sin x}{x} = \lim_{x\to 0}\dfrac{\cos x}{1} = 1$$ (I used L'Hopital)
Let's do the first:
$$\lim_{x\to0}\lim_{y\to0} \dfrac{\sin xy}{xy} = \lim_{x\to 0}\lim_{y\to0} \dfrac{x\cos{xy}}{x} = \lim_{x\to 0}\lim_{y\to0}\dfrac{\cos{xy}}{1} = \lim_{x\to 0}1 = 1$$
Let's do the second :
$$\lim_{y\to0}\lim_{x\to0} \dfrac{\sin xy}{xy} = \lim_{y\to 0}\lim_{x\to0} \dfrac{y\cos{xy}}{y} = \lim_{y\to 0}\lim_{x\to0}\dfrac{\cos{xy}}{1} = \lim_{y\to 0}1 = 1$$
For the third part we do a little trick, we declare that $x(t)$ and $y(t)$ are differentiables on the neighborhood of $t_0$ and $x(t_0)=y(t_0)=0$ so:
$$\lim_{t\to t_0} \dfrac{\sin (x(t)y(t))}{x(t)y(t)}= \lim_{t\to t_0} \dfrac{\cos (x(t)y(t))(x'(t)y(t)+y'(t)x(t))}{x'(t)y(t)+y'(t)x(t)}=\lim_{t\to t_0}\cos (x(t)y(t)) =\cos(0\cdot0)=1$$
Comment: I have added my initial post... Could you take a look and tell me if we show it in that way??


get id from url (or kwargs) for django class-based form

Question: I am trying to use a basic form to add data into my app, but I've setup a structure to seperate everything into 'projects'.
Everything is filtered by the project you select, and a project has a manytomanykey to a user.
my urls are then setup to be:
**//backend/projects
**//backend/projects[projid]/capabilities
Then when I add a 'capability' element, I need to read the 'project' id from the URL or kwargs
how do I access the capability_project (when rendering the form template)
i.e. the submit button redirecting the user back to the right project page
<form id="capability-form" method="post" action="backend/projects/{{ project_capability }} /capabilities">

I think because its a many to many it returns an iterator error. How should I be doing this? and how should I make sure the project_id value is always correctly when adding items?
// edit to update NoReverseMatch error im getting from aparently not being able to access kwargs of the form.
NoReverseMatch at /backend/projects/1/capabilities/add/
Reverse for 'capability-list' with arguments '()' and keyword arguments '{}' not found. 0 pattern(s) tried: []
Request Method: POST
Request URL:    http://c9589d:8000/backend/projects/1/capabilities/add/

and further trace info
Request information

GET No GET data
POST Variable   Value
status  u'12'
domain  u'2'
capability_num  u'1'
level   u'a'
description u'asasd'
submit  u'Create Capability'
project u'1'
csrfmiddlewaretoken u'RW2vMwu3BhCOOhDTRaSeCglDSBCctqeF'
name    u'asdsd'

models.py
class Project(models.Model):
    proj_name = models.CharField()
    assigned_to = models.ManyToManyField(User, related_name='assigned_to')

class Capability(models.Model):
    name = models.CharField()
    project = models.ManyToManyField(Project)
    current_status = models.BooleanField()
    future_status = models.NullBooleanField()

class Function(models.Model):
    name = models.CharField()
    capability = models.ForeignKey(Capability)
    current_status = models.BooleanField()
    future_status = models.NullBooleanField()

views.py
class Add_Capability(CreateView):
    template_name = 'backend/add_capability.html'
    model = Capability
    fields = ['name', 'description', 'status', 'capability_num', 'project', 'domain']
    form_class = CapabilityForm

forms.py
class CapabilityForm(forms.ModelForm):
    name = forms.CharField(max_length=255, help_text="Please enter a name for the capability")
    description = forms.CharField(max_length=255, help_text="Please enter a description for the capability")
    status = models.CharField(max_length=255, choices=STATUS_CHOICES, default="Approved")
    capability_num = forms.DecimalField(max_digits=5, decimal_places=4, help_text="Enter a number")
    project = models.ManyToManyField(Project, null=True)
    level = models.CharField(max_length=1, choices=LEVELS_CHOICES, default="Approved")
    domain = models.ManyToManyField(Domain)

    class Meta:
        model = Capability
        fields = ('name', 'description',)

urls.py
# /backend/1/capabilities
url(r'^projects/(?P<capability_project>\d+)/capabilities/$', views.CapabilityList.as_view(), name='capability-list'),

# /backend/capabilities/add - Add capabilities
url(r'^projects/(?P<capability_project>\d+)/capabilities/add/$', views.Add_Capability.as_view(), name='add-capability'),

# /backend/projects
url(r'^projects/$', views.ProjectList.as_view(), name='projects'),

Comment: OK, but that should be done in the `get_redirect_url` method of the view, not in the form action.
Comment: I'm not sure why you want to specify anything in the form action. It needs to post back to the same AddView, otherwise it won't be processed.
Comment: paste the error traceback you're getting
Comment: does `Add_Capability` view have `form_class = CapabilityForm` in its definition?
Comment: ill add the error im getting when i get back home tonight, but I want to redirect to 'projects/1/capabilities page (depending on proj id) after a successful postback. @Anentropic, ive shown the whole form class above, does it need a form_class definition?
Comment: @heymishy how does the view know to use your custom form class otherwise? https://docs.djangoproject.com/en/dev/ref/class-based-views/mixins-editing/#django.views.generic.edit.FormMixin
Comment: @entropic apologies I did have it i just temporarily commented it out and forgot to include in this question. What I am failing to see is how to include / pass the project_I'd back so that get_redirect_url can evaluate the url as defined in urls.py which expects the  passed in
Comment: @entropic or @anush any ideas? When I try to use the kwargs like `kwargs={'project': self.project}` I get an error or a manytomany manager error. I thought it should be just the value of the selected project from the form post?
Answer: In the views.py you can access kwargs this way:
class Add_Capability(CreateView):
    template_name = 'backend/add_capability.html'
    model = Capability
    fields = ['name', 'description', 'status', 'capability_num', 'project', 'domain']
    form_class = CapabilityForm
    project_id = False

    def get_success_url(self):
        self.success_url = reverse('project-detail')+self.kwargs['capability_project']+'/'
        return super(Add_Capability, self).get_success_url()

Comment: arent you supposed to use something like   `def get_absolute_url(self):
        return reverse('project-detail', kwargs={'capability_project': self.capability_project})`
Comment: I get the following: `mproperlyConfigured at /backend/projects/1/capabilities/add/
No URL to redirect to.  Either provide a url or define a get_absolute_url method on the Model.
Request Method: POST`
Comment: @heymishy i have edited the code. I have added a get_success_url function.
Comment: thank you, its almost there, it except it doesnt seem to be accssing the value of the kwargs properly. I have edited my above post with the additional info


How do I list user rights and group memeberships of a domain user with Powershell?

Question: I would like to list all the windows group memberships of a domain user using powershell. How can I do this?
Comment: what you mean for "user rights"?
Comment: @Christian Rereading my post I will edit it and include only Windows Group memberships. Thanks for letting me know
Answer: I am not sure exactly what you mean by "list all user rights" - but group membership is straightforward from your domain:
$de = New-Object System.DirectoryServices.DirectoryEntry
$ds = New-Object System.DirectoryServices.DirectorySearcher($de)
$ds.Filter = "(&(objectClass=User)(sAMAccountName=goyuix))"
$user = $ds.FindOne()
$user.Properties.memberof

The memberof property should contain a list of all the group memberships for that account. You would need to update the Filter to use the appropriate login rather than my hard-coded user.
Comment: nested groups are not listed in memberof. Use quest module for cmdlets to achieve all groups membership
Answer: gyurisc! Is it that you mean?
function Get-ProfilesList {
  $hive = "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList"

  #проверка нужна лишь на время отладки
  if (Test-Path $hive) {
    gci $hive | % -b {$prof = @()} -p {$dest = "" | select UserName, Sid, ProfilePath
    $dest.Sid = $_.PSChildName
    $dest.ProfilePath = (gp ($hive + "\" + $_.PSChildName)).ProfileImagePath
    $dest.UserName = Split-Path $dest.ProfilePath -leaf
    $prof += $dest
    } -end {$prof}
  }
}

Get-ProfilesList | ft -auto


How to make deployment of PHP files with Gradle

Question: I have Android Studio project, where I store (in separate folder) PHP files (implementing web services). The whole project is built and maintained by Gradle plugin to Studio.
How to make Gradle to deploy (copy) these files from project to external Apache \htdocs localisation? Is it possible to copy theses files during Gradle build process? if so, how to copy them only if there are differences between files in project folder and files in external \htdocs localisation.
I will appreciate any help.
Answer: Assuming your Apache server runs on the same machine, you can create a Gradle copy task to copy single files or whole directories to a specific location:
<code>task copyWebFiles(type: Copy) {
    from 'src/web'             // source directory
    into '../path/to/htdocs'   // target directory
}

preBuild.dependsOn copyWebFiles  // execute task before build
</code>
As part of your app's build.gradle file.
Comment: I was looking into this documentation but I have omitted this fragment about copy task. Great thanks.


Is every correlation matrix positive semi-definite?

Question: I am generating correlation matrix by some new algorithm. Generated matrix is non positive semi-definite matrix.
I'm getting a few negative eigenvalues. The rest of eigenvalues are quite equal to the ideal matrix.
Can I use that non positive semi-definite matrix? If not, why?
If my estimated correlation matrix has all positive but complex value and imaginary terms are close to zero then is it possible?
Comment: Near-duplicates: [Is every correlation matrix positive definite?](http://stats.stackexchange.com/questions/182875/is-every-correlation-matrix-positive-definite) which has more focus on the definite versus semi-definite angle, and [Is every covariance matrix positive definite?](http://stats.stackexchange.com/questions/56832/is-every-covariance-matrix-positive-definite) which is relevant because a covariance is essentially a rescaled correlation.
Comment: Possible duplicate of [Is every covariance matrix positive definite?](http://stats.stackexchange.com/questions/56832/is-every-covariance-matrix-positive-definite)
Comment: One of a number of very similar questions on CV http://stats.stackexchange.com/q/69114/3277.
Comment: it cannot be correct, so the algorithm has some issues.
Answer: A correlation matrix is really the covariance matrix of a bunch of variables which have been rescaled to have variance one.
But every population covariance matrix is positive semi-definite, and if we rule out weird cases (such as with missing data, or "numerical fuzz" turning a small eigenvalue to a negative one), so is every sample covariance matrix.
So if a matrix is supposed to be a correlation matrix, it should be positive semi-definite.
Note that the semi-definite is important here. In the bivariate case, take your two variables to be perfectly positively correlated and then the correlation matrix is $\pmatrix{1 & 1 \\ 1& 1}$ which has eigenvalues of $2$ and $0$: the zero eigenvalue means it is not positive definite.
Answer: Negative eigenvalues would imply that by the diagonalizing transformation the random vector would have negative variance in some components. Negative variances do never exist.
Answer: A correlation matrix is positive semi-definite, period.  Numerics, however, might refuse to acknowledge that mathematical fact depending on how you arrive at the numeric representation of the correlation matrix.
The solution is to choose a representation of your matrix that cannot fail to be positive semi-definite by representing the matrix in a suitable decomposed form.  I am not up to scratch, but there are things like LUD decompositions or square root forms that essentially are unable to represent anything but truly positive semi-definite matrices and which you can usually update incrementally similarly to how you would update the full matrix, possibly even easier.


Pushing Objects Into Array

Question: I am trying to push objects into an array using an each loop, while the amount of objects is right in the end the data in the object is repeated with the last entry.
When I log the object inside the loop it is giving me two sets of data, the first one is correct, the second one is the last set of data for the loop.
I have tried logging the data to the console but it is giving me really weird values.
$.getJSON(url, function(data) {

  var entry = data.feed.entry;

     var nietos = [];
     var obj = {};

  $(entry).each(function(){

   var title = String(this.gsx$title.$t);
   var lastName = String(this.gsx$lastname.$t);
   var firstName = String(this.gsx$firstname.$t);
   var email = String(this.gsx$email.$t);
   var department = String(this.gsx$department.$t);

      console.log(lastName);

      obj["title"] = title;
      obj["lastName"] = lastName;
      obj["firstName"] = firstName;
      obj["email"] = lastName;
      obj["department"] = department;

      //console.log(obj);
      /*
      (index):345 {title: "Cook II", lastName: "Woon", firstName: "Ra", email: "Wson",
      department: "LR"}department: "WL Span"email: "Zuniga"firsame: "Lea"lastName: "Zga"title: "Regular Teer"__proto__: Object*/

      //nietos.push(obj);

});
//console.log(nietos);
//97: {title: "Regular Teacher", lastName: "Zga", firstName: "Leticia", email: "Zuga", department: "WL S"}

});
Expected:
{title: "Teacher", lastName: "Teacher1", firstName: "Teachername", email: "firstname.lastname@example.com",},
{title: "Teacher2", lastName: "Teacher2", firstName: "Teachername2", email: "firstname.lastname@example.com",},,
{title: "Teacher3", lastName: "Teacher3", firstName: "Teachername3", email: "firstname.lastname@example.com",},

Actual output:
{title: "Teacher3", lastName: "Teacher3", firstName: "Teachername3", email: "email@example.com",},
{title: "Teacher3", lastName: "Teacher3", firstName: "Teachername3", email: "email@example.com",},
{title: "Teacher3", lastName: "Teacher3", firstName: "Teachername3", email: "email@example.com",},

Answer: You create just one object with var obj = {};, then you modify that one object in the "loop" and add references to that object to the array.
Create a new var obj = {} inside of each loop iteration (inside the .each callback).
As a sidenote, let profesor = {}; would be much better ...
Comment: Thanks! Why would let be better?
Comment: Cause it has stricter scoping rules (and is therefore easier to reason about)

Pushing Objects Into Array


find & replace part of text

Question: I am trying to do a search and replace using GREP/Regex
Here is what I am searching for

.+

I want to remove the ,
 tags, and place
 tags around what .+ finds.
But I can't get what .+ finds to stay when it does the replace.
For example, I want to find this

This is the section I want intact

to change to this

This is the section I want intact

Any help is appreciated.
Answer: Use sed instead of grep:
# Modify the file in-place
sed -i~ 's|\(
\)\(.\+\)
|\1
\2
|' the-file

Comment: What about "perl -ple 's/search regex/replace string/"?
Grep has absolutely no replace functionality, only searching. Sed is the usual answer.
Answer: It depends exactly what system you're using, but if you put something in parenthesis you can refer to it later
So it might be something like
s/(.+)
/
\1<\/h3>/

Comment: I am just using the search/replace in textwrangler.
Comment: Ok, then use what I said. Text to find = (.+) Replace with: \1 If you need to match the stuff at the beginning, do (foo)(.+) and \1\2
Answer: In TextWrangler:
search for:

(.+?)

replace with:

\1

The '\1' will be replaced with the string matched inside the parens in the search pattern.
Comment: (Also, using the question mark (.+?) instead of (.+) will ensure that you don't match too much -- otherwise, (.+) might match everything up to the last  in the text)

find & replace part of text


Fetching JSON in C# Xamarin Forms

Question: I'm trying to use my JSON API in Xamarin Forms (C#). The API can return:
When it fails: {"status":"failed"}
When everything is okay: {"status":"success","time":"03.06.2016 13:15","data":["Thats a cool text","And thats another text"]}
I'm currently doing this:
var client = new HttpClient ();
var url = "http://domain/plan/web/get/" + userclass;

var response = await client.GetAsync (url);
var data = response.Content.ReadAsStringAsync ().Result;

But now, I need to:

Fetch the JSON data into an array or something
Display an alert when the API returns failed (with DisplayAlert())
When the API returns success, get the time, and set the data (the two or more strings at the end of the API result) to a label or a list view.

How can I do that?
Comment: Create a class that handle you JSON data. Use Newtonsoft.json Nuget Package to convert json to object.
Comment: But how to fetch the data like the two strings, because it's like an array in the JSON.
Comment: Create a model to handle your data and then you use `JsonConvert.Serialize(yourobject)` to turn it into a string and when you fetch it you Deserialize your string to get back your object `JsonConvert.Deserialize(jsonstring);`.
Answer: The first step you need to do is creating a class that can handle your JSON Data. In this case I would do:
public class APIResponse
{
    public string status { get; set; }
    public string time { get; set; }
    public List<string> data { get; set; }
}

Then you need to create you need to create a method to make the GET request. This can be acomplished by doing this:
    public async Task <APIResponse> GetDataFromAPI(string query){
                try
                {
                    var client = new System.Net.Http.HttpClient ();
                    client.BaseAddress = "http://domain/";
                var response = await client.GetAsync("plan/web/get"+query);

                response.EnsureSuccessStatusCode();
                var responseJSON = await response.Content.ReadAsStringAsync();

                var msg  = JsonConvert.DeserializeObject<APIResponse>(responseJSON);

                return msg;
            }

            catch (Exception exc)
            {
                var msg = new APIResponse ();
                msg.status = "failed";

                return msg;
            }

}

Then in your page you call the method using:
APIResponse data = await GetDataFromAPI("yourquery");
if(data.status.Equals("success"){
     //Assign object values to label text property
}
else {
     DisplayAlert("Error","Something wrong happened", "OK");
}

Comment: Strange thing is here, I have the same code, except I get an error on the `APIResponse data = await GetDataFromAPI("yourquery");`, it gives `The name 'GetDataFromAPI' does not exist in the current context.` Any ideas?

Fetching JSON in C# Xamarin Forms


Paired t-test, Kruskal-Wallis rank sum test and chisquare test. Which test should I choose?

Question: I observed moth oviposition in orchard for 4 moths (400 clusters per week). The moth oviposit on leaf and apple. I want to compare the means between oviposit on leaf and apple. I suppose to use paired t-test, however, the result of shapiro.test showed my data do not fit normality. I was wondering if I should use  Kruskal-Wallis rank sum test for this case?
Moreover, I did some treatment on apples to prevent the ovipostion. I observed eggs on treated and untreated clusters weekly (treated and untreated apple trees were in the same orchard). Then I got a table like:
<code>               Treated  untreated
Egg Observed       40       1852
Non-observed     6000      12600
</code>
Can I use a chisquare test in this case?
Answer: If your data are paired (as they seem to be, since you are supposed to use paired t-tests) the Kruskal Wallis is not appropriate as it is for independent data. I think a Friedman test is better suited to your question (see Wikipedia article)
On the table - you certainly can use chi-square here, but since you have a dependent variable (egg observed) and an independent variable (treatment) it might be better to use logistic regression. But, it appears that your data are not independent since the trees were in clusters; therefore, a nonlinear mixed model might be even better (if trees in clusters are likely to be similar, as seems likely); there are also other methods for dealing with clustered samples
Comment: @jack, Friedman test ia a valid test 2+ paired samples. With 2 samples, it gives results very close to those obtained by **sign test**, so you may use this. (The latter is a little bit more liberal, in normal approximation regime.)
Comment: Thank you for your reply. I read the link you provided. I also googled for more information. However, I found that Friedman test is suitable for data in 3 groups or more than 3 groups. The code for R is in this way: friedman.test(y, ...).

## Default S3 method:
friedman.test(y, groups, blocks, ...)

## S3 method for class 'formula'
friedman.test(formula, data, subset, na.action, ...)        However, my data was only in 2 groups ( ovipostion on leaf and oviposition on apple). I was wondering how can I use Friedman test in this case?
Comment: In your second paragraph, you suggested that it might be better to use logistic regression. My task was count the times of moth ovipostion on treated apple and untreated apple trees(number of eggs), I considered Poisson Regression might be a better choice. As I metioned treated and untreated trees were in the same orchard. Therefore, my data are not independent. You said: nonlinear mixed model might be even better. I checked The R Book. I found there were dozens of methods, How can I know which one is suitable?
Comment: @jackWei choosing among different nlme is a huge job.
Comment: @ttnphns thank you for your suggestion. I will be able to use friedman test once I put my data into matrix. As the sign test you metioned, I do not considered it is proper test. Becasuse oviposit on leaf and apple are dependent events. If moth lay more eggs on leaf, obviously, there will be less eggs on apple.
Comment: @jackWei Why do you think that "more eggs on leaf" $\implies$ "fewer eggs on apple" would invalidate the sign test? Conditioning on the total sample size, as is done in the sign test, more in one category necessarily implies fewer of the other. I don't see the problem at all.
Comment: @ttnphns, you also metioned the latter is a bit more liberal,in normal approximation regime.However, Peter Flom pointed out that my data are not independent, since the trees were in same orchard. I note that you suggested binomial test in another case. Is it suitable in my case?
Comment: @Glen_b Thank you for your reply. I tried sign test. However the result is very different from friedman test. The result of sign test implied moth oviposite more egg on leaf. The result of friedman test implied there is no significant difference between moth oviposite on leaf and apple. However, ttnphns said:The result of friedman test is very close to those obtained by sign test, Something must be wrong.


Vectorized Splitting of String on Character Count using Numpy or Pandas

Question: Is there a way to split a Numpy Array in a vectorized manner based upon character count for each element?
Input:
In [1]: import numpy as np

In [2]: y = np.array([ 'USC00013160194806SNOW','USC00013160194806SNOW','USC00013160194806SNOW' ])

In [3]: y
Out[3]:
array(['USC00013160194806SNOW', 'USC00013160194806SNOW',
       'USC00013160194806SNOW'],
      dtype='|S21')

I want each element of the array split according to a certain number of characters.
Desired Output:
In [3]: y
Out[3]:
array(['USC00013160', 'USC00013160',
       'USC00013160'],
      dtype='|S21')

I've executed this using standard python loops, but I'm dealing with millions of values, so I'm trying to figure the fastest method.
Comment: Are you intentionally asking for the dtype of the desried output to be `'S21'`?  `'S11'` might make more sense.
Comment: @WarrenWeckesser no, I'm not specifically looking for a desired output to S21.
Comment: How would you do this in a loop?  `[astr[:11] for astr in a]` or some character search?  `np.char` applies a bunch of `string` functions to elements of an array.
Answer: You can create a view using a data type with the same size as y's dtype  that has subfields corresponding to the parts that you want.  For example,
In [22]: y
Out[22]:
array(['USC00013160194806SNOW', 'USC00013160194806SNOW',
       'USC00013160194806SNOW'],
      dtype='|S21')

In [23]: dt = np.dtype([('part1', 'S11'), ('part2', 'S6'), ('part3', 'S4')])

In [24]: v = y.view(dt)

In [25]: v['part1']
Out[25]:
array(['USC00013160', 'USC00013160', 'USC00013160'],
      dtype='|S11')

In [26]: v['part2']
Out[26]:
array(['194806', '194806', '194806'],
      dtype='|S6')

In [27]: v['part3']
Out[27]:
array(['SNOW', 'SNOW', 'SNOW'],
      dtype='|S4')

Note that these are all views of the same data in y.  If you modify them in place, you are also modifying y.  For example,
In [32]: v3 = v['part3']

In [33]: v3
Out[33]:
array(['SNOW', 'SNOW', 'SNOW'],
      dtype='|S4')

Change v3[1] to 'RAIN':
In [34]: v3[1] = 'RAIN'

In [35]: v3
Out[35]:
array(['SNOW', 'RAIN', 'SNOW'],
      dtype='|S4')

Now see that y[1] is also changed:
In [36]: y
Out[36]:
array(['USC00013160194806SNOW', 'USC00013160194806RAIN',
       'USC00013160194806SNOW'],
      dtype='|S21')

Answer: One possible solution I've found is just completing the operation using Pandas Series, but I'm wondering if this can be done using an only Numpy arrays slicing methods.  If not, it is fine, more curious about the best practice.
Starting Pandas Series:
In [33]: x = pd.read_csv("data.txt", delimiter='\n', dtype=str, squeeze=True)
In [34]: x
Out[34]:
0         USC00013160194807SNOW
1         USC00013160194808SNOW
2         USC00013160194809SNOW
3         USC00013160194810SNOW
4         USC00013160194811SNOW, dtype: object

Vectorized String Processing based on Character Count:
In [37]: k = x.str[0:11]

Output:
In [38]: k
Out[38]:
0         USC00013160
1         USC00013160
2         USC00013160
3         USC00013160
4         USC00013160


Free statistical textbooks

Question: Are there any free statistical textbooks available?
Comment: Look at Statistics Topics ebook on Amazon by Mehta, and his free web log Statistics Ideas that has lecture slides. Nearly free and better in some pedagogical topics, than the ones you cite on your list of resources.
Answer: Online books include

http://davidmlane.com/hyperstat/
http://vassarstats.net/textbook/
https://dwstockburger.com/Multibook/mbk.htm
https://web.archive.org/web/20180122061046/http://bookboon.com/en/statistics-ebooks
http://www.freebookcentre.net/SpecialCat/Free-Statistics-Books-Download.html

Update: I can now add my own forecasting textbook

Forecasting: principles and practice (Hyndman & Athanasopoulos, 2012)
Answer: The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman is a standard text for statistics and data mining, and is now free:
https://web.stanford.edu/~hastie/ElemStatLearn/
Also Available here.
Comment: Why pay $70 for the book?  To support the authors and to have a physical copy.  That's why I did it, anyway!
Comment: I agree with Shane... and a hardcopy makes reading it much easier.
Comment: Did that, too (previous edition). But it is nice to be able to look things up online when the book is not at hand. And to have access to the new edition.
Comment: (+1) Just wanted to comment that this is a GREAT book, it is my all-time favorite. I also ordered a physical copy of it :-).
Comment: If you attend Hastie and Tibshirani's Statistical Learning and Data Mining seminar, you'll receive a free copy of the text. And you can have your book signed!
Comment: I have a second edition. It is the best printed book I have - four color printing throughout, excellent typography, great paper.
Comment: Would it be possible to point to the solutions of Elements of Statistical Learning ? Thanks!
Comment: Found it answered here: https://stats.stackexchange.com/questions/108449/most-efficient-way-to-check-answers-to-exercises-when-learning-from-a-statistics
Answer: Introduction to Statistical Thought
Comment: I'm teaching out that book this semester.  It may be a good *statistics* book, but I'm having second thoughts regarding its light probability content.
Comment: Michael Lavine is a clear lecturer and thinker, so I have little doubt his book is worth a look.
Answer: There's a superb Probability book here:
https://web.archive.org/web/20100102085337/http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/book.html
which you can also buy in hardcopy.;
Answer: I've often found the Engineering Statistics Handbook useful. It can be found here.
Although I've never read it myself, I hear Introduction to Probability and Statistics Using R is very good. It's a full ~400 page ebook (also available as an actual book). As a bonus, it also teaches you R, which of course you want to learn anyways.
Comment: +1 For this resource. As the name says, excellent for a practical approach to engineering problems.
Answer: Machine Learning

One the most, if not the most, popular textbooks on machine learning is  Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, which is fully available online (currently 10th printing). It is comparable in scope e.g. to Bishop's Pattern Recognition and ML or Murphy's ML, but those books are not free, while ESL is.

Hastie & Tibshirani also co-wrote freely available An Introduction to Statistical Learning, With Applications in R which is basically a simpler version of The Elements and focuses on R.

In 2015, Hastie & Tibshirani co-authored a new textbook Statistical Learning with Sparsity: The Lasso and Generalizations, also available online. This one is quite a bit shorter and focuses specifically on lasso.

Another freely available all-encompassing machine learning textbook is David Barber's Bayesian Reasoning and Machine Learning. I did not use it myself, but it is widely considered to be an excellent book.

$\hskip{5em}$

Switching now to more specialized topics, there are:

Rasmussen & Williams Gaussian Processes for Machine Learning, which is the book on Gaussian processes.

Much awaited Goodfellow, Bengio and Courville Deep Learning textbook that is about to be published by MIT Press. It isn't published yet, but the book is already available online. On the official website one can view it in browser but cannot download (as per agreement with the publisher), but it is easy to find a combined PDF e.g. here on github.

Csaba Szepesvári, Algorithms for Reinforcement Learning, a concise book on RL. A classical, much more detailed but a bit dated textbook is Sutton & Barto, Reinforcement Learning: An Introduction which is also freely available online but only in a cumbersome HTML format.

Boyd and Vandenberghe, Convex Optimization.

$\hskip{4em}$
Answer: I really like  The Little Handbook of Statistical Practice by Gerard E. Dallal
Answer: Here's a fresh one: Introduction to Probability and Statistics Using R . It's R-specific, though, but it's a great one. I haven't read it yet, but it seems fine so far...
Comment: I downvoted because of "it's a great one.  I haven't read it...."
Comment: First edition at http://cran.r-project.org/web/packages/IPSUR/vignettes/IPSUR.pdf and second edition at http://ipsur.org/
Comment: Link is broken.  Try http://www.lulu.com/items/volume_68/8123000/8123594/3/print/IPSUR.pdf
Answer: A New View of Statistics by Will G. Hopkins is great! It is designed to help you understand how to understand the results of statistical analyses, not how to prove statistical theorems.
Answer: Norman Matloff has written a mathematical statistics textbook for computer science students that's free.  Kind of a niche market, I suppose.  For what it's worth, I haven't read it, but Matloff has a Ph.D. in mathematical statistics, works for a computer science department, and wrote a really good R book, that I recommend for people who want to go to the next stage of programming R better (as opposed to just fitting models with canned functions).
Answer: OpenIntro Statistics
http://www.openintro.org/stat/textbook.php
Inexpensive paperback copies are also available on Amazon.
Comment: But if authors want to put out material on the internet for free that is their choice. We all can benefit from that.
Comment: You could make the same arguments about software, but we all benefit from R.
Comment: I moved this from answer to comment since I am expressing my opinion about free books on the net rather than providing a a list of some. As both an author and owner of books I see both sides. I appreciate that most statistics books, especially when they deal with specialized topics are very expensive. However as an author who has put in a lot of effort to try to write useful and informative books I think the authors and publishers deserve remuneration for the efforts they go through to produce the books. So I don't think books should be free.
Answer: "An Introduction to Statistical Learning with Applications in R" https://www.statlearning.com/ by two of the 3 authors of the well-known "The Elements of Statistical Learning" plus 2 other authors.  An Introduction to Statistical Learning with Applications in R is written at a more introductory level with less mathematical background required than The Elements of Statistical Learning, makes use of R (unlike The Elements of Statistical Learning), and was first published in 2013, some years after this thread was started.
Answer: Not Statistics specific, but a good resource is:  http://www.reddit.com/r/mathbooks
Also, George Cain at Georgia Tech maintains a list of freely available maths texts that includes some statistical texts.  http://people.math.gatech.edu/~cain/textbooks/onlinebooks.html
Answer: I really like these two books by Daniel McFadden of Berkeley:

Lecture Notes: Econometric Tools
http://elsa.berkeley.edu/users/mcfadden/e240a_sp98/e240a.html

Lecture Notes: Econometrics/Statistics  http://elsa.berkeley.edu/users/mcfadden/e240b_f01/e240b.html
Answer: For getting into stochastic processes and SDEs, Tom Kurtz's lecture notes are hard to beat.  It starts with a decent review of probability and some convergence results, and then dives right into continuous time stochastic processes in fairly clear, comprehensible language.  In general it's one of the best books on the topic -- free or otherwise -- I've found.
Answer: Cosma Shalizi, CMUs ML guru, occasionally updates a draft of a stats book soon to be published by Cambridge Press titled Advanced Data Analysis from an Elementary Point of View. Can't recommend it highly enough...
Here's the Table of contents:
<code>I. Regression and Its Generalizations
Regression Basics
The Truth about Linear Regression
Model Evaluation
Smoothing in Regression
Simulation
The Bootstrap
Weighting and Variance
Splines
Additive Models
Testing Regression Specifications
Logistic Regression
Generalized Linear Models and Generalized Additive Models
Classification and Regression Trees

II. Distributions and Latent Structure
Density Estimation
Relative Distributions and Smooth Tests of Goodness-of-Fit
Principal Components Analysis
Factor Models
Nonlinear Dimensionality Reduction
Mixture Models
Graphical Models

III. Dependent Data
Time Series
Spatial and Network Data
Simulation-Based Inference

IV. Causal Inference
Graphical Causal Models
Identifying Causal Effects
Causal Inference from Experiments
Estimating Causal Effects
Discovering Causal Structure

Appendices
Data-Analysis Problem Sets
Reminders from Linear Algebra
Big O and Little o Notation
Taylor Expansions
Multivariate Distributions
Algebra with Expectations and Variances
Propagation of Error, and Standard Errors for Derived Quantities
Optimization
chi-squared and the Likelihood Ratio Test
Proof of the Gauss-Markov Theorem
Rudimentary Graph Theory
Information Theory
Hypothesis Testing
Writing R Functions
Random Variable Generation
</code>
Comment: I point people to Cosma's notes regularly. There's some good material in there
Answer: Some free Stats textbooks are also available here.
Answer: I know other authors have gone to some trouble to make their books available here on stack exchange ...  The printed version of our 2002 edition was printed 3 times and sold out 3 times;  Springer and Google recently started selling it (book only) as a PDF eBook (no software) on the Springer and Google sites for $79.
We are delighted to be able to make the PDF eBook version (2002 edition) available for FREE to stackexchange users at:
http://www.mathstatica.com/book/bookcontents.html

This is a complete PDF version of the original 2002 printed edition. Although no software is included (neither Mathematica nor mathStatica), the methods, theorems, summary tables, examples, exercises, theorems etc are all useful and relevant ... even as a reference text for people who do not even have Mathematica.
One can either download:

the entire book as a single download file ... with live clickable Table of Contents etc,  ... or
chapter by chapter.

iBooks installation
To install as an iBook:

Download the entire book as a single PDF file
Then drag it into iBooks (under the section: PDF files).

iPad installation
To install on an iPad:

First install it as an iBook (as above)
Open iTunes; select your iPad; click on Books: select the book and sync it over to your iPad.
Answer: It's nice to see academics freely distribute their works. Here is trove of free ML / Stats books in PDF:
Machine Learning

Elements of Statistical Learning Hastie, Tibshirani, Friedman
All of Statistics Larry Wasserman
Machine Learning and Bayesian Reasoning David Barber
Gaussian Processes for Machine Learning Rasmussen and Williams
Information Theory, Inference, and Learning Algorithms David MacKay
Introduction to Machine Learning Smola and Vishwanathan
A Probabilistic Theory of Pattern Recognition Devroye, Gyorfi, Lugosi
Introduction to Information Retrieval Manning, Rhagavan, Shutze
Forecasting: principles and practice Hyndman, Athanasopoulos (Online Book)

Probability / Stats

Introduction to statistical thought Lavine
Basic Probability Theory Robert Ash
Introduction to probability Grinstead and Snell
Principle of Uncertainty Kadane

Linear Algebra / Optimization

Linear Algebra, Theory, and Applications Kuttler
Linear Algebra Done Wrong Treil
Applied Numerical Computing Vandenberghe
Applied Numerical Linear Algebra James Demmel
Convex Optimization Boyd and Vandenberghe

Genetic Algorithm

A Field Guide to Genetic Programming Poli, Langdon, McPhee
Evolved To Win Sipper
Essentials of Metaheuristics Luke
Comment: If LADW is mentioned, perhaps LADR by Axler could have been mentioned too; the latter is way better book imo.
Answer: Statsoft's Electronic Statistics Handbook ('The only Internet Resource about Statistics Recommended by Encyclopedia Britannica') is worth checking out.
Comment: This link now redirects to a product page for a data science/AI software. Can you update the link, if this resource is still freely available?
Answer: Not properly an entire textbook, but the part IV of Mathematics for Computer Science is about probability and random variables.
Answer: Some downloadable notes on probability, which seems interesting:
http://www.math.harvard.edu/~knill/teaching/math19b_2011/handouts/chapters1-19.pdf
Applied probability:
http://www.acsu.buffalo.edu/~bialas/EAS305/docs/EAS305%20NOTES%202005.pdf
http://www.ma.huji.ac.il/~razk/Teaching/LectureNotes/LectureNotesProbability.pdf
Answer: A write up of probability tutorials and related puzzles along with R code for learning. Hope it helps
Answer: http://www.probabilitycourse.com/ is a website hosting free online-based Probability and Statistics textbook. It also has extra features such as graphing tools and lecture videos
Answer: Here is also a great free book on multivariate statistics by Marden, primarily concerned with the normal linear model linked on this page:
Multivariate Statistics Old School by John I. Marden
Answer: Gelman et al. "Bayesian Data Analysis" (3rd edition).
Comment: How do I shrink the picture? :)
Comment: on the image file if you add some whitespace it will shrink and fit to width. Alternatively, you may use a lower resolution image.
Comment: Use [ImageMagick](https://duckduckgo.com/?t=ffab&q=imagemagick&atb=v235-1&ia=web). E.g., `convert xxx -density 600 -resize 25% xxx`.
Comment: @chl, thanks. I was looking for a built-in solution. Otherwise there are multiple services to choose from. I use FastStone Photo Resizer every now and then.
Comment: No built-in skrinking feature, AFAIK. +1, BTW, this was a very nice free update from the authors of this memorable textbook.
Answer: It's not a textbook but Bayesian Methods in the Search for the MH370 is a great introduction to particle filters.
S. Davey, N. Gordon, I. Holland, M. Rutten, and J. Williams, Bayesian Methods in the Search for MH370 (2016)
Answer: A digital textbook on probability and statistics by M. Taboga can be found at https://www.statlect.com The level is intermediate. It has hundreds of solved exercises and examples, as well as step-by-step proofs of all the results presented.
Answer: For an introduction to the statistical analysis of networks, Statistical Analysis of Networks by Konstantin Avrachenkov and Maximilien Dreveton. The paper version isn't free, but the PDF version is. It is published under a Creative Commons By-NC licence.

Free statistical textbooks


Python pandas how to merge multiple row into one?

Question: I have A dataframe as below:
Car        SeriesNo     White       Red        Black
Proton     A111           2
Proton     A111                      4
Proton     A111                                  5

My Expected output will be:
Car        SeriesNo     White       Red        Black
Proton     A111           2          4           5

Anyone have ideas on this?
Comment: There are multiple groups by first 2 columns?
Answer: Use GroupBy.first for get first non missing values per groups:
#if necessary replace empty strings to missing values
#df = df.replace('',np.nan)
df = df.groupby(['Car','SeriesNo'], as_index=False).first()
print (df)
      Car SeriesNo  White  Red  Black
0  Proton     A111    2.0  4.0    5.0

Comment: @Chris - It depends what OP need, but I guess there are multiple groups, so then `groupby` here is necessary.
Comment: Interesting use of `groupby` here. Nice answer as always, much better than mine :)


How to make dictionary from one line of text with integers?

Question: I have text a 1 b 2 c 3.
How can I make a dictionary in which letters would be keys, and numbers values?
I know how I would make it if it was in in multiple rows, I do not know how to do it when it is in one row.
Answer: Convert the string to list. And then list to dictionary.
str1 = "a 1 b 2 c 3"
lst = str1.split(" ")
dct = {lst[i]: lst[i + 1] for i in range(0, len(lst), 2)}

Answer: text = "a 1 b 2 c 3"
myDictionary = {}
for i in range(0,len(text),4):
    myDictionary[text[i]]=text[i+2]
print(myDictionary)
#output : {'c': '3', 'a': '1', 'b': '2'}

Comment: This code will not work with the string given in the question.
Comment: if you have spaces you can use this, if you have two digit value then you should check for that , if you do not specify actually what you need then it become difficult to answer.
Comment: Now it works with the exact string given in the question, but what if there are numbers greater than 9 involved, for example `text = "a 10 b 12 c 32"`?
Comment: just split the string and convert into a list so it will be easier for you to make dictionary
myList = list(text.split())
Comment: so list will become [a,12,b,45,...] like that so using indexing you can make dictionary


How to overflow a column inside a modal?

Question: I have spent a few hours trying to get this to work. I have a (full screen) modal, and it flexes its body to grow according to the modal. Inside this body, I have container, a row, and 2 columns. Since the modal-body doesn't have a height set, I am unable to limit the height (max-height, maybe) of the column to make it overflow.
How do I properly achieve the expected behavior in this scenario? I want the modal body to respect the screen's height, but activating column's overflow.
https://jsfiddle.net/f3t5p6xq

Full Screen Modal

asd

            Right Column

            new line
new line
new line
new line
new
            line
new line
new line
new line
new line
new
            line
new line
new line
new line
new line
new
            line
new line
new line
new line
new line
new
            line
new line
new line
new line
new line
new
            line
new line
new line
new line
new line
new
            line
new line
new line
new line
new line
new
            line

        <button class="btn btn-secondary">
            Save
        </button>

<button
    id="fullModalButton"
    type="button"
    class="btn btn-primary"
    data-toggle="modal"
    data-target="#fullModal"
>
    Click me
</button>

CSS
p {
  line-height: 50px;
}

.modal-dialog {
  padding: 0.5rem;
  margin: 0;
  width: 100%;
  max-width: 100%;
  height: 100vh;
  max-height: 100vh;
}

.modal-content {
  display: flex;
  flex-direction: column;
  height: 100%;
  max-height: 100%;
}

.modal-body {
  /* modal-body dimensions are automatically calculated, since it uses flex */
  /* the body expands, while header and footer have fixed height */
}

.container-fluid {
  /* since modal-body doesn't have a height set, I can't set max-height in this element */
}

.row {
  /* since container-fluid doesn't have a height set, I can't set max-height in this element */
}

.col-4 {
  overflow: auto;
  /* since row doesn't have a height set, I can't set max-height in this element */
  /* this causes this column to overflow */
}

Answer: I got it working by adding .overflow-auto to all parent divs (starting at .modal-content) and specifying .h-100 on .container-fluid, .row and .col-4
Chrome DevTools are of great help for this kind of issue.
Answer: Since you're using bootstrap, I'm not sure why you would add some custom CSS that is already built-in with bootstrap. Here is what I would do. I set the column's height limit to 300px as an example or you can set it based on the screen size using @media or percentage and set the overflow to its element:

.h-300 {
  height: 300px;
}
<link href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" rel="stylesheet" />

Modal title

      <button type="button" class="close" data-dismiss="modal" aria-label="Close">
          ×
        </button>

            ...

              new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new
              line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new line
new
              line

      Footer

Comment: I didn't use bootstrap classes to help me try different combinations of styles. Thank you for your help.


ERROR: could not open directory /lib/modules/4.2.0-25-generic: No such file or directory

Question: I upgraded Ubuntu 15.10 to 16.04 and somewhere there must happened my trouble.
I firstly realised my trouble when I wanted to do sudo iptables -A INPUT -m state --state NEW -p tcp --dport 80 -j ACCEPT. There I got:
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.2.0-25-generic/modules.dep.bin'
modprobe: FATAL: Module ip_tables not found in directory /lib/modules/4.2.0-25-generic
iptables v1.6.0: can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

Ok so there is an issue with /lib/modules/4.2.0-25-generic...
Then I tried to repair with sudo depmod. But I got:
depmod: ERROR: could not open directory /lib/modules/4.2.0-25-generic: No such file or directory
depmod: FATAL: could not search modules: No such file or directory

Now I wanted to rebuild the kernel with sudo apt-get install --reinstall linux-..., here were some versions like:
linux-image-3.13.0-24-generic
linux-image-3.4.0-4-goldfish
linux-image-4.4.0-21-generic
linux-image-4.4.0-21-lowlatency
linux-image-4.4.0-22-generic
linux-image-4.4.0-22-lowlatency

but there is no linux-image-4.2.0-25-generic. Strange thing. So by upgrading Linux there was set the old and wrong version of the kernel somehow.
Now I want to see where this version is saved in. And I found it in the uname -r command. So apt-get install linux-image-'uname -r'-dbgsym, was like:
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package linux-image-4.2.0-25-generic-dbgsym
E: Couldn't find any package by glob 'linux-image-4.2.0-25-generic-dbgsym'
E: Couldn't find any package by regex 'linux-image-4.2.0-25-generic-dbgsym'

But I have no clue how to change the uname -r by hand. And I also don't know why I got this wrong version. And most importantly if it's a good idea to change the uname.
Btw I use Digitalocean and upgraded via do-release-upgrade.
Edit #1
I just wanted to see my current kernel. And it is actually 4.4.0-22:
sudo update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-22-generic
Found initrd image: /boot/initrd.img-4.4.0-22-generic
Found memtest86+ image: /boot/memtest86+.elf
Found memtest86+ image: /boot/memtest86+.bin
done

but uname still says:
uname -r
4.2.0-25-generic

Can someone explain me what might have happened and how to solve?
Answer: You can't install your own kernel or kernel modules on a VPS.  You should consult with your host if you absolutely need certain kernel modules.
I think the filter module should certainly be included.
Comment: `iptables` is installed. But he does not continue because of `/lib/modules/4.2.0-25-generic`
Comment: Yes, but your kernel doen't have the necessary parts included or available as modules.  You can **not** solve this yourself on a VPS.
Comment: But this cannot be. So my VPS is damaged and there is nothing I can do. As I used the official way of upgrading this would have happened to many others already. And DO is a big company. mhh...
Comment: They are indeed a big company and they offer customer support which you should contact!
Comment: Done and solved :) Added a picture.
Comment: @kwoxer: That's good!  Their configuration is surprisingly flexible.
Answer: I just needed to set the kernel in Digitalocean. Now I have the correct kernel (184.108.40.206) and iptables are working, too.