O que significa o operador "|=" em Python?

Question: Estava analisando um código e me deparei com o operador |=, gostaria de saber o que isso significa, e qual é a sua aplicação prática.
Exemplo:
x |= y

Answer: Assim como o operador += incrementa o valor de uma variável, algo como x +=y ser equivalente a x = x + y, o operador |= executa o ou binário entre o valor atual e o outro parâmetro, algo como x |= y ser equivalente a x = x | y.
O ou binário nada mais é que a operação ou executada bit a bit. Por exemplo, vamos tomar dois números inteiros positivos:
x = 5
y = 13

Podemos verificar a representação binária de cada valor através da função nativa bin.
print(bin(x), bin(y)) # 0b101 0b1101

Que pode ser expandido para 8 bits para 00000101 e 00001101. O ou binário fará a operação ou bit a bit, retornando 1 quando pelo menos 1 dos operandos for 1 ou 0 quando ambos forem 0.
00000101 | 00001101 = 00001101

Perceba que o resultado, neste exemplo, será igual ao segundo operando, então fazer 5 | 13 resulta em 13. Assim:
x |= y
print(x) # 13

Veja funcionando no Ideone.

Aplicação
Uma das aplicações mais comuns desse operador é de definir flags utilizadas na configurações de algum sistema. Flags são comumente valores booleanos, que podem receber apenas valores 0 ou 1. Utilizando o tipo boolean, é comum que compiladores e interpretadores aloquem mais de apenas 1 bit para tal tipo, mesmo que este ocupe, de fato, um único bit. Para contornar isso, é possível armazenar todas as flags em um outro tipo de variável, utilizando o operador |. Considerando que o tipo char seja representado por 8 bits, que é o padrão em C, seria possível armazenar 8 flags diferentes em um char. Com Python, por ser tipagem dinâmica, isso acaba ficando um pouco abstrato, mas a filosofia por trás acaba sendo a mesma.
Vejamos um exemplo com código:
Vamos considerar que nossa aplicação possuem 8 diferentes opções de configuração. Para simplificar, darei nomes genéricos de A a H:
CONFIG_A = 1     # bin: 0b00000001
CONFIG_B = 2     # bin: 0b00000010
CONFIG_C = 4     # bin: 0b00000100
CONFIG_D = 8     # bin: 0b00001000
CONFIG_E = 16    # bin: 0b00010000
CONFIG_F = 32    # bin: 0b00100000
CONFIG_G = 64    # bin: 0b01000000
CONFIG_H = 128   # bin: 0b10000000

Perceba que com esses determinados valores inteiros, cada configuração representará apenas um bit na representação binária. Isso permitirá que se a nossa configuração possuir o valor 0b00000101, significará que as configurações CONFIG_A e CONFIG_C foram definidas. Esta definição pode ser feita através do operador |:
config = CONFIG_A | CONFIG_C  # bin: 0b00000101

Veja funcionando no Ideone.

No código, para se verificar se determinada configuração foi definida, basta utilizar a operação inversa & (e bit a bit). Por exemplo:
def foo(config):
    if (config & CONFIG_A): print("Configuração A foi definida")
    if (config & CONFIG_B): print("Configuração B foi definida")
    if (config & CONFIG_C): print("Configuração C foi definida")
    if (config & CONFIG_D): print("Configuração D foi definida")
    if (config & CONFIG_E): print("Configuração E foi definida")
    if (config & CONFIG_F): print("Configuração F foi definida")
    if (config & CONFIG_G): print("Configuração G foi definida")
    if (config & CONFIG_H): print("Configuração H foi definida")
    print(" -- + --")

Testando a função:
foo(CONFIG_A)
foo(CONFIG_B | CONFIG_D)
foo(CONFIG_H | CONFIG_E | CONFIG_F)

A saída será:
Configuração A foi definida
 -- + --
Configuração B foi definida
Configuração D foi definida
 -- + --
Configuração E foi definida
Configuração F foi definida
Configuração H foi definida
 -- + --

Veja funcionando no Ideone.
Comment: Suspeitei desde o princípio! A explicação da aplicação ficou perfeita (+1)
Answer: Para números inteiros, é o operador de atribuição de compostos bit a bit ou OR. Como todos os operadores de atribuição compostos, a | = b é a mesma coisa que a = a | b. Onde o | é o símbolo de OR bit a bit.

|=    Bitwise inclusive OR
  https://en.wikipedia.org/wiki/Augmented_assignment
|= (bitwise OR assignment) Performs bitwise OR and assigns value to
  the left operand.
  http://python-reference.readthedocs.io/en/latest/docs/operators/

Meu entendimento após pesquisar... O |= atribuirá o valor após executar um OR entre duas variáveis, mas bit a bit.
Por exemplo:
   0101 (decimal 5)
OR 0011 (decimal 3)
 = 0111 (decimal 7)

Outro exemplo:
a |= 10 //Supondo que a vale 12.

   1100 (decimal 12)
OR 1010 (decimal 10)
 = 1110 (decimal 14)

//Dessa forma a será igual a 14.

Da uma olhada nesse link tirei ele como base.
É a mesma pergunta que a sua com bastante exemplos e caso de uso:
https://stackoverflow.com/questions/18884075/usecase-of-in-python
Comment: Boa resposta (+1), faltou só exemplificar com uma aplicação prática.


Sharing folder with no password from WS2012 AD to non-domain computer

Question: I have a network with a domain on Windows Server 2012 and a few clients with Windows 7 Pro and 2 with Windows 7 Home, so I can't add them to the domain.
On server I have a shared folder with software updates (not Windows updates).
On each computer I have software that is updated from this shared folder.
Problem is with non-domain computers. They can't see the shared folder.
I have tried:

I add new user named updates on server, set rights on this user to folder and write it to credentials window in client. It work, but I must do it manually
Second way - this isn't look good for me - I enable Guest account on server then I add permission for guest in Share / Security od shared folder.

What is the best solution to this problem?
Comment: Using the builtin FTP server would be a good fit for anonymous, read-only access.
Answer: If you don't want to enable guest, your choices are:

Manually or scriptomatically map the share with different credentials.
Create an account on the machines with Windows 7 home that uses a username/password combination that's identical to a domain account and do all updating from that account.


question about congruences

Question: I'm attempting to learn something about modular arithmetic.  I'm a retired engineer that obviously has a lot of time on his hands.
My question is this:
Given: $C_1 * x + C_2*y \equiv C_3*(x+y) \pmod K$, does this imply that $C_1, C_2 \text{and} C_3$ are congruent $\pmod K$.  Nothing is divisible by $K$.
Thanks,
Ron
Comment: Well, if that holds for *every* $x,y$, then, yes.
Comment: Thanks for you reply.  I believe that answers my question.
Answer: We can produce examples of non-congruence. For instance, let $K=5$ and let $x=y=1$. Let $C_1=1$, $C_2=3$, and $C_3=2$.  There is really nothing special about these choices.
Comment: Thanks for the reply.  I unfortunately did not state the problem very well.  There are some other constraints such as C1,C2,X,Y are relative prime. I see now that was an obvious mistake.
Comment: It si happens that in the example above, they all are. If you edit your question to better reflect what you are looking for, perhaps I can find an answer.

question about congruences


Scalaz library import technique reasoning

Question: Do any of you know why examples from Scalaz always use this import technique:
import scalaz._
import Scalaz._

rather than:
import scalaz.Scalaz._

?  I'm trying to understand what the reasoning behind the preference is.
Thanks!
Answer: I believe it's because import scalaz._; import Scalaz._ imports all members from the scalaz package and from the scalaz.Scalaz object.
If you will just import import scalaz.Scalaz._, then you only import members from the scalaz.Scalaz object.
Comment: In the first case, also from the `scalaz` package object.
Answer: import scalaz._ imports all [type]classes from the core package.
import Scalaz._ imports implicits which make all these classes useful. Like conversions from standard collections to MA and getting Option wrapper and so on.
So you can use one without another.
I believe this is a conscious design decision to allow us survive any kind of implicit-related issues, like this one.
Comment: Thanks!  This is great additional information to take into consideration.
Comment: Specific elements of a full import can be disabled with the following mechansim: import com.foo.{unwanted => _, _}, and "unwanted" will not be brought in.
Comment: why I really love Scala is because it has tons of great features I needn't learn until the time comes. @pyrony thanks for the tip!


MVVM RaisePropertyChanged

Question: I have a custom datagrid in my application that when its source is empty it should display a message saying "No results", and when the source is not empty it should display the source data and no message should be displayed.
My custom datagrid is:
namespace GenericControls
{
    [TemplatePart(Name = "EmptyDataGridTextBlock", Type = typeof(TextBlock))]
    public class CustomDataGrid : DataGrid
    {
        private TextBlock _txtEmptyDataGrid = null;

        public static readonly DependencyProperty IsEmptyDataGridProperty = DependencyProperty.Register("IsEmptyDataGrid", typeof(bool), typeof(CustomDataGrid), null);
        public static readonly DependencyProperty EmptyDataGridTextProperty = DependencyProperty.Register("EmptyDataGridText", typeof(string), typeof(CustomDataGrid), null);

        public CustomDataGrid()
        {
            IsEmptyDataGrid = true;
        }

        public bool IsEmptyDataGrid
        {
            get
            {
                return (bool)base.GetValue(IsEmptyDataGridProperty);
            }
            set
            {
                base.SetValue(IsEmptyDataGridProperty, value);
                if(_txtEmptyDataGrid != null)
                {
                    _txtEmptyDataGrid.Visibility = IsEmptyDataGrid ? Visibility.Visible : Visibility.Collapsed;
                }
            }
        }

        public string EmptyDataGridText
        {
            get
            {
                if (_txtEmptyDataGrid != null)
                {
                    return _txtEmptyDataGrid.Text;
                }

                return (string)base.GetValue(EmptyDataGridTextProperty);
            }
            set
            {
                if (_txtEmptyDataGrid != null)
                {
                    _txtEmptyDataGrid.Text = value;
                }
                base.SetValue(EmptyDataGridTextProperty, value);
            }
        }

        public override void OnApplyTemplate()
        {
            base.OnApplyTemplate();

            _txtEmptyDataGrid = GetTemplateChild("EmptyDataGridTextBlock") as TextBlock;
            if (_txtEmptyDataGrid != null)
            {
                _txtEmptyDataGrid.Text = EmptyDataGridText;
                _txtEmptyDataGrid.Visibility = IsEmptyDataGrid ? Visibility.Visible : Visibility.Collapsed;
            }
        }
    }
}

My XAML is as follows: my:CustomDataGrid Height="180" Name="dgChildren" SelectedItem="{Binding SelectedChild, Mode=TwoWay}" ItemsSource="{Binding Childrens}" IsEmptyDataGrid="{Binding IsEmptyDataGrid}">

In my ViewModel I have the property IsEmptyDataGrid:
public bool IsEmptyDataGrid
{
    get
    {
        return _isEmptyDataGrid;
    }
    set
    {
        _isEmptyDataGrid = value;
        RaisePropertyChanged("IsEmptyDataGrid");
    }
}

my problem is that even though the RaisePropertyChanged("IsEmptyDataGrid") in my view model is hit, it does not go inside the IsEmptyDataGrid property in the custom datagrid and the empty message is displayed along with the data in the source.
What am I doing wrong?
Thanks you in advance,
Kruvi
Answer: The setter of a DependencyProperty is not always called in this scenario. Use a Callback function instead:
public static readonly DependencyProperty IsEmptyDataGridProperty = DependencyProperty.Register("IsEmptyDataGrid", typeof(bool), typeof(CustomDataGrid), new UIPropertyMetaData(false,Callback));

private static void Callback(DependecyObject d, DependencyPropertyChangedEventArgs e)
{
    CustomDatagrid cdt = d as CustomDatagrid;
    if(cdt._txtEmptyDataGrid != null)
        {
            cdt._txtEmptyDataGrid.Visibility = cdt.IsEmptyDataGrid ? Visibility.Visible : Visibility.Collapsed;
        }
}

MVVM RaisePropertyChanged


null object value when query user data from parse.com database

Question: null object value when query user data from parse.com database
final ParseUser user= ParseUser.getCurrentUser();
    if (user!=null){
        Email.setText(user.getEmail());
        ParseFile file=user.getParseFile("ProfilePicture");
        file.getDataInBackground(new GetDataCallback() {
            @Override
            public void done(byte[] bytes, ParseException e) {
                Bitmap pic= BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
                Bitmap croped=getCroppedBitmap(pic);
                Bitmap scaled=Bitmap.createScaledBitmap(croped, 70, 70, true);
                ProfilePicture.setImageBitmap(scaled);

            }
        });

    }else {
        Toast.makeText(this,"Please Login",Toast.LENGTH_SHORT).show();
    }

my app is showing only bitmap image where obtained email value is null on using log.d("Email:",user.getEmail()+"")
is there any problem in code.
Answer: Sounds like either the user did not enter an email, or the code to save the email to parse is incorrect.  Check on Parse to see if the value exists in your database for that user.  If it does maybe set a breakpoint at the line and inspect the user object you receive.
Comment: Yes user class in parse.com database have values of email


WARNING: recursion requested but not available, 3 servers only one fails

Question:
Primary nameserver at pp.pp.pp.pp
Secondary namseserver ss.ss.ss.ss
third namserver at tt.tt.tt.tt

<code>primary named.conf
acl "trusted" {
  ss.ss.ss.ss;
  tt.tt.tt.tt;
  localhost;
};
</code>

from third

dig @pp.pp.pp.pp www.google.com

I get answers.
But from the secondary:

dig @pp.pp.pp.pp www.google.com

I get
WARNING: recursion requested but not available
and no answer.
If I comment out the third server in the primary's named.conf trusted section, and rndc reload, I get the WARNING: recursion requested but not available.  Uncomment, rndc reload, re-dig, warning it goes away.
The primary sees the secondary as external apparently and is not obeying the trusted setting.
Appreciate any suggestions to fix.
Answer: I found the problem, which leads to a different question.
The secondary server is sending it's query from it's secondary IP address.  It has a secondary on the same interface (eth0:1).
I added that to the trusted lists and it cured the DNS lookup.
But why is it querying from the secondary IP instead of the primary?
Answer: Found the fix for that, too.
in cfg-eth0 after gateway, add:
METRIC=1
And in cfg:eth0:1
METRIC=999

reboot.  cured.
Answer: And found the probably better answer.  Remove the gateway from the alias file (*:1) and do not use the metric setting.  Seems to fix it as well.  I gather 2 gateways are bad


Using server side html+js in phonegap (multiplatform mobile dev)

Question: Phonegap uses html source located in www folder. I was testing what happens if index.html is still in www, but it links to other html that are located in at the server side. It will open the server side html in the web browser instead of handle it as part of the app.
Is there any way to make phonegap work with server side html + js source?
It is not a bad idea if you need to mix usage of libraries (jars + ios libraries), local phonegap html+js with server side dynamic html code (like php output).
thanks.
Answer: Phonegap is designed to deploy a client, a web app inside a native web view. You can write a server side API for returning any information your app may need and you can modify the DOM using JS. There are great templating solutions out there such as mustache.js. The idea is to write the app fully in JS and retrieve information from the server using any of Ajax implementations in your favourite JS framework. You shouldn't need any dynamic JS. Just information to template and insert into the DOM.
Comment: but what if you want to be able to update the app without a full build/upload cycle ?
Comment: In that case you can serve templated html like a normal website, this is what Facebook do with their app. But if you take this approach then your app will never be able to run fully offline.
Comment: This doesn't answer the question at all. Unless the answer is "you can't do that".
Answer: I can't find any reasonable explanation anywhere, but the reason why you can't invoke PhoneGap API's when html is hosted on server-side is because cross-site-scripting issue as the webkit does not allow scripts coming from one domain (i.e. server) be executed on different domain (localhost-device)


Building a simple chat client

Question: Im building a simple chat client which is only supposed to be able to send and receive messages.
Im using a server running on my own computer that sends back whatever message is sent to it, to all the user that's connected to the server.
When I send messages to the server by clicking my "send button", the server isn't sending back the message to me as it's supposed to. So either my output stream isn't working, or my listener for input messages isn't working but can't figure out what is wrong.
I might add, that i don't get any error messages/exceptions and connecting to the server works
<code>public class Chatt extends JFrame implements Runnable{
    private JPanel topPanel = new JPanel();
    private JPanel bottomPanel = new JPanel();
    private JTextArea chattArea = new JTextArea();
    private JButton sendButton = new JButton("Skicka");
    private JLabel chattPerson = new JLabel("Du chattar med: ");
    private JTextField chattField = new JTextField(15);
    private Thread thread;
    private int port;
    private String ip;
    private DataInputStream in;
    private DataOutputStream out;
    private Socket s;

    public Chatt(String ip, int port){
        this.ip=ip;
        this.port=port;
        Konstruktor();
        }
    public Chatt(){
        ip="127.0.0.1";
        port=2000;
        Konstruktor();
        }
    public Chatt(String ip){
        this.ip=ip;
        port=2000;
        Konstruktor();
        }

    public void Konstruktor(){
        setLayout(new BorderLayout());

        chattArea.setSize(70, 50);
        add(chattArea, BorderLayout.CENTER);

        bottomPanel.setLayout(new BoxLayout(bottomPanel, BoxLayout.X_AXIS));
        bottomPanel.add(sendButton);
        bottomPanel.add(chattField);
        sendButton.addActionListener(new sendListener());
        add(bottomPanel, BorderLayout.SOUTH);

        topPanel.setLayout(new BoxLayout(topPanel, BoxLayout.X_AXIS));
        topPanel.add(chattPerson);
        add(topPanel, BorderLayout.NORTH);

        try {
            //s = new Socket("atlas.dsv.su.se", 9494);
            s=new Socket(ip, port);
            }
        catch (UnknownHostException e) {
                System.out.println("Connection failed");
                }
        catch (IOException e) {
                    }
        try{
        in= new DataInputStream(new BufferedInputStream(s.getInputStream()));
        out= new DataOutputStream(new BufferedOutputStream(s.getOutputStream()));
        }
        catch(UnknownHostException e){
                System.out.println("Host unknown");
            }
        catch(IOException e){

        }
        thread = new Thread(this);
        thread.start();

        setTitle("Connected to "+ip+" på port "+port);
        chattArea.setEditable(false);
        setSize(400, 500);
        setVisible(true);
        setResizable(false);
        setDefaultCloseOperation(EXIT_ON_CLOSE);
    }
    public void run() {

            while(true){
                System.out.println("tråden igång");
                try {
                    String temp = in.readUTF();
                    System.out.println(temp);
                    chattArea.append(temp);
                } catch (IOException e) {

                }

            }
    }

    public class sendListener implements ActionListener{
        public void actionPerformed(ActionEvent e){
            String chattString = chattField.getText();
                try {
                    out.writeUTF(chattString);
                    out.flush();
                    }
                catch (IOException e1) {

                    }
                chattArea.append("Du: "+chattString+"\n");
                chattField.setText("");

            }

    }

    public static void main(String[] args){
        //new Chatt("127.0.0.1", 2000);
        //new Chatt();
        new Chatt("127.0.0.1");
    }

}
</code>
Comment: You are digesting most of the exceptions `catch(IOException e){

        }` are you sure you do not have any? - consider putting `e.printStackTrace()` in those.
Comment: Looks like you have pasted client code, and your issue seems to be with the server. You may require to post both.
Comment: Yeah it's only the clientcode, but this is a homework assignment and we've gotten the server from out teacher so im guessing it is working. Don't have access to the source-code to the server nor any documentation, but it's stated the the server sends back whatever message you send to it. Not supposed to call any specific server-methods
Comment: added a printline after all the `catch(IOEXception e)` so pretty sure I don't have any
Comment: Looking at your code, it should work. Looks like an issue with the server. Create a test "echo server" for your purpose and validate your client.
Comment: If your teacher is attempting to make you test web services style applications then you should have gotten some form of documentation (i.e. what the request to the server should look like). I've run into Client/Server issues before because my request was not terminated with a blank line or something similar... See if your teacher can provide any insight into what kind of communications may be allowed or if the server will ever respond with error messages.
Comment: Are you sure the question didn't specify what signals an end of message? My guess would be the server is still waiting for you to finish. For example, a newline might be expected if the server uses readline
Comment: It can be that your teacher's chatserver is still waiting for input, although you have sent some bytes to it. Have you trying to sent either an EOF signal or another delimiter to signal the server that it should start processing the input bytes?
Answer: I can confirm that the chatserver wasn't working correctly. I did build my own server and sending/reciving messages works fine, so nothing wrong with my code.

Building a simple chat client


Private file folder outside document root

Question: I am setting up a private files folder outside of www. I have set the permissions to allow access to the folder by www-data to read and write but not execute, however, my drupal 8 site is now showing an error:

PRIVATE FILES DIRECTORY Not fully protected See
  https://www.drupal.org/SA-CORE-2013-003 for information about the
  recommended .htaccess file which should be added to the
  /var/drupal_data directory to help protect against arbitrary code
  execution.

That link is to an advisory and fixes regarding Drupal 6 and 7. There is no mention of of Drupal 8 and it appears reasonably out of date:

20 Nov 2013

I didn't think apache could even utilise htaccess outside of the www folder in such a way.
Is there some other configuration or permission I need to be changing?
Up until now, for testing, I have had a private folder within the public files directory. I wish to move the private files directory outside the root directory as I am lead to believe it is better for security.
While many guides discuss the best practice of moving this folder outside the root directory few go in to much detail about either why that is beneficial or how that changes the site setup. If there are good recent explanations for this practice I would appreciate being directed to them.
Answer: Looking at the code in system.install that raises this error it just checks if private://.htaccess exists:
$htaccess_files['private://.htaccess'] = [
    'title' => t('Private files directory'),
    'directory' => drupal_realpath('private://'),
];

...

foreach ($htaccess_files as $htaccess_file => $info) {
  // Check for the string which was added to the recommended .htaccess file
  // in the latest security update.
  if (!file_exists($htaccess_file) || !($contents = @file_get_contents($htaccess_file)) || strpos($contents, 'Drupal_Security_Do_Not_Remove_See_SA_2013_003') === FALSE) {
    $url = 'https://www.drupal.org/SA-CORE-2013-003';
    $requirements[$htaccess_file] = [
      'title' => $info['title'],
      'value' => t('Not fully protected'),
      'severity' => REQUIREMENT_ERROR,
      'description' => t('See @url for information about the recommended .htaccess file which should be added to the %directory directory to help protect against arbitrary code execution.', [':url' => $url, '@url' => $url, '%directory' => $info['directory']]),
    ];
  }
}

At no point does it check if your private file directly is actually inside webroot or not.
This means in your case it's a false warning and you can ignore it, as your right in saying there is no point putting an .htacess file to deny access in a folder outside of webroot.
Comment: changing the owner of the folder also seemed to get rid of the warning.


Cohen's kappa and confounders

Question: I'm interested in how certain factors affect the inter-rater reliability of two driving examiners (sitting in the back seats) in deciding whether to pass or fail the examinee.
For example, age, gender, traffic intensity, and number of previous failed tests (all known to both examiners) may affect the examiner decision and possibly the agreement with the other examiner.
One can simply do several sub-group analyses, but I'm not sure this is a good method.
How would you evaluate the effects of such factors on a measure of concordance (e.g., Cohen's kappa)?
Is it possible to "adjust" Cohen's kappa for such factors?
Answer: I don't think kappa is the route to go here, because that requires calculation over multiple "ratees" (here drivers). For each particular driver, the agreement can be either a) One of two things or b) One of three things or c) One of four things, depending on what your interest is:
A. One of two things - agree or disagree. Then you could use this as the dependent variable and use logistic regression.
B. One of three things - both pass, both fail, disagree. Then you would use multinomial logistic or maybe ordinal logistic.
C. One of four things - both pass, both fail, A pass B fail, A fail B pass.   Here you would use multinomial logistics.
For each driver you would have the DV (one of the three just described) and whatever IVs you wanted.
Of course, if you have multiple pairs of examiners, things get more complicated.
Comment: Many thanks for the speedy answer!

Cohen's kappa and confounders


Ranking Order Test

Question: I am analyzing my capstone data - I had students, from different grades, rate 6 feedback forms from 1-6, where 1 was the most helpful feedback type and 6 the least helpful feedback type. Is there a statistical test I can run to assess whether there are differences in ranking order by grade? Is there a test that looks at the rating order as a whole or do I need to run 6 ANOVAs (one per feedback type)?
Answer: What you probably want to do is build a single model with both Feedback and Grade (and their interaction) as independent variables.  So in R notation, you would have something like, Rating ~ Feedback + Grade + Feedback:Grade.  This will allow you to see if there is an effect of Grade overall, but also if there are differences among the different Feedbacks.  Perhaps most interesting, it will allow to inspect the interaction, to see if the ratings for the different Feedbacks are different among different grades.
This is analogous to a two-way anova.
However, because your Ratings are individual Likert-type items, a traditional anova is probably not applicable for this data.
The ideal approach would be to use ordinal regression. This is relatively easy in some software packages, particularly R †.
Another approach would be to conduct a nonparametric two-way "anova" using ARTool.  This is also relatively easy in R.

† Note: link to my own webpage.

Ranking Order Test


Combine two regular expressions in Javascript

Question: I need a regular expression that picks up the author in the following scenarios
Letter from author to recipient regarding topic 11-10-2018
Letter from author, regarding topic 10-11-2018
I tried following the suggestions found here and this is what i have:
let commaRegex =  new RegExp(/(?<=from) (.+?),/, 'gi','$1');
let fromToRegex = new RegExp(/(?<=from) (.+?) (?=to)/, 'gi','$1');
let combinedRegex = new RegExp(commaRegex + '|' + fromToRegex);
let author = document.getElementById('userInput').value.match(combinedRegex);
console.log(author)

But the console.log(author) returns 'null'.
I am using this on Chrome as look behinds are not supported in all browsers. Any suggestions?
Comment: Have you properly read the approaches? They use `regex.source`- have you also tired using that?
Comment: Which word are you trying to get?
Comment: @IslamElshobokshy I am trying to get the name of the author that will always vary but will be between either the words 'from' and 'to' or between 'from' and a ','
Answer: There's no need to combine them "automatically", just use the or operator for the last part:
(?<=from )(.+?)(?=,| to)

https://regex101.com/r/xzlptd/2/
To make it work in all browsers, get rid of the look-arounds
from (.+?)(?:,| to)

and pick the first group from the match: .match(...)[1]
Answer: Apart from the solution provided by georg, just in case you want to combine any other two that aren't as easily substituted by one:
You have to add the sources instead of the whole object:

let commaRegex =  new RegExp(/(?<=from) (.+?),/, 'gi','$1');
let fromToRegex = new RegExp(/(?<=from) (.+?) (?=to)/, 'gi','$1');
let combinedRegex = new RegExp(commaRegex.source + '|' + fromToRegex.source);

let sampleText = 'Letter from Joseph Tribbiani to recipient regarding topic 11-10-2018';

console.log(sampleText.match(combinedRegex));

sampleText = 'Letter from Joseph Tribbiani, regarding topic 11-10-2018';

console.log(sampleText.match(combinedRegex));


Steps to move an ubuntu 14.04 server install disk (damaged) with lvm to a new disk on the same system

Question: Need some clarification on how to move the following :
I have a Ubuntu 14.04 server install with LVM and its disk was failing. I have managed to clone the disk without any errors to a new disk of same capacity (i.e 1 TB ) using GNU ddrescue. I have checked the disk imaged and the data is there and no problems accessing it. Now I would like to put the new cloned disk back into the same system.
Will I have to do anything further to put the new disk on the same system and boot off it ?
I am aware that I may need to change fstab entries only, is there anything else I need to change on the new disk to boot correctly without any errors ?
Comment: It's not clear what your question is, or it is too broad. If you cloned the whole disk (i.e., not a single partition) then everything is exactly as it was before. As if nothing ever happened.
Answer: I resolved it myself.
Once you clone your failed disk to another disk, in case of lvm partitions there is no need of changing the UUID of the disk in /etc/fstab.
Therefore you can just clone the disk and directly boot off it on the new server. Thats it !


Java Quicksort help understand

Question: I'm trying to understand the Quicksort algorithm in java.
I have commented my question below beside the part i'm trying to comprehend.
public class Quciksort {

  static void qsort( char items[]) {
    qs(items,0,items.length-1); /*how can this method reference its parameters before the method is defined (below)?*/
  }

  private static void qs(char items[], int left, int right) //
  {
    int i, j;
    char x, y;

    i=left; j=right;
    x=items[(left+right)/2];

    do{
        while((items[i] < x) && (i<right)) i++;
        while((x < items[j]) && (j>left)) j--;

        if(i<=j){
            y=items[i];
            items[i]=items[j];
            items[j]=y;
            i++;j--;
            }
    }while (i<=j);

    if(left<j) qs(items, left, j);
    if(i<right) qs(items, i, right);
  }

}

Comment: @Tacocat thanks for the response, and sorry for not being to clear. I'm trying to understand how qsort() can make a call to the method of qs() and give values to its parameters, if qs(char items[], int x, int y) has not been created until after?
Comment: @vefthym oh ok, i just assumed since you could not do something like>               {int r=x+y;  int x, y; } you could not do the same to methods.
Comment: I believe @vefthym has answered your question then ~
Answer: You might be confusing Java with C, in which, if I remember well after 10+ years since last using it, it is a good practice, or maybe required, to declare a method before calling it.
In Java, there is no such requirement. Thus, you can call a method that is implemented at a later point in your code.
About the parameters, I am not sure I got your question correctly, but in your example, you are calling qs with the array items as the first argument (as it was given to the qsort method), with 0 for the value of left and the length of the items array -1, for right.

Java Quicksort help understand


Minimum Edges per Vertex in a Graph with a Given Number of Edges and Vertices

Question: Say we have a graph with 6 nodes/vertices.  Say that within this graph, we draw 12 edges connecting the nodes.  Why can we guarantee that at least one of the nodes will have 3 edges connected to it?
I understand why this has to be true when drawn out on paper, but I am wondering if there is a principle or theory that could help in explaining this.  I'd like to understand it from a mathematical perspective rather than pure intuition.  Thanks!
Answer: If all $6$ nodes have degree $3$ or less, then there will be at most $6\cdot 3=18$ ends of edges, which means at most $\frac{18}{2}=9$ edges in total. This contradicts your specification that there must be $12$ edges.
So in fact there must be at least one node with degree $4$ or more.
(You cannot guarantee that there must be a node of degree exactly $3$: the octahedral graph a 4-regular graph with 6 nodes and 12 edges).


How do I extract the values from this data using bash and awk?

Question: I grepped these, how do I extract the values?

...
cavity_2mgl_wt_strip57001.out: Total cavity volume (A3)               : (  1.240E+01)
cavity_2mgl_wt_strip58001.out: Total cavity volume (A3)               : (  2.408E+00)
cavity_2mgl_wt_strip60001.out: Total cavity volume (A3)               : (  4.935E+00)
cavity_2mgl_wt_strip61001.out: Total cavity volume (A3)               : (  1.319E+00)
cavity_2mgl_wt_strip63001.out: Total cavity volume (A3)               : (  1.532E-01)
cavity_2mgl_wt_strip64001.out: Total cavity volume (A3)               : (  1.137E+01)
...

and I need the index # in the filename in bold:

cavity_2mgl_wt_strip76001.out: Total cavity volume (A3)               : (  1.276E+01)

and I need the number in the parenthesis:

cavity_2mgl_wt_strip76001.out: Total cavity volume (A3)               : (  1.276E+01)
Comment: Could you be a bit more err specific?
Comment: I want to create a file that contains the numerical value from the file name. mached with the value in the parenthesis.
Answer: $ ..<commands>.. | awk -F"[:)(]" '{gsub(".*strip|.out","",$1);print $1,$(NF-1)}'
57001   1.240E+01
58001   2.408E+00
60001   4.935E+00
61001   1.319E+00
63001   1.532E-01
64001   1.137E+01

or if your grepped values are already in a file
$ awk -F"[:)(]" '{gsub(".*strip|.out","",$1);print $1,$(NF-1)}' file

Answer: sed 's/.*strip\(.*\).out.*(\([^:].*\))/\1 \2/' file

Answer: Sure is a lot shorter in perl than awk. I don't know how flexible your format is; I put in a few wildcards just in case:
perl -ne 'if ($_ =~ /cavity_[0-9]+mgl_wt_strip([0-9]+)\.out:[^:]+: *\( *([0-9]+\.[0-9]+E[+-][0-9]+)\)/) {print "$1 $2\n"}' in.txt > out.txt

Answer: How about using sed?
sed -e 's/.*strip\([^.]*\).*( *\([^ )]*\) *).*/\1 \2/' infile > outfile

The first capture is of the part of the line between "strip" and the next dot.
Then the line until the last opening bracket is skipped.
The second capture is of the number (with any leading and trailing space removed) between the last pair of brackets.
Answer: In pure bash:
while read line; do
   tmp="${line#*strip}"; index="${tmp%.out*}"
   tmp="${line##*(}";    value="${tmp%)*}"
   printf "%s:%s\n" "$index" "$value"
done < file


Determine if folder or file using Dropbox Python API

Question: I am trying to use the Dropbox API v2 to get a list of all folders and all files, and view the metadata associated with each.
I have been able to make it list all files and folders in my Dropbox root, and return the metadata, however I can't figure out how to determine if the output metadata is for a file FileMetadata or folder FolderMetadata.
When I print the returned values from my query it shows them as File or Folder
import dropbox

def print_metadata(dbfile):
    md = dbx.files_get_metadata(dbfile)
    print (md)

dbx = dropbox.Dropbox('MYAPIKEY')

myfiles = ['/someotherfile.jpg', '/Camera Uploads']

for myfile in myfiles:
    print_metadata(myfile)

File:
FileMetadata(name='someotherfile.jpg', id='id:1234567890', client_modified=datetime.datetime(2013, 1, 15, 20, 51, 3), server_modified=datetime.datetime(2017, 8, 24, 21, 32, 52), rev='1234567890', size=162012, path_lower='/someotherfile.jpg', path_display='/someotherfile.jpg', parent_shared_folder_id=None, media_info=None, sharing_info=None, property_groups=None, has_explicit_shared_members=None, content_hash='1234567890')

Folder:
FolderMetadata(name='Camera Uploads', id='id:0987654321', path_lower='/camera uploads', path_display='/Camera Uploads', parent_shared_folder_id=None, shared_folder_id=None, sharing_info=None, property_groups=None)

However as I can't determine which is which from within python, I can't print specific metadata values such as the file size
print(md.size)

AttributeError: 'FolderMetadata' object has no attribute 'size'

What do I need to do to check if the returned object is File or Folder, or specify to only return folders or only return files in my lists?  That way I should be able to loop through all folders and files throughout my entire dropbox.
Answer: You can use this function/line if you don't want to read through the accepted answers code:
def isFile(dropboxMeta):
    return isinstance(dropboxMeta,dropbox.files.FileMetadata)

Answer: You can use isinstance to check the type of the object, which can be FileMetadata, FolderMetadata, or DeletedMetadata. There's an example here:
https://github.com/dropbox/dropbox-sdk-python/blob/master/example/updown.py#L91
Comment: That was exactly what I was looking for thank you.  So simple but when you don't know you don't know :)
Answer: I find this to be more useful for downstream folder creation.
def get_file_exists(dbx,db_file):
    try:
        md = dbx.files_get_metadata(db_file)
        exists_bool = True
        return exists_bool
    except Exception as error_response:
        exists_bool = False
        return exists_bool

dropbox_exists_boolean = get_file_exists(dbx,'/{}'.format(filename))
if dropbox_exists_boolean == False:
    # create dropbox folder
    res = dbx.files_create_folder_v2('/{}'.format(filename))


my regex work on online test but not in my c# app

Question: I have a regex pattern (?<=base64,)(.*?)(?=") to get part of text.
This pattern works in an online tester web site, but in my C# app always returns false in IsMatch() and return empty in Match().
string imga = _blogPosts[position].PostImage;
if (_blogPosts[position].PostImage !=null)
{
    Regex re2 = new Regex(@"(?<=base64,)(.*?)(?="")");
    bool m = re2.IsMatch(imga); // always retun false
}

Can anyone help me?
reg pic
Comment: Try to escape the two consecutive double quotes => `?=\"` and remove the @ before the string
Comment: There is no `"` at the end of your string. Use `@"(?<=base64,)\S+"` and if you plan to actually get the value, use `re2.Match`.
Comment: @sle1306 `new Regex("(?<=base64,)(.*?)(?=\")");` still return false
Comment: @arman That is because there is no `"`. And `@"\""` is not a valid verbatim string literal.
Comment: @WiktorStribiżew , no end of my string have `"` , `"data:image/Jpeg;base64,9j/OPozEktreOC.....GNY4o1CIijAUAYAA9MUVLRWhif/Z"`
Comment: @arman you are wrong, the *string* you have **does not** contain a single `"` char. Run my code from the answer, and you will get it working.
Answer: What you copied into RegexStorm is just the debugger row data. There is no actual " in your string.
Note that there is no sense using the regex for this task, you might as well use
if (imga !=null && imga.Contains("base64,"))  { /* yes */ }

If you need to actually grab all non-whitespace characters after base64, substring, you might use
Regex re2 = new Regex(@"base64,(\S+)");
Match m = re2.Match(imga);
string result = "";
if (m.Success)
{
    result = m.Groups[1].Value;
}

See the regex demo used above. (\S+) captures into Group 1 any one or more non-whitespace chars after a base64, substring.
But you can really get it using a Substring:
string result = "";
if (imga !=null && imga.Contains("base64,"))
{
    result = imga.Substring(imga.IndexOf("base64,")+7);
}

Comment: You are right. Characters are "a string sign!". My bad brain


Uses of Poisson process in stock price models

Question: If I want to find the probability that a stock is going to touch a support or resistance at least once in the next 5days, can I use a Poisson distribution?
The textbook examples usually say that Poisson distribution is used to model either number of defaults or jumps on a stock price, so was wondering what are some other applications?
Answer: The stock prices are often modeled as jump-diffusion processes. There's a ton of papers and books on this subject. You can also look up Levy processes.
However, judging by the way you framed your question, I think that all this stuff is not applicable to you. You seem to be attempting technical analysis. If this is the case, then you won't believe in any of this stuff :) Otherwise, you wouldn't be doing technical analysis.
Also, you may find more involved audience in Quant SE
Answer: Judging by your question. You may still studying the significance of Poisson process maybe in a college? I would say the best way of approaching such kind of problem is think the time of period, which is 5, and number of occurances, which is more than 1. So we can use poisson process to model by doing total probability - the probability of 0 occurance. 1 - e^(-5E(X))
Comment: It is unclear why this is appropriate or how it would be applied.  In fact, it looks incorrect, because it obviously does not account for intraday stock movements.  Could you (a) state your probability assumptions about the underlying process and (b) perhaps show a small example?
Comment: -1 this makes little sense.
Answer: Some responses to this are incredibly arrogant. It is possible to focus on a different type of event than simply intraday stock prices. Imagine we try to plot a line between two time periods, along with their slopes and correlation coefficients, and count the number of times a stock would touch x rate of change and/or direction x periods forwards, we could theoretically use poisson distribution to calculate the probability an event could happen within a time period. This would be a reasonable heuristic. I am working on an application at the moment to do this, whether i use Poisson Distribution is uncertain but it doesn't have to be statistical in the classic sense to offer insight.
Comment: Could you clarify what you mean by *not* "to be statistical in the classic sense"? Also, if any of the responses do appear arrogant, please be so kind as to flag them for moderator attention.  Because you haven't specified which "responses" you mean (and whether they are answers or comments), and I can detect arrogance in this thread only by reading it into the text of a single (downvoted and commented) answer, I'm unsure what you're referring to.


Win10 NFS Client a SAMBA Killer?

Question: We finally got the last of our recalcitrant Windows 7 users to Windows 10 thanks to MS' recent withdrawal of support from the former, so the entire enterprise now is either Ubuntu 18.04 or Windows 10.
Because Windows 10 has a NFS client, the question now is whether to ditch SAMBA in favor of NFS.
Specifically, does any reason exist to retain SAMBA now that all our Windows clients support NFS?
Comment: In my personal experience SMB & SAMBA has faster throughput than NFS only second to HTTP (yes, even FTP is slower). NFS also has some file state issues such as being slow detecting file changes across the network compared to SAMBA. But these are my personal experience so YMMV and also why I'm not posting this as an answer.
Comment: In my (admittedly limited) experience managing NFS and Samba servers, permissions issues plague NFS in the end user world. In server-;land you can sync UNIX users and then everyone is happy. In the world of end users I have found there to be an never ending stream of "I can't delete that file I created" or "I can't write to that directory Bob created" support cases. Samba seems to deal more gracefully with these issues (mainly by completely ignoring permissions and rewriting everything).
Comment: What OS is your fileserver running on?  If your fileserver is Windows server you should almost certainly be using SMB.
Comment: Sorry, I should have addressed our infrastructure a bit more clearly.our servers are all Ubuntu 18.04.  The business side generally runs Windows 10 natively
Comment: I got time constrained while editing-the above should read:  Sorry, I should have addressed our infrastructure a bit more clearly.  Our servers are all Ubuntu 18.04.  Client machines on the business side generally run Windows 10 natively.  Client machines for some of the more technical staff (nwk admin, dba, finance quants) often run Ubuntu natively and Win10 virtually.
Comment: @slebetman nfs hasn't any caching issue, you only need to use noac in the client side :)
Answer: SMB 3.xx has a better tuned performance over "generic" TCP connectivity and has features like RDMA and multichannel support Microsoft didn't implement with "their" (actually - licensed) NFS client.
Comment: Samba supports both SMB Multichannel & SMB Direct just fine. https://www.samba.org/~metze/presentations/2018/SDC/StefanMetzmacher_SDC2018-SMB-Direct-Status-rev1-presentation.pdf
Comment: @Zoredache By now, I think the Samba developers could very well understand SMB/CIFS better than Microsoft.
Comment: Not sure from the information the OP provided, but I wonder if the advanced SMB3.0 features matter if they are running Samba on a Linux box, or some other non-Microsoft SMB implementation.
Comment: @AndrewHenle sure, I guess I am just used to using the more 'enterprisy' Linux distros like Centos/Debian and it often feels like there is an extremely long lag time between new functionality in Windows, and it being available in the stable Linux release for 'production' usage, and not just in some upstream dev repo.  From what I see it looks like Debian only had the full functionality in Buster(~2017)?
Comment: @Zoredache: Debian did take a while to bring Samba up to date, but before that, I believe you could get Samba packages from Sernet if you wanted commercial support. (Meanwhile I built from sources for my home lab AD...)
Answer: Server Message Block (SMB), the protocol used by the Samba software, might be more easily deployed with sufficient security.  Network File System, abbreviated NFS, has jokingly been called "No File Security".  That's the joking name, but "No File-Level Security" may be a name with some accurate implications.  In other words, NFS security is based on sharing a partition, not an individual file, so file-level permissions are not enforced by the NFS protocol.
From my reading, it is possible for an NFS server to pay attention to files and reject invalid requests.  However, not all NFS software will do that.  The protocol tends to have the client request a block of data on a drive, and a server could fulfill that request by reading the block from the disk without necessarily require paying attention to what file that block is a part of.
Even if you found out that an NFS implementation is secure, what prevents the possibility of a change down the road resulting in a less secure implementation/deployment of NFS?  If you have security concerns, then having an answer to such a question may be very worthwhile.
With SMB, people share directories rather than partitions.  This can help you to feel confident that you're sharing just a directory you want to, and not other directories that are located in a different part of a drive's hierarchy.
Edit: Here comes a new challenger.  A comment to this answer has made a claim that this is off-target.  So, I sought some time-honored documentation that helps to back this up.  And I easily found material backing up claims from my answer:
First and foremost, "Secure Networks Inc." posted a "Security Advisory" from March 7, 1997, titled "4.4BSD NFS File Handles".  (That Hyperlink is from the OpenBSD website: SecList.org BugTraq Mailing List Archive from 1997: 4.4BSD NFS File Handles shows the same thing posted as part of an old mailing list, but adds a header.  PacketStormSecurity: SNI BSD File Handles Advisory also shows the same document.)
This article discusses the block-based nature of how NFS served data (and is likely the source of my understanding of this particular vulnerability).
That document has been hosted by multiple organizations.  Here is a different report, apparently quite unrelated to that document: parts of "Why NFS Sucks", by "Olaf Kirch" of "SUSE/Novell Inc." email@example.com say:

"NFS does not care if it is reiser, ext3 or XFS you export, a CD or a DVD. A direct consequence of this is that NFS needs a fairly generic mechanism to identify the objects residing on a file system." ... "Only the server needs to understand the internal format of a file handle." ... "Linux introduced the concept of the directory cache, aka the dcache. An entry in the dcache is called a dentry" ... "virtually all functions in the VFS layer expect a dentry as an argument instead of (or in addition to) the inode object they used to take." "This made things interesting for the NFS server, because the inode information is no longer sufficient to create something that the VFS layer is willing to operate on" ... "attackers could intercept a packet with valid credentials and massage the NFS request to do their own nefarious biddings."

As for my claim about the nickname,
https://news.ycombinator.com/item?id=10967064 backs up:

in 1987, it was common knowledge among the engineers at Sun that NFS stood for "No File Security"

The first screen brings up a few different vulnerabilities, including trusting someone based on the host name that is reported by the client computer.
Google Books: quoted material from "A Practical Guide to Ubuntu Linux" notes:

Default NFS security is marginal to nonexistent (a common joke is that NFS stands for No File Security or Nightmare File System) so such access should not be allowed outside your network to machines that you do not trust.

eTutorials.org section on NFS Configuration notes:

If you mount your filesystems over the Internet, the transferred files can be interfered and even tampered with at any time (some people joke that NFS is short for "No File Security").
Comment: That's...not how NFS works at all.
Comment: *in 1987, it was common knowledge among the engineers at Sun that NFS stood for "No File Security"*  That's **literally** 33 years ago. -1.
Comment: SMB/CIFS didn't introduce any encryption until 3.0 - in 2012.  NFS had full end-to-end encryption available in NFSv4 in 2000.  So your "If you mount your filesystems over the Internet, the transferred files can be interfered and even tampered with at any time" quote applied to SMB **a lot longer than it applied to NFS**.
Comment: @AndrewHenle : This has just been getting further and further from the answer's focus. May ongoing discussion commence in [our chat room](https://chat.stackexchange.com/rooms/103936/discussion-between-toogam-and-andrew-henle).
Comment: (Maybe you're thinking of iSCSI, or of "pNFS Volume Layout", which is an addon feature for block-device sharing?)
Comment: This is somewhat mangled, but it's true that NFS authentication is extremely poor and you can just impersonate UIDs.
Comment: @pjc50 Only if the NFS server is using `AUTH_SYS` authentication.  And anyone who already has root/admin privileges and the knowledge to get inside their system and spoof UIDs/GIDs over NFS likely has the ability to do the same to SMB, too.  And hack your entire system anyway.
Comment: @AndrewHenle : root may be unnecessary. "hackers" ... "who attacked Unix systems in (say) 1995 had a copy of NFS shell." "a user
can then mount any exported filesystems, performing arbitrary filesystem
operations as any non-root user, assuming they are on a host in the server's
export list.  Such access will commonly result in the user obtaining root
privileges." "userland code was sufficient for authentication. And the mountd returned the root handle of the file system. When you presented that handle to nfsd it would happily serve you even when you are no longer in the exports table."
Comment: @TOOGAM *who attacked Unix systems in (say) 1995* So hacks against Windows 95 are valid arguments against Windows implementations today?
Comment: @AndrewHenle Yes. If there is a protocol that is known for having a ton of vulnerabilities, I would want to know that the protocol has removed such design flaws. At which point, it is essentially a new protocol.  Otherwise, you may be relying on various workarounds that people try to tack on to secure things better, but who knows when an update or a different implementation will fail to implement those same workarounds, or when another issue arises because of the initially insecure design.  Besides, I essentially wasn't promoting absolute final conclusions, just considerations to keep in mind.
Comment: (The reason why Sun was so significant is that Sun invented NFS.)

Win10 NFS Client a SAMBA Killer?


relation between first kind Chebyshev poly and second kind Chebyshev poly

Question: How do you prove following relation between Chebyshev poly of first kind and Chebyshev poly of second kind:
$$dT_n(x)/dx=nU_{n-1}(x)$$
Comment: Since both sequences of polynomials have nice recursions, one idea is to do this by induction. Have you tried?
Comment: I am not sure what you mean...you mean indicator function?
Answer: Since
$T_n(x)
=\cos(n\arccos(x))
$
and
$U_n(x)
=\frac{\sin((n+1)\arccos(x))}{\sin(\arccos(x))}
$,
$\begin{array}\\
T_n'(x)
&=-(n\arccos(x)))'\sin(n\arccos(x))\\
&=-n\frac{-1}{\sqrt{1-x^2}}\sin(n\arccos(x))\\
&=\frac{n}{\sqrt{1-(\cos(\arccos(x))^2}}\sin(n\arccos(x))
\qquad\text{(this is a sort of sneaky part)}\\
&=\frac{n\sin(n\arccos(x))}{\sin(\arccos(x))}\\
&=nU_{n-1}(x)
\end{array}
$
Comment: I see...is there a proof for conditions Tn(x)=cos(narccos(x)) and Un(x)=sin((n+1)arccos(x))sin(arccos(x)) you used? Or it is theorem that people can directly use it without proving?
Comment: These follow from the recurrences.


Why Specifically Use Poisson Regression For Count Data?

Question: I am a MBA Student that is taking some statistics courses, a colleague recommended this site as a useful resource! So far there seems to be a lot of interesting information here! I posted a previous question on the same topic and was advised to break it into individual parts.
We are learning about "Count Data" - for example, we are interested in learning how to make a model that predicts the number of complaints a customer might file. In a previous stats course, I learned about basic regression models and the Poisson Distribution. Now, I am trying to understand how these two can be put together!

Why should Poisson Regression be used for Count Data instead of a "vanilla linear regression"? I understand the basic argument : Count Data is by definition discrete and you would rather use a model in which predictions are always discrete (i.e. Poisson Regression) ... but to me, this seems like a formality. Couldn't I just use a linear regression model and round the predictions to the nearest whole number?
Comment: Related https://stats.stackexchange.com/questions/204696/what-regression-model-is-the-most-appropriate-to-use-with-count-data
Comment: Welcome to Cross Validated! A (perhaps) mind-blowing fact about Poisson regression is that it can predict any positive value, not just positive integers. (Once you dig into the details of how Poisson regression and generalized linear models in general work, this is entirely unsurprising, but it still might come as a surprise.) Consequently, having to round predictions from a vanilla linear regression is not a valid criticism of linear regression that would be resolved by Poisson regression.
Comment: Which "linear regression model" might you have in mind?  Probably not OLS, because that's inconsistent with Poisson distributions (except for large parameter values, where those distributions begin to become indistinguishable from Normal distributions).
Comment: @whuber I got the feeling that the OP meant shoehorning data into vanilla OLS. (We're always able to click "add trendline" in Excel, after all.)
Comment: I tried to answer a similar question at [Goodness of fit and which model to choose linear regression or Poisson](https://stats.stackexchange.com/questions/142338/goodness-of-fit-and-which-model-to-choose-linear-regression-or-poisson/142353#142353)
Answer: Because you're an MBA student and not a statistician, I will list a couple of reasons which may be applicable.

First, note that expected number of complaints can not be negative.  A customer can not complain -2 times on average.  So there is an explicit requirement that the prediction be positive.  It need not necessarily be an integer (because regression predicts the expected number of complaints, and expectations or averages need not be integers).  The poisson makes this natural since the canonical link is the natural log, meaning $E[y] = \exp(X\beta)$.

You may say, "well, I can just take the log of my counts and presto, I can use OLS".  This could work, say if there are a moderate number of complaints.  However if you include customers who issue 0 complaints, then we have a problem since $\log(0)$ is undefined.  Again, this is no problem for the poisson.  The support of the poisson distribution is on the non-negative integers, so no matter what $\lambda = \exp(X\beta)$ is, 0 is always a possibility though it may not be probable.  Excluding the 0s would mean you're modelling the expected number of complaints for those people who complain, which is different.  You could do the $\log(1+y)$ trick, but its just more details to keep track of unnecessarily in my opinion.

Lastly, were you to adopt a Bayesian perspective and compute a prediction interval, the prediction interval will necessarily be within a realistic range without you having to intentionally force it to be non-negative.
Comment: Wow, I didnt expect an answer so quickly! Thank you!
Comment: I am still a bit confused - why can't I take the logarithm, and just say customers with 0 complaints have 0.1 complaints (for argument sake) and then fit a regular OLS model? And then, I just round the predictions down? E.g. If the model predicts 0.872 complaints, I just say its basically 1 complaint. And if the model predicts 0.21 complaints, I say its 0 complaints. Wouldn't this work as well?
Comment: First, the 0.1 is arbitrary. Call this offset $\delta$.  If $\delta$ is too small then $\log(\delta)$ will be very big and negative and could harm your model by having high leverage.  You can do anything you want, but its just more that could go wrong.  A poisson regression is no harder than a linear model.  Additionally, there is no need to round the predictions because the predicted average need not be an integer.  If you want to predict any one user's number of complaints, a different method is needed.
Answer: Short version:
Poisson regression deals with some of the quirks of modeling count rates, including the requirement that rates be positive and the presence of zero count cases in the data, which will cause problems in ordinary least squares.
Longer version:
When you are dealing with count data, you will generally assume that the counts come in at some average rate (which is possibly affected by one or more independent variables -- a.k.a. "covariates"), but the actual counts that you see will vary around that average.  This variation is assumed to be truly random, not predictable even in principle, so your model is going to try to predict that average, and in particular it's going to try to tease out the influence of your covariates on the average.  With that in mind, let's think about how we might model the system.
The first thing we'll note is that although the counts are integers, the average need not be.  It's perfectly fine to have an average rate of 3.5 counts per minute, even though we will observe an integer number of counts in each actual minute, so the discreteness isn't going to cause us any problems, which is good.
The second thing we have to consider is that the average rate must be positive, which is problematic for ordinary least squares (OLS).  If the counts are low, and the effect of the covariates is strong, then we could easily end up with a model that predicts a negative or zero rate, which is not good.  One way around this problem is, instead of having our model predict the average count rate, we could have it predict the log of the rate.  The log of the rate can be any real number, so we don't have to worry about our model giving impossible results.  So, you could do an OLS fit to the log of the counts and interpret the model predictions as the log of the rate.
Unfortunately, there's still a problem.  If the rate is low, then you will occasionally get zero counts, and with the log of zero being $-\infty$, these cases are going to mess up your fit.  You could try setting a small positive number, say $10^{-8}$ as a minimum, but if you do that then you'll find that your answer depends on the precise value that you select for that cutoff.  Here's where Poisson regression comes to the rescue.
In nature, a lot of counting processes show a specific relationship between the average number of counts and the distribution of observed counts.  That relationship is called the Poisson distribution, and if we call the average rate $\lambda$, then the Poisson distribution looks like this:
$$
P(y=k; \lambda) = \frac{\lambda^k e^{-\lambda}}{k!}.
$$
The important thing here is that P(y=0) doesn't blow up this expression; it gives a perfectly well-behaved number.
So, the name of the game is that we let $\lambda$ be a function of our coefficients:
$$
\log(\lambda) = \alpha + \beta_1 x_1 + \beta_2 x_2 + \ldots,
$$
and we solve for coefficients $\alpha$, $\beta_1$, etc. that maximize $P(y_i = k_i; \lambda_i)$ for all of our observations. This takes care of the requirement that $\lambda > 0$, deals with any zero-count observations, and as an added bonus accounts for the fact that for low rates the distribution of observed counts tends to be rather skewed (OLS is based on the normal distribution, which is symmetric).  All of these are desirable, and that is why Poisson regression is preferred to OLS for count data.
These kinds of models, by the way, are called Generalized Linear Models (GLMs), and Poisson regression is just one possibility.  For example, if we had started with the assumption that our observations were True/False observations of a binary experiment, and that our model had to produce probabilities bounded by 0 and 1, we would have arrived at a formula for logistic regression, a type of GLM for predicting probabilities of binary experiments.  There are a whole slew of GLM formulations out there for different kinds of problems, with Poisson regression and logistic regression being the most common.


Quartic Solution on Wikipedia special cases problem $S=0$ how to "change the choice of cubic root"?

Question: So, I've posted a question regarding Wikipedia's quartic page.  This was from the first question.

I'm trying to implement the general quartic solution for use in a ray tracer, but I'm having some trouble. The solvers I've found do cause some strange false negatives leaving holes in the tori I'm testing with.
Most implementations use the depressed quartic solutions, I don't understand the math involved and can't figure out why I'm having false non-intersections (link to layman explanation would be great). So I'm trying to implement the general solution at this wikipedia page. I got the stuff up until the special cases implemented, but at that point I have an issue.

With lots of rays being traced most of the special cases become common.  I've found a set of coefficients that Wolfram Alpha tells me has two real roots, but my code was just returning NaN, further searching I found my S was coming up as $\sqrt{-4.9 \times 10^{-11}}$ Floating point precision error, means this should equate to 0, so I need the special case for S=0, it says we need to "change choice of cubic root in Q" but it does not explain how to do this.  I did try changing the sign of Q when S=0, but that doesn't work.  Does anyone know what this means and how I can do it?
Comment: For most practical purposes, the "exact" solution of the quartic in radicals is not very useful: besides being complicated, it can have severe roundoff errors.  A more useful idea might be to isolate the roots in intervals using Sturm's theorem and then approximate them using Newton-Raphson.
Comment: @RobertIsrael interesting point, I've already seen some round-off errors as differences between my discriminant and WolframAlpha's, fixed by going from calculating the 16 terms outright, to using the $\Delta_1^2-4\Delta_0^3=-27\Delta$ identity.

Do you have a good link for the Strum's/Newton-Raphson?
Comment: @MatrixPeckham https://en.wikipedia.org/wiki/Root-finding_algorithm is a good starting point. I second the recommendation that you may as well use a rootfinding algorithm - especially since whatever square root/cube root algorithm you're finding is doing that behind the scenes anyway.
Answer: Try this version. Given,
$$x^4+ax^3+bx^2+cx+d=0$$
then,
$$x_{1,2} = -\tfrac{1}{4}a+\tfrac{1}{2}\sqrt{u}\pm\tfrac{1}{4}\sqrt{3a^2-8b-4u+\frac{-a^3+4ab-8c}{\sqrt{u}}}\tag1$$
$$x_{3,4} = -\tfrac{1}{4}a-\tfrac{1}{2}\sqrt{u}\pm\tfrac{1}{4}\sqrt{3a^2-8b-4u-\frac{-a^3+4ab-8c}{\sqrt{u}}}\tag2$$
where,
$$u = \frac{3a^2-8b}{12} +\frac{1}{3}\left(v_1^{1/3}+\frac{b^2 - 3 a c + 12 d}{v_1^{1/3}}\right)$$
and $v_1$ is any non-zero root of the quadratic,
$$v^2 + (-2 b^3 + 9 a b c - 27 c^2 - 27 a^2 d + 72 b d)v + (b^2 - 3 a c + 12 d)^3 = 0$$
P.S. This is essentially the method used by Mathematica, though much simplified for aesthetics.
Comment: @MatrixPeckham: How does this method fare on your ray tracer implementation?
Comment: It might work, but after adding complex numbers to my existing code I think I have it working, but need to test more.
Answer: If $\alpha$ is a root of $x^3 = A$, then so too is $\gamma \alpha$, where $\gamma$ is a complex cube root of unity.  This follows since $(\gamma\alpha)^3 = \gamma^3\alpha^3 = \alpha^3 = A$.
The nontrivial values of these are given by $\gamma = -\frac{1}{2} +  \frac{i\sqrt{3}}{2}$ or $\gamma = -\frac{1}{2} -  \frac{i\sqrt{3}}{2}$.
In your case, you need to replace $Q$ with $\gamma Q = \left(-\frac{1}{2} +  \frac{i\sqrt{3}}{2}\right)Q$ in this case, if I'm reading Wikipedia's algorithm correctly.
(Implementation might be easier if you used the note that this case is always accompanied by the depressed quartic being biquadratic, in which case the solutions follow from applying the quadratic formula.)
Comment: This would require Imaginary numbers, which my implementation language does not natively support, The biquadratic method requires a re-write of the rest of my code... I may implement the imaginary numbers just to make everything consistent with the overall formula.
Comment: Even for cubic equations, you can't get by without complex numbers.  That is, if an irreducible (over the rationals) cubic has three real roots, those roots can't be expressed using only real radicals.  This would extend to a quartic that is the product of a linear factor with rational root and one of those cubics.  See https://en.wikipedia.org/wiki/Casus_irreducibilis
Comment: That is true, however my need is for real roots only (ray-torus intersection) so I was hoping to scrape by without, but I have since implemented complex numbers and I think my code is working, however I don't know for sure, I need some more test cases with known solutions.
Answer: As noted in a comment by Robert Israel, if you are using floating point numbers, it's better to use a root-finding algorithm than the exact formula for quartic roots, as root finding algorithms tend to have much better numerical stability, and avoid issues like this one. They also tend to work much more generally (i.e., they won't be limited to just a single degree of polynomial, or just polynomials of degree < 5).
Answer: If you are interested in the various ways of representing the quartic solution symbolically, the SymPy code goes through a few methods and tries to pick the best one for the given polynomial. There are also some useful references there for further reading.


Is it necessary to learn a new scale to improvise over it?

Question: I'm a beginning guitar theory learner and I'm confused about certain things when improvising/soloing/figuring songs out. I've just started learning, and certain things seem hard to imagine.
Let's say I learn the major/minor/pentatonic scales and I become decent at improvising, soloing and figuring songs out by ear. This, however, would only apply to the scales I know. Let's say I hear a song that seems to be based on an exotic scale I don't know (something typically Japanese, Indian, etc).
Let's say I want to improvise to that song, or I want to figure the song out. Do I need to figure out the scale and learn all the possible patterns first, or does a good guitarist just do it by intuition? What's the process of this kind of stuff?
Answer: Most scales, or sets of notes, will have a fairly close relationship with each other.
The major scale has the same notes as the natural minor, just centred on a different note. The major pentatonic has the same notes as its relative minor pentatonic. The blues scale has pentatonic notes plus one more. Modes all use the same notes as their parent scale.
So, knowing those, a new scale will most likely be easily adaptable from one already known.
Asian, Indian and Eastern scales may contain notes which are 'in the cracks', and consequently, it'll be impossible to relate those to known Western scales.
So, to sum up, if a new scale is diatonc, adapt your knowledge.If it has quarter tones, etc., then it'll be possible to play on guitar, but a bit of bending may be required,but based on what you already know.
Comment: I meant actual diatonic scales like these http://www.jazzguitar.be/exotic_guitar_scales.html, not genuine eastern music which has entirely different theory than western. Knowing this, do you think it's necessary to learn these scales and all their patterns explicitly later on, or will it be easy adapting basic scales to these?
Comment: @Darren Ringer - that quote from Longfellow, mid 19th C, was probably made with no knowledge of Eastern music - he was American, and not a musician. Just because someone famous said something, doesn't make it true ! "What do you mean, the world's not flat ?"
Comment: Adapting is what we do in life anyway. Or should do. Look for patterns amongst new stuff that remind you of things you already know, and work from there.For example, some folks will learn,say, a Dorian scale from scratch, whereas I see it as the parent scale, but going from the second note - same pattern as major.
Comment: So basically it depends on the guitarist, but there's always a bit of new practice involved, as far as I can understand. Thanks.
Comment: Classical western theory has plenty of shortcomings, but to say anything has 'entirely different theory than western' is refusing to bridge the communication gap.  I know western theory can fully accomodate eastern traditions, and I wouldn't be surprised if western scales can be seen as a special case in eastern theory. I get somewhat of a bad feeling in my gut for even using the terms 'eastern' and 'western' in the 21st century, though, at least particularly in regard to something like this. Music is supposed to be a universal language, yet we insist on our own vocabularies.
Comment: @Tim: I'm honestly not sure what you mean.  I was replying to Ahmed's claim that it is 'entirely different theory'.  If you mean my use of the term 'universal language' I was definitely not borrowing the phrase from Longfellow, regardless of whether he said it.
Answer: Scales are double-edged swords, if you don't learn them at all it becomes harder to improvise and transcribe especially for beginners (it's easier to play with 5 or 7 notes instead of 12 if you're just starting out).
But if you rely on scales too much, you'll be missing out on a lot of interesting ideas. What you need to develop as a musician is the ability to recognize all twelve notes regardless of the context. You don't always get to know the chords and scale before improvising ; you'll have to listen and identify some notes before you do anything else.
For example: you may hear a minor third, a flat seventh and start with an idea over the minor scale, but suddenly you feel like playing a major sixth and then you realize that Dorian was the scale you were looking for. If you don't know well enough what a major 6th sounds like, you probably would have played the minor 6th and sounded awful in that particular context. But because you know very well what that 6th sounds like, you confidently use it in your solo, and you can always use the minor 6th later on as well if you feel it would sound nice again, or you like the tension it would create...
If a person were to analyse you solo, they would probably say that it is in Dorian and moves the Aolian mode, and that it's a modal solo in general... But you couldn't care less because you're just expressing whatever comes to your mind. You didn't make a conscious and calculated decision of using whatever scale degree over a given chord because the theory said you could. I'm not against thinking that way, we all rely on theory at some point. And many guitarist will explain their work that way but it doesn't mean that they created it that way. Don't learn more theory than you can handle. I made that mistake as a beginner, and I ended up using theory too much when playing. Theory can grantee your next note is going to sound nice, but it doesn't get you the note you want. For that you need ear training (a lot of it).
So how should you learn the scales. The approach I would recommend focuses at learning what the additional degrees (minor degrees in a major context, major degrees in a minor context, and the odd sounding flat 5th and flath 2nd):
   Dorian : teaches you what a major 6th sounds like in a minor context.
   Mixolydian : teaches you what a flat 7 sounds in a major context.
   Phrygian : teaches you the minor 2nd
   Blues scale : teaches you the flat 5th, and minor third in a major context (it's common to use the major third in blues as well)
Experiment with these modes, and focus on what the altered notes (compared to the minor (Aolian mode) and major (Ionian mode) scales) sound like. There are two more mode I didn't talk about which are Lydian and Locrian. Locrian is just a mess, it doesn't even have a perfect fifth, I never came up with an idea that uses the Locrian scale, it's a waste of time in my opinion, because event when I want to use a lot of flat 5th and flat 2nds, I can't give up the perfect fifth. Lydian is like the major scale, but with a sharp 4th. I sounds very nice but it's confusing to the ear, and it won't help you learn as much as the others.
So what I'm saying is learn new scales to better recognize all 12 notes, and you'll notice that these new notes will start to stand out for you as you listen to music, and you won't be too much concerned about which scales to use or learn. It will give you much more freedom when playing. So you may indeed end up using a scale you have never used before. But be careful, after learning these new "exotic" notes, never use them just for the sake of using them, keep it natural! You goal is to play the music in you hear mind, not being a smartass.
Answer: To use a scale one has to at least know what the relationship is between the degrees. It's not necessary to learn the pattern all over the fretboard in order to use the scale. One can experiment by playing it on one string, for example.
Most improvisation is built on a foundation of theory and pre-learned patterns, very few players "just play", so as a beginner, I think it's a good approach to begin with the scale...
Comment: Agree- Begin with a scale then, see where you can break it to stop it getting too stale :-D


O que é codificação elegante?

Question: Sempre vejo algumas respostas de perguntas sugerindo que se fizer algo de forma x é mais elegante que a forma y. Comecei pensar o que seria elegante no nosso contexto.
9 páginas de 50 registros com o termo: https://pt.stackoverflow.com/search?q=elegante
Dicionário:

que se caracteriza pela harmonia, leveza ou naturalidade.
  "traje e."
  2.
  frequentado por pessoas elegantes (diz-se de lugar ou ambiente); seleto.
  3.
  requintado na escolha das palavras; apurado, fino.
  "sintaxe e."
  4.
  moral ou intelectualmente correto; honrado, nobre.
  "disputa e."
  5.
  relativo às pessoas elegantes ou a seus atos.
  "jantar e."
  6.
  adjetivo e substantivo de dois gêneros
  que ou quem revela bom gosto na escolha de suas vestes e no modo de usá-las.
  "homem e."
  7.
  adjetivo e substantivo de dois gêneros
  que ou quem revela correção e fineza nas atitudes; distinto, delicado.

Quando este termo começou ser utilizado em programação? Em programação o que pode-se definir como elegante?
Comment: **Relacionado:** [O que define um código limpo?](http://pt.stackoverflow.com/q/33702/3635) - obs: desde que explicado que não é algo padronizado ou técnico ou qualquer coisa assim, a pergunta me parece valida se a(s) resposta(s) forem bem explicadas acho, o problema são as respostas ruins que costumam atrair, mas nem por isso deve deixar de perguntar.
Comment: Apesar de ser uma excelente pergunta, acho que pede respostas baseadas no achismo. Coisa que, se não me engano, é contra as diretrizes do stackoverflow. Correto ?
Answer: Primeiro entenda que a avaliação sempre será subjetiva, ainda que possa haver critérios objetivos.
A definição do dicionário é boa. Claro que não ajuda entender os pontos a considerar, mas também acredito que nem em um livro dá para definir de forma concreta. E existem vários que tentam fazer isso.

Obviamente que todo código elegante é correto, isso eu coloco como hors concours. Não basta "funcionar", tem que ser correto. Ele faz o que deveria fazer em qualquer situação planejada e lida bem com as situações não planejadas.

Acho que a principal característica é o código ser legível por profissionais qualificados. Note a diferenciação. Ser legível para leigos, estudantes, amadores os pseudo-profissionais não é um requisito para ser elegante. O código deve ser facilmente percebido, ainda que eventualmente precise de comentário que façam sentido.

Outra característica muito forte é que ele deve ser simples, estúpido. O problema pode ser complexo, a solução deve ser simples, que é o oposto de complicado.

Ser um código limpo é outra característica. Não deve ter nada que não precise. Não deve ter mais de uma responsabilidade. Mas também não deve encurtar ao ponto de não ser fácil entender o que está ocorrendo ali. De nada adianta criar um monte de camadas para atingir este objetivo e piorar outro.

O código deve ser claro e expressar bem o que deseja. Deve ser uma solução óbvia. Deve ser tão conciso quanto possível sem comprometer a legibilidade. Para isso deve ter as abstrações certas, e o nível abstrativo adequado.

Geralmente ele deve fazer da forma como já é conhecida por todos e não tentar inventar uma forma esquisita sem necessidade. Deve escolher os algoritmos adequados. Não reinventar a roda, no bom sentido.

Deve seguir convenções de codificação estabelecidas em termos gerais, pela linguagem e pela equipe. Assim fica mais fácil acompanhar o que está escrito. Código elegante está bem formatado, não tem brancos demais e nem de menos (vertical ou horizontal). Não tem abreviaturas desnecessárias, usa nomes significativos.

Deve ter uma forma que facilite a manutenção e evite bugs. Você pode confiar no código. Pode mudar sem susto?

Provavelmente ele não deve fazer otimização prematura. Mas ele deve ser tão rápido quanto possível sem comprometer outras características. E é claro que deve ter a performance desejada, se possível. Mas mais que ser rápido, ele precisa ser eficiente.

Não deve ser clever. Essa é uma palavra geralmente usada para indicar que o programador tentou fazer um código muito inteligente, performático, curto, que parece fazer algo curioso, que se aproveita de alguma característica da linguagem ou da matemática para obter o resultado, mas que não é facilmente compreensível. Clever code pode parecer elegante, mas é o oposto.

Você só obtém isso se domina a computação (no sentido matemático e técnico), sabe como o computador, a linguagem e outros recursos funcionam.
Só pode fazer código elegante se entende o problema que está resolvendo e fez um planejamento prévio do que precisa, do que pode dar errado, de todas necessidades.
Então dá para dizer que só escreve código elegante sempre, quem tem experiência, e não é teimoso :) Claro que copiar código elegante de outros faz aquele trecho elegante, mas pra produzir código inovador elegante precisa "chão".
Elegância é fazer o que é mais adequado para aquele caso. E seguir regras prontas cegamente não ajuda muito. Por isso boas práticas não ajudam diretamente a fazer código elegante. São coisas distintas.
O código elegante é, possivelmente, aquele que alguém te mostra como você poderia ter feito e você fala "por que eu não pensei nisso antes?" :).
Um código WTF não é elegante.
Tem linguagens que incentivam código elegante, outras nem tanto.
Ficou mais claro? Talvez não, mas é o tipo de coisa que não se mostra no papel. Como o nome indica, envolve (bom) gosto.
Coloquei no GitHub para referência futura.
Comment: Ficou bem mais claro. Quando perguntei pensei que talvez seria difícil de explicar, mas creio que entendi a "essência".
Answer: Excelente pergunta.
Embora isto possa trazer várias interpretações baseado na experiência profissional de cada desenvolvedor, o destino final de um código elegante deve atender alguns requisitos que, no final das contas, leva-nos a um código que atenderá a exigências de qualidade para alguém com vários anos de experiência na linguagem.
Listei alguns pontos abaixo, onde alguns deles podem ser entendidos como consequência natural dos outros, mas é importante listar todos:

O código é direto e objetivo no seu propósito, não fazendo nada além de sua responsabilidade.
O código evita "super engenharias" desnecessárias para o tamanho do problema que está disposto a resolver, com base nas informações disponíveis no momento.
Classes, métodos e variáveis tem nomes bem definidos, fazendo exatamente o que espera-se deles ao ler o código.
Não viola aspectos das boas práticas da linguagem e não contém vulnerabilidades óbvias.
A solução aproxima-se do mínimo possível de código utilizado, não sendo tão curta a ponto de ficar indecifrável e não tão longa a ponto de desviar a atenção do entendimento.
O código é facilmente testável por testes automatizados.
Não viola outros pontos importantes não-funcionais, como segurança e desempenho, cumprindo o primeiro e sendo (ao menos) aceitável no segundo.
Os comentários para descrever o que o código faz são ausentes, pois o código já é claro o suficiente. Comentários são encontrados apenas para esclarecer dúvidas de negócio.
Se o código é contestado, o autor consegue facilmente defendê-lo, pois foi ele tem base em diferentes fontes para esclarecer sua qualidade, como: bons livros, autores, Internet (Stack Overflow, por exemplo) e ferramentas da área. Isto é importante, pois o entendimento de um código elegante pode divergir muito entre os desenvolvedores e uma "comprovação" pode ser necessária para consenso.

O que é codificação elegante?


Is the P-adic limit $\lim_{n \to \infty} p^{p^n}$ 0 or 1?

Question: I was playing around making up some limits of things I thought would be interesting to check out and came to something I thought was quite simple on the surface but found it wasn't very consistent with other things in my mind.
$$\lim_{n \to \infty} p^{p^n} = 0$$
To me this seemed very obviously 0, since more and more times you multiply p by itself you really get smaller and smaller with respect to the p-adic metric. But something about the concept of exponents and "counting" just doesn't seem to make sense in the context of p-adic numbers since there is no ordering.
In talking about a similar limit for the p-adic logarithm I've heard it mentioned that the exponent would be getting smaller and smaller, so perhaps it's just as fine to interpret this as,
$$\lim_{n \to \infty} p^{p^n} = p^0 = 1$$
Which is perhaps a bit dubious at first, but if I were to expand this as the Mahler series,
$$p^x = \sum_{n=0}^\infty (p-1)^n \binom{x}{n} = 1 + (p-1)x + (p-1)^2\frac{x(x-1)}{2} + \cdots$$
Then I run into this issue of taking the limit from this definition a bit more seriously,
$$\lim_{n \to \infty} 1 + (p-1)p^{n} + (p-1)^2\frac{p^{n}(p^{n}-1)}{2} + \cdots$$
$$=1$$
However maybe I'm doing some sort of circular reasoning or even running into simply some kind of choice of convention here, I'm not quite so sure.
Answer: Your first thought was exactly correct: the limit is $0$.  You are getting confused here about the domain of exponentiation: $p^n$ is only defined when $n$ is an integer, not when $n$ is a $p$-adic number.  So you shouldn't be surprised if this function ends up being ill-behaved when you think of $n$ as a $p$-adic number instead.  In particular, the function $n\mapsto p^n$ is not continuous as a function $\mathbb{Z}\to\mathbb{Q}_p$ when you give $\mathbb{Z}$ the $p$-adic topology, so you cannot compute the limit of $p^{p^n}$ by taking the limit of the exponent as you have done.
(You might try defining $p^x$ for $p$-adic $x$ using the $p$-adic exponential and logarithm by $p^x=\exp(x\log p)$.  However, this does not work since $p$ is not in the domain of the $p$-adic logarithm.)
Comment: Tangentially, one can define $g^x$ for $p$-adic integers $x$ if $g\in 1+p\Bbb Z_p$ (and then I imagine it is continuous).
Comment: Ahhh I see, I had assumed that it was continuous, but since it's not, Mahler's theorem is invalid here. Thanks!
Answer: I hope I'm not adding to your confusion by saying this:
In your expression $p^{\text{something}}$, the base, $p$, is a $p$-adic integer, but the exponent "something" is just an ordinary counting number, a genuine integer (though it can be negative as well). It is definitely not a $p$-adic number. So you have to think of what you've written as "$p^{\text{huge}}$", thus an expression with limit zero.
EDIT:What I originally wrote below was careless and wrong, since I interchanged $t$ and $z$ in the formula.
And now I'm really going to undo everything and confuse you totally. For $p$-adic numbers of form $u=1+pa$, where $a$ can be any $p$-adic number, in other words, $\vert u-1\vert_p<1$, you may raise to an exponent that is not an ordinary integer, but in fact a $p$-adic integer in $\Bbb Z_p$. That is, you can define $(1+pa)^z$ for $a,z\in\Bbb Z_p$. Just use the Binomial Theorem that you learned in Calculus,
$$
(1+t)^z=1+zt+\frac{z(z-1)}{2!}t^2+\frac{z(z-1)(z-2)}{3!}t^3+\cdots\,
$$
and verify that with $t=pa$ and $z$ as I specified, the whole mess is $p$-adically convergent. This takes some work, but the expression is continuous in both $a$ and $z$, just as you had been hoping in the other, worse, situation.
Comment: Aha I see now, I guess in a sense this was sort of my starting point. I was recognizing by Euler's theorem that we have $x^{\varphi(p^n)}\equiv 1 \mod p^n$ and so naturally I recognized the relationship, $\lim_{n \to \infty} \frac{x^{\varphi(p^n)}-1}{p^n} = \frac{1}{p} \log(x^{p-1})$. Now since $x^{p-1}$ basically sneaks in every single x into being of the form $u=1+pa$, the only kind of value I couldn't was numbers of the form $u=pa$ which in a sorta vague way took me to thinking about the limit in the original question here.
Answer: This seems to be a classic case of unjustified interchange of summation and limits.  In truth, the Mahler series has only finitely many (non-zero) terms for any given $n$, in fact when $x$ is a power of $p$ I think it only has two: the $n=0$ term and the $n=x$ term.  But it is not necessarily the case that the limit of the sums is equal to the sum of the limits.  Your series is highly analogous to the following sequence of infinite sums:
$$
1 + -1 + 0 + 0 + 0 +\cdots, \\
1 + 0 + -1 + 0 + 0 + \cdots, \\
1 + 0 + 0 + -1 + 0 + \cdots, \\
1 + 0 + 0 + 0 + -1 + \cdots, \\
\vdots.
$$
It is obvious that in each row, the sum is $0$, so the sequence is identically $0$ (and thus clearly converges to $0$).  However, if you instead take the limit of each term, the first column converges to $1$ and all other columns converge to $0$, so the sum of the limits is $1$.
Comment: @StevenGrigsby Why do you assert that the rest of the terms are divisible by $p$?  The term that contains the coefficient $\binom{p^n}{p^n}$ is not divisible by $p$, that's the whole point.
Comment: Oh, you're completely right. I think I'm so used to using power series that I forget that I'm not working with them sometimes.
Comment: I don't think this is quite as similar, since I'm taking the limit after plugging in $x=p^n$ to that series. At that point, for all n>1 the terms in the rest of the series have no way of ever affecting the leading term of 1 since the rest are divisible by p and by the ultrametric inequality (which becomes equal to the max in the case when the two terms have different absolute values) $|1+p^n|_p = 1$
Comment: @StevenGrigsby I contend that they are very similar: after you plug in $x = p^n$ into the series you are evaluating the limit of each individual *term* (this is how you get $0$ in each term after the first) and then summing those **limits** to obtain $1$.  This is not necessarily equivalent to evaluating the sum for an arbitrary $p^n$ and then taking the limit as $n\to\infty$.


Is there a way to avoid the implementation of the full sort.Interface for slices of structs?

Question: If I have an array/slice of structs in Go and want to sort them using the sort package it seems to me that I need to implement the whole sort interface which contains 3 methods:

Len
Swap
Less

It seems that Len and Swap should always have the same implementation no matter the type of struct is in the array.
Is there a way to avoid having the implement Len and Swap every time or is this just a limitation from the lack of Generics in Go?
Answer: If you are implementing several different comparison operations on the same slice type, you can use embedding to avoid redefining Len and Swap each time. You can also use this technique to add parameters to the sort, for example to sort in reverse or not depending on some run-time value.
e.g.
package main

import (
    "sort"
)

type T struct {
    Foo int
    Bar int
}

// TVector is our basic vector type.
type TVector []T

func (v TVector) Len() int {
    return len(v)
}

func (v TVector) Swap(i, j int) {
    v[i], v[j] = v[j], v[i]
}

// default comparison.
func (v TVector) Less(i, j int) bool {
    return v[i].Foo < v[j].Foo
}

// TVectorBarOrdered embeds TVector and overrides
// its Less method so that it is ordered by the Bar field.
type TVectorBarOrdered struct {
    TVector
}

func (v TVectorBarOrdered) Less(i, j int) bool {
    return v.TVector[i].Bar < v.TVector[j].Bar
}

// TVectorArbitraryOrdered sorts in normal or reversed
// order depending on the order of its Reversed field.
type TVectorArbitraryOrdered struct {
    Reversed bool
    TVector
}

func (v TVectorArbitraryOrdered) Less(i, j int) bool {
    if v.Reversed {
        i, j = j, i
    }
    return v.TVector[i].Foo < v.TVector[j].Foo
}

func main() {
    v := []T{{1, 3}, {0, 6}, {3, 2}, {8, 7}}
    sort.Sort(TVector(v))
    sort.Sort(TVectorBarOrdered{v})
    sort.Sort(TVectorArbitraryOrdered{true, v})
}

Comment: Very interesting. I had wondered about how to use multiple sorts on the same Slice type.
Answer: Your own answer is right. In your case of an array or slice the implementations of Len() and Swap() are simple. Like len() Go could provide a native swap() here. But the interface which is used now can also be used for more complex data structures like BTrees. It still allows the Sort() function to work (like my parallel quicksort, which uses the same sort interface).
Answer: If you want to sort slices (for which Len and Swap always have the same implementation), the sort package now has a function that only requires an implementation of Less:
func Slice(slice interface{}, less func(i, j int) bool)
Answer: Although this is an old question, I'd like to point out
the github.com/bradfitz/slice
package.
But as an example or proof of concept only, I would not recommend this actually be used (it's documented with the word "gross"):

It uses gross, low-level operations to make it easy to sort arbitrary slices with only a less function, without defining a new type with Len and Swap operations.

In actual code, I find it completely trivial, quick, short, readable, and non-distracting to just do something like:
type points []point

func (p []points) Len() int      { return len(p) }
func (p []points) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p []points) Less(i, j int) bool {
        // custom, often multi-line, comparison code here
}

Here gofmt insists on a blank line between the type and func lines
but it has no problem with
multiple one-line functions with no blank lines
and it nicely lines up the function bodies.
I find this a nice readable compact form for such things.
As for your comment that:

It seems that Len and Swap should always have the same implementation no matter the type of struct is in the [slice]

just the other week I need a sort that kept pairs of elements in a slice together (for input to strings.NewReplacer) that required a trivial variation like:
type pairByLen []string

func (p pairByLen) Len() int           { return len(p) / 2 }
func (p pairByLen) Less(i, j int) bool { return len(p[i*2]) > len(p[j*2]) }
func (p pairByLen) Swap(i, j int) {
        p[i*2], p[j*2] = p[j*2], p[i*2]
        p[i*2+1], p[j*2+1] = p[j*2+1], p[i*2+1]
}

This is not supported by an interface like the one in github.com/bradfitz/slice.
Again, I find this layout easy, compact, and readable.
Although (perhaps more so in this case), others may disagree.


tooltip for select in html

Question: I have a list of car that can be selected from dropdown as
<select>
  <option value="volvo">Volvo</option>
  <option value="saab">Saab</option>
  <option value="opel">Opel</option>
  <option value="audi">Audi</option>
</select>

this code gives me code of dropdown, I need a hover message as "select your car" on hovering on the box. There is a option of select that gives  message for each option, however my need is to display tooltip only when hovering the select box.
Answer: The easiest way is to add a title attribute:
<select title="Select your car">
    <option value="volvo">Volvo</option>
    <option value="saab">Saab</option>
    <option value="opel">Opel</option>
    <option value="audi">Audi</option>
</select>

It doesn't look that great though.  If you want a fancier tooltip you'd have to go the JavaScript route.

tooltip for select in html


Which key Harmonica to use in conjunction with Distorted Electric Guitar in C#/D half step down

Question: I'm in a melodic metalcore band, and we're looking to include some harmonica riffs/solos in some songs. We're currently writing in C# or D half step down and would like some help in this subject. We're thinking C# or G#, would either of these work?
Thanks!
Answer: For a song in C#, the more melodic parts need a C# harp. To get a bluesy sound/feel, you'll need an F# harp, which will give a blues sound when sucked, or drawn, producing C#7 and C#9.
Comment: Okay, I was a little hesitant to try an F# because I thought it was uncomfortably high. Thoughts? Also, I may go with C#, but does that rule out any other lower harps, like G#?
Comment: How do you mean - uncomfortably high? The more important factor is will it play the right notes.
Comment: Oh, I just feel that it's very high pitched, I'm not sure what it'll sound like with C# distorted guitar. I see what you're saying though. I think I'll get a cheapo F# if I can find one and test the waters. Thanks you for your expertise!


mixed sorting in extjs grid

Question: I want to sort a column based on text and numbers.
values i have are sorted like this:
<code>- List:010
 - List:100
 - List:134
 - List:2
 - List:204
</code>
but what i need is
<code>- List:2
 - List:010
 - List:100
 - List:134
 - List:204
</code>
the column type i am using is string. i tried to use sortType but that was not working.
<code>{
            name: 'listItems',
            type: 'string',
            mapping: 'listItems'
        }
</code>
Answer: If you are using ExtJS 4.x, you can take advantage of the SortTypes class.
<code>Ext.apply(Ext.data.SortTypes, {
    asPerson: function(person){
        // expects an object with a first and last name property
        return person.lastName.toUpperCase() + person.firstName.toLowerCase();
    }
});

Ext.define('Employee', {
    extend: 'Ext.data.Model',
    fields: [{
        name: 'person',
        sortType: 'asPerson'
    }]
});
</code>
More information on the SortTypes doc page.

mixed sorting in extjs grid