How to load more than on CSV file in highcharts?

Question: I'm trying to make a plot in highcharts, incorporating variables from 2 CSV files. I made an attempt based on these examples: load multiple csv files (correct order) in one chart and load data in highcharts from 2 different csv files but I can't make it work, nothing appears in the webpage, it's just blank. If you see the error in chrome, throws "Uncaught SyntaxError: Unexpected end of input".
I would really apreciate any help you could give me. Here's the code and two sample files I was using:
    <!DOCTYPE HTML>

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>2 csv_v3</title>

test1.csv
Date, Puren
18-07-16 0:00,152
18-07-16 2:00,175
18-07-16 4:00,188
18-07-16 6:00,209
18-07-16 8:00,241
18-07-16 10:00,254
18-07-16 12:00,262
18-07-16 14:00,267
18-07-16 16:00,290
18-07-16 18:00,324
18-07-16 20:00,341
18-07-16 22:00,343
19-07-16 0:00,344
19-07-16 2:00,346
19-07-16 4:00,348
19-07-16 6:00,353
19-07-16 8:00,362
19-07-16 10:00,374
19-07-16 12:00,375
19-07-16 14:00,373
19-07-16 16:00,376
19-07-16 18:00,379
19-07-16 20:00,376
19-07-16 22:00,367
21-07-16 0:00,347
21-07-16 2:00,322
21-07-16 4:00,306
21-07-16 6:00,278
21-07-16 8:00,237
21-07-16 10:00,216
21-07-16 12:00,204
21-07-16 14:00,198
21-07-16 16:00,174
21-07-16 18:00,140
21-07-16 20:00,123
21-07-16 22:00,120

test2.csv
Date, Ferroviario
19-07-16 0:00,172
19-07-16 2:00,171
19-07-16 4:00,169
19-07-16 6:00,169
19-07-16 8:00,169
19-07-16 10:00,165
19-07-16 12:00,154
19-07-16 14:00,121
19-07-16 16:00,82
19-07-16 18:00,67
19-07-16 20:00,63
19-07-16 22:00,59
21-07-16 0:00,56
21-07-16 2:00,53
21-07-16 4:00,50
21-07-16 6:00,44
21-07-16 8:00,32
21-07-16 10:00,24
21-07-16 12:00,21
21-07-16 14:00,20
21-07-16 16:00,17
21-07-16 18:00,14
21-07-16 20:00,13
21-07-16 22:00,12
22-07-16 0:00,11
22-07-16 2:00,11
22-07-16 4:00,11
22-07-16 6:00,9
22-07-16 8:00,9
22-07-16 10:00,9
22-07-16 12:00,9
22-07-16 14:00,9
22-07-16 16:00,9
22-07-16 18:00,9
22-07-16 20:00,9
22-07-16 22:00,9

Answer: Best way would be crating your own CSV parser. Example below:
Array.prototype.concatAll = function() {
  var results = [];
  this.forEach(function(subArray) {
    results.push.apply(results, subArray);
  });

  return results;
};

function csvToData(csv) {
  return csv.split('\n')
    .splice(1)
    .map((line) => line
      .split(',')
      .map((val, i) => {
        if (i === 0) {
          const a = val
            .split(' ')
            .map((s, i) => i === 0 ? s.split('-') : s.split(':')).concatAll().map(s => Number(s))
          return Date.UTC(2000 + a[2], a[1], a[0], a[3], a[4])
        } else {
          return Number(val)
        }
      })
    )
}

const test1_csv = `Date, Puren
18-07-16 0:00,152
18-07-16 2:00,175
18-07-16 4:00,188
18-07-16 6:00,209
18-07-16 8:00,241
18-07-16 10:00,254
18-07-16 12:00,262
18-07-16 14:00,267
18-07-16 16:00,290
18-07-16 18:00,324
18-07-16 20:00,341
18-07-16 22:00,343
19-07-16 0:00,344
19-07-16 2:00,346
19-07-16 4:00,348
19-07-16 6:00,353
19-07-16 8:00,362
19-07-16 10:00,374
19-07-16 12:00,375
19-07-16 14:00,373
19-07-16 16:00,376
19-07-16 18:00,379
19-07-16 20:00,376
19-07-16 22:00,367
21-07-16 0:00,347
21-07-16 2:00,322
21-07-16 4:00,306
21-07-16 6:00,278
21-07-16 8:00,237
21-07-16 10:00,216
21-07-16 12:00,204
21-07-16 14:00,198
21-07-16 16:00,174
21-07-16 18:00,140
21-07-16 20:00,123
21-07-16 22:00,120`
const test2_csv = `Date, Ferroviario
19-07-16 0:00,172
19-07-16 2:00,171
19-07-16 4:00,169
19-07-16 6:00,169
19-07-16 8:00,169
19-07-16 10:00,165
19-07-16 12:00,154
19-07-16 14:00,121
19-07-16 16:00,82
19-07-16 18:00,67
19-07-16 20:00,63
19-07-16 22:00,59
21-07-16 0:00,56
21-07-16 2:00,53
21-07-16 4:00,50
21-07-16 6:00,44
21-07-16 8:00,32
21-07-16 10:00,24
21-07-16 12:00,21
21-07-16 14:00,20
21-07-16 16:00,17
21-07-16 18:00,14
21-07-16 20:00,13
21-07-16 22:00,12
22-07-16 0:00,11
22-07-16 2:00,11
22-07-16 4:00,11
22-07-16 6:00,9
22-07-16 8:00,9
22-07-16 10:00,9
22-07-16 12:00,9
22-07-16 14:00,9
22-07-16 16:00,9
22-07-16 18:00,9
22-07-16 20:00,9
22-07-16 22:00,9`
const data1 = csvToData(test1_csv)
const data2 = csvToData(test2_csv)
const mergedData = [...data1, ...data2]
const options = {
  xAxis: {
    type: 'datetime'
  },
  series: [{
    data: mergedData,
    type: 'column'
  }]
}
const chart = Highcharts.chart('container', options);

console.log(data1, data2, mergedData)

Live example:
https://jsfiddle.net/a9zocb40/
[EDIT] FOR LIVE DATA
const options = {
  chart: {
    events: {
      load: function() {
        const chart = this
        const data1 = csvToData(test1_csv)
        const data2 = csvToData(test2_csv)
        const mergedData = [...data1, ...data2].sort((a, b) => a[0] - b[0])
        console.log(data1, data2, mergedData)
        chart.series[0].setData(mergedData)
      }
    }
  },
  xAxis: {
    type: 'datetime'
  },
  series: [{
    data: [],
    type: 'column'
  }]
}

const chart = Highcharts.chart('container', options);

Live example:
https://jsfiddle.net/g8ocqwwr/
Comment: thanks for the help, but I need it to read the csv file without putting the series values inside the script because eventually it's going to be automatized and the csv's are going to be constantly changing


Does zero sample correlation between regressor and residual mean Correlation between regressor and outcome = 0?

Question: In my econometrics professors' slides he states that:

"We said that $\sum_{i=1}^nX_i\hat{u}_i=0$ (where $\hat{u}$ are the residuals) implies zero sample correlation between X and Y".

My problem is in understanding WHY this is (the statement in bold), or if he made a mistake and actually meant that each regressor always has zero correlation with residuals estimator, due to first order conditions used to grant OLS (even in a multivariate regression, not necessarily single variable regression).

IMPORTANT EDIT: X is the regressor in a single variable linear regression and Y is the outcome of such regression
Answer: That is clearly wrong as stated. As you say, orthogonality between regressors and residuals is true for any regression estimated by OLS. At the same time, the OLS estimator in a bivariate regression is known to be
$$
\hat\beta_1=\frac{\widehat{cov}(x,y)}{\widehat{var}(x)}
$$
Hence, if the statement were true, we would have a proof that, always, $\hat\beta_1=0$...
So I suppose that is a typo indeed.
Comment: Thank you very much!


PHP Simple HTML DOM Parser funciona sem referência de id ou classe?

Question: estou querendo utilizar o PHP Simple HTML DOM Parser para retirar certos links de uma página gerada no frontpage o problema que encontrei é o seguinte como o frontpage não gera nenhum elemento html com id ou class eu gostaria de saber se é possível indicar um caminho para que isso possa ser feito

a página na qual quero retirar os links não é minha, estou apenas extraindo dados dela
Answer: Sim. Você deve usar uma propriedade chamada XPath, que indica o caminho de um elemento no DOM (document object model).
Para pegar o XPath de um elemento, basta acessar a aba Elements do Developer Tools (no caso do Google Chrome), inspecionar o elemento em questão e depois copiar seu XPath:

No caso do link acima, por exemplo, o XPath dele é:
//*[@id="supertopo"]/div[1]/div[1]/ul/li[1]/a

Espero ter ajudado. :)
Comment: o meu XPath retornado foi o seguinte

´/html/body/table/tbody/tr[1]/td[2]/table/tbody/tr[2]/td/table/tbody/tr/td[2]/table/tbody/tr[3]/td/table/tbody/tr[1]/td/div/p[1]/font/strong´

mas retorna

´ Notice: Array to string conversion in C:\xampp\htdocs\noticias\titulo.php on line 5
Array ´
Comment: É porque o método provavelmente retorna um array de nós, mesmo que tenha apenas um nó. Você deve iterar por esse array.
Comment: desculpe mas não entendi! tem algum exemplo?
Comment: Qual a linha do código que te retorna "Array to string conversion"?
Comment: `include('simple_html_dom.php');
$content = file_get_html('http://www.abresi.com.br/noticia_700.htm');
$elems = $content->find("/html/body/table/tbody/tr[1]/td[2]/table/tbody/tr[2]/td/table/tbody/tr/td[2]/table/tbody/tr[3]/td/table/tbody/tr[1]/td/div/p[1]/font/strong");
echo $elems;` na linha 5
Comment: @GuilhermeHenrique veja a documentação dele: http://simplehtmldom.sourceforge.net/. O método `find` retorna um array, então você precisa percorrer os elementos ou ir direto no índice do elemento com `find("/html/body/table/tbody/tr[1]/td[2]/table/tbody/tr[2]/td/table/tbody/tr/td[2]/table/tbody/tr[3]/td/table/tbody/tr[1]/td/div/p[1]/font/strong", 0)`. Veja se dá certo.
Comment: Vamos [continuar esta discussão no bate-papo](http://chat.stackexchange.com/rooms/20271/discussion-between-guilherme-henrique-and-rodrigo-rigotti).


Setting the font for the languages other than english

Question: I am doing a Font application for the Mac. I got all the font names from the location "/Library/Fonts/" and saved them in an array named FontArray. I just have to preview all the fonts in a NSTextView. This is my code for the NSTextView:
NSTextView *textView=[[NSTextView alloc]initWithFrame:NSMakeRect(250,500,450,30)];
[textView setString:@"ABCDEFGHIJKLMMOPQRSTUVWXYZ"];
[textView setFont:[NSFont fontWithName:[FontArray objectAtindex:rowid] size:40]];

This function will be called when a tableview row has been clicked. This works correctly for all the english fonts. But whenever non english fonts clicked means it gives an error like            *** -[NSTextView setFont:]: nil NSFont given. How can I change my code to correct this error.
Comment: anyone please give the suggestion...
Answer: You're on a completely wrong road. A font is not a font file; the name of the font contained inside a font file is not necessarily the same as the name of the file, as you've discovered. Instead of trying to work with the font files directly, use NSFontManager.
Comment: May I know how to get the Font name from the font file?


Dynamic vs static array in c

Question: The following code creates an array with the malloc function. But i know that this can be done much simplier with just int array[size]. I think this is static array. But with the malloc function is it dynamic array? I found this code in the net... What is really happening and whats the difference between a static array and a dynamic array(and static memory between heap memory). Can you change the size of the dynamic array during runtime? or... what i dont know exactly... If someone could explain i would appreciate :)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
    int size;
    int i;
    printf("Choose size of array: ");
    scanf("%d",&size);

    /*---MALLOC/FREE---*/
    int *m_array = (int *)malloc((size+1)*sizeof(int));
    memset(m_array,'\0',size+1);

    for(i=0; i<size ; i++)
    {
        m_array[i]=i;
        printf("%d ",m_array[i]);
    }
    printf("\n");
    free(m_array);

    return 0;
}

Comment: `int *m_array` is a `Pointer` not an `Array`.
Comment: `int array[size]` is a [variable length array](https://en.wikipedia.org/wiki/Variable-length_array). Also, you may be interested in the `realloc` function.
Comment: `The following code creates an array with the malloc function` No is not. With `malloc` you request an Amount of memory from your `OS` and if you get it you set your `pointer` to points there.
Comment: @Michi the C Standard says that `malloc` creates an array
Comment: @Michi i ment using the malloc...
Comment: @M.M no, the C standard does not say that `malloc` creates an array
Answer: There are several flavors of arrays, depending on how and where they are declared.
Fixed-length Arrays
Fixed-length arrays must have their size determined at compile time.  You cannot change the size of a fixed-length array after it has been defined.
Fixed-length arrays are declared in one of the following ways:
T a[N];
T a[N] = { /* initializer list */ };
char_type a[N] = "string literal";
T a[]  = { /* initializer list */ };
char_type a[]  = "string literal";

In the first three cases, N must be a constant expression whose value must be known at compile time.  In the first three cases, the size of the array is taken from N; in the last two cases, it's taken from the number of elements in the initializer list or the size of the string literal.
The initial contents of a fixed-length array depend on its storage duration and whether an initializer has been supplied.
If the array has static storage duration (meaning it was declared at file scope outside of any function body, or was declared with the static keyword) and no initializer is present, then all of the array elements are initialized to 0 (for scalars) or NULL (for pointers).  If T is an aggregate type such as a struct or an array type, then each member of the aggregate is initialized with a 0 or NULL.  union types are similarly zeroed out.
If the array has auto storage duration (meaning it was declared within a function or block without the static keyword) and no initializer is present, then the contents of the array are indeterminate - basically, garbage.
If the array is declared with an initializer list (regardless of storage duration), then the initial values of the array elements correspond to the initializer.  If there are fewer elements in the initializer than the array (for example, N is 10 but you only initialize the first 5 elements), then the remaining elements are initialized as though the array had static storage duration.  IOW, given the declaration
int a[10] = {0, 1, 2};

then the initial contents of the array are {0, 1, 2, 0, 0, 0, 0, 0, 0, 0}.
Fixed-length arrays containing string values may be initialized using a string literal.  C allows for "wide" character strings, so char_type may be either char or wchar_t.  The rules are the same for regular initializer lists, except that N (if specified) must be at least 1 more than the length of the string to account for the string terminator.
This means that
char a[10] = "test";

will be initialized as {'t', 'e', 's', 't', 0, 0, 0, 0, 0, 0} and
char a[] = "test";

will be initialized as {'t', 'e', 's', 't', 0}.
Arrays with static storage duration are stored such that they are available as soon as the program is loaded, and aren't released until the program exits.  This usually means that they're stored in a memory segment like .data or .bss (or the equivalent for whatever executable format your system uses).
Arrays with auto storage duration are stored such that they are allocated at block or function entry and released at block or function exit (in practice, they'll probably be allocated at function entry, regardless of whether they're limited to a smaller scope within the function) - this typically translates to the stack, although it doesn't have to.
Variable-length Arrays
Variable-length arrays were added in C99 - they behave mostly like fixed-length arrays, except that their size is established at run time; N does not have to be a compile-time constant expression:
int n;
printf( "gimme the array size: ");
scanf( "%d", &n );
T a[n]; // for any type T

Contrary to what their name implies, you cannot change the size of a variable-length array after it has been defined.  "Variable-length" simply means that the size isn't fixed at compile time, and can change from definition to definition.
Since their size isn't set until runtime, variable-length arrays may not be declared at file scope or with the static keyword, nor can they be declared with an initializer list.  Exactly how the space for VLAs is managed is up to the implementation; it may be (and usually is) taken from the stack, but AFAIK may be taken from somewhere else.
Dynamic Arrays
Dynamic arrays are not really "arrays" as such, at least in terms of the data types of the objects we use to manage them.  Dynamic arrays are allocated at runtime using one of malloc, calloc, or realloc, and that storage is held until released with a call to free.
T *p = malloc( sizeof *p * N ); // where N may be either a compile-time or
                                // run-time expression
...
free( p );

A dynamic array may be resized using the realloc library function, like so:
/**
 * If realloc fails, it will return NULL and leave the original array in
 * place.  We assign the result to a temporary variable so we don't risk
 * losing our only reference to that memory.
 */
T *tmp = realloc( p, sizeof *p * new_size );
if ( tmp )
  p = tmp;

While the memory for the array elements themselves is taken from the heap (or whatever dynamic memory pool), the memory for the pointer variable p will be allocated from either a .bss or .data segment or from the stack, based on p's storage duration (static or auto).
Memory allocated with malloc or realloc is not initialized; the contents of that memory will be indeterminate.  Memory allocated with calloc will be initialized with zeros.
Arrays vs. Pointers
At some point, somebody is going to tell you that "an array is just a pointer".  That person is not correct.
When you declare an array (either fixed- or variable-length), enough storage is set aside for the elements of that array and nothing else; no storage is set aside for any metadata such as the array length or a pointer to the first element.  Given the declaration
T a[N];

then the storage will look something like this:
    +---+
 a: |   | a[0]
    +---+
    |   | a[1]
    +---+
    |   | a[2]
    +---+
     ...
    +---+
    |   | a[N-1]
    +---+

There is no object a apart from the array elements themselves (or, more properly, the object a is the elements of the array), and the expression a may not be the target of an assignment.
But...
The expression a[i] is defined as *(a + i); that is, given a pointer value a, offset i elements (not bytes!) from that address and dereference the result.  But if a is not a pointer, how can that work?
Like this - except when it is the operand of the sizeof or unary & operators, or is a string literal used as an array initializer in a declaration, an expression of type "N-element array of T" will be converted ("decay") to an expression of type "pointer to T", and the value of the expression will be the address of the first element of the array.
This has several implications:

The expressions a, &a, and &a[0] will all yield the same value (the address of the first element of the array), but the types of the expressions will be different (T *, T (*)[N], and T *, respectively);
The subscript operator [] works equally well with both array expressions and pointer expressions (indeed, it's defined to work on pointer expressions);
When you pass an array expression to a function, what you are actually passing is a pointer value, not the entire array;

For dynamic arrays, the situation is different.  Given the line
T *p = malloc( sizeof *p * N );

then your storage will look something like this:
   +---+
p: |   | ---+
   +---+    |
    ...     |
     +------+
     |
     V
   +---+
   |   | p[0]
   +---+
   |   | p[1]
   +---+
    ...
   +---+
   |   | p[N-1]
   +---+

In this case, p is a separate object from the array.  Thus, &p won't give you the same value as p and &p[0], and its type will be T ** as opposed to T (*)[N].  Also, since p is just a pointer variable, you can assign a new value to it (although if you do so without freeing the memory it points to first, you'll create a memory leak).
Similarly, sizeof p won't behave like sizeof a; it will simply return the size of the pointer variable, not the size of the allocated memory that the pointer points to.
Comment: Amazing answer! But one nitpick about the statement *"except that N (if specified) must be at least 1 more than the length of the string to account for the string terminator"*. That's not technically correct. If N is specified, it must be greater than **or equal** to the length of the string. If N is equal to the length of the string, the result is an array of characters without a NUL terminator. For example, `a[4] = "test";` is allowed by the C specification, see paragraph 6.7.9/14
Answer: Static arrays are allocated memory at compile time and the memory is allocated on the stack. Whereas, the dynamic arrays are allocated memory at the runtime and the memory is allocated from heap.
This is static integer array i.e. fixed memory assigned before runtime
int arr[] = { 1, 3, 4 };

This is dynamic integer array i.e memory assigned on runtime
int* arr = new int[3];

Comment: Int *arr is not an Array, it's a Pointer
Comment: I know. Dynamic memory is allocated in pointer form. It has name of arr, it is not an array
Comment: The memory doesn't got allocated in Pointer. The pointer points there.
Comment: I am saying the same thing mate. Dynamic memory is allocated in pointer form and pointer points on that location where the data is, either it is in distributed or sequential form.
Comment: static arrays do not use the stack... that is automatic arrays.
Comment: @Muhammad Noman: `new` is from C++ but question is for C.
Answer: You're using dynamic arrays of size+1 and adding elements to it (from 0 to size) and releasing the space at the end before returning 0. So when in your case, int *m_array is a pointer to an int. What you're doing on line 13 is declaration:
(int *m_array) =...

and allocation:
...(int *)malloc((size+1)*sizeof(int));

Since it is a dynamic array, you are allocating on heap and is kept until it is freed (that's why you have free(m_array) at the end). If it was static, you could've initialized the array this way:
int m_array[size];

And it would've been allocated on static storage area (unless is automatic) and would be deallocated as soon as the program ended. You can't change the size of an static array in C, so you need to use a dynamic array. You use realloc when you want to change to size of a dynamic array.

Dynamic vs static array in c


Exception while calling WPF-GUI-Changes with BackgroundWorker

Question: I want to set a Control on my GUI transparent through manipulating opacity value.
Therefore, I initialize a BackgroundWorker on the constructor of my WPF-Form like :
volumeWorker = new BackgroundWorker();
volumeWorker.DoWork += new DoWorkEventHandler(VolumeBarDoWork);
volumeWorker.ProgressChanged += new ProgressChangedEventHandler(VolumeBarChanged);
volumeWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(VolumeBarCompleted);
volumeWorker.WorkerReportsProgress = true;

After that, my GUI waits for an event, which is triggered by another class
soundController = new SoundController();
soundController.VolumeChanged +=new SoundController.VolumeChangedEventHandler(VolumeChanged);

On the event handler i finally will start the volumeWorker (BGW) to manipulate the GUI:
private void VolumeChanged(float pVolume)
{
    Logger.InfoWrite("Event raised ## New volume : {0}", pVolume);
    volumeWorker.RunWorkerAsync();
}

But on BackgroundWorkers ChangeEvent, i got a InvalidOperationException every time.Did anyone got a idea?
private void VolumeBarDoWork(object sender, DoWorkEventArgs e)
{
    BackgroundWorker worker = sender as BackgroundWorker;
    Logger.InfoWrite("Worker");
    for(int _opacity = 99 ; _opacity >= 0 ; _opacity--)
    {
        Logger.InfoWrite("Start on {0}", _opacity);
        worker.ReportProgress(_opacity);
        System.Threading.Thread.Sleep(20)
    }
    e.Result = e.Argument;
}

private void VolumeBarChanged(object sender, ProgressChangedEventArgs e)
{
    this.volumeBar.Opacity -= 0.01f;
    Logger.InfoWrite("Changed opacity to : {0} on {1}", this.volumeBar.Opacity, e.ProgressPercentage);
}

private void VolumeBarCompleted(object sender, RunWorkerCompletedEventArgs e)
{
    Logger.InfoWrite("Completed");
    this.volumeBar.Opacity = 0.0f;
}

Answer: You cannot change UI directly outside the UI thread, since the UI controls are created on the UI thread. Use the Dispatcher.Invoke or BeginInvoke method instead.


Como funcionam os números de serie em um sistema?

Question: Sabe quando você faz uma compra de um jogo online e recebe a CD-KEY, e então abre um aplicativo e registra essa CD-KEY pra você? Ou então você comprou seu novo Sistema Operacional e vai registrar e insere uma key e mesmo offline ela é aceita? Como elas são feitas afinal?
São apenas um banco de dados enorme contendo todas as keys válidas e para quem foram atribuídas, ou existe uma lógica por trás disso?
Acredito que não seja uma pergunta ampla, pois deve haver algum algoritmo ou padrão utilizado que responda a essa pergunta.
Comment: A propósito, eu acho a pergunta interessante, só não sei se é "respondível". Você diz que deve haver algum algoritmo ou padrão, mas dificilmente esse padrão seria **público** - pois do contrário, ia facilitar muito o trabalho dos "piratas" rsrs.
Comment: Em inglês: http://stackoverflow.com/questions/3002067/how-are-software-license-keys-generated
Comment: Eu não posso dizer com certeza o que é usado na prática, já que são sistemas proprietários e eu não conheço seu funcionamento. Mas posso afirmar com segurança que é baseado em criptografia. O esquema pode ser desde simples - o software contém uma chave pública, e o CD-KEY é uma chave privada ou algo assinado com essa chave - até algo bem mais complexo. Já estudei por exemplo uma técnica que permite "desabilitar" um dispositivo crackeado simplesmente gerando um código que funciona em todos os outros menos naquele. Também posso inferir algumas coisas baseado na experiência pessoal com o Windows.
Answer: Não há padrão. Cada empresa faz de um jeito, com maior ou menor nível de sofisticação/qualidade. Raramente o algoritmo é publicado, uma vez que o desconhecimento do mesmo é o principal fator que busca (mas raramente consegue) assegurar a proteção contra o uso indevido (ver "segurança por obscurantismo"). As técnicas empregadas evoluem com o tempo, ao mesmo tempo que as técnicas de "quebra" ou evasão* também o fazem.
Entretanto, pode-se observar alguns padrões nessas técnicas, dependendo do cenário em particular que elas se aplicam (software online ou offline, que "liga pra casa" ou que não liga).

*Nota: me refiro a "quebra" quando se descobre um meio de se gerar códigos válidos que não foram emitidos pelo produtor/distribuidor, sem alteração ("crack") no programa, e a "evasão" quando o programa é modificado para simplesmente não fazer mais a checagem da chave (de modo que funcione sem uma chave válida) ou para "enganá-lo" de qualquer outra forma, simulando um servidor de valiadação, bloquando o aplicativo no firewall, etc.

Offline, sem "call home"
Nesse cenário procura-se impedir que um usuário que não possua a chave - qualquer chave - execute o programa. É o cenário mais simples, porém mais limitado - pois basta que uma única chave seja divulgada para que qualquer um com uma cópia do programa possa usá-la.
A técnica geral é embutir no software um algoritmo verificador, que recebe um código e diz "válido" ou "inválido", e separadamente entregar ao cliente uma chave única que o validador aceitará. O formato tem de ser tal que um terceiro não possa facilmente criar códigos válidos a partir do nada.
Já se usou algoritmos bastante ineficazes em sistemas mais antigos (uma resposta a uma pergunta semelhante no SOen [indicada por rubStackOverflow nos comentários] cita um caso em que apenas 10 tentativas eram suficientes para adivinhar uma chave válida!), mas alguém que quisesse implementar isso hoje em dia poderia fazê-lo de forma inquebrável (mas não inevasível) através da criptografia de chave pública:

Crie um par de chaves, embutindo a chave pública na mídia de instalação;
Cada licença teria um número de série, e a CD-Key seria esse número de série assinado pela chave privada (que só o distribuidor tem);
Quando o usuário entrar com a chave, o sistema só tem que verificar se a assinatura é válida ou não.

Sem a chave privada, é impossível criar um keygen que gere (com alta probabilidade) chaves válidas, não importa quantas outras chaves o atacante conheça. De modo que a única maneira de copiar o software sem alterá-lo seria reutilizar uma chave já usada (e se o fornecedor tem meios de rastrear uma chave ao seu dono - via registro no ato da compra, por exemplo - isso aumenta a exposição de quem compartilha uma chave, e consequentemente o risco de ser pego).
Offline, "calling home"
Se a ativação do produto envolve a intervenção do produtor/distribuidor, isso permite uma proteção um pouco mais sofisticada. Pode-se por exemplo fazer com que o software só funcione na primeira máquina em que foi instalado, e que o código seja inútil em qualquer outra*. Além é claro de tentar autenticar quem usa o CD-Key, de modo a evitar que a mesma chave seja usada duas vezes por pessoas diferentes.

* disclaimer: ao menos no Brasil e nos EUA isso é ilegal (por razões diferentes), a menos que o fornecedor garanta ao consumidor o direito de reinstalar o sistema em outra máquina (após desinstalar da primeira, ou se a primeira estiver inutilizável), seja qual for o meio técnico usado pra tal. No Brasil é permitido até mesmo crackear um sistema legalmente para garantir seus direitos de consumidor, desde que esse crack não incorra simultaneamente numa violação dos direitos autorais do fornecedor.

Essa intervenção não precisa necessariamente envolver conexão com a internet, como já observado (recentemente validei minha cópia do Windows 7 por telefone). De novo, as técnicas variam em qualidade, mas segue um exemplo de como isso poderia ser feito:

De novo, chaves assimétricas são criadas, mas nesse caso dois pares são necessários, um pra assinar e outro pra cifrar;
Um CD-Key é criado tal como no caso anterior, assinado pelo fabricante;
O software, após instalado, coleta algumas informações sobre o sistema do usuário difíceis de serem forjadas (como seu endereço MAC) e pede o CD-Key pro usuário (verificando a assinatura);
Ele então cifra ("encripta") ambos os dados gerando um novo código, que precisa ser enviado de volta pro fabricante (seja via internet, telefone, ou qualquer outro meio);
O fabricante então decifra esses dados, cadastra-os no seu banco de dados (como dito, "um banco de dados enorme contendo todas as keys válidas e para quem foram atribuídas") - caso já não tenham sido usadas, é claro - efetivamente ligando (locking) aquele número serial àquele usuário e àquela máquina específica;
Um código de liberação específico praquele serial é criado então, e assinado pelo fabricante; esse código é então inserido no software (automaticamente ou manualmente) e a assinatura é verificada, liberando o acesso.

De novo o atacante não pode gerar um CD-Key do nada (pois não tem a chave privada de assinatura), mas nesse caso não dá nem pra reusar uma chave anterior - pois o código final, aquele retornado no final do processo, só é válido pro computador com aquele endereço MAC específico.
Online
Se o software exigir conexão com a internet, então nada disso é necessário - basta usar um valor aleatório e único como CD-Key (um UUID basta, desde que gerado de forma segura), guardá-lo no seu banco de dados a priori (necessário, do contrário não se poderia distinguir chaves válidas de inválidas) e na hora de validar checar se o código está no banco e associá-lo ao usuário logado (se isso já não tiver sido feito), rejeitando se ele for enviado por um usuário diferente. E é claro tomando as medidas cabíveis para evitar que múltiplas pessoas façam login com a mesma conta de usuário.


Standard Chord Naming Conventions

Question: I am developing some software for music production, I am now coding some classes for Chords, but I came to realize that I understood much less than I thought. What I am trying to do is to rip apart all the pieces that make up a chord, and this is what I currently came up with.
So chords are made up by 4 main parts: Chord Quality, Highest Degree, Seventh Modifier, and Secondary Chord Modifiers
Chord Quality contains the main information about the chord, whether it is:
M      -> Major          -> (1-3-5)
m      -> Minor          -> (1-3b-5)
aug    -> Augmented      -> (1-3-#5)
dim    -> Diminished     -> (1-b3-b5)
sus2   -> Suspended 2    -> (1-2-5)
sus4   -> Suspended 4    -> (1-4-5)
sus2/4 -> Suspended 2/4  -> (1-2-4-5)
??     -> Power Chord    -> (1-5)

If there is not a quality written on a chord it is implied to be Major.

The highest degree well, it's the highest degree of the chord wether it is:
5  -> (1-3-5)
6  -> (1-3-5*-6)
7  -> (1-3-5*-7)
9  -> (1-3-5*-7-9)
11 -> (1-3-5*-7-9-11)
13 -> (1-3-5*-7-9-11-13)

* -> The note can be omitted

The seventh modifier is the only modifier that is in some way standard a chord, this tells what the interval from the root to the seventh.
maj7 -> Major Seventh
min7 -> Minor Seventh

And the secondary chord modifiers are the ones that are used upon the previous characteristics. This is the list of modifiers I have seen:
Add(X)  -> This modifier adds X note to the chord
No(X)   -> This modifier omits X noto from the chord

I have some concerns with this approach, I am not sure that Suspended chords are a quality or a modifier, and also I don't know if a chord can only have one modifier, or can it have more?
Can someone lead me to a good post or book on this topic?
Comment: @ggcg I have heard the term chord quality to refer to major or minor. I think that is common.
Comment: I am not familiar with your definitions.  Is chord quality and highest degree used in music or did you make them up.  Most musicians I know would not consider a 7th a modifier.  But again, is this your definition?
Comment: Please note that chords are rarely played in the exact order They are spelled. A lot of time the lowest note heard is important and leads to chords like c/g or c over g which indicates that the c triad should be played with the lowest note being a g (the 5th) for example. Also known as the inversion that it's played in. Sometimes the player will choose what note is the highest so they can get the melody note up on top.
Comment: It is not clear if the OP wants to simply catalog chords or have the program make sensible chord voicing sequences that follow harmony theory.  In which case even the root is not most important.  Also, the notes he has listed as optional are not the only optional notes.
Comment: @b3ko yes that is chord voicing, but chord voicing is an additional step afterwards, you can a C chord with many different voicings, but the building notes of the voicing would be given by the chord made up by a quality and modifiers.
Comment: @ggcg Which are the other optional notes, where can I find a lecture about that?
Comment: @ChecheRomo, when playing a 13th chord the dom7 is essential but the 9th and 11th are not.  So you could get away with (1, 3, b7, 13) and most guitarists do.  Again, it is not clear what you intent and needs are.
Comment: You seem to be treating the voicing as a modification of a chord.  Maybe not totally wrong but in the context of true music theory the choice comes from the environment.  Are you planning to code the rules of music theory too?  Just curious.  Also, you will encounter enharmonic chords like CMaj6 and Amin7.  They are exactly the same notes and Amin7 (root inversion) is the 3rd inversion of CMaj6.  Something to look out for.
Comment: It seems that what I'm trying to do is quite complicated, since there is no such thing as "Standards". There are many things that are merely ambiguous when creating a chord, I want to create an algorithm that can take a set of notes, and find all the possible Chords without voicings, nor modifiers. So the user can then modify the ones he would like to change, also to change the voicings. But I can't make the right structure of a chord in my mind.
Comment: I recommend starting with triads. Major, minor, and diminished (diatonic triads). This will be much easier and you will learn a lot and then can extend it to add diatonic 7ths later. And then add other "flavors" later.
Answer: In order of building the chord symbol:
Letter and accidental for the root.
Chord (triad) types or qualities: major, minor, augmented, diminished. No symbol implies major.
Power chords are often shown with a 5 like A5 meaning just an A root and a fifth above, no third. I suppose this can be consider as applying at the same level as triadic type.
The next part is a little tricky. Basically, after triads there are seventh chords, then extended chords for ninths and higher. You called this "highest note." You might simply use the term "extension" for sevenths through thirteenths, extension being applied to simple triads.
About "seventh modifier", the default qualities of the 7 through 13 are those of a diatonic, dominant thirteenth chord. In other words, the default is to use only a number with the assumed qualities m7, M9, P11, M13. Use a sharp/flat modifier on the extension only if it differs from those assumed qualities.
Modifiers. This also is a bit tricky, because you have to account for the fact that modifiers overlap with the basic triad types. Example: Cmb5 is logical but unnecessary, because you can use the simple triad type Cdim.
Regarding omissions, I've seen 'omit' written on chord symbols, I think this isn't a well defined standard.

I am not sure that Suspended chords are a quality or a modifier

That's because the chord symbol system isn't totally logical and the sus isn't used consistently, often it isn't even used to label an actual suspension. Logically it should refer to a suspension of the tone below so a sus4 should be a suspension of the third. That's the triad level. Logically you might expect Cmin-sus4-7 but my convention the modifiers go last Cmin7sus4. Also, logically it's a modifier not a quality. Csus4 means C F G without the third you can't know the quality, you have to resolve the suspension to know the quality Csus4 Cm - in that example the ambiguous suspension resolved to a minor chord.

...also I don't know if a chord can only have one modifier, or can it have more?

You can have multiple modifiers, like G7#5sus4, there doesn't seem to be a convention about the order other than putting them last.
So, the object is something like:
<root><triadic-type><extension><modifiers>
Comment: Thanks for your reply, It makes much more sense in my head now, but I have one more question, which notes can be sharpened/flattened on a chord?
Comment: I think most of the sharp/flat modifications are covered here https://music.stackexchange.com/a/86961/23919
Answer: See https://www.scales-chords.com/chordid.php for a remarkably thorough and accurate application of chord naming conventions.
I think (naively) the problem is comparable to converting values between number systems. The cumbersome example that comes to mind would be developing a systematic conversion between hexadecimal and roman numerals. Fundamentally, the issue focuses on the identification of multiple stacked intervals using "natural" language. Chord naming is based on conventions developed from the 10th century on-wards. Then add to that the complexity of translation between languages. The example I have is the German usage of "moll", "dur", "klein", and "groß". The German term "großer Dur Septakkord" becomes in English "major–major seventh". The German is more descriptive while the English sounds inarticulate and redundant. An exercise left to the student: get the French, Italians, Czechs, Russians, Poles, Chinese, Japanese, Koreans, and Arabs to agree on a standard naming system.
Comment: It is very interesting the perspective you are giving to this problem, yes, of course I'll check on naming conventions on other languages, this is something I would have never thought of. Also thanks for the web page referenced.

Standard Chord Naming Conventions


Qual código HTTP eu devo usar quando não consigo autenticar em serviços de terceiros com login e senha providos pelo cliente anteriormente?

Question: Estou implementando um serviço de um site que faz o seguinte:

O meu site (A) conecta em alguns serviços externo providos por terceiros (B, C e D) usando webservices.
Na verdade A é uma API REST no backend que é acessado pelo javascript do frontend. O frontend não necessariamente é o meu, o cliente pode desenvolver o seu próprio frontend acessando a minha API REST se assim ele preferir.
Cada usuário do meu site A tem um login e senha no site B, um no site C e um no site D. Cada um desses logins e senhas dos sites B, C e D ficam salvos no meu banco de dados, por meio de uma funcionalidade oferecida ao usuário para este fim. Esses logins e senhas são diferentes entre si e diferentes do login e senha que uso para logar o cliente no site A.
Há algumas funcionalidades que o cliente acessa no meu site A que dependem da integração com os serviços B, C e D, e neste momento o login e a senha do respectivo serviço integrado é usado.
Por vezes, o login e senha enviados ao B, C ou D estão errados porque o meu usuário (do site A) os salvou incorretamente ao usar a funcionalidade que salva esses logins e senhas no banco de dados.

Pois bem, a minha dúvida é qual é o código de status HTTP que devolvo ao navegador do cliente quando o login e senha para a integração estão errados.

400 - Bad Request não parece ser bom porque isso é para requisições HTTP malformadas e a requisição recebida pelo meu site A é bem formada.
401 - Unauthorized não parece ser bom porque isso seria para uma falha de autenticação ou autorização no site A que é onde o navegador está se conectando. Ocorre que a falha de autenticação foi na integração com o site B, C ou D e não no site A. Além disso, em caso desse código ser aplicado, um cabeçalho WWW-Authenticate deveria estar presente para permitir o navegador a se (re)autenticar no meu site, o que não faz nenhum sentido no meu caso.
403 - Forbidden também não parece correto porque o cliente tem sim acesso a funcionalidade solicitada, o problema é apenas que ela não foi configurada adequadamente.
404 - Page Not Found também não é correto porque a página de fato existe e o cliente deve saber disso.
405 - Method Not Allowed também não é correto porque o método HTTP usado (GET, POST, PUT, etc) é o correto.
406 - Not Acceptable e 506 - Variant Also Negotiates também não parecem ser corretos porque isso tem a ver com o content negotiation que não tem relação com o problema ocorrido.
407 - Proxy Authentication Required também não parece ser correto porque isso diz para o cliente se autenticar em um proxy antes de tentar a requisição novamente com o site A. Ocorre que a requisição com o site A é válida e não precisa de proxy nenhum, o problema é entre A e B, C ou D.
409 - Conflict também não parece ser correto porque isso é para o caso de ocorrer um conflito entre duas ou mais requisições que alteram o estado do servidor.
412 - Precondition Failed e 417 Expectation Failed também não parece ser corretos porque isso tem a ver com o conteúdo dos cabeçalhos da requisição, que no meu caso não têm qualquer problema.
421 - Misdirected Request também não parece ser correto porque isso ocorre quando o servidor não é capaz de produzir uma resposta ao cliente, e não é esse o caso.
422 - Unprocessable Entity também não parece ser correto porque isso é para o caso de a requisição ter algum erro semântico. Ocorre que a requisição está correta afinal de contas.
424 - Failed Dependency é usado pelo WebDav no método PROPPATCH para especificar que a alteração de uma propriedade falhou porque isso desencadeou a alteração de uma outra propriedade e esta outra alteração falhou, tal como consta na RFC4918. Eu acho que usar isso para representar um erro de login e senha seria algo inadequado.
500 - Internal Server Error é para quando ocorre um erro imprevisto e não tratado no servidor, ou então para erros genéricos do lado do servidor. Não é esse o caso.
502 - Bad Gateway também não parece ser correto porque isso é para sinalizar uma resposta inválida do serviço integrado B, C ou D. Ocorre que a resposta de B, C ou D é válida, embora seja uma resposta de erro. Até faria sentido se o login e senha fossem algo privativos do site A com um desses outros sites, mas no caso eu estou usando um login e senha dados pelo cliente.
503 - Service Unavailable também não parece ser correto porque isso é para quando o servidor estiver congestionado ou em manutenção, o que não é o caso.
510 - Not Extended não parece ser correto porque após ler a RFC2774, isso daí tem a ver com extensões especificados em cabeçalhos da requisição. No meu caso não existe cabeçalho nenhum com qualquer propriedade especial relevante.
511 - Network Authentication Required também não parece ser correto porque isso é para quando ocorre um problema com a infraestrutura de rede do lado do cliente para ele obter acesso ao site A. No meu caso, se a requisição chegou ao site A, é porque ele já tem o acesso.
Quanto aos códigos
402 - Payment Required,
408 - Request Timeout,
410 - Gone,
411 - Length Required,
413 - Payload Too Large,
414 - Request-URI Too Long,
415 - Unsupported Media Type,
416 - Request Range Not Satisfiable,
418 - I'm a Teapot,
423 - Locked,
426 - Upgrade Required,
429 - Too Many Requests,
431 - Request Header Fields Too Large,
451 - Unavailable For Legal Reasons,
501 - Not Implemented,
504 - Gateway Timeout,
505 - HTTP Version Not Supported,
507 - Insufficient Storage e
508 - Loop Detected,
estes claramente não tem relação nenhuma com o meu problema.

Há alguns status não-oficiais usados por aí, mas nenhum deles parece se adequar também.
Assim sendo, nenhum dos códigos HTTP 4xx e 5xx oficiais que achei especificados em algum lugar parecem fazer sentido para o meu caso. Então, o que devo fazer? Que código HTTP devo utilizar e por quê? Se nenhum dos códigos de status oficiais forem adequados, seria melhor eu inventar um para isso ou forçar o uso de algum já existente? Se for forçar o uso de algum já existente, qual seria e por quê?
Além disso, noto que esse meu problema deve ser algo comum e recorrente, e portanto já deveria estar coberto em algum dos códigos existente.
Um outro ponto é que não tenho certeza se é problema no cliente (4xx) ou no servidor (5xx), uma vez que a requisição do cliente não pôde ser atendida devido a um problema na integração no lado do servidor (5xx), entretanto a razão de ocorrer essa falha é porque o cliente forneceu informações incorretas (4xx), mas essas informações não foram passadas pelo cliente no momento da requisição, e sim estavam armazenadas no lado do servidor (5xx).
Nota 1: Eu estou ciente das questões acerca de armazenar login e senha de clientes para serviços de terceiros ao invés de solicitá-los em cada requisição e das implicações em relação a segurança disso. Entretanto, o que estou pedindo tem pouca relação com isso.
Nota 2: Considerando-se os comentários e as duas respostas postadas, esclareço que não posso simplesmente retornar 200 indiscriminadamente para qualquer requisição para então impor ao cliente que interprete a resposta para saber se houve um erro ou não. Retornar um 400 para todo e qualquer erro e esperar o mesmo do cliente também está fora de cogitação. É um requisito importante que o código de status HTTP deva ser um código informativo.
Nota 3: Por enquanto, estou usando um código personalizado 432 para esta situação.
Comment: @VictorStafusa Talvez eu não tenha me feito claro. Se as credenciais do usuário estão erradas, e o serviço remoto não as aceitou, isso não é um erro do seu sistema - você precisa implementar um fluxo que corrija a configuração. Entretanto, *sua aplicação ainda está funcional*; vocêe precisará guiar o usuário por um outro fluxo do processo, mas isso não implica num erro: `200 OK` indica que sua aplicação ainda funciona. A não ser que ela seja apenas um *proxy*.
Comment: @VictorStafusa, o cliente acessa o seu Site A, então, sempre que estiver online a resposta do seu site A será 200. Se a resposta dos outros serviços for diferente, você indica em uma página a resposta de cada serviço. Se você "assumir" o erro do site B ou site C, é o seu site A que está com problema.
Comment: http://stackoverflow.com/questions/942951/rest-api-error-return-good-practices, mesmo tema, tem várias respostas e diagramas.
Comment: na minha opinião, HTTP Status refere-se ao seu website, e não dos serviços integrados. Se o site A está funcionando corretamente, a resposta deve ser 200, caso haja algum problema de integração por erro de digitação, o seu site A deve informar ao seu cliente que a integração não aconteceu por erro de digitação, e indicando que as configurações do perfil do cliente devem ser corrigidas. Se você retornar um erro 4xx ou 5xx, isso indicará um erro no seu site.
Comment: A resposta me parece simples - seu site é a referência de estado. Se funcional (integração não causa uma falha catastrófica) então você conseguiu gerenciar o estado: `200 OK`. Se a integração torna impossível a execução da ação, `500 Internal Server Error`. (Edit) O ponto do Ricardo é perfeito. Integrações são *parte da sua aplicação*.
Comment: @Ricardo Responder algo com um "200 Erro de autenticação" é meio estranho, não acha?
Comment: @OnoSendai Um `500 Internal Server Error` solicitando o cliente a configurar adequadamente o login e senha não é exatamente o que pode se considerar um erro interno do servidor.
Comment: Se a sua aplicação, entretanto, não possui um fluxo de tratamento para credenciais erradas e *falha catastroficamente* por causa disso, isso é um erro da aplicação (e `5xx` me parece válido.)
Comment: @OnoSendai Ok, editei a pergunta para clarificar isso. Na verdade a aplicação A tem duas partes, uma API REST no backend e o frontend. Se for só para a API conversar com o meu front, posso colocar o código que quiser e pronto, mas a minha dúvida é sobre o ponto de vista da API REST se for usada por outros aplicativos, qual seria o código mais adequado.
Comment: @Ricardo Veja o último comentário que fiz pro OnoSendai.
Comment: Eu sei que parece zoeira, mas eu sempre consulto os gatos HTTP https://http.cat/ para isso... A resposta real depende muito de como vai ser sua _implementação cliente padrão_, o modelo de consumidores do seu serviço.
Comment: Essa é uma pergunta bem heurística, seu caso em específico é um método não comummente utilizado. Pois bem, ao meu ver acredito que seja uma falha do servidor  5xx se não foi possível se comunicar com o os serviços de terceiro para autenticar. Porém, se seu serviço está funcionando e as credenciais fornecidas pelo usuário não estão sendo aceitas pelos serviços de terceiro, então é código 4xx na condição de que as credenciais não foram aceitas e por estarem incorretas ou foram fornecidas incorretamente.
Comment: @LeonanCarvalho Sim, no primeiro caso que você citou, um 502 resolve. Já o segundo caso é que é o problema que descrevo na pergunta. Também cheguei a conclusão que isso é um 4xx porque é algo em que o cliente é que tem que atuar para resolver e a "culpa" é dele. Atualmente eu coloquei um código customizado 432 para essa situação.
Comment: As APIs 3rdparty podem ser configuradas pelo usuário final ou não? Quero dizer, ele pode alterar senhas do A, B, C e D sendo assim o possível responsável por falhas de autenticação?
Comment: @GuilhermeNascimento Sim, o usuário é que configurava as suas senhas para os serviços A, B, C e D. Entretanto, para ser sincero, já não estou mais envolvido com o projeto em questão, mas mesmo assim, ainda estou curioso em saber qual seria a melhor abordagem.
Comment: Eu não vou fazer uma resposta porque é apenas opinião minha e fazia um tempo que queria comentar sua pergunta. Penso eu que se o usuário depois de conectado no "A" acessa o "B" e "C" e pelo painel "A" pode ajustar e as mensagem de notificações com erros são informadas como sendo dos 3rdparty, então seria 403. Mas se a alimentação é um tipo de sincronização e os ajustes tem que ser feito diretamente por paineis nos 3rdparty então seria 500, isso porque para o servidor "A" a situação é inesperada e o processo é indireto. Espero que eu tenha conseguido explicar meu ponto de vista :)
Answer: Como você faz múltiplas operações, algumas funcionam e outras não, você pode utilizar o multi status (207) e listar os status apropriados por operação.
Comment: Abordagem interessante. Não sei o que pensar sobre ela ainda. Vou refletir um pouco.
Answer: Eu considero que 403 (Forbidden) faz sentido nesse caso. Já cheguei a usar 400 (Bad request) em casos parecidos – é pouco preciso, mas ainda acho aceitável.
A justificativa para o uso da faixa 4xx é que se trata sim de um erro do cliente, apesar de detectado no servidor. Foi o cliente (o usuário, não o software que ele está usando) que informou credenciais incorretas.
Answer: Um usuário havia postado uma resposta, porém ele a apagou. Pessoalmente, não concordo com ela porque não acho correto retornar 200 em caso de erro, mas de qualquer forma, copio a resposta apagada dele aqui do jeito que estava quando foi apagada para que o conteúdo não seja perdido:

Depois de ler sobre o assunto:

http://blog.thefrontiergroup.com.au/2012/08/http-status-codes-and-restful-api-crafting
http://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/common-rest-api-error-codes
http://racksburg.com/choosing-an-http-status-code/

eu ficaria com a resposta encontrada no último:

There's a lot of smart people at Facebook and they built an API that only ever returns 200.
The basic argument against bothering with specific status codes is this: the existing status codes are much too general for a modern website/API

Traduzindo: Os códigos de status são muito vagos para as aplicações modernas.
A Microsoft apresenta 25 possíveis situações para o Código 400.
Answer: Eu iria de 500. Explico.
Pense que quem está integrando com sua API "A" não se importa - pelo que entendi da sua pergunta - com a autenticação para estas APIs internas. O fato é que o usuário da sua API tentou fazer uma requisição e não conseguiu completá-la.
Como uma 401 é mais esperada para a request atual e todos 4xx são Client Errors, eu trataria este problema como um erro imprevisto no servidor. Ou seja, um erro 500. Como está no link que enviou:

The server encountered an unexpected condition that prevented it from
  fulfilling the request.

Parece ser uma condição inesperada. Não há nada que o usuário possa fazer no rest service de "A" e o suporte (ou equipe de desenvolvimento) precisará atuar para resolver a situação. Para ajudar, adicione ao retorno do erro uma mensagem (ou código de erro) com a explicação: a autenticação a um sistema interno não foi possível de ser completada por um problema de usuário ou senha para a API B/C/D (se puder fornecer estes detalhes, claro).
Mesmo que não concorde com a minha resposta, tenha em mente que nem sempre você vai encontrar o código HTTP ideal para o seu problema. Um exemplo: qual código HTTP usar para validar se um campo de um cadastro é maior que o permitido? Verá que há uma boa discussão na Internet sobre isto.
O ideal é escolher o código HTTP que fizer mais sentido para você e estabelecer ele como padrão para o problema que está resolvendo.
Comment: Não concordo. O erro no caso é culpa do cliente, e não do servidor. E também não é um erro imprevisto. Pensando bem, o meu problema corresponde a uma classe de erros onde a requisição não obtém sucesso porque o cliente deve primeiramente realizar outras ações antes de tentar efetuar a requisição novamente (ações essas que não são simplesmente se autenticar no servidor). Assim sendo, tendo a pensar que um código 4xx seria adequado. Quanto à validação, sei que há um debate forte entre o 400, o 422 e outras alternativas.
Comment: @VictorStafusa, se o cliente pode tomar alguma ação, creio possa entender como ClientError mesmo (4xx). Preciso olhar deste ponto de vista o problema novamente, mas parece que o 401 seria a melhor escolha.
Answer: Realmente não parece ter um código que atenda essa situação, já que no seu caso a requisição será processada corretamente.
Acredito que código 200 seja o mais indicado, com uma mensagem de erro personalizada.
Comment: Código 200 é OK,  significa que tudo está certo, sinal verde, etc. A situação requer um código de falha de autenticação ou erro interno do servidor.


Confirmatory factor analysis for data reduction (prior to regression)

Question: I  have a data set from an opinion survey with many variables, and to conduct regression analysis I would like to reduce the number of variables; because curently I actually have more variables than responses. I've tried principal component analysis(PCA)/exploratory factor analysis(EFA), but I find it difficult to "interpret" the different factors. However, I think one can group the variables well based on theoretical considerations alone, so I thought confirmatory factor analysis (CFA) might be an option.
Unfortunately most resources on data reduction techniques point me to PCA/EFA. Does anybody know any resources that explain the process with CFA?
Alterantively, can anybody help me with the following more specific questions?
1) If my sole purpose of is to use the factor scores in regression analysis, do i have to worry about the fit statistics of the CFA at all? It seems to me that all I need are the factor scores.
2) As far as I understand, one can either constrain the variance or set set one of the variables in each of the groups to 1, again for my purpose of data reduction, does it matter which approach I use? Does the choice affect my factor scores? Or more specifically does setting one factor loading in each factor to 1 the comparison across factor scores?
3) I am struggling with the actual computational step going from the CFA results to the factor scores. I was using the Lavaan package in R, and can't seem to get the factor scores. or do i have to calculate the manually using the loadings? Does anybody know how this is done, or can reccomend a different package? (I realise this is a more a programming question than a conceptual qustion, but thought I'll add it nonetheless, hoping that somebody who might know about CFA in theory also has practical experience with it in R)
Thanks in advance!
NB: In case anybody is alaremd by the statement about the relatively sample size above. I don't really care so much about inference here, as my sample is pretty much the entire population.
NB2: I am planing to use the factors only on the right-had side of the regression. The left-hand side variable comes from elsewhere.
Comment: I would look into partial least squares for this type of problem. I wrote about [PLS](http://www.statisticalanalysisconsulting.com/partial-least-squares/) on my blog
Comment: Thanks, @PeterFlom! Good idea. I'll give that a try. Am I correct that PLS is more data-driven, i.e. more similar to PCA, than theory driven like CFA?
Comment: I'd still be curious to hear any thoughts about my original question, on whether/how CFA can be used for data reduction. Would anybody have any pointers?
Answer: It's my understanding that EFA and CFA are essentially very similar, but differ in whether a solid theoretical foundation or researcher's arguments for a set of factors that is related to studied phenomena. Being a beginner and planning to perform EFA, CFA and SEM analyses for my dissertation, I collected a lot of resources. Below I will share some of them, which I think are relevant to your question.

http://www.jvank.nl/PP/down/BeamerFASEM.pdf (The best presentation on EFA that I've seen so far! I just love the detective/judge analogy!) Unfortunately, CFA explanation in this document is not comprehensive. The following slides do a better job at that and complement the first set of slides: http://people.ucsc.edu/~zurbrigg/psy214b/09SEM6a.pdf;
http://courses.ttu.edu/isqs6348-westfall/images/6348/Exploratory_Factor_Analysis_in_R.pdf;
http://www.statpower.net/Content/312/R%20Stuff/Exploratory%20Factor%20Analysis%20with%20R.pdf;
http://jeromyanglim.blogspot.com/2009/10/factor-analysis-in-r.html;
Resources on James Gaskin's StatWiki: http://statwiki.kolobkreations.com/wiki/Exploratory_Factor_Analysis, http://statwiki.kolobkreations.com/wiki/Confirmatory_Factor_Analysis;
James Gaskin's YouTube channel: https://www.youtube.com/watch?v=X-O-OcJPCe8 (This is just for EFA, check other videos in his SEM Series there);
https://personality-project.org/r/#factoranal;
http://tgmstat.wordpress.com/2014/01/15/computing-and-visualizing-lda-in-r (Dimension reduction using PCA and (!) LDA);
Somewhat "unusual", top-down method, called Bass-Ackwards technique: http://projects.ori.org/lrg/PDFs_papers/Goldberg_2006_BassAckwards_JRP.pdf (description), http://blogs.scientificamerican.com/beautiful-minds/2013/11/25/openness-to-experience-and-creative-achievement (example of application);
On selection of estimators for CFA: http://redfame.com/journal/index.php/ijsss/article/viewFile/27/29;
http://www.r-bloggers.com/structural-equation-modeling-separating-the-general-from-the-specific-part-ii (CFA)
'qgraph' R package for PCA/EFA/CFA/SEM analysis and visualization: http://sachaepskamp.com/qgraph (details), http://www.jstatsoft.org/v48/i04/paper (more details);
'semPlot' R package for SEM visualization: http://sachaepskamp.com/semPlot
CFA and SEM can also be performed by using 'plspm' R package. Details on PLS-PM and some other PLS-based CFA methods are available in free book by plspm's author Gaston Sanchez: http://gastonsanchez.com/PLS_Path_Modeling_with_R.pdf;
A nice book, integrating various multivariate techniques into a coherent description. After reading "inside", I concluded that it should be good for a beginner like me. I went ahead and ordered it - looking forward to learning from this book: http://www.amazon.com/The-Essence-Multivariate-Thinking-Applications/dp/041587372X.

I might update this answer with other materials I will run across. Hope it helps!


Which Object detection model will give the best result on images when the speed is not a problem for Text Images

Question: I want to develop a model for cropping the equations from the Maths questions as people like me are struggling a lot for doing it manually for the research purpose. I want to know if we can do this? and if we can out of all the possible solutions out there for object recognition models, which one will produce the best results on Text images.
As there is tensorflow's object recognition API, RCNN, Fast RCNN, Faster RCNN, YOLO (v-1,2,3,4,5).
An if there is any other , please do suggest. What I want to do is to detect the gray areas of equations in this image.

Note: The grey region shown in the image is for just demonstrating. My actual images are simple cropped questions from books with with background and black letters (most of the books)
Comment: Are math segments always highlighted with gray background? If yes, a simple deterministic image processing routine will do the trick.
Comment: No no!!,! This is just to show what exactly I want to do
Answer: Note that there are two problems in this case: segmentation and classification. A neural net might be a solution for both steps in this case because you can easily generate zillions of labelled test images. Nevertheless, a classic approach should yield comparable results with much less efforts:

Use a simple page segmentation alorithm like runlength smearing or bounding box merging for segmenting the image into regions
Classify each region with an arbitrary classifier. You can use a NN on all normalized input pixels for this, but other classifiers like kNN should also work with gradient histograms as features (the gradients are computed on quasi grayscale images, which are generated from the onbit images by blurring). Gradient histograms were the state-of-the art features before the renaissance of neural nets.

Out of curiosity, I have tried out step one with the python library Gamera (gamera.sf.net) with the following code:
<code>from gamera.core import *
init_gamera()

img = load_image("MathExpressionInputExample.png")
img = img.to_onebit()

img.remove_border()
segments = img.runlength_smearing()

# now you could process each segment (e.g. saving it to a file)
for seg in segments:
    # do some stuff

# visualize the result
color_ccs = img.graph_color_ccs(segments)
color_ccs.save_PNG("segments.png")
</code>
The result looks reasonable to me(note that the colors only indicate the segmentation, with adjacent segments having different colors):
Comment: Could you please give an example of this approach? And how can I crop using your idea?
Comment: I do not understand. Waht do you mean by "crop"? And waht do you mean by "an example"? Example code? This will depend on the programming language and libraries at hand.
Comment: I have added sample code for doing the page segmentation (it heavily depends on the pythin library Gamera, of which I am the lead developer). Is this what you wer looking for?
Comment: I am looking to extract the portion of the code which can crop the equation from an image of question. For example if I put the program to a script, I want it to give me 2 cropped images consisting of both the equations seperately.
Comment: My code above already solves the segmentation problem (write each segment to a separate file). Then you must only find a classification method. You write in your question, you are experienced with neural nets, so this is an option: generate a million sample images, store in a label file the classes of segments, use the above code for segmentation and train the NN with the labelled data. You cannot expect everything to be done by someone else for you ;-)


LibGit2Sharp - Merging Branch

Question: I'll try to merge a branch to another but I've a MergeConflictException all time. So I try to stage all repository.Index.Conflicts but without any change.
string[] conflictArray = new string[10000];
try {
    repository.Merge(branch[selected], GitHelper.getSignature(), options);
} catch(MergeConflictException e) {
    if (repository.Index.Conflicts.Count() > 0) {

        foreach (var c in repository.Index.Conflicts) {
            repository.Index.Stage(c.Theirs.Path);
        }

    }
}

I' search a lot to have any help, I read MergeFixture.cs from LibGit2Sharp.Tests.
EDIT: sorry I don't tell you why i'm working on this but I'm student and my teachers gave to me this project to do with .NET 3.5 so I use latest 3.5 release of LibGit2Sharp.
Comment: What do you mean "without any change"? Are you committing after the merge resolution? Note, however, that if you get a merge conflict, you need to resolve it. If you stage all these files without resolving the conflicts you'll include the conflict markers in them. The right thing to do there is either give up or ask the user depending on what kind of program you're writing.
Comment: I show this before asking question : http://stackoverflow.com/questions/33812895/libgit2sharp-how-to-resolve-a-conflict

I tell 'without any change' because i've already the same exception each time. So can you help me how to resolve conflict ?
Comment: Why do you upload commented code? This is confusing.
Comment: I remove commented code, I forgot to remove them and i'm sorry for my english, I'm french.

I search to resolve conflict but I've to use Ancestor or not ?
Answer: MergeConflictException is admittedly a misleading name.  It doesn't mean that there was a merge conflict.  That's not exceptional, that's normal, and it doesn't raise an exception.  It means that there was a conflict writing the files during merge.  (But more on this in a second.)
In fact, you have files in your working directory that have been modified and were impacted by the merge.  You can perform a merge if you have files modified in your working directory if that file wasn't changed in the branch you're merging or in the current branch (relative to the common ancestor).
But what you almost certainly want to do is to only start a merge when you do not have any changes in your working directory.  Please stash or commit your changes before merging.  If you believe that you truly do not have any chanes in your working directory, then correct your line endings.
(Note that MergeConflictException doesn't exist anymore, it's been renamed to help avoid this confusion.)
Comment: I use the latest version of .Net 3.5 LibGit2Sharp because the project is based on the prerequisite given by my teachers. I have to interface and integrate its capabilities to Git .

I tried to list all the conflicts path but I noticed Ancestor and Theirs are similar but Ours is null. All of this conflicts happened with deleted files. It's normal ?
Comment: If you have a conflict where `Ours` is null then that was deleted in your branch and modified in the other.  Perfectly normal.

LibGit2Sharp - Merging Branch


How is it that 12 eighth notes fit in a measure labeled as common time?

Question: I am looking at Franz Liszt's Consolation S.172 No.3. I have checked out two different music scores of the same composition and they both have the same time signature (standard), however the base line includes 12 eighth notes (making it a 3/2 time signature?). My question is how is that possible? How am I supposed to compare the base line with the treble line when they seem to be on different signatures?
I am still relatively new to music theory and appreciate any constructive criticism as to my approach or my vocabulary.
Here is a snippet of what I am talking about. The first two measures of the composition:

Later on you will see this gets even more confusing when the treble is certainly divisible by 8 eighth-notes and the bass is doing its own thing:

Note that this is still standard time. Also note that the second measure here uses eighth notes for both the treble and bass but the treble's eight notes clearly take up more "space" within the measure (four eighth notes are clearly different between the two staffs).
Comment: If I can state the obvious. Sometimes people break the rules. Changing time signature temporarily without actually telling you. Usually returning to a very clear 'one' immediately afterwards. Here it appears that they've broken the rules by using triplets, and not told you about it. Is there anything at the head of the sheet which might indicate this?
Comment: I like the theory, but there anything there to suggest that is true here. Sorry @AJFaraday!
Comment: I said that it appears to be implicit triplets in this case.
Comment: Sorry, I didn't mean to discredit your observation. Simply responding to your question about if something at if there was anything at the head of the sheet to indicate your theory. Not that I can tell at least. Sorry for being too quick with my reply :)
Comment: Written out the way it is, l.h. can only be triplets. There's no other explanation. At least the engraver has been thoughtful enough to write each hand as it's played, from a vertical point of view.
Answer: They are actually eighth note triplets instead of eighth notes. The alternative notation to this would be to group the eighth notes and rests in threes and put a 3 over them like a standard triplet, but it's easy enough to see that you are fitting 12 equally spaced notes in a measure which end up being eighth note triplets which would kind of screw up the legato flow of the passage.
Another thing to note is it is very common if you expect to be playing in 4/4 with almost exclusively triplets you would play in 12/8 so you don't have to put triplets over each set of three eighth notes they would just work out to 12 eighth notes a  measure.  This piece takes this idea , but just keeps the piece in 4/4 and it is just implied that the eighth notes are triplets.
Comment: I'm not sure if I would agree that it's "easy" to see it, except once you compare the right and left hands in the last quoted measure it becomes the only explanation that makes sense.
Comment: @ToddWilcox when you have 4 quarter notes per measure and you're putting in 12 eighth notes in, it's the thing that comes to mind first.
Comment: Well it wasn't for me (or the asker, apparently), perhaps I'm more the exception in this case.
Comment: One interpretative note: if you notate it with triplets, it would imply a bit of emphasis on the first note of each triplet. Rather than being a shortcut, Liszt probably wanted all the bass notes played with an even strength with all the emphasis on the sustained D. The notes should stream out as a soft flowing tail to a profound D (it's like a "three voices-two hands" part). I imagine he wanted the performer to focus on an even legato flow, not math correctness.
Comment: @AustinHaskew the composer chooses the notation for his piece how he wants to it doesn't have to be the standard. He can omit a time signature completely as seen in the second movement of _Quartet for the End of Time_ or make the score [completely graphic](http://music.stackexchange.com/questions/23725/how-do-you-read-graphic-scores).
Comment: I immediately recognized them as unmarked triplets, but this may reflect my loooooooong experience in playing music :-)
Comment: Easy to see or not, the notation is just plain wrong.  From a theory standpoint, it's simply not valid.
Comment: @AustinHaskew I'm going to have to go with Liszt, Rachmaninoff et al. here and say that you're being overly pedantic.
Comment: @AustinHaskew I second Dom and Kyle here.  This sort of thing happens all the time, falling into the category of "conventions that everyone understands."  Similar to Bach's use of dotted-8th-16th to mean a triplet 8th plus a 16th.
Comment: @Dom I would point out that if you wrote this in 12/8, then you'd have to put a 2 over the 8th note duples in the right hand or let those be implied as duples.  I think that given the *bel canto* style, the melody needs to be controlling as far as time signature, which I think is why he left it in common time.
Comment: @AustinHaskew - 'simply not valid'. All that's missing are the triplet signs, which for a lot of us are superfluous. It's not difficult at all to work out that 12 quavers, as quavers, won't fit into a 4/4 bar.
Answer: The eighth notes in the left hand are all triplets.  The ones in the right hand are normal.  Note how the note heads line up vertically in measure 4.
On a purely technical level, this is incorrect notation.  But it's something that can be figured out pretty easily, so I guess Liszt either didn't care or wrote it like that for artistic reasons.
Comment: @KJPrice The treble (right hand) notes in the second half of the last measure are regular eighth notes. 1) They are beamed in a group of four. 2) They add up along with the half note to a 4/4 measure. 3) They are spaced in the measure like 8th notes. The triplets/12tuplets in the left hand are *not* spaced the way 8th notes are which is why they aren't lined up with the "real" 8th notes. Also the left hand notes (bass) are all beamed in one group which would not be done with normal 8th notes. Finally, they don't add up to a 4/4 measure (which is what prompted your question, of course).
Comment: This lax notation is actually not uncommon for Liszt (and some of his contemporaries like Chopin).  Many times, a run will be seen with an arbitrary number of notes that doesn't rhythmically divide into the measure in any fashion whatsoever, like 17 notes or something like that.  He was generally more concerned with the melodic flow of the notes than their mathematical precision.
Comment: Further to this, it's pretty common for the first bar to be notated as triplets, then write "simile" on the next bar, and job done for the rest of the piece.
Comment: Can you please explain the bit about the eight notes in the right hand being normal? Are you referring to those in the base? They seem to be the same as the eighth notes on the left. Or are you referring to the treble notes on the right?
Answer: In support of the other answers here, I have re-notated this passage in your example to emphasize the triplets. This is the exact same passage of music (unless I have made a typo or two) but using extra symbols to make it more explicit.
Note that in measure 4 you are required to play "two against three": your right hand is in a duple rhythm while your left hand is in a triple rhythm. This is tricky!
Comment: To my mind, the first measure should have a 12-[tuplet](https://en.wikipedia.org/wiki/Tuplet) notation across it, with a *simile* marking at the start of measure 2. Problem solved with elegance *and* accuracy.
Comment: @KJPrice, I don't think that "simple" and "Liszt" can exist in the same sentence. Neither can "simple" and "Chopin". I'm not a pianist but I'm sure there are other Romantic-era composers that would be an easier place to start.
Comment: This is perfect! Thanks @wheat! Yeah I keep picking songs that I think will be simple and then I get a good ole romantic-era slap in the face!
Comment: Do you agree the original gives a better impression of continuous musical flow?
Comment: @LaurencePayne, I agree. The original is how Franz Liszt or his publisher wanted it to appear. I created my version only to analyze it -- to show the triplets clearly to demonstrate how to count the subdivisions of the beat. That is all.
Comment: Lol. Live and learn.
Comment: Watch out for this in Debussy and Brahms, too.
Comment: @AakashM And the question shows a snippet beginning with measure 3.  It's possible that the score actually does.
Comment: The uppermost measures are the first two measures (starting with the word "Piano").
Answer: I would hold a slightly different opinion to those already given and say the proper notation is 12 in the time of eight which is played the same as four triplets but still this is 12 in the time of eight.
The person that did the transcription probably felt the marking for twelve in the time of 8 would be to hard and left you to scratch you head instead.
Comment: So why beam in two groups of 6?   And what 'transcription'?  The piece was composed, not transcribed from a recording.
Comment: This would be more representative if all the quavers were joined together, otherwise there would be unnecessary emphasis on the 2nd part of the bar, which I imagine Liszt wanted to avoid.


Expression's integral domain

Question: Given the next expression:
$$f(x)=\int_{0}^{x^2}{e^{-2\sqrt{t}}}dt$$
How can i calculate its domain? I figured that it was related to its derivative, but not sure how to get there.
Help? Thanks
Comment: This is defined for any real $x$.
Comment: You are not given a function. You are given an expression. The question is: For which real numbers $x$ is the expression defined?
Comment: i've edited, thanks
Comment: Do you mean the domain of $t$? It's from 0 to a positive real number $x^2$. If you want to solve the integral substitute $t=y^2$.
Comment: take a look [here](https://proofwiki.org/wiki/Continuous_Function_is_Riemann_Integrable)
Comment: @Agapita An "integral domain" is sadly not the domain of an integral; it's a kind of ring. (The word "integral" here means "a bit like the integers".) Accordingly I have removed that tag.
Answer: To find the domain of $f(x)$, we will begin by evaluating the integral.
$$\begin{aligned}
f(x)&=\int_0^{x^2}e^{-2\sqrt t}dt \\
&=\int_0^x2ue^{-2u}du;t=u^2 \\
&=\left(ue^{-2u}+\frac 12e^{-2u}\right)_0^x \\
&=\left(xe^{-2x}+\frac 12e^{-2x}\right)-\left(\frac 12e^0\right) \\
&=xe^{-2x}+\frac 12e^{-2x}-\frac 12.
\end{aligned}$$
This is clearly defined over all of $\mathbb R$.

Expression's integral domain


SQL query to update 1000 records

Question: I have created 1000 records with ID as sequence Id. Now, I have list of 1000 unique codes, which I need to assign to each record.
Like this:

I am looking for suggestions to achieve it using SQL script.
Comment: Start over and insert the unique codes and id at the same time.
Comment: If you don't have access to the codes, then  how are you going to assign them?
Comment: @jarlh At this point, I dont have access to the codes. Thanks for the quick response.
Comment: In addition to the question by @grootygroot ( he beat me to it) , even when you have access to the codes, how do you determine which code is assigned to which id?
Comment: @a_horse_with_no_name codes will be given to another team, I have to provide update script for them to update the codes
Comment: @EdStevens that order doesn't matter. but each id should be assigned a unique code
Answer: You could use a MERGE statement:
MERGE INTO destination_table dst
USING (
  SELECT ROWNUM rn,
         value
  FROM   your_codes_table
) src
ON ( src.rn = dst.id )
WHEN MATCHED THEN
  UPDATE SET code = src.value
WHEN NOT MATCHED THEN
  INSERT ( id, code ) VALUES ( destination_table__id__seq.NEXTVAL, src.value );

However, that will increment the sequence for every row updated and inserted and this may not be what you want.
The simplest way is to delete the existing rows (without codes) from your destination table and the delete the sequence you are basing the id values on and then recreate the sequence (to reset the sequence back to its starting value):
DELETE FROM destination_table;
DROP SEQUENCE destination_table__id__seq;
CREATE SEQUENCE destination_table__id__seq;

and insert the rows:
INSERT INTO destination_table ( id, code )
SELECT destination_table__id__seq.NEXTVAL,
       value
FROM   your_codes_table

db<>fiddle
Answer: Do you just want row_number()?
select code, row_number() over (order by code) as id
from t;

SQL query to update 1000 records


Comparing intervention/control group

Question: I am planning to conduct a study which is based on an RCT design.
I am intending to conduct an intervention. During this intervention, participants and control group enter some medical data on daily base for a predefined time frame (like 6 months).
I wonder whether there is a better approach to investigate the effect of intervention besides averaging medical data per participants and doing an repeated ANOVA. I wonder about this because averaging does not really take into account whether the intervention stables good health behaviour over the time compared to control group.
Thanks a lot!
Answer: The basic model for the ANCOVA is:
$$ E[Y_{t>t_0} | X, Y_{t_0}] = \alpha + \beta_1 Y_{t_0} + \beta_2 X$$
Where $t_0$ is the baseline and $Y$ is the time series of the response and $X$ is the indicator for receipt of intervention. The coefficient $\beta_2$ summarizes the average difference in change from baseline between treated and control. $\alpha$ is the expected change from baseline in the control group.
Of course, I've suppressed the actual expression of the error term, you can use any number of model based or robust estimates (GEE, mixed model, etc.) to handle repeated measures. The point is even if there's a time-varying trend in the post-baseline response, such as comes from a learning effect or growth effect, the hypothesis of a mean-difference-in-differences is still well motivated. In other words, rejecting the null, you can still state that participants in the intervention group on average have a greater or lesser change from baseline than those in control. This is supported both by theory (exact derivations can show hits to power but the test retains suitable power depending on the design) and by practice (guidance on estimands supports fitting the model that most closely parametrizes the quantity of interest based on initial hypotheses and settings).
However you can generalize the ANCOVA to fit any complex time-series for the mean change-from-baseline process in treated and control using splines, filters, or other. Splines would be preferable in my book because it would be quite simple to construct a hypothesis test using a likelihood ratio test. You could argue that the ANCOVA is a time series model using a "constant" for the change-from-baseline-process. This family of tests is often called "interrupted time series" but the expression of time is almost always no more complicated than a first order, i.e. linear effect. The consequent likelihood ratio test for the interaction slope and interaction intercept parameters are sensitive to detect a "jump" or "growth" effect.
Comment: Thanks! Any recommendations for a suitable R package?
Comment: Sadly you'll need a reference on how to fit these models using any of the repeated measures packages, I think LME4 is more than capable.
Comment: what do you exactly mean by needing a reference? Can you elaborate further, please?
Comment: What I mean is that there is no simple R program (that I trust) into which you can just dump your data and get a result. The syntax in LME4 is tricky, and even harder in NLME. By reference I mean either a statistical primer or reviewer to help assess the model fit, adequacy, diagnostics, etc.
Comment: Thanks for clarifying

Comparing intervention/control group


Oracle 11g SELF member procedure not working

Question: I have the following:
create type customer as object (
id number, name varchar2(10), points number,
member procedure add_points(num_points number)
) not final;
/

create type body customer as
member procedure add_points(num_points number) is
begin
   points := points + num_points;
   commit;
end add_points;
end;
/

create table customer_table of customer;
/

insert into customer_table values (customer(123,'joe',10));
/

i then do this is an anonymous block:
declare
cust customer;
begin
select treat(value(c) as customer) into cust from customer_table c where id=123;
c.add_points(100);
end;

but nothing happens - the points value remains at 10.
What have i missed? If i make my member procedure an update...set...commit and pass in the points and a given id, it works.
Thanks.
Comment: I know from dealing with one of your previous questions that you are on a training course of some sort.  Unfortunately they do seem to be training you in some rather arcane and pointless features of Oracle!
Answer: The PL/SQL you posted is invalid.  I guess you meant to post this:
declare
  cust customer;
begin
  select treat(value(c) as customer) into cust from customer_table c where id=123;
  cust.add_points(100);
end;

i.e. "cust" not "c" on line 5?
If so, all you have done there is updated the value of points in the variable cust, not in the table.  You can see that like this:
declare
  cust customer;
begin
  select treat(value(c) as customer) into cust from customer_table c where id=123;
  cust.add_points(100);
  dbms_output.put_line(cust.points);
end;

Output:
110

To update the data in the table does indeed require an UPDATE statement.


Deleting file after progeam execution in batch

Question: I am having a simple question, having simple batch script:
for /l %%x in (1, 1, 3) do start /wait c:\some.exe -verbose c:\someLog.txt del c:\someLog.txt

after each execution file creates log, and I want to remove that log before next loop execution, execution works fine but when I add del command its having problems, it looks like log is being deleted to early.
Is there a possible way to delay del command ??
Comment: Possible duplicate of [How to run two commands in one line in Windows CMD?](http://stackoverflow.com/questions/8055371/how-to-run-two-commands-in-one-line-in-windows-cmd)
Answer: When running multiple command I usually nest them for readability.
for /l %%x in (1, 1, 3) do (
    start /wait c:\some.exe -verbose c:\someLog.txt
    del c:\someLog.txt
)

You could probably do it this way as well
for /l %%x in (1, 1, 3) do (start /wait c:\some.exe -verbose c:\someLog.txt &del c:\someLog.txt)

But this does not mean that your executable is actually respecting the WAIT option.  Many programs do not.  I would just use the exe directly without using START at all.


What is the density of a brownian bridge $X_t=B_t-tB_1$?

Question: The original question is:
For $t\in [0,1]$, we define $X_t=B_t-tB_1$, where $\{B_t:t\geq 0\}$ is a standard Brownian motion. Find the density of $X_t$ .
After reading several resources, I think $X_t$ is a normal distribution. However, since most of the books concerning on the process itself other than the distribution of $X_t$, can someone confirm this?
Answer: Hints:

$X_t=(1-t)B_t - t(B_1 - B_t)$.
$B_1 - B_t \sim N(0,1-t)$ and $B_t \sim N(0,t)$ are independent.
Comment: @Gono Thanks for the correction!
Comment: Actually it's $$B_t-B_1 \sim N(0,1-t)$$ because $0 \le t \le 1$
Answer: Yes, it is normal. Brownian motion is a Guassian process, which means that its finite dimensional distributions are all multivariate normal, and in particular any linear combination of $B_{t_1},\ldots,B_{t_n}$ is normal for any indices $t_1,\ldots,t_n$.
To find the density of $X_t$, we need only calculate its mean and variance. Its mean can be found as
$$E(X_t)=E(B_t)-tE(B_1)=0$$
and its variance can be found by calculating its second moment using $E(B_sB_t)=s\wedge t$:
$$\operatorname{Var}(X_t)=E(X_t^2)=E(B_t^2)-2tE(B_tB_1)+t^2E(B_1^2)=t-2t^2+t^2=t(1-t).$$
Hence for $t\in(0,1)$, $X_t$ is $\mathcal N(0,t(1-t))$ and so has density
$$f(x)=\frac1{\sqrt{2\pi t(1-t)}}e^{-x^2/2t(1-t)}.$$
Comment: @Jango I'm not totally sure what you mean by affecting the distribution - what you mention is an alternative characterization (or definition) of a Brownian bridge, but this fact requires some proof.
Comment: You missed a $t^2$ while calculating the variance…
Comment: Books said $X_t$ is a Brownian motion conditional on $X_1 = 0$. Does this affect anything about the distribution of $X_t$?
Comment: @Gono Thanks for the correction!


How to put google map field in my form of my custom module?

Question: i like to build a form field based on google map, where user can put in their address, city and country, then the map will automatically search for it.
and then you can put a pin to define the exact location, which the expected output is latitude & longitude which will be saved during form submit.
how to approach this? is there any module in Drupal 8 that i can use during form generation?
maybe something like:
<code> $form['map'] = array(
      '#type' => 'gmap',
      '#otherProperty' => propertyValues,
 );
</code>
Thank you,
AnD
Answer: use this
it works for me
https://www.drupal.org/project/google_map_field
.
.
.
.
.
.
Comment: Welcome to Drupal Answers! Whilst this may theoretically answer the question, [it would be preferable](http://meta.stackoverflow.com/q/8259) to include the essential parts of the answer here, and provide the link for reference.


Options for optimizing exponential regression

Question: I have an exponential regression equation that I use to predict the future condition of roads:
$$y=21-e^{a x}$$
I've come up with an initial estimate for $a$ using the normal equation.
$$a=\frac{\sum_{i=1}^n x_iz_i } { \sum_{i=1}^n x_i^2
 }=\frac{\sum_{i=1}^n x_i \log(21-y_i)} { \sum_{i=1}^n x_i^2 }$$

<code>C9     =21-EXP($C$33*A9)
C33    {=SUM(VALUE(A9:A29*LN(21-B9:B29))) / SUM(VALUE(A9:A29^2))}
C34    {=SUM(VALUE(  (B9:B29 - C9:C29)^2  ))}
</code>

Question:
What are the options for tuning the exponential regression estimate to optimize the coefficient $a$? In other words, I want to minimize the Error Sum of Squares (ESS).

I'm wondering what the options are, not because the options I've found so far have problems, I just want to explore what different methods are possible as as a learning exercise.

I'm looking for options that are reproducible/explainable. I want to be able to show my colleagues how the math was done so that they can thoroughly understand where the results came from. In other words, I'm looking for options that aren't a black box.

My preference is to do the calculations in Excel using basic formulas. Reason: My department is an Excel shop (we also have access to Python through GIS tools). But we don't have access to statistical software. So, installing software, even if it's free, is difficult due to IT security restrictions.

Edit: To my surprise, I was able to install R without admin privileges on Windows 10. Cool, although I don't know how to use it yet. Download.

A side-benefit to using simple Excel formulas is that we can migrate the logic to other programming languages if we need to (such as Oracle SQL).

With that said, I'm open to other ideas if they have significant benefits.

Here are the options I've come across so far:

Excel: Newton-Raphson iterations (Calculus)

Uses basic Excel formulas; relatively easy to explain. Can be migrated to other programming languages.

Excel: data table (hit and trial)

Works, but is somewhat labor intensive. And is somewhat of a black box.

Excel Solver using nonlinear regression

Works, but is a black box. I don't have access to information about how the calculations were done.
Related: Mimic Excel Solver nonlinear regression? (to reduce ESS of exponential regression)

As someone who is inexperienced with math/stats, are there other options that I might have overlooked?
Comment: What do you mean by "tuning"? The value of $a$ you get from the normal equations is the value that minimises the squared error of your model. This is a standard decision criterion in practice. Why would you change $a$ after selecting that value?
Comment: Apologies, I now realise what you mean. The value of $a$ you get from the normal equations is the value that minimises the error given by $\| ax - \log (21 - y) \|_2^2$, but not necessarily the error given by $\| y - (21 - e^{ax}) \|_2^2$. I think your approach of taking an initial solution via normal equations and then optimising the secondary loss function via Newton-Raphson is fine.
Comment: @JoseAvilez Good question. I edited the post. I want to minimize the Error Sum of Squares (ESS). For example, I can minimize the ESS via Newton iterations or Excel Solver.  And I'm wondering if those were my only options, or if there are others.
Comment: The normal equations give you the value of $a$ that minimises the ESS for your model. There is no need to tune any further.
Comment: @JoseAvilez Forgive my ignorance, but it's my understanding that the value of $a$ from the normal equation (8.27990876) can be reduced to 6.665716708, using Newton iterations, Excel Solver, Excel data table (hit and trial) etc. But it's entirely possible that I've misunderstood something.
Comment: @JoseAvilez Do you think the title or wording of the post should be improved? For example, should the title be something like *"Normal equation solution: Options for optimizing secondary loss function?"* Or something else? Also, feel free to edit the post, if that's easiest.
Comment: The title is fine
Answer: If you are trying to minimize the ESS, the given exponential equation is the best one. That is, Excel has given you the absolute best exponential equation possible. That being said, there are 2 options for producing "better" results, or more what I assume you are looking for.
Firstly, you could perhaps try to smooth out or fix any errors in your data. There seems to be a number of spikes and inconsistencies. The best way to fix this issue is to gather more data and use an average. However, that may not be feasible, which is why there's another way.
Excel has a wonderful built in trendline menu. Open the "Format Trendline" menu and under "Trendline Options" select "Polynomial". Increase the order to at least 3 and possibly 4 or even higher. To see the formula select the "Display Equation on chart" (although I believe you already have done this). Additionally, you could select the "Display R-squared value on chart". You can play around with settings like the order and observe this R-squared value. The closer it is to 1, the better. You should then have a clear equation that's easy to work with.


Prove that in a simple graph with $\geq 2$ nodes at least one node can be removed without disconnecting the graph

Question: Prove that in any simple graph $G$ with number of nodes $\geq 2$ there is at least one node $v$ that can be removed with its all edges, and keep the graph connected?
From my point of view I can say if there are nodes with a single edge then it can be removed without affecting the connectivity of the graph, and if there are two or more edges connected with a node, then we have at least a cycle, but a cycle can contain some removal nodes and some of non-removal. Could someone give me a concrete proof for this situation?
See the picture :

As we can see, it is possible to remove node A, or B, or C or D
Answer: Trees:
This is a tree, and, as you noted, there is a pendant vertex in the tree that can be removed without disconnecting the graph.
All other connected graphs:
Then this graph has a spanning tree, and you can remove one of the pendant vertices from the spanning tree. This clearly preserves some path between all the remaining vertices, because removing the pendant vertex does not disconnect the spanning tree.

There seems to be some confusion, so I'll illustrate the point with the image provided.
Take the subtree with edges $\{ab,bc,cd,ce\}$. We can see that in this tree, vertices $a$, $e$, and $d$ are pendant. As we demonstrated in the first case, removing one of these pendant vertices does not disconnect the subtree.
For example, if we delete $a$, then the rest of the graph ($\{b,c,d,e\}$) is still connected within the subtree, and so returning the other edges to it will not disconnect the graph.
Comment: it is not a tree, because it could have a cycle
Comment: @CuddlyCuttlefish In some cases there are   cycles and of course it is not a tree
Comment: "spanning tree" - cycles, like all graphs, have spanning trees ($P_n$). The tree case needs to be stated so that we know the subtrees remain connected in the second case.
Comment: All connected graphs, I should say.
Answer: Since $G$ is connected, there is a walk $W$ that passes through every vertex of $G$ such that the end vertex $v$ occurs exactly once in $W$ (we stop the walk when we've seen every vertex).
Since $W \setminus v$ is a walk through every vertex in $G \setminus \{v\}$, it is also connected.
Comment: This is by far the more elegant proof! +1