Use more complex where clauses in Scout

Question: I am a happy user of Laravel Scout.
Now I'd like to extend my search:
        $data
            = new UserOverviewResourceCollection(User::search($searchphrase)
            ->currentStatus('active')->orderBy('lastname', 'asc')
            ->orderBy('firstname', 'asc')
            ->paginate(config('pagination.length')));

currentStatus comes from https://github.com/spatie/laravel-model-status .
Now I am getting a response, that currentStatus is not supported. I thought I could be a good idea to filter the result of User::search after it has been returned from scout?
Another idea: I'd like to add more complex where clauses:
->where([
                [
                    'starts_on',
                    '<',
                    Carbon::now()->toDateTimeString(),
                ],
                [
                    'ends_on',
                    '>',
                    Carbon::now()->toDateTimeString(),
                ],
            ])

Maybe you got a better idea?
Answer: I really need similar functionality as well but I'm afraid it's not possible at the moment (Laravel 8)
According to the docs; Laravel\Scout\Builder (which is what search() returns) only has a really basic implementation of the where method an can only handle exactly 2 arguments.
Here is the code from the scout package:
/**
 * Add a constraint to the search query.
 *
 * @param  string  $field
 * @param  mixed  $value
 * @return $this
 */
public function where($field, $value)
{
    $this->wheres[$field] = $value;

    return $this;
}

See https://laravel.com/docs/8.x/scout#where-clauses
Putting the search() behind all regular eloquent builder methods won't work either since search will only be a static function of the model.
Hopefully this will be improved in the future.
Answer: You can extend Builder and add currentStatus method to store required status in builder. See example
https://github.com/matchish/laravel-scout-elasticsearch/issues/47#issuecomment-526287359
Then you need implement your own engine and build query using builder there.
Here is example for ElasticSearch engine
https://github.com/matchish/laravel-scout-elasticsearch/blob/06c90914668942b23ffaff7d58af5b9db1901fb1/src/Engines/ElasticSearchEngine.php#L135
Answer: I first stumbled upon your question when I was having the same problem a couple of days ago, which is when i decided to build my own solution.
I created a laravel scout fork, and added the following features :

An array of arrays can be passed as a parameter : where([['id', '>', 5],['id', '<', 10]]).
Any operator supported by the Eloquent Builder is now supported as well : '=', '<', '>', '<=', '>=', '!=', etc.
Clauses containing the operator like so : where('id', '>', 5).
All changes are fully backward compatible, If no operator is passed as a parameter, it will be assumed that the desired operator is '='

Inside your laravel project, scout is found in : vendor/laravel/
Replace the scout directory by the cloned repository, and you will be able to use the complex where clause normally.
You can find the repository here : https://github.com/alibaddd/scout
Answer: Well rewriting it as this should work.
 $data
            = new UserOverviewResourceCollection(User::search($searchphrase)
            ->where('starts_on','<',now())
            ->orderBy('lastname', 'asc')
            ->orderBy('firstname', 'asc')
            ->paginate(config('pagination.length')));

Notice that now() is just a global helper function returning current moment Carbon instance. It's just shorter to write, no other reason for using it.
And if you want grouped where queries you do it like this:
 $data
            = new UserOverviewResourceCollection(User::search($searchphrase)
            ->where(function($q) {
               $q->where('starts_on','<',now())->where('ends_on','>',now())
            })
            ->orderBy('lastname', 'asc')
            ->orderBy('firstname', 'asc')
            ->paginate(config('pagination.length')));

And then you should be able to export this to the UserOverviewResourceCollection as a local scope, for example:
public function scopeActive($q) {
    return $q->where('starts_on','<',now())->where('ends_on','>',now())
}

and then you can use something like this:
 $data
            = new UserOverviewResourceCollection(User::search($searchphrase)
            ->active()
            ->orderBy('lastname', 'asc')
            ->orderBy('firstname', 'asc')

I've written this from my head so there might be typos.
Comment: https://laravel.com/docs/5.7/scout#where-clauses Seems complex (whatever that means) queries are not allowed so that could be reason why it's not working. As a long shot, you might try putting the `where` query before the `search()` otherwise you'll probably have to index the `starts_on` and `ends_on` fields in Algolia and then do the query there through their API.

Another, ugly, workaround might be to get the ID's from Algolia and then re-do the query on your database using the `whereIn('id',$arrayOfIDsFromAlgolia)` but there should be a way to do it in Algolia.
Comment: Hm, the scope does not work, actually (`Method Laravel\\Scout\\Builder::active does not exist`).  Unfortunately the grouped where does not work, too (https://laravel.com/docs/5.7/scout#where-clauses) -> `Too few arguments to function Laravel\\Scout\\Builder::where(), 1 passed in .. line 91 and exactly 2 expected`
Comment: Let's go one step at a time. Does the first chunk of code works? If it doesn't, can you remove the `->where('starts_on','<',now())` part and verify that it works then?
Comment: Alright... No, the first chunk does not work:

    "message": "Invalid syntax for numeric condition:starts_on=<",
    "exception": "AlgoliaSearch\\AlgoliaException",
    "file": "/home/vagrant/code/vendor/algolia/algoliasearch-client-php/src/AlgoliaSearch/Client.php",

If I remove the `where` clause it works.
Answer:  $data
        = new UserOverviewResourceCollection(User::search($searchphrase)
        ->currentStatus('active')->orderBy('lastname', 'asc')
        ->where(function($q){
            $query->where('starts_on', '<',Carbon::now()->toDateTimeString());
            $query->where('ends_on', '>',Carbon::now()->toDateTimeString());
         })
        ->orderBy('firstname', 'asc')
        ->paginate(config('pagination.length')));

Try this code. This will satisfy your complex where condition
Comment: Hi and thank you.

Unfortunately it still returns `Method Laravel\\Scout\\Builder::currentStatus does not exist.`
Comment: Hi, currentStatus functin is written in you question itself.

please remove "->currentStatus('active')" and try.
Comment: Yes, it's written there for a good reason - I need it ;-)

`currentStatus` comes from currentStatus comes from https://github.com/spatie/laravel-model-status as i wrote on my initial question :-)

Use more complex where clauses in Scout


using drush php-eval db_query() are there differences with db_query() from code?

Question: I think the answer should be easy but I can't find it.
maybe I've been put in the wrong direction finding sites telling how to clear my flood table like so:
drush php-eval 'db_query("DELETE FROM `flood`");'

But I think is should be like this so it also works for prefixed tables:
drush php-eval 'db_query("DELETE FROM `{flood}`");'

Or does db_query() works different in drush php-eval?
As far as I know it just executes php but does it has the drupal 'library' available or not?
Now I could just try and see what happens but I think it's good to really know if there's a difference or not in running db_query() from drush or from code.
Answer: I don't know this for definite but conceptually speaking, if Drush didn't use a standard Drupal bootstrap to perform these tasks it would have to re-implement all of that functionality itself.
Drush uses your site's settings.php file for the database connection, which leads me to believe it will also have access to things like $db_prefix.
I think it's safe to say exactly the same thing is happening when you run db_query() from Drush as if you run it from any other bootstrapped instance of Drupal.
Comment: Yes, that is absolutely correct for php-eval: the code will execute in the same environment that it does when running from within a Drupal site, so prefixed tables will be handled for {flood} and similar.  However, please note that the sql-query command avoids modifying your query in any way unless you specify the --db-prefix option.  The sql-query version of the above example would therefore be: `drush sql-query --db-prefix 'DELETE FROM {flood};'` (please excuse quoting variation; stack exchange code filter in effect)


como ler este escopo?

Question: Preciso ver detalhadamente, pelo que eu li e o seguinte:

SE A VARIAVEL $NAME ESTIVER PRESENTE E $NAME NAO ESTIVER VAZIA EXECUTAR: ADICIONAR NA DATA BASE COLUNA "AVEIME" LOGIN, E SENHA AS VARIAVEIS $NAME E $PASSWORD, MEDIANTE $RESULTADO_USUARIO, SE EXECUTADO $RESULTADO_USUARIO EXECUTAR  .. AI JA NAO SEI MAIS... PODEM ME AJUDAR?

if (isset($name) && !empty($name)){

$result_usuario = "INSERT INTO aveime (login, senha) VALUES ('$name', '$password')";

$resultado_usuario = mysqli_query($conn, $result_usuario);

    if($resultado_usuario){

            $_SESSION['id'] = mysqli_insert_id($conn);

            header("Location: form2.php");
    }

Answer: if($resultado_usuario){

    $_SESSION['id'] = mysqli_insert_id($conn);

    header("Location: form2.php");
}

Se o registro foi salvo no banco com sucesso (if($resultado_usuario)) então pega o ID inserido na última consulta ao banco (mysqli_insert_id($conn)) e salva ele numa session ($_SESSION['id']), então redireciona o usuário para a página form2.php (header("Location: form2.php"))
Ps: caso a session não esteja salvando o id, verifique se você está inicializando a sessão no início do arquivo com session_start();

como ler este escopo?


configuring sendmail for the first time

Question: I am a newbie to all of this. I just installed sendmail however, I am getting the following error:
Apr 13 05:24:06 localhost sendmail[27071]: My unqualified host name (localhost) unknown; sleeping for retry
Apr 13 05:25:44 localhost sendmail[27074]: My unqualified host name (localhost) unknown; sleeping for retry
Apr 13 05:27:01 localhost sendmail[27076]: My unqualified host name (localhost) unknown; sleeping for retry

What should I do?
I am guessing that it has to do with my /var/hosts setting. Please guide me through.
My hosts is:
127.0.0.1 localhost
188.8.131.52 admin.api.frapi
Answer: While it may not directly answer your question, I would suggest you apt-get install postfix - this is just personal preference but I find it easy to configure & manage. Lots of people love sendmail, I'm just not one of them :)
Comment: I actually just uninstalled postfix and revert back to sendmail
Answer: I don't have the exact answer for you, but I would recommend you to check out this page for configuring Sendmail.

configuring sendmail for the first time


What acoustic instruments are great for Droning?

Question: I want a drone, using only acoustic instruments, with as little effort as possible, to accompany singing.
Comment: I'm voting to close this question because this question is just asking for a list, making every answer essentially of equal value.
Comment: @phoog is there a stack exchange for instrument design info or something of the like?
Comment: Also see https://music.stackexchange.com/help/dont-ask and https://music.stackexchange.com/help/on-topic on the inadvisability of list questions and questions seeking recommendations.
Comment: Why does playing untouched matter?  That has nothing to do with drone.
Comment: @DataProcessing Interested in using it for singing only. So the least amount of "playing" possible.
Comment: This can easily be edited into a non-list question just by focusing on your need. "I want a drone, using only acoustic instruments, with as little effort as possible." Spoiler alert, pretty sure it's a sruti box.
Comment: @AndyBonner I don't know why, but I originally thought they were all electronic. I'm gonna look into them now. And wouldn't it still be a question asking for recommendations or a "list" of some sort? That's why I'm confused about where to have a question/discussion like this.
Comment: "Here's my problem, what's the best approach" can have multiple valid answers, but the answers tend to be substantive. One of the problems with "list questions" is that their answers are often very superficial, like "what are all the pieces Chopin wrote in C major"
Comment: Also, any answers are not likely to be useful to anyone else, since no one but OP knows the parameters of acceptance for their case.
Comment: For more of the rationale behind the policy, see [this meta post](https://music.meta.stackexchange.com/a/162/78419). See also [this post](https://stackoverflow.blog/2010/09/29/good-subjective-bad-subjective) about the notion that there can be good questions that still provoke differing valid answers. Your need might be a bit niche but it's totally possible that it help someone with the same need in the future.
Comment: Btw, you might explain why electronics are off-limits. You could easily do a lot just by putting an acoustic signal through a delay effect, or using an E-Bow. The current edit might be a bit *too* minimalist; e.g. does "as little effort as possible" is about making low demands on the performer during the performance, but could be mistaken for "the readiest available solution." You might also spell out that this is for a singer to self-accompany; thus e.g. most mouth-blown wind instruments are out.
Comment: One can play drone on any instrument capable of playing pitched notes. Which one is "great" for that purpose is opinion based. The question about "the least amount of effort" is subjective as well. The suggestion to list "personal favorites" is clearly off-topic.
Comment: "Interested in using it for singing only." Do you mean you want to sing while playing the instrument? This would make bagpipes, listed in both present answers, not a great choice.
Comment: @user1079505 It's ok, I understand what I did wrong, I'll just have to speculate for myself. Although, Andy Bonner answered for me before this post will get closed. It was something I did not realize before. Thanks.
Comment: @Lecifer Ask in [The Practice Room](https://chat.stackexchange.com/rooms/440/the-practice-room). That's the perfect place for this kind of discussion.
Answer: Trivially every wind instrument with a reservoir, like bagpipes, harmoniums, organs.  When the player's mouth cavity is the reservoir, this is called "circular breathing" and can be used for wind instruments with little air use.  The didgeridoo is regularly played in that manner, but circular breathing is also an advanced technique in some demand at professional level for instruments such as oboe and flute.
Then there are instruments with a continuous tone even if not of indeterminate length, like bowed strings.  The hurdygurdy even gives arbitrary-length notes by employing a circular bow and a crank.
Then there are instruments that are kept vibrating mechanically continuously, like singing glasses or even the glass harmonica, or percussion instruments played with soft mallets in a roll, like cymbals or timpani.  There are even plucked string instruments commonly played in a semicontinuous technique like mandolins and balalaikas.
Instrument sections with limited tone length can deliver drones by unsynchronizing breaks in their tone.
Answer: Organ, bagpipe, wind gong/tamtam, bells (including church bells as well as plate and tubular bells), Hurdy-Gurdy
The pipe organ is obviously the acoustic drone instrument, as with a single keypress it can sound for ever. And even when the key is released the sound will continue, thanks to the long reverb in churches


Numeros primos em letras a-z A-Z em python

Question: Neste problema você deve ler um conjunto de palavras, onde cada palavra é composta somente por letras no intervalo a-z e A-Z. Cada letra possui um valor específico, a letra a vale 1, a letra b vale 2 e assim por diante, até a letra z, que vale 26. Do mesmo modo, a letra A vale 27, a letra B vale 28 e a letra Z vale 52.
Você deve escrever um programa para determinar se uma palavra é uma palavra prima ou não. Uma palavra é uma palavra prima se a soma de suas letras é um número primo.
Estou em duvida de como fazer o for, eu já criei as variaveis, a = []
for i in range(ord('a'), ord('z')+1):
    a.append(chr(i))

Comment: Não entendi qual a dúvida
Comment: Então eu tenho que fazer um for que recebe palavras como numero, so que na hora de eu passa o for para o resultado se é primo ou não, não aceita a string
Answer: Basta você definir uma função que retorne o número que represente cada letra:
def get_ord_of_char(char):
  if 'a' <= char <= 'z':
    return ord(char) - ord('a') + 1
  else:
    return ord(char) - ord('A') + 27

Depois, calcular a soma dos números que representam a palavra:
soma = sum(get_ord_of_char(char) for char in palavra)

E, por fim, verificar se soma é um número primo. Ficaria assim:
palavra = 'teste'
soma = sum(get_ord_of_char(char) for char in palavra)
print('A palavra é prima' if is_prime(soma) else 'A palavra não é prima')

Basta só você implementar a função is_prime que recebe um número inteiro como parâmetro.
Comment: cara agradeço a ajuda mais vou desisti disso ai.
Comment: Essa pergunta é de uma entrevista de estagio, foi dessa maneira que foi pedido, se poder mandar o codigo todo eu agradeço.
Comment: Duvida fico maior ainda agora, e como eu faço a soma verificar se é primo, na hora que eu passo soma como atributo da erro
Comment: @Guilherme `soma` será um número inteiro. Você sabe verificar se é primo? Passar como atributo para onde?
Comment: if soma % get_ord_of_char(palavra) == 0:
  print('é primo')
  soma+=1
else:
  print('não é primo')
Comment: pelo menos o que eu entendi seria isso, o usuario informa a palavra e tenho que verificar se ela é primo ou não de acordo que cada palavra vale um valor, porém, eu não consigo fazer o if nela passando a função que cada numero vale um valor.
Comment: @Guilherme adicionei ao final da resposta um breve exemplo de como ficaria
Comment: eu ainda não consegui entender, pois não consigo fazer o for percorre, já falaram pra eu fazer ate com dicionario mais ta dificil.
Comment: @Guilherme que parte exatamente não entendeu? Eu praticamente coloquei o código pronto ali, não precisa alterar nada, basta implementar a função `is_prime`.
Comment: eu travei na parte de implementar a função, pq quando eu tento implementar a função eu coloquei a outra função como "numero" e acaba sempre dando erro.
Comment: Então edite a pergunta com o seu código, pois está difícil de entender qualquer coisa.
Comment: @Guilherme será melhor se editar a pergunta e colocar o que fez. Só assim irei conseguir ver onde está com dificuldade.


Using Network Creds in dotnet core app on a mac using HttpClient

Question: Writing a dotnet core app. I need to log in with network credentials as the service (which happens to be a TFS on-prem server) uses those to authenticate. From my (and another team members') windows machine, the following code works:
Console.WriteLine("Type in your DOMAIN password:");
var pass = GetPassword(); //command line secure string magic from SO
var networkCredential = new NetworkCredential("USERNAME", pass, "DOMAINNAME");
string tfsDefaultCollection = "https://TFSURL/DefaultCollection";

string testUrl = $"{tfsDefaultCollection}/_apis/tfvc/changesets/1234/changes?api-version=2.2";

var httpClientHandler = new HttpClientHandler
{
    Credentials = networkCredential
};

var client = new HttpClient(httpClientHandler)
{
    BaseAddress = new Uri(testUrl)
};
httpClientHandler.PreAuthenticate = true;

var test = client.GetAsync(testUrl).Result;

Console.WriteLine(test);

But it doesn't work from my mac. I get a 401 unauthorized. Both used the same, hardwired connection. AND this works on my mac:
curl --ntlm --user "DOMAINNAME\USERNAME" "https://TFSURL/DefaultCollection/_apis/tfvc/changesets/1234/changes?api-version=2.2"

So that rules out a connectivity question, I would think. Am I missing something I need to be doing on my mac? Can anybody point me to some documentation or way to troubleshoot what both of these requests are doing at the lowest level to see if there is a difference?
Comment: Have you checked this question? It doesn't have an accepted answer but looks to be close to your issue: https://stackoverflow.com/questions/47902642/net-core-and-ntlm-on-osx I think the key difference being related to the CredentialCache - https://learn.microsoft.com/en-us/dotnet/api/system.net.credentialcache?view=netframework-4.8
Comment: Thanks, I just tried that and no go. Although I think that was one of my many iterations before as well.  Works on my windows machine tho...
Comment: Have you tried using the client for accessing TFS? https://www.domstamand.com/accessing-tfs-2017-programmatically/
Answer: Well finally some google-foo got me there. There's a bug in dotnet core for linux/mac. This issue describes the fix:
https://github.com/dotnet/corefx/issues/25988#issuecomment-412534360
It has to do with the host machine you are connecting to uses both Kerberos and NTLM authentication methods.
Implemented below:
AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

Console.WriteLine("Type in your DOMAIN password:");
var pass = GetPassword(); //command line secure string magic from SO
var networkCredential = new NetworkCredential("USERNAME", pass, "DOMAINNAME");
string tfsDefaultCollection = "https://TFSURL/DefaultCollection";

string testUrl = $"{tfsDefaultCollection}/_apis/tfvc/changesets/1234/changes?api-version=2.2";

var myCache = new CredentialCache
{
    {
        new Uri(testUrl), "NTLM",
        networkCredential
    }
};

var httpClientHandler = new HttpClientHandler
{
    Credentials = myCache
};

var client = new HttpClient(httpClientHandler)
{
    BaseAddress = new Uri(testUrl)
};
httpClientHandler.PreAuthenticate = true;

var test = client.GetAsync(testUrl).Result;

Console.WriteLine(test);

Thanks to @dmcgill50 for getting me on the right googling track.


Trigger an SSRS Report from ASP.Net without user intervention

Question: I'd like to be able to fire off an SSRS Report from an ASP.Net application without pulling it up in the application and having to print it from there to a specified printer.  This seems like it would be a highly desired feature, but I'm having trouble finding any good solutions online.  Can anyone help me with this?
Comment: for programmatically creating subscriptions: http://stackoverflow.com/questions/20330071/can-i-send-ssrs-custom-subscription-e-mails/20330200#20330200
Answer: rFirst you will need to set up the report to render from an execution snapshot. You should find this under the reports property tab, execute, select the "Render this report from a report execution snapshot" radio button.
Next you will need to set up a subscription for whom you want it delivered to. Under the subscriptions tab, add new subscription, enter the e-mail information, etc. Under "Subscription Processing Options" select "when the report content is refreshed".
Now you need to be able to trigger a new snapshot programmatically. Fortunately, the reports service exposes a web service for this purpose. Add a service reference to:
http:// your_report_server:your_port/ReportServer/ReportService.asmx
Once you add the reference then you simply need to call the UpdateReportExecutionSnapshot method to cause you report to execute and mail out to your subscribers.
A simple c# command line app to this might look like:
<code>static void Main(string[] args)
    {

        // The first argument should be the full report path
        // and name. It is passed directly to the Ssrs web service

        if (args.Length == 0)
        {
            throw new ArgumentException("Full report name must be the first parameter");
        }

        // create the endpoint

        ReportingServiceSoapClient ssrs = new ReportingServiceSoapClient();

        // Update the snapshot

        ssrs.UpdateReportExecutionSnapshot(null,args[0]);
     }
</code>
For the report name, you will need to specify the full report name including any sub folders you have. So if you have your reports say organized by division you would have to specify all the parent folders for example:
/DivisionName/DailyReports/SalesReport
Comment: Interesting... Good information.  I'll look into to the web service.  I see that I neglected to mention that I wanted the report to go directly to a specified printer.  Do you know if that is an option?
Comment: Not sure, there is not an option for that in the version we are using.  Could be there in a later version. However, you can have it go to a network share instead of e-mail. You could probably throw a windows service together to pick that up and shove to specific printer, assuming someone else has not already written something to do that.
Comment: Okay.  I've though about that.  With both these answers, it looks like I'm back to creating a mini-project to accomplish the task.
Answer: From a Web Application standpoint, in modern prowsers, I don't believe you can specify a printer, and you certainly cannot "auto-start" printing.  Reporting services used to have the ability to print unattended with an MS browser plugin, but that was removed a number of years ago as it's now considered a security risk.
The best you can do is "one-click" print, that is programmatically render the report using the Report Execution service from the ReportService.asmx web service and initiate a programmatic solution to print the rendered report.
Unfortunately, printing an SSRS report automatically is quite involved. You can read about How to Print Reports Programmatically here.
Comment: Interesting... I think I saw the link but couldn't believe there wasn't a simpler way.  Thanks, though.
Comment: well that blog is from Sep 2011 - there may well be a better, more up-to-date, approach, but honestly I haven't needed to research it so... :)


How do I run an asynchronous 'find' in a loop while incrementing the find parameter so I can generate unique custom id's?

Question: I'm new to mongoose/mongodb and I am trying to do some sort of error handling with my document save.
I am trying to create a stub id to store into the db for easier data retrieval later on (and also to put into the url bar so people can send links to my website to that particular page more easily -- like jsfiddle or codepen).
Basically I want to search for a document with a page_id and if it exists, I want to regenerate that page_id and search until it gets to one that's unused like this:
while(!done){
    Model.findOne({'page_id': some_hex}, function (err, doc) {
        if(doc){
            some_hex = generate_hex();
        }
        else
        {
            done = true;
        }
    });
}
model.page_id = some_hex;
model.save();

However, since mongoose is asynchronous, the while loop will pretty much run indefinitely while the find works in the background until it finds something. This will kill the resources on the server.
I'm looking for an efficient way to retry save() when it fails (with a change to page_id). Or to try and find an unused page_id. I have page_id marked as unique:true in my schema.
Comment: Can't you generate an id that's pretty much guaranteed to be unique to begin with? [`unik`](https://github.com/ricardobeat/unik), [`flocon`](https://github.com/Yosee/flocon), [`ObjectId`](http://mongoosejs.com/docs/api.html#types_objectid_ObjectId)
Comment: The reason for the stub is so people can grab that particular page if they plug it into the URL. It would be hard to remember the ObjectId() string.
Answer: Retrying should be performed asynchronously:
var tryToSave = function(doc, callback) {
  var instance     = new Model(doc);
  instance.page_id = generate_hex();
  instance.save(function(err) {
    if (err)
      if (err.code === 11000) { // 'duplicate key error'
        // retry
        return tryToSave(doc, callback);
      } else {
        // another error
        return callback(err);
      }
    }
    // it worked!
    callback(null, instance);
  });
};

// And somewhere else:
tryToSave(doc, function(err, instance) {
  if (err) ...; // handle errors
  ...
});

Comment: Thanks, this worked. The key was to do recursion asynchronously using the 'return tryToSave()' on error statement.


see if a 2d collider is between 2 points

Question: i have a fruit ninja sort of game where your mouse is a blade and you can cut items like fruit. So the fruit has a collider and so does the blade.
Problem is when you cut fast, the collider might move too fast to collide with the fruit.
To overcome this i need a way to see if a collider is between two points.
I can do this to get the 2 points;
vector 3 lastPos = Vector3.zero;

void Update()
{
    /*If collider between Input.mouse.position & lastPos {
        collided();
    }*/
    lastPos = Input.mouse.position;
}

void collided()
{
   ...
}

Is this possible to do and what would be the best way?
Answer: Within blade:
private Vector3 bladeLastPos;

private void Start(){
    bladeLastPos = new Vector3(blade.position.x,blade.position.y,blade.position.z);
}

private void Update()
{
    Vector2 v1 = new Vector2(this.transform.position.x, this.transform.position.y);
    Vector2 v2 = new Vector2(bladeLastPos.x, bladeLastPos.y);
    float maxRange = Vector2.Distance(v1,v2);

    RaycastHit2D[] hits = Physics2D.RaycastAll(v1, v2 - v1, maxRange);

    for (int i = 0; i < hits.Length; i++)
    {
        var component = hits[i].transform.GetComponent


How to implement a group & role based permission system?

Question: I am currently building an application in PHP using the Symphony 2 library, but I guess this question is applicable to any sort of web application. Here is the basic infrastructure I would like to implement:

every user is part of one or multiple groups
every group implements one or more roles
roles implemented by a group apply to all users in that group
a user can implement additional roles not in its group

An example

a group 'writers' implement the 'writer' role and the 'comment moderator' role
an group 'administrators' implements the 'admin' role
a user 'Henry' is part of the writer group, and the administrator group
a user 'Henry' implements the 'owner' role

The roles affective for that user would be 'writer', 'comment moderator', 'administrator' and 'owner'.
Edit
Is it a good practice or not to have this behavior : user can inherit roles from its own group and can have individual roles as well.
And if so, how to make it real ?
I thought of 5 tables :
Users :

id
name

Role :

id
name

UserRole :

id_user (FK)
id_role (FK)

Group :

id
name

GroupRole :

id_group (FK)
id_role (FK)

UserGroup :

id_user (FK)
id_group (FK)

This could work and the main problem would be to prevent adding an individual role a user already has from the groups it belongs to.
Be it seems to be a little complicated. Is there any better way to do so ?
Thanks
Comment: One of the most common forum systems (phpBB) does exactly that, and it's perfectly happy.  The main thing to remember is to ensure you craft your queries and tables (indexes) right to keep the speed optimal.
Comment: So... what is the question?  It seems like you have it all planned out in your mind, so what part of it do you want our help with?
Comment: Edited my own post ;)
Comment: You are basically describing a fairly normal system there, and have worked out the normal tables you would have.  Is there a problem with a user having a role that is in their group as well? Does that really make a difference? They have the role, it doesn't matter where they get the role from, or if they have the role twice - if they have it they have it.
Comment: Okay so it is not a chocking to implement it this way ? Is it how people do ?
Comment: If John gives me a chocolate bar I can eat chocolate.  If both John and Arthur give me chocolate bars I can eat chocolate.  So I can eat more chocolate with the latter scenario - but that just makes me happier (and fatter), but it doesn't stop me being able to each chocolate...
Answer: What you are describing is a fairly normal and common permissions system.  It is a system used throughout the world in many different forms.  In fact, you are most probably using it in some similar form by just using this website.  Certainly, if you ever used a forum using phpBB you will have used it.
The tables you have described are pretty much standard.  One user, many groups, a table to link them.  One user, many roles, a table to link them.  One group, many roles, a table to link them.  It's all pretty standard stuff.
The main caveat, with a more heavily loaded system, is to ensure that your tables and queries are optimized in such a way as to keep server load down.  That means making sure you have indexes on the right columns, etc.  The MySQL command explain can really help you to check you have it right.
There are other similar ways of achieving the same results using less tables, which in some situations may be more optimal, or may be less optimal, it depends entirely on your system.  The most common way is to basically compress the joining tables into a single field in the "single" end table, so the user would have a field that has a list of the roles in it, and another that has a list of the groups in it.  Similarly with the groups - a field that has a list of the roles in it.  How you encode that information is up to you.  I commonly use JSON to encode array information into table fields as it is self-escaping.  All it does though is shift the processing from the SQL server to the interface script.
And does it matter if a user has the same role both individually granted and inherited from a group?  It shouldn't.  If a role consists of a flag that says the user is (or is not) allowed to perform a specific function, and they get that flag twice, do they have more of the flag?  Can a flag be off, on, and more on?  Not normally, no.  You'd have to go out of your way to program a system that would act like that.
Of more interest is what happens when you have two roles that conflict - one that says they can do something and one that says they can't do it.  Which do you pick - the one that lets them do it or the one that stops them doing it?  That is entirely your choice, of course.  Incidentally, phpBB has 3 states for a permission - YES, NO and NEVER.  YES can override NO, but can't override NEVER.
Comment: You couldn't have wrote something easier for me to understand ! Thank you very much @Majenko. I'll do it the way I wrote it in my question post.
Answer: You are talking about RBAC. It could by implemented for example via PHP-RBAC. For Symphony i found UserRbac.
Comment: I know this discussion is a bit old but in case someone looks for an maintained rbac library, I want suggest my own: https://github.com/doganoo/simple-rbac


How to do a public site with sharePoint on premise

Question: I have to build a faccing public site with his own visual identity with sharepoint 'on premise', have someone know some tutorial or online class about how to do this. Because didn't find anything clear about how to do this with sharepoint in my researches
Comment: Narrow your question, do you want to make your site anonymous?  how to customize your sharepoint look and feel? Seems like you don't have a clue about Sharepoint in the broad sense, google for tutorials first (http://andreasglaser.com/learn-sharepoint)
Comment: Thanks for the comment man, I'll look at the link.
Answer: Although I am not an SharePoint expert, I believe what you are asking is a loaded question that encompasses several topics of knowledge and technical hurdles you would have to learn or overcome. There are a lot of details missing from your request, so I can only offer you some topics you will need to be familiar with as I have not seen any single tutorial that shows what you are asking for. But you should be able to find tutorials on individual topics that would be relevant to your request:
Public Facing Site

Opening your on-premise SharePoint server to the public can be done by changing network settings which is a straight forward task for any network admin.
Best to use a domain (ex mysite.com). If you don't have one, you can purchase a domain. Then have it registered and configured to connect with your server(s),
Some other things to consider is security, firewalls, load balancing, DMZ zones and business policies.

Own Visual Identity (a custom look of SharePoint)

One method is to create a custom master page for SharePoint which can be done via Visual Studio or SharePoint Designer.  This requires knowledge on web development such as CSS, HTML and possibly javascript. Developing a on-premise SharePoint environment with Visual Studio requires a lot (i mean alot) of setup and work.
Another method is to use the SharePoint theming engine to create custom themes.
You can also buy pre-built themes from other sources.

Final thoughts

Unless absolutely required, I would avoid using a on-premise SharePoint to host a public site as there are much more simpler and cost effective solutions out there. There is also SharePoint online which will probably provide most of what you need out of the box.  Need to ask yourself what this public facing site will do and if SharePoint is the right product to use as I consider SharePoint complex and suited for specific needs. I worked with a on-premise SharePoint solution and it takes a crew of IT professionals and tons of money for maintenance and development.


Getting the calling function in a custom ExcelDNA exception handler without MacroType

Question: I'm aware that there is the method shown in the ErrorHAndler.dna example:
private object ErrorHandler(object exceptionObject)
{
    ExcelReference caller = (ExcelReference)XlCall.Excel(XlCall.xlfCaller);

    // Calling reftext here requires all functions to be marked IsMacroType=true, which is undesirable.
    // A better plan would be to build the reference text oneself, using the RowFirst / ColumnFirst info
    // Not sure where to find the SheetName then....
    string callingName = (string)XlCall.Excel(XlCall.xlfReftext, caller, true);

    [...]

}

But has anyone determined how the issue described in the comment might be resolved?  I.e. is there a way to do this without setting IsMacroType=true?
Answer: You don't need IsMacroType=true to get the caller (the first line in your sample works fine without it). As the comment in your sample indicates, if you want to build a text description for the range, you need to build it yourself from the Row/Column info in the ExcelReference. You can get the sheet name with a call to XlCall.Excel(XlCall.xlSheetNm, caller).


Neural network for predicting a non-linear function

Question: I want to build a neural network to predict $f(x)=\exp(-x)$ for every $x$ in the interval $(0, 5)$. I randomly generated the training set uniformly in the given interval with 500K training examples, then we generate the target exactly with $f$.  Which model would be best suited for this simple task? I understand this is not generally the best use case for Neural Nets. I have tried a bunch of networks and they do alright. I want to get extremely precise accuracy. Any tips or words of advice?
Comment: What is "extremely precise accuracy"? How do you measure it? What numerical quantity of "accuracy" would satisfy you? What network architecture have you tried and what level of "accuracy" did you achieve? Are you familiar with the universal approximation theorem? https://en.wikipedia.org/wiki/Universal_approximation_theorem
Answer: import keras as K
import numpy as np
DATA_SIZE = 500000
def gen_data():
    x_ = 5 * np.random.rand(DATA_SIZE)
    y_ = np.exp(-x_)
    return x_, y_

x, y = gen_data()
x_test, y_test = gen_data()
model = K.Sequential([
    K.layers.Dense(8, input_shape=(1,)),
    K.layers.Activation("elu"),
    K.layers.Dense(1),
])

sgd = K.optimizers.sgd(0.01, momentum=0.9)
model.compile(optimizer=sgd, loss="mse")

model.fit(x=x,y=y,epochs=1, batch_size=32,validation_data=(x_test,y_test))

A 1-layer with 8 units network trained for 1 epoch achieves validation loss of val_loss: 6.3314e-06
Comment: elu has nice properties https://arxiv.org/abs/1511.07289 and because dying relu can be a problem https://stats.stackexchange.com/questions/379884/why-cant-a-single-relu-learn-a-relu/379957#379957 if your activation function exactly matches what you want to approximate, then you don't need an activation function or weights or biases or a neural network
Comment: @JohnDoe you misunderstood Sycorax's comment. You asked him: "Could the activation function be designed to match exp(-x)?". He told you: " your activation function exactly matches what you want to approximate, then you don't need an activation function or weights or biases or a neural network", i.e., if you design your activation function to match the function you want to approximate (exp(-x)), then you don't need a neural network in the first place! This has nothing to do with ELU or ReLU
Comment: Sycorax, re: `relu`, I think that the reason you get orders of magnitude better results using `elu` with respect to `relu`, has nothing to do with dying neurons, but it's just because `elu` has an exponential component. I plan to prove this with a few experiments including LeakyReLU (which don't suffer from the dying neurons issue).
Comment: @DeltaIV That would be an interesting experiment! If you ask & answer your own question, you'll probably garner a great many points. I think you're probably correct, since on $[-5, 0]$, the activation function is essentially the function we want to model, and it's easy to fit the appropriate weights' -- so the real effect is what I hinted at above: the activation matches what we seek to model.
Comment: Why Elu and not Relu? Could the activation function be designed to match exp(-x)?
Comment: @Sycroraw it's unclear how that applies here. How would a Relu function exactly match what I want to approximate?
Comment: I didn't say that it did; I thought I was addressing your second question. What is your question?
Comment: Could you just explain this statement: "if your activation function exactly matches what you want to approximate, then you don't need an activation function or weights or biases or a neural network?" I don't understand what you mean.
Comment: @DeltaIV I see. I thought the 2 comments were related.
Comment: "Many points" is probably a bit optimistic :-) but I do hope it will be well received. I will write it in the next days.
Comment: @Sycorax I have a bit more spare time now. Do you still think it would be a good idea to ask & answer such a question?
Comment: @DeltaIV I'd be interested to read your findings.


Regression and causality in econometrics

Question: In regression in general and in linear regression in particular, causal interpretation of parameters is sometimes permitted. At least in econometrics literature, but not only, when causal interpretation is permitted is not so clear. For a discussion, you can see Regression and Causation: A Critical Examination of Six Econometrics Textbooks - Chen and Pearl (2013).
For proper handling of causality in a statistical model the best way probably is to use Structural Causal Model as explained, for example (shortly), in Trygve Haavelmo and the Emergence of Causal Calculus – Pearl 2012 Feb.
However, currently, these are not the standard method in basic econometrics model (classic multiple linear regression). Indeed is frequently used the concept of "true model" or "data generating process" that sometimes has explicit causal meaning. In any case, I want to consider only causal sense. Therefore if we estimate the sample counterpart of "true model" we achieve causal interpretation of parameters.
Keeping in mind the above consideration, my attempt is to grasp

the link between the concept of "true model" (of current econometrics textbooks) and structural causal model (of Pearl) … if any.

The link between the previous point and the concept of  randomized controlled experiment, as used in laboratory, that sometimes is the reference point in econometrics observational study (as good as it). For example Stock and Watson (2013) spend a lot of discussion about that (particularly cap 13). Moreover, in Pearl 2012feb page 14, there is a debate review between "structuralists" and "experimentalists" that is strongly related to this point.

Can you explain to me something about these two points in the simplest possible scenario?
Comment: There are multiple schools of thought about causality in econometrics. See e.g., Heckman and Pinto (2015) commenting on Pearl, or the applications of the Rubin potential outcomes model (for example in Mostly Harmless Econometrics by Angrist and Pischke).
Comment: @Frank: thank you for the paper suggested, they will be surely useful. Regarding the Angrist and Pischke book, I read it. I surveyed about ten econometrics textbooks and Mostly Harmless Econometrics represent the best about causality (my opinion but probably not only). I learned a lot from that. However, at least in my opinion, even this book is exhaustive. For example it says nothing about structural causal model and link between it and potential outcome language. Finally it does not offer an answer a my two question above. Or, at least, I "don't see" it.
Comment: @Aksakal: answer like your says both, everything or nothing. In my opinion says nothing because it don't says me nothing about: firstly, why in many econometric books "true model" is used (and seems very useful if not essential); what are the links among your answer and my questions? If links do not exist the presence of "true model" became very hard to explain. In any case I known that "true model" is sometimes criticized concept and that "real world" is something different and links between them can be artificial, but this is the theory. My question is about econometric theory.
Comment: there are no "true" models in economtrics
Comment: @markowitz: Angrist and Pischke do discuss this, but their language might be confusing you. See section 3.2.1: "the function $f_i(s)$ tells us what $i$ would earn for _any_ value of schooling, $s$. In other words, $f_i(s)$ answers causal 'what if' questions." See also the discussion of the linear constant effects causal model (at the end of the section).
Comment: @markowitz, it was a comment, not an answer. The subject matter is too unstable to have a "true" model in sense that we have models in natural sciences. Instead, economic "models" are "stories" and vignettes. That's the reason why economists love "stories", and always talk about them when you discuss the papers. This still just a comment, not an answer.
Comment: @markowitz, my views on econometric models is similar to Rubinstein's description of microeconomics, see the first part of the Introduction in his lecture notes here http://assets.press.princeton.edu/rubinstein/front.pdf See also his review of the book http://arielrubinstein.tau.ac.il/papers/100.pdf , He says this about the models: "A story is not meant to be "useful" in
the sense that most people use the word...
During my forty years in the profession, I
have not encountered a single model that
could persuade me that economic theory
might have direct practical use."
Comment: @Aksakal, you are wrong. Have you ever opened an econometrics textbook? Start with the classics such as Greene "Econometric Analysis", Kennedy "A Guide to Econometrics" or Wooldridge "Introductory Econometrics" - all of them include the notion of the "true model".
Comment: @RichardHardy, I know they talk about them "true model", but it's delusional. None of their models are true in conventional sense.
Comment: @Aksakal, who defines the conventional sense? Or where can such a definition be found? My first bet would be on a classical textbook in the field. I presume you are not an econometrician by education and what seems conventional to you might be specific to you alone, or perhaps to your field. It is the same as if I was going to define what is conventional in, say, chemistry (I am not educated as a chemist but an econometrician). Even using the word "delusional" in this context is not only ignorant but also inappropriate, IMHO.
Answer: I will start with the second part of your question, which pertains to the difference between randomized control studies and observational studies, and will wrap it up with the part of your question pertaining to "true model" vs. "structural causal model".
I will use one of Pearl's examples, which is an easy one to grasp. You notice that when the ice cream sales are highest (in the summer), the crime rate is highest (in the summer), and when the ice cream sales are lowest (in the winter), the crime rate is lowest. This makes you wonder whether the level of ice cream sales is CAUSING the level of crime.
If you could perform a randomized control experiment, you would take many days, suppose 100 days, and on each of these days randomly assign the level of sales of ice cream. Key to this randomization, given the causal structure depicted in the graph below, is that the assignment of the level of ice cream sales is independent of the level of temperature. If such a hypothetical experiment could be performed you should find that on the days when the sales were randomly assigned to be high, the average crime rate is not statistically different than on days when the sales were assigned to be low. If you had your hands on such data, you'd be all set. Most of us, however, have to work with observational data, where randomization did not do the magic it did in the above example. Crucially, in observational data, we do not know whether the level of Ice Cream Sales was determined independently of Temperature or whether it depends on temperature. As a result, we'd have to somehow untangle the causal effect from the merely correlative.
Pearl's claim is that statistics doesn't have a way of representing E[Y|We Set X to equal a particular value], as opposed to E[Y|Conditioning on the values of X as given by the joint distribution of X and Y]. This is why he uses the notation E[Y|do(X=x)] to refer to the expectation of Y, when we intervene on X and set its value equal to x, as opposed to E[Y|X=x], which refers to conditioning on the value of X, and taking it as given.
What exactly does it mean to intervene on variable X or to set X equal to a particular value? And how is it different than conditioning on the value of X?
Intervention is best explained with the graph below, in which Temperature has a causal effect on both Ice Cream Sales and Crime Rate, and Ice Cream Sales  has a causal effect on Crime Rate, and the U variables stand for unmeasured factors that affect the variables but we do not care to model these factors. Our interest is in the causal effect of Ice Cream Sales on Crime Rate and suppose that our causal depiction is accurate and complete. See the graph below.

Now suppose that we could set the level of ice cream sales very high and observe whether that would be translated into higher crime rates. To do so we would intervene on Ice Cream Sales, meaning that we do not allow Ice Cream Sales to naturally respond to Temperature, in fact this amounts to us performing what Pearl calls "surgery" on the graph by removing all the edges directed into that variable. In our case, since we're intervening on Ice Cream Sales, we would remove the edge from Temperature to Ice Cream sales, as depicted below. We set the level of Ice Cream Sales to whatever we want, rather than allow it be determined by Temperature. Then imagine that we performed two such experiments, one in which we intervened and set the level of ice cream sales very high and one in which we intervened and set the level of ice cream sales very low, and then observe how Crime Rate responds in each case. Then we'll start to get a sense of whether there is a causal effect between Ice Cream Sales and Crime Rate or not.

Pearl distinguished between intervention and conditioning. Conditioning here refers merely to a filtering of a dataset. Think of conditioning on Temperature as looking in our observational dataset only at cases when the Temperature was the same. Conditioning does not always give us the causal effect we're looking for (it doesn't give us the causal effect most of the time). It happens that conditioning would give us the causal effect in the simplistic picture drawn above, but we can easily modify the graph to illustrate an example when conditioning on Temperature would not give us the causal effect, whereas intervening on Ice Cream Sales would. Imagine that there is another variable which causes Ice Cream Sales, call it Variable X. In the graph is would be represented with an arrow into Ice Cream Sales. In that case, conditioning on Temperature would not give us the causal effect of Ice Cream Sales on Crime Rate because it would leave untouched the path: Variable X -> Ice Cream Sales -> Crime Rate. In contrast, intervening on Ice Cream Sales would, by definition, mean that we remove all arrows into Ice Cream, and that would give us the causal effect of Ice Cream Sales on Crime Rate.
I will just mention that one Pearl's greatest contributions, in my opinion, is the concept of colliders and how conditioning on colliders will cause independent variables to be likely dependent.
Pearl would call a model with causal coefficients (direct effect) as given by E[Y|do(X=x)] the structural causal model. And regressions in which the coefficients are given by E[Y|X] is what he says authors mistakenly call "true model", mistakenly that is, when they are looking to estimate the causal effect of X on Y and not merely to forecast Y.
So, what's the link between the structural models and what we can do empirically? Suppose you wanted to understand the causal effect of variable A on variable B. Pearl suggests 2 ways to do so: Backdoor criterion and Front-door criterion. I will expand on the former.
Backdoor Criterion: First, you need to correctly map out all the causes of each variable and using the Backdoor criterion identify the set of variables that you'd need to condition on (and just as importantly the set of variables you need to make sure you do not condition on - i.e. colliders) in order to isolate the causal effect of A on B. As Pearl points out, this is testable. You can test whether or not you've correctly mapped out the causal model. In practice, this is easier said than done and in my opinion the biggest challenge with Pearl's Backdoor criterion. Second, run the regression, as usual. Now you know what to condition on. The coefficients you'll get would be the direct effects, as mapped out in your causal map. Note that this approach is fundamentally different from the traditional approach used in estimating causality in econometrics - Instrumental Variable regressions.
Comment: "regressions in which the coefficients are given by $E[Y|X]$ is what he says authors mistakenly call "true model", mistakenly that is, when they are looking to estimate the causal effect of $X$ on $Y$ and not merely to forecast $Y$." This seems me not strictly true because $E[Y|X]$ is not always the true model. Any OLS regression is a conditional expectation but not every of these are counterpart of the true model frequently used in econometric textbooks.
Comment: I understand your explanation but I fear that it does not give an answer about my question. Except, maybe, the final phrase but I partially disagree with it.
You firstly explain the usefulness of an experiment. I agree with you about that. After you said that (shortly) in observational context we have to "somehow untangle the causal effect from the merely correlative" without experimental manipulations. I known.
Comment: After, since the conceptual difference between conditioning and intervention, you introduce Pearl interventional notation's. I understand the usefulness of interventional notation. I'm not confident with causal graph but surely it is useful too. Pearl's contributions are important.
Finally you said that: "Pearl would call a model with causal coefficients (direct effect) as given by $E[Y|do(X=x)]$ the structural causal model.". Ok, I understand.
Comment: Let me use your/Pearl example: $X1$= temperature, $X2$= ice cream quantity, $Y$= crime rate. It's possible to define the true (causal) model as: $Y = beta_0 + beta_1 *X1 + beta_2 * X2 + u$ but we estimate the underspecified model $Y = alfa_0 + alfa_1 *X1 + e$. It's possible demonstrate that alfas parameters are biased estimates of the true causal parameters (betas); however the latter remain a conditional expectation. This is omitted variables problem. Indeed if we estimate the correct specified form, or a "longer", the estimates parm. are unbiased and its causal interpretation is permitted.
Comment: The explanation, at least in simple case like above, can be complete also without Pearl's (and related) contributions.
Exactly for reasons like the above I looking for the relations among: Structural Causal Model (from Pearl), experimental language (from Rubin); usual true model used in its causal sense (from many econometrics textbooks). I'm convinced that links exist but I'm doubt about the form of them.
Comment: This answer is a good exposition of the Pearl framework, but it doesn't shed any light on OP's question: what's the relationship to the methods used in econometrics?
Comment: Good feedback. Thank you. I added 2 paragraphs to the end to address the link to econometrics practice and pointed out that it is different than Instrumental Variables. Maybe somebody else can directly attack the difference between Pearl's Backdoor Criterion and Instrumental Variables, as I believe this is the implicit heart of this not-so-surgically stated question.
Comment: Thanks @ColorStatistics. To your recent addition: econometrics sometimes uses selection on observables (Conditional Independence Assumption), which appears to be similar to (but less rigorous than) the backdoor criterion.
Comment: Do I understand correctly that "Variable X -> Ice Cream Sales -> Crime Rate" is a problem only when Variable X affects also Crime Rate through a path that does not go via Ice Cream Sales, just like it is with Temperature? Otherwise, controlling for Temperature would still be enough.
Answer: In the context of the Pearl paper you've given, what most econometricians would call a true model is input I-1 to the Structural Causal Model: a set of assumptions $A$ and a model $M_A$ that encodes these assumptions, written as a system of structural equations (as in Models 1 and 2) and a list of statistical assumptions relating the variables. In general, the true model need not be recursive, so the corresponding graph can have cycles.
What's an example of a true model? Consider the relationship between schooling and earnings, described in Angrist and Pischke (2009), section 3.2. For individual $i$, what econometricians would call the true model is an assumed function mapping any level of schooling $s$ to an outcome $y_{si}$:
$$
y_{si} = f_i(s).
$$
This is exactly the potential outcome. One could go further and assume a parametric functional form for $f_i(s)$. For example, the linear constant effects causal model:
$$
f_i(s) = \alpha + \rho s + \eta_i.
$$
Here, $\alpha$ and $\rho$ are unobserved parameters. By writing it this way, we assume that $\eta_i$ does not depend on $s$. In Pearl's language, this tells us what happens to expected earnings if we fix an individual's schooling at $s_i = s_0$, but we don't observe $\eta_i$:
$$
E[y_{si} \mid do(s_i = s_0)] = E[f_i(s_0)] = \alpha + \rho s_0 + E[\eta_i].
$$
We haven't said what queries we're interested in, or what data we have. So the "true model" is not a full SCM. (This is generally true, not just in this example.)
What's the connection between a true model and a randomized experiment? Suppose an econometrician wants to estimate $\rho$. Just observing $(s_i, y_i)$ for a bunch of individuals isn't sufficient. This is identical to Pearl's point about statistical conditioning. Here
$$
E[y_{si} \mid s_i = s_0] = E[f_i(s_0) \mid s_i = s_0] = \alpha + \rho s_0 + E[\eta_i \mid s_i = s_0].
$$
As Angrist and Pischke point out, $\eta_i$ may be correlated with $s_i$ in observational data, due to selection bias: an individual's decision about schooling might depend on her value of $\eta_i$.
Randomized experiments are one way to correct for this correlation. Using Pearl's notation loosely here, if we randomly assign our subjects to $do(s_i = s_0)$ and $do(s_i = s_1)$ then we can estimate $E[y_{si} \mid do(s_i = s_1)]$ and $E[y_{si} \mid do(s_i = s_0)]$. Then $\rho$ is given by:
$$
E[y_{si} \mid do(s_i = s_1)] - E[y_{si} \mid do(s_i = s_0)] = \rho(s_1 - s_0).
$$
With additional assumptions and data, there are other ways to correct for the correlation. A randomized experiment is only considered the "best" because we may not believe the other assumptions. For example, with the Conditional Independence Assumption and additional data, we could estimate $\rho$ by OLS; or we could bring in instrumental variables.
Edit 2 (CIA): This is mainly a philosophical point, and Angrist and Pischke may disagree with my presentation here. The Conditional Independence Assumption (selection on observables) lets us correct for selection bias. It adds an assumption about joint distributions: that
$$
f_i(s) \perp\!\!\!\perp s_i \mid X_i
$$
for all $s$. Using just conditional expectation algebra (see the derivation in Angrist and Pischke) it follows that we can write
$$
y_i = f_i(s_i) = \alpha + \rho s_i + X_i' \gamma + v_i
$$
with $E[v_i \mid X_i, s_i] = 0$. This equation allows us to estimate $\rho$ in the data using OLS.
Neither randomization nor the CIA goes into the system of equations that defines the true model. They are statistical assumptions that give us ways to estimate parameters of a model we've already defined, using the data we have. Econometricians wouldn't typically consider the CIA part of the true model, but Pearl would include it in $A$.
Comment: @Frank you can give any partial specification of a structural causal model, the set of assumptions $A$ that Pearl refers to. Traditionally, this will be qualitative statements of the functional relationships that you can defend on scientific grounds. The CIA is a partial specification of an SCM as well, you are asserting that the potential response $Y_{s}$ does not depend on $S$ given a set of covariates, this imposes constraints on the possible systems of equations.
Comment: I already see the Angrist and Pischke example and I thought about it. That was the best example/explanation that I never found. Your addition represent what I was looking for. Thank you.
Comment: However let me add some points and specific questions. Angrist and Pischke call the model as "linear constat effect causal model" but my doubt was exactly if that model was interpretable as "true model" that I usually see in many econometrics textbooks (for simple example see my comment on ColorStatistics answer).  My favourite answer was no, because in the usual "true model" is explicitly valid some form of exogeneity that is indeed precisely violated in the present case. This is exactly your final consideration about my phrase but I was aware of this problem.
Comment: Now, Angrist and Pischke never speak about "true model" but you suggest this interpretation for it. Probably you are right. In any case you write: "So the "true model" is just one part of the SCM.". Ok! Your is a well posed form of one guess that I already had in mind. This conclusion hold even in the case of the "true model"  that I was remember before?
Comment: In the Angrist and Pischke's exposition the CIA plays a key role, and actually play a key role in causality. This is essential for eliminating selection bias. However in the example the causal assumptions explicitly preceded the CIA intervention. I think that if we take the general model $y = a + bx + cZ + e$, where $Z$ is a set of controls, the CIA condition (on $Z$) is never enough. Causal assumption must precede $y$,$x$ relationship shown in the equation. It's correct? I think yes otherwise the saying "no causal assumpions no causal conclusions" is violated.
Comment: Finally in the case of randomized controlled experiment, is correct to says that randomization (about sampling and treatment assignment) is enough in the last case because even if causality assumptions are not explicit are nested in randomization?
Comment: @markowitz I added a section on the CIA. Does this address your question?
Comment: Frank, there are some imprecise statements here. CIA is a causal assumption (constraints on the causal model is a causal assumption, and CIA imposes constraints on the potential outcome---a causal quantity). Also, it's $E[Y|do(s)] = E[Y_{s}]$, when you $do(s)$ that's already the potential outcome had $s$ been $s$.
Comment: So this sentence is mathematically incorrect: "Neither randomization nor the CIA adds any causal content – nothing there goes into the system of equations that defines the true model. They are only statistical relationships that give us ways to estimate parameters of a model we've already defined, using the data we have."
Comment: As to SCMs, it seems you are conflating partial specification with the definition of an SCM. In your model, you don't need to give all inputs of an SCM, you give the assumptions you can defend. That is, usually your give only a partial specification of the model. You can give a partial specification of an SCM with a DAG, for instance. Or, in your example, with the CIA, $Y_{s} \perp S$, which defines a class of SCMs.
Comment: @CarlosCinelli I've removed the statement that the CIA isn't causal, but the CIA is entirely separate from the statement that $E[Y \mid do(s)] = E[Y_s]$ (that came much earlier). I don't get your second point – the system of equations (as given here) is one of the inputs of an SCM, isn't it?
Comment: @Frank no, the SCM (just like Rubin's "truth table", a table with all potential outcomes filled in) is the complete causal model. If you have that, then you don't need to estimate anything. The inputs in practice are partial specifications of the causal model. The CIA, for instance, is one such partial specification (you are stating that potential responses do not depend on treatment assignment, given a set of covariates). A DAG, for instance, is another such partial specification (you are stating qualitative functional relationships among variables).
Comment: 1) I think you misunderstand the CIA, see above (I see you fixed this)
2) I'm having trouble connecting your definition of the SCM with the definition given in "Trygve Haavelmo and the Emergence of Causal Calculus"
Comment: @markowitz I don't understand your objection -- Angrist and Pischke describe common practice in their book, and then criticize it in their article. It's still common practice.
Comment: @Frank: No Frank. Angrist and Pischke, in the 2017 article, criticize the "old style econometrics" that is still common practice, or at least is so in many currently econometrics books. In his 2009 book the same authors use the "new style econometric". In the old style econometric the "true model" is a key concept and in it control variables is absent or no important. In the new style econometric the "true model", at least in the usual sense understood, is absent and the controls variables is very important.
Comment: Nonetheless, the exposition in AP 2009, section 3.2, is faithful to old style econometric practice, and what it calls the "true model" is consistent with old style econometrics.
Comment: No Frank, I'm sorry but it's not like that. AP 3.2 is exactly the point of the book where the distinction in argument become more clear. AP 3.2 is completely inside new style econometrics. In this section the "true model" is never mention, is mention "linear causal model" that is something else. In this section the difference between causal variable (of interest) and controls variables is crucial and clear. CIA is used primarily for this distinction.
Comment: In the "old style" (past and also present books) so called "true model", always with exogenous error, is used. Distinction between causal and controls variables is absent or not clear and, in fact, CIA is not used.
Comment: Apparently comments are not for extended discussion; we can take this up in chat. I'll just point you to AP 2009 pages 58-59, the 1981 paper cited there, and every "old style" econometrics paper that uses instrumental variables (of which there are many).
Comment: Frank, you're right, the comments are not for extended discussions. Let me take up my point again. I appreciate a lot your answer, this is what I looked for. The "objection" the I made is primarily terminological. The sentence " Therefore if we estimate the sample counterpart of "true model" we achieve causal interpretation about parameters." seems me not wrong, rather it represent one "mantra" of old style econometrics. In my opinion for benefit of other readers is better to delete or modify your Edit. That's it.
Comment: Others argument as Matching and/or IV estimators are related but different question that is better do not insert here.
Comment: @Frank: about the definitional question of "true model" and particularly to your first Edit e my comment let me add that the model used in Angrsit and Pischke (2009), essentially the above example, is not the usual "true model". In fact they speak about "linear causal model". This position is clarified in the article Undergraduate Econometrics Instruction: Through Our Classes, Darkly; Angrist and Pischke (2017).
Comment: The authors do not have a great opinion about the so called "true model" and in fact affirm: "The pedagogical emphasis on statistical efficiency and functional form, along with the sophomoric narrative that sets students off in search of "true models" as defined by a seemingly precise statistical fit, is ready for retirement" pag 126. The critics is exactly about the search of the true model that is the usual (old) econometrics goal. In the old sense, what I had in mind, my sentence that you underscore is correct.
Answer: The use of 'causal' in such regression/correlation based approaches is misleading, in my opinion. Path analysis, structural equation modeling, Granger causality, etc attempts to license causal inferences by imposing some fairly tenuous assumptions. In the case of Structural equation modeling for instance, the paths are directional and A appears to 'cause' B, but this simply means that the model as structured is 'plausible' in that it reproduces an observed covariance matrix (in fact, the direction of the paths don't even matter much - just the constraints).
Comment: You write: "In the case of Structural equation modeling for instance, the paths are directional and A appears to 'cause' B, but this simply means that the model as structured is 'plausible' in that it reproduces an observed covariance matrix" I disagree. Structural equation do not have the role of "reproduces" any statistical quantity. Moreover this question is quite old. Main points are no more obscure for me. I summarized main points here: https://stats.stackexchange.com/questions/493211/under-which-assumptions-a-regression-can-be-interpreted-causally/493905#493905


Some drupal database operations won't complete

Question: On a development server (and it's sister staging server), I'm not sure how I got myself into this situation. When running drush updatedb I get this error below.
$ drush updatedb
The following updates are pending:

redirect module :
  7101 -   Add status field.
  7102 -   Disable redirects that could cause infinite loops.

webform module :
  7322 -   Add index on {webform_submitted_data}.data.

Do you wish to run all pending updates? (y/n): y
Cannot add field redirect.status: field already exists.                                       [error]
Performed update: redirect_update_7101                                                        [ok]
Cannot add index data to table webform_submitted_data: index already exists.                  [error]
Performed update: webform_update_7322                                                         [ok]
Finished performing updates.

I occasionally dump the production server database onto dev and stage to have an updated copy.
Is there a way I can push through this?
Answer: Is everything working fine on the production server?  If so, try erasing the development database, and then copy the production database over it again.  Drush sql-sync does not delete the target database by default.
drush @dev sql-create
drush sql-sync @prod @dev


jmeter finding max sustainable throughput

Question: I would like to use Jmeter to keep sending requests at the maximum sustainable interval - where the number of running threads (displayed before the / sign in the top right corner) stay constant e.g. 2. Currently I need to manually experiment by changing the number of threads during a large ramp up time. I need to do this after each change on the server. Is there an automated way to find this max. throughput?
Answer: It's not very clear what do you mean by "unanswered requests".
There is Auto Stop Listener which can stop JMeter test if response time exceeds some pre-defined value or latency is bigger than expected or error rate is more than pre-defined threshold.
When the test is stopped by the listener a special environment variable: auto_stopped is being created with the value of true so you can use it for checking whether test has been stopped in script for setting the exit status code.
Auto Stop listener can be installed using JMeter Plugins Manager.
Comment: I have changed "unanswered requests" to "running threads". I hope it's clearer now.  Auto Stop Listener does not seem to be a good choice since it can't watch the number or running threads and simply stops at the criteria instead of reducing throughput.


What inferential method produces the empirical CDF?

Question: The empirical cdf is an estimate of the cdf. What kind of estimation method (such as method of moments, MLE, ...) constructs the empirical cdf?
Is the empirical cdf a nonparametric estimate? Do nonparametric estimates have construction methods different from MOM, MLE, and others?
For example, the empirical pmf of a finite-valued discrete distribution is constructed by MLE. But that is a special case, and not the whole story. Moreover, it is not an empirical cdf but an empirical pmf.
Comment: You don't seem to realise that estimates can be produced by any method you like. I could have an estimator that said always guess 42; it would be a lousy estimator in all but very special circumstances, but it would be an estimator. In other words, the class of estimators is not restricted to good or sometimes good named methods such as moments or MLE. The method used for calculating CDF is often just empirical fractions; it doesn't have to be thought of as formal estimation.
Comment: @NickCox But it *would* answer the ultimate question about [life, the universe, and everything](https://en.wikipedia.org/wiki/Life,_the_Universe_and_Everything) quite nicely. ;)
Comment: @Alexis Glad someone caught the allusion.
Comment: @NickCox I remember the precise *moment* I first heard about those books in a radio interview with Douglas Adams I was listening to as an 11-year old in the car on 580 in the Bay Area. Worth catching, and they stand rereading well. :)
Comment: Not asking the definition of empirical cdf, but by what kind of estimation method is it constructed.
Comment: This Wikipedia article will be helpful.
Answer: In An Introduction to the Bootstrap, Efron and Tibshirani find it useful to characterize the empirical cumulative distribution function (ecdf) as the nonparametric maximum likelihood estimate of the "underlying population" $F$.

Given data $x_1, x_2, \ldots, x_n$, the likelihood function (by definition) is the product of the probabilities
$$L(F) = \prod_{i=1}^n {\Pr}_F(x_i).$$
E&T claim this is maximized by the ecdf.  Since they leave it as an exercise, let's work out the solution here. It's not completely trivial, because we have to account for the possibility of duplicates among the data.  Let's take care with the notation, then.  Let $x_1, \ldots, x_m$ be the distinct data values, with $x_i$ appearing $k_i \ge 1$ times in the dataset. (Thus, $x_{m+1}, \ldots, x_n$ are all duplicates of the first $m$ values.) The ecdf is the discrete distribution that assigns probability $k_i/n$ to $x_i$ for $1 \le i \le m$.
For any distribution $F$, the likelihood $L(F)$ has $k_i$ terms equal to $p_i = {\Pr}_F(x_i)$ for each $i$.  It therefore is completely determined by the vector $p=(p_1, p_2, \ldots, p_m)$ and can be computed as
$$L(F) = L(p) = \prod_{i=1}^m p_i^{k_i}.$$
Since the likelihood for the ecdf is nonzero, the maximum likelihood will be nonzero.  Therefore, for any distribution $\hat F$ that maximizes the likelihood, $p_i = {\Pr}_{\hat F}(x_i)$ must be nonzero for all the data.  The Axiom of Total Probability asserts the sum of the $p_i$ is at most $1$. This reduces the problem to a constrained optimization:
$$\text{Maximize } L(p) = \prod_{i=1}^m p_i^{k_i}$$
subject to
$$p_i \gt 0, i=1, 2, \ldots m;\quad \sum_{i=1}^m p_i \le 1.$$
This can be solved in many ways. Perhaps the most direct is to use a Lagrange multiplier $\lambda$ to optimize $\log L$, which produces the critical equations
$$\left(\frac{p_1}{k_1}, \frac{p_2}{k_2}, \ldots, \frac{p_m}{k_m}\right) = \lambda\left(1, 1, \ldots, 1\right)$$
with unique solution $$\hat p_i = \frac{k_i}{k_1+\cdots+k_m} = \frac{k_i}{n},$$
precisely the ecdf, QED.

Why is this point of view important?  Here are E&T:

As a result, [any] functional statistic $t(\hat F)$ is the nonparametric maximum likelihood estimate of the parameter $t(F)$.  In this sense, the nonparametric bootstrap carries out nonparametric maximum likelihood inference.

[Section 21.7, p. 310]
Some words of explanation: "as a result" follows from the (easily proven) fact that the MLE (maximum likelihood estimate) of any function of a parameter is that function of the MLE of the parameter.  A "functional statistic" (or "plug-in" statistic) is one that depends only on the distribution function.  As an example of this distinction, E&T point out that the usual unbiased variance estimator $s^2 = \sum (x_i-\bar x)^2/(n-1) $ is not a functional statistic because if you were to double all the data, the ecdf would not change, but the $s^2$ would be multiplied by $2(n-1)/(2n-1)$, which does change (albeit only slightly).  Functional statistics are crucial to understanding and analyzing the Bootstrap.

Reference
Bradley Efron and Robert J. Tibshirani, An Introduction to the Bootstrap.  Chapman & Hall, 1993.
Comment: And it turns out this is not just a fun exercise: with right censored data, the Kaplan Myer curves can be shown to be an MLE in a similar fashion, although they also have a MOM justification. But for interval censored data, maximizing the non-parametric likelihood is, as far as I know, the **only** justification for the basic CDF estimator.
Comment: @gondolier In what way does it make no sense?  A great many (very successful) applications of bootstrapping use models with continuous distributions.
Comment: It seems this only makes sense for discrete problems. For continuous distributions with a density, this makes no sense.
Comment: @whuber I'm a bit confused by this definition of the likelihood function. For any continuous distribution $F$, the likelihood would be $L(F) = 0$, right? (Since point mass is $0$ for any given point.)
Comment: @zxmkn That's correct.  It follows that the (usual) bootstrap always estimates a *discrete* $\hat F.$  In retrospect this is kind of obvious because no matter how large the dataset is, there are only finitely many possible bootstrap samples.
Answer: For a discrete random variable, the standard definition of the empirical cumulative distribution function (cdf) can be seen as a Method-of-Moments estimator. Consider the discrete random variable $X$ taking values $\{k_1 <k_2 <...\}$. Then its cdf is defined as
$$F_X(k_m) =\Pr(X\le k_m)= \sum_{i=1}^m\Pr(X=k_i)$$
We have that $\Pr(X=k_i) = E[I_{\{X=k_i\}}]$, where $I_{\{X=k_i\}}$ is the indicator function taking values $1$ if $X=k_i$, $0$ otherwise. Substituting we have
$$F_X(k_m) = \sum_{i=1}^mE[I_{\{X=k_i\}}]$$
If we have available a sample of size $n$, $\{x_j,\, j=1,...,n\}$, of realizations of $X$, the sample analogue of the RHS is
$$\sum_{i=1}^m\left(\frac 1n\sum_{j=1}^nI_{\{x_j=k_i\}}\right) = \frac 1n\sum_{j=1}^nI_{\{x_j\le k_m\}} = \hat F_X(k_m)$$
i.e it is the standard expression for the empirical cumulative distribution function. So, since it uses the sample analogue of expected values (which here are moments of the indicator functions which in turn are Bernoulli r.v.'s), it can be seen as a Method-of-Moments estimator.


Are event keys same on windows and mac?

Question: If I want to check if enter is pressed, I will do something like:
addEventListener('keypress', function (e) {
    if (e.key === 'Enter') alert('You pressed enter!');
});

So, will that same code work on a Mac, will it register mac's return as enter?
Comment: Yes they are same,Possible duplicate of [How to detect pressing Enter on keyboard using jQuery?](https://stackoverflow.com/questions/979662/how-to-detect-pressing-enter-on-keyboard-using-jquery)
Comment: 13 is equal for enter key, you can try `e.key === 13` which should return true on every where mac, windows etc
Comment: [javascript - Replacement for deprecated `keypress` DOM event - Stack Overflow](https://stackoverflow.com/questions/52882144/replacement-for-deprecated-keypress-dom-event)
Comment: As @AyazShah stated, use the code - this will be more reliable.
Comment: @Justcode This is not a jQuery question but a pure javascript question
Comment: @Justcode This is pure javascript, how did jquery come into this? And I still don't have a definite answer. So `e.key` will be equal to `Enter` on a mac?
Comment: @jukenduit I know this is javascript, the question is about the key code, instead of the enter you can use 13 that's it, its not about javascript or jquery its about keycode.
Comment: @Justcode No, the Q is about key not keycode. I can use e.keycode 13 but not e.key 13.
Comment: @jukenduit there are 2 questions included, both are answered in my comment. please check
Answer: As we can see in the MDN table the answer is "Yes", they are the same.

Are event keys same on windows and mac?


How to delete block on hook_uninstall?

Question: I have a custom module, mymodule, that will create custom blocks, block_test and block_test2 on mymodule_install().
When the module is uninstalled, I want to remove any blocks created by mymodule. How would I delete the block generated by a module being uninstalled?
drush command (local dev):
drush pmu mymodule -yv

mymodule.uninstall()
function mymodule_uninstall() {
  // Delete Blocks generated by module.

  /** @var \Drupal\Core\Entity\EntityStorageInterface $block_storage */
  $block_storage = \Drupal::service('entity_type.manager')->getStorage('block');
  /** @var \Drupal\block\Entity\Block[] $blocks */
  $blocks = $block_storage->loadMultiple();
  foreach ($blocks as $block) {
    // I am stuck here.
  }
}

Comment: What kind of blocks? content blocks? You could try using enforced config dependencies, see https://www.drupal.org/node/2404447
Comment: @Berdir the boxes are of `basic block` types that come with Core
Answer: Yes, then the best thing would be to follow the enforced config dependencies part above, if you create the content blocks with a fixed uuid then you can just put the block config entities in your config/install folder, with the config dependencies as explained there and it should be uninstalled automatically.
At least the config entities, if you want to delete the block_content entities too then load them by UUID with $block_content = \Drupal::service('entity.repository')->loadEntityByUuid('block_content', 'YOUR_UUID'); $block_content->delete()

How to delete block on hook_uninstall?


What harmonic devices approach the tonic's root from a semitone above?

Question: Usually, a dominant will have two voices approaching the tonic's root. Most traditionally, the leading melody comes from a semitone below, while the other (generally the bass) simply jumps between the roots:
X:1
L:1/4
M:
K:Am
%%score (T1 T2) (B1 B2)
V:T1           clef=treble
V:T2           clef=treble
V:B1  middle=d clef=bass
V:B2  middle=d clef=bass
%            End of header, start of tune body:
% 1
[V:T1] "V⁷"">↑"^g,    "i"a,
[V:T2]       e,    e,
[V:B1]       d,    c,
[V:B2]    ">↑"e,,    a,,

The other voice may also approach the root from less far away than a fourth and from above, like
X:1
L:1/4
M:
K:Am
%%score (T1 T2) (B1 B2)
V:T1           clef=treble
V:T2           clef=treble
V:B1  middle=d clef=bass
V:B2  middle=d clef=bass
%            End of header, start of tune body:
% 1
[V:T1] "♯vii°⁷"">(↓)"b,   "i"a,
[V:T2]    f,    e,
[V:B1]       d,    c,
[V:B2]    ">↑"^g,,    a,,

but this whole step down has rather less impressive leading character than a half-step, as here in the diminished seventh chord both the bass leading upwards and the alto leading down to the tonic's fifth.
The tonic's third can also be approached from a semitone above
X:1
L:1/4
M:
K:A
%%score (T1 T2) (B1 B2)
V:T1           clef=treble
V:T2           clef=treble
V:B1  middle=d clef=bass
V:B2  middle=d clef=bass
%            End of header, start of tune body:
% 1
[V:T1] "V⁷"g,    "I"a,
[V:T2]       e,    e,
[V:B1]     ">↓"d,    c,
[V:B2]      e,,    a,,

But how about the root – can this only be approached either from a semitone below or somewhere more distant? There are of course the augmented sixth chords, but these always seem to have secondary dominant function, and don't directly establish a tonic.
X:1
L:1/4
M:
K:Am
%%score (T1 T2) (B1 B2)
V:T1           clef=treble
V:T2           clef=treble
V:B1  middle=d clef=bass
V:B2  middle=d clef=bass
%            End of header, start of tune body:
% 1
[V:T1] "viIt⁺⁶"a, "V"^g,    "i"a,
[V:T2]         ^d,     e,    e,
[V:B1]       a,,       b,,    c,
[V:B2]    ">↓"f,,      e,,    a,,

Comment: Neapolitan sixth could easily go to the tonic, with the tonic commonly in 2nd inversion before hitting the dominant.
Comment: @thrig: interesting, the Neapolitan seems indeed to work without properly going to the dominant first. Could you elaborate how the voices should be lead in that scenario?
Comment: @thrig  When the Neapolitan goes to "tonic in 2nd inversion," that's not really a tonic chord, it's the dominant with a 6th and a 4th above it that needs to resolve to a 5th and a 3rd. Otherwise known as a "cadential 6/4." Most theorists just call it a dominant function chord, some occasionally call it pre- or subdominant, but it's never tonic function.
Comment: And old question, but are you still interested in this?  And are you looking for just chords that are based on the root or any chord that includes a resolution from the flat second scale degree to the tonic?  Also, are you interested in minor keys or just major?  I can think of quite a few examples in real music if you're interested still.
Comment: @Some_Guy I'm looking for any ways to harmonise a ♭Ⅱ-Ⅰ movement in a melody, such that the Ⅰ note is given a sense of finality.
Answer: The classic example of a root being approached from a semitone above is when using the "tritone substitution" or "b5 substitution" for a dominant-function chord.  In the example see how the F - B tritone is retained, giving Db7 its "dominant of C" quality,  and how the parallel 5ths don't hurt the ear at all in this type of chromatic harmony.  You're slipping a Db triad down to a C triad - revel in it!
The idea of b5 substitutions also opens a door to understanding how the Augmented 6th chord works.  Look at it as a b5 substitution for ii7.
Comment: @leftaroundabout I think if you establish a sense of C major, then shift to a bunch of I–ii–V7–I progressions and then suddenly replace the V with bII7 (the TT sub is usually written as a dominant 7th, I think Laurence wrote it as an +6 to emphasize the notes held in common between it and V7) it might work better. I–ii–bII7–I has a very convincing chromatic bass line and resolution of ^7 and ^4. Also, you might want to look into Phrygian and (to a lesser degree) Locrian for other contexts with a half-step downward resolution to the tonic.
Comment: Ah, yeah, the Italian augmented sixth. But I just can't feel the C after that D♭⁶⁺ as having tonic character – it always seems to lead on to F-minor. Does it perhaps need a Jazz context with extra notes to avoid that secondary dominant feel?
Comment: `s/Italian/German/`


Adacency matrix of a line digraph

Question: Let $G$ be directed graph with adjacency matrix $A$ and let us assume that $A$ is primitive, i.e. there exists $N\in\mathbb{N}$ such that $(A^N)_{i,j}>0$ for all $i,j$.
Let now $L(G)$ be the line digraph of $G$ and denote by $B$ its adjacency matrix.
(Two vertices representing directed edges from $u$ to $v$ and from $w$ to $x$ in $G$ are connected by an edge from $uv$ to $wx$ in the line digraph when $v = w$).
My guess is that the matrix $B$ is again primitive and this seems to fit with some examples I calculated. However, I am not sure how to prove it rigorously. Can anybody share some good ideas?
I have tried to look for ways of expressing $B$ in terms of $A$, but for digraph I have found no connection...
Thank you very much for your help!
Comment: Just to be sure I understand: the vertices of $L(G)$ are the edges of $G$ and $ee' \in L(G)$ if $e$ ends where $e'$ starts?
Comment: Funny, I just thought about that. My graphs in question are undirected and regular, but I think this could also work for you: There is an entry in $B$, if an edge $(a,b)$ is connected to an edge $(b,c)$, so I ended at $$B=\sum_a \sum_b \langle b|Aa\rangle \sum_{|c\rangle\in A|b\rangle} E_{a,b}\otimes E_{b,c}$$, where $E_{i,j}$ is a matrix with a $1$ at the $i$th row and $j$ column. What do you think?
Answer: I think you're looking for what is close to what is defined here as $W_1$:

We define the 0,1 edge matrix $W_1$ by orienting the $m$ edges of $G$ and labeling them as in formula (2.1). Then $W_1$ is the $2m×2m$ matrix with $ij$ entry $1$ if edge $e_i$ feeds into $e_j$ provided that $e_j\neq e^{−1}_i$, and $ij$ entry $0$ otherwise. By "$e_i$ feeds into $e_j$," we mean that the terminal vertex of edge $e_i$ is the same as the initial vertex of edge $e_j$.

Let $u,w,x$ be labels of vertices of $G$ and not caring about the no-backtracking thing for now, the following should give an $n^2$ dimensional representation of $B$:
$$
B=\sum_u \sum_w \langle w|Au\rangle \sum_{|x\rangle\in A|w\rangle} E_{u,w}\otimes E_{w,x}
$$
Setting $E_{u,w}\otimes E_{w,u}=0$ we can remove the backtrackers, to get a $n^2$ dimensional representation of $W_1$...

Adacency matrix of a line digraph