How to practise, ear piano playing after years of sightreading only?

Question: I am stuck in sheet music. I know theory, have average sight reading skill, know chords and their inversions, scales, accidentals, ect.
But if someone took the notes away I couldn't play even familiar pieces.
I'm 65 years old and I want to finally learn ear playing, finding chords, improvising...
How to practise? I learned from the internet that first, you find the root of a chord, then scales, then accidentals... How?
What are practical ways to proceed?
I'm slow to learn but motivated to spend rest of the years I have to learn to play without sheets or at least partly without.
Comment: What kinds of things do you want to play? Just anything or can you narrow it down?
Comment: This is funny. I always thought that being able to sightread and in general be a reader so good that you can avoid completely and consistently to resort to memory had to imply a degree of ear training in order to read a bar and having at least *some* idea of how it will sound.
Comment: IM so pleased to get repplies. IM year actually Been alone strugling and In spite Of all sometimes  enjoying ...;)
Comment: I play (try to) play differentiaali King's Of which ever sould nice to me. However most evergerrns, peaceful. Toutching lyrics ...  Also would like to accomp singing Of common songs ( after I get over The treshold Of openiin my mouth)
Comment: I am struggling with the same issue. I've been sight reading about 10 years, on a different instrument, and now want to learn piano. But I can not even play the simplest of tunes without the written notes. I find that the hardest part is starting the first note on a key that makes sense. Unless I know where to start, couldn't I find myself needing to play in a key that I rarely use, with many sharps or flats? How should I select that first note?
Answer: I suggest starting with lead sheets, with melody and chords only.
The term "playing by ear" was always a pejorative when I was young. It meant some kind of illiterate flailing at the instrument, maybe learning rote patterns with no idea of how they worked together musically. (Think of little kids playing "Heart and Soul" with four hands - great fun.)
But what I think you are aiming to do is very different than that, and lead sheets can get you there.
When you take an improvisatory approach to playing - which IS what I think you are aiming for - two powers will help you, and happily those two powers aid in one another's development.
The first power is that of music theory, all the stuff you list in your question about chords and their relationships and scales.
The second is the ability to hear and replicate intervals and melodic passages without the aid of transcripted music.
Using lead sheets, or fake book music to begin will move you along rapidly to not needing any kind of manuscript at all.
Your left hand and the fingers of your right hand not engaged in playing the melody will dance to the tune according to the chord structure that is written above the melody line.
Getting over that hurdle of comfortably comping a melody - with just your sense of rhythm and a chord name to guide you - takes some time. If you sing, it can be advantageous to sing the melody, and occupy your hands with an improvised accompaniment. It is quite literally a dance your fingers perform according to the rhythm you feel.
Once you have mastered playing from a lead sheet, your ability to discard all manuscripts will follow. Playing by ear? Perhaps, but while deploying all your musical knowledge as you do so.
It seems obvious, but for me it was a hard won insight to realize that the music happens under my hands, not on any page. I make the music with my instrument, not because someone else wrote out instructions for me.
The great classical players absorb themselves in their pieces to the extent that they create the sensation in themselves and in their audience of generating the music in the moment it is being played, just like you would expect any jazz player to do.
Good luck and go for it.
Comment: Pleased to notice help is around ... So long been alone.  I've been learning theory bit by bit with applying in practical playing. ( I a bit too  inpatient tu long theoretical approaches ;)) I think in ear training might develope the same,  got now a pile of fake books which are I terestonf approach to me. The jazzy/ bluesy world is also interesting. However the plentiful chord sets and changes have given me new technical challenges ( theoretically I think I can just manage) . So speeding up finding chords quicker is one learning aim. However hearing better might speed up predicting ...
Comment: ... But the basics in hearing better I think with  right kind of practices finding key signatures, root notes of chords, accidentals are my weak points now on the table.
Answer: Here are some suggestions, have a try and let us know how it goes:
Get a feel for the tune

CD player / Media player, play a tune that is not too complicated you have never learnt
Listen to the whole tune a few times until you can hear in your head what the next melody will be before it plays

Learn in parts
 - Play a few seconds at a time (a phrase) and just find the root note by ear on the piano

Build up the chord until it sounds most similar to the tune
Play the next phrase on the CD / Media player and do this again

Test your playing

When you think you have got a section to a good standard, listen to the whole thing on the CD / Media player to see if you have any differences and correct them, repeat this
Sometimes you will struggle and need to check the sheet music to see if you are correct, if it sounds right and the sheet music is different don't assume you are wrong, the sheet music may be wrong or your piano may just sound better playing the chord in a different arrangement.

Eventually after doing the above for a lot of songs you will find you may not even need to hear the song, you can just do it from memory of what you think the song sounds like. But this takes time and practice.
Comment: Hi Dave.  I think you are now near my situation.  As I wrote to my pervious supporter ... ... But the basics in hearing better I think with  right kind of practices finding key signatures, root notes of chords, accidentals are my weak points now on the table..  Try to follow your Suggs... Let y know thanks. William
Answer: Start simple.  Take a familiar tune like "Happy Birthday" and pick a starting note.  Lets just pick "G" (this means you will be playing the tune in the key of C Major).  Keep trying to play the song, re-starting from the beginning every time until you can get through the entire thing.
At first, this will be difficult.  You'll play the first 8 notes, and really struggle to find the 9th but I promise if you keep at it you will succeed.  Then, once you master the tune in the key of C, transpose it to another key.  If you start "Happy Birthday" on a "D" note, what key will you be playing the song in?  Play the whole tune in this new key, and then go back and play it in the key of C.
I was in your same position about ten years ago, and I just kept trying to pick out songs by ear and now I'm pretty good at it.  Just do it over and over and over.
Comment: Hi.  You are also right. I'll the to keep it simple.  Although changing the key signature without sheet is not simple to me even I'll manage it from sheets.  Strange and chilly by so true..  Thank you so much for all of you.  Nice to have you around.  Winter came back to Finland, william
Comment: Transposing is a great way of learning to play by ear


Join de tabelas para o Tableau

Question: Precisei criar uma extração para um relatório do Tableau, onde meu time desenvolveu o banco de dados e o sistema web e outra companhia desenvolveu o relatório do Table.
Esse é o esquema do problema

Para alimentar o relatório do tableau eu desenvolvi a solução abaixo, mas não sei se essa foi a melhor solução
DECLARE @Pillars table (Id int , Name varchar(30), ExhibitionOrder tinyint)
DECLARE @Initiatives table (Id int identity(100,2) , PillarId int,  Name varchar(35), ExhibitionOrder tinyint)
DECLARE @Goals table (Id int identity(10,30) , PillarId int, Descriptions varchar(60), ExhibitionOrder tinyint)

INSERT @Pillars (Id, Name, ExhibitionOrder) values
 (1, 'Communication',1)
,(2, 'Environmental Responsibility',2)

INSERT @Initiatives (PillarId, Name, ExhibitionOrder) Values
(1,'Social Network',2)
,(1,'Television',1)
,(1,'Brand Content',3)
,(2,'Financial education',1)
,(2,'Your garbage, Your responsibility',0)

INSERT @Goals (PillarId, Descriptions , ExhibitionOrder) Values
 (1, 'Improves the number of direct contact on call center in 10%',2)
,(1, 'Reduces paper media in 2%',1)
,(2, 'Reduces in 35% the recycle garbage in the office',1)
,(2, 'Reduces in 2% the carbon footprint on the office',1)
,(2, 'Reduced 2% Hydro bill',1)

;WITH
lv0(n) AS (SELECT 0 FROM   (VALUES (0), (0))G(n)), --2
lv1(n) AS (SELECT 0 FROM   lv0 a CROSS JOIN lv0 b), -- 4
lv2(n) AS (SELECT 0 FROM   lv1 a CROSS JOIN lv1 b), -- 16
lv3(n) AS (SELECT 0 FROM   lv2 a CROSS JOIN lv2 b), -- 256
lv4(n) AS (SELECT 0 FROM   lv3 a CROSS JOIN lv3 b), -- 65,536
--lv5(N) as (select 0 from lv4  a cross join lv4 b), -- 4,294,967,296
tally(n) AS (SELECT Row_number() OVER( ORDER BY (SELECT NULL)) FROM   lv4),
t1 AS (SELECT Row_number() OVER( ORDER BY id ) N ,* FROM   @Pillars),
t2 AS (SELECT Row_number() OVER( ORDER BY id ) N ,* FROM   @Initiatives),
t3 AS (SELECT Row_number() OVER( ORDER BY id ) N ,* FROM   @Goals)
SELECT  A.Name Pilar
       ,b.Name Initiative
       ,c.Descriptions Goals
       ,a.id
       ,b.PillarId
       ,c.PillarId
FROM   tally t
       CROSS JOIN t1 a
       LEFT JOIN t2 b
              ON t.n = b.n
                 AND a.id = b.PillarId
       LEFT JOIN t3 c
              ON t.n = C.n
                 AND a.id = c.PillarId
WHERE  a.id IS NOT NULL
       AND ( b.id IS NOT NULL
              OR c.id IS NOT NULL )
ORDER  BY  IsNUll(a.ExhibitionOrder,255)
          ,IsNUll(b.ExhibitionOrder,255)
          ,IsNUll(c.ExhibitionOrder,255)

            SELECT
                    A.Name Pilar
                   ,b.Name Initiative
                   ,c.Descriptions Goals
                   ,a.id
                   ,b.PillarId
                   ,c.PillarId
            FROM
                @Pillars A
            INNER JOIN @Initiatives B
                ON
                A.Id = B.PillarId
            INNER JOIN @Goals C
                ON
                A.Id = B.PillarId
            ORDER  BY  IsNUll(a.ExhibitionOrder,255)
                      ,IsNUll(b.ExhibitionOrder,255)
                      ,IsNUll(c.ExhibitionOrder,255)

Resultado
Pilar                          Initiative                          Goals                                                        id          PillarId    PillarId
------------------------------ ----------------------------------- ------------------------------------------------------------ ----------- ----------- -----------
Communication                  Television                          Reduces paper media in 2%                                    1           1           1
Communication                  Social Network                      Improves the number of direct contact on call center in 10%  1           1           1
Communication                  Brand Content                       NULL                                                         1           1           NULL
Environmental Responsibility   Your garbage, Your responsibility   Reduced 2% Hydro bill                                        2           2           2
Environmental Responsibility   Financial education                 Reduces in 2% the carbon footprint on the office             2           2           2
Environmental Responsibility   NULL                                Reduces in 35% the recycle garbage in the office             2           NULL        2

(6 row(s) affected)

Comment: A coluna ExhibitionOrder é de preenchimento obrigatório? // Para um mesmo valor de PillarID, podem existir valores repetidos para a coluna ExhibitionOrder? // Se valores repetidos forem permitidos, como decidir qual é a ordem de exibição?
Comment: Olá @JoséDiz, o campo ExhibitionOrder é um campo opcional distinado para ordernar os registros na apresentação, ele pode possuir qualquer valor inteiro e não é unico.
Answer: Para as 3 tabelas existe o campo de ligação PillarId. Ou seja, uma hierarquia em 3 níveis. Mas o campo de ligação entre os 3 níveis é o mesmo. Então o resultado irá ser esse mesmo, muitas linhas, pois o resultado será retornar todos os registros da tabela 3 para cada PillarId da tabela 2 para cada ID da tabela 1. Neste caso, para o id 1 da tabela 1 existem 3 registros na tabela 2 e 2 registros na tabela 3, isso resultará em, 2 registros (tabela 3) para cada registro na tabela 2.
Para não ocorrer isso, é necessário haver um outro campo de ligação entre as tabelas 2 e 3.
Comment: As iniciativas podem ser para atender a um ou mais objetivos ou os objetivos podem ser alcançados por uma ou mais inicciativas. Representar essa abstração tornaria no sistema seria muito complexo uma vez que qualquer combinação acima é válida.
Answer: Me parece que a consulta pode ser montada com o uso de junção do tipo FULL OUTER JOIN entre as tabelas Initiatives e Goals, acrescentando sequenciamento por pilar em cada uma dessas duas tabelas.
-- código #1 v4
with
cte_I as (
SELECT *,
       Seq= row_number() over (partition by PillarId
                               order by coalesce(ExhibitionOrder, Id))
  from @Initiatives
),
cte_G as (
SELECT *,
       Seq= row_number() over (partition by PillarId
                               order by coalesce(ExhibitionOrder, Id))
  from @Goals
)
SELECT coalesce(I.PillarId, G.PillarId) as PillarId,
       P.Name as Pilar,
       I.Name as Iniciativa,
       G.Descriptions as Objetivo
  from cte_I as I
       full outer join cte_G as G on I.PillarId = G.PillarId
                                     and I.Seq = G.Seq
       inner join @Pillars as P on P.Id = coalesce(I.PillarId, G.PillarId)
  order by P.ExhibitionOrder, coalesce(I.Seq, 255), coalesce(G.Seq, 255);

O sequenciamento por pilar é confiável nos casos em que a coluna ExhibitionOrder ou é informada para todas as linhas ou então para nenhuma linha de um mesmo pilar. Mas, se para um determinado valor de PillarId houver informação de ExhibitionOrder em somente parte das linhas, o sequenciamento por pilar pode (ou não) gerar ordem inesperada.

Join de tabelas para o Tableau


How long would it take you to code a PHP shopping cart like osCommerce from scratch?

Question: I'm just learning PHP, but I was wondering to the advanced PHP programmers; how long would it take you to code a shopping cart exactly like osCommerce from scratch?
I'm just interested in seeing how quickly others can code a fully functional shopping cart like osCommerce, and hopefully what I can look forward to do one day once I learn PHP fluently.
Comment: A while back I looked at the innards of osCommerce. What I saw made me run away - the biggest red flag being "you must have register_globals() turned on" in the requirements. That indicates a level of code quality that compares about equally with the stuff that floats around in sewage plant settling ponds.
Comment: Around 48 years! http://www.ohloh.net/p/osCommerce
Answer: It really depends on the feature set you want to implement. A really simple cart for a basic personal website wouldn't take more than a day of casual coding, but a heavy-duty feature-rich cart would take substantially more time.
It also depends on the site you're integrating with; some sites are an absolute pain to work with, while others are clear and easy to decipher. The most complex sites can have hundreds of different cases for the cart to handle, which can easily multiply your development time.
Answer: I've built a few online shops now, the last one with probably most (not all) of the features of osCommerce. It took me about 4 or 5 days to get it up and running, and I've been building on it, off and on since then.


UNNEST array stored inside object?

Question: I am trying to unnest an array which is stored inside an object in BigQuery. I am relatively inexperienced when it comes to JSON's, but I have been looking for a solution for this for a while with no luck.
I have other cases similar to this where I have created UDF's inside BigQuery to define the structure of JSON arrays and JSON objects where they are separate, but this is the first case I have come across where one is inside the other.
Example Structure (JSON name is "routes");
{"legs":

        [{"eta": "2020-04-25T00:00:00.000Z", "etd": "2020-03-31T00:00:00.000Z", "change": "new"}],

"arrival_time": "2020-04-25T00:00:00.000+00:00", "cut_off_date": "2020-03-29",
"departure_time": "2020-03-31T00:00:00.000+00:00", "last_pickup_on": "2020-03-29"}

I have tried so many different things it would be difficult to list so I am wondering if anyone has a simple solution to this which I might have overlooked? I have read through BQ documentation and tried things like...
UNNEST(routes) as routes_unnest

Which does not work, along with stored UDF's which have a similar structure to below...
CREATE OR REPLACE FUNCTION `project`.udfs.extract_routes(input STRING)
  RETURNS STRUCT<
    arrival_time TIMESTAMP,
    cut_off_date STRING,
    departure_time TIMESTAMP,
    last_pickup_on STRING,
    legs ARRAY<
    STRUCT<
      eta TIMESTAMP,
      etd TIMESTAMP,
      change STRING,
  LANGUAGE js AS "return JSON.parse(input);";

I believe my failure lies within my query, if that is the case then I am stuck with how to query the array section. Help is most appreciated here! Thank you for your time.
Comment: Bigquery has (very) limited support of JSON functions. I would look at changing data ingestion and load the JSON string in a table of its own as BQ would parse all nodes for you.
Comment: Can you please show sample data from your BigQuery table? It can be a screenshot, or schema. Also, don't provide to add at least a row data to make it easier to understand the data.
Comment: Hi Sabri, the best I can do is provided above in the "Example Structure". The data is quite sensitive so a screenshot or anything else would not work. I thought the example above is sufficient?
Comment: Is the data stored as JSON in data, or is it already a BQ table with nested columns. I want to see how the data is stored, so I can help. You can show dummy data with same structure.
Comment: It is just stored as a JSON in the data. I am at the stage where I am trying to make the data inside the array "queryable" - hope this helps explain it better?
Answer: Below is for BigQuery Standard SQL
#standardSQL
CREATE TEMP FUNCTION json2array(input STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS '''
  return JSON.parse(input).map(x=>JSON.stringify(x));
''';
SELECT
  JSON_EXTRACT_SCALAR(routes, '$.arrival_time') AS arrival_time,
  JSON_EXTRACT_SCALAR(routes, '$.cut_off_date') AS cut_off_date,
  JSON_EXTRACT_SCALAR(routes, '$.departure_time') AS departure_time,
  JSON_EXTRACT_SCALAR(routes, '$.last_pickup_on') AS last_pickup_on,
  JSON_EXTRACT_SCALAR(leg, '$.eta') eta,
  JSON_EXTRACT_SCALAR(leg, '$.etd') etd,
  JSON_EXTRACT_SCALAR(leg, '$.change') change
FROM `project.dataset.table`,
UNNEST(json2array(JSON_EXTRACT(routes, '$.legs'))) leg

You can test, play with above using dummy data as in below example
#standardSQL
CREATE TEMP FUNCTION json2array(input STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS '''
  return JSON.parse(input).map(x=>JSON.stringify(x));
''';
WITH `project.dataset.table` AS (
SELECT '''
  {"legs":
  [
    {"eta": "2020-04-25T00:00:00.000Z", "etd": "2020-03-31T00:00:00.000Z", "change": "new"},
    {"eta": "2020-04-25T00:00:00.000Z", "etd": "2020-03-31T00:00:00.000Z", "change": "new1"},
    {"eta": "2020-04-25T00:00:00.000Z", "etd": "2020-03-31T00:00:00.000Z", "change": "new2"}
  ],
  "arrival_time": "2020-04-25T00:00:00.000+00:00",
  "cut_off_date": "2020-03-29",
  "departure_time": "2020-03-31T00:00:00.000+00:00",
  "last_pickup_on": "2020-03-29"}
''' routes
)
SELECT
  JSON_EXTRACT_SCALAR(routes, '$.arrival_time') AS arrival_time,
  JSON_EXTRACT_SCALAR(routes, '$.cut_off_date') AS cut_off_date,
  JSON_EXTRACT_SCALAR(routes, '$.departure_time') AS departure_time,
  JSON_EXTRACT_SCALAR(routes, '$.last_pickup_on') AS last_pickup_on,
  JSON_EXTRACT_SCALAR(leg, '$.eta') eta,
  JSON_EXTRACT_SCALAR(leg, '$.etd') etd,
  JSON_EXTRACT_SCALAR(leg, '$.change') change
FROM `project.dataset.table`,
UNNEST(json2array(JSON_EXTRACT(routes, '$.legs'))) leg

with below output
Row arrival_time                cut_off_date    departure_time          last_pickup_on  eta etd change
1   2020-04-25T00:00:00.000+00:00   2020-03-29  2020-03-31T00:00:00.000+00:00   2020-03-29  2020-04-25T00:00:00.000Z    2020-03-31T00:00:00.000Z    new
2   2020-04-25T00:00:00.000+00:00   2020-03-29  2020-03-31T00:00:00.000+00:00   2020-03-29  2020-04-25T00:00:00.000Z    2020-03-31T00:00:00.000Z    new1
3   2020-04-25T00:00:00.000+00:00   2020-03-29  2020-03-31T00:00:00.000+00:00   2020-03-29  2020-04-25T00:00:00.000Z    2020-03-31T00:00:00.000Z    new2

Or you can wrap-up the query with TO_JSON_STRING if you need output still as a [unnested] JSON
#standardSQL
CREATE TEMP FUNCTION json2array(input STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS '''
  return JSON.parse(input).map(x=>JSON.stringify(x));
''';
SELECT TO_JSON_STRING(t) AS route
FROM (
  SELECT
    JSON_EXTRACT_SCALAR(routes, '$.arrival_time') AS arrival_time,
    JSON_EXTRACT_SCALAR(routes, '$.cut_off_date') AS cut_off_date,
    JSON_EXTRACT_SCALAR(routes, '$.departure_time') AS departure_time,
    JSON_EXTRACT_SCALAR(routes, '$.last_pickup_on') AS last_pickup_on,
    JSON_EXTRACT_SCALAR(leg, '$.eta') eta,
    JSON_EXTRACT_SCALAR(leg, '$.etd') etd,
    JSON_EXTRACT_SCALAR(leg, '$.change') change
  FROM `project.dataset.table`,
  UNNEST(json2array(JSON_EXTRACT(routes, '$.legs'))) leg
) t

in this case output is like below
Row route
1   {"arrival_time":"2020-04-25T00:00:00.000+00:00","cut_off_date":"2020-03-29","departure_time":"2020-03-31T00:00:00.000+00:00","last_pickup_on":"2020-03-29","eta":"2020-04-25T00:00:00.000Z","etd":"2020-03-31T00:00:00.000Z","change":"new"}
2   {"arrival_time":"2020-04-25T00:00:00.000+00:00","cut_off_date":"2020-03-29","departure_time":"2020-03-31T00:00:00.000+00:00","last_pickup_on":"2020-03-29","eta":"2020-04-25T00:00:00.000Z","etd":"2020-03-31T00:00:00.000Z","change":"new1"}
3   {"arrival_time":"2020-04-25T00:00:00.000+00:00","cut_off_date":"2020-03-29","departure_time":"2020-03-31T00:00:00.000+00:00","last_pickup_on":"2020-03-29","eta":"2020-04-25T00:00:00.000Z","etd":"2020-03-31T00:00:00.000Z","change":"new2"}

Where each row is a JSON that can be easily queried further with JSON_EXTRACT_SCALAR functions
{
"arrival_time": "2020-04-25T00:00:00.000+00:00",
"cut_off_date": "2020-03-29",
"departure_time": "2020-03-31T00:00:00.000+00:00",
"last_pickup_on": "2020-03-29",
"eta": "2020-04-25T00:00:00.000Z",
"etd": "2020-03-31T00:00:00.000Z",
"change": "new"
}

Comment: It may be an exception thrown by your mapping function when parsing a null value. At first sight I would suggest to check data source for missing fields or to handle the null values in your js json2array function.
Comment: Thank you for this, your first solution works with dummy data however when plugged into mine it returns the following error which I cannot seem to find a solution for; TypeError: Cannot read property 'map' of null at json2array(STRING)

UNNEST array stored inside object?


Which test is appropriate for comparing Pre and Post intervention data for an Intervention. Within Sample

Question: Im running a study that tests whether an intervention is effective in reducing depression levels, increasing well-being, optimism and self-esteem and increasing positive affect. My DV's are: Depression, Well-being, Optimism, Self-Esteem, Positive/Negative affect.
my iv is the intervention itself.
There are 62 participants in the Pre-assessment condition and 70 participants in the post-assessment condition however its a within sample.
which statistical analysis should i use? A T-test or Anova?
Comment: Look at:  https://stats.stackexchange.com/questions/3466/best-practice-when-analysing-pre-post-treatment-control-designs
Answer: With this data alone you will be unable to do any meaningful analysis of whether the intervention is effective.
You would need something that would tell you what would have happened, if the intervention had not been given. Patients will typically improve even in the absence of an intervention due to regression to the mean (especially if a certain patient state was required for entry), the placebo effect, the Hawthorne effect, more attention being paid to them as part of a study and so on.
Ideas for making some valid conclusions could be using historical controls (e.g. using the robust meta-analytic predictive approach or power prior approaches), concurrent controls (e.g. using propensity score methods, especially if you compare versus another intervention so that both interventions get a placebo effect) and other methods for observational studies.
Comment: The design has now changed.The sample is within as there are only 40 participants in the intervention (treatment group) so there is no control group. I applied paired samples-t test
Comment: Not sure what you want to learn from that. It certainly will not tell you whether the intervention has an effect.
Comment: Do you perhaps have any other suggests regarding the analysis and the sample i described so that i could find out wehther or not it has an effect.
Comment: the specific study was conducted on healthy students, it was basically a way to enhance well-being through a positive psychology intervention. There were controls but my analysis is only on the intervention (treatment group)
Comment: An analysis in the intervention group only cannot tell you whether the intervention was effective.
Comment: Can you estimate the values of the DVs in the general (I'm assuming student) population?
Comment: @GuillaumeLoignon I guess if it is a representative sample of what happens in the population of interest,  then I guess so.
Comment: Yes the sample is students. I now have a within sample and i applied paired t-test to the 40 participants i have in the intervention to asses the differences in the dv's before and after the intervention
Comment: Can you clarify the description of the experimental design? You mentioned 40 that had the intervention. Did the remaining 30 also had a treatment or are they control?


read only access to prepopulated solr

Question: I'm new to apachesolr/solr/drupal so please be patient (:
I have running solr-4.10.3 inside tomcat 8 and have "successfully" connected to it from drupal 7 via apachesolr.
The solr core that I've set up is already populated with external data and I can search it successfully from the SOLR web interface. The population was not done through drupal/apachesolr indexing mechanism, but directly into solr.
How can I search my data?
I can see my fields when I go to
http://localhost:8080/site/#overlay=admin/reports/apachesolr/solr
but they are all listed as "NOT INDEXED"
I'd appreciate any help or advice.
Nat
Comment: Why not repopulate with the Apachesolr Search module or the Search API + solr addons?
Comment: Thanks for the comment, but I'm not sure I can do that. The "database" is a large collection of continuously updated/uploaded XML files which are batch inserted into SOLR using software already in place. Is it possible to programmatically add these files to the site without using the browser interface? Nat
Comment: So the data in the solr database is *not* Drupal pages?  I'm afraid I can't help you there.
Answer: I can recommend taking a look at Sarnia

Sarnia allows a Drupal site to interact with and display data from Solr cores with arbitrary schemas, mainly by building views. This is useful for Solr cores that index large, external (ie, non-Drupal) datasets that either aren't practical to store in Drupal or that are already indexed in Solr.

I am using to search a solr index with emails imported with the dih
Answer: I have a website that uses the apachesolr module with a completely external, read-only index.
What I have done is:

Indexing happens outside of Drupal.
I have added all my index fields to the solr schema.xml file.
I implement hook_facetapi_facet_info() for the fields I want to use facets with. Note that this isn't needed for basic search to work.

and this works fine.
In the report I get legit stats for number of items in the index and it shows my fields. I don't get distinct term counts though because I have more than 20000 items in the index.
I do not see "NOT INDEXED" anywhere.
I also have some customisations to facet blocks and a custom sort block but that is just for my specific use case, it isn't a requirement.
The other part to my setup is my search pages are all custom pages in a custom module that manually run queries using apachesolr_drupal_query() and apachesolr_do_query().
I'm not sure how feasible it is to use the out of the box search page with an external index.
The apachesolr_search module is a useful example of search functionality. You might find there are chunks of that module you can copy and paste and modify to suit your needs.
To implement a custom search page you need a hook_menu() callback for your page, for which there are many examples around the internet.
There are a lot of ways you could implement it based on your requirements and preference but the way I have it is my menu item points to a search results page that by default shows x number of results, not filtered in any way.
If there is a url segment after the search url that is taken as the search keys (entry into search box), just like the default drupal search and apachesolr_search work.
Then I have the normal drupal search box, which has been form altered to add a custom submit handler that redirects (via $form_state['redirect']) to my search page with any keys appended.
I then use facetapi facet blocks as usual. The facet blocks should link to whatever the current search is and they pass facets in as query string params (from memory I think that's how it works).
So for those to work the custom search page also checks for any facet query strings to pass into its query.
As mentioned before I have a custom search block, which also passes search options via the query string.
This is heavily based on the apachesolr_search module sort block.
So your custom page really assembles search keys and query string params for facets and sort into a query, runs it, then returns the results.
You can return the results in any format you choose.
As per hook_menu() usage you will return a render array or a string of HTML.
I would recommend using custom theme functions to output your results.
So you get the results from your query and pass them as variables into your template, which returns the markup.
See hook_theme() for more information on creating theme templates.
So that brings you to the main part, the solr query.
For this you can use apachesolr_search_search_results() as an example and take from that the parts you find relevant to your usage.
The main parts are:
apachesolr_get_solr() to load the search environment you want to use to search on.
apachesolr_search_run() to run the actual search.
I found that apachesolr_search_run() was not suitable for my needs because it did some things I didn't want, like processing the response in a way that doesn't suit your my data.
So I took the parts from there that I needed instead, which are:
apachesolr_drupal_query() to load a query object.
apachesolr_do_query() to run the query.
apachesolr_has_searched() to tell the system that a search has been run. This is used for things like facet blocks later in the page load.
So in the end the basis of the search is:
apachesolr_get_solr()
apachesolr_drupal_query()
apachesolr_do_query()
apachesolr_has_searched()
Now your use case may mean you want to utilise some other parts of the apachesolr module but if you start from apachesolr_search_search_results() and follow it through you can see how it works and take the parts you want.
I'm sure given more time I could improve my solution but it currently gives the results I require.
I'm not sure how complex your searching is but be aware that there is a fair bit of caching that goes on when running these searches. It is used for things like facets, so that they know what the current page's search was to show you its facets.
If you are doing multiple searches on a single page load you will run into issues, but that's probably outside the scope of this specific question, which is just to get basic searching running.
It's worth noting that my search index structure was very different from a normal drupal content one and my search requirements were very custom, so there may be simpler ways to do this for a simpler setup, but this is definitely one option that works.
Comment: Thanks for taking the time to answer rooby. What you've described is **exactly** what I'm trying to do except that after reading your message I can see that I obviously lack the knowhow! Your first 2 bullet points were clear and I've made those changes. Your last bullet point about  hook_facetapi_facet_info() and the custom search page are obviously what I'm missing. I'll start reading up on your last point. If you have the time (and the inclination (: ) to let me have some sample code, it would make life much easier for me. Thanks again, nice to hear what I'm trying is doable. nat
Comment: I added some information about making a custom search page.
Comment: Excellent! Thank you very much rooby. I've already started experimenting with the code. I'll no doubt have to ask more questions later and hope for good answers like yours (: Nat
Comment: Just a quick update. rooby's comments were very helpful and got me started on Drupal module development. Thanks rooby! I know I'm being very impatient and asking for a lot, but if you have a any sample code that you feel you can contribute, I'd be glad for it. I've got going with simple module development to do very simple things like make changes to forms, but when I've not quite figured out how to use what's available in apachesolr.api.php. A very simple, even hard-coded, apachesolr search to return something from SOLR would be a fantastic start. Sorry to bug you about this ):

read only access to prepopulated solr


Can the smallest solution of $\varphi(k)=n$ be an even positive integer?

Question: Suppose, $n$ is a positive integer and the equation $$\varphi(k)=n$$ has a solution. Upto $n=20\ 000$, the smallest solution $k$ (if existent) is an odd positive integer.

Do positive integers $n$ exist, such that the smallest solution of $\varphi(k)=n$ is an even integer ? If yes, which is the smallest such $n$ ?
Comment: How is this question different from [this one](https://math.stackexchange.com/questions/2866737/totient-number-for-which-the-smallest-element-of-the-inverse-is-even), which you asked and which was answered last July?
Answer: I don't know which $n$ is smallest, but I know $n=2^{32}$ satisfies the condition (look up Fermat numbers if you want to figure out why for yourself).
Since $n$ is a power of 2, $k$ must be of the form $2^a*p_1*...p_n$ where $p_i$ are distinct odd primes that are equal to a power of 2 plus 1. As it happens, $2^1+1$, $2^2+1$, $2^4+1$, $2^8+1$ and $2^{16}+1$ are all prime (see Fermat numbers) but up to $2^{2048}+1$ no other such primes are known, so any solution of the equation $\varphi(k)=2^{32}$ must have $k$ divisible by 4 (since $\varphi(p_1*...p_n)$ can only get you to $2^{31}$ and in particular $k$ will be even.
Edit: I can't comment but the mistake in the other answer is where he implicitly uses near the end that $\varphi(2q')=2\varphi(q')$ but $q'$ is odd so in fact $\varphi(2q')=\varphi(q')$
Comment: You could add $\phi(2^{64})=2^{32}$ to make clear that this isn't a case where no solution exists.
Comment: @Ingix You mean $\phi(2^{33}) = 2^{32}$, not $\phi(2^{64})$.
Comment: Yes, sorry. Those big numbers got into my head :-(
Comment: Good answer. But you missed to mention one detail : Should a larger Fermat prime exist, we cannot use it because we would then get beyond $2^{32}$
Comment: I think it was Euler who observed that we can prove that the prime $641$ divides $F_5=2^{2^5}+1$ by noting that in the field $\Bbb Z_{641}$ we have $0=641=5^4+2^4=1+5\cdot 2^7,\;$ so $2^4=-5^4$ and $2^7=-1/5,\;$ so $2^{32}=2^4\cdot (2^7)^4=2^4 (-1/5)^4=(-5^4)(-1/5)^4=-1.$
Answer: Building on the ideas in the answer by anonymous, I have managed to construct a smaller example, namely $2^{16}\cdot257 = 16842752$.  As was shown with $2^{32}$, I'll show that if $\phi(k) = 2^{16}\cdot257$, then $k$ must be even.
Start by considering what prime power $p^a$ in the factorization of $k$ could yield the factor of $257$ in $\phi(k)$.  A priori, there are two possibilities:

$a = 1$: In this case $\phi(p) = p - 1$ must be a multiple of $257$ and divisor of $2^{16}\cdot257$, so $p = 2^b\cdot257 + 1$ for some $b$, with $0\le b\le 16$.  It turns out that there is no such prime.  When $b$ is even, $2^b\cdot257 + 1$ is divisible by $3$; when $b\equiv1\pmod4$, it is divisible by $5$; when $b\equiv3\pmod8$, it is divisible by $17$; and when $b = 7$ or $15$, it is divisible by $67$ or $1123$, respectively.  (This fortunate numerical coincidence is why $257$ was chosen.)
$a \gt 1$: In this case, $p$ divides $\phi(k)$, so $p$ is either $2$ or $257$.  But $\phi(2^a) = 2^{a-1}$ is never a multiple of $257$, so $p = 257$.  Moreover, if $a$ were greater than $2$, then $\phi(257^a)$ would be divisible by $257^2$.  Thus $a=2$, so $k$ must have a prime-power factor of $257^2$.

Since $\phi(257^2) = 2^8\cdot257$, that leaves a factor of $2^8$ in $\phi(k)$ still to be accounted for.  As in anonymous's argument, it must come from some combination of factors of $2$ and Fermat primes that divide $k$.  But since we've already used $257$, the remaining smaller Fermat primes cannot by themselves produce the additional eight factors of $2$ in $\phi(k)$.  Hence $k$ is even; in particular, the smallest possible $k$ is $2^9\cdot257^2$.
Having figured out that $16842752$ works, it is now easy find out that it has been discovered before.  In particular, it is the first entry in sequence A143510 in the On-Line Encyclopedia of Integer Sequences.  This sequence shows all nine known integers $n$ such that $\phi(k) = n$ only for even values of $k$.  ($2^{32}$ is the largest of the nine.)
This still does not establish that $16842752$ is the smallest value for the question asked here, but it seems likely.  The value is small enough that it could reasonably be verified computationally.
EDIT: I should have said that sequence A143510 contains the nine smallest integers $n$ such that $\phi(k) = n$ only for even values of $k$; additional larger values are known, but there may be others unknown in between.  In a note titled Numbers Like 16842752, T.D. Noe lists many larger values, as well as observing that each is a member of an infinite family obtained by multiplying by suitable powers of $2$.


Como faço para sortear nomes aleatórios de cada lista sem repeti-los

Question: Eu quero sortear nomes aleatórios de 3 listas diferentes, tenho um programa em Python que faz mais ou menos isso. O problema é que eles fica repetindo vários nomes em combinações diferentes, gostaria que o programa pegasse apenas um nome de cada lista e formasse uma equipe.
from random import choice
from time import sleep

lista = []

j1 = ['Jose', 'Bruno', 'Lucas', 'Eduardo', 'Pedro', 'Luciano', 'Vitor', 'Diego', 'Rômulo', '*Pisca']
j2 = ['Carlinhos', 'Carlos', 'Davi', 'Thiago', 'Paulo', 'Igor', 'Felipe', 'Marcelo', 'Matheus', 'Fabio']
j3 = ['Artur', 'Anderson', 'Gustavo', 'Rogerio', 'Marcus', 'Nando', 'Jorge', 'Rodrigo', 'Caio', 'Jonas']

for c in range(1, 11):
    print('PROCESSANDO.....')
    sleep(2)
    print('-' * 20)
    print(f'      \033[34mEQUIPE {c}\033[m')
    print('-' * 20)
    sorteio = (f'{choice(j1)}\n'
               f'{choice(j2)}\n'
               f'{choice(j3)}')
    print(sorteio)
    print('-' * 20)

Answer: Você pode simplesmente embaralhar as listas e em seguida percorrê-las com zip (que serve para percorrer várias listas ao mesmo tempo):
from random import shuffle

j1 = ['Jose', 'Bruno', 'Lucas', 'Eduardo', 'Pedro', 'Luciano', 'Vitor', 'Diego', 'Rômulo', '*Pisca']
j2 = ['Carlinhos', 'Carlos', 'Davi', 'Thiago', 'Paulo', 'Igor', 'Felipe', 'Marcelo', 'Matheus', 'Fabio']
j3 = ['Artur', 'Anderson', 'Gustavo', 'Rogerio', 'Marcus', 'Nando', 'Jorge', 'Rodrigo', 'Caio', 'Jonas']

shuffle(j1)
shuffle(j2)
shuffle(j3)

for n, (c1, c2, c3) in enumerate(zip(j1, j2, j3), start=1):
    # imprime as outras mensagens, etc
    print(f'      \033[34mEQUIPE {n}\033[m')
    print(f'{c1}\n{c2}\n{c3}')

Eu também uso enumerate para obter o índice ao mesmo tempo em que percorro as listas (e uso o argumento start para que a contagem comece com 1, já que por padrão ela começa em zero). Isso é melhor do que usar um range específico, pois assim funciona independente do tamanho das listas (assumindo que todas elas têm o mesmo tamanho, já que zip se encerra quando a menor delas terminar).
Assim, a cada iteração, n é o índice e c1, c2 e c3 são um dos nomes das listas j1, j2 e j3. E como elas foram embaralhadas anteriormente, eu garanto que as equipes serão escolhidas aleatoriamente.

O problema é que shuffle acaba modificando a própria lista.
Se quiser manter as listas originais intactas, você pode usar sample, que retorna outra lista embaralhada:
from random import sample

j1 = ['Jose', 'Bruno', 'Lucas', 'Eduardo', 'Pedro', 'Luciano', 'Vitor', 'Diego', 'Rômulo', '*Pisca']
j2 = ['Carlinhos', 'Carlos', 'Davi', 'Thiago', 'Paulo', 'Igor', 'Felipe', 'Marcelo', 'Matheus', 'Fabio']
j3 = ['Artur', 'Anderson', 'Gustavo', 'Rogerio', 'Marcus', 'Nando', 'Jorge', 'Rodrigo', 'Caio', 'Jonas']

def embaralha(lista): # retorna uma cópia da lista, só que embaralhada
    return sample(lista, k=len(lista))

for n, (c1, c2, c3) in enumerate(zip(embaralha(j1), embaralha(j2), embaralha(j3)), start=1):
    print(f'      \033[34mEQUIPE {n}\033[m')
    print(f'{c1}\n{c2}\n{c3}')

Ou ainda:
for n, (c1, c2, c3) in enumerate(zip(*map(embaralha, [j1, j2, j3])), start=1):
    print(f'      \033[34mEQUIPE {n}\033[m')
    print(f'{c1}\n{c2}\n{c3}')

Comment: Aff, ia responder isso bem agora :( [repl.it](https://repl.it/@acwoss/BigJudiciousGlitch#main.py) já estava pronto...
Comment: @Woss Eu tava vendo se o `itertools` tem alguma coisa pronta, mas não achei (mas talvez dê pra fazer alguma "mágica", sei lá).. Fica aí a sugestão pra outra possível resposta :-)
Answer: Você pode remover o item sorteado da lista. Veja abaixo:
from random import choice
from time import sleep

lista = []

j1 = ['Jose', 'Bruno', 'Lucas', 'Eduardo', 'Pedro', 'Luciano', 'Vitor', 'Diego', 'Rômulo', '*Pisca']
j2 = ['Carlinhos', 'Carlos', 'Davi', 'Thiago', 'Paulo', 'Igor', 'Felipe', 'Marcelo', 'Matheus', 'Fabio']
j3 = ['Artur', 'Anderson', 'Gustavo', 'Rogerio', 'Marcus', 'Nando', 'Jorge', 'Rodrigo', 'Caio', 'Jonas']

for c in range(1, 11):
    print('PROCESSANDO.....')
    sleep(2)
    print('-' * 20)
    print(f'      \033[34mEQUIPE {c}\033[m')
    print('-' * 20)
    #
    j1_player = choice(j1)
    j1.remove(j1_player)
    j2_player = choice(j2)
    j2.remove(j2_player)
    j3_player = choice(j3)
    j3.remove(j3_player)
    #
    sorteio = (f'{j1_player}\n'
               f'{j2_player}\n'
               f'{j3_player}')
    print(sorteio)
    print('-' * 20)

Exemplo de saída
PROCESSANDO.....
--------------------
      EQUIPE 1
--------------------
*Pisca
Paulo
Caio
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 2
--------------------
Pedro
Igor
Jorge
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 3
--------------------
Diego
Carlos
Artur
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 4
--------------------
Vitor
Felipe
Jonas
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 5
--------------------
Luciano
Fabio
Anderson
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 6
--------------------
Rômulo
Carlinhos
Nando
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 7
--------------------
Lucas
Thiago
Rodrigo
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 8
--------------------
Jose
Marcelo
Gustavo
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 9
--------------------
Bruno
Davi
Rogerio
--------------------
PROCESSANDO.....
--------------------
      EQUIPE 10
--------------------
Eduardo
Matheus
Marcus
--------------------

UPDATE
Caso todos os nomes possam estar em uma lista única, tem outra forma (mais simples) de fazer que a primeira, veja abaixo:
>>> j = ['Jose', 'Bruno', 'Lucas', 'Eduardo', 'Pedro', 'Luciano', 'Vitor', 'Diego', 'Rômulo', '*Pisca', 'Carlinhos', 'Carlos', 'Davi', 'Thiago', 'Paulo', 'Igor', 'Felipe', 'Marcelo', 'Matheus', 'Fabio', 'Artur', 'Anderson', 'Gustavo', 'Rogerio', 'Marcus', 'Nando', 'Jorge', 'Rodrigo', 'Caio', 'Jonas']
>>>
>>> while len(j) != 0:
...     random.shuffle(j)
...     print(j[:3])
...     j = j[3:]
...
['Rômulo', 'Rodrigo', '*Pisca']
['Luciano', 'Nando', 'Carlos']
['Jonas', 'Eduardo', 'Igor']
['Fabio', 'Davi', 'Lucas']
['Thiago', 'Marcus', 'Paulo']
['Vitor', 'Felipe', 'Bruno']
['Anderson', 'Pedro', 'Marcelo']
['Caio', 'Artur', 'Matheus']
['Rogerio', 'Diego', 'Carlinhos']
['Jose', 'Jorge', 'Gustavo']
>>>

Espero que ajude


I want to cycle through users and only print the information when the directory exists - BASH

Question: I'm trying to cycle through all users on the system, and print only the users who have the directory .junkdir in their Home (The end product of this, is that only my group profiles will have the directory, and I want to total the disk usage for that directory for all users together - I will sort that after I sort this problem)
Can somebody advise where I'm going wrong?
<code>#Function to list total usage of all users in bytes
{
 #Get list of all users in aphabetical order
 get_users="$(getent passwd | cut -d: -f1 | sort)"

 IFS=$'\n' #Use newline as delimiter for for-loop
 for user_usage in $get_users
  if [ -d /home/"$user_usage"/.junkdir ]; then
    do
      #Do processing on each line
      echo 'Username:' $user_usage
      echo 'Usage(bytes):' "$(du -h --bytes /home/"$user_usage"/.junkdir)"
      echo '********************************'
    done
  fi
}
</code>
Answer: You could be better off using the 6th field, which is the home folder of the user, rather than guessing they are all in /home/
Comment: Sorry I've only just learning BASH. I'm aware that only a few of the users are in the /home/, I was under the impression that checking that the .junkdir directory exists would filter these out? Thats what I'm looking for help for.
Answer: I don't think you need that IFS edit. Personally, I would set IFS, but not use the variable, and pull field 6 for the dir because it might not be in /home.
<code>IFS=:
while read usr x x x x home x
do if [[ -d $home/.junkdir ]]
   then echo "Username: $usr"
        echo "Usage(bytes): $(du -h --bytes $home/.junkdir)"
        echo '********************************'
   fi
done < /etc/passwd
</code>
Comment: .junkdir is basically a directory that I send files I want to delete to. Only human users are going to have the .junkdir directory added, so I'm really wanting to filter out all the other users. Thats why I've been trying to implement it the way I have. You have to excuse my ignorance, I'm just struggling to see how I would change my code to your suggestions.
Comment: Try that. We could also add a threshold and automatic removal of oldest files first if you wanted to get ambitious. ;)


Sum of Beta-Bernoulli variables

Question: Assume you have  $x_i \sim \operatorname{Bernoulli}(p_i)$ with $p_i \sim \operatorname{Beta}(\alpha,\beta)$.
I am exploring $Z=X_1+ \dots +X_n$
According to this page, it is
$Z \sim \operatorname{BetaBinomial}(n,\alpha,\beta)$
but according to this page, with simulation, it is $Z \sim \operatorname{Binomial}(n,\frac{\alpha}{\alpha+\beta})$
Two different distributions. Which one is correct?
Also equally important, what is the difference in assumptions for each? So I know when to use them correctly and can simulate either one in the right context. Thanks!
Answer: Short summary: if the $p_i$s are independent, it's the binomial. If the $p_i$s are all equal, it's the beta-binomial.
By $X_i \sim \textrm{Bernoulli}(p_i)$, you must mean the conditional distribution $X_i\mid p_i \sim \textrm{Bernoulli}(p_i)$. The marginal distribution of $X_i$ (that is, the distribution obtained by averaging over different values of $p_i$s) is obtained, e.g., by noting that $X_i$ is Bernoulli, and computing the expectation using the tower law:
\begin{equation}
\mathbb{E}(X_i) = \mathbb{E}(\mathbb{E}(X_i \mid p_i)) = \mathbb{E}(p_i) = \frac{\alpha}{\alpha+\beta}.
\end{equation}
So, $X_i \sim \textrm{Bernoulli}(\frac{\alpha}{\alpha+\beta})$, for all $i$. However, this does not yet determine the distribution of $Z$ as you have not specified enough information to deduce the joint distribution of $X_i$s. Two additional things are needed:

The conditional distribution of $X_i$s given the  $p_i$s, $p(X_1,\ldots,X_n \mid p_1,\ldots,p_n)$.
The joint distribution of the $p_i$s: $p(p_1,\ldots,p_n)$.

For the first part, I guess&assume you intend the $X$s to be conditionally independent given the $p$s. The difference of the two distributions mentioned in your question stems from the second point.
If we assume the $p$s to be mutually independent, then the $X$s will be mutually independent, too, as each $X$ depends on only one $p$ and the $X$s are conditionally independent given the $p$s. Then, $Z$ is just the sum of iid. Bernoulli random variables. But this is the definition of the binomial distribution, and thus indeed
\begin{equation}
Z \sim \textrm{Binomial}\left(n,\frac{\alpha}{\alpha+\beta}\right).
\end{equation}
Note that in this case it did not make much sense to define the $p$s in the first place: if each $p_i$ influences only the corresponding $X_i$, nothing is learned about the distribution of $p_i$ except via the value of $X_i$. Thus, the $p_i$s are useless in the sense that the exactly same model would have been  easier to specify by just stating that $X_i$s are independent Bernoulli random variables with parameter $\alpha/(\alpha+\beta)$.
The beta-binomial distribution is actually defined so that there is only one $p$ parameter drawn from the beta distribution, and then all $X_i$s are Bernoulli with this $p$. This is obtained as a special case of your question by stating in my "step 2" that the joint distribution of the $p_i$s are all the same.
By defining some other dependence structures for the $p_i$s (not independent, but not constrained to be equal either), other distributions for $Z$ would be obtained, but I don't know if any of these have special names.

Sum of Beta-Bernoulli variables


Concatenating the contents of multiple files passed from command line

Question: I am trying to concatenate the contents of all the argv[] arguments passed while executing the code. count here is the same as argc. **files contains argv.
char *merge_everything(int count, char **files) {
        size_t size = 0;
        char *buf = NULL;
        buf = malloc((size + 1) * sizeof(*buf)); /* size + 1 byte for the \0 */
        if (buf == NULL) {
            printf("MALLOC ERROR\n");
            exit(1);
        }
        char *temp = buf;

        for (int i = 0; i < count - 1; i++) {
            FILE *f = fopen(files[i], "rb");

            fseek(f, 0, SEEK_END);
            long bytes = ftell(f);
            fseek(f, 0, SEEK_SET);

            fread(temp, (size_t) bytes, 1, f);
            temp += bytes;
            fclose(f);
        }
        *temp = 0;
        free(buf); //If I don't free buf here then memory leak? If I do temp has null?
        return temp; //temp has all the concatenated stuff
    }

I am in this dilemma that freeing buf at the end is preceding the return statement.
Please help me rectify the approach.
I am keen on using buffer (via malloc) instead of writing it into a new file. Hence, this approach.
Comment: *If I don't free buf here then memory leak*. Firstly you should be returning `buf` not `temp` as the former is the start of the string whereas the latter is somewhere in the middle of the string. Secondly, if that is done then it is not a memory leak as the caller is then responsible for doing the free.
Comment: Do you realize that the amount of memory you allocate isn't at all related to the size of any one file, much less all of them?
Comment: `size_t size = 0; malloc((size + 1) * sizeof(*buf)); ` How much memory do you think that allocates?
Comment: That allocates nothing. I realized that but instead of giving it a hardcoded macro like 512 or something. I want it to be adaptive to the file 1 size then file 2 size and so on.
Comment: Fair enough. But it doesn't happen magically. Where is the code to do that? Once the allocation is made it doesn't change size automatically.
Comment: temp has been keeping track of where 1 file ends and where to start reading the next file in the buffer. So, it moves farther from the origin over time. We return buf because it is still at the 0 mark?
Comment: Well if you don't return the start of the buffer how do you expect the caller to know where the data starts?
Comment: btw, If I return buf here at the end of the function, how do I free the memory malloc is supposed to provide?
Comment: you must not free the memory. If should be freed by the caller when it is not needed anymore
Answer:
You allocate 1 byte and read plenty. It is Undefined Behaviour.
You need to increase the allocated memory size for every file.
You should check file I/O errors.
I do not know why count-1 but maybe you have your reasons.

char *merge_everything(int count, char **files) {
        size_t size = 0;
        long bytes;
        char *buf = NULL, *temp;

        for (int i = 0; i < count - 1; i++) {
            FILE *f = fopen(files[i], "rb");

            fseek(f, 0, SEEK_END);
            bytes = ftell(f);
            fseek(f, 0, SEEK_SET);

            temp = realloc(buf, size + bytes);
            if(!temp)
            {
                free(buf);
                break;
            }
            buf = temp;

            fread(buf + size, (size_t) bytes, 1, f);
            size += bytes;
            fclose(f);
        }
        return buf; //temp has all the concatenated stuff
    }

Comment: This helps! About your statement "the caller should free memory", I am guessing that could be done by freeing the char * which collects buf, right?
Comment: Also, in your fread(), how can I get rid of the \n characters? it is tricky because we are keeping track of bytes read.
Answer: You need to grow the buffer as you go.  eg:
/* Read all named files into a buffer. */

#include <stdio.h>
#include <stddef.h>
#include <ctype.h>
#include <stdlib.h>

struct buffer {
        char *start;
        char *end;
        size_t cap;
};

void push(int c, struct buffer *);
int pop(struct buffer *a);
void * xrealloc(void *buf, size_t num, size_t siz, void *endvp);
FILE * xfopen(const char *path, const char *mode);

int
main(int argc, char **argv)
{
        struct buffer a = {0};
        int c;
        for( int i = 1; i < argc; i++ ){
                FILE *ifp = xfopen(argv[i], "r");
                while( (c = fgetc(ifp)) != EOF ) {
                        push(c, &a);
                }
        }
        while( (c = pop(&a)) != EOF ) {
                putchar(c);
        }
        return 0;
}

int
pop(struct buffer *a)
{
        return (a->start end) ? *a->start++ : EOF;
}

void
push(int c, struct buffer *b)
{
        if( b->start == NULL || b->end >= b->start + b->cap ) {
                b->start = xrealloc(b->start, b->cap += BUFSIZ, 1, &b->end);
        }
        *b->end++ = c;
}

void *
xrealloc(void *buf, size_t num, size_t siz, void *endvp)
{
        char **endp = endvp;
        char *b = buf;
        ptrdiff_t offset = b && endp && *endp ? *endp - b : 0;
        b = realloc(b, num * siz);
        if( b == NULL ){
                perror("realloc");
                exit(EXIT_FAILURE);
        }
        if( endp != NULL ){
                *endp = b + offset;
        }
        return b;
}

FILE *
xfopen(const char *path, const char *mode)
{
        FILE *fp = fopen(path, mode);
        if( fp == NULL ){
                perror(path);
                exit(EXIT_FAILURE);
        }
        return fp;
}


Programmatically save altered file

Question: As I mentioned in my previous question I'm attempting to write tests for a patch that will, ideally, trigger the creation of a redirect if the URI of a file changes.
My patch uses hook_file_update() which is called whenever file_save() runs.
Currently this chunk of code:
  // Move file to new URI
  $new_uri = 'public://qwerty.png';
  file_save($test_file);
  $test_file->uri = $new_uri;
  file_save($test_file);

This calls my hook and creates the redirect, but the redirect fails because drupalGet($new_url) returns a 404.
file_save() returns the error, filesize() [function.filesize]: stat failed for public://qwerty.png.
I've tried file_save() on URIs that linked to existing files, which gives an error, I've tried it on not-yet-extant URIs (as above), which gives an error, and I've tried file_move(), which doesn't call hook_file_update().
How can I programmatically change the URI of a file and trigger hook_file_update() inside a test?
Comment: Have you tried invoking the same code outside of the test environment? i.e. with a file that you know definitely exists and can't return a 404? It would help to work out if the problem is within the test environment or the patch logic
Comment: BTW this is really interesting, if you're able to upload the full module somewhere I'd be happy to have a play around and see if I can work it out. Can't promise anything but you never know I might get lucky :)
Comment: It's probably just a typo in the question but are you calling `drupalGet($new_url)` or `drupalGet($new_uri)` (with an `i`)?
Comment: I've tried both. But I know from stepping through with xdebug that the node_save() is failing so neither will ever work. Any by "both" I mean `$test_file->uri` and the output of `file_create_url($test_file->uri)`
Comment: I haven't tried it outside of a test (yet) but I have tried it with a file I know exists and doesn't return (on its own) a 404. That creates separate issues with file_save() because it doesn't want to use an already-in-use uri. The earlier question linked in the OP contains the entirety of my modifications to the redirect.module and redirect.test files (one custom  function on each), if you want to cut, paste, and try it out.
Comment: Trying it with the uri of an already-extant managed file causes `PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://docnotes_1.txt' for key 'uri'`
Comment: BTW This happens in both the test environment and a non-test sandbox now.
Comment: Cool that's something solid to go on, I've got some stuff do now but I'll take a look later on tonight
Answer: The way I ended up getting around this was:

Upload my file as an unmanaged file at the new URI
$test_file->uri = $new_uri;
file_save($test_file);
file_unmanaged_delete($old_uri);

It didn't like it when I used a URI that was not yet in existence, nor one that was already in use by a managed file, but using one that goes to an unmanaged file seemed to work fine.

Programmatically save altered file


How do I create a role-specific registration page?

Question: I would like to create a "hidden" registration page that I can send to our wholesale customers in order to sign up as dealers, instead of standard authenticated users.
As our dealers have access to wholesale pricing, I will require that this type of user be approved by the admin.
I will need to add a few new fields that I would only like to be shown on this dealer-specific registration page (vs. std), e.g. company, company address, POC. Do I create a new template for this form?
How can I add a Honeypot field to this form?
Answer: I suggest you to try the Multiple Registration module (https://www.drupal.org/project/multiple_registration), since this is precisely what it does "adds ability to create role-specific registration pages."
And as for the need to add the "admin approval" functionality, too, for specific user roles on your site, I would recommend you the Registration Role with Approval module (https://www.drupal.org/project/registration_role_with_approval)
Also, see if you will find this post of any help: https://www.optasy.com/blog/how-create-role-specific-user-registration-forms-drupal
Good luck!


Typeset a slur from a chord's middle note

Question: I'm trying to typeset a piano piece, where the following two chords challenge my Lilypond skills:

I'm sorry for the bad quality of the printed score (which is actually the reason why I'm retypesetting it). The snippet is from the upper staff, treble clef, in A major.
The first problem is the slur departing from g'' in the right hand.
It is a repeated pattern in this piece that slurs are attached to inner notes of a chord, so it seems intentional.
I found no other solution than using two voices, both rendered as \voiceOne so their stems collide and they appear as a single chord:
\relative c''' {
  \key a \major
  << { cis4 s8 } \\ { \voiceOne <g cis,>4( fis8) } >>
}

It feels like an ugly hack, but it works (except for a warning about colliding columns).
However, when I add the first chord of the left hand, it isn't moved to the right, resulting in a collision of c#'' and b':

I was unable to shift the chord through flipping \voice<N>, and even NoteColumn.force-hshift had no effect (why?!).
This is my current attempt:
\relative c''' {
  \key a \major
  << { cis4 s8 } \\
     { \voiceOne <g cis,>4( fis8) } \\
     { \voiceTwo
       \set doubleSlurs = ##t
       \once \override NoteColumn.force-hshift = #1.8
       4( <ais ais,>8) }
  >>
}

I see two possible solutions: There might be a better way to change the attachment of the slur in the right-hand chord, which doesn't involve abusing collisions. Or there is a way to (effectively) force-shift the left-hand chord.
Does anybody know how to achieve either of these?
Answer: With a reasonably recent version of 2.19 you'll be able to write
\fixed c'' {
  \key a \major
  << { <cis g( cis'>4 <fis)>8 } \\
     { 4^(_(\p <ais,, ais,>8) }
  >>
}

in order to get

In-note slur positioning like for the upper slur is not overly convincing yet, but at least the manner of getting double per-chord slurs (used in the lower voice in order to get nicer overall results than when using the \= construct for multiple per-note slurs) is slightly less cumbersome than meddling with the doubleSlurs property and also works for phrasing slurs.
Comment: Excellent! I upgraded to the dev version right away. Just a quick follow-up: You you know if in-note slur positioning is going to work for multiple slurs per chord (right now it doesn't)? I have another beast in this piece...


Prediction of propensity of new customer dataset based on old customer dataset

Question: We have 3 customer data sets. One has customer informations with around 25 attributes like
customerID, Gender,Relationship,Job_Level (manager, non-manager), Income level (<25000 USD, between 25000 to 50000 USD, <70000 USD etc)
The second dataset consiste of all those cutomers who bought the car.
The attributes are cutomerID, Month& year of purchase,Model (like sports, eco etc), Feature 1 used (say cruise control) and so on.
The thrid data set is new customer data set which has similar attributes as that in dataset1 (around 25 attributes).
Based on correlation between data set 1&2, we want to identify cutomers from data set 3 with high propensity to buy a car.
In addition, the expected solution can also indicate the kind of products and features the potential leads will be interested in along with tentaive time i.e calender month.
In total we have 1000 sample values in data set 1. Out of which 891 are "No" and 109 are "yes" i.e bought the car.
We are exploring Weka for classification, and clustering. However we have basic doubts about selection of algorithem for each stage

preprocessing Stage - what filters to be used such that attributes which are not significant will be automatically removed. i.e Is there any algo. which  will suggest/identify unimportant attributes ?
Which Classification algorithem we should we explore to predict pattern of cutomers who would buy the car?
We are not sure how to determine probable possible month in which new set of customer may be interetsed in buying.

Would really appreciate any inputs which will take us in right direction.
So far, We tried  WEKA decision tree(J48) & Rules (PART, Jrip,Prism).However none of them are giving us rules or tree for "yes" path i.e rule which leads us to customer pattern who bought the car.
If needed, shall post the output of each of the algorithms we tried in WEKA.
Answer: In case anybody new in statistics stumble upon, here is summary of steps followed to address the use case.

Pre-processing &  Attribute selection.
Pre processing involves data clean up like handling missing values in data set, conversion of continuous values into discrete values etc. In our case, we converted the Age into suitable age groups, This is to avoid too many possible prediction based on the Age value.

Attribute Selection:
Weka GUI offers good support for Attribute selection. You need to supply original dataset with necessary pre processing steps. Then apply various methods like use "BestFirst" or "rank". The output is set of attributes which you should consider for prediction.
Useful link - http://machinelearningmastery.com/feature-selection-to-improve-accuracy-and-decrease-training-time/

Classification Algorithm selection:
In this case, we wanted to predict a discrete class value i.e weather customer will Buy a vehicle or Not? And if he is gonna Buy, then what is the probability of purchase. Hence its a supervised learning case,with known response variable (Buy or Not-Buy)
Since old sales data is  available, hence it falls in category of "Rule-based reasoning" methods providing explicit knowledge model (Old sales data), which can be expressed by formal rules or not, to be applied for further prediction. So we decided to go with decision tree, we chose random Tree algorithm assuming random data distribution.
Now we chose Cross Validation option to ensure that model built is tested on available data.
So java code looks like,
<code>String [] agrTraning = {"-t","traningData.arff","-d","treemodel.model"};
        RandomTree.runClassifier(new RandomTree(),  agrTraning);
        //to predict by applying model over test data. above model is applied over test data testData.arff to predict output
        String [] agrTest = {"-l","treemodel.model","-T","testData.arff","-p","1-8","-distribution"};
        RandomTree.runClassifier(new RandomTree(),  agrTest);
</code>

3.To determine probable possible month in which new set of customer may be interested in buying.
Here we had two predictions to be done,  one is car model prediction and second month prediction.In WEKA GUI under "Cluster" tab, there are multiple algorithm which can be applied on data set & select one with lower value of "sum of squared errors" (Even though some claim that lower value of SSE does not guarantee accuracy of predictions since this SSE is for given trained data & est data may be altogether different than it)
Sinnce we could get lower SSE for KNN, hence we applied same to predict the model
<code> IBk i = new IBk();
            String [] agrTraning = {"-K","1", "-W" ,"0", "-A", "weka.core.neighboursearch.LinearNNSearch -A \"weka.core.EuclideanDistance -R first-last\"","-t","model_predict_traindata.arff","-d","KNNmodel.model"};
            i.runClassifier(new IBk(),  agrTraning);
            //to predict by applying model over test data. above model is applied over test data testData.arff to predict output
            String [] agrTest = {"-l","KNNmodel.model","-T","predict_model_test_data.arff","-p","0","-distribution"};
            i.runClassifier(new IBk(),  agrTest);
</code>
We also felt that MEKA which has multiple class variable is better option to predict Model & time. But we are yet to get satisfactory results on "time prediction".
Also a good link describing how to approach such problems
https://www.ibm.com/developerworks/library/os-weka2/
Hope this helps someone!


Topology a first course, exercise 2 from section 3.2

Question: Let n be a positive integer. Given that the function $f: \mathbb{R} \rightarrow \mathbb{R} $ defined by $f(x)= x^n$ is continuous, show that for each $ a\geq0 $ there is exactly one $ b\geq0 $ such that $ b^n=a$.
I thought in to do for contradiction but I'm using only algebra: let $b<c$ such that $b^n=a$ and $c^n=a$, but if $0<b<c$ implies $b^n<c^n$ so we can't have $b^n=a$ and $c^n=a$, contradiction. But I not get an idea to prove this using connected sets in the real line or intermediate value theorem.
Comment: They take the continuity of $f$ as given and want to show that $x^n=a$ has a solution for all $a\geq0$. That sounds like a setup to use the intermediate value theorem. So, all one needs to do is show that there are $x_0,x_1\geq0$ such that $x_0^na$. This can be done in many ways. One way can be to exploit [Bernoulli's inequality](https://en.wikipedia.org/wiki/Bernoulli%27s_inequality) $(1+x)^n\geq 1+nx$ to get that if $x_1$ is an integer greater than $(\max(a,2)-1)/n$, then $1+nx_1>a$ and therefore $x_1^n>a$. Similarly, find $x_0$ in the form $1/m$ such that $1/aa$.
Answer: As $\lim_{x \to \infty} = \infty$, exists $d$ with $a < f(d)$.
Also $f(0) \leq a$.  Now apply IVT.
To show $f$ is strictly increasing, prove $f'$ is positive except at $0$.
Topological proof:
$A = f[[0,\infty)] = [0,\infty)$ is connected.
If $a$ not in $A$, then $A$ is disconnected,
Thus $a$ in $A$, exists $b$ with $f(b) = a$.
Exercise: What happens if the domain of $f$ is $Q$?


Parameter estimation of exponential distribution depending on multiple factors/attributes

Question: for a simulation I'll need to simulate project delays.
I've data on 20k projects with delays in quarters. Some of them were finished ahead of plan (i.e. neg. delays) others with a delay of 1-16 quarters. I assume a delay of 0 for everything finished ahead of plan as this does not get awarded in my case. The histogram looks like an exponential distribution would be a good fit.
I want to differentiate the distribution parameter (ie. lamda) depending on multiple attributes/factors of the project (3-5 factors, eg. region (south, north, west, east), project manager (X, Y, Z), type (basic, premium), etc.)). So I'm wondering what is the best approach to estimate the matrix of coefficients?
Individual estimation (for each combinations of factors):

Should I extract subsamples for each possible combination of factors and estimate the lambdas in a regular way for each (eg MLE, LSE, etc.)?
What must hold, that I could run the parametrization on each dimension (eg region) independently and then somehow cross the coefficient vectors to get the matrix? (rather then going through all combinations)
Which test would you recommend to identify the 1-2 most sensitive dimensions/factors for the delay distribution (would ANOVA work assuming an exp. distribution and 5-8 groups each dimension)?

Multi-dimensional estimation:

Any method allowing to run a multi-dimensional parametrization on the whole data set and will this be more accurate?

Many thanks for any input and ideas!
Serge
Answer: If you're familiar with R (or Python, or many other statistical packages), using a sparse.model.matrix with a general model-builder like xgboost will automatically separate out the categories for variables and give you their relative importance.
If you actually need to encode the model somewhere else instead of just making predictions on a new data set, many generalized linear models can give you coefficients, and as long as you cast your data as a sparse matrix, those individual categories will survive.
Comment: Many thanks! It's been a while since I was using R/Matlab myself but am taking a deeper look into it. Serge
Answer:
for a simulation I'll need to simulate project delays.

I've data on 20k projects with delays in quarters. Some of them were finished ahead of plan (i.e. neg. delays) others with a delay of 1-16 quarters. I assume a delay of 0 for everything finished ahead of plan as this does not get awarded in my case.

The histogram looks like an exponential distribution would be a good fit.

Presumably that's the marginal distribution you're looking at; this can be misleading, since what you're trying to discuss is the conditional distribution (conditional on the attributes you mention)
Also being quarterly, it's discrete (or, arguably interval-censored) rather than continuous. The exact zeroes may be an issue because they probably don't fit with the patter of the rest; you might perhaps need to consider zero-inflated distributions

I want to differentiate the distribution parameter (ie. lamda) depending on multiple attributes/factors of the project (3-5 factors, eg. region (south, north, west, east), project manager (X, Y, Z), type (basic, premium), etc.)). So I'm wondering what is the best approach to estimate the matrix of coefficients?

Likely a generalized linear model (or similar) with a conditional distribution suited to the fact that times will be right skew and likely heteroskedastic. Perhaps a negative binomial or a zero-inflated gamma model.

Should I extract subsamples for each possible combination of factors and estimate the lambdas in a regular way for each (eg MLE, LSE, etc.)?

No, generally it's better to do it all in one model

What must hold, that I could run the parametrization on each dimension (eg region) independently and then somehow cross the coefficient vectors to get the matrix? (rather then going through all combinations)

This won't work in general. e.g. see Simpson's paradox

Multi-dimensional estimation:
Any method allowing to run a multi-dimensional parametrization on the whole data set and will this be more accurate?

See the above suggestion.
Comment: Thank you! The zero-inflated distribution is a good path. An approach I am currently looking at:
* adjust for zeros
* for 3 factors calculate the avg. delay (only on delayed projects) for the respective combination
* smooth the means with a lin. regression (to avoid big jumps as some sub-samples are rel. small)
* take the smoothed means as MLE for the Exp- distribution


definite integral with recurence

Question: The following is taken from Cohen-Tannoudji book: "Quantum Mechanics" Vol. 1 page 679:

I don't understand how he does this integration by parts.
thanks for any help.
Answer: Set $f=u$, $df=du$, and $dg=u(1-u^2)^{l-1}du$, $g=\int u(1-u^2)^{l-1}du=-\frac1{2l}(1-u^2)^l$. Then the integral equals
$$fg\big]_{-1}^1-\int_{-1}^1 g df=
-\frac1{2l}u(1-u^2)^{l}\big]_{-1}^1+\frac1{2l}\int_{-1}^1(1-u^2)^ldu
=\frac1{2l}\int_{-1}^1(1-u^2)^ldu.$$

definite integral with recurence


Is it necessary to install Electron module and save it as devDependencies in project folder or install it globally?

Question: I'm working with Electron to build an app and I find that we can install Electron module globally by npm install -g electron instead of npm install --save-dev electron as the documentations said. I think this will reduce the app size. But I'm afraid of there will be an error when packaging the source code or while the user use the app. Can we install Electron module globally without any error and if can, will the app size reduce when we package it?
Answer: No, you need to install it locally in the package, because as you say, packaging the app will fail.
Actually, you won't even be able to build your app.
Something like what you're describing has been suggested on their Github page. See here, but I doubt they'll implement this any time soon.
Comment: Beat me by 30 seconds.
Answer: Installing Electron locally (within your project) or globally will make no difference to the size of the built application.
It is common practice to install Electron locally within your project as a dev-dependency. Doing so will prevent the need to constantly change its version number when working on different Electron applications should their Electron version numbers differ.
When your application is built, all the necessary binaries, etc are packaged up within your application. IE: Your built Electron application is self-contained.


Möbius transformation on Upper half plane

Question: If I have the function $\phi (x)= \frac{-z+i} {-iz+1} z \in \mathbb{C}  $
from D to H where $ D = { z \in \mathbb{C} | |z| <1}    $ and  H is the upper half plane
It's not that hard to see that  $\phi^{-1} (x)= \frac{z-i} {iz-1}  $
g is given as the standard metric of the upper half plane
how can i show that: $(\phi^* g)[id](z) = \frac 4 {(1-|z|^2)^2} id_{[\mathbb{R}^2]}  $
Comment: Sorry, I'm confused about your notation. Can you unpack the last line a little so I can try to work out an answer?
Comment: I think it should be the pullback of $\phi$ on g. where g the standart metric is.
Comment: Yeah, on second thought I'm not sure why that wasn't apparent to me to begin with. I'm scratching out some notes right now, hopefully I can come up with something.
Answer: The metric on the upper half-plane is given by
$$g_{|w} = \frac{\left|dw\right|^2}{\mathrm{Im}(w)^2}$$
To find the expression of the the pull-back of this metric by $\phi$ at some point $z$, just do $w = \phi(z)$ in the previous expression. You find
$$(\phi^*g)_{|z} = \frac{\left|d\phi(z)\right|^2}{\mathrm{Im}(\phi(z))^2}$$
with $$d\phi(z) = \phi'(z) dz = {2 dz \over (z+i)^2}$$
and
$$\mathrm{Im}(\phi(z)) = {1 - \left|z\right|^2 \over \left|z+i\right|^2}$$
you indeed get
$$(\phi^*g)_{|z} = \frac{4 \left|dz\right|^2}{( 1 - \left|z\right|^2)^2}$$


Nginx routes undesignated traffic to a seemingly arbitrary subdomain

Question: The problem
I have an nginx server serving 3 subdomains - say a.example.com, b.example.com, and c.example.com.
The configuration files are a.example.com.conf, b.example.com.conf, and c.example.com.conf respectively. They are stored in /etc/nginx/sites-available and soft-linked from /etc/nginx/sites-enabled.
In the configuration files, each server clause has its appropriate server_name directive. If it matters, all three subdomains use the same SSL certificate issues by Let's Encrypt (which works fine).
When b.example.com.conf is removed from sites-enabled, I expect an error message when trying to browse to it. Surprisingly, nginx redirects the traffic to a.example.com. In fact, every incoming connection without a matching server name in /etc/nginx/sites-enabled - including just the machine IP - is routed to a.example.com.
How to configure nginx to make the server directive exclusive, so that requests to b.example.com will never be served by a.example.com?
Configuration
a.example.com.conf
server {
    listen 443 ssl;
    server_name a.example.com;

    ssl_certificate     /etc/letsencrypt/live/a.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/a.example.com/privkey.pem;

    location / {
        proxy_pass http://localhost:8080;
    }
}

b.example.com.conf
server {
    listen 443 ssl;
    server_name b.example.com;

    # I'm using a multi-domain certificate called `a.example.com`, which is
    # serving a.example.com, b.example.com and c.example.com - not an error

    ssl_certificate     /etc/letsencrypt/live/a.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/a.example.com/privkey.pem;

    location / {
        proxy_pass http://localhost:8090;
    }
}

Temporary solution
I have created a default server that catches all default requests and returns error 404. However, I would like to have a more holistic solution preventing requests from any given server served by another server.
server {
    listen       443  default_server;
    server_name  _;
    return       404;

    ssl_certificate     /etc/letsencrypt/live/a.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/a.example.com/privkey.pem;
}

Comment: @MichaelHampton I've rephrased.
Answer: From nginx documentation:

In this configuration nginx tests only the request's header field "Host" to determine which server the request should be routed to. If its value does not match any server name, or the request does not contain this header field at all, then nginx will route the request to the default server for this port. In the configuration above, the default server is the first one — which is nginx's standard default behaviour.

So your temporary solution seems to be the right solution.


Organic Groups - How do I set a default role for new group members?

Question: I have created a custom group role with some custom permissions and I am trying to find a way to allocate this role to group members by default. Ideally I would like every member of every group to be allocated this role, and be able to remove this role from a user if I no longer want the particular user to have my custom permissions.
I believe OG User Roles had this functionality and user roles were put into core organic groups for 7 but I am not sure if this default roles functionality was also implemented?
Comment: OG user roles has this functionality.
Comment: I am using Drupal 7 and User Roles is only for 6, as I said above, the role functionality got added into core Organic Groups but I am unsure of the default role functionality.
Answer: I found a solution to this by using og_role_grant within hook_og_membership_insert and hard coding my role in there. This works by granting the role to a user who has just been added as a member of a group.
If I had more time I would create myself a form in the backend so it wouldn't be hard coded but for now this seems to work fine.
<code>function MODULENAME_og_membership_insert(OgMembership $og_membership) {

  if('user' == $og_membership->entity_type && is_numeric($og_membership->etid)){
    $roles = og_roles($og_membership->gid);
    foreach($roles as $key => $value){
      if ('ROLE_NAME' == $value) {
        og_role_grant($og_membership->gid, $og_membership->etid, $key);
      }
    }
  }
}
</code>
Answer: Yep, this is not provided by default, i'd spen hour understanding that by default users have no role in group where they was added. Maybe i cant find that setting... But seems like for now using hook_og_membership_insert is only solution


ModuleNotFound quanto tento importar uma biblioteca externa

Question: pip install pillow

>>> Requirement already satisfied: pillow in c:\users\miguel\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (8.3.1)

import pillow

>>> Traceback (most recent call last):
  File "C:\Users\Miguel\PycharmProjects\Testing\main\__init__.py", line 1, in <module>
    import pillow
ModuleNotFoundError: No module named 'pillow'

por algum motivo eu sempre consigo esse erro quando tento importar uma biblioteca externa, mesmo depois de baixa-la. tentei pesquisar mas sem resultados. alguém sabe o motivo desse erro?
Answer: Se a biblioteca for para manipulação de imagens o correto é:
Instalação
pip install Pillow

Nota Pillow com a letra P em maiúsculo.
Carregando
import PIL

ou
from PIL import Image


In Java can changing the value in a map

Question: In java can you change the value of a map to be a string,double, or int. For example, if all my key's are names, but my values can be numbers,names, or a decimal. A output would be something like this(bold is user input)
Greg marley
Mandy 12
John  1.8

**{Greg=marley, Mandy=12, John=1.8}**

Comment: He has not even asked anything! Yet so many answers. Lolz
Answer: Possible. Use
Map<String,Object>

But are you sure you are using this in a right way?
Answer: Your Map declaration should be:
Map<String, Object>

Example Code:
Map<String, Object> map=new HashMap<>();
map.put("Greg","marley");
map.put("Mandy",12);
map.put("John",1.8);
System.out.println(map);

Output:
{Mandy=12, John=1.8, Greg=marley}

Answer: Map<String, Object> mapName = new HashMap<>();

Key- String
Value- Object type
Answer: Yes you can ,
As the key and value both have to be Objects not primitive,
String is Obvious and Other primitives i.e int, double are Wrapped to their corresponding Wrappers ie. Integer, Double
Answer: Yes you can.
If you are using JDK 1.5 and above
Map<String, Object> map;

But for JDK 1.4 and other
Map map;

Answer: You can still use Map<String, String>.
Store int/decimal value as String and parse it back to specific type.
Answer: Use Map , in which key is String and value is type of object.. so that cast..
 Map<String, Object>

Later you can iterate the map.. If want to print..

In Java can changing the value in a map


Is there any way to see the categorical content of a specific column?

Question: If, for example, a SQL 'categorical' column measures extreme, high, medium, low or no risk, I would like to see those values as output from a SQL query. I can't find it anywhere.
Is it possible? Thankyou.
Comment: Are you looking for the values that are stored in the table?  Or are you looking for the possible values the column can take on?
Comment: The first one ! :)
Answer: <code> SELECT DISTINCT categorical
 FROM YOUR_TABLE
</code>
May be you need this one?


Are there "left-hand" pianists that keep time with their right hand?

Question: Typically keeping rhythm is a low-end job: bassists in a guitar band, and the left hand in playing the piano.  Admittedly I know nearly nothing about playing the piano.  But I thought I would ask:
Are there pianists who keep time with their right hand, and play the melody with the left, bass hand?  Are there famous piano pieces which follow this pattern?
Comment: Professional pianists are expected to be able to keep time with either hand, whatever they're playing.
Comment: By "keep time," do you mean "keep the tempo," or "play something other than the melody"?
Comment: Surely the question is if there are any _pieces_ that do this, not if there are any _performers_ do this? The musical role of the left hand and the right hand is mostly determined by the piece, not the performer. It seems a bit like asking "are there any bus drivers who drive out to Five Mile Road?" instead of "are there any bus routes that go out to Five Mile Road?"—the former question doesn't make a lot of sense, because the driver doesn't decide whether to go to Five Mile Road or not; that's determined by the route.
Comment: It's common for pianists to use their left hand rhythmically and the right hand melodically, but this has nothing to do with handedness — it's musical convention. Left-handed pianists also use their left and right hands as above, and any pianist with enough experience can reverse the relationship.
Comment: You may be interested in [this left-handed piano ad](https://linkshaenderklavier.de/Shop/product_info.php?language=en&info=p16_bluethner-modell-4-grosser-salonfluegel-fuer-linkshaender.html&=) or [this performance by a left-handed musician on a left-handed custom-built piano](https://www.classicfm.com/discover-music/instruments/piano/lefthand-piano-christopher-seed/). The video contains a few minutes of explanations by the pianist.
Answer: By "keep time in the right hand," I'm assuming you mean "play something other than the melody in the right hand." And quite a lot of composers have done this over the years! There are countless examples, so ultimately I'll just point to one: Chopin's Op. 28 B-minor Prelude.
And as an interesting aside: another type of composition, one that only uses one arm/hand, is also an important tradition. It became much more commonplace, I'm sorry to say, following World War I. Many pianists returned from the war with injured or missing arms, which led to an uptick in pieces written for just one arm. My own favorite from this time period is Ravel's "Piano Concerto for the Left Hand," but other similar works are listed on that Wikipedia page. You can watch a performance here.
Comment: @JasonPSallinger: My prototypical model of music is something like a Bach fugue, where there are 3 or 4 melodies going on at once and nothing is accompanying something else (but Bach took great care to make sure it still all fits together harmonically).  I realize your model might differ but it's important to realize there are other ways of thinking about how music is put together.
Comment: ermm… but no half decent musician thinks that way. The 'time' is in their head. They don't need to plonk along 4-to-the-bar to know that. Also, there is a far higher than average percentage of left-handed musicians compared to 'other jobs', so you'd think someone would have noticed by now if that's what they're doing.
Comment: Thank you, great answer.  To answer your question, I meant play something along the lines of quarter notes, something basic and rhythmic, while the left hand plays something intricate, which I assume in this case would be considered the melody.
Comment: @Tetsujin  Agreed.  I don't know of many, if any, compositions where the performer is playing strict quarter notes and nothing else on one hand.  What I meant is to be providing a voice similar to what a drummer might provide, or what a bassist might provide, while the other hand does the accompaniment, like a guitarist playing something heavily involved.
Comment: @AlexanderWoo - Bizarrely enough, even though Bach fugues are quintessential polyphony, I've *always* still listened to them as if they had only one melody line and multiple accompaniment lines (one of the first Bach fugue recordings I listened to was by a student who relentlessly emphasized the topmost line at all times; I still have a soft spot for Bach fugue recordings that emphasize the subject whenever it reappears).
Comment: @Dekkadeci, if you look at a partimento fugue, it's basically what your describe, subject entries only on a single staff with changing clefs, the other two or three voices to be filled in ad lib like thorough bass.
Answer: In any kind of music with regular beats, the most important factor for a performance to be perceived to be 'keeping time', is the performer's ability to play in time to their own internal sense of time, and the ability of all of the musicians within any ensemble to be able to synchronize their internal metronomes with each other.
Some styles of music, for example, boogie-woogie piano are clearly driven by the left hand patterns, but the performance is still guided by the performer's own internal sense of time. A listener, or even the performer, will perceive the left hand to be 'keeping time'.
It is possible to play a melody on the piano, unaccompanied, with either hand and still be perceived as 'keeping time'.


swiftmailer plugin does not exist

Question: I am having trouble sending emails with drupal and swiftmailer. When an email is sent it throws this error
The website encountered an unexpected error. Please try again later.

Drupal\Core\Entity\EntityStorageException: The "swiftmailer" plugin does not exist. in Drupal\Core\Entity\Sql\SqlContentEntityStorage->save() (line 829 of core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
Drupal\Core\Plugin\DefaultPluginManager->getDefinition('swiftmailer') (Line: 16)

I am running drupal 8.5 and v5.4.10 of swiftmailer/swiftmailer and 1.0.0-beta-2 of drupal/swiftmailer
I originally had a error of "class swift_message not found" and when i went to update the packages via composer, I got this error.
Answer: So it turns out after I updated it, a previous coworker disabled it from the drupal UI, hence why it wasn't working. Enabling it from the drupal UI made it work.

swiftmailer plugin does not exist


Can we extend the basis of a subspace of a tensor product space to the defining tensor product basis of the space?

Question: Suppose we have a finite-dimensional Hilbert space $\mathcal{H}_{C}\subseteq \mathcal{H}_{A}\otimes\mathcal{H}_{\overline{A}}$ which is a subspace of a tensor-product space as so. All bases I refer to are implicitly orthonormal. Choose a basis $\{|i\rangle\}$ of $\mathcal{H}_{C}$ and extend it to a basis $\{|j\rangle\}$ of $\mathcal{H}_{A}\otimes\mathcal{H}_{\overline{A}}$. By definition of a tensor product space, we know that one possible basis of $\mathcal{H}_{A}\otimes\mathcal{H}_{\overline{A}}$ is $\{|a\overline{a}\rangle\}=\{|a\rangle\otimes|\overline{a}\rangle\}$ where $\{|a\rangle\}$ and $\{|\overline{a}\rangle\}$ are bases of $\mathcal{H}_{A}$ and $\mathcal{H}_{\overline{A}}$ respectively.
My question is whether we can always choose the basis $\{|i\rangle\}$ in such a way that the basis $\{|j\rangle\}$ we extend it to will be equivalent to the defining $\{|a\overline{a}\rangle\}$ basis. I suppose an alternative phrasing would be whether we can always choose $\{|i\rangle\}$ such that each basis element $|i\rangle$ can be written as a tensor product $|i\rangle=|a\overline{a}\rangle$. My gut says no, but I can't think up a counterexample - a proof evades me too, if it is true. Any help would be much appreciated!
Comment: What is the relationship between $A$ and $\overline A$? If no relationship then it'd be better to replace $\overline A$ with $B$.
Comment: @WhatsUp This has an application to quantum erasure correction where in context, $\overline{A}$ is an erased subsystem so the notation makes a little more sense.
Answer: The answer is "no" as long as $A$ and $\overline A$ have dimension larger than $1$, so that there exist non-pure tensors.
For an example, just take any non-pure tensor $v \in A \otimes \overline A$ and let $C$ be the one dimensional subspace generated by $v$. A basis of $C$ is necessarily a constant multiple of $v$, which is not a pure tensor by construction, and hence any extension of that basis contains some non-pure tensor.
(On the existence of non-pure tensors: imagine that $A$ and $\overline A$ are $n$-dimensional spaces, so that $A\otimes \overline A$ can be identified with the space of $n \times n$ matrices. Pure tensors correspond to matrices of rank $\leq 1$, hence as long as $n \geq 2$ there exist non-pure tensors.)