PHP editing Microsoft Word document str_replace and preg_replace don't work

Question: Assume, I've got MSWord file source.doc with next content "Content of Microsoft Word file".
For example, I'd like to open it via PHP and replace word "Microsoft" to "Openoffice" and save the result into result.doc.
Here is the code using preg_replace:
$content = file_get_contents( SOMEPATH . '/source.doc' );
$new_content = preg_replace( '/Microsoft/i', 'Openoffice', $content );
file_put_contents( SOMEPATH . '/target.doc', $new_content );

Or using str_replace:
$content = file_get_contents( SOMEPATH . '/source.doc' );
$new_content = str_replace( 'Microsoft', 'Openoffice', $content );
file_put_contents( SOMEPATH . '/target.doc', $new_content );

None of them doesn't work. Code runs without any exceptions, but target.doc is the same as source.doc. Replacement not performs.
I've tried a lot of different reciepts, such as regular expression modificators, iconv and so on, but nothing helps.
var_dump of $content shows raw structure of source.doc that is full of unusual characters and as I suppose some of it stops str_replace or preg_replace scanning. Can't figure out which char is it and what should I do if I'll find it.
var_dump of $new_content is identical to $content.
Thanks forward for any help!
Comment: MS Word saves its files in a compressed format, so you won't be able to see or edit the contents without decompressing it first. But even if you do that, you'd have to know the details of the file format (there are several), and there's no guarantee that a word on the page is saved as contiguous characters in the file.
Answer: If you have a DOCX file you need to replace something in, its basically a zipped up xml archive.
Here's an example on how to replace the word "Microsoft" with "Openoffice" in a DOCX file.
$zip = new ZipArchive;
//This is the main document in a .docx file.
$fileToModify = 'word/document.xml';
$wordDoc = "Document.docx";

if ($zip->open($wordDoc) === TRUE) {
    //Read contents into memory
    $oldContents = $zip->getFromName($fileToModify);
    //Modify contents:
    $newContents = str_replace('Microsoft', 'Openoffice', $oldContents);
    //Delete the old...
    $zip->deleteName($fileToModify);
    //Write the new...
    $zip->addFromString($fileToModify, $newContents);
    //And write back to the filesystem.
    $return =$zip->close();
    If ($return==TRUE){
        echo "Success!";
    }
} else {
    echo 'failed';
}

Hope this helps!
Comment: It would be nice if you also added the source as a link to your answer.
Comment: How do you mean? Source code, or a link to some other source?
Comment: Ah, sorry. I misread the answer. I thought you found the answer from google. If that was the case, a link to the source would be a good thing. Anyway, welcome to Stack Overflow.
Comment: This works great!! But one tip, because this gave me some grief. If you use replace codes as I did (such as {MYVAL} or etc), and it doesn't change...emit the changed contents to the browser and inspect. You'll find tons of xml garbage around your replace codes. Construct your code accordingly.
Comment: how to replace many strings in this scenario? like $newContents1 = str_replace('Microsoft1', 'Openoffice1', $oldContents); $newContents2 = str_replace('Microsoft2', 'Openoffice2', $oldContents);
Comment: @Swarne27 - Just do a replace on the $newContents again. So it'd be like `$newContents = str_replace('Microsoft1', 'Openoffice1', $oldContents); $newContents = str_replace('Microsoft2', 'Openoffice2', $newContents);`
Comment: How can one save into a new file or redirect to download?
Answer: I think this is what you are looking for :) http://phpword.codeplex.com/ since doc files are not ordinary text files (try opening one with notepad..you'll get my point)
Comment: Just keep in mind that PHPWord project only allows you to open and manipulate DOCX files (compressed openXML format files). It won't work if you need to deal with the older DOC format.
Comment: http://phpword.codeplex.com/ link is broken, not opening in the browser. Thank you.


Problems with running the test in Java

Question: I have a petService class where it lists pets from a shelter and imports the pet file where it gets the id or name of the shelter through the Java class's Scanner and the name of the .csv file also through the Scanner. Then I created a test for importing pets, when running it gives me an error saying that it was not possible to find a line for the scanner.
I used the Junit and Mockito libraries to create the tests.
The ImportPets code:
public void importarPetsDoAbrigo() throws IOException, InterruptedException {
        System.out.println("Digite o id ou nome do abrigo:");
        String idOuNome = new Scanner(System.in).nextLine();

        System.out.println("Digite o nome do arquivo CSV:");
        String nomeArquivo = new Scanner(System.in).nextLine();

        BufferedReader reader = null;
        try {
            reader = new BufferedReader(new FileReader(nomeArquivo));
        } catch (IOException e) {
            System.out.println("Erro ao carregar o arquivo: " +nomeArquivo);
        }
        String line;
        while ((line = reader.readLine()) != null) {
            String[] campos = line.split(",");
            String tipo = campos[0].toUpperCase();
            String nome = campos[1];
            String raca = campos[2];
            int idade = Integer.parseInt(campos[3]);
            String cor = campos[4];
            Float peso = Float.parseFloat(campos[5]);

            Pet pet = new Pet(raca, nome, tipo, cor, idade, peso);

            String uri = "http://localhost:8080/abrigos/" + idOuNome + "/pets";
            HttpResponse<String> response = client.dispararRequisicaoPost(uri, pet);
            int statusCode = response.statusCode();
            String responseBody = response.body();
            if (statusCode == 200) {
                System.out.println("Pet cadastrado com sucesso: " + nome);
            } else if (statusCode == 404) {
                System.out.println("Id ou nome do abrigo não encontado!");
                break;
            } else if (statusCode == 400 || statusCode == 500) {
                System.out.println("Erro ao cadastrar o pet: " + nome);
                System.out.println(responseBody);
                break;
            }
        }
        reader.close();
    }

The test code:
@Test
    public void deveVerificarSeImportouOsPetsDoAbrigo() throws IOException, InterruptedException {
        String userInput = String.format("Test%spets.csv", System.lineSeparator());
        ByteArrayInputStream bais = new ByteArrayInputStream(userInput.getBytes());
        System.setIn(bais);

        when(client.dispararRequisicaoPost(anyString(), any())).thenReturn(response);
        petsService.importarPetsDoAbrigo();
        verify(client.dispararRequisicaoPost(anyString(), anyString()), atLeast(1));
    }

The error generated:
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Digite o id ou nome do abrigo:
Digite o nome do arquivo CSV:

java.util.NoSuchElementException: No line found

    at java.base/java.util.Scanner.nextLine(Scanner.java:1656)
    at br.com.alura.service.PetsService.importarPetsDoAbrigo(PetsService.java:57)
    at br.com.alura.service.PetsServiceTest.deveVerificarSeImportouOsPetsDoAbrigo(PetsServiceTest.java:31)
    at java.base/java.lang.reflect.Method.invoke(Method.java:578)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

I performed the debugging and managed to identify that the first scanner was performed successfully but the second was not. I've tried to do some things but I couldn't.
Can anyone help me?
Comment: I suspect it is related to using multiple Scanners on the same input stream (i.e. System.in).
Comment: This looks like a problem people had after upgrading Java versions in IntelliJ IDEA.  Are you using that IDE?  See https://stackoverflow.com/questions/54205486/how-to-avoid-sharing-is-only-supported-for-boot-loader-classes-because-bootstra
Comment: The code requests 2 inputs with the Scanner, but in the test I am using only one input.
I tried to duplicate the input code but it didn't work the same way.
Comment: In general, consider using `Console.readLine` to prompt for user input rather than the scanner.
Answer: I found the error!
I managed to solve it simply by changing the code in the petsService class.
I changed the scanner code to:
    Scanner scanner = new Scanner(System.in);
    System.out.println("Digite o id ou nome do abrigo:");
    String idOuNome = scanner.nextLine();

    System.out.println("Digite o nome do arquivo CSV:");
    String nomeArquivo = scanner.nextLine();

    scanner.close();

The previous code was:
    System.out.println("Digite o id ou nome do abrigo:");
    String idOuNome = new Scanner(System.in).nextLine();

    System.out.println("Digite o nome do arquivo CSV:");
    String nomeArquivo = new Scanner(System.in).nextLine();

Apparently this was what generated the error of not finding the second line of the input.
Thank you all for your help!

Problems with running the test in Java


Titanium build failed

Question: I have an app with iOS module added. I wanted some to methods to be executed so I created a proxy class with a method (id) doSomething;
I saved the changes to the module and repackaged it - so the olde version is rewritten. Now while trying to build a project I get an error:

[ERROR] :  ** BUILD FAILED **
[ERROR] :  The following build commands failed:
[ERROR] :     Ld build/Intermediates/Test.build/Debug-iphonesimulator/Test.build/Objects-normal/i386/Test normal i386
[ERROR] :   Ld build/Intermediates/Test.build/Debug-iphonesimulator/Test.build/Objects-normal/x86_64/Test normal x86_64
[ERROR] :  (2 failures)
Answer: The error most likely happens because the .framework file is not linked correctly. Ensure it is linked in Build Phases, as well as in you module.xcconfig like this:
FRAMEWORK_SEARCH_PATHS=$(SRCROOT)/../../modules/iphone/ti.googlemaps/1.0.0/platform "~/Library/Application\ Support/Titanium/modules/iphone/ti.googlemaps/1.0.0/platform"
OTHER_LDFLAGS =$(inherited) -framework Accelerate -framework AVFoundation -framework CoreBluetooth -framework CoreData -framework CoreGraphics -framework CoreLocation -framework CoreText -framework GLKit -framework ImageIO /usr/lib/libz.dylib /usr/lib/libc++.dylib /usr/lib/libicucore.dylib /usr/lib/libobjc.A.dylib -framework OpenGLES -framework QuartzCore -framework Security -framework SystemConfiguration -framework GoogleMaps

If that still does not work, please link the module you are working on.

Titanium build failed


Pegar todos valore do banco de dados

Question: Alguém poderia me ajudar a descobrir o erro:
<?php
$mysqli = new mysqli("localhost", "user",
"senha", "dbname");

if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}

$sth = $mysqli->prepare("SELECT * FROM clientes");
$sth->execute();
$result = $sth->fetchAll();

foreach($result as $item){
 echo $item->id;
}

Eu queria pegar os valores do banco e imprimi-los, tentei usando o fetchAll, porém nem consegui êxito.
Comment: Só para esclarecer, não tem `fetchAll()` como método, e nesse caso é desnecessário a utilização do `prepare` (não é errado, mas, é desnecessário, sem utilidade), até existe o `fetch_all` mas, ai a sintaxe é diferente, o ideal é como eu respondi, o dado resposta é mais coerente ao que precisa.
Comment: Bestfastfire, segue o manual oficial do PHP com o método `fetch_all()` da `mysqli` para você ver a sintaxe correta: https://php.net/manual/en/mysqli-result.fetch-all.php
Answer: O seu código inicial tem problemas de digitação e também falta comandos para funcionar, um exemplo básico do seu código é:
<?php

    $mysqli = new mysqli("localhost", "root", "senha", "dbname");

    if (mysqli_connect_errno())
    {
        printf("Connect failed: %s\n", mysqli_connect_error());
        exit();
    }

    $sth = $mysqli->prepare("SELECT * FROM clientes");
    $sth->execute();
    $result = $sth->get_result();

    $items = $result->fetch_all(MYSQLI_ASSOC);
    foreach ($items as $row)
    {
        echo $row['id'];
        echo '
';
    }

e tem que tomar cuidado porque, o fetch_all funcione a partir do PHP >= 5.3.
Mas, sinceridade o seu código não precisa de prepare (porque, nesse caso é sem utilidade, utilize quando precisar preparar paramentos para essa SQL, ou seja, a sua SQL não tem e não precisa), com o query e o seu retorno com o método fetch_assoc, é mais simples e fácil de manipular nessa SQL sem parâmetro, um exemplo funcional com outra forma:
<?php

    $mysqli = new mysqli("localhost", "user", "senha", "dbname");

    if (mysqli_connect_errno())
    {
        printf("Connect failed: %s\n", mysqli_connect_error());
        exit();
    }

    if ($sth = $mysqli->query("SELECT * FROM clientes"))
    {
        while ($row = $sth->fetch_assoc())
        {
            echo $row['id'];
        }
    }

Pegar todos valore do banco de dados


using bottom navigator sheet with bloc showing only once

Question: I am crated a float action button, on click a bottom sheet will popup. i am using a bloc with freezed library. when i tape to the float action bottom once the application hot restart the bottom sheet app appeared, when i click again on the float action button there is no action happening.
bloc code:
class NoteBloc extends Bloc<NoteEvent, NoteState> {
  NoteBloc() : super(const _Initial());

  @override
  Stream<NoteState> mapEventToState(
    NoteEvent event,
  ) async* {
    if(event is AddNoteClickedEvent){
      yield const AddNoteClickedState();
    }
  }
}

event code:
@freezed
class NoteEvent with _$NoteEvent {
  const factory NoteEvent.started() = _Started;
  const factory NoteEvent.addNoteClickedEvent() = AddNoteClickedEvent;
}

state code:
@freezed
class NoteState with _$NoteState {
  const factory NoteState.initial() = _Initial;
  const factory NoteState.addNoteClickedState() = AddNoteClickedState;
}

bottom sheet code is:
class HomePage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    GlobalKey<ScaffoldState> _scaffoldKey = GlobalKey<ScaffoldState>();
    return BlocConsumer<NoteBloc, NoteState>(
      listener: (context, state) {
        state.maybeMap(
          orElse: () {},
          addNoteClickedState: (AddNoteClickedState state) {
            return _scaffoldKey.currentState!.showBottomSheet(
              (context) => const AddNewNoteBottomSheet(),
            );
          },
        );
      },
      builder: (context, state) {
        return  Scaffold(
            key: _scaffoldKey,
            floatingActionButton: InkWell(
              onTap: () {
                BlocProvider.of<NoteBloc>(context)
                    .add(const NoteEvent.addNoteClickedEvent());
              },
              child: Icon(
                  Icons.add,
                  // Icons.save,
                  color: const Color(whiteColor),
                  size: 9.h,
        ),
      },
    );
  }
}

Answer: I usually write the code like this
class NoteBloc extends Bloc<NoteEvent, NoteState> {
  NoteBloc() : super(const _Initial());

  @override
  Stream<NoteState> mapEventToState(
    NoteEvent event,
  ) async* {
    yield* event.when(
     started: () async*{
        yield initial(),
      },
     addNoteClickedState: ()async*{
        yield addNoteClickedState();
      },
    );
  }
}

and it always work
Answer: When using freezed classes, the operator== performs deep equals, so every instance of your bloc state is equal to the previous. In such a case, a new state won't be emitted. You need your new state to be different than the previous one.


Cannot be undone because a file already exists at "tfs file path here". The file must be deleted from disk for the undo to succeed

Question: I have copied and paste that existing Uploads folder on same solution. I have renamed existing Uploads folder from "Uploads" to "Uploadsold". Then renamed that newly pasted(created) folder from "Uploads - Copy" to "Uploads".
Then I have Gated check-in my code on TFS without .csproj file. Committed was successfully.
But here after I could not check in any changes.
Whenever try to commit, cannot be undone because a file already exists at tfs file path here. The file must be deleted from disk for the undo to succeed.
Even I have rollback whole changeset but still exists same error?
Comment: What's the source control type do you use? TFVC or Git?
Comment: Hi @AndyLi-MSFT . source control is TFVC
Answer: Just try below things to narrow down the issue:

Clean the Cache folder on client computer. The folder path should be
like this: C:\Users\username\AppData\Local\Microsoft\Team Foundation\{version}\Cache.
Rename the file under the specific path to something else (e.g:
test.config  to newtest.config), then check it again.
Clone a new local Git repository if you are using Git. Create and map
a new workspace for TFVC if you are using TFVC, then try it again.

UPDATE:
For the issue in get source step, just try to set clean to true and select source directory as the clean option in your build definition, then try it again.
Seems it's related to the temp workspace which created during the build process. So, if that still not work, then try to change another agent for building or clean caches and restart the build agent server, check if that works.
Comment: I tried you said that above method.but still throw an exception of "##[error]Exit code 100 returned from process: file name 'tf', arguments 'vc undo /recursive C:\agent\_work\33\s /loginType:OAuth /login:.,*** /noprompt'."
Comment: @GowthamAlan This is another error, just try to set `clean` to `true` and select `source directory` as the clean option in your build definition, then try it again.
Comment: Hi, @AndyLi-MSFT I have just restart the agent and simply Gated Check-in from another one user's Id with different workspaces. the problem was solved.
then I got the latest version from my workspace(local machine) and can able to check-in my code from my workspace too. I don't know what causes to happened this.your instruction was great.


Correlation of motion or movement timeseries

Question: Assuming we have tracked the movement of a group of people, is it possible to detect, if one person follows another? Or if even a group of people follows one person?
To be more specific: In a crowd of approx. 500 moving people, I want to identify "leaders", which direct the movement of people around, or identify at least groups of people which behave similar.
Latter one is more a spatio-temporal clustering, for which I haven't found a working approach so far.
The usual timeseries analyses I know fail, as there are two dimensions per timepoint and individuum (X,Y coordinates).
Comment: The data is derived from 200 image frames of a video. I can extract and track successfully objects. So for each track I have a sequence of 3-tuples (X,Y,t), meaning the X,Y coordinates and time point. In total there are about 500 to 700 tracks.
Comment: Could you describe your data?
Answer: Interesting problem. You want to know if there's a leader in the movement pattern observed, correct?
Idea: Granger causality type analysis on the position and velocity time series could do the trick. If someone is always leading the position and velocity, then they could be considered the leader.


How to translate the Automated Logout module administration?

Question: How to translate language of the Automated Logout module administration?
I was searching for translation of polish language and tired other modules.
Comment: Did you try to translate strings in Drupal admin interface "admin/config/regional/translate/translate"?
Comment: Yeah its good way to do it, but i think there's should be change in autologout-7.x-4.3.pl.po
Answer: There IS a Polish translation of the autologout module available. Though it is currently only translated for 42%.
The way I understand (translate???) your question is something like so: How can I show the admin interface for the Automated Logout module in the Polish language?
To answer that question, just swich to the Polish version of your site, and then use the admin link. As an alternative, enable the "language switcher block" so that it is shown within the theme you're using for Administration. Then navigate to anywhere within admin (whatever language), and use the language switcher block to show the admin interface in Polish (for about 42%). If you also want the remaining 58% to be shown translated, then it's up to you to actually add your custom translations of the various missing translations.
Comment: It looks like that was less than 42%. There's only 5 think's translated in autologout-7.x-4.3.pl.po, i know there's licences and I got no idea i could change it.
Comment: I don't understand what you mean by that, want to retry?
Comment: Inside of this librarie, it's kind empty, or @msg not working.           :

msgid "user"
msgstr "użytkownik"
msgid "Enable"
msgstr "Włącz"
msgid "Yes"
msgstr "Tak"
msgid "No"
msgstr "Nie"
msgid "@msg"
msgstr "@msg"
Comment: Hm, accepted, merci (thank you)! So my "translation" of the translation issue was close (even correct)? Next question?


Soma de valores em duas colunas diferentes baseada em outras colunas em R

Question: Olá,
Estou tentando fazer a soma de gols feitos e gols sofridos de um conjunto de dados, alguém consegue me ajudar?
Basicamente tenho os dados da seguinte forma:
Mandante    Gols Feitos    Gols Sofridos    Visitante
Cruzeiro         0              1            Grêmio
Vitória          2              2            Flamengo
   .             .              .               .
   .             .              .               .
Fluminense       1              0            Cruzeiro

Quero então fazer uma tabela, como por exemplo:
Time       Gols Feitos    Gols Sofridos
Cruzeiro        0              2
Vitória         2              2
Fluminense      1              0

A pergunta se resume a, como somar o número de gols feitos quando mandante e o número de gols "sofridos" quando é visitante. Isso é relativamente simples fazer para um time, mas quero algo que sirva para todos os times. Alguma sugestão?
Amostra dos dados:
structure(list(Mandante = c("Cruzeiro", "Vitória", "Santos",
"América-MG", "Internacional", "Corinthians", "Vasco", "Atlético-
PR","São Paulo", "Botafogo", "Bahia", "Flamengo", "Fluminense", "Paraná",
"Ceará", "Chapecoense", "Palmeiras", "Atlético-MG", "Grêmio",
"Sport"), `Gols feitos` = c(0, 2, 2, 3, 2, 2, 2, 5, 1, 1, 1,
2, 1, 0, 0, 1, 1, 2, 0, 1), `Gols sofridos` = c(1, 2, 0, 0, 0,
1, 1, 1, 0, 1, 0, 0, 0, 4, 0, 1, 0, 1, 0, 1), Visitante = c("Grêmio",
"Flamengo", "Ceará", "Sport", "Bahia", "Fluminense", "Atlético-MG",
"Chapecoense", "Paraná", "Palmeiras", "Santos", "América-MG",
"Cruzeiro", "Corinthians", "São Paulo", "Vasco", "Internacional",
"Vitória", "Atlético-PR", "Botafogo")), row.names = c(NA, -20L
), class = c("tbl_df", "tbl", "data.frame"))

Answer: Aqui vai a minha sugestão
Havia um problema com o nome do Atlético-PR (tinha um \n no meio).
> test <- structure(list(Mandante = c("Cruzeiro", "Vitória", "Santos",
+ "América-MG", "Internacional", "Corinthians", "Vasco", "Atlético-PR","São Paulo", "Botafogo", "Bahia", "Flamengo", "Fluminense", "Paraná",
+ "Ceará", "Chapecoense", "Palmeiras", "Atlético-MG", "Grêmio",
+ "Sport"), `Gols feitos` = c(0, 2, 2, 3, 2, 2, 2, 5, 1, 1, 1,
+ 2, 1, 0, 0, 1, 1, 2, 0, 1), `Gols sofridos` = c(1, 2, 0, 0, 0,
+ 1, 1, 1, 0, 1, 0, 0, 0, 4, 0, 1, 0, 1, 0, 1), Visitante = c("Grêmio",
+ "Flamengo", "Ceará", "Sport", "Bahia", "Fluminense", "Atlético-MG",
+ "Chapecoense", "Paraná", "Palmeiras", "Santos", "América-MG",
+ "Cruzeiro", "Corinthians", "São Paulo", "Vasco", "Internacional",
+ "Vitória", "Atlético-PR", "Botafogo")), row.names = c(NA, -20L
+ ), class = c("tbl_df", "tbl", "data.frame"))
>

Fiz uma dataframe
> test <- data.frame(test)
> head(test)
       Mandante Gols.feitos Gols.sofridos  Visitante
1      Cruzeiro           0             1     Grêmio
2       Vitória           2             2   Flamengo
3        Santos           2             0      Ceará
4    América-MG           3             0      Sport
5 Internacional           2             0      Bahia
6   Corinthians           2             1 Fluminense

Fiz 2 data frame com os visitantes e mandantes, ambos com a mesma ordem de nomes:
> a <- test[,-4]
> a <- a[with(a, order(Mandante)), ]
>
> b <- test[,-1]
> b <- b[with(b, order(Visitante)), ]

Confirmar que os nomes estão na mesma ordem:
> all.equal(b$Visitante, a$Mandante)
[1] TRUE

E no final é somar os golos marcados e sofridos:
> head(a)
      Mandante Gols.feitos Gols.sofridos
4   América-MG           3             0
18 Atlético-MG           2             1
8  Atlético-PR           5             1
11       Bahia           1             0
10    Botafogo           1             1
15       Ceará           0             0
> head(b)
   Gols.feitos Gols.sofridos   Visitante
12           2             0  América-MG
7            2             1 Atlético-MG
19           0             0 Atlético-PR
5            2             0       Bahia
20           1             1    Botafogo
3            2             0       Ceará
> a[,2] <- a[,2]+b[,2]
> head(a)
      Mandante Gols.feitos Gols.sofridos
4   América-MG           3             0
18 Atlético-MG           3             1
8  Atlético-PR           5             1
11       Bahia           1             0
10    Botafogo           2             1
15       Ceará           0             0
> a[,3]<- a[,3]+b[,1]
> head(a)
      Mandante Gols.feitos Gols.sofridos
4   América-MG           3             2
18 Atlético-MG           3             3
8  Atlético-PR           5             1
11       Bahia           1             2
10    Botafogo           2             2
15       Ceará           0             2

E agora fiz a tabela final:
> final <- a
> colnames(final) <- c("Time", "Gols Feitos", "Gols Sofridos")
> head(final)
          Time Gols Feitos Gols Sofridos
4   América-MG           3             2
18 Atlético-MG           3             3
8  Atlético-PR           5             1
11       Bahia           1             2
10    Botafogo           2             2
15       Ceará           0             2

Espero que ajude.


`runtime/trace` viewer on browser

Question: when i run go tool trace m.trace
it opens this link http://127.0.0.1:37721/trace
on both chrome & firefox i go an empty page
in chrome Console i found
this error

GET http://127.0.0.1:37721/trace_viewer_html 404 (Not Found)
  Uncaught ReferenceError: tr is not defined

i found this question talk about issue in firefox and chrome version
i use ubuntu16.04/chrome66.0/golang1.10

so what should i do to be able to view the trace
is there is an online tool could i upload my m.trace file to browse online
Answer: the problem was "I install Go from your OS package manager"
after reinstalling Go following official instructions the problem solved
you can find more in issue created with golang repo here

`runtime/trace` viewer on browser


Error: could not find function "filterStream", any suggestion on why this error?

Question: I am trying to run the below code, that I found in the example of CRAN package streamR link here, but am getting an error when running it, can anyone tell me why this error and how can I resolve this. Thanks for your help.
## Not run:
## An example of an authenticated request using the ROAuth package,
## where consumerkey and consumer secret are fictitious.
## You can obtain your own at dev.twitter.com
library(ROAuth)
requestURL <- "https://api.twitter.com/oauth/request_token"
accessURL <- "https://api.twitter.com/oauth/access_token"
authURL <- "http://api.twitter.com/oauth/authorize"
consumerKey <- "xxxxxxxxxxxxxx"
consumerSecret <- "xxxyyyyyyyyyyyzzzzzzz1111111122222222"
my_oauth <- OAuthFactory$new(consumerKey=consumerKey, consumerSecret=consumerSecret, requestURL=requestURL, accessURL=accessURL, authURL=authURL)
my_oauth$handshake(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl"))
filterStream( file="tweets_rstats.json",track="rstats", timeout=3600, oauth=my_oauth )
## capture 10 tweets mentioning the "Rstats" hashtag
filterStream( file.name="tweets_rstats.json", track="rstats", tweets=10, oauth=my_oauth )
## capture tweets published by Twitter's official account
filterStream( file.name="tweets_twitter.json", follow="783214", timeout=600, oauth=my_oauth )
## capture tweets sent from New York City in Spanish only, and saving as an object in memory
tweets <- filterStream( file.name="", language="es",locations=c(-74,40,-73,41), timeout=600, oauth=my_oauth )
## capture tweets mentioning the "rstats" hashtag or sent from New York City
filterStream( file="tweets_rstats.json", track="rstats",locations=c(-74,40,-73,41), timeout=600, oauth=my_oauth )
## End(Not run)

Comment: You need a package called streamR. Install streamR and then call filterStream
Answer: May be the first filterStream, after "my_oauth$handshake"
filterStream( file="tweets_rstats.json",track="rstats", timeout=3600, oauth=my_oauth )

should be instead:
filterStream( file.name="tweets_rstats.json",track="rstats", timeout=3600, oauth=my_oauth )

Comment: you load also another library `library(streamR)` (this is what a see at this [link](http://pablobarbera.com/blog/archives/1.html)
Comment: Nope doesn't work, please check the link!
Answer: Did you register a Twitter application on: Twitter Apps?
If so, did you set the callback URL on: http://127.0.0.1:1410?
What type of error did you get? I'm thinking about some problems with the authentication process, filterStream needs to pass the my_oauth to get authenticate and initiate the download.


Derivation of Hessian for multinomial logistic regression in Böhning (1992)

Question: This question is basically about row/column notation of derivatives and some basic rules. However, I couldn't figure out where I'm wrong.
For multinomial logistic regression, I'm trying to get the following observed information given in Böhning's paper [1]:
$$
-\nabla^2L=\begin{pmatrix}
p_1(1-p_1)\mathbf{xx}^T & -p_1p_2\mathbf{xx}^T & \dots & -p_1p_k\mathbf{xx}^T \\ \vdots & p_2(1-p_2)\mathbf{xx}^T & & \vdots \\
-p_kp_1\mathbf{xx}^T & \dots & & p_k(1-p_k)\mathbf{xx}^T
\end{pmatrix}
$$
where $p_i=p(y_i=1|\mathbf{x})=\frac{\exp(w_i^T\mathbf{x})}{\sum_{j=1}^K\exp{w_j^T\mathbf{x}}}$ for $i=1,...,K$, $\mathbf{x}$ is a single observation (a column vector of size $d$), $\omega_i$ is the parameter vector of $i$th class (a column vector of size $d$). $y_i \in \lbrace 0, 1 \rbrace$, which is the class label.
I can write the log-likelihood (for a single observation) of as follows:
$$
\mathcal{L}=\sum_{k=1}^K y_{k}\omega_k^T\mathbf{x}-\ln\sum_{j=1}^K\exp{(\omega_j^T\mathbf{x})}
$$
According to Matrix Cookbook, $\frac{\partial \omega_k^T \mathbf{x}}{\partial \omega_k}=\mathbf{x}$. By using this, I wrote the following first and second derivatives:
$$
\frac{\partial \mathcal{L}}{\partial \omega_k}=\left( y_{k} -\frac{\exp(\omega_k^T\mathbf{x})}{\sum_{j=1}^K\exp{(\omega_j^T\mathbf{x})}} \right)\mathbf{x}
\\
\frac{\partial \mathcal{L}}{\partial \omega_k \partial \omega_k}=-\left( \frac{\exp(\omega_k^T\mathbf{x})}{\sum_{j=1}^K\exp{(\omega_j^T\mathbf{x})}}-\frac{(\exp(\omega_k^T\mathbf{x}))^2}{(\sum_{j=1}^K\exp{(\omega_j^T\mathbf{x})})^2} \right)\mathbf{x} \mathbf{x}
$$
So instead of $\mathbf{xx}^T$, I get $\mathbf{xx}$. How can I correct this?
In some sources like this one the second derivative is defined as $\frac{\partial^2\mathcal{L}}{\partial\omega_k\partial\omega_k^T}$. Is this the correct notation? (Even if it is, I still can't correct my derivatives.)
[1] D. Böhning, "Multinomial Logistic Regression Algorithm", Ann. Inst. Statist. Math, vol. 44, No. 1, 197-200, 1992.
Comment: Because dimensionality of $\mathbf{x}$ is not specified, assume it's a $d$-dimensional column vector for sake of argument. Concerning your question and specifically 2nd derivatives, *what operation do you have in mind when you write $\mathbf{x} \mathbf{x}$?*
Comment: For the avoidance of doubt, please may you clarify what follows because you have used notation that departs from the paper. 1\. The dimensionality of $\omega_k$ and $y_k$. 2. Am I correct in understanding that you are specifying *gradients* (i.e. derivatives of a scalar function with respect to a vector), rather than derivatives with respect to a scalar? Because I think this is where the source of the misunderstanding lies.
Comment: Thank you for clarifying and editing. In response, can you see why the notation you've chosen, as it stands, is problematic, to the extent, I suspect, of compounding confusion? Because you state that $\omega_k$ is a vector, but then simultaneously refer to each element of the Hessian as $\partial \mathcal{L} / \partial \omega_k \partial \omega_k$ suggesting that each $\omega_k$ is an element, not a vector.
Comment: @microhaus You're right, according to the paper, $p_i(1-p_i)\mathbf{xx}^T$ should be a matrix, so it can't be a single element of the Hessian. However, I still can't see why it is $\mathbf{xx}^T$. $\frac{\partial\exp(\omega_k^T\mathbf{x})}{\partial \omega_k}=\exp(\omega_k^T\mathbf{x})\mathbf{x}$, right? But when deriving the Hessian, the second $\mathbf{x}$ has a $T$ over it although the term we're taking the derivative of is the same thing.
Comment: @microhaus $\mathbf{xx}$ is meaningless, but I can't show why it should be $\mathbf{xx}^T$. Why the outer product but not inner product? Should I just say "Hessian should be a matrix of dimensions $dK \times dK$, so let me transpose the last vector."?
Comment: @microhaus Yes, I'm trying to calculate gradients and the elements of the Hessian matrix. I've tried to correct the notation. Since $\omega_k$ is the parameter vector of class $k$, and $\mathbf{x}$ is the observed sample vector, their inner product is a scalar, so it is the derivative of a scalar function with respect to a vector. But the second one is an element of the Hessian on the diagonal (since $\partial \omega_k\partial\omega_k$).
Answer: The source of the issue in my view comes from a confusion concerning dimensionality, and because the Hessian departs from the usual context in that there is sub-partitioning going on. Other than a mild bit of calculus, this is just routine linear algebra with some tricks like the Kronecker delta function and the Kronecker product to compress the notation.
I have no doubt there may be a quicker and cleaner solution to this using a suitable selection of recipes from the Matrix Cookbook, or Einstein notation. But I've generally found that unless it's immediately obvious which recipe you should be applying, then there is no harm in building the condensed abstractions presented piece-by-piece from the ground up, particularly when you are in doubt.
For this reason, I will derive the relevant partial derivatives, and show how these can be stacked into the data-structures (gradient and Hessian) whose expressions you are seeking to validate.
Notation and dimensionality.
$\mathbf{x} \in \mathbb{R}^D$ is an observation vector.
$\mathbf{y} \in \mathbb{R}^K$ is also a vector.
$\omega^{(k)} \in \mathbb{R}^D$ for $k = 1, \dots, K$ are parameter vectors., where
$$\omega^{(k)} =
\begin{bmatrix}
\omega_1^{(k)} \\
\vdots \\
\omega_D^{(k)}
\end{bmatrix}$$
$\omega \in \mathbb{R}^{DK}$ is a $DK$-dimensional column vector, consisting of all of the column parameter vectors $\omega^{(1)}, \dots, \omega^{(K)}$ stacked vertically on top of one another. Where for each element $\omega_d^{(k)}$, the superscript index $k$ indexes the $k$th class, and the subscript index $d$ indexes the covariate element.
The log-likelihood $\mathcal{L}(\omega)$ is a scalar function, in that $\mathcal{L} : \mathbb{R}^{DK} \rightarrow \mathbb{R}$. For a single observation $\mathbf{y}$, the log-likelihood is
$$\mathcal{L}(\omega) = \left( \sum^K_{k=1} y_k {\omega^{(k)}}^T \mathbf{x} \right) - \ln \left(1 + \sum^K_{j=1} \exp( {\omega^{(j)}}^T \mathbf{x})  \right).$$
Computing the score vector/gradient $\nabla_{\omega} L(\omega)$.
Note that the gradient $\nabla_{\omega} L(\omega) \in \mathbb{R}^{DK}$ is a $DK$-dimensional column vector.
Each element is the partial derivative of $\mathcal{L}$ with respect to $w_d^{(k)}$. Using dummy indexes $c$ and $l$ to avoid confusion, we have that
\begin{align}
\frac{\partial \mathcal{L}}{\partial \omega_c^{(l)}} &= \frac{\partial}{\partial w_c^{(l)}} \left\{ \left( \sum^K_{k=1} y_k {\omega^{(k)}}^T \mathbf{x} \right) - \ln \left[1 + \sum^K_{j=1} \exp( {\omega^{(j)}}^T \mathbf{x})  \right] \right\} \\
&= y_l \cdot  \frac{\partial}{\partial w_c^{(l)}} {\omega^{(l)}}^T \mathbf{x} - \frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})} \cdot \frac{\partial}{\partial w_c^{(l)}} {\omega^{(l)}}^T \mathbf{x} \\
&= y_l x_c - \frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})} x_c \\
&= \left[y_l - \frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})} \right] x_c \\
&= (y_l - \hat{p}_l) x_c,
\end{align}
where
$$\hat{p}_l = \frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})}.$$
Writing out the gradient, we have that:
$$\begin{align}
\nabla_{\omega} \mathcal{L}(\omega) =
\begin{bmatrix}
(y_1 - p_1)x_1 \\
\vdots \\
(y_1 - p_1) x_D \\
(y_2 - p_2) x_1 \\
\vdots \\
(y_2 - p_2) x_D \\
\vdots \\
(y_K - p_K) x_1 \\
\vdots \\
(y_K - p_K) x_D \\
\end{bmatrix}
=
\begin{bmatrix}
(\mathbf{y} - \hat{\mathbf{p}})_1 \mathbf{x} \\
(\mathbf{y} - \hat{\mathbf{p}})_2 \mathbf{x} \\
\vdots \\
(\mathbf{y} - \hat{\mathbf{p}})_K \mathbf{x} \\
\end{bmatrix}
= (\mathbf{y} - \hat{\mathbf{p}}) \otimes \mathbf{x}.
\end{align}$$
Where $(\mathbf{y} - \hat{\mathbf{p}}) \otimes \mathbf{x}$ is the Kronecker product of the "matrices" $(\mathbf{y} - \hat{\mathbf{p}}) \in \mathbb{R}^{K \times 1}$ and $\mathbf{x} \in \mathbb{R}^{D \times 1}$, resulting in a $DK$-dimensional column vector.
Computing the observed information matrix/negative Hessian $-\nabla^2 \mathcal{L}(\omega)$.
Because the gradient $\nabla_{\omega} L(\omega) \in \mathbb{R}^{DK}$, the Hessian $-\nabla^2 \mathcal{L}(\omega) \in \mathbb{R}^{DK \times DK}.$
Each element of the Hessian, which itself consists of $D^2K^2$ elements is given by
\begin{align}
-\frac{\partial^2 \mathcal{L}}{\partial \omega_{c'}^{(l')} \partial \omega_c^{(l)}} &= -\frac{\partial}{\partial \omega_{c'}^{(l')}} \frac{\partial \mathcal{L}}{\partial \omega_c^{(l)}} \\
&= -\frac{\partial}{\partial \omega_{c'}^{(l')}}  \left[y_l x_c - \frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})} x_c \right].\\
\end{align}
The left-hand term in the square brackets vanishes, and so the above simplifies to
$$-\frac{\partial^2 \mathcal{L}}{\partial \omega_{c'}^{(l')} \partial \omega_c^{(l)}} = \frac{\partial}{\partial \omega_{c'}^{(l')}} \left[\frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})} x_c \right].$$
For a function $f = g / h$, the quotient rule gives $f' = (g'h - gh') / h^2$, where $f'$, $g'$, and $h'$ represent partial derivatives.
Now in the case where $l' \neq l$, where $l'$ does not refer to a partial derivative, we have that the partial derivative $g'= 0$, giving
\begin{align}
-\frac{\partial^2 \mathcal{L}}{\partial \omega_{c'}^{(l')} \partial \omega_c^{(l)}} &= \frac{-\exp({\omega^{(l)}}^T \mathbf{x}) x_c \cdot \exp({\omega^{(l')}}^T \mathbf{x}) x_{c'}}{\left[1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x}) \right]^2}.
\end{align}
When $l' = l$, then we have that
\begin{align}
-\frac{\partial^2 \mathcal{L}}{\partial \omega_{c'}^{(l')} \partial \omega_c^{(l)}} &= \frac{\exp({\omega^{(l)}} ^T \mathbf{x}) x_c x_{c'} \cdot  [1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x}) ]- \exp({\omega^{(l)}}^T \mathbf{x})x_c \cdot \exp({\omega^{(l')}}^T \mathbf{x})x_{c'}}{\left[1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x}) \right]^2} \\
&= \left( \frac{\exp( {\omega^{(l)}}^T \mathbf{x})}{1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x})} - \frac{\exp({\omega^{(l)}}^T \mathbf{x}) \cdot \exp({\omega^{(l')}}^T \mathbf{x})}{\left[1 + \sum^K_{j=1} \exp({\omega^{(j)}}^T \mathbf{x}) \right]^2} \right) x_c x_{c'}. \\
\end{align}
The author compresses the cases where $l' \neq l$ and $l' = l$ using the Kronecker delta function, meaning that both cases above can be written generally as
$$-\frac{\partial^2 \mathcal{L}}{\partial \omega_{c'}^{(l')} \partial \omega_c^{(l)}} = (\delta(l, l') \hat{p}_l - \hat{p}_l \hat{p}_{l'}) x_c x_{c'}.$$
With a view to stacking these $D^2K^2$ elements into a $DK \times DK$ Hessian $- \nabla^2 \mathcal{L}$, note that the outer product $\mathbf{x} \mathbf{x}^T$ is a $D \times D$ matrix:
$$\mathbf{x} \mathbf{x}^T =
\begin{bmatrix}
x_1 \\
\vdots \\
x_D \\
\end{bmatrix}
\begin{bmatrix}
x_1 \cdots x_D
\end{bmatrix}
=
\begin{bmatrix}
x_1^2 & x_1 x_2 &\cdots &x_1 x_D \\
x_2 x_1 & x_2^2 &\cdots &x_2 x_D \\
\vdots & \vdots &\ddots & \vdots \\
x^D x_1 & x_D x_2 &\cdots  &x_D^2 \\
\end{bmatrix}$$
And also note that if we cycle through $l = 1, \dots K$ and $l' = 1, \dots K$, storing each element $(\delta(l, l') \hat{p}_l - \hat{p}_l \hat{p}_{l'})$ into a $K \times K$ matrix, which we can refer to as $\Lambda_{\hat{\mathbf{p}}} - \hat{\mathbf{p}} \hat{\mathbf{p}}^T$, then:
$$\Lambda_{\hat{\mathbf{p}}} - \hat{\mathbf{p}} \hat{\mathbf{p}}^T =
\begin{bmatrix}
\hat{p}_1( 1 - \hat{p}_1) & -\hat{p}_1 \hat{p}_2 &\cdots &-\hat{p}_1 \hat{p}_K \\
-\hat{p}_2 \hat{p}_1 & \hat{p}_2 (1 - \hat{p}_2)  &\cdots & -\hat{p}_2 \hat{p}_K \\
\vdots & \vdots &\ddots & \vdots \\
-\hat{p}_K \hat{p}_1 & -\hat{p}_K \hat{p}_2  &\cdots & \hat{p}_K (1 -\hat{p}_K) \\
\end{bmatrix}$$
We can now think of populating the $DK \times DK$ Hessian matrix one $D \times D$ sub-matrix at a time. Indexing a total of $K^2$ individual $D \times D$ sub-matrices with a co-ordinate $(i, j)$, where $i = 1, \dots, K$ and $j = 1, \cdots, K$, the $(i,j)$-th $D \times D$ submatrix is
$$(\Lambda_{\hat{\mathbf{p}}} - \hat{\mathbf{p}} \hat{\mathbf{p}}^T)_{ij} \mathbf{x} \mathbf{x}^T.$$
Populating our Hessian with a total of $K^2$ submatrices, and using the definition of the Kronecker product to compress the notation we have that:
\begin{align}
- \nabla^2 \mathcal{L} &=
\begin{bmatrix}
\hat{p}_1( 1 - \hat{p}_1) \mathbf{x} \mathbf{x}^T & -\hat{p}_1 \hat{p}_2 \mathbf{x} \mathbf{x}^T &\cdots &-\hat{p}_1 \hat{p}_K \mathbf{x} \mathbf{x}^T \\
-\hat{p}_2 \hat{p}_1 \mathbf{x} \mathbf{x}^T & \hat{p}_2 (1 - \hat{p}_2) \mathbf{x} \mathbf{x}^T  &\cdots & -\hat{p}_2 \hat{p}_K \mathbf{x} \mathbf{x}^T \\
\vdots & \vdots &\ddots & \vdots \\
-\hat{p}_K \hat{p}_1 \mathbf{x} \mathbf{x}^T & -\hat{p}_K \hat{p}_2 \mathbf{x} \mathbf{x}^T  &\cdots & \hat{p}_K (1 -\hat{p}_K) \mathbf{x} \mathbf{x}^T \\
\end{bmatrix}
= (\Lambda_{\hat{\mathbf{p}}} - \hat{\mathbf{p}} \hat{\mathbf{p}}^T) \otimes \mathbf{x} \mathbf{x}^T.\\
\end{align}
Where the final equality is a Kronecker product of the matrices $(\Lambda_{\hat{\mathbf{p}}} - \hat{\mathbf{p}} \hat{\mathbf{p}}^T) \in \mathbb{R}^{K \times K}$ and $\mathbf{x}\mathbf{x}^T \in \mathbb{R}^{D \times D}.$


How to perform rapid notes with the pinky finger on piano?

Question: I'm playing a piece on piano which contains the following two-voice rhythm for the right hand:
.
The piece is at a relatively rapid tempo, around 120 BPM. I currently finger the chord as 1-2-3-5 and play the fast notes with the pinky finger.
However, I'm struggling to perform this rhythm at speed. Since I need to hold down the lower notes from the chord with my 1-2-3 fingers, I find myself using purely my pinky finger for the repeats, which is quite weak and slow.
One thing I've tried was to rotate my arm and wrist so that the pinky is as low as possible on its key and it helps a bit, but with my thumb on a black key, I have to rotate a fair amount, to a point of some discomfort.
Is there a better way to handle this part?
Comment: Please add the left hand part to the image.
Comment: Would it ruin the sound to use the pedal? Also, can you reach the B with finger 4?
Comment: It's not very pianistic. Is it a reduction? What's the left hand doing? If it's occupied then you do need a middle pedal to sustain the chord. I personally would look the musical director defiantly in the eye while _omitting the first B_ but playing the others very crisply, using 5, 1, 3, 5
Answer: I see no purpose in holding down the lower three notes, when they could be pedalled - either with sustain of sostenuto pedals. That then leaves the pinky free to play the top note each time, or even move the hand across and use the stronger ring finger. Using a pedal will hardly affect the top notes, as they all get played in time.
Comment: Holding the damper pedal while playing notes will start a big humming sympathetic resonance. Sometimes it's just what the piece needs but other times not. If it was intended, there would be a pedal marking.
Comment: @ojs - it would with the sustain pedal, but not with the sostenuto. One could always half-pedal, depending on what else is going on, which it seems we're not party to.
Comment: that's why I specifically mentioned damper pedal. But there are a lot of pianos without sostenuto pedal out there.
Comment: @ojs - seems to depend on which side of the Atlantic one is.


Partial derivative $\frac{\partial}{\partial x_i} ( {\bf x}-{\bf c})^T A ( {\bf x}-{\bf c})$

Question: How to find the following partial derivative
\begin{align}
\frac{\partial}{\partial x_i} ( {\bf x}-{\bf c})^T A  ( {\bf x}-{\bf c}),
\end{align}
where ${\bf x}$ is a vector of length $n$ and  $A$ is a strictly positive definite matrix, and  ${\bf c}$ is just a fixed vector.
My plan was to re-write it in terms of a double sum and take the derivative.
Answer: I like the approach of using the definition of the partial derivative.  In particular,
$$
\frac{\partial}{\partial x_i} f(\mathbf x) = \lim_{t \to 0} \frac{f(\mathbf x + t\mathbf e_i) - f(\mathbf x)}{t}
$$
where $\mathbf e_i$ denotes the $i$th canonical basis vector.  With that, we have
$$
f(\mathbf x + t\mathbf e_i) =
([\mathbf x - \mathbf c] + t\mathbf e_i )^TA([\mathbf x - \mathbf c] + t\mathbf e_i)\\
= f(\mathbf x) + t[(\mathbf x - \mathbf c)^TA\mathbf e_i + \mathbf e_i^T A(\mathbf x - \mathbf c)] + t^2 \mathbf e_i^T A \mathbf e_i.
$$
Notably, we have
$$
[(\mathbf x - \mathbf c)^TA\mathbf e_i]^T = \mathbf e_i^T A^T(\mathbf x - \mathbf c)
$$
and any scalar is equal to its own transpose.  So, the above simplifies to
$$
f(\mathbf x + t\mathbf e_i) =
f(\mathbf x) + t[\mathbf e_i^T (A + A^T)(\mathbf x - \mathbf c)] + t^2 \mathbf e_i^T A \mathbf e_i.
$$
Comment: Can you explain how you are getting the term  $te_i(A+A^T) (x-c)$? I am getting $t (e_i^TA(x-c)+(x-c)^T Ae_i)$. Are these two equal?
Comment: Yes, they are equal.  What I tried to communicate in my answer is that
$$
t(e_i^TA(x-c) + (x-c)^TAe_i) = \\
t(e_i^TA(x-c) + e_i^TA^T(x-c)) =\\
te_i(A + A^T)(x-c)
$$
Comment: I did forget the transpose on the left side a few times; maybe that confused you.  The current version is correct.
Comment: Thank you. Final quation.  Should there be a transpose on $e_i$ in the last equation?
Comment: Yes there should be


filtering bad words from text

Question: This function filer the email from text and return matched pattern
  function parse($text, $words)
  {
    $resultSet = array();
    foreach ($words as $word){
      $pattern = 'regex to match emails';
      preg_match_all($pattern, $text, $matches, PREG_OFFSET_CAPTURE );
      $this->pushToResultSet($matches);
    }
    return $resultSet;
  }

Similar way I want to match bad words from text and return them as $resultSet.
Here is code to filter badwords
TEST HERE
$badwords = array('shit', 'fuck'); // Here we can use all bad words from database
$text = 'Man, I shot this f*ck, sh/t! fucking fu*ker sh!t f*cking  sh\t ;)';
echo "filtered words
";
echo $text."
";
$words = explode(' ', $text);
foreach ($words as $word)
    {
        $bad= false;
        foreach ($badwords as $badword)
            {
                if (strlen($word) >= strlen($badword))
                {
                    $wordOk = false;
                    for ($i = 0; $i < strlen($badword); $i++)
                    {
                        if ($badword[$i] !== $word[$i] && ctype_alpha($word[$i]))
                        {
                            $wordOk = true;
                            break;
                        }
                    }
                    if (!$wordOk)
                    {
                        $bad= true;
                        break;
                    }
        }
            }
            echo $bad ? 'beep ' : ($word . ' '); // Here $bad words can be returned and replace with *.
    }

Which replaces badwords with beep
But I want to push matched bad words to $this->pushToResultSet() and returning as in first code of email filtering.
can I do this with my bad filtering code?
Comment: [Related](http://stackoverflow.com/q/273516/1578604)
Comment: Let me share [this gem](http://stackoverflow.com/questions/6096634/regular-expression-preg-quote-symbols-are-not-detected/6099598#6099598) with you.
Answer: Roughly converting David Atchley's answer to PHP, does this work as you want it to?
$blocked = array('fuck','shit','damn','hell','ass');
$text = 'Man, I shot this f*ck, damn sh/t! fucking fu*ker sh!t f*cking  sh\t ;)';
$matched = preg_match_all("/(".implode('|', $blocked).")/i", $text, $matches);
$filter = preg_replace("/(".implode('|', $blocked).")/i", 'beep', $text);
var_dump($filter);
var_dump($matches);

Comment: Ha! Son of a gun, not sure why I did that in Javascript.  Late night.
Comment: Haha we've all been there :). At least now there's two solutions :P.
Answer: JSFiddle for working example.
Yes, you can match bad words (saving for later), replace them in the text and build the regex dynamically based on an array of bad words you're trying to filter (you might store it in DB, load from JSON, etc.).  Here's the main portion of the working example:
var blocked = ['fuck','shit','damn','hell','ass'],
    matchBlocked = new RegExp("("+blocked.join('|')+")", 'gi'),
    text = $('.unfiltered').text(),
    matched = text.match(matchBlocked),
    filtered = text.replace(matchBlocked, 'beep');

Please see the JSFiddle link above for the full working example.

filtering bad words from text


Suitable way to hide a paragraph entity

Question: I have read Hide an entity with hook_entity_view_alter(), and by all rights this code should hide every paragraph on a page:
/**
 * Implements hook_ENTITY_TYPE_view_alter().
 */
function mymodule_paragraph_view_alter(array &$build, EntityInterface $entity, EntityViewDisplayInterface $display) {
  $build['#access'] = false;
}

There are no errors, I know the function is being called because the debugger stops there on a breakpoint, I've cleared cache with every test.
I've tried hook_entity_view, hook_entity_view_alter, hook_ENTITY_TYPE_view and hook_ENTITY_TYPE_view_alter.
None of them have any effect whether I'm logged in or not.
['#access'] is not showing up in the build array in any of these hooks which makes me think the advice might be out of date since 8.4 dropped? If it does work - why should it work?
Are there any other ways, better ways of hiding paragraph entities (or any entity really)?
Obviously there will be some logic involved later which is why I want to do it in one of these hooks (or similar).

Edit 1: It has been suggested that there should be a $build['content'] array or similar. There is only the entity object in whichever hook I try of the four I have listed:
Comment: Inspect the `$build` variable. Are you sure it shouldn't be something like `$build['content']['#access'] = false`?
Comment: I'd try try adding that #access property everywhere until something sticks. Maybe to #paragraph? Maybe the each field separately? If all else fails you could always try unsetting stuff?
Comment: Hehehe, yeah... that's why I'm here, I tried that already =)
Answer: Try this:
$build['#printed'] = TRUE

Answer: $build['#printed'] = TRUE works but is heavily cached..you can set your cache though or still try
$build['body']['0']['#access'] = FALSE; OR
$build['filter_this_view']['actions']['submit']['#access'] = FALSE;
depending on your $build array structure. The fields will not be cached
Answer: I know it didn't work for you, but I am using  mymodule_comment_view_alter() (i.e. hooking into hook_ENTITY_TYPE_view_alter) to selectively hide individual comments.
Although $build didn't have an '#access' key, eventually I tried
$build['#access'] = false;

and that does work, for comments anyway.
Also I don't need to clear the cache at all whilst editing the hook (I do have Twig debugging on it local.services.yml, but with twig.config.cache still enabled.)
(Drupal 8.5.5)
Comment: Unfortunately this question is specifically about **paragraph** entities, not comments. The bit about 'any other entity really' is in reference to other ways of achieving this :)

Suitable way to hide a paragraph entity


Magento Admin Dashboard modification

Question: A client noticed that the Admin Dashboard calculations in her system aren't matching up with the Order report, in that what's being called 'Revenue' is actually displaying profit as demonstrated in the attached shots.

What I want to do is modify the Dashboard page to add a 5th element which would be the actual Revenue and not Profit.
I've gotten as far adding the element to the page.

To do this I copied Totals.php from /app/code/core/Mage/Adminhtml/Block/Dashboard/ to /app/code/local/Mage/Adminhtml/Block/Dashboard/.  This is the code that I modified:
    $totals = $collection->getFirstItem();
    $this->addTotal($this->__('Revenue'), $totals->getProfit());
    $this->addTotal($this->__('Profit'), $totals->getRevenue());
    $this->addTotal($this->__('Tax'), $totals->getTax());
    $this->addTotal($this->__('Shipping'), $totals->getShipping());
    $this->addTotal($this->__('Quantity'), $totals->getQuantity()*1, true);

I just can't seem to find the actual variable that I need to call.  Does anyone have any idea how I can locate it or what it might be? I've tried Sales, SalesTotal, Invoiced and they all return zero.
It's the value that's being displayed as Sales Total in the 1st image.
Thanks.
EDIT
The variable that I'm looking for isn't loading in this file.  I've hunted down Grid.php in /app/code/core/Mage/Adminhtml/Block/Report/Sales/Sales/ which is what's drawing the table in the 1st image.
I've tried inserting
Mage::log($totals->debug());

in various places but it causes an error in each instance.
Here's a snippet of code from Grid.php, either of these values would work for me..
$this->addColumn('total_income_amount', array(
        'header'        => Mage::helper('sales')->__('Sales Total'),
        'type'          => 'currency',
        'currency_code' => $currencyCode,
        'index'         => 'total_income_amount',
        'total'         => 'sum',
        'sortable'      => false
    ));

    $this->addColumn('total_revenue_amount', array(
        'header'        => Mage::helper('sales')->__('Revenue'),
        'type'          => 'currency',
        'currency_code' => $currencyCode,
        'index'         => 'total_revenue_amount',
        'total'         => 'sum',
        'sortable'      => false,
        'visibility_filter' => array('show_actual_columns')
    ));

Comment: Bumping for Monday morning...Anyone? :)
Comment: Comments don't bump a question any higher. Also I added a `magento` tag since more people are likely to see that.
Answer: Turn on logging in System > Configuration > Developer and then next to your overridden code put:
Mage::log($totals->debug());

Watch the "var/log/system.log" file as you refresh the page and it will dump $totals for you. Find the index of the value you want and convert it to camel case, for example if you see sales_total then you can use $totals->getSalesTotal() (which you have already tried, this is only an example).
Remember to turn off logging when the site goes live.
Comment: There's no other variable being pulled in on this page. I'm updating my question with some additional info

Magento Admin Dashboard modification


Não consigo mostrar os dados da função "detalhes()" que criei

Question: Eu criei um programa sobre livros, como se fosse uma biblioteca. Então no final, minha intenção seria mostrar o nome do livro, quem está lendo, total de páginas, etc.
Criei todas as funções, mas ao tentar mostrar os dados como string na função detalhes() não dá certo.
O código abaixo mostra o que eu tentei fazer, coloquei só o titulo do livro para ficar fácil de compreender. Nenhum desses dois códigos funcionou, procurei algumas perguntas sobre isso mas não encontrei.
public String detalhes() {
        return "Livro: " + tituloLivro;
    }

public void detalhes() {
        System.out.println("Livro: " + tituloLivro);
    }

Answer: Vc tem que passar o parâmetro do titulo do livro para quem chama a função, se não como a sua função vai saber qual titulo do livro é?
public String detalhes(string tituloLivro) {
    return "Livro: " + tituloLivro;
}


Lebesgue integral of $f^{p}$

Question: $f : \mathbb{R}^d\rightarrow [0,\infty], p\geq1$ show that:
$\int_{\mathbb{R}^d}f^p = p \int_0^\infty y^{p-1}m(\{x \in \mathbb{R}^d: f(x)>y\})\,dy$.
Any hints? Thanks in advance!
Comment: https://math.stackexchange.com/q/1254330/321264
Answer: You can use this (general) identity for positive functions :
$\int_X f(x) d\mu = \int_0^{\infty} \mu( x\in X, f(x) > t) dt$
Comment: Correct! Never seen this identity before though.
Comment: Then you should try to demonstrate it if you're not supposed to know it ;)

Lebesgue integral of $f^{p}$


How does Guitar Pro work out fret numbers/strings?

Question: If one imports say a C Major scale in Guitar Pro, it auto calculates the fret numbers/strings.
Does anyone know by what logic it does this (since there are many possible ways to play a C major scale on a guitar)?  Why one way and not another?
Comment: @FreddyD are you wondering specifically how software does it (with little information about your intent) or in general about the patterns down the neck, including what a guitar teacher would say?  i imagine a guitar teacher will choose the *position* that is easiest for you to stay in or near, provided neither of you are too worried about voicing.
Comment: I am wondering specifically and technically, from a maths point of view. I am looking at building an app that would include auto frettings. I am having issue working out what, if any, logic exists. It is of course possible no logic exists and its just convention developed over time.
Comment: We can't really comment on their design decisions unless they've published a reason, but I can definitely see a use for finding out what method is used.
Comment: Let me clarify, I am not concerned specifically with Guitar Pro, many other guitar notation app auto calculate frets. So an answer as to how any of them do it, or any rationale around the logic would be intresting. It seems guitar teachers in general seem to use specific patterns down the neck rather than across.
Comment: I suggest you get some midi tracks and import them into Guitar Pro. You may be able to identify a pattern. Also you could investigate transposing some tracks.
Comment: @FreddyD unfortunately, i assume that's where they make their money, with proprietary algorithms that help them separate their software from someone else's.  i do believe there is logic, but maybe it would depend on your intended audience?  if this is for beginners, for example, you might default to chords in the open position since many consider them to be the easiest.  maybe you could make them choose a category, like a style of jazz, and then default to close position voicings or drop 2 or whatever.
Comment: @aeroNotAuto, your comments make a lot of sense. As I do more research it seems an array of very similar logics exist and they are in fact genre specific and difficulty specific. Finale tends to use few strings and many frets which would be harder than Guitar Pro which seems to use many strings with minimum fret movement.
Comment: @FreddyD yea it would be fascinating to port this over to datascience.se or kaggle and build up a data set to analyze.  i wonder if any of them would sue us if we figured their algorithm out...
Answer: If you are wondering on how to get a suitable/optimal fingering position, there are some existing research that develop such algorithms.
For Guitar fingers position, take a look at the following paper: http://www.csc.kth.se/utbildning/kth/kurser/DD143X/dkand13/Group7Anders/final/Vladimir.Grozman.Christopher.Norman.report.pdf
The algorithm is graph-based where each node represents few elements such as String Number, Fret Number, Note duration, etc. And it tries to find a solution that will minimize the cost of transition between the notes.
For Piano fingers position, take a look at the following paper:
http://ijcai.org/papers07/Papers/IJCAI07-469.pdf
It is based on a statistical model called Hidden Markov Model.


SimpleDateFormat am/pm format retorna "da tarde" em vez de "PM"

Question: Usando uma solução desta pergunta:
https://stackoverflow.com/questions/6907968/how-to-convert-24-hr-format-time-in-to-12-hr-format
Ao fazer o seguinte:
        Int hour = 13;
        SimpleDateFormat _24HourSDF = new SimpleDateFormat("HH:mm");
        SimpleDateFormat _12HourSDF = new SimpleDateFormat("hh:mm a");
        Date _24HourDt = _24HourSDF.parse((Integer.toString(hour)+":00"));
        System.out.println(_24HourDt);
        System.out.println(_12HourSDF.format(_24HourDt));

O resultado do segundo print é:
01:00 da tarde

Como faço para retornar:
01:00 PM

Comment: Provavelmente é o *locale default* da sua JVM. Por favor veja qual é o resultado de `System.out.println(Locale.getDefault())` (não esqueça de fazer o `import java.util.Locale`)
Comment: Retorna: pt_PT
Como faço para mudar?
Answer: Eu não consegui reproduzir (testei no Java 8 e 11, com diferentes locales) mas provavelmente é por causa do locale default da JVM (veja qual é o seu, com System.out.println(Locale.getDefault())).
O que acontece é que SimpleDateFormat usa o locale default da JVM, e este locale pode afetar algumas strings de formatação, como é o caso do AM/PM.
Por exemplo, se eu fizer:
// locale default será o do Japão
Locale.setDefault(new Locale("ja", "JP"));
int hour = 13;
SimpleDateFormat _24HourSDF = new SimpleDateFormat("HH:mm");
SimpleDateFormat _12HourSDF = new SimpleDateFormat("hh:mm a");
Date _24HourDt = _24HourSDF.parse((Integer.toString(hour) + ":00"));
System.out.println(_12HourSDF.format(_24HourDt));

A saída será :
01:00 午後

Pois SimpleDateFormat usa o locale default que está configurado no momento em que ele é instanciado. E este locale pode afetar as strings que são usadas na formatação.
Mas se você quer strings específicas, que não dependem do que está configurado na JVM, então a solução é setar o locale na criação do SimpleDateFormat:
SimpleDateFormat _12HourSDF = new SimpleDateFormat("hh:mm a", Locale.US);

No caso, Locale.US usa o locale correspondente ao inglês americano, que usa as strings "AM" e "PM".
Esta solução eu acho melhor porque você só altera o que precisa (a instância específica de SimpleDateFormat). Se você mudar o locale default (com Locale.setDefault), isso afeta todas as aplicações que estiverem rodando na mesma JVM, e isso pode ter efeitos colaterais indesejados.

Java >= 8
A partir do Java 8 você pode usar a API java.time, mais moderna e que corrige vários problemas de Date e SimpleDateFormat (veja mais aqui).
Por exemplo, para criar um horário correspondente a 13:00, não precisa daquele "malabarismo" todo que você fez com Date. Basta criar um java.time.LocalTime (uma classe que só possui os campos de horário, que parece ser o que você precisa):
import java.time.format.DateTimeFormatter;
import java.time.LocalTime;

...
// criar o horário 13:00
LocalTime time = LocalTime.of(13, 0);
// formatar (não esqueça do locale, para ter as strings AM/PM corretamente)
DateTimeFormatter fmt = DateTimeFormatter.ofPattern("hh:mm a", Locale.US);
System.out.println(fmt.format(time)); // 01:00 PM

Para formatar, também é necessário setar o locale, senão você acaba tendo o mesmo problema (ele assume o default da JVM, e as strings "AM/PM" podem não ser exatamente essas).
Comment: Funcionou! Muito obrigado. Boa resposta e explicação!
Comment: @lemario Não sei qual a versão do Java que vc está usando, mas caso seja Java >= 8, tem uma alternativa - na minha opinião, muito melhor - que é usar a API `java.time`. Adicionei esta opção no final da resposta.


display content type in view

Question: I have 2 content types courses and courses_type, I added courses_types as node reference in courses.
when going to build view to dispaly all courses with thier courses_type the courses_type column is empty. what i am doing wrong?
Comment: Did you create a relationship in the view using the specific node reference?
Answer: For Drupal 7, use the Entity and Entity Reference modules. This is just a simple view. You don't need to create any relationship. Just add course type in Fields.

display content type in view


R desconsiderando um data_frame na Function

Question: Estou trabalhando para automatizar um processo de manipulação de dados que tenho aqui na minha demanda pessoal no trabalho.
Criei uma function que abre as bases de dados de todos os anos que tenho vendas, o intuito é juntá-las para depois manipular os dados e posteriormente exportar como excel para alimentar o dashboard no Power BI. Juntando todas as bases de dados somam 122.651 linhas, porém depois que a function roda inteira, ela gera um dataset com 122.553 linhas (o que corresponde às somas de 2017 até 2021, ignorando o dataset de 2022).
Comment: A função está mal definida, tente `f<-function(){ o seu código de leitura, join e return }` seguido de `base_ok <- f()`.
Answer: Tente a seguinte função.
<code>ler_ficheiros <- function(anos, path = "."){
  pattern <- paste0("^", anos, ".xls")
  pattern <- paste(pattern, collapse = "|")
  fich <- list.files(path = path, pattern = pattern, full.names = TRUE)
  df_list <- lapply(fich, read_excel)
  base_actual <- Reduce(full_join, df_list)
  base_actual
}

base_ok <- ler_ficheiros(2017:2022)
dim(base_ok)
</code>


What is a typical volume range for a singer?

Question: I have always struggled from having a quiet voice, both singing and talking, but have no easy way to measure my singing volume objectively - I know when I was in a singing group for fun I had to be turned up a lot higher than everyone else.
It's only a very approximate test but today I used a decibel meter app on my phone, holding it at arm's reach in front of me and singing notes on a scale.
But then I have no idea how others would compare. Is there a good way I can measure myself and see how my volume compares to a typical singer, similar to how I can measure my vocal range?
Comment: Its hard to be your own judge on something like this, a vocal teacher should be able to point you in the right direction. But yeah, I know it is a tangent that I suggest.
Comment: There are techniques to improve your volume: http://music.stackexchange.com/questions/10992/techniques-to-improve-vocal-volume?rq=1. I think there are indeed proper and improper techniques to be louder that can be learned. I think they missed some, but I am no expert there. They also give some alternative thoughts about amplification.
Comment: Interesting but without knowing how quiet I am to begin with... I mean I'm sure I'm quiet but not if I'm so quiet it's a problem.
Comment: Also A teacher will help prevent you from damaging up your voice, if you have a good one.
Comment: @amalgamate a tutor would probably be able to tell me subjectively if I really am that quiet. I was hoping for a more objective way to see how quiet I am compared to the average guy but it seems unless you lot all sing into a decibel meter, that doesn't exist :)
Comment: I guess we were all trying too hard to read between the lines....
Answer: While I feel amalgamate's post is the answer you are looking for, I can offer some suggestions to objectively improve/compare your singing volume.
Seek one-on-one vocal instruction. While singing in a group you learn to focus on blending your voice with others, this could definitely deter your ears from remembering what your voice sounds like and could definitely lead to volume issues. Every choir has people who do not sing loudly enough - rest assured your situation has a solution. In choir, the focus is pitch and blend. 40 other voices singing different pitches at once would wig anyone out.
Remember, our ears are on the sides of our faces - not directly in front of our mouths. People sometimes find the sound of their voice on recordings unpleasant because they are not used to hearing it that way before. Well...there is no way they possibly could. The nature of how your ears works changes the vary EQ of your voice! So too does their position!
By going to a vocal instructor you will be given the opportunity to hear your voice by itself. Singing in a choir or with instrumentation behind you is not the same! In this element you are forced to truly hear your voice. This gives an instructor an opportunity notices you naturally singing too softly (for anyone to hear), maybe then he/she can offer some suggestions to improving your tone. Most of the time this involves

Proper technique in singing/breathing from your diaphragm
Vocal exercises to warmup/cool down so that you can sing comfortably at full volume
Tongue twisters to properly form certain vowel/consonance sounds to ensure clarity and less mumbling (How now brown cow)

Is there a good way I can measure myself and see how my volume
  compares to a typical singer, similar to how I can measure my vocal
  range?

While I understand your intentions, a word of caution. By comparing one's technique or practice habits to someone else, one can derive great benefit. However, there are variables out of our control. I believe singing volume to be one of them. Is your case hopeless? Of course not! Might you need some help? Perhaps.
Other ideas. Keep the same microphone settings and

Record your voice without accompaniment
Record your voice with accompaniment
Record your voice in choir

Compare all of these to each other. Are you quieter solo than with a guitar? Perhaps you are just focusing on nailing your pitches. Can you pick out your voice in a choir? The same thing could be said for all these situations.
Lastly,
<code>Pitch || blend || tone > volume (within reason)
</code>
Comment: I humbly submit that this is the correct response to the question.
Answer: This site says that the average whisper is 30db and the Fortissimo (loud) singer is around 80 db. I found another site that suggested that Opera singers can reach past 100db (it was less specific so I take it with a grain of salt). This site and others say that average shouting voice is about 88db.  So I would conclude that the range of useful singing loudness starts somewhere between 70db and is unlikely to go higher then 88db for the average singer. Also that 80db would be a very respectable unassisted volume level.
Note that distance has an intense effect on the measured volume level of sound for small distances. I would suspect that these average measurements stated on these sites are from a distance. This site accounts for distance. Based on it, I would say the other sites are basically taking it's measurements at arms length from the source (Although I am sure that changes for cases involving headphones).
Incidentally, levels above 90db are considered dangerous to your hearing at sustained intervals of time.
Answer: I think we've established that your voice is at the quiet end of the range.  Numbers don't matter.
It may be that you're 'singing out' well, but it's just a small voice.  If you're going to be singing in an environment where individual microphones are used, and you make a sound useful to the group, this may not matter.  Or you may have just never discovered how to 'turn the motor on' - and you're in the group for inclusion reasons (common in church groups) or because you look good (common in pop groups).
You need a session with a vocal teacher.  Period.
Answer: There's been a study about this in 2008 by the Royal Conservatory of The Hague / University Utrecht / Voice Quality Systems.
The report shows that a female opera singer can reach a 90 dB in a traditional practice session and can reach a 130 dB in a performance setting, depending on years of experience/training and vocal range.
The subject interests me as I am a singer looking for a decent vocal booth... most of them won't isolate more than a lousy 40dB. Not enough in my case. Deep sigh...
Answer: Does it really matter? The loudest a voice would go often takes it from singing into shouting, so the distinction is blurred, anyway. The volume may well not be the same over a range of pitches. The sound, as in a word, or just 'lah' will muddy the waters,and a larger built person will probably have more projection, partly due to bigger lungs, and maybe mouth, than a smaller one. So there is too much subjection here.
When miced up with one band, the others used to ask why my mic had to have more gain than someone else's. Partly their mic was far better than mine, partly, I never want to swallow mine while singing. If you think you have a voice that's not too powerful, accept it, and turn up your mic! It's nothing shameful. A typical singer probably doesn't exist. One who uses a mic all the time, or an opera singer who can 'break glasses'? Could a soprano be as loud as a tenor? I'm afraid there's no concrete answer here.
Comment: @Mr.Boy - no doubt your conductor is the best person to ask. Surely if he feels others are singing too loudly over you, and maybe others, he would try to rectify what should be seen (or heard) as a problem. In any case, ask the question, as here, and in a quiet moment (rehearsal) let him be the judge. He's bound to have heard loads of voices in his time - it's his job!
Comment: @Mr.Boy  I would suggest that one with a really quiet voice is just not going to stand out in a choir (that's okay) and if playing un amplified and without a mic, might need to play more softly and ask for the audience to please be quiet.  Pushing your voice to sing louder than what is comfortable might put undue strain on voice and cause damage to vocal chords.  Accept the limitations and learn to compensate.
Comment: Well if I want to sing in a choir it's an issue - even if we're group-miked (unlikely) I am quiet in comparison. And sometimes I am involved in unplugged music. I'm not interested in being super-loud, but in some approximate way to see how loud I am... I can't tell since I cannot hear what my audience hears! I don't know if my natural volume is enough or if I need to singing as loud as I (comfortably) can - I know people who are far too loud and that's as annoying as someone who is too quiet.
Comment: I will ask someone to listen out for me next time - "can you even hear me" but I still think it's a reasonable topic for a question. For instance if I play acoustic guitar and sing, singer-songwriter style, I get much more guitar bleed in my vocal mic since I (I think) have to turn my mic up so much.
Comment: Its hard to find someone that can give you an honest and educated appraisal. I was playing a regular gig last year where I knew my volume was off (electric guitar), and people came up saying how I was really great and that I really added something. You need someone you trust with some musical knowledge to be your ears.
Comment: @Mr.Boy  To cut down on guitar bleed, you might want to try a mic with a supercardioid or hypercardiod polar pattern.  These mics are more directional than a typical cardiod pattern mic (such as an SM58) and will better reject off axis sound.
Comment: @RockinCowboy true but that's just an example where a very quiet voice has real implications


Help understand a pathological example for the Ford Fulkerson Algorithm

Question: Consider the graph below. All edges are undirected and have capacity $\frac{1}{1-\sigma}$ where $\sigma = \frac{\sqrt{5}-1} {2}$ except for the ones indicated otherwise (those in red). I further understand that $\sigma^{n+2} = \sigma^{n}-\sigma^{n+1}$.
I was told that the Fulkerson Algorithm does not terminate on it. However, I do not understand why.  Could you explain that to me?
Comment: Have you run a few iterations of the algorithm and observed how the flow changes in each iteration, with the flow increasing in each iteration but by a smaller and smaller amount?
Answer: None of the pathological examples for the Ford-Fulkerson algorithm have the property "the algorithm does not terminate for it". All networks have a maximum flow (well, all finite networks with finite capacity on each edge, and a source and sink for the problem to make sense). If we know a maximum flow, it is easy to find a sequence of augmenting paths that will let the Ford-Fulkerson algorithm find it in finitely many steps.
(In this example, the maximum flow is $\frac{4}{1-\sigma}$, and is easy to find in four augmenting steps: just take four edge-disjoint paths of length $3$ from $s$ to $t$ that don't use any of the red edges.)
So pathological examples for Ford-Fulkerson must come with a pathological choice of augmenting paths which is valid at every step, but does not terminate. Essentially, what we're saying is this: "The Ford-Fulkerson algorithm does not fully specify which augmenting path to choose at each step. So it's possible that it will choose these augmenting paths. Then it will never terminate. As a result, we conclude that it's possible for Ford-Fulkerson to never terminate, unless we come up with a rule to pick augmenting paths that avoids outcomes like this one".
Without knowing the pathological choice of augmenting paths, it's hard to reconstruct it. Presumably, however, the bad augmenting paths were specified wherever you found the example.
If not, here is a different example for you to try to understand. It is a $6$-vertex, $8$-edge example that is:

Originally described in "The smallest networks on which the Ford-Fulkerson maximum flow procedure may fail to terminate" by Uri Zwick.
Elaborated on in these lecture notes on pages 335-336.
Worked out in excruciating detail in the last section of my lecture notes.


$f(x)=[x]+^{0.5}$ Is this continuous and differentiable at $x=1$?

Question:
$f(x)=[x]+<x>^{0.5}$ Is this continuous and differentiable at $x=1$?
Where $[x]$ is the greatest integer function and $<x>$ represents the
  integer part of x.

Please help with this.
Is it discontinuous because when $x=1-\varepsilon $ , $f(x)=0+\sqrt{\varepsilon}$
and when  $x=1+\varepsilon $ , $f(x)=1+\sqrt{\varepsilon}$
Comment: Both the answers you have received take $\lt x \gt$ as the fractional part of $x$.  Your question says it is the integer part, which would match $[x]$ as you have defined it.  If you meant $\lt x \gt$ to be the fractional part, please edit your post.  Your computation at $x=1+\varepsilon$ uses the fractional part.  Your computation at $x=1-\varepsilon$ does not use either one correctly.
Answer: The fractional part of $x$ is $\langle x\rangle=x-[x]$, so when $0<x<1$ then the integer part of $x$ is zero, while the fractional part of $x$ is $\langle x\rangle=x-[x]=x-0$. Hence $f(x)=0+\sqrt{x}$ for $0<x<1$. On the other hand, when $1\le x<2$ then the integer part of $x$ is $1$ and so fractional part of $x$ is $\langle x\rangle=x-[x]=x-1$. Hence, $f(x)=1+\sqrt{x-1}$. I'll leave it to you to check continuity and differentiability.
Comment: Is it continuous but not differentiable?
Comment: yes, it is continuous but not differentiable
Answer: For $\,0 \le x < 1$,
$$f(x) = \sqrt{x}$$
For $1 \le x < 2$,
$$f(x) = 1 + \sqrt{x-1}$$
It follows that $f$ is continuous at $x=1$, since
\begin{align*}
&\bullet\;f(1) = 1\\[4pt]
&\bullet\;\lim_{x \to 1^{-}}f(x) = \lim_{x \to 1^{-}}\sqrt{x} = 1\\[4pt]
&\bullet\;\lim_{x \to 1^{+}}f(x) = \lim_{x \to 1^{+}}1+\sqrt{x-1} = 1
\end{align*}
But $f$ is not differentiable at $x=1$, since
\begin{align*}
\lim_{h \to 0^{+}}
\frac{f(1+h)-f(1)}{h}
&=
\lim_{h \to 0^{+}}
\frac{\left(1 + \sqrt{h}\right)-1}{h}\\[4pt]
&=
\lim_{h \to 0^{+}}
\frac{\sqrt{h}}{h}\\[4pt]
&=
\lim_{h \to 0^{+}}
\frac{1}{\sqrt{h}}\\[4pt]
&=\infty
\end{align*}


Why does the pivotal quantity $g(X,θ)$ have to be a monotonic function of $θ$ for a fixed $X$?

Question: I'm learning about pivotal quantities, which are functions $g(X,θ)$ of the data $X$ and the parameter $θ$. One property they must satisfy is that $g(X,θ)$ is a monotonic function of $θ$ for a fixed $X$.
I'm told the reason for this, is to be able to manipulate inequalities.
For example, if we have a random sample
$X_1, X_2, ... , X_n$ from $N(θ,σ^2)$, we can construct a confidence interval for $θ$ by creating a pivotal quantity: $\frac{\bar X - \theta}{\sigma/\sqrt n}$
So that for example, $P(-1.96 \leq \frac{\bar X - \theta}{\sigma/\sqrt n} \leq 1.96) = 0.95$
Now, it is apparently this property of $\frac{\bar X - \theta}{\sigma/\sqrt n}$ being monotonic in $θ$ for a fixed $X$ that allows us to manipulate these inequalities and write this as
$P(\bar X - 1.96 \frac{σ}{\sqrt n} \leq θ \leq \bar X + 1.96 \frac{σ}{\sqrt n}) = 0.95$.
But I do not understand any part of this. Why does it need to be monotonic in $θ$? What does it mean for it to be monotonic in $θ$? Why does $X$ need to be fixed? Why does this let us manipulate the inequalities?
Comment: possible duplicate (or partial duplicate): https://stats.stackexchange.com/questions/141768/confidence-intervals-from-pivotal-quantities
Answer: In a sense you want a sort of inverse to go from $\mathbb P(c_1 \le g(X, \theta) \le c_2) \le k$ to one of:

(if monotonic increasing): $\mathbb P(h(X,c_1) \le  \theta \le h(X,c_2)) \le k$ or
(if monotonic decreasing): $\mathbb P(h(X,c_2) \le  \theta \le h(X,c_1)) \le k$

and having $g(X,\theta)$ monotonic in $\theta$ for given $X$ enables such an inverse $h(X,c)$ to exist


Central Limit Theorem and preventing the distribution from collapsing and degenerate

Question: I know question on central limit theorem has been asked numerous times and I have searched around the website but still couldn't get what I wanted to find out (probably most people who asked the question is stuck at different parts of the theory).
Attached in my question is a lecture slide for the search for a transformation of x̅ that has a limiting distribution. I do not understand why x̅  and µ is multiply by √n instead of n. Also, why when the statistic √n(x̅  - µ) is used, µ becomes zero?
Thank you in advance!
Comment: Suggestion to type in math mode and not use figures.
Answer: A previous slide defines
$$ \bar{X} = \frac{X_1+\cdots+X_n}{\sqrt{n}}, $$
where the random variables $X_1,\ldots,X_n$ are independent copies of the same distribution, which has mean $\mathbb{E}[X_i] = \mu$ and variance $\mathbb{V}[X_i] = \sigma^2$. One calculates that the mean of $\bar{X}$ is also $\mu$, and its variance is $\sigma^2/n$.
We want to normalize $\bar{X}$ so that it has mean $0$ and variance $1$. If we subtract $\mu$ from $\bar{X}$ we obtain a random variable $\bar{X} - \mu$ with mean $0$ and the same variance. If we multiply the result by $\sqrt{n}/\sigma$ then the mean remains $0$ and the variance becomes $1$. The new random variable is
$$ \tilde{X} = \frac{\sqrt{n}(\bar{X} - \mu)}{\sigma}. $$
It so happens that as $n\to\infty$, $\tilde{X}$ tends to a normal distribution $N(0,1)$. This is the central limit theorem.
In order for a theorem of this sort to hold, the mean and variance of $\tilde{X}$ have to be fixed. This is why we normalized it to have mean $0$ and variance $1$. We could have chosen other values and would have obtained an equivalent central limit theorem, but the choice of zero mean and unit variance is the conventional standard.
The slide tries to explain why other normalizations won't work:

$\bar{X}$ has fixed mean but the variance tends to $0$. Therefore $\bar{X}$ tends to the constant $\mu$ distribution. This is the law of large numbers. The law tells us what $\bar{X}$ tends to, but not how the deviation from the mean behaves.
$\sqrt{n} \bar{X}$ has fixed variance $\sigma^2$ but the mean tends to infinity (or minus infinity). So $\sqrt{n} \bar{X}$ cannot converge to any fixed distribution, unless $\mu = 0$. This is because if $\sqrt{n} \bar{X}$ tends to a distribution with mean $m$, then the mean of $\sqrt{n} \bar{X}$ must tend to $m$.
$\sqrt{n}(\bar{X} - \mu)$ has mean $0$ and variance $\sigma^2$, and we can hope for a limit theorem. Indeed, the central limit theorem shows that $\sqrt{n}(\bar{X} - \mu)$ tends to $N(0,\sigma^2)$, a normal distribution with mean $0$ and variance $\sigma^2$. This is equivalent to the central limit theorem stated above.
Comment: Thank you. Your explanation is very thorough! :) I still have some questions:

in 2) we can basically choose any combination value of µ and σ^2 but it's meaningless, hence we choose the standard normal values of mean o and variance 1 so that we can use the standard normal statistical table, right?

also, I do not understand why  x̅ = (X1+⋯+Xn) / √n and consequently, why  σ^2/n can just become σ^2 when everything else is multiplied by √n? does my question make sense? why √n is used instead of just n?
Comment: 1) You can choose whichever mean and variance you want, and it's meaningful. While it's true that standard "tables" (nowadays called library functions) usually work only for standard normals, it's not difficult to use them to analyze arbitrary normals. 2) When you add $n$ i.i.d. random variables, the order of magnitude of the error is the standard deviation, which is of order $\sqrt{n}$. This is a very basic fact that follows from the central limit theorem. 3) A calculation shows that the variance of $(X_1+\cdots+X_n)/\sqrt{n}$ is $\sigma^2$. Make sure that you know how to perform it.
Comment: We divide by $\sqrt{n}$ rather than by $n$ since we want the variance to be constant. If we divided by $n$, the variance would tend to zero and we wouldn't be able to see the errors. You can try it – add 100 numbers, each chosen uniformly at random among $1,-1$. If you divide by 100 and do a histogram (i.e., repeat the experiment many times and plot how many times each value was obtained) then the histogram will be very close to 0. Replace 100 by 1000, and it becomes even closer to 0.
Comment: In contrast, if you divide by 10 (that is, $\sqrt{100}$) then you should see a Gaussian. You should see a very similar Gaussian if you replace 100 with 1000 and 10 with $\sqrt{1000}$. I encourage you to do this experiment and see for yourself.
Comment: I got it!! thanks!!!


rodar o puppeteer as 8 da manha todos os dias

Question: opa, eu sei o básico de javascript, mas com muita pesquisa eu consegui criar uma aplicação para pegar algumas informações de uns sites e fazer uns cálculos. é um sistema simples pra um amigo que é trader e precisa fazer esses calculos manualmente no excel.
seguinte, o codigo seguinte pega as informações dos sites e salva num json (no caso 3)

const puppeteer = require('puppeteer');
const fs = require('fs');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://br.advfn.com/investimentos/futuros/di-depositos-interfinanceiros/cotacoes',{
    waitUntill: 'load',
    timeout: 0
  });

  const textNode = await page.evaluate(()=>{
    const nodeText = document.querySelector(".even.first").innerText;
    const text = [nodeText];

    return text
  });

  fs.writeFile('arreglo2.json', JSON.stringify(textNode), err =>{
    if (err) throw new Error ('algo deu errado')
      console.log('deu certo')
  })

})();

//************************************************************************/
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://br.tradingview.com/symbols/TVC-DXY/',{
    waitUntill: 'load',
    timeout: 0
  });

  const textNode = await page.evaluate(()=>{
    const nodeText = document.querySelector(".tv-symbol-price-quote__change-value").innerText;
    const text = [nodeText];

    return text
  });

  fs.writeFile('arreglo.json', JSON.stringify(textNode), err =>{
    if (err) throw new Error ('algo deu errado')
      console.log('deu certo')
  })

})();

/**********vai pegar a cotação do fechamento do colar */

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://br.advfn.com/bolsa-de-valores/fx/USDBRL/cotacao',{
    waitUntill: 'load',
    timeout: 0
  });

  const textNode = await page.evaluate(()=>{
    const nodeText = document.querySelector(".qs-current-price").innerText;
    const text = [nodeText];

    return text
  });

  fs.writeFile('cotacaoFechamento.json', JSON.stringify(textNode), err =>{
    if (err) throw new Error ('algo deu errado')
      console.log('deu certo')
  })

})();

quando eu abro o terminal e dou node index.js ai ele roda de boa, cria o json e tudo, ai quando eu dou refresh na página já ta tudo calculado.
so que eu queria saber agora como faço pra fazer o index.js rodar exatamente as 820 todo dia.
pq meu amigo precisa dos dados as 830h quando o mercado abrir.
Comment: Seja benvindo ao StackOverflow em Português! Essa aplicação roda num servidor? Computador do usuário? Já testou crontab?

Para melhorar sua experiência aqui, fazer a pergunta de forma que possa ser respondida, recomendo ler estes dois posts: [Guia de sobrevivência do Stack Overflow em Português](https://pt.meta.stackoverflow.com/questions/8045/guia-de-sobreviv%c3%aancia-do-stack-overflow-em-portugu%c3%aas) e [Como criar um exemplo Mínimo, Completo e Verificável](https://pt.stackoverflow.com/help/minimal-reproducible-example). Obrigado!
Comment: então, quando eu subir para um site de hospedagem, isso caracteriza um servidor? eu só manjo de front-end, é minha primeira "aplicação" por assim dizer com algo de back-end. mas eu pretendo subir ela pra ficar disponível. hoje eu to rodando no xampp pra testar.
Answer: Da uma pesquisada sobre a biblioteca node-cron, com ela você vai conseguir agendar as execuções do seu script. Como usar.
Para o agendamento do node-cron funcionar você vai precisar deixar o script executando, para isso eu gosto de usar o PM2. Como usar.
Comment: obrigado, eu tava aqui pesquisando, no caso a minha dúvida ainda é maior. por exemplo, um site estatico eu vou lá e coloco as pastas no site e em poucos segundos ele ja ta no ar. nesse caso é mais complexo né? pq tem um arquivo .js, tem json e as pastas do node. eu tenho que contratar algum tipo de serviço especifico de servidor?
Comment: Esse seu script teria que rodar separado do site porque se não ele executa só quando você abrir o site. Você teria que deixar esse script rodando em algum servidor ou até mesmo na sua máquina local e criar uma API que retorne os dados do seu arquivo JSON. No frontend você apenas chamaria essa API para buscar os dados do JSON..
Comment: tem alguma empresa que tipo presta esse serviço pra eu hospedar esse script num servidor?
Comment: Sim,  já usei o Replit pra isso, mas não sei se continua gratuito. Hoje em dia tenho alguns bots online e deixo eles rodando através do PM2 em uma EC2 na AWS, mas dai você vai ter que configurar tudo, com o replit você só jogaria o código lá dentro e botaria pra rodar o script


MDX object not found in cube

Question: I have districts with areas. and i'm trying to fetch data from my cube. the problem is:

i add new area to district in database.
the data in cube is recalculated only once a day. so rigth after i have added new area it will be not present in my cube.
and when i try to get info from cube i get the error 'MDX object '[ATD].[Area].&[6007d9a7-a137-4bba-9d72-0020ee5d2db0]' not found in cube'
after the cube is recalculated everything is ok.

how can i fight this error?
with member [Measures].[District]
     member [Measures].[Area]
     member [Measures].[AreaID]
select
                    {
                         [Measures].[District],
                         [Measures].[Area],
                         [Measures].[AreaID]

                    }  on columns,
                    {
                          {{[MyDimension].[Area].&[6007d9a7-a137-4bba-9d72-0020ee5d2db0]}}
                    } on rows
                    from SomeCube

Thanks in advance.
Answer: You should set the ignoreInvalidMembers property to true.
You can find more information on Mondrian's configuration page.
Comment: I use tomcat and I put a mondrian.properties file in bin directory of tomcat but it seems it's not being used.
Comment: i have solved the problem: just put the file in \mondrian\WEB-INF\classes\

MDX object not found in cube


Why does maven 2 try to download dependencies that I already have?

Question: When I launch the "mvn install" command, maven sometimes tries to download dependencies that it has already downloaded. That's expected for SNAPSHOT but why does maven do that for other JARs?
I know I can avoid that behavior by "-o" flag but I just wonder what the cause is.
Answer: I'd look for dependencies that don't have a specified version number.  Maven will periodically check to make sure that it has the most up-to-date version of these artifacts.
Comment: Dependencies _must_ have a version specified. Plugins do not, but you should. Here's why: http://www.sonatype.com/people/2008/04/maven-209-released/
Comment: It may also be updates to plugins so specifiying versions on them could also help.
Comment: Yep ... I guess I generically included them in "dependencies".  You also have to realize that lifecycle phases are often bound to specific plugins.  Who actually specifies a version number for their compiler plugin?
Answer: This is probably not what you're seeing, but in the past I've had to manually install artifacts into my local repository and if you forget to include the -Dgenerate.pom=true option there will be no pom in the repo for that artifact and Maven will go out to central (and any other remote repos you have configured) to try to download that pom on every build.
Answer: While we're on the subject of this, I've run into a major bug in Maven 2.0.x.  In offline mode, maven will still attempt to download the latest snapshot, and when it can't find your snapshot repo, it fails the build. Imagine the hilarity that ensues when this happens on site with a client and you just needed to make a small change (but I digress).
Here's the bug:
http://jira.codehaus.org/browse/MNG-2433
here's a workaround:
http://firstname.lastname@example.com%3E
Comment: This was fixed in 2.0.10: http://jlorenzen.blogspot.com/2009/04/better-offline-capabilities-with-maven.html
Comment: actually in my experience Maven 2.0.x was so full of bugs that back then I thought it ain't worth using at all. I don't recall exactly if it was 2.1.x or 2.2.x but it was definitely **post 2.0** version which made me first accept then fell in love with Maven
Answer: The -o flag still wasn't working for me, but this did:
<code>find ~/.m2/repository -name '_maven*' | xargs rm
find ~/.m2/repository -name '*lastUpdated' | xargs rm
</code>
Which will delete all the .lastUpdated and _maven.repositories files in your local repo. I ran into this issue because we have a corporate Nexus repo that was unreachable, and I needed to do some work. Using Eclipse's Maven integration may have also contributed to this.


VueJS + Axios - CORS policy : No Access-Control-Allow-Origin

Question: I have an error on a web application developed with VueJs that uses Axios.
Access to XMLHttpRequest at `URL API` from origin `URL APP` has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Example of a request :
axios
        .get(
          store.getters.getURL +
            "/TennisStats?MatchId=" +
            this.$route.params.id +
            "&season=" +
            season,
          { }
        )
        .then(function (response) {

I have a devServer set up:
module.exports = {
  devServer: {
    proxy: 'https:/URL_APP/api/',
  },

The application is hosted on AWS.
The error appears randomly and when the cache is deleted, it disappears again for a while.
I don't know what to do... can you reproduce the error and guide me in my search for the cause?
Answer: Can you try going incognito and trying? Might be a caching issue since you are working on the same machine. Also if this does not work try restarting the AWS server, or try running the hosted application on a different machine.