Create table from a string in cell by extracting values using regex

Question: I am trying to write a pretty big macro to pull data from one sheet and parse and transform it into different sheets.
Going pretty well in everything else. However, I am stuck with 1 part.
I have multiline text in a cell (11 lines in this example, but could be dynamic)
The sample text I am trying to build my code on is below (the lines are separated by a space and Alt+Enter after the last character in each line):
Flat where a figure is quoted but theshold based on the following lanes
IL STD  $5.00 above $75.00 USD
SG STD  $11 above SGD400
TR STD  $3 above €30 EUR
AU EXP $2 for value>AUD 1000
IL EXP $2 for value>USD 75
JP EXP $2 for value>USD 65
NZ EXP $2 for value>NZD 400
PH EXP $2 for value>USD 165
SG EXP $2 for value>SGD 400
TW EXP $2 for value>TWD 2000

My requirement is to parse this text into a table as below:

I started with the logic that I will count the number of line breaks/carriage returns and then create arrays using loop for the number of line count.
But I haven't been able to achieve this.
I have tried the below variation of codes to get no. of lines but keep getting 0 for the output.
Sub tresholds()

Dim strTest As String
Dim NewLines As Long

strTest = ThisWorkbook.Sheets("Rate Card").Range("D10").Text
NewLines = UBound(Split(strTest, Chr(32) & vbCrLf))
'NewLines = UBound(Split(strTest, " " & vbCrLf))
'NewLines = UBound(Split(strTest, vbCrLf))
'NewLines = UBound(Split(strTest, vbLf))
'NewLines = UBound(Split(strTest, Chr(32) & vbLf))

Debug.Print NewLines

End Sub

Also, I'm not too good at regex. And there are multiple patterns in the string.
Some lines have currency symbols and others dont. Some have 'value above' and others have value >. Couldn't figure out how to account for these variations.
Would really be grateful if you guys could help.
Comment: What specific problem did you run into getting your attempt to work?
Comment: 1st I tried to get the number of lines in the string. But my code keeps returning 0 lines.
This is what I have tried.
Sub tresholds()

Dim strTest As String
Dim NewLines As Long

strTest = ThisWorkbook.Sheets("Rate Card").Range("D10").Text
NewLines = UBound(Split(strTest, Chr(32) & vbCrLf))

Debug.Print NewLines

End Sub

Also, I'm not too good at regex. And there are multiple patterns in the string line some lines have currency symbols and others dont. Some have 'value above' and others have value >. Couldn't figure out how to account for these variations.
Comment: The data you pasted does not have a space at the end of each line, only the first 3 lines have a space. It seems unnecessary to include the space in your split. Alt+Enter is `vbLf` not `vbCrLf`. The last 2 commented out lines should return `NewLines = UBound(Split(strTest, vbLf))` = 10 and `NewLines = UBound(Split(strTest, Chr(32) & vbLf))` = 3.
Answer: This solution uses several regex to match various patterns. You could easily pick out the individual country lines without regex, but some of the fields within each line would be difficult without it. You might as well get better with regex. You'll find it useful in the future.
Using regex with VBA requires setting a reference (Tools/References) to Microsoft VBScript Regular Expressions 5.5.
This code has no error checking; that's up to you. Also, I'm sure you can write the output to a worksheet, so I'll leave that up to you also. I've noted in the sub where the output code belongs.
If you want help with the regex, ask specific questions in a comment. Also, SO has a great regex summary:
How to use Regular Expressions (Regex) in Microsoft Excel both in-cell and loops
Sub ParseThreshold()
    Dim rngInput As Range, regexMatch As RegExp, strPattern As String, strInput As String, vntRc As Variant, strPartLine As String
    Dim nCount As Integer, nIndex As Integer, vntMatches As Variant, arstrItems() As String, strCountry As String, strSE As String
    Dim dblDollars As Double, dblThreshold As Double

    Set regexMatch = New RegExp
    Set rngInput = ActiveSheet.Range("a1")
    strInput = rngInput.Value

    ' Define the regex pattern to match each country's line
    strPattern = "^[A-Z]{2} *(STD|EXP) *\$[0-9.]* (above|for value>[A-Z]{3}).*$"

    With regexMatch
        .Pattern = strPattern
        .Global = True
        .MultiLine = True
        nCount = .Execute(strInput).Count
    End With

    Set vntMatches = regexMatch.Execute(strInput)

    If nCount > 0 Then
        ReDim arstrItems(0 To nCount - 1)

        For nIndex = 0 To nCount - 1 '
            arstrItems(nIndex) = vntMatches(nIndex)
            strCountry = Left(arstrItems(nIndex), 2)
            strSE = Mid(arstrItems(nIndex), 4, 3)

            dblDollars = DollarAmount(arstrItems(nIndex))
            dblThreshold = Threshold(arstrItems(nIndex))

            ' Output the values to your table here
        Next nIndex
    End If

End Sub  ' ParseThreshold

Function DollarAmount(strInput As String)
    Dim regexMatch As RegExp, strPattern As String, vntRc As Variant

    strPattern = "(STD|EXP) *\$[0-9.]*"
    Set regexMatch = New RegExp

    With regexMatch
        .Pattern = strPattern
        .Global = False
        .MultiLine = False
        nCount = .Execute(strInput).Count
    End With

    Set vntMatches = regexMatch.Execute(strInput)
    If nCount > 0 Then
        vntRc = vntMatches(0)
        vntRc = Replace(vntRc, "STD ", "")
        vntRc = Replace(vntRc, "EXP ", "")
        vntRc = Replace(vntRc, "$", "")
        vntRc = Trim(vntRc)
    End If

    DollarAmount = vntRc

End Function  ' DollarAmount

Function Threshold(strInput As String)
    Dim regexMatch As RegExp, strPattern As String, vntRc As Variant, nLength As Integer, vntMatches As Variant

    nLength = Len(strInput)
    Set regexMatch = New RegExp

    strPattern = "[0-9.]+"
    strTarget = "above"
    nPosition = InStr(strInput, strTarget)
    If nPosition > 0 Then
        strPartLine = Mid(strInput, nPosition + Len(strTarget) + 1, nLength)
    Else
        strTarget = "value>"
        nPosition = InStr(strInput, strTarget)
        If nPosition > 0 Then
            strPartLine = Mid(strInput, nPosition + Len(strTarget) + 1, nLength)
        End If
    End If

    With regexMatch
        .Pattern = strPattern
        .Global = False
        .MultiLine = False
        nCount = .Execute(strPartLine).Count
        Set vntMatches = .Execute(strPartLine)
    End With

    If nCount > 0 Then
        vntRc = vntMatches(0)
    End If

    Threshold = vntRc

End Function  ' Threshold

Comment: Hi @Richard Cook, Thank you so much! This worked perfectly. I added the piece for the output to be written in the required cell addresses. So far I have only tested with the string I put into the question. Will try with more test strings to see if the regex holds. But I am hopeful that it will. Once again, Thank you so much!
Answer: Another way....
Sheet1 cell A1 value is just like your sample data
Sub test()
Dim rslt As Range, cnt As Integer, splitVal as String
Dim i As Integer, j As Integer, k As Integer, arr

Set rslt = Sheet2.Range("A" & Rows.Count).End(xlUp).Offset(1, 0)
arr = Split(Sheet1.Range("A1").Value, Chr(10))

For i = 1 To UBound(arr)
    rslt.Value = application.trim(Split(arr(i), " ")(0))
    rslt.Offset(0, 1).Value = application.trim(Split(arr(i), " ")(1))
    Set rslt = rslt.Offset(0, 2)
    For j = 2 To UBound(Split(arr(i), " "))
        splitVal = Split(arr(i), " ")(j)
        If splitVal Like "*#*" Then
            For k = 1 To Len(splitVal)
                If Mid(splitVal, k, 1) = "." Then Exit For
                If Mid(splitVal, k, 1) >= "0" And Mid(splitVal, k, 1) <= "9" Then rslt.Value = rslt.Value & Mid(splitVal, k, 1)
            Next k
            Set rslt = rslt.Offset(0, 1)
        End If
        If rslt.Column = 5 Then Exit For
    Next j
    Set rslt = Sheet2.Range("A" & Rows.Count).End(xlUp).Offset(1, 0)
Next i

End Sub

rslt is a range variable where the result will be put. In this case example the result will be put in sheet2 starting from cell A2.
arr variable has value from sheet1 cell A1 value splitted with line break.
There are three loops in the sub.

Loop for each line
Loop for each "word" in the looped line splitted with space
Loop for each character in the looped "word"

The first loop is to put the first word to sheet2 column A and put the 2nd word to sheet2 column B from the looped line string as i variable. Then it set the rslt to offset(0,2).
In the 2nd loop, it loop to each word in the looped line. Since the first word and the second word not needed anymore, it starts from 2 as j variable. And it put the looped word into splitVal variable, and check if the splitVal value contains number then it go to the third loop.
In the third loop, it check each character of the splitVal (the looped word) value as k variable. if the looped char is "." then it exit the loop  ---> this will ignore the decimal value (since I assume that you don't need a decimal for the result).
Then it check if the looped char is >= 0 and <=9, it put the result to sheet2 column  C (the rslt variable). and set rslt offset(0,1) so the next result will be put in column D.
Please note, the code will fail if the looped word is something like this SGD.400 (there is a dot which is located in front of the number)
Comment: "Code will fail ... if the first word and the second word is separated with more than one space on each line". Also "if the first word started with space".

Both of those can be avoided by calling WorksheetFunction.Trim() before parsing the line. That replaces multiple spaces with a single space, and removes leading and trailing spaces. Note that it is NOT the same as the VBA Trim() statement, which ignores multiple spaces within the line.
Comment: @RichardCook, thank you very much for your help . I will never know it if you don't tell me. I've edited the code as your suggestion.


Text field formatter with link to the entity?

Question: I have a text field that I use to store an entity title.
It is not the "native" entity title, but just a field named field_title.
Now I want to display this text field as a link to the entity (e.g. product/123).
(I am working with Display Suite here, but the question is not limited to that)
I know I could simply create a custom formatter for this. But isn't there an easier way? I think I even did this once, just can't remember.
I also found the Field Value Link Formatter module , but this doesn't seem to be it.
EDIT: There are some known solutions like DS custom fields, or Custom Formatters, or hook_field_formatter_info() in a custom module, or node template override, or field template override, or the awesome hook_ds_field_theme_functions_info().
However: This seems such a recurring use case that I consider to write a new (reusable) module and publish it. So, I am looking for a solution so easy that it invalidates the desire to write a new module :) I think I saw something like this once, just can't remember.
Answer: you could do this with a Display Suite custom field and use the tokens to build the link.
I would do it like that if I wouldn't like to write a custom template. Or, as already mentioned, create a template for that node. Due to the fact that you don't use the native title, you will have it available in the node template and you can change the markup there.
Comment: Yes, the DS custom fields.. This does work, but I am still looking for something easier. This is such a recurring case. Still +1.


Jquery .click() with an unanonymus method issue

Question: i'm following and modifing a tutorial in hope to create an interactive skilltree sheet
var main = function() {
    var points = parseInt($('.resume').text());
    var pointExpended = 0;

    $('.node').click(function() {
        var cost = parseInt($(this).children('.info').children('.cost').text());
        if($(this).hasClass('active-node')) {
            //l'abilità viene rimossa
            $(this).removeClass('active-node');
            pointExpended = pointExpended - cost;
            $('.resume').text(points-pointExpended);
        }
        else {
            //l'abilità viene comprata
            if(points-(pointExpended+cost) < 0) {
                alert("Limite dei punti superato");
            }
            else {
            $(this).addClass('active-node');
            pointExpended = pointExpended + cost;
            $('.resume').text(points-pointExpended);
            }
        }
    });
   };

   $(document).ready(main);

to add some recursion i'm trying to put the handler into an external function
var click_handler = function() {

        alert(points);

        var cost = parseInt($(this).children('.info').children('.cost').text());
        if($(this).hasClass('active-node')) {
            //l'abilità viene rimossa
            $(this).removeClass('active-node');
            pointExpended = pointExpended - cost;
            $('.resume').text(points-pointExpended);
        }
        else {
            //l'abilità viene comprata
            if(points-(pointExpended+cost) < 0) {
                alert("Limite dei punti superato");
            }
            else {
            $(this).addClass('active-node');
            pointExpended = pointExpended + cost;
            $('.resume').text(points-pointExpended);
            }
        }
    }

var main = function() {
    var points = parseInt($('.resume').text());
    var pointExpended = 0;

    alert(points);

    $('.node').click(click_handler);
};

$(document).ready(main);

but the code does not respond to the clicks.
I'm pretty new to jscript/jquery so i can't figure why the first try works and not the latter
Comment: try to put var points = parseInt($('.resume').text());
    var pointExpended = 0;  in the click_handler function not in main function
Answer: variable points and pointEtended are not visible to your click handler (variable scope is a lexical stuff, and does not work on the call stack..) so you need to pass them in with a closure:
var main = function() {
    var points = parseInt($('.resume').text());
    var pointExpended = 0;

    alert(points);

    $('.node').click(function(evt) {
              click_handler(evt, points, pointExtended)
    });
};

also modify you click_handler to accept named values
click_handler(evt, points, pointExtend) { ...
}

Comment: Yes, thanks. I've fixed in a little bit uglyer way than you suggested and made points and pointExpended as global and calling click_handler(evt,$(this))
Comment: nice1... beware the global space though - but I guess you are already aware what a minefield that is


3d animation in android

Question: I am developing a android application which involves rotating the images in a horizontal way.
Can anyone help me in this regard.
Comment: What most people would do would be to look for sample apps, demos, tutorials etc.
Answer: There are 3D flip example in ApiDemos:
In your Exclipse create a new Android project and select "Create project from existing sample". Choose target android version and then ApiDemos.
Run the program and go to Views>Animation>3D transition
There are many good examples there what you can do.
Answer: One of the most convincing 3D flip animation I've seen is done here https://code.google.com/p/android-3d-flip-view-transition.
A lot of the other tutorials and sample codes don't produce believable 3D flips. A simple rotation on the y-axis isn't what's done in iOS.
There is also a video here: http://youtu.be/52mXHqX9f3Y

3d animation in android


Rest framework django

Question: Boa noite, estava estudando um pouco de rest framework no django e desenvolvi uma api simples para consumir com o Angular deu tudo certo porém a minha api que eu fiz é limitada a
{
  [
    {
      "id":1,
      "name":"lima",
      "phone","000000000",
      "email","email@example.com",
      "photo","localhost/members/profile/foto1.jpg"
    },
    {
      "id":2,
      "name":"Carlos",
      "phone","000000000",
      "email","email@example.com",
      "photo","localhost/members/profile/foto2.jpg"
    }
  ]
}

Ela era limitada a essa estrutura json porém eu queria fazer algo mais completo e um pouco mais complexo kk vou dar um exemplo aqui de como eu queria:
{
   [
      "id": 1,

      "User": {

         "name":"João Vitor",

         "phone":"000000000",

         "email": "firstname.lastname@example.com",
         "configuracoes": {
               "aqui vem algumas configurações setadas pelo usuário
         }

     },

     "pedidos": [

          {

             "Aqui vem uma lista de pedidos desse usuário que vai chegar através de outra aplicação
          }
     ]
   ]
}

Consumir a e manipular a api usando o Angular eu to conseguindo de boa o problema ta sendo em como eu vou criar essa api mais complexa ai. Vou deixar o aquivo de model onde eu descrevo a minha api e depois faço o migrate desse arquivo.
Esse é o arquivo de migrate 0001
# Generated by Django 3.0.6 on 2020-05-28 14:57

from django.db import migrations, models

class Migration(migrations.Migration):

    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Member',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=100)),
                ('surname', models.CharField(max_length=100)),
                ('phone', models.CharField(max_length=100)),
                ('email', models.EmailField(max_length=254)),
                ('addres', models.CharField(max_length=100)),
                ('photo', models.ImageField(upload_to='members_profile')),
            ],
        ),
    ]

E esse é o model que gerou esse arquivo:
from django.db import models

# Create your models here.
class Member(models.Model):
    name = models.CharField(max_length=100)
    surname = models.CharField(max_length=100)
    phone = models.CharField(max_length=100)
    email = models.EmailField()
    addres = models.CharField(max_length=100)
    photo = models.ImageField(upload_to='members_profile', blank=True, null=True )

    def __str__(self):
        return f'{self.name} {self.surname}'

Comment: Olá viva. Partilha os models que estás a utilizar que precisas para a tua api terás um objeto member outro user e por fim configuracoes? Outra coisa, elimina o migrations, não tendo nenhum erro associado, não é relevante apresentares aqui, não acrescenta valor à tua questão. Outra coisa que fiquei com a impressão, posso estar enganado, é que desenvolveste o client e agora estás no backend, dá uma vista de olhos no conceito api-first https://dzone.com/articles/an-api-first-development-approach-1. É possivel obter a api como queres, mas terá de ser criado a estrutura de dados correta (models).
Answer: A estrutura que você deseja retornar não corresponde as boas práticas (acredito que seja até inviável), até onde eu entendi é uma lista que tem um char, um objeto e uma outra lista de sets.
A solução que eu irei informar é uma lista de objetos onde nesse objeto irá ter um id, o objeto do usuário e a lista de pedidos.
Uma solução que facilitaria a construção dessa estrutura seria utilizar o SerializerMethodField, esse atributo funciona a gente definir o retorno de um determinado campo a partir de um metodo no nosso serializer.
Segue o serializer para a estrutura que mencionei:
class UsuarioSerializer(serializers.Serializer):
    name = serializers.CharField()
    phone = serializers.CharField()
    email = serializers.CharField()
    configuracoes = serializers.SerializerMethodField()

    def get_configuracoes(self, objeto):
        configuracoes_usuario = ... #Filter ou alguma forma de associar as configurações ao objeto usuario.

        return configuracoes_usuario # Retorna uma lista ou dicionario de configuracoes

class InterfaceComplexaSerializer(serializers.Serializer):
    id = serializers.IntegerField()
    User = serializers.SerializerMethodField()
    pedidos = serializers.SerializerMethodField()

    def get_User(self, objeto):
        serializer_usuario = UsuarioSerializer(objeto)

        return serializer_usuario.data

    def get_pedidos(self, objeto):
        lista_pedidos = ... #Um filter ou alguma forma de associar os pedidos ao objeto usuario.

        return lista_pedidos

...: meu_serializer = InterfaceComplexaSerializer(
...:    [
...:        objeto_usuario1,
...:        objeto_usuario2,
...:        objeto_usuario3
...:    ],
...:    many=True
...:)

...: print(meu_serializer.data)

Resultado Final:
>   [
...:   {
...:      "id": 1,
...:      "User": {
...:         "name":"João Vitor",
...:         "phone":"000000000",
...:         "email": "firstname.lastname@example.com",
...:         "configuracoes": {
...:            "config1": "teste",
...:            "config2": "teste2",
...:            "config3": "test3"
...:         }
...:     },
...:     "pedidos": [
...:        "Aqui vem uma lista de pedidos desse usuário que vai chegar através de outra aplicação",
...:        "Outra mensagem",
...:        "Outra mensage2"
...:     ]
...:   }
...:   ...
...:]

Importante:
O único problema que você tem que se atentar é que a cada objeto usuário, os metódos "get_configuracoes", "get_User" e "get_pedidos" irá ser executado. Ou seja, se em get_configuracoes tiver uma busca com o filter do models, e eu passar 10 objetos usuários para o "InterfaceComplexaSerializer" irá ser feito 10 hits/queries para o banco de dados.
Answer: Para ter algo mais completo e complexo, como você disse, é preciso criar um campo do tipo ManyToManyField no seu Model. Exemplo:
from django.db import models
from configuracoes.models import Configuracao

# Create your models here.
class Member(models.Model):
    name = models.CharField(max_length=100)
    surname = models.CharField(max_length=100)
    phone = models.CharField(max_length=100)
    email = models.EmailField()
    addres = models.CharField(max_length=100)
    photo = models.ImageField(upload_to='members_profile', blank=True, null=True )
    configuracoes = models.ManyToManyField(Configuracao)

    def __str__(self):
        return f'{self.name} {self.surname}'

Porém, apenas isso não basta. Você também precisa implementar no Serializer do usuário o Serializer da configuração. Exemplo:
from rest_framework.serializers import ModelSerializer
from member.models import Member
from configuracoes.api.serializers import ConfiguracaoSerializer

class MemberSerializer(ModelSerializer):
    configuracoes = ConfiguracaoSerializer(many=True)

    class Meta:
        model = Member
        fields = (
            'id', 'name', 'surname', 'phone', 'email', 'addres', 'photo',
            'configuracoes'
        )

Espero ter ajudado.
Abraços,

Rest framework django


Pearson correlation - sample independence assumption

Question: I have a couple of specific questions regarding the underlying assumptions of the pearson's correlation. I have googled this question but I have not managed to find a consistent answer. In particular whether there is an assumption of sample by sample independence. I will give an example to clarify as I may not be using the correct nomenclature. Assume, I have taken 50 measurements of height and weight from 40 people. These measurements are:
-Measurement of weight and height of 10 people.
-Measurement of weight and height of the same 10 people but repeated at a different point in time.
-Measurement of weight and height of a different group of 30 people.
If I conduct a pearson's correlation for height and weight, using all 50 measurements, is there an underlying assumption of sample independence that is being violated? If there is no underlying assumption of independence being violated, how can generalisation of the result be interpreted?
Any responses are appreciated.
Answer: Well, as far as I'm aware there is no strict assumption of sample-by-sample independence, the way you mean it. The closest thing I can find is the assumption of no auto-correlation in a variable, meaning one sample should not depend on its previous values (e.g. as is the case in time series).
The main problem in the example you described is that your data, while measuring the same thing, originates from different distributions. Which will hurt your outcome in any case (not just linear regression.
Comment: Thank you for your response. Can you clarify what are the different distributions you are referring to as I am not sure I understand this part of your answer.
Comment: I mean that in the case of a simple height/weight measurment, your data should follow a normal distribution. When measuring the height of a single person over 10 years your distribution should be uniform (height stays the same on adults) while the weight should have ups and downs (this case will definitely have auto-correlation). If your data consisted samples generated by these two techniques your variables would be [multimodal](https://en.wikipedia.org/wiki/Multimodal_distribution).
Comment: Thanks. Would doing a boot strapped pearson correlation be appropiate?
Comment: I can't answer that for sure...
Comment: I've read about a little bit and although no auto-correlation is a requirement for regression it does not appear to be a requirement for conducting pearson correlation.
Comment: But in order to achieve a high r, you need your data to have no (or at least little) auto-correlation...
Comment: Sure but there is no fundamental reason why you can't conduct the pearson correlation. You are just limited by how you can interpret the result.
Comment: No you can conduct anything you want (even linear regression), but the underlying assumption that would make it successful would be broken. Another thing you should take a look at is the statistical importance of the correlation you computed (or the [p-value](https://en.wikipedia.org/wiki/P-value)) which would give a lower score on auto-correlating data.


Is every closed ball (or open ball) in the Eucledean Space $R^n$ convex?

Question: I am solving a problem and I need to use this fact:
Every closed ball (or open) in  the Eucledean Space $R^n$ convex?
Hoever, I am not sure if it is true or not. Can anyone help? Thanks!
Comment: What you would need to show is that if $d(x,x_0)\leq \epsilon,$ then for every $y$ on the line segment from $x_0$ to $x$, $d(y,x_0)\leq \epsilon$. Is this true?
Comment: @Kevin Carlson: What I wanted to show is if you pick any two points $x$ and $y$ inside the ball, then the line segment joining those two points lie inside the ball as well.
Answer: Suppose that $x$ and $y$ are in $B(z,r)$. Then, for $\theta\in[0,1]$,
$$
|\theta x+(1-\theta)y-z|=|\theta(x-z)+(1-\theta)(y-z)|\\
\leq\theta|x-z|+(1-\theta)|y-z|<\theta r+(1-\theta)r=r
$$
so that $\theta x+(1-\theta)y$ is also in $B(z,r)$. The case for the closed ball is similar (the only difference is that the last inequality above will be weak.)


Cant debug with react native debugger

Question: I am starting project as instructed in the readme file (mcnamee/react-native-starter-kit). I integrated my firebase with app when I signup it stucks at spinner... though saves email and password to firebase and it signs in without problem. I want to debug whats going on when I hit the signup button but it doesnt show any action etc nor in the react native debugger app. Why react native debugger cant connect to my app. How can I fix this issue?
Answer: You can only have one debugger attached at a time. If you are using the debugger in a web browser. You cannot use React Native Debugger.
The simplest solution is to make sure that you have React Native Debugger running before you do react-native run-ios And you should connect automatically to the React Native Debugger. If it doesn't connect you can always press cmd+d to connect to the debugger.
If you are already connected to the web browser debugger. Press cmd+d to bring up the menu on the simulator, disable remote debugging. Close the web browser. Launch React Native Debugger. Press cmd+d and enable remote debugger, this should connect you to the React Native Debugger.
Regards to the information in the console. Both debugging optionswill give you the same information. If you are wanting to debug what is happening in more detail you'll need to use the built in Debugger in your IDE and step through the lines of code or put in some well placed console.logs().
Comment: I tried What you said, it fires chrome automatically and opens web ui debugger. also I still cant see actions in the redux chrome plugin neither.
Comment: I am tryingto see which actions fired with which data.. I still not able to see actions.
Comment: It is directly openning debugger on chrome even if RND open before npm start
Comment: @beratuslu can you switch to the RND?
Comment: How can I switch it?
Comment: @beratuslu If you are already connected to the web browser debugger. Press cmd+d to bring up the menu on the simulator, disable remote debugging. Close the web browser. Launch React Native Debugger. Press cmd+d and enable remote debugger, this should connect you to the React Native Debugger.
Comment: think you need to add "postinstall": "rndebugger-open --expo" to "scripts": in package.json then run yarn run postinstall then expo start
Comment: I was facing this issue with Debugging. Yes, running react-native-debugger before running ios simulator worked for me.
Answer: The problem was RND was opening in different port. I changed the port from RND menu and it worked like expected.
Comment: The same fix worked for me. Check that the port being used is what you expected. Mine suddenly changed from 19001 to 19004. Include this in the array of default RN ports in your RN debugger config file.
Answer: @beratuslu
you wrote in the comments

I tried What you said, it fires chrome automatically and opens web ui
debugger. also I still cant see actions in the redux chrome plugin
neither.

This was the same problem with me and after spending more than 10 working hours, I solved it by unchecking the small tick saying Maintain Priority

So, in case someone else is still trying to solve this issue when doing all the configuration as mentioned in the docs, still the default debugger fires automatically instead of connecting to react native debugger, follow these steps.

Uncheck the Maintain Priority checkbox in default debugger-ui
Close the default debugger
in simulator CTRL + M and stop remote debugging
CTRL + T in the React Native Debugger window
Enter your debugger port (in my case it was 19000) and click Confirm
In your android emulator, enable the remote debugging again (CTRL + M -> Start remote debugging)
Now it should be connected to the React Native debugger instead of default debugger-ui

Hope it helps someone to save time in figuring out why the default debugger is firing up ever time instead of connecting to react native debugger.
Answer: @beratuslu You wrote in the comments:

I tried What you said, it fires chrome automatically and opens web ui
  debugger. also I still cant see actions in the redux chrome plugin
  neither.

How do you start your debugging-process? Might be, that you try to start with Jetbrains-Product like WebStorm, or PHPStorm.
In that case, you need to know, that this doesn't work yet. You need to start react-native-debugger manually (not via jetbrains software), and if running, use only "run" (in jetbrain-software) to start your DEV-Build.

This way, the Connection to the react-native-debugger will be established, if the App (with DEV_Flag) is up and running.
If you need to debug with breakpoints, you have to go with Chrome-Browser, but there you don't have redux-output... while on the other hand you react-native-debugger support it.
If you wan't to see jetbrain-debug working with react-native-debugger feel free to vote for this Issue: https://youtrack.jetbrains.com/issue/WEB-35251
Answer: Here are some steps you can follow:
Use Console Logs:

console.log(),
console.warn(),
and console.error()

to print out values and follow the flow of your application.

React Native Debugger:
This is a standalone app that provides a powerful debugging environment. It integrates with the Chrome Developer Tools.
Error Boundaries:
Implement error boundaries in your React components to catch JavaScript errors anywhere in their child component tree.

Cant debug with react native debugger


The basic logic of constructing a confidence interval

Question: Consider a model with a parameter of interest, $\theta$, and its point estimator, $\hat\theta$. For simplicity, assume  $\hat\theta\sim N(\theta,\sigma^2/n)$ (in numerous instances this could be justified asymptotically). There are two ways of constructing an interval that happens to be the shortest possible $(1-\alpha)$ level confidence interval.

For any true value $\theta$, I want the shortest possible interval $(\hat\theta_{lower},\hat\theta_{upper})$ that has $(1-\alpha)$ probability of capturing $\theta$. I select the highest density region in the distribution of $\hat\theta$ given $\theta$, $f(\hat\theta;\theta)$, so that the cumulative probability for that region is $(1-\alpha)$. I define the interval estimator such that for every point estimate $\hat\theta$ in the region, the corresponding interval estimate would cover $\theta$.
Since the distribution of $\hat\theta$ is the same for any true value $\theta$ except for a location shift, the mechanism (the rule) for constructing the interval is independent of what the actual $\theta$ is. Hence, it will cover any true $\theta$ with $(1-\alpha)$ probability.
Given a point estimate $\hat\theta$, I am considering under what true value $\theta$ it is likely to have been generated. Knowing the distribution of $\hat\theta$ for any given true $\theta$, $f(\hat\theta;\theta)$, I select those $\theta$s that yield the highest density values. I limit the selection to only include  values $\theta$ that have the cumulative probability $\geq\alpha$ for values at least as extreme as $\theta$; in other words, the values $\theta$ for which the corresponding $p$-value associated with $\hat\theta$ is at least $\alpha$.

The first approach focuses directly on ensuring that whatever the true $\theta$, it is included in $(1-\alpha)$ share of sampling instances. The second approach looks for the best candidate $\theta$s that make the realization $\hat\theta$ likely, while discarding $\theta$s under which $\hat\theta$ is unlikely. The line between the two (likely vs. unlikely) is drawn somewhat arbitrarily from the perspective of the original goal, but it happens to be the right line.
The two rules for constructing an interval give the same answer in this simplified example.
Which (if any of the two) is the correct motivation for, or the correct way of thinking about, the construction of a confidence interval?
(Perhaps removing the distributional assumption for $\hat\theta$ above would invalidate one of the approaches, making it clear that it is generally inappropriate and only gives the right answer in this example by coincidence?)
Comment: What (if any) is your motivation here? It seems like a very subtle difference for the vast majority of cases.
Comment: @Xiaomi, thank you for your interest! The result of the two approaches is the same, but the way of arriving at it is apparently very different (at least that is my perception). I wonder which way one should proceed to stay true to the logic (and probably the history) of confidence interval estimation. Perhaps one of the ways would only occasionally give the right answer. I am quite concerned, even though the difference might appear subtle to some. Would you happen to know the answer?
Comment: After Martijn Weterings's answer I am starting to think that the second approach might be a special case of constructing a *credible* interval (with a flat prior on $\theta$).
Comment: Related question about the difference between confidence interval and credible interval with flat prior  https://stats.stackexchange.com/questions/355109/if-a-credible-interval-has-a-flat-prior-is-a-95-confidence-interval-equal-to-a  (they are *not* the same, and especially noteworthy is that a confidence interval does not change with a change of variables whereas a credible interval, where the prior needs to be changed if you want to keep it 'flat', does not remain the same)
Comment: The last sentence of the paragraph explaining the second method *"I limit the selection to only include...the values $\theta$ for which the corresponding p-value associated with $\theta$ is at least $\alpha$"* is actually very much the same as the first method.
Answer: Example with 100 Bernoulli trials
The construction of confidence intervals could be placed in a plot of $\theta$ versus $\hat{\theta}$ like here:
Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?
In my answer to that question I use the following graph:

Note that this image is a classic and an adaptation from  The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial C. J. Clopper and E. S. Pearson Biometrika Vol. 26, No. 4 (Dec., 1934), pp. 404-413
You could define a $\alpha$-% confidence region in two ways:

in vertical direction $L(\theta) < X < U(\theta)$ the probability for the data $X$, conditional on the parameter being truly $\theta$, to fall inside these bounds is $\alpha$ .

in horizontal direction $L(X) < \theta < U(X)$ the probability that an experiment will have the true parameter inside the confidence interval is $\alpha$%.

Correspondence between two directions
So the key-point is that there is a correspondence between the intervals $L(X),U(X)$ and the intervals $L(\theta),U(\theta)$. This is where the two methods come from.
When you want $L(X)$ and $U(X)$ to be as close as possible ("the shortest possible ($1−\alpha$) level confidence interval") then you are trying to make the area of the entire region as small as possible, and this is similar to getting $L(\theta)$ and $U(\theta)$ as close as possible. (more or less, there is no unique way to get the shortest possible interval, e.g. you can make the interval shorter for one type of observation $\hat\theta$ at the cost of another type of observation $\hat\theta$)

Example with $\boldsymbol{\hat\theta \sim \mathcal{N}(\mu=\theta, \sigma^2=1+\theta^2/3)}$
To illustrate the difference between the first and second method we adjust the example a bit such that we have a case where the two methods do differ.
Let the $\sigma$ not be constant but instead have some relation with $\mu= \theta$ $${\hat\theta \sim \mathcal{N}(\mu=\theta, \sigma^2=1+\theta^2/3)}$$
then the probability density function for $\hat \theta$, conditional on $\theta$ is $$f(\hat\theta, \theta ) = \frac{1}{\sqrt{2 \pi (1+\theta^2/3)}} exp \left[ \frac{-(\theta-\hat\theta)^2}{2(1+\theta^2/3)} \right] $$
Imagine this probability density function $f(\hat \theta , \theta)$ plotted as function of $\theta$ and $\hat \theta$.

Legend: The red line is the upper boundary for the confidence interval and the green line is the lower boundary for the confidence interval. The confidence interval is drawn for $\pm 1 \sigma$ (approximately 68.3%). The thick black lines are the pdf (2 times) and likelihood function that cross in the points $(\theta,\hat\theta)=(-3,-1)$ and $(\theta,\hat\theta)=(0,-1)$.
PDF In the direction from left to right (constant $\theta$) we have the pdf for the observation $\hat \theta$ given $\theta$. You see two of these projected (in the plane $\theta = 7$). Note that the $p$-values boundaries ($p<1-\alpha$ chosen to be the highest density region) are on the same height for a single pdf, but not for not at the same height for different pdf's (by height that means the value of $f(\hat\theta,\theta)$)
Likelihood function In the direction from top to bottom (constant $\hat \theta$) we have the likelihood function for $\theta$ given the observation $\hat\theta$. You see one of these projected on the right.
For this particular case, when you select the 68% mass with the highest density for constant $\theta$ then you do not get the same as selecting the 68% mass with the highest likelihood for constant $\hat \theta$.
For other percentages of the confidence interval you will have one or both of the boundaries at $\pm \infty$ and also the interval may consist of two disjoint pieces. So, that is obviously not where the highest density of the likelihood function is (method 2). This is a rather artificial example (although it is simple and nice how it results in these many details) but also for more common cases you get easily that the two methods do not coincide (see the example here where the confidence interval and the credible interval with a flat prior are compared for the rate parameter of a exponential distribution).
When are the two methods the same?
This horizontal vs vertical is giving the same result, when the boundaries $U$ and $L$, that bound the intervals in the plot $\theta$ vs $\hat \theta$ are iso-lines for $f(\hat \theta ; \theta)$. If boundaries are everywhere at the same height than in neither of the two directions you can make an improvement.
(contrasting with this: in the example with $\hat \theta \sim \mathcal{N}(\theta,1+\theta^2/3)$ the confidence interval boundaries will not be at the same value $f(\hat \theta, \theta)$ for different $\theta$, because the probability mass becomes more spread out, thus lower density, for larger $\vert \theta \vert$. This makes that $\theta_{low}$ and $\theta_{high}$ will not be at the same value $f(\hat \theta ; \theta)$, at least for some $\hat \theta$, This  contradicts with method 2 that seeks to select the highest densities $f(\hat \theta ; \theta)$ for a given $\hat \theta$. In the image above I have tried to emphasize this by plotting the two pdf functions that relate to the confidence interval boundaries at the value $\hat \theta= -1$; you can see that they have different values of the pdf at these boundaries.)
Actually the second method doesn't seem entirely right (it is more a sort of variant of a likelihood interval or a credible interval than a confidence interval) and when you select $\alpha$% density in the horizontal direction (bounding $\alpha$ % of the mass of the likelihood function) then you may be dependent on the prior probabilities.
In the example with the normal distribution it is not a problem and the two methods align. For an illustration see also this answer of Christoph Hanck. There the boundaries are iso-lines. When you change the $\theta$ the function $f(\hat\theta,\theta)$ only makes a shift and does not change 'shape'.
Fiducial probability
The confidence interval, when the bounds are created in vertical direction, are independent of the prior probabilities. This is not the case with the 2nd method.
This difference between the first and the second method may be a good example of the subtle difference between fiducial/confidence distribution $\frac{d}{d\theta}F(\hat\theta,\theta)$ and the likelihood function $\frac{d}{d\hat\theta}F(\hat\theta,\theta)$ (where $F$ is the cumulative distribution function).
Comment: @whuber the image is pretty old,  I used the image from here https://www.jstor.org/stable/2331986   *The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial
C. J. Clopper and E. S. Pearson
Biometrika
Vol. 26, No. 4 (Dec., 1934), pp. 404-413*   (and I agree that the idea of choosing an area such that you get 95% of the mass is not correct, only one of those regions will result in confidence intervals, the problem might be in the statement of the question "I want the shortest possible interval" which is ambiguous. There is no single unique way to get this.)
Comment: Good points. I was suspecting some prior probabilities might be be trying to sneak in in the second approach...
Comment: I will try and see whether I can come up with some better visual representation. When you would plot the $f(\hat \theta; \theta)$ as a surface then you get some ridge shape, but in the case of the Bernouilli trials that shape is smaller and higher at the edges. In the case of the normal distribution it is more symmetric.
Comment: That (a new visual representation) might help a lot! Besides, could you elaborate on *Imagine the probability density function $f(\hat \theta ; \theta)$ for $\hat \theta$ conditional on $\theta$ plotted in 2D $\theta$ versus $\hat \theta$*? Could you rephrase it somehow? I am having trouble understanding this and then consequently what is meant by horizontal vs. vertical in the remainder; perhaps you could give names like "the $\theta$ direction" for horizontal and "the $\hat\theta$ direction" for vertical (or otherwise, whichever is correct).
Comment: I would like to add a picture of that. It is similar to the current image. We normally see $f(\hat \theta ; \theta)$ as a function with $\theta$ fixed but we could change it into a function $f(\hat \theta, \theta)$ with $\theta$ *not* fixed. Then when we make a confidence interval we create boundaries $L(\theta)$ and $U(\theta)$ that bound, in vertical ($\hat \theta$) direction, $\alpha$% of the mass. Since we do this for every $\theta$ we will in the 2D image have boundaries that contain $\alpha$% of the mass. We could imagine doing the same in the other direction (but it will be different).
Comment: Thanks! When I think of the bivariate function $f(\hat\theta,\theta)$, I think of the function value to be on the vertical axis with the two horizontal directions being $\hat\theta$ and $\theta$; that is why I was asking. A picture would really help :)
Comment: I edited the question slightly to disambiguate the method 2; that is, I crossed out a small piece but left it visible (crossed out). After that, how do you reconcile **(1a)** the definition of the second method in my question **and (1b)** *the second method doesn't seem entirely right* in your (this) answer **with (2)** *You might see the confidence intervals as being constructed as a range of values for which an $\alpha$ level hypothesis test would succeed* in your answer in the linked post? Aren't **(1a)** and **(2)** these the same (at least for the newly edited version)?
Comment: @RichardHardy I guess that it is still problematic and it is due to the phrase *"I select those $\theta$s that yield the highest density values at $\hat \theta$"*. A confidence interval is not (in general) the selection of $\theta$ that give the highest density values $f(\hat \theta; \theta)$ for a fixed $\hat \theta$ (in other words the values $\theta$ with the highest likelihood). The follow up part where you speak about *"I limit the selection to only include..."* is confusing.
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/83990/discussion-between-richard-hardy-and-martijn-weterings).
Comment: Your image reminds me of graphics I posted at http://www.quantdec.com/envstats/notes/class_12/ucl.htm. I think "area of entire region" has to be understood not as an actual area, though, for two reasons. First, without any prior on $\theta$ there is no basis for any measure in the horizontal direction; and second, there *are* measures in the vertical direction: namely, the distribution of $x$ determined by each $\theta.$ Thus I think your key point is a step in the right direction but needs more elaboration and nuance. Regardless, only one of the two methods is actually a confidence interval.
Comment: I cannot say I have understood the topic well, but your answer is certainly very helpful and clearly the best of all those I have receivedwe have (1 out of 1 at the moment). I am accepting it.


Scraping thread-href

Question: import bs4
from bs4 import BeautifulSoup as soup
from urllib.request import urlopen as uReq
import requests
import re
from pyquery import PyQuery as pq
from requests.exceptions import RequestException

my_url = 'http://club.baby.sina.com.cn/forum-112-1.html'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

page_soup = soup(page_html, "html.parser")
page_soup.find_all('span')

The above codes return:
[,

 亲子论坛 »

            ,
 ,
 ,
 ,
  ,
 **关于育儿专家答疑论坛的一些问题**,
 亲子专家张思莱免费讲座通知,
 轮滑是幼儿时期的一项最好的运动,
 关于婴儿身上的各种"记"(转贴),
 关于推荐婴幼儿钙和维生素D的参考摄入量的问题,
 婴幼儿奶粉怎么选择?,
 关于辅食添加的问题,
 再一次希望妈妈好好看看我的文章:如何向医生介绍孩子病情,
 婴幼儿便秘产生的原因及对策,
 工晶体是不是越贵越好,

In this case, I'm trying to parse out the  so that I can go to each thread post to scrape the first thread of each post. Can anyone help on parsing this href tag?
Answer: First of all, finding all span elements is too much - you are matching more than you are up to. Let's scope our search to span elements that have id attribute values starting with thread:
threads = page_soup.select('span[id^=thread]')

where ^= means "starts with".
Then, if you want to get the thread titles, just get the text of the inner a elements:
threads = page_soup.select('span[id^=thread]')
for thread in threads:
    print(thread.a.get_text())

Prints:
**关于育儿专家答疑论坛的一些问题**
亲子专家张思莱免费讲座通知
轮滑是幼儿时期的一项最好的运动
...
剖蓓舒要火,因为一份爱!
小儿肺热有那些症状

Scraping thread-href


Measure Defined by Lebesgue Integral Of Another Measure

Question: Let $(\Omega, \mathcal{A}, \mu)$ be a measure space and $f: \Omega \to \mathbb{R}$ a measure nonnegative function that is summable against $\mu$.
It is known that $$\nu(A) = \int_{A} f \mu$$ is also a measure on $\mathcal{A}$. I have to prove that $$\int_{\Omega} g \nu = \int_{\Omega} gf \mu$$ for all functions $g: \Omega \to \mathbb{R}$ that are summable against $\nu$.
I don't quite understand the notation, though. Specifically, on the proof statement, what does $gf \mu$ mean? Am I taking the composition of $g$ and $f$? How does one even evaluate $\int_{\Omega} g \nu$?
Thanks in advance for any help.
Comment: The more common notation is $\nu(A)=\int_Af\,d\mu$; the integral of $f$ with respect to the measure $\mu$. You're then supposed to show $\int_{\Omega}g\,d\mu=\int_{\Omega}g\cdot f\,d\mu$; the RHS has a product of $g$ and $f$. In words: show Lebesgue integral of $g$ with respect to $\nu$ equals Lebesgue integral of $gf$ (a product) with respect to $\mu$. These are all Lebesgue integrals with respect to the measures involved, so the definition goes like non-negative simple functions $\to$ non-negative measurable functions $\to$ real integrable $\to$ complex-valued integrable functions.
Comment: So to start off with simple functions, if I just consider indicator function, $g = a * X_A,$, $$\int g \nu = a \cdot \nu (A) = a \cdot \int_{A} f \mu = a \cdot \int X_{A} f \mu = \int a X_{A} f \mu = \int gf \mu.$$ Does that work?
Comment: Yes that's exactly right. Now use linearity to get all simple functions, and monotone convergence to get all non-negative measurable functions.
Comment: @peek-a-boo does "all functions summable against $\nu$" mean I have to go further than non-negative measurable functions? Do I need Dominated Convergence Theorem like how Mahdi said? Or is this much enough?
Comment: if you know the equality for all non-negative functions then for integrable functions the statement becomes clear by considering positive and negative parts.
Answer: Regarding your questions on the notation $\nu(A)=\int f d\mu$ and $\int g d \nu=\int fg d \mu$, and in particular $fg$ means multiplication of $f$ and $g$.
The question you are asking can be answered using Radon Nikodym theorem. However, you can also directly approach this question by assuming first that $g$ is a simple function. Then using the fact that you can approximate any measurable function with simple functions and dominated convergence theorem you can generalize it to any function.


Problema usando RotativaPdf no aspnet core 3

Question: Tenho seguido as configurações recomendadas na documentação, realizei várias pesquisas, não tenho erro no exemplo mais o relatório não e gerado. Inicialmente fiz a instalação do :
Install-Package Rotativa.AspNetCore -Version 1.2.0-beta

Configuração do Startup
RotativaConfiguration.Setup(env.WebRootPath, "Rotativa");

Exemplo do meu controller Home (Dentro da pasta Home, eu tenho uma View com o nome = VisualizarPDF)
  public class Person
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string LastName { get; set; }
}

public IActionResult RotativaPDF()
{
    Person person = new Person()
    {
        Id = Guid.NewGuid(),
        Name = "Primeiro",
        LastName = "Ultimo"
    };

    var relatorioPDF = new ViewAsPdf
    {
        ViewName = "VisualizarPDF",
        IsGrayScale = false,
        FileName = "Fornecedor.pdf",
        PageOrientation = Rotativa.AspNetCore.Options.Orientation.Landscape,
        PageSize = Rotativa.AspNetCore.Options.Size.A4,
        Model = person
    };

    return relatorioPDF;
}

public IActionResult Index()
{
    RotativaPDF();
    return View();
}

Answer: A solução e instalar a versão
Install-Package Rotativa.AspNetCore -Version 1.1.1

No Startup basta fazer a configuração:
//Configuração do rotativa
            RotativaConfiguration.Setup(env2, "Rotativa");

Altera:
   [Obsolete]
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env, Microsoft.AspNetCore.Hosting.IHostingEnvironment env2)

Isso resolve o problema


How to publish and unpublish content depending on the type of user?

Question: I have a site with content for two different user types: primary and secondary. I want to publish and unpublish some content depending on whether the current user is anonymous user, user primary or secondary user.
If the current user is an anonymous user:

to publish primaria/docentes
to publish secundaria/docentes
to unpublish primaria/docentes/kit
to unpublish secundaria/docentes/kit

If the current user is a primary user:

to unpublish primaria/docentes
to publish primaria/docentes/kit

If the current user is a secondary user:

to unpublish secundaria/docentes
to publish secundaria/docentes/kit

I try to do this with the the Rules module. The result of my configuration is this:
 - In the computer 1, I login as a primary user. Everything works fine.
 - In the computer 2, I login as a secondary user. Everything works fine.
 - If I return to the computer 1, the things are not good. I see that everything works as if logged in as a secondary user.
My settings do not respect the rules for the user already logged on. Every time someone logs on to any part of the world, that user settings apply to all other users in the world.
Please help me. I attached screenshots of my settings. :(
All my rules:
Log in as an anonymous user.
When primary user logs.
When primary user logs off.
Comment: Sounds like what you want is different access permissions (ie: an anon user doesn't see primaria/docentes/kit but a primary user does) instead of publishing states, which are global (ie: no one can see vs anyone with the appropriate access permissions can see).
Comment: OK. I did not know that publish and unpublish are global actions. You're right: An anonymous user can not see "primaria/docentes/kit". But a primary user does. @ErinMcLaughlin, thanks for your support. I follow the direction of No Sssweat.
Answer: Publishing and unpublishing is not the way to go.
Step 1)
Install the Node Access module and give access to each node or content type accordingly, this way people can't share direct links with other users that have a different role.
Step 2)
What you need is to create 3 different view blocks
One for anonymous which lists primaria/docentes & secundaria/docentes nodes.
One for primary which lists all the primaria/docentes nodes
One for secondary which lists all the secundaria/docentes/kit nodes.

Hopefully node access module lets you filter your view by node access.
  Otherwise, you'll need to make the "kit", "primaria" and "secundaria"
  be a different content types so then you can filter by content type in
  your view.

Use view's access to role and select the role you want to be able to see the view. (see picture below)

Then you place all 3 view blocks wherever you need them (home page perhaps?) and since you have the view access by role, they will only see 1 view (the one they have access to).
Comment: It works now! Thanks @NoSssweat. I did not use Views. Everything worked only with Nodeaccess module.


How to create a video that plots the scatter plots one by one

Question: I'm plotting some data as scatter plots which is overlaid on an image. I would like to make an animation of this by plotting one scatter point at a time. This is the code I have right now using and editing the answer from here:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

x = random.sample(range(0, 287), 20)
y = random.sample(range(0, 380), 20)
size =  [20 for x in range(20)]
colors = ["r" for x in range(20)]

cm = plt.get_cmap('jet')

fig = plt.figure(figsize=(18,9))

graph = plt.scatter([], [],marker='+')
url = 'https://raw.githubusercontent.com/kornelski/pngquant/master/test/img/test.png'
im = plt.imread(url)
def animate(i):

    implot = plt.imshow(im)
    graph.set_offsets(np.vstack((x[:i+1], y[:i+1])).T)
    graph.set_sizes(size[:i])
    graph.set_facecolors(colors[:i+1])
    return graph

ani = FuncAnimation(fig, animate, repeat=False, interval=0.1)
plt.show()

There are two things I would like help with.

I would like the color of my scatterplot to change based on a third variable, i.e use a cmap. However, the set_facecolors does not accept such an argument.
When I try to save my animation using ani.save('files/animation.gif',writer='imagemagick', fps=60) my jupyter notebook crashes.
Answer: The background image of the graph is drawn by adding ax. The color map is also created according to the number of data, 20, and a list is created so that each color can be displayed. Since the coordinates of the image and the coordinate basis of the graph are different, the y-axis is set in the opposite direction.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
import random
from PIL import Image
import urllib.request

random.seed(20210702)
N = 20
x = random.sample(range(0, 380), N)
y = random.sample(range(0, 287), N)
size =  [20 for x in range(N)]
colors = []

cm = plt.get_cmap('jet', N)

fig,ax = plt.subplots(figsize=(9, 4.5))
plt.xlim(0, 380)
plt.ylim(287, 0)
graph = ax.scatter([], [], marker='+')#

url = 'https://raw.githubusercontent.com/kornelski/pngquant/master/test/img/test.png'
im = Image.open(urllib.request.urlopen(url))
print(im.size)

def animate(i):
    ax.imshow(im)
    graph.set_offsets(np.vstack((x[:i+1], y[:i+1])).T)
    graph.set_sizes(size[:i+1])
    colors.append(cm(i))
    graph.set_facecolors(colors)
    return graph

ani = FuncAnimation(fig, animate, frames=20, repeat=False, interval=200)
plt.show()

Comment: This works perfectly, thank you. For the `size` array, am I correct in understanding that it should be the size of the number of frames?
Comment: Yes, that's right. You can change the size of the marker by changing the value of the size array.


Por que o Pycharm não mostra os métodos de uma classe?

Question: Quero saber por que o Pycharm não mostra os métodos de uma classe ao apertar control + spaço. Normalmente as IDE'S mostram todos os métodos presente na classe. Na imagem abaixo mostro o que a IDE me sugere de métodos para qualquer classe.
Comment: https://pt.stackoverflow.com/questions/306887
Answer: Consegui solucionar o meu problema atualizando o interpretador do Python dentro do Pycharm.
1) Primeiro cliquei na aba superior em File ->Settings.
2) Cliquei em Project: untitled (Nome do meu projeto) -> Project interpreter -> Show All..

3) O próximo passo foi entrar em add interpreter(+).

4) Por fim, entrei em Virtualenv Environment, selecionei New environment, selecionei o interpretador Python em Base interprete. Depois marquei as opções e cliquei Ok.

O Pycharm fez algumas atualizações do interpretador e ao acabar passou a mostrar todos os métodos pertencentes a classe. O processo é o mesmo tanto em Linux quanto em Windows.


equivalent expressions for trace norm

Question: I understand the trace norm (or nuclear norm) of a matrix $X\in\mathbf{R}^{n\times m}$is usually defined as
$$\|X\|_{tr}=\sum_{i=1}^{\min\{m,n\}}\sigma_i$$
where $\sigma_i$'s are the singular values of $X$.
However, some papers uses an alternative definition:
$$\|X\|_{tr}=\min_{X=AB'}\|A\|_{Fro}\|B\|_{Fro}$$
where $X=AB'$ is some arbitrary decomposition of $X$ and $\|*\|_{Fro}$ is the Frobenius norm.
But why are these two definitions equivalent?
As far as I understand, if $X=USV'$ is the SVD of $X$ and if $A=UC, B=VSC^{-1}$ where $C$ is diagonal s.t. $X=AB$, then $C=S^{1/2}$ leads to the minimum of $\|A\|_{Fro}\|B\|_{Fro}$.
However, I don't see how to prove the equivalency when $A,B$ are some arbitrary decomposition (not constraint to the form $A=UC,B=VSC^{-1}$)
Can anyone help me ?
Answer: Note that, since $X^*X$ is positive-semidefinite (and square), the square roots of its eigenvalues are the eigenvalues of the square root. Thus
$$
\|X\|_{\rm tr}=\sum_j\lambda_j(X^*X)^{1/2}=\sum_j\lambda_j((X^*X)^{1/2})=\sum_j\lambda_j(|X|)=\text{Tr}\,(|X|)
$$
(hence the name of the norm). Now, if $X=AB$, then with $AB=W|AB|$ the polar decomposition,
\begin{align}
\|AB\|_{\rm tr}&=\text{Tr}\,(|AB|)=\text{Tr}\,((W^*A)B)\leq\text{Tr}(A^*WW^*A)^{1/2}\,\text{Tr}\,(B^*B)^{1/2}\\ \ \\
&=\text{Tr}(A^*A)^{1/2}\,\text{Tr}\,(B^*B)^{1/2}=\|A\|_2\,\|B\|_2
\end{align}
(where the inequality is due to Cauchy-Schwarz).
Comment: Thanks for the reply! It seems the first equation relies on $X$ is a square matrix? What if $X$ is non-square, in general?
Comment: No, whether $X $ is square or not, $ X^*X$ will always be square.
Comment: Huh, I see. Thank you very much!

equivalent expressions for trace norm


Why is this not working? New to Java. (Bukkit)

Question: Making an alert system in Java for a 1.8 Bukkit test server.
Extremely new to java.
            if (args.length == 0) {
                sender.sendMessage("§cUsage: /alert <message>");
                return true; // << "return true" means that it will stop running the code
            }

            if (args[0].equalsIgnoreCase(args[0])) {
                String message = args[0];
                for (int i = 1; i < args.length; i++) {
                    message += args[i] + " ";
                }
                Bukkit.broadcastMessage(ChatColor.translateAlternateColorCodes('&', message + " "));

I don't know what's wrong with this.
Whenever I do /alert in game it sends the first word fine but then no space then it goes straight to the next word. Then the rest of the message is fine.
Answer: You currently initialize message with a value, but in your loop you don't prepend the space. Change
message += args[i] + " ";

to
message += " " + args[i];

Also, args[0] will always equals args[0] (and in the original case, so ignoring case for comparing it to itself is odd). Finally, and alternatively, if using Java 8+ you can use a collector like
String message = Arrays.stream(args).collect(Collectors.joining(" "));
Bukkit.broadcastMessage(ChatColor.translateAlternateColorCodes('&', message));

Comment: it just says "the value of the variable local message is not used"
Comment: Did you remove `Bukkit.broadcastMessage(ChatColor.translateAlternateColorCodes('&', message));`? If so, why?


Practice to minimize finger pressure on guitar strings?

Question: For years I did not notice now I need to correct my bad habit of putting a lot pressure on guitar strings when I play. How to get rid of this bad habit.
I know I put my finger on a guitar string and after buzz, the minimum pressure will give me enough to get a clear sound but how to train myself to do so?
Comment: I do chromatic scale without pressing, I do chromatic scale just buzzing but even though I have been doing this for the last 1 week still when I go back to play some thing the old habit of pressing hard is back
Answer: Try to think about your whole hand at once. The goal here is to relax as much as possible. Obviously, you have to use some muscles to actually push the string, but to use them no more than necessary—AND also not to use any muscles that aren't necessary.
As a violin teacher, when I see tension in a student's left hand, the thumb is often part of it; they're clenching their entire hand into a fist. I ask them to tap their thumb against the neck a few times. This does a few things: it focuses their attention on a part of their hand they weren't thinking about, but also, to "tap," you have to first release the pressure to move the finger away; after the tap they can replace the finger with a relaxed contact.
Try reminding yourself to do this; at random times while playing, tap your thumb; if any fingers are on strings but are not the "front fingers" (e.g. if one finger is on fret 5 but another is resting behind it on fret 4, not changing the pitch) then tap those. Try to consciously relax your entire hand (and arm!).
See also this answer, and its question about where to press the string. The answer makes the good point that you only need to press until the string meets the fret, not the wood.
Comment: My guitar teacher told me at some point to put less pressure on the back of the neck with my thumb. This answer reminded me of that and it helped me a lot. When it comes down to it, pressure of your thumb also means more pressure from the fingers. When I learned to put just the right amount of pressure on my thumb (and position it optimally), the pressure from my fingers naturally decreased. Another thing is trying to not cramp your wrist. Thumbs up! (no phun intended)
Answer: This might sound crazy, but try playing for a few weeks pressing the strings down on the fretboard so they hardly sound at all.
Then, gradually increase that pressure until most notes are clear. Continue in this silly way until all the notes are actually clear. All this will depend on how well or badly your guitar is set up, of course: with a well set up guitar you'll hardly going to need to press hard any way.
Also be aware that your thumb does not need to make a vice-like grip on the back of the neck - use the right arm to pull the guitar body in, particularly on barre chords, so your fretting hand hardly needs any pressure from the thumb to hold the strings from rattling. You may also consider the angle your guitar is held at, both from a vertical and a horizontal aspect.
Comment: Good point! After years of bad habits, releasing an unnecesary amount of pressure applied with thumb (also proposed by Hans) was, in my case, one of the biggest problems, and also the guitar angle and wrist position. On the other hand, some similar propioception exercises as the one you mention (pressing down strings) helped me slow but massively to get relaxed and apply not much more of the required pressure (even nowadays, as I keep practicing them as much as I can). I must reinforce that a good teacher -as it was for my case- is a key factor on this matter...
Answer: In addition to Andy and Tim's excellent answers, one of the things which helped me was playing without using my thumb at all - this shows you just how little pressure you need on the frets.
Comment: @Iplayguitar - at least eight - on the ends of those fingers...
Comment: Great is there any more tips?
Answer: I suppose you are OK with your right hand and can hit the strings in time?
So let the right hand do its thing and play a nice rhythm.
Now pay attention to the left hand. Pay attention to the exact feel in the fingers, in the fingertips, how does it sound?

If you press harder how does it sound?
If softer?
Does each string sound as good?

If you pay close attention to both how the fingertips on the left hand feel as you play, and to how it sounds, you will be able to teach your brain and hands to apply the right pressure.
If it sounds right it is right.
This will probably take about 1-3 weeks if you do a few hours each week.


Google app engine shading consistency

Question: I'm having problems with Google App Engine attempts to ensure strong constancy when aggregating shard-ed entities. I originally was under the assumption that batch "gets" by keys ensured that read's were up to date, but have since learned that that is incorrect. Now I'm confused as to how I can achieve strong constancy.
Here's what I'm trying to do:
I'm building an "game" in which users vote in polls. The voting takes place rapidly (about once per user minute), so I believe a shading technique is necessary to ensure users are not bottle-necked when writing their votes. When a user places a vote, a poll entity is read to make sure that the poll is still running, then assuming the writing of the vote to the shard is successful, the user should receive notification that their vote was counted (on the returned page). After a poll ends, the shards are assembled and computations are done based on the votes of each user; the results are then displayed.
It is important that a user's vote, which received confirmation, is actually seen during the shard aggregation. It's ok if the server rejects a user's vote (especially if its near the end of polling), but they should receive a failure notification. I'm worried that with GAE's eventual consistency assurances that cases may arise (when using only basic shading) in which a user successfully writes their vote to a shard (and get success notification), but the accumulator does not yet see the update and does not count that user vote in the final tally.
It it possible to prevent this? Do I even need to worry about this in practice?
Thanks!
Comment: You said  **I originally was under the assumption that batch "gets" by keys ensured that read's were up to date, but have since learned that that is incorrect.**   what leads you to believe this.  gets by Key are consistent, queries without using an ancestor won't be consistent.
Comment: Can you show the code for how you are issuing your get operations?  Maybe they aren't actually get by key operations (which should be consistent).
Answer: If you're trying to achieve real-time results, then use memcache.  The incr() method should be of particular interest to your voting game.  You can write the results to the datastore on a much less frequent basis (say, every 100 votes and again at the end of voting) so your performance will be much better.  You may even be able to avoid sharding this way.

Google app engine shading consistency


Is letsencrypt registration email address stored in certificate

Question: I'm using letsencrypt in two ways: a docker stack using certbot, and another using traefik (which performs certificate management automatically, using lego). In both cases there's a setting for the registration email address.
Is that email address stored in the certificate, or elsewhere? And can anyone (other than letsencypt) discover that email address?
Answer: From the letsencrypt forum:

The only ones who know that (very powerful) email address are you, and Let's Encrypt
It's used by Let's Encrypt to inform you of a pending expiration
It won't be encoded into the certificate
It won't be published anywhere, and so won't be contained in any "transparency logs" service

And:

The correlation of email to account/domains only exists in two places:

LetsEncrypt's database, which associates an Email Address to an Account ID. LetsEncrypt staff can pull this info themselves. You can pull this info yourself by presenting LetsEncrypt with the current AccountKey, which happens under secure https connections. No third party can access this information from LetsEncrypt. Intercepting this information during an API call would require a MITM or other attack.
If LetsEncrypt sends an expiry email, that email may be relayed to you by trusted third parties. It is possible, but incredibly unlikely, that (i) LetsEncrypt's email service providers or (ii) your email service provider, are mining this information from the emails they relay and store.
Answer: The beginning of the certificate process starts with creating a CSR or Certificate Signing Request.  This request allows the optional entry of an email address which would then be embedded into the certificate once it was provided.
The documentation and video at the link below shows how they are created.
https://www.ssl.com/how-to/manually-generate-a-certificate-signing-request-csr-using-openssl/
Comment: Thanks for your help. But apparently the email address is not encoded into the certificate. See the other answer for details.


Strange Server Problems

Question: I'm having some server issues, hopefully someone might have some insight.
There is one server in my workplace which is set up strangely and I don't understand it. Every machine can ping/access/share with every other machine on the network except this one server. It is inaccessible except for an online website it hosts which can be found at (say):
help.company.com/webapp
However pinging help.company.com gets no response although the IP is resolved to (say) 192.168.9.108.
To access this server you have to Remote Desktop onto a different server (say: helpdepartment) and from that machine you can Remote Desktop to help.company.com (or just "help" which seems to be sufficient). So basically you have to go through 2 layers of Remote Desktop. From this machine other machines on the network are unreachable and it appears that it is the only computer on the network. I don't have much experience with networks so this confuses me (are they bridged or something?).
EDIT: Just realised that the weird server hosts a public website. Must be why it's segregated from the rest of the network. Security?
Anyway my problem is that I need to write an application which can connect to a MS SQL Server which is running on the weird "help" machine (the same database the internal website accesses - it's a bug tracking application). However any attempts at connecting are "refused". But it's refused not "host unknown" so there is definitely some interaction. Clearly this problem is linked with the strange setup described above (for all I know it's standard practice - I'm ignorant).
The firewall on the "help" machine is disabled. The ports are all configured to 1433 (default) on the MS SQL Server. I've been through about a million articles with various different fixes but nothing helped. The servers run Windows Server 2003 by the way.
So, anyone have any insight?
Thanks in advance; this has been driving me crazy.
Carl
Comment: Ask the person that set it up this way, this is not really a standardish setup.
Comment: So it's not normal then. Yeah I should probably contact somebody internal if that's the case.
Comment: Sounds like your network has been carved up into vlans (or possibly just subnets) with extremely limited connectivity between them -- You'll need to find the person responsible for your network architecture/firewalls and ask them to give you a diagram...
Comment: It sounds to me like the SCW has been run on that server. http://technet.microsoft.com/en-us/library/cc757109(v=ws.10).aspx
Answer: The server is a "DMZ" because it faces out onto the Internet. That's why I couldn't connect to it easily.
The solution was to use a different machine with a copy of the database I needed.

Strange Server Problems


Is there a name for the technique of alternating left and right hand in piano?

Question: There is a technique in piano in which one plays a melody by alternating the left and right hand.
A well known example is in Bach's toccata and fugue in D minor, in the toccata part (right after the intro).
Is there a name for this technique?

(copied from IMSLP)
Comment: If the point is not so much the actual fingering that you use, but an alternation between low notes and high notes, there are some relevant terms. For the toccata, the low notes repeat the same pitch (for several measures); that's a form of [pedal point](https://music.stackexchange.com/a/130485/78419). There's also "[hocket](https://music.stackexchange.com/a/105187/78419)," in which multiple voices take turns with their notes to create an "interlocking" texture.
Comment: There doesn't seem to be a word for it! Michael Curtis's answer to an earlier question [here](https://music.stackexchange.com/questions/93222/what-is-the-term-for-a-rapid-succession-of-chords-in-alternating-hands) may be useful. The problem (*my* problem, at any rate) with using the word _hocket_ to describe those bars is that it would be anachronistic.
Comment: Is there any reason not to use the term "alternating hands"?
Answer: This passage is suspected to be a transcription from a stringed instrument, where you would execute this on alternating strings. That technique is called bariolage. No equally standard term exists for keyboard music.
Comment: The fugue subject is also an example of bariolage, but it does not call for alternating hands on the keyboard.


Como otimizar a consulta em um banco remoto utilizando ClientDataset?

Question: Estou utilizando um ClientDataSet para consulta em um banco remoto, a tabela possui mais de 100 mil registos e esta ficando muito lento, a SQL que estou utilizando é:
select (coalesce (cast(A.con_baixa as date),current_date) - cast(A.con_vencto as date) )Diastraso,
        A.CLI_CODIGO, A.CON_DOCUMENTO, A.CON_PARCELA, A.CON_DOC_ORIGEM,cast(A.CON_LANCTO as date) CON_LANCTO,
        cast(A.CON_VENCTO as date)CON_VENCTO, A.CON_VALOR, A.PLA_CODIGO, A.CON_TIPO, A.CON_PREVISAO,  A.CON_ESPECIE,
        cast(A.CON_BAIXA as date)CON_BAIXA,A.CON_PAGO, (A.CON_JUROS + A.CON_MULTA) CON_JUROSMULTA, A.CON_SITUACAO,
        A.BANCO, A.AGENCIA, A.CONTA, A.CONTA_R, A.CHEQUE, A.EMITENTE, A.CIDADE, A.CLI_REPASSE_CODIGO, A.CLI_REPASSE_NOME,
        A.CON_VL_BRUTO, A.CON_ID, A.BOL_CONTA, A.BOL_NOSSONUMERO, A.BOL_LINHADIGITAVEL, A.BOL_LOTE, A.NOSSONUMERO,
        A.CON_HISTORICO, A.IDORDEMSERVICO, A.ECF_NUMERO,  A.STATUSCONTA, A.CON_FPAGTOID, A.CONTRATOLOC, A.CON_DESCONTO,
        A.CON_OUTROS, A.IDLOTE, A.IDPAI, A.MOVORIGEM, A.IDCARTAO, A.CON_DATA, A.IDBORDERO, A.CCUSTOID, A.IDDESCONTO,
        A.IDLOJA,B.cli_nome, C.PLA_DESCRICAO, D.DESCRICAO CENTROCUSTO, E.FDESCRICAO, NomeFantasia
from contas A
left join CLIENTES B on (B.cli_codigo = A.cli_codigo)
left join PLANOCONTAS C on A.PLA_CODIGO = C.PLA_CODIGO
left join CCUSTO D on A.CCUSTOID = D.CCUSTOID
left join FPAGTO E on A.CON_FPAGTOID = E.FPAGTOID
inner join Loja F on F.IdLoja = A.IdLoja
where  (A.CON_LANCTO >='03/05/2013 00:00:00' and A.CON_LANCTO <= '03/24/2015 23:59:59' )
        and A.CON_TIPO = 'R' and A.IdLoja in (1) and A.STATUSCONTA <> 'AGRUPAMENTO'
        order by A.IdLoja, A.CON_LANCTO

Comment: Vamos lá ? qual SGBD ? As tabelas tem índice ? As estatísticas estão atualizadas ? Foi feito o Plano de Execução da query ?
Comment: Qual a necessidade de carregar todos os registros?
Comment: estou utilizando o firebird, não tem indice, estatisticas não estão atualizadas e não foi feito nenhum plano, sou novato e não sei como funciona...
Comment: http://www.firebirdsql.org/manual/isql-set.html#isql-set-plan
Comment: Relacionada: http://pt.stackoverflow.com/questions/54034/como-melhorar-o-desempenho-da-consulta-do-clientdataset-com-muitos-registros
Answer: Você conseguiu identificar se o que está lento é a execução da query ou o carregamento no dataset ? quantos registros estão retornando na consulta, pelo que entendi, provavelmente o problema é que muitos registros estão sendo retornados, e como sabemos o clientdataset coloca todos em memória.
Criar índices para alguns campos pode ajudar muito, porém não adianta sair criando um monte de índices, isso pode prejudicar a performance. É necessário analisar o plano de consulta para avaliar os melhores índices para serem criados.
Answer: Utilize a propriedade PacketRecords para carregar seu TClientDataSet incrementalmente.
Por experiência, 500 é um bom valor. -1 serve para trazer um único bloco (seu caso) e 0 traz apenas metadados sobre a consulta em questão.
Evite usar valores mais altos que 5000. Fazendo isto, você estará transferindo o gargalo de memória para a consulta em banco.
Answer: Se não tem nenhum índice está fazendo full scan table e esse é o motivo da demora. Crie índices para cada campo no Join e no Where e rode a query novamente
Comment: Os campos que estão nos joins são chave primaria, já possuem um index criado para cada um deles e mesmo assim fica lento
Comment: E os campos do Where ?  Esses também precisam de índices
Comment: Não tinha vistos o Order by, esses se não tem, tem que ter índices também
Comment: eu criei para todos do where e order by, mesmo assim continua a mesma coisa ;s
Comment: Bom agora que seu banco está cheio de índices e a culpa não pode ser imposta a ele, o problema é a quantidade de registros trazidas de uma só vez para o client, faz a sugestão do Cigano Morrison mendez que postou uma resposta limitando a quantidade de registros trazidos por vez


R - Download de dados do portal Hidroweb

Question: A Agência Nacional de Águas disponibiliza em seu portal Hidroweb o download de séries históricas referentes aos dados obtidos por diversas estações de monitoramento.
Gostaria automatizar o download dessas séries históricas, contudo a ANA alterou seu site e estou com dificuldades para realizar a tarefa. Nessa pergunta está o código em R que eu me baseei para baixar os arquivos.
A princípio eu havia substituído os links de http://hidroweb.ana.gov.br/ para  http://www.snirh.gov.br/hidroweb/, contudo, o download não está conseguindo ser realizado corretamente.
Imagino que o problema esteja na linha r <- POST(url = paste0(baseurl[1], est, baseurl[2]), body = list(cboTipoReg = "10"), encode = "form"), visto que o comando de POST parece não estar conseguindo acessar devidamente o link http://www.snirh.gov.br/hidroweb/Estacao.asp?Codigo=2851050&CriaArq=true&TipoArq=1.
Quando acessado com o query &CriarArq=true&TipoArq=1 o site mostra apenas a mensagem "Solicitação não permitida", não sendo possível realizar o scraping da página para encontrar o arquivo .zip.
Eu havia imaginado que apenas o comando POST com a URL anterior e o form cboTipoReg = '10' bastaria, mas parece não estar acessando a página da maneira esperada. Alguém saberia me explicar o que há de errado?
Comment: Perguntas que dependem de recursos externos fogem do objetivo do site, mas você pode [edit] a questão e reduzir o problema a um [mcve] (simulando os dados localmente, por exemplo, de forma que a pergunta fique autocontida sem depender de recursos que podem mudar a qualquer momento, e de forma a não se confundir com suporte de APIs de terceiros).  Ao editar, a pergunta entra numa reavaliação da comunidade via voto.
Answer: Oi
Podemos fazer de forma semelhante atualizando a url base para: "http://www.snirh.gov.br/hidroweb/rest/api/documento/convencionais?tipo=1&documentos="
Sugestão em R:
## Download automático dados de estações convencionais SNIRH -----

library(httr)

baseurl = "http://www.snirh.gov.br/hidroweb/rest/api/documento/convencionais?tipo=&documentos="
ListaEstacaoes = c(40025000,40050000,40070000,40100000)

setwd("D:/PastaParaDownload")
destino = getwd()

# tipo=1 arquivo access *.mdb
# tipo=2 arquivo texto  *.txt
# tipo=3 arquivo excel  *.csv
tipo = 1

#substituindo o tipo
baseurl = gsub("tipo=",paste0("tipo=",tipo),baseurl)

for(i in 1:length(ListaEstacaoes)){
  baseurl_est = paste0(baseurl,ListaEstacaoes[i])

  #Conexao
  r = POST(url = baseurl_est, body = list(cboTipoReg = "10"), encode = "form")
  if (r$status_code == 405) {
    cont = content(r, as = "text", encoding="ISO-8859-1")
    download.file(baseurl_est, paste0(ListaEstacaoes[i], ".zip"), mode = "wb")
  }
}

Adicionalmente, para estações telemétricas:
## Download automático dados de estações telemetricas SNIRH -----

library(httr)

# URL Base 1 - gera o arquivo
baseurl1 = "http://www.snirh.gov.br/hidroweb/rest/api/documento/gerarTelemetricas?codigosEstacoes=&tipoArquivo=&periodoInicial=&periodoFinal="
# URL Base 2 - baixa o arquivo
baseurl2 = "http://www.snirh.gov.br/hidroweb/rest/api/documento/baixarTelemetricas?codigosEstacoes="

# Informação necessaria das estacoes telemetricas: ID Codigo
# para obter o ID consultas ficha descritiva da estacao em http://gestorpcd.ana.gov.br/
idEstacaoes     = c(84863570,91264360,84763570,84663550,94765311)
CodigoEstacaoes = c(15400000,15341000,15360000,15380000,15326010)

#setwd("D:/PastaParaDownload")
destino = getwd()

#Parametros necessarios e comuns as estacoes
    # Estacoes telemetricas disponiveis apenas em dois formatos
    #tipo = 2 # tipo=2 arquivo texto  *.txt
    tipo = 3 # tipo=3 arquivo excel  *.csv

    # Data de inicio e fim do recorte da serie no formato aaaa-mm-dd
    dinicio = "2019-01-01"
    dfim = "2019-12-31"

    # Hora de inicio e fim do recorte da serie no formato hh:mm
    hinicio = "00:00"
    hfim = "23:00"

#substituindo os parametros comuns as estacoes
baseurl1 = gsub("&tipoArquivo=",paste0("&tipoArquivo=",tipo),baseurl1)
baseurl1 = gsub("&periodoInicial=",paste0("&periodoInicial=",dinicio,"T03:",hinicio,".000Z"),baseurl1)
baseurl1 = gsub("&periodoFinal=",paste0("&periodoFinal=",dfim,"T03:",hfim,".000Z"),baseurl1)

for(i in 1:length(CodigoEstacaoes)){
  baseurl1i = gsub("codigosEstacoes=",paste0("codigosEstacoes=",idEstacaoes[i]),baseurl1)
  baseurl2i = gsub("codigosEstacoes=",paste0("codigosEstacoes=",idEstacaoes[i]),baseurl2)

  #Conexao
  BROWSE(baseurl1i)
  BROWSE(baseurl2i)
}

Comment: Interessante resposta! Sempre usei esse script. Mas noto que em alguns casos, para estações pluviométricas, muitos dados vem vazios, o que não ocorre no portal HidroWEB mesmo


Passing some rows based on a threshold

Question: I am wondering how can I adapt the code (or add extra calls) below to pass some rows based on a threshold (here, a percentage value). My xyz file:
21.4    7
21.5    141
21.6    4
21.7    43
21.8    26
21.9    133
22      305
22.1    216
22.2    93
22.3    33
22.4    13
22.7    23
22.8    2
22.9    10
23      39
23.1    22
23.2    33
23.3    8
23.4    9
23.5    2
23.6    270
23.7    724
23.8    2349
23.9    2
24      1
24.1    11
24.2    376
24.3    1452
24.4    92

with the following awk call I obtain the corresponding percentage of the value in the 3rd column:
awk 'FNR==NR { s+=$2; next; } { printf "%s\t%s\t%s%%\n", $1, $2, 100*$2/s }' xyz xyz | sort -k3 -g

which gives:
24      1       0.0155304%
22.8    2       0.0310607%
23.5    2       0.0310607%
23.9    2       0.0310607%
21.6    4       0.0621214%
21.4    7       0.108713%
23.3    8       0.124243%
23.4    9       0.139773%
22.9    10      0.155304%
24.1    11      0.170834%
22.4    13      0.201895%
23.1    22      0.341668%
22.7    23      0.357198%
21.8    26      0.403789%
22.3    33      0.512502%
23.2    33      0.512502%
23      39      0.605684%
21.7    43      0.667806%
24.4    92      1.42879%
22.2    93      1.44432%
21.9    133     2.06554%
21.5    141     2.18978%
22.1    216     3.35456%
23.6    270     4.1932%
22      305     4.73676%
24.2    376     5.83942%
23.7    724     11.244%
24.3    1452    22.5501%
23.8    2349    36.4808%

So, I want to automagically filter the last Nth rows if the sum of the last values in the 3rd column is just greater than 60%, in the case above it will be 36.4808% + 22.5501% + 11.244% = 70.2749%:
23.7    724     11.244%
24.3    1452    22.5501%
23.8    2349    36.4808%

Any hints are appreciated,
Comment: You might want to change `100*$2/s` to `(s ? 100*$2/s : 0)` or similar to guard against the time when you run it and all `$2` values are `0` or the file is empty.
Comment: I mean the sum of the last values in the 3rd column, thanks for pointing that out.
Answer: Assumptions:

for rows with duplicate values in the 2nd column, precendence goes to rows with a higher row number (via awk/FNR)

One GNU awk idea:
awk '
BEGIN {OFS="\t" }

      { a[$2][FNR]=$1                                # [FNR] needed to distinguish between rows with duplicate values in 2nd column
        s+=$2
      }

END   { PROCINFO["sorted_in"]="@ind_num_desc"        # sort array by numeric index (descending order)
        for (i in a) {                               # loop through array (sorted by $2 descending)
            for (j in a[i]) {                        # loop through potential duplicate $2 values (sorted by FNR descending)
                pct=100*i/s
                out[--c]=a[i][j] OFS i OFS pct "%"   # build output line, store in out[] array, index= -1, -2, ...
                sum+=pct
                if (sum > 60) break                  # if the sum of percentages > 60 then break from loop
            }
            if (sum > 60) break                      # if the sum of percentages > 60 then break from loop
        }
        for (i=c;i<0;i++)                            # print contents of out[] array starting with lowest index and increasing to -1
            print out[i]
      }
' xyz

NOTE: requires GNU awk for:

multidimensional arrays (aka array of arrays)
PROCINFO["sorted_in"] support

This generates:
23.7    724     11.244%
24.3    1452    22.5501%
23.8    2349    36.4808%

Answer: That could be done in a single awk command, but I think this version is shorter:
awk -v OFS="\t" 'FNR==NR { s+=$2; next; } { $3=100*$2/s "%" }1' xyz xyz |
sort -k3 -g |
awk '(t+=$3)>40'

prints out:
23.7    724     11.244%
24.3    1452    22.5501%
23.8    2349    36.4808%

Passing some rows based on a threshold


Como ordenar campo 1/5 de parcelas no datatables

Question: $.ajax
        ({
            type: 'POST',
            dataType: 'json',
            url: BASE_URL + '/conta_pagar/pesquisa_contas',
            data: {palavra: palavra},
            success: function (json)
            {
                var colunaNome = [{ "data": "nome_desc" },
                { "data": "id" },
                { "data": "nome_desc_despesa" },
                { "data": "nr_parcelas","targets": 0, "orderable": false},
                { "data": "dt_parcelas"},
                { "data": "posicao" },
                { "data": "valor_parcelas", render: $.fn.dataTable.render.number('.', ',', 2, 'R$ ') },
                { "data": "dt_pago"}];

                var colunaAcoes = [
                    {
                                                    "aTargets": [8], //  é o nº da colunaacoes
                        "mRender": function (data, type, dadosjson) { //aqui é uma funçãozinha para pegar os ids
                            return '\n\
                            \n\
                            \n\
                            \n\
                            ';
                        }
                    }
                ];
                $.fn.dataTable.moment('DD/MM/YYYY');
                datatable('#table_cont_pag', json, colunaNome, colunaAcoes);

            }
        });
$(tabela).dataTable(
    {

        "aaData": json,//retorno ajax com informaçoes do banco
        "aoColumns": colunaNome,// colunas
        "aoColumnDefs": colunaAcoes,// coluna açoes
        "language": {
            "scrollX": true,
            "sEmptyTable": "Nenhum registro encontrado",
            "sInfo": "Mostrando de _START_ até _END_ de _TOTAL_ registros",
            "sInfoEmpty": "Mostrando 0 até 0 de 0 registros",
            "sInfoFiltered": "(Filtrados de _MAX_ registros)",
            "sInfoPostFix": "",
            "sInfoThousands": ".",
            "sLengthMenu": "_MENU_  Resultados por Página",
            "sLoadingRecords": "Carregando...",
            "sProcessing": "Processando...",
            "sZeroRecords": "Nenhum registro encontrado",
            "sSearch": "Pesquisar",
            "oPaginate": {
                "sNext": "Próximo",
                "sPrevious": "Anterior",
                "sFirst": "Primeiro",
                "sLast": "Último"
            },
            "oAria": {
                "sSortAscending": ": Ordenar colunas de forma ascendente",
                "sSortDescending": ": Ordenar colunas de forma descendente"
            }
        }
    }
);

Answer: pra quem precisar
jQuery.fn.dataTableExt.oSort['num-html-asc'] = function(a,b) {
var x = a.replace( /[/]/g, "" );
var y = b.replace( /[/]/g, "" );
x = parseFloat( x );
y = parseFloat( y );
return ((x < y || isNaN(y) ) ? -1 : ((x > y || isNaN(x)) ? 1 : 0));
};

jQuery.fn.dataTableExt.oSort['num-html-desc'] = function(a,b) {
var x = a.replace( /[/]/g, "" );
var y = b.replace( /[/]/g, "" );
x = parseFloat( x );
y = parseFloat( y );
return ((x < y || isNaN(x)) ? 1 : ((x > y || isNaN(y) ) ? -1 : 0));
};

exemplo uso
{ "data": "nr_parcelas","type":"num-html"},


Date conversion from string not happening

Question: I have a date within order_date in this format May 31, 2018 at 3:35 PM
I'm converting it to Date like so…
if let order_date = value["order_date"] as? String {

let dateFormatter = DateFormatter()

dateFormatter.dateFormat = "yyyy-MM-dd HH:mm:ss"

let date = dateFormatter.date(from: order_date)

}

But I'm getting date as nil. What could be the issue..?
Comment: [DateHelper](https://github.com/melvitax/DateHelper) will help you
Comment: Is the date is at the format you give? I don't think so
Comment: change your dateFormatter like this "MMMM dd, yyyy 'at' hh:mm a"
Comment: @FredericP I'm sending in this format..`yyyy-MM-dd HH:mm:ss`. But in the server, some dates are in this format `May 31, 2018 at 3:35 PM`. Hence the issue...
Comment: @bwv - what the input you get and what the OP you expect
Comment: @Anbu.karthik Even if the format is `May 31, 2018 at 3:35 PM` I want it in this format...`yyyy-MM-dd HH:mm:ss.`
Comment: @bwv - check the updated answer
Comment: Look at your date string. Now look at your date format. They are nothing like each other.
Answer: your dateformat is wrong change your format to
dateFormatter.dateFormat = "MMMM dd, yyyy 'at' hh:mm a"

for e.g if you get the month as Mar then use - MMM, if you get March then use - MMMM
you can check the dateformat's in here
for full answer
 if let order_date = value["order_date"] as? String {

let dateFormatter = DateFormatter()
 dateFormatter.dateFormat = "MMMM dd, yyyy 'at' hh:mm a"
  var date = dateFormatter.date(from: order_date)
    if date == nil{
        dateFormatter.dateFormat =  "yyyy-MM-dd HH:mm:ss"
        date = dateFormatter.date(from: order_date)
    }
 dateFormatter.dateFormat = "yyyy-MM-dd HH:mm:ss"
 let myStringafd = dateFormatter.string(from: date!)
 print(myStringafd)

}

for detail dateforamt conversion you get here
Comment: .@Anbu.karthik `"MMM dd, yyyy 'at' hh:mm a"` won't be the format always. It could be `am` or `pm`
Comment: But @Anbu.karthik isn't `"MMMM dd, yyyy 'at' hh:mm a"` always for A.M ? In my case it could be either AM or PM
Comment: @bwv - based on the time format it will change AM or PM , its the 12 hour dateformat
Comment: .@Anbu.karthik In the answer that you gave, should we not give an if condition to check for the date format..? because the date that I'll get in `order_date ` can be either in the formats `"MMMM dd, yyyy 'at' hh:mm a"` or `"yyyy-MM-dd HH:mm:ss"`
Comment: @bwv - check the updated answer
Comment: Ok @Anbu.karthik Thanks..
Answer: Replace your code with below :
if let order_date = value["order_date"] as? String
    {

        let dateFormatter = DateFormatter()

        dateFormatter.dateFormat = "MMM dd, yyyy 'at' hh:mm a"

        let date = dateFormatter.date(from: order_date)

    }


Fundamental time period of a function

Question: I have the following continuous time signal:
$$x(t) = \sum_{n=-\infty}^{\infty}e^{-(2t-n)}u(2t-n)$$ where $u(t)$ is the unit function. I had previously determined that this signal was not periodic. However, it seems that it is. However, I'm not sure how I'd determine the fundamental period of such a function. Here is my work so far.
$$u(2t-n) = 1 \text{ for all } n \leq 2\lfloor t \rfloor \text{ and } 0 \text{ for all } n \geq 2\lfloor t\rfloor + 1$$
$$e^{-2t}*\frac{e^{\lfloor 2t \rfloor}}{1-e^{-1}}$$
But how do I proceed from here?
Comment: Hint: what happens when you substitute $t-(1/2)$ for $t$ in that summation?
Comment: I'll get t-1 =n. However, how do I know I can make that substitution?
Comment: I don't see how that whole summation becomes $t-1=n$ when you replace $t$ in it everywhere by $t-(1/2)$. But, anyway, what could possibly stop you from making that substitution? The beauty of functions is that you can replace their variables by anything you like, so long as you are consistent about it.
Comment: Are you still here?
Comment: @GerryMyerson Very sorry for the delay; I was away on travel! I have figured out a solution and would be glad to post it if this could be reopened. Thank you.
Comment: You can now post your solution.
Comment: ((Comment deleted, which was addressing the "period" aspect but not the "fundamental period" question. Sorry for the noise.))
Comment: @DanielFischer I have added the solution. Please consider removing the downvote. Thank you.
Comment: Removing other people's downvotes is  not an ability moderators have.
Comment: @GerryMyerson please see new answer
Answer: Let's consider slightly expanding out the equation.
$$x(t) = \sum_{n=-\infty}^{\infty}e^{-(2t-n)}u(2t-n)$$
$$x(t) = ... + e^{2t-0}u(2t-0)+e^{2t-1}u(2t-1)+...$$
$$x(t) = x(t+T)$$
$$x(t+T) = ...+e^{2(t+T)-0}u(2(t+T)-0)+e^{2(t+T)-1}u(2(t+T)-1)+...$$
Here, I'm defining $T$ as the fundamental period. Now, the adjacent period would allow adjacent terms to be equal. By adjacent, I mean for $x(t) \text{ when n=0 } = x(t+T) \text{ when n = 1}$ .
So, let's set those terms equal to each other.
$$e^{2t-0}u(2t-0) = e^{2(t+T)-1}u(2(t+T)-1)$$
From this equation, we can set the following two terms equal to each other.
$$2t = 2t + 2T -1$$
$$1 = 2T$$
$$T = \frac{1}{2}$$
**EDIT: **To show that this holds for all terms, we have the following $$e^{2t-n}u(2t-n) = e^{2(t+T)-(n+1)}u(2(t+T)-(n+1))$$
$$2t - n= 2t + 2T -n-1$$
And from here, you can see how $T = \frac{1}{2}$.
Comment: That's the idea – you have shown that one of the terms in the expansion at $t$ goes over to one of the terms in the expansion at $t+(1/2)$. Now if you can show that works for all the terms in the expansion, and not just for the one with $n=0$....
Comment: @GerryMyerson I have added the general term

Fundamental time period of a function