Unique service names for gitlab auto devops

Question: While using gitlab auto devops I notice each project being created in its own namespace, defining the service name as production-auto-deploy.

$kubectl get services --all-namespaces

NAMESPACE                NAME                                     TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                      AGE
app-13094854             production-auto-deploy                   ClusterIP      10.245.23.224    <none>           5000/TCP                     11h
app-13094854             production-postgres                      ClusterIP      10.245.202.205   <none>           5432/TCP                     11h
config-server-13051179   production-auto-deploy                   ClusterIP      10.245.138.49    <none>           5000/TCP                     40m
default                  kubernetes                               ClusterIP      10.245.0.1       <none>           443/TCP                      11h
gitlab-managed-apps      ingress-nginx-ingress-controller         LoadBalancer   10.245.200.23    188.8.131.52   80:30888/TCP,443:30962/TCP   11h
gitlab-managed-apps      ingress-nginx-ingress-controller-stats   ClusterIP      10.245.104.211   <none>           18080/TCP                    11h
gitlab-managed-apps      ingress-nginx-ingress-default-backend    ClusterIP      10.245.202.171   <none>           80/TCP                       11h
gitlab-managed-apps      tiller-deploy                            ClusterIP      10.245.31.107    <none>           44134/TCP                    11h
kube-system              kube-dns                                 ClusterIP      10.245.0.10      <none>           53/UDP,53/TCP,9153/TCP       11h
some-microservice-13093883          production-auto-deploy                   ClusterIP      10.245.97.62     <none>           5000/TCP                     11h
some-microservice-13093883          production-postgres                      ClusterIP      10.245.245.253   <none>           5432/TCP                     11h

Can this service name be customized? For example I want it to include the project name thus mapping production-auto-deploy -> app-production-auto-deploy and some-microservice-production-auto-deploy.
The reason I want these service names to be unique is because I am evaluating spring-cloud-kubernetes and I need unique service names for ribbon discovery using feign clients.
Additionally I am wondering why each project is given its own namespace, is this some kind of best-practice? Can auto devops be configured to deploy all projects in the same namespace?
Answer:
Can this service name be customized?

Yes, it can be, by using custom helm chart.
In simplification, service name is generated from two variables (Release name + Chart name)
printf "%s-%s" .Release.Name $name | trimSuffix "-app" ...

By default Auto DevOps uses its own helm chart, source code available here.
And by changing the 'name' inside Chart.yaml file (which contains chart's metadata), you can influence the final service name.
There is also another way to customize service name: by using overrides values to 'helm upgrade' command inside 'Deploy.gitlab-ci.yml' template with '--set nameOverride=<CUSTOM_SVC_NAME>'

Additionally I am wondering why each project is given its own
  namespace, is this some kind of best-practice? Can auto devops be
  configured to deploy all projects in the same namespace?

By default Auto Deploy uses this technique and naming convention for K8S namespaces during apps deployment (as described here), and there is no way to change it according official documentation.
You can try at your own risk to override it with use of custom project variable: KUBE_NAMESPACE
Answer: Been looking everywhere, ended up going really deep into documentation. All you need is to override Autodevops template. Here's how I've changed namespace name per project also as project name itself, so it would use a project name and not project slug. Open your .gitlab-ci.yml and add this:
include:
  - template: Auto-DevOps.gitlab-ci.yml

production:
  environment:
    url: http://$CI_PROJECT_NAME.$KUBE_INGRESS_BASE_DOMAIN
    kubernetes:
        namespace: $CI_PROJECT_NAME

If you just want to use environments as namespaces, you can do this in gitlab "Kubernetes" tab configuration. There's a checkbox in there for such option.


How can I create an ecs service for my deploy

Question: I'm learning CI/CD pipelines and I've come up with a question.
I have a CloudFormation file that creates some static resources for my app (ECR repository, ECS Cluster and some roles)
When creating my deployment pipeline in GitHub Actions I've set up my task-definition.json to run my ECS Task.
But when do I create my ECS Service?
Should it be already created or I need to create in the deployment stage?
If it needs to be already created in my CloudFormation template, how will I put a task definition resource pointing to an empty ECR?
Answer: Exactly, your guess is right. You need to create the service in advance with a task definition pointing to your empty ECR, clearly this won't go up, but once you run your pipeline, a new task definition will be created pointing to your newly created image and update the service.


How do I check if variables of two different turtle breeds match?

Question: I am trying to move a turtle towards it's offspring based on a shared variable. In my initial code I ask a male (breed) that has a display, but not a bower, to establish a bower (different breed) by hatching. I use create-link-with myself to so that the male and the bower share a variable called bird-id.
to establish-bower
  ask males[
   if has-bower? = false[
     if has-display? = true[
        hatch 1 [
        set breed bowers
        set color yellow
        set bower-id who
        set shape "house"
        create-link-with myself
        ]
           ]
      ]
    ]
end

In the second part of my code, I ask males that have a bower and a display to return-to-bower. If their distance to THEIR OWN bower is greater than 0, the male faces that bower and moves forward one. If the distance is 0, two other variables and the energy are adjusted and the male moves a bit randomly.
to return-to-bower
  ask males[
    if has-bower? = true[
      if has-display? = true[
        (ifelse
        distance one-of bowers with [bird-id = (bird-id of myself)] > 0[
          face one-of bowers with [bird-id = (bird-id of myself)]
          fd 1]
        distance one-of bowers with [bird-id = (bird-id of myself)] = 0[
          set has-display? false
          set nr-display nr-display + 1
          rt random 50
          lt random 50
          fd 1
          set energy energy - 3])
      ]
    ]
  ]
end

I have trouble coding the part that identifies a bower as belonging to the male that made it. So far I have
bowers with [bird-id = (bird-id of myself)]

This obviously doesn't work, but it conveys what I want to do.
How  can I write this part?
Comment: how exactly is this java related?
Answer: I think your bird-id is not being shared as expected between your males and bowers- in establish-bower you have the line set bower-id who, but the who there should be in the context of the bower instead of the context of the male. So, male 0 may ask bower 4 to set its bird id to 4, rather than 0. If you want to use that option (which is certainly doable), I think you'd need to alter to set bower-id [who] of myself- this mirrors what you did with the create-link-with myself line.
However, if you are already using links, you might like an alternative that makes use of the link already in place. The other-end the primitive returns the agent on the other end of the link owned by the asking turtle (better explained in that link). So whenever it makes sense to have the turtle figure out which bower is its own, have it query the associate link. For example:
breed [ males male ]
breed [ bowers bower ]

males-own [ has-bower? go-time? ]

to setup
  ca
  ask n-of 10 patches [
    sprout-males 1 [
      set go-time? true
      set has-bower? false
    ]
  ]
  ask males [
    establish-bower
  ]
  reset-ticks
end

to establish-bower
  if not has-bower? [
    hatch-bowers 1 [
      set shape "house"
      create-link-with myself
    ]
    set has-bower? true
  ]
end

to go
  ask males with [has-bower?] [
    ifelse go-time? [
      rt random 60 - 30
      fd 1
      if random-float 1 < 0.05 [
        set go-time? false
      ]
    ] [
      let my-bower [other-end] of one-of my-links
      ifelse distance my-bower > 1 [
        face my-bower
        fd 1
      ] [
        move-to my-bower
        if random-float 1 < 0.02 [
          set go-time? true
        ]
      ]
    ]
  ]
  tick
end

I use let in the above to have the bowers only be temporarily checked by the males- you could also simplify this by having a males-own variable like my-bower and assign the bower directly to that variable and then the males would permanently track it.


Is it still legal to do pointer arithmetic on a deleted array?

Question: Today I wrote something which looked like this:
void foo(std::vector<char>&v){
    v.push_back('a');
    char*front=&v.front();
    char*back=&v.back();
    size_t n1=back-front+1;
    v.push_back('b');//This could reallocate the vector elements
    size_t n2=back-front+1;//Is this line valid or Undefined Behavior ?
}

If a reallocation occures when I push 'b' back, may I still compute the difference of my two pointers ?
After reading the relevant passage of the standard a few times, I still cannot make my mind on this point.
C++11 5.7.6:
When two pointers to elements of the same array object are subtracted, the result is the difference of the
subscripts of the two array elements. The type of the result is an implementation-defined signed integral
type; this type shall be the same type that is defined as std::ptrdiff_t in the  header (18.2). As
with any other arithmetic overflow, if the result does not fit in the space provided, the behavior is undefined.
In other words, if the expressions P and Q point to, respectively, the i-th and j-th elements of an array object,
the expression (P)-(Q) has the value i − j provided the value fits in an object of type std::ptrdiff_t.
Moreover, if the expression P points either to an element of an array object or one past the last element of
an array object, and the expression Q points to the last element of the same array object, the expression
((Q)+1)-(P) has the same value as ((Q)-(P))+1 and as -((P)-((Q)+1)), and has the value zero if the
expression P points one past the last element of the array object, even though the expression (Q)+1 does not
point to an element of the array object. Unless both pointers point to elements of the same array object, or
one past the last element of the array object, the behavior is undefined.
Of course I know that it works, I just wonder if it is legal.
Comment: Doesn't a deleted object cease to be an object?
Comment: Didn't anybody notice that it should be `back-front`, not `front-back`?
Comment: Since `front` and `back` may be dangling after `push_back`, that implies there is no array, and if there's no array, the two can't point to it. This potentially invokes UB.
Comment: @jbruni That is an implementation detail. This question concerns the standard.
Comment: @Henrik I corrected the code
Comment: Even if the backing store of the vector is moved, the difference between the two address values won't change. In fact, an optimizing compiler could move the computation of n2 to before the push_back and not change the meaning of the program.
Answer: Pointers to deleted objects are toxic: don't touch then for anything other than giving them a new value. A memory tracking system may trap aby use of a reclaimed pointer value. I'm not aware if any such system in existence, however.
The relevant quote is 220.127.116.11 [basic.stc.dynamic.deallocation] paragraph 4:

If the argument given to a deallocation function in the standard library is a pointer that is not the null pointer value, the deallocation function shall deallocate the storage referenced by the pointer, rendering invalid all pointers to any part of the deallocated storage. The effect of using an invalid pointer value (including passing it to a deallocation function) is undefined.

When resizing a std::vector<...> it jumps through a number of hoops (allocators) and, by default, eventually calls a deallocation function.
Comment: @jbruni I don't see how cargo cult applies. Do you maybe mean something else?
Comment: @jbruni: I didn't have the time to add the relevant quote. However, my statement is solidly founded on the standard. The quote is added now.
Comment: cargo cult programming.
Comment: +1: But is subtracting two pointers *using* those pointers?  ;)
Comment: @Yakk: Since the quote doesn't use _pointer_ but _pointer value_ there is very little room for interpretation, I'd think.
Comment: A bigger issue than memory tracking systems is compilers that may draw some rather "interesting" inferences based upon Undefined Behaviors in cases which would otherwise have had only benign consequences. I wish the Standard would specify a normative dialect of C which would forbid feeding invalid pointers to arithmetic operations that return pointers, but would guarantee that pointers to distinct objects of overlapping lifetimes would compare distinct, all pointers could be ranked, and pointers to the same dead object could be subtracted from each other and yield the same value as when live.
Answer: Strictly speaking, it's UB. But you can always convert your char * pointers to uintptr_t (provided it is present) and then safely subtract the resulting integers.
void foo(std::vector<char>&v){
    v.push_back('a');
    auto front= uintptr_t (&v.front());
    auto back = uintptr_t (&v.back());
    size_t n1=back-front+1;
    v.push_back('b');//This could reallocate the vector elements
    size_t n2=back-front+1;
}

Comment: @chill: Cray vector systems (C90, T90, SV1). The machines were 64-bit word addressed, with no hardware support for 8-bit byte access, so a machine was a 64-bit pointer to a 64-bit word. But the systems ran Unix, so it was necessary to support 8-bit bytes (C permits `CHAR_BIT==64`, but POSIX doesn't, and interoperability would have been a nightmare). So the C compiler implemented byte pointers *in software* by storing a 3-bit byte offset in the otherwise unused 3 high-order bits of a pointer.
Comment: However, now there's no guarantee that the result is meaningful.
Comment: @R.MartinhoFernandes: I've actually worked on systems where converting pointers to integers yielded values that were not meaningfully related to the pointer values; for example, `(uintptr_t)p1 - (uintptr_t)p0` might yield a completely different and meaningless result than `(char*)p1 - (char*)p0`.
Comment: But, _why_?  This may be _defined_ behavior, but almost certainly not what someone would want.
Comment: @KeithThompson, what was the reason for it?
Answer: This particular case is safe but ugly and misleading.
Line v.push_back('b');//This could reallocate the vector elements can cause reallocation of your container. In this case next line will use a non existent front and back pointers. Computing difference of two addresses is safe even if are dangling pointers. What is not safe is dereferencing them.
The correct solution is to use vector::count() function the will be always in sync. If you (for some reason) don;t want to call vector::count() you should at leas use ++n1.
Comment: It's only safe until it stops being. Treating this as safe assumes compilers won't get smarter.
Comment: Pointer arithmetic remains pointer arithmetic. Playing with pointer values is safe. There is no (smart) scenario that can lead to dereference pointer (and crash) when doing pointer arithmetic. As i pointed, the correct solution is to use `vector::count()`
Comment: Who said anything about crashes? There are other undesired behaviours that could result from a smart compiler (say, entirely removing an operation involving such bad pointers because it obviously must be dead code: a well-behaved program that follows C++ rules would not execute it, therefore it must be dead). Undefined behaviour is not about crashes.


Find where the limit does not exist for the function

Question: Given the function: $f(x,y) = \frac{xy^4}{x^2+y^8}$, find a path where the limit does not exist at the origin.
I am having problems with this because of lot of paths go to $0$ but I know the limit does not exist.
I have tried things like $y^2$, $\sqrt(y)$ but I am getting nowhere.
Thank you!
Answer: Put $x=y^4$. Anywhere on this curve, our function is equal to $\frac{1}{2}$. So as we approach $(0,0)$ along the path $x=y^4$ (say for positive $y$), our function approaches $\frac{1}{2}$.
Put $x=-y^4$. As we approach $(0,0)$ along ths path, our function approaches $-\frac{1}{2}$.
If we want a single path such that as we approach $(0,0)$ along it, we combine the first path above with the path $y=0$. We describe the combined path geometrically.
For $1/2\le y\le 1$, let $x=y^4$. At $y=1/2$, we reach the point $(1/2^4,1/2)$.
Travel straight down to the $x$-axis.  So we reach the point $(1/2^4,0)$. Then travel back up to $(1/2^4,1/2)$.
Now for $1/3\le y \le 1/2$, travel along $x=y^4$, until we reach the point $(1/3^4,1/3)$. Travel down to the $x$-axis, then back up again.
Continue. For $1/4\le y\le 1/3$, travel along the curve $x=y^4$, then down to the $x$-axis, back up, and so on.
On the points of our path that are on $x=y^4$, our function is $1/2$, and on the $x$-axis, it is $0$, so if we travel towards $(0,0)$ on the complex path just described, the limit does not exist.


Prove that $\sin^{6}{\frac{\theta}{2}}+\cos^{6}{\frac{\theta}{2}}=\frac{1}{4}(1+3\cos^2\theta)$

Question: A question on submultiple angles states...

Prove that:$$\sin^{6}{\frac{\theta}{2}}+\cos^{6}{\frac{\theta}{2}}=\frac{1}{4}(1+3\cos^2\theta)$$

My efforts
I tried using the formula
$$a^3+b^3=(a+b)^3-3ab(a+b)$$
and
$$\cos^2{\frac{\theta}{2}=\frac{\cos\theta + 1}{2}}$$
Then I tried simplifying it:
$$\require{cancel} \begin{align} \sin^{6}{\frac{\theta}{2}}+\cos^{6}{\frac{\theta}{2}} & = (\sin^{2}{\frac{\theta}{2}}+\cos^{2}{\frac{\theta}{2}})^3 - 3\sin^2{\frac{\theta}{2}}\cos^2{\frac{\theta}{2}}(\sin^{2}{\frac{\theta}{2}}+\cos^{2}{\frac{\theta}{2}})\\
& = 1 - 3\sin^2{\frac{\theta}{2}}\cos^2{\frac{\theta}{2}}\\
& = 1 - 3(1 - \cos^2{\frac{\theta}{2}})\cos^2{\frac{\theta}{2}}\\
& = 1 - \frac{3}{2}(\cos\theta+1) + \frac{3}{4}(\cos\theta+1)^2\\
& = \frac{4\cancel{-6\cos\theta}-2+3\cos^2+3+\cancel{6\cos\theta}}{4}\\
& = \frac{1}{4}(5+3\cos^2\theta)\end{align} $$
I suspect I must have messed up with some sign somewhere. The trouble is, I can't seem to find where. Please help me in this regard.

Update: I am not accepting an answer because all the answers are equally good. It would be an injustice to the other answers.

Note to the editors: I also suspect that my post is a little short on appropriate tags. If required, please do the needful.
Comment: -2 is wrong. It's -3*2=-6
Comment: @Exodd Oh... that's where I slipped up. I ought to be more careful. Thank you very much!
Answer: $$\sin^6\frac{\theta}{2}+\cos^6\frac{\theta}{2}=\cdots=1-3\left(1-\cos^2\frac{\theta}{2}\right)\cos^2\frac{\theta}{2}$$
is correct. From here, note that
$$\begin{align}\sin^6\frac{\theta}{2}+\cos^6\frac{\theta}{2}&=1-3\left(1-\color{red}{\cos^2\frac{\theta}{2}}\right)\color{red}{\cos^2\frac{\theta}{2}}\\&=1-3\left(1-\color{red}{\frac{\cos\theta+1}{2}}\right)\cdot\color{red}{\frac{\cos\theta+1}{2}}\\&=1-3\cdot\frac{1-\cos\theta}{2}\cdot\frac{1+\cos\theta}{2}\\&=\frac{4-3(1-\cos^2\theta)}{4}\\&=\frac{1+3\cos^2\theta}{4}\end{align}$$
Answer: You may begin factoring:
\begin{align*}
\sin^6\frac\theta2+\cos^6\frac\theta2 &=\Bigl(\sin^2\frac\theta2+\cos^2\frac\theta2\Bigr)\Bigl(\sin^4\frac\theta2-\sin^2\frac\theta2\cos^2\frac\theta2+\cos^4\frac\theta2\Bigr)\\
&=\Bigl(\cos^2\frac\theta2-\sin^2\frac\theta2\Bigr)^2+\sin^2\frac\theta2\cos^2\frac\theta2=\cos^2\theta+\frac14\sin^2\theta\\
&=\frac14(3\cos^2\theta+1).
\end{align*}
You also may linearise:
$$\cos^2\theta+\frac14\sin^2\theta=\frac12(1+\cos2\theta)+\frac18(1-\cos2\theta)=\frac18(3\cos2\theta+5).$$
Answer: You may use
$$
2\sin x \cos x= \sin (2x) \tag1
$$
giving
$$
\begin{align} \sin^{6}{\frac{\theta}{2}}+\cos^{6}{\frac{\theta}{2}} & = (\sin^{2}{\frac{\theta}{2}}+\cos^{2}{\frac{\theta}{2}})^3 - 3\sin^2{\frac{\theta}{2}}\cos^2{\frac{\theta}{2}}(\sin^{2}{\frac{\theta}{2}}+\cos^{2}{\frac{\theta}{2}})\\
& = 1 - 3\sin^2{\frac{\theta}{2}}\cos^2{\frac{\theta}{2}} \\
& = 1 - \frac34\sin^2{\theta}\quad (\text{using} \,(1))\\
& = 1 - \frac34(1-\cos^2{\theta})\\
& = \frac14(1+3\cos^2{\theta})
\end{align}
$$ as desired.
Answer: Hint: $$\sin^2\theta/2\cos^2\theta/2\\=\frac{1}{4}(1-\cos\theta)(1+\cos\theta)=\frac{1}{4}(1-\cos^2\theta)$$
Answer: A little life-saving trick:
$$
\require{cancel} \begin{align} \sin^{6}{\frac{\theta}{2}}+\cos^{6}{\frac{\theta}{2}} & = (\sin^{2}{\frac{\theta}{2}}+\cos^{2}{\frac{\theta}{2}})^3 - 3\sin^2{\frac{\theta}{2}}\cos^2{\frac{\theta}{2}}(\sin^{2}{\frac{\theta}{2}}+\cos^{2}{\frac{\theta}{2}})\\
& = 1 - 3\sin^2{\frac{\theta}{2}}\cos^2{\frac{\theta}{2}}\\
& = 1 - 3(1 - \cos^2{\frac{\theta}{2}})\cos^2{\frac{\theta}{2}}\\
& = 1 - \frac{3}{2}(\cos\theta+1) + \frac{3}{4}(\cos\theta+1)^2\\
& = \frac{4\cancel{-6\cos\theta}-2+3\cos^2+3+\cancel{6\cos\theta}}{4}\\
& = \frac{1}{4}(5+3\cos^2\theta)\end{align}
$$
is wrong because, if you evaluate at $\theta=0$ you get
$$
\require{cancel} \begin{align} 1 & = 1\\
& = 1 \\
& = 1 \\
& = 1 \\
& = 2\\
& = 2\end{align}
$$
and this also shows the passage that is wrong.
Comment: Life-saver indeed. I believe that this is similar to testing the formula $$\cos {3\theta}=4\cos^3\theta - 3\cos\theta$$ by putting $\theta=0$.
Comment: It isn't a test for validity, but a way to find your errors
Answer: Let $\theta/2=x$. Then the LHS is:
$$\sin^6 x+\cos^6 x=\underbrace{\left(\sin^2 x+\cos^2 x\right)}_{=1}\left(\sin^4 x-\sin^2 x\cos^2 x+\cos^4 x\right)$$
$$=\left(1-\cos^2 x\right)^2-\left(1-\cos^2 x\right)\cos^2 x+\cos^4 x\tag{1}$$
The RHS is:
$$\frac{1}{4}\left(1+3\cos^2 2x\right)=\frac{1}{4}\left(1+3\left(2\cos^2 x-1\right)^2\right)\tag{2}$$
Now prove $(1)=(2)$.


How to get data from database?

Question: I have created 'smphas' table in my database. To retrieve the data, I write this code:
function goBack() {
  window.location.replace("dashboardproject.jsp");

}
 function getQuarter(phase){
  if(phase==""){
     alert("PLEASE CHOOSE PHASE");
  } else {
  document.getElementById("phase").value = phase;
  constructionprogress.submit();
  }

}
  function selected(){
  $('#phaseCons').val('<%=phase%>');

}
Mysql command is:
    public List<String[]> constPeriodPhase (String projCde,String co_no){
    System.out.println("/*** constPeriodPhase ***/");

    String sql = "SELECT distinct smh_phase_num, smh_phase_nme FROM c"+co_no+".smphas WHERE smh_proj_cde = '"+projCde+"'";
    System.out.println("Execute = "+sql);
    return super.execListStrArr(sql,false);
}

However, the error is:
/* constPeriodPhase */
Execute = SELECT distinct smh_phase_num, smh_phase_nme FROM cnull.smphas WHERE smh_proj_cde = 'null'
nullnulle-Solicitor[cnull.smphas]
2020-03-12 09:53:58.165 SGT EXECUTE: SELECT distinct smh_phase_num, smh_phase_nme FROM cnull.smphas WHERE smh_proj_cde = 'null'
java.sql.SQLSyntaxErrorException: Unknown database 'cnull'
How can i retrieve the database from mysql?
Answer: Unknown database 'cnull'
with a query:
...FROM c"+co_no+".smp

mean co_no is null when the function was called.

How to get data from database?


Evaluating if statement in python from a string

Question: I'm working on a project that allows users to define filters to sort data in a csv file. They can create as many or as few filters as they want so I am trying to create an if statement from a string that will get longer as users define more filters.
I have an output for my string variable which is this: row[0]=="6/1/2014 0:00:00"
So for simplicity sake I'll just hard code that into a variable name in the sample code below
The problem is that it doesn't correctly evaluate this if statement. It returns every row of the file.
f = open("somefile.csv", "r")
r = csv.reader(f)

statement = 'row[0]=="6/1/2014 0:00:00"'
for row in r:
    if statment:
       print(row)

Comment: Of Course it will evaluate to `True` for every iteration, since you are checking against a value that is not `False`...
Comment: How much trust do you have in these users?  In some ways, this looks like a case for `eval`, but if you don't have 100% trust in your users, then `eval` is not safe and you'll need to parse the string yourself (e.g. via `ast`).
Comment: Is there a way I could use that string in the if statement the same way I would do it if I type it out myself?
Comment: there is always a better way to do things than `eval`. might do you some good to create some sort of api for your users that will create filters in a safer manner
Comment: the string is a result of some other code I created to parse through user input so yeah I'll try eval
Comment: Why not dump the CSV into a [`sqlite3`](https://docs.python.org/2/library/sqlite3.html) database -- Then your users can query it using more familiar SQL and you don't need to use `eval` or `ast` to parse the filters.
Answer: Despite the obvious security reasons you will have to take care of, you can use eval:
if eval(statement):
    print row

Comment: [Eval is really dangerous](http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html)
Comment: Yup, better create a high level interface for your users to define filters and then you may interpret the choices into actual python statements. This was just a quick answer to your quick workaround, but doesn't mean this is the way to go.
Comment: Should consider `ast` module instead
Comment: If the users are explicitly trusted (i.e., this is a piece of code that runs on a local machine, and not a web service or the like), then eval() is perfectly fine.
Comment: What would be another approach that would be safer? what is the ast module?


Make Hook in main-menu

Question: I created a module containing the following files:

menu_imoveis.info
menu_imoveis.module

The content of menu_imoveis.info is the following.

    name = Menu Imoveis
    description = Adiciona imoveis ao menu.
    package = Imoveis
    core = 7.x
    version = 7.x-dev

    files[] = menu_imoveis.module

**The content of menu_imoveis.module is the following.
function menu_imoveis_menu() {
  $items['node/27'] = array(
    'title' => 'Example Page',
    'menu_name' => 'main-menu',
    'type' => MENU_NORMAL_ITEM,
  );

  return $items;
}

The code I wrote doesn't work.
Comment: Did you clear your cache "Admin -> Config -> Performance -> clear cache". Or `drush cc all`. When you use hook_menu() you should clear your cache to see the changes. Also, I get you speak 2 languages -- I recommend trying to write your code in 1 language (english) so it's more maintainable.
Comment: Tanks tenken, But I clear the cache and run cron.
Comment: Where is the page callback?
Comment: It's not clear what you're trying to accomplish. I suggest menu_example for learning how Menu API works: http://api.drupal.org/api/examples/menu_example%21menu_example.module/7
Answer: Taken from this comment, written by zifiniti:

I was having trouble getting a menu item to display with the following code:
function mymodule_menu() {
  $items['custom-page'] = array(
    'title' => 'Custom Page',
    'menu_name' => 'main-menu',
    'access arguments' => array('access custom pages')
  );

  return $items;
}

It was continuously denying me access to the specified path in addition to not displaying the item in the menu. I discovered that the reason behind this is if you do not specify a page callback, then the access callback will acquire a value of 0 in the database and thus render that path/menu item inaccessible.
Working code:
function mymodule_menu() {
  $items['custom-page'] = array(
    'title' => 'Custom Page',
    'menu_name' => 'main-menu',
    'page callback' => 'mymodule_custom_page',
    'access arguments' => array('access custom pages')
  );
}

Answer: Ok Looking at it you are trying to edit a menu item which is already created. So you need to use the HOOK_menu_alter function.
function  mymodule_menu_alter(&$items) {
  $items['node/27']['title'] = 'Something';
}

Also you might need to clear cache for this function to be triggered.
Comment: Actually, node/27 is not defined; it is node/%node that is defined.
Comment: Yes that is correct. My mistake.

Make Hook in main-menu


Is there an name for a graph with this property?

Question: Just had this thought and wondered if there was a name for this type of graph that satisfies this transitivity property (would more easily let me read what other people have learned about it if so).
A weighted graph where given any vertices $v_i, v_j, v_k$, the weight of edge
$$w(v_i, v_k) = w(v_i, v_j) + w(v_j, v_k)$$
Comment: Does the existence of edges $(v_i,v_j)$ and $(v_j,v_k)$ mean that there _must_ exist a $(v_i,v_k)$ with the weight you specify, or only that _if_ it exists _then_ its weight must be as given by your equation?
Answer: Suppose $G$ has at least $3$ vertices, and satisfies the specified edge-weight conditions.

Let $a,b,c$ be distinct vertices of $G$, and let
$$r=w(a,b),\;\;\;s=w(b,c),\;\;t=w(c,a)$$
Then we get the system of equations
$$
\begin{cases}
r=s+t\\[4pt]
s=t+r\\[4pt]
t=r+s\\
\end{cases}
$$
which implies $r=s=t=0$.

Thus, all edge weights must be zero.
Comment: that is assuming $G$ is a complete undirected graph
Comment: Yes, I'm assuming that for any two distinct vertices $a,b$, the edge $ab$ has _some_ weight, and that $w(a,b)=w(b,a)$.
Comment: Clearly, not a realistic assumption, as your argument most vividly demonstrates. I think the OP is thinking to impose this on a directed incomplete graph
Comment: Well, I'll edit or delete my answer as soon as the OP clarifies the specs. In the meantime, my answer should at least alert the OP that more explicit assumptions need to be stated in order to avoid having only a trivial solution.
Comment: I am not saying you should delete your answer; on the contrary -- it's pointing out a **very** useful feature


Poisson Regression : expectation vs probability for each outcome

Question: The Poisson regression can be used to predict the expectation count.

e.g: E(number of car | male = 1 & income = 40000) = 1.3

To get this, we can use the predict function in R, e.g:
predict(poisson_model, data, type="response")

Question:

How can I get in R the probability of a specific income using the probability mass function? Is there a specific function in Rto do so?

e.g: P(number of car = 0 | male = 1 & income = 40000) = 13%
e.g: P(number of car = 1 | male = 1 & income = 40000) = 28%
Comment: Related:  https://stats.stackexchange.com/questions/142338/goodness-of-fit-and-which-model-to-choose-linear-regression-or-poisson/142353#142353
Answer: If you have a fitted expectation (as you show you can get), using that as a parameter in a Poisson pdf should work.
That is, given $\hat{\lambda}_i = E(Y=0|X=\boldsymbol{x}_i)$ you can use that to calculate whichever probability you want:
$$\hat{P}(Y=k|X=\boldsymbol{x}_i) = \frac{\exp(-\hat{\lambda}_i)\, \hat{\lambda}_i^k}{k!}$$
(In R you'd use dpois() with the parameter set to the fitted mean for that kind of calculation. The process should be about as easy in any number of other packages.)
Answer: Just to illustrate the canonical answer by Glen (and with the peace of mind of being late for the party to the point that the answer to the OP has rightfully been adjudicated), let me run a toy example in R, crediting this post also by Glen.
Instead of cars and money, I'll make it more picturesque, and focus on the number of fish caught by a fishing company, in relation to the number of vessels out in the sea - if the numbers seem small, imagine that the units are in $10,000$, or something else. The number of fish caught will follow a Poisson distribution with a rate parameter identical to the number of boats from $\lambda=1$ to $\lambda = 14.$
set.seed(0)
sam = 1000         # Random values from a Pois for each lambda.
lambda = c(1,4,7,10,13) # And these are the lambda values.
x = c(rep(lambda[1], sam), rep(lambda[2], sam), rep(lambda[3], sam),
      rep(lambda[4], sam), rep(lambda[5],sam)) # These are the number of boats
peixos = rpois(length(x), x)  # And the number of fish caught.

We are establishing a relationship between the expected mean and the $x$ axis strictly linear in this simulation:
$\mathbb E(Y\;\vert\;\text{no. boats})=  X\beta + \varepsilon$
but with a Poisson distribution of errors, which parameter changes with the value of the explanatory variable.
The rate parameter $\lambda$ increases strictly linearly with the number of boats. In the case of a single boat $(\lambda = 1)$ the distribution around the expected mean is manifest in the density of points close to $1$ in the $y$-axis on the plot below - the data points produce a stripped pattern due to the discrete nature of the Poisson distribution. As the value of lambda increases, the distribution of points, and the corresponding histogram, assume more of a normal shape:

The errors (or residuals if you want) are differently spread depending on the number of boats, because we used $5$ different rate parameters. However, once you know the rate parameter for a given value of the explanatory variable, the probability mass function is determined, allowing you to calculate the probability of any other value.
If we tried to recover the lambdas ($\lambda =\{1,4,7,10,13\}$), we'd only have to run the following command:
> fit = glm(peixos ~ x, family = poisson(link = identity))
> unique(predict(fit))
[1]  1.038018  4.023809  7.009600  9.995391 12.981182

Notice that in keeping with the set up of the toy dataset, the rate parameter was linearly dependent with an identity link. The output would have been off if we had run
> fit <- glm(peixos ~ x, "poisson")
> exp(unique(predict(fit)))
[1]  2.270660  3.602671  5.716065  9.069216 14.389387

implicitly using a $\log$ link function - see this post.

code here


A matrix identity

Question: Suppose $K^TX=0$ and $K$ has full column rank. Suppose furthermore that the columns of $X$ form a basis for the nullspace of $K^T$. If $H$ is positive definite, then
$$
K(K^THK)^{-1}K^T = H^{-1} − H^{-1}X(X^TH^{-1}X)^{-1}X^TH^{-1}
$$
The identity can be found on slide 44 at http://www.ltcc.ac.uk/media/qmul-images/REML-Lecture-1.pdf. A quick R script has also verified the formula. However, I can't seem to prove the identity. I have tried using the Schur formula for inverses, but it does not seem to help in this case.
Comment: You need to specify that the columns of $X$ give a basis for the nullspace of $K^\top$. Then you should see why the formula holds when $H$ is the identity.
Comment: @TedShifrin Thank you Prof. Shifrin. That makes perfect sense now: In the case where $H=I$, then for any vector $v$ we may express $v$ uniquely as $v=K \gamma_1 + X\gamma_2$. Then, $K(K^TK)^{-1}K^T(K \gamma_1 + X\gamma_2) + X(X^TX)^{-1}X^T(K \gamma_1 + X\gamma_2) = K \gamma_1 + X\gamma_2$ is straightforward to see which yields the formula $K(K^TK)^{-1}K^T + X(X^TX)^{-1}X^T = I$. When $H \neq I$, if we could somehow express $v=HK \gamma_1 + X \gamma_2$, we could then use the same strategy to show the formula holds - but it is not obvious to me that $v=HK \gamma_1 + X \gamma_2$
Answer: Although you have not revised appropriately, we agree that the columns of $X$ give a basis for the nullspace of $K^\top$. This subspace is the orthogonal complement of the column space $W$ of $K$. Assuming $H=I$, we therefore have
\begin{align*}
I &= \text{proj}_W + \text{proj}_{W^\perp} \\
&= K(K^\top K)^{-1}K^\top + X(X^\top X)^{-1}X^\top.
\end{align*}
Here $\text{proj}_V$ is the matrix for the orthogonal projection onto the subspace $V$.
In the general case, we let $\tilde K = \sqrt H K$, where $\sqrt H$ is the unique positive square root of the positive definite symmetric matrix $H$. We set $\tilde X = \sqrt H^{-1} X$. Then the columns of $\tilde X$ give a basis for the nullspace of $\tilde K$. Thus, as before, we have
\begin{align*}
I &= \tilde K(\tilde K^\top\tilde K)^{-1}\tilde K^\top + \tilde X(\tilde X^\top\tilde X)^{-1}\tilde X^\top \\
&= \sqrt H K(K^\top H K)K^\top \sqrt H + \sqrt H^{-1}X^\top(X^\top H^{-1}X)^{-1}X^\top\sqrt H^{-1}, \text{so} \\
H^{-1} &= K(K^\top HK)K^\top + H^{-1}X^\top(X^\top H^{-1}X)^{-1}X^\top H^{-1},
\end{align*}
as desired.
Comment: Thank you again, Prof. Shifrin, fully understood. I did not think of forming the matrices $\bar{K}$ and $\bar{X}$ - that was nice. I will update my question to include the correct  assumptions on $X$ for future readers

A matrix identity


Chi-Squared Goodness of Fit Test Alternative? - Zero Can't Be in Denominator

Question: I have 5 zones(categories) in which a certain percentage of total sinkholes exist. I have 5 different maps that I am testing to see which one provides me with the best fit to my expected percentages per category.
My hope is that 0% of the sinkholes will be in category 1, 10% in category 2, 20% in category 3, 30% in category 4, and 40% in category 5 (summing up to 100% total).
I am trying to compare my observed set of percentages (for each map) with my desired set of percentages.
I thought using a Chi-Squared Goodness of Fit test would be the best option, but when I attempt to calculate the Test Statistic, I run into problems, obviously, since one of my desired values is 0%, and 0 can't be in the denominator.
Any ideas as to how I can approach this statistical analysis without using the Chi-Square Test?
Thank you!
Answer: Actually, I think you can take the limit just fine.
Consider $\lim_{E_i\to 0} (O_i-E_i)^2/E_i$ under two cases:
Case 1: $O_i > 0$. In this case the term goes off to infinity in the limit, and the overall chi-square statistic goes with it.
Case 2: $O_i = 0$. In this case the term equals $E_i^2/E_i = E_i$, which goes to $0$ in the limit. So if $O_i=0$, the term adds nothing to the chi-square statistic.
Your statistic doesn't have a chi-squared distribution under the null, but it's perfectly possible to simulate it under the null (whence $O_i$ must be $0$), as long as you replace the contribution $(O_i-E_i)^2/E_i$ for the cell with $E_i=0$ with the limiting value $0$.
Then if $O_i$ for that cell is 0, you can compute the overall chi-square and compare with the simulated distribution. If $O_i$ is anything but 0, you can reject the null immediately; it's not possible to observe that if the null were true.
You can't simply run it through a canned routine as-is, but with a little bit of effort you can still do a test as described.

An alternative might be to use something from the power-divergence statistics
$\frac{2}{\lambda(\lambda+1)}\sum_{i=1}^kO_i[(O_i/E_i)^\lambda-1]$
where $\lambda$ is chosen so the statistic will always exist.
I believe an appropriate reference for these is Cressie and Read (1984)$^{[1]}$.
e.g. something like the Freeman-Tukey $F^2 =4\sum_i (\sqrt{O_i}-\sqrt{E_i})^2$
However, instead of auto-rejecting, an $O_i$ of 1 would only contribute 4 to the statistic. You would likely need to simulate the null distribution of the statistic here; the asymptotic chi-square approximation may not be so good.

A bigger issue, perhaps, is to note that your categories are ordered. I don't think it would make sense to use a chi-square-like statistic in any case, since it throws away a lot of power in the rest of the table. You might use something like an Anderson-Darling statistic, but (again) with simulated distribution under the null (you have a similar issue to the above in applying this test, but it should have better power if you use an analogous approach to solving it).
[I wouldn't use Kolmogorov-Smirnov because it won't automatically reject the impossible case. At least I wouldn't use it as-is, but a test of that form could be adapted to behave as you'd hope.]

[1]: Cressie, N. A. C. and Read, T. R. C., (1984),
"Multinomial goodness-of-fit tests,"
J. Roy. Statist. Soc. Ser. B, 46, 440-464


OpenWeatherMap API, How do I fetch the temp?

Question: Using Openweather JSON API, I am fetching the data from cities. I can retrieve all the data,the data I am focused on is the temp, and dt_txt.
I have tried getting the temp by doing this but it doesn't work:
import requests

def getCity():
    allCityName = ["London"]
    for cityName in allCityName:
        url = "http://api.openweathermap.org/data/2.5/forecast?q="+ cityName +"&appid="
        response = requests.get(url)
        data = response.json()
        print(data["main"]["temp"]

I've tried many combinations but always seem to get an error or no result.
Any help would be appreciated.
Answer: maybe you don't realize this but there are more than 1 temp and dt_text returned for each city. So you can't just pull one out unless you know you want the first, last or somewhere in the middle. Or if you want them all you have to collect them in a list, for example
data = getCity()
for d in data['list']:
    print(d['main']['temp'])
    print(d['dt_txt'])

281.44
2020-12-15 06:00:00
281.33
2020-12-15 09:00:00
283.54
2020-12-15 12:00:00
283.19
2020-12-15 15:00:00
281.88
2020-12-15 18:00:00
281.4
2020-12-15 21:00:00
281.17
2020-12-16 00:00:00
281.59
2020-12-16 03:00:00
281.75
2020-12-16 06:00:00
281.95
2020-12-16 09:00:00
283.61
2020-12-16 12:00:00
282.86
2020-12-16 15:00:00
282.99
2020-12-16 18:00:00
283.17
2020-12-16 21:00:00
281.61
2020-12-17 00:00:00
281.19
2020-12-17 03:00:00
281.27
2020-12-17 06:00:00
280.81
2020-12-17 09:00:00
282.71
2020-12-17 12:00:00
282.64
2020-12-17 15:00:00
281.16
2020-12-17 18:00:00
282.16
2020-12-17 21:00:00
282.52
2020-12-18 00:00:00
283.4
2020-12-18 03:00:00
284.03
2020-12-18 06:00:00
283.86
2020-12-18 09:00:00
284.4
2020-12-18 12:00:00
284.19
2020-12-18 15:00:00
283.87
2020-12-18 18:00:00
284.21
2020-12-18 21:00:00
284.34
2020-12-19 00:00:00
284.53
2020-12-19 03:00:00
284.44
2020-12-19 06:00:00
283.48
2020-12-19 09:00:00
284.81
2020-12-19 12:00:00
284.55
2020-12-19 15:00:00
283.7
2020-12-19 18:00:00
283.33
2020-12-19 21:00:00
282.42
2020-12-20 00:00:00
281.76
2020-12-20 03:00:00

temps = [d['main']['temp'] for d in data['list']]

In [59]: print(temps)
[281.44, 281.33, 283.54, 283.19, 281.88, 281.4, 281.17, 281.59, 281.75, 281.95, 283.61, 282.86, 282.99, 283.17, 281.61, 281.19, 281.27, 280.81, 282.71, 282.64, 281.16, 282.16, 282.52, 283.4, 284.03, 283.86, 284.4, 284.19, 283.87, 284.21, 284.34, 284.53, 284.44, 283.48, 284.81, 284.55, 283.7, 283.33, 282.42, 281.76]

Comment: @Chinez no problem
Comment: @taylorMcdonnell, accept his answer then, click on the tick button


Problema com condiçao no While em C++

Question: Olá, estou estudando a partir do livro "PROGRAMMING Principles and Pratice Using C++" do Bjarne. Estou tentando fazer Drill número 1 do capitulo quatro. O exercício diz para fazer um programa que consiste em um while-loop que leia e imprima dois inteiros e pare quando for escrito |, porem, quando escrevo | (ou qualquer outra coisa que não seja um número) o programa fica em um loop infinito. Se alguém puder me explicar o que tem de errado no meu código, já tentei ler no livro sobre conversões de int pra char e vice e versa e mesmo assim o erro persiste, deem uma olhada.
#include <...\Projects\std_lib_facilities.h>

int main()
{
    //drill's
    //1 - inicio

    /*escrever um programa que consiste em um while-loop que leia dois int e termine quando
é escrito |  */

    int n=0, n1=0;

    cout << "Write a int. To stop the program hit |";

    while (n != '|' || n1 != '|' )
    {
        cin >> n >> n1;

        system("cls");

        cout << n << " " << n1 << "\n";
    }
}

Comment: Tem alguma coisa que diz que as variáveis precisam ser `int`? Dá para colocar o enunciado do exercício? Ou é só isso aí?
Comment: Estou tentando entender a intenção de fazer isto para dar uma resposta.
Comment: Isso foi perguntado no SO, mas parece que o exercício é outro :)
Comment: /*escrever um programa que consiste em um while-loop que leia dois int e termine quando
é escrito |  */ é este, irei comentar abaixo o original do livro.
Comment: Write a program that consists of a while-loop that (each time around the loop) reads in two int and then prints them. Exit the program when a terminating '|' is entered.
Comment: Ok muito obrigado enquanto isso farei uma revisão no Cap se eu achar algo posto aqui @bigown
Comment: É o gomiero passou o link
Answer: Conforme o post no SO (em inglês):
https://stackoverflow.com/questions/10349857/how-to-handle-wrong-data-type-input

A razão para o programa entrar em um looping infinito é por causa de
  um flag (sinalizador) de entrada inválida do dispositivo
  std::cin.
O que precisa ser feito é limpar a flag e descartar a entrada inválida
  do buffer de entrada.

Após a correção, o código fica assim:
while (n != '|' || n1 != '|')
{
    // limpa a flag
    std::cin.clear();

    // Descarta a entrada
    std::cin.ignore(std::numeric_limits<std::streamsize>::max(), '\n');

    std::cin >> n >> n1;

    system("cls");

    std::cout << n << " " << n1 << "\n";
}

A entrada "errada" está ocorrendo por causa da leitura de um (tipo) caracter "| " em uma variável do tipo numérica n ou n1.
Para mais detalhes (em inglês):
C++ FAQ
Outras possibilidades para corrigir:

ler o conteúdo | para uma variável tipo char
ler cada valor para um buffer do tipo string e depois converter para inteiro com a função std::stoi

Atualização:
Após aplicar a segunda sugestão de correção, o looping fica assim:
int n = 0, n1 = 0;

std::string buffer;

std::cout << "Write a int. To stop the program hit |";
while (n != '|' || n1 != '|')
{
    // Lê a primeira entrada
    std::cin >> buffer;

    // Se for um "|", sai do looping
    if (buffer == "|")
        break;
    // Converte para inteiro
    n = std::stoi(buffer);

    // Lê a segunda entrada
    std::cin >> buffer;

    // Se for um "|", sai do looping
    if (buffer == "|")
        break;

    // Converte para inteiro
    n1 = std::stoi(buffer);

    // Limpa a fila std::cin
    std::cin.clear();
    std::cin.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
    // std::cin >> n >> n1;

    //system("cls");

    std::cout << n << " " << n1 << "\n";
}

Mesmo com essa correção, o código acima ainda está sujeito a outros erros.
Abaixo, segue uma sugestão de leitura para entender melhor como funciona o sistema de entrada e saída de dados em C++:
Entrada e saída de dados
Comment: Muito obrigado gomiero irei testar e ler o FAQ para ficar mais adentro.
Comment: bom o looping infinito resolveu porem para min não mudo nada é como se a condiçao do while não existise agora pois escrever | ele simplismente descarta o | e pede para ler outro digito no lugar ao invez de terminar o while, eu ja tentei ler o conteudo | em um char irei testar fazendo a conversao de string para int agora que foi a unica opçao que nao tentei até agora. obrgado mais uma vez pela ajuda
Comment: meu professor comentou o seguinte "Se vc não mudar o valor de uma das variáveis dentro do loop. Ela sempre vai ter o valo | e portanto o laço vai ser repetido...."
Comment: converti para string depois pra int vlw ajudou bastante ainda mais os links muito obrigado.
Comment: perfeito exatamente essa logica que usei obrigado mais uma vez
Comment: viu eu sempre venho aqui dar uma olhada nessa explicação ela resolve altas coisas ainda XD tenho que tomar mais cuidado nessas coisas obrigado novamente cara

Problema com condiçao no While em C++


Spark (Java) - Adding a single average as a new column

Question: I have two dataframes, df1 and df2, and I'd like to add a new column to the second one. This new column should be the average of a column from the first dataframe. Something like this:
df1                  df2                   df2
userid count value   userid count          userid count value
11     2     5       10     1              10     1     5
22     3     4       20     1     ======>  20     1     5
33     5     6       30     1              30     1     5

I'm trying
df2 = df2.withColumn("value", avg(df1.col("value")));

which is not working. How can I do this?
Thank you!
Comment: you need to join both dataframes before you can do any operation. Spark doesn't know how to relate df1 to df2.
Comment: Oh. That seems tedious, since in the end I'd want to `.unionAll()` them into a single df, but I can't do that until they have the same no of columns...
Comment: I think the problem you try to solve is related with http://stackoverflow.com/a/29950853/702002
Answer: It's similar to Append a column to Data Frame in Apache Spark 1.3
withColum() should have a column related to the DateFrame, so you can make a transformation:

cal the avg value
when adding a new column, set the original value as 0, and then add the avg value
import org.apache.spark.sql.functions._
val avgValue = df1.select(avg(df1("value"))).collect()(0).getDouble(0)
df2 = df2.withColumn("value", rand() * 0 + avgValue)

Comment: Ah! `rand() * 0 + avgValue` is really clever to genereate a column of data from a single value. Thanks! I'll test it on Monday, but I'll trust you on this and accept your answer ;)


Which MLB team had the shortest "longest winning streak" in a pennant-winning season?

Question: Which team won the pennant but never won more than X games in a row, with X being the lowest number?
For example, the hypothetical team the Rainbow Sox won the pennant in 1717, but never won more than 3 games in a row during that pennant winning season, and no team had a lower number, then the answer would be the "Rainbow Sox in 1717, longest streak of 3".
Answer: There are three teams in MLB history who have won the National League pennant but never won more than 5 games in a row during their winning season.
The 2010 San Francisco Giants, 2011 St. Louis Cardinals and 2018 Los Angeles Dodgers all won the pennant with their longest winning streak being 5 games.


oracle character to number conversion error from .net

Question: I have a procedure in oracle having input fields varchar2 and number. I am adding parameters from my .Net code using Oracle.DataAccess.dll. I am getting an exception like

ORA-06502: PL/SQL: numeric or value error: character to number conversion error\nORA-06512: at line 1

If I run procedure directly from oracle sql developer, it works fine.
code:-
OracleCommand cmd = (OracleCommand)_dbFactory.CreateCommand();
            try
            {
                cmd.Connection = (OracleConnection)_conOracle;
                _conOracle.Open();
                cmd.CommandText = "SERVICE_STATUS";
                cmd.CommandType = System.Data.CommandType.StoredProcedure;
                cmd.Parameters.Clear();

                cmd.Parameters.Add("PANUMBER", OracleDbType.Int32).Value = 10;
                cmd.Parameters.Add("PA_LINK", OracleDbType.Varchar2).Value = "Test";
                cmd.Parameters.Add("PO_MSG", OracleDbType.Varchar2, 4000).Direction = System.Data.ParameterDirection.Output;
                try
                {
                    cmd.ExecuteScalar();
                }
                catch(Exception ex1) {
                    log.Debug("Exception ex1 "+ ex1.Message+" inner--"+ex1.InnerException);
                }

                string isValid = cmd.Parameters["PO_MSG"].Value.ToString();
                if (isValid == "SUCCESS")
                    return true;
                else
                    return false;
            }
            catch (Exception ex)
            {
                throw ex;
            }
            finally
            {
                _conOracle.Close();
            }

procedure

create or replace
PROCEDURE SERVICE_STATUS( panumber number, pa_link varchar2 default NULL, po_msg OUT VARCHAR2 )
 is

BEGIN
   --logic comes here

END;

Comment: "Logic comes here" might be responsible. It seems that you're somewhere doing TO_NUMBER on a string that contains something else but digits; for example, TO_NUMBER('12F') will fail.
Comment: Do you still have a problem if you *comment out* all the `--logic comes here` but assign `po_msg := "Some Test Message";`?
Comment: @Littlefoot i understand, but i passed numeric field and it failed. I have not given a conversion like this.
Comment: @DmitryBychenko I dont have permission to edit the procedures. So i couldn't check that.
Comment: Try switching `OracleDbType.Int32` to `OracleDbType.Decimal` - Oracle maps a NUMBER without precision to a .NET Decimal not an integer - Also you should be calling `ExecuteNonQuery`...
Answer: Whenever you have a procedure with one OUT parameter I would rather prefer to create a function instead.
Anyway, looks like you cannot change that, so take what you have.

Try to add the parameter like this:

cmd.Parameters.Add("PO_MSG", OracleDbType.Varchar2, 4000, null, ParameterDirection.Output);
cmd.Parameters["PO_MSG"].DbType = DbType.String;

(Only relevant for older version of ODP.NET providers)
The cmd.ExecuteScalar(); seems to be wrong. See documentation:

This method executes the query using the connection, and returns the
  first column of the first row in the result set returned by the query.

You don't execute a query, you want to invoke a procedure. Use cmd.ExecuteNonQuery();
But the real issue should be the size of your output parameter. Apparently 4000 characters is not sufficient. As opposed to SQL (unless you set MAX_STRING_SIZE = EXTENDED) the max. size of VARCHAR2 datatype is 32767, see PL/SQL Program Limits.
So, change your parameter to
cmd.Parameters.Add("PO_MSG", OracleDbType.Varchar2, 32767).Direction = ParameterDirection.Output;

Comment: No luck.I think, in my case,output doesn't matters for this issue
Comment: Then the error seems to be inside the procedure. Try to execute it manually from sqlplus window and see if you get the same error.


FreeBSD pre-mount script for gnop device?

Question: How do I run a script in FreeBSD before filesystems are mounted?
I want to install a gnop device and mount it (no, not for ZFS). Is there any canonical place where I can put a gnop create ...?
Answer: Create an rc script, e.g.

#!/bin/sh

# PROVIDE: gnop disks
# BEFORE: fsck

. /etc/rc.subr

name="gnop"
start_cmd="${name}_start"
stop_cmd=

gnop_start()
{
    local disk="/dev/da1"
    /sbin/gnop create -S 4096 "$disk" || err 1 "Creating gnop on $disk failed"
}

load_rc_config $name
run_rc_command "$1"

where /dev/ada1
This could be extended to created multiple nop devices based on a configuration in rc.conf.


Converting a dataframe column containing variable length lists to mutliple columns in dataframe

Question: I have pandas dataframe that looks like the following.
The column props contain lists and the elements in the list is varying in length. I know the maximum number of entries in the list is 5. I also know that the list is ordered, i.e. I know that the second item always belong to the column with a specific header say "Tense" or "number". Then how can I convert each of the entry in lists into separate columns?
id  source   type   target          props                        subtype
2   wyrzucić    V   wyrzucisz       [FUT, 2, SG]                 NaN
6   śniadać     V   śniadać         [NFIN]                       NaN
7   bankrutować V   bankrutujący    [PST, ACT, PL, MASC, HUM]    PTCP
8   chwiać      V   będą chwiały    [FUT, 3, PL]                 NaN
23  dobyć       V   dobyłaś         [PST, 2, SG, FEM]            NaN

I have tried solutions with usntack() and also with tolist() methods. But the solutions do not work for the specific case.
Comment: check the link ? https://stackoverflow.com/questions/35491274/pandas-split-column-of-lists-into-multiple-columns
Answer: apply is usually slow. You can use
In [34]: df.join(pd.DataFrame(df.props.values.tolist()))
Out[34]:
   id                      props     0     1     2     3     4
0   2               [FUT, 2, SG]   FUT     2    SG  None  None
1   6                     [NFIN]  NFIN  None  None  None  None
2   7  [PST, ACT, PL, MASC, HUM]   PST   ACT    PL  MASC   HUM
3   8               [FUT, 3, PL]   FUT     3    PL  None  None
4  23          [PST, 2, SG, FEM]   PST     2    SG   FEM  None

Details
In [33]: df
Out[33]:
   id                      props
0   2               [FUT, 2, SG]
1   6                     [NFIN]
2   7  [PST, ACT, PL, MASC, HUM]
3   8               [FUT, 3, PL]
4  23          [PST, 2, SG, FEM]

Answer: You can try this UDF and see if it works -
def col_gen(x):
    props = x['props']
    for i in range(len(props)):
        x['Item'+str(i+1)] = props[i]
    return x

df = df.apply(lambda x: col_gen(x), axis=1)

This is taking every row, extracting the props column and appending it to additional columns
Comment: The solution is intuitive and great, and I was thinking why this didnt occur to me. Seems my pandas really got rustic. Thanks a lot for your time.
Answer: Consider this simplified dataframe
df = pd.DataFrame({'id': [2,6,7,8,23], 'props': [['FUT', 2, 'SG'], ['NFIN'], ['PST', 'ACT', 'PL', 'MASC', 'HUM'], ['FUT', 3, 'PL'],['PST', 2, 'SG', 'FEM']]})

You can split the list column using
df[[1,2,3,4,5]] = df.props.apply(pd.Series)

You get
    id  props                       1       2   3   4       5
0   2   [FUT, 2, SG]                FUT     2   SG  NaN     NaN
1   6   [NFIN]                      NFIN    NaN NaN NaN     NaN
2   7   [PST, ACT, PL, MASC, HUM]   PST     ACT PL  MASC    HUM
3   8   [FUT, 3, PL]                FUT     3   PL  NaN     NaN
4   23  [PST, 2, SG, FEM]           PST     2   SG  FEM     NaN

Note: You can specify more relevant column names, I just used 1,2,3,4,5


Subseries converge

Question: I want to ask:
Let's say we have the series of non-negative terms $\sum_{n=0}^\infty a_n$ which is convergent.
Can I directly say that the series $\sum_{k=0}^\infty a_{2k}$  is convergent because all of its terms contain in the first serie which is convergent and they are non-negative?
And what about if they can be negative?
Comment: For the negative case look at $\sum_n(-1)^n/n$, then the even terms is $\sum_n1/(2n)=2\sum_n1/n$.
Answer: Yes, it is true, because $0\leqslant\sum_{k=1}^na_{2k}\leqslant\sum_{k=1}^{2n}a_k$.
But if the $a_n$'s can be negative, then it is false. Consider the convergent series$$1-1+\frac12-\frac12+\frac13-\frac13+\cdots$$It converges, but$$-1-\frac12-\frac13-\cdots$$diverges.
Comment: Written as it is, this assertion is false. The correct assumption is that the terms of the bigger series are positive. Otherwise you have counterexamples like the one given by José above.
Comment: @spyer Yes (assuming that the terms of the "big" series are non-negative).
Comment: So, if a serie's terms are a subset of the set of terms of a convergent serie and are non-negative I can say that the first serie also converges?
Comment: Yes, I meant this. Thank you very much!!!

Subseries converge


Configuring a linux machine for high inbound network load

Question: I'm running Ubuntu 12.04 on an a dual processor Intel E5-2620-HexCore, 64GB of RAM, 1Gb network link.
When downloading a single test file from a location in the same datacenter (Public network) I get d/l rate of 80-100 MB/s (Bytes/second).
When crawling web pages using multi-process PHP design, no matter how many processes I run, the d/l rate stays steady on 200Mb/s (bits/second).
rate of packets per second is around 300K inbound.
I've tried some of the settings recommended here
http://http-kit.org/600k-concurrent-connection-http-kit.html
but that didn't make any difference.
Any ideas?
Comment: The large file download is apparantly limited by bandwidth and the other task runs into for instance a CPU or IO limit before reaching the bandwidth limit instead.
Comment: @Hbrujin hdparm shows my HD speed can get to 150MB/s write, where iotop shows current load of only 2M/s. cpu and memory load are not even close to their limit. so that's probably not the issue.
Comment: hdparm  is a horrible tool for testing hard drive performance.  You really should be using something like bonnie++, or iometer.  `hdparm -t` is a sequential raw-read operation, it doesn't tell you anything about the writes, or random access.  It also completely ignores the filesystem overhead.
Comment: @Zoredache bonnie++ shows a not so different rate of 137MB/s. how does this rate limit d/l rate in any way?
Answer: Have you looked at the firewall that is between your server and the internet? I know most firewalls will have default limits on packets and concurrent connections for any single host inside or outside the network. Especially if the firewall comes with IDS.


Outlook 2010 fails to connect to Exchange 2013

Question: I have an Exchange Server 2013 and I am trying to connect using Outlook 2010. Both are running on the same server. The Outlook installation is just for testing mailboxes.
Outlook first refused to connect because the network adapter didn't have a default gateway and apparently Outlook requires a default gateway to connect to 127.0.0.1.
Now the network adapter has a default gateway and Outlook continues to pretend that it cannot find the Exchange server.

The connection to Microsoft Exchange is unavailable. Outlook must be
  online or connected to complete this action.

This happens regardless of whether I use the Exchange server's name ("exchange") or its (fixed) IP address or "localhost" or 127.0.0.1.
What am I doing wrong?
Update:
After that error message, this small window appears with a slightly different error message.

For some reason Outlook cannot "resolve" the name 127.0.0.1. The same message appears when I try any other name or IP address that points to this same local server including localhost and the server's name and the server's IP address in 192.168.0.0/24.
Comment: Outlook installed on the Exchange Server? Im not sure about 2013, but that used to be a worse-practice in previous versions.
Comment: Is there any other way to test an Exchange installation other than with Outlook? I am thinking this is more of an Outlook issue.
Comment: Same thing happens with Outlook on another machine in the same domain, then with both IP address and name of the Exchange server.
Comment: I always find Outlook is happier if you use the proper FQDN for the exchange server (e.g. exchange.ad.mycompany.com). Make sure the machine can ping it first. If the client (especially if it's the server itself) can't resolve the FQDN you've got more basic problems). Always amazes me how complex and awkward Microsoft's protocols are.
Comment: I can ping the Exchange server using IP address, host name and FQDN. But Outlook still claims it cannot resolve the name.
Answer: Found it! It was so easy...
This site had the answer.
http://www.msoutlook.info/question/531
<code>Key: HKEY_CURRENT_USER\Software\Microsoft\Exchange\Exchange Provider
Value name: DS Server
Value type: REG_SZ
Value: FQDN of the GC server
</code>
Apparently the global catalog server (I have only one domain controller for the root domain and it is running as it is also the DNS) was missing.
Once I added its name to the "DS Server" value, the connection works!
But why Outlook needs the global catalog server to resolve a DNS name when DNS is working I don't know.
Comment: And why Outlook cannot find the global catalog server without knowing its name is a bit of a riddle too.
Answer: first of all, are you using a proxy server in IE? Those tend to mess things up quite a bit. Secondly, could you tell me what kind of output you get from right clicking the outlook icon in the task tray (next to the clock :)) and performing a "test e-mail autoconfiguration"? Uncheck both guesssmart boxes...
This will try and connect to the autodiscover website (localhost/autodiscover/autodiscover.xml) to get the URLS for the services.
Comment: No, there is no proxy server configured. In fact nothing is configured. The server doesn't even have Internet access. There is no Outlook icon in the tray. When I start Outlook it asks me to configure a profile and that's the part that doesn't work as I explained above.
Answer: Exchange 2013 uses RPC over http rather than MAPI, on "Connection Settings" try setting up the RPC settings.
Comment: What "Connection Settings"?
Comment: I found a "More Settings" button and a Connection tab. There is an option "Connect to Microsoft Exchange using HTTP" under "Outlook Anywhere". I activated it but nothing changes I am still getting the same name resolution error message.
Answer: In my case I was getting following message while setting up Outlook 2010 for Exchange 2013.
The action cannot be completed. The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action.
The issue was because of Proxy Settings in the "Internet Options". The Proxy Server was automatically pointing to 127.0.0.1. Disabling Proxy Server in Registry did not help. It still kept automatically pointing to same IP.
The solution was to add Exception in the Proxy Settings:
Open "Internet Options" from control panel or IE.
Click "Advanced".
Click on "LAN Settings".
Click "Advanced" under Proxy Server.
In the "Exceptions" box, add the mail server. for example "mail.yourdomain.com"
Then Outlook worked!
Answer: Use your active directory dns domain.
Comment: More detail would make this a better answer.


Is there a proof for the maximum principle without the Cauchy integral theorem?

Question: All the theorems about holomorphic functions seem to rely on the Cauchy integral theorem: Liouvilles theorem about bounded whole functions, the maximum principle, the open mapping theorem for holomorphic functions, Riemanns theorem about removabe singularities, etc. I see that the Cauchy integral theorem is powerful but are there other ways to prove one of these theorems that do not directly or implicitely use the Cauchy integral theorem?
Comment: Holomorphic functions are harmonic, and the maximum principle for real-valued harmonic functions implies the maximum modulus principle for holomorphic functions. But the usual way via the Cauchy theorems is easier, IMO.
Comment: @DanielFischer That's an answer :-)
Comment: @DanielFischer  I think to get that they are harmonic, we need at least that they are second differentiable, whose proof passes through the cauchy integral theorem.
Comment: @StevenGubkin Yes! If you already knew that holomorphic $\Leftrightarrow$ analytic, the rest would be no problem!
Comment: @StevenGubkin Depends on what you want to throw at it. From the Cauchy-Riemann equations it follows that they are harmonic in the sense of distributions. And we have regularity theorems, harmonic distributions are in fact $C^\infty$. _Checks_: Yes, I did say the usual way was easier ;)
Comment: I heard that there was some sort of a dispute between Liouville and Cauchy. It sounded like Liouville proved his theorem without Cauchys integral theorem. How would you do that?
Comment: Also, I read [here](http://mathoverflow.net/questions/116896/liouvilles-theorem-with-your-bare-hands) that you can deduce Liouvilles theorem from Riemanns theorem for removable singularities. But I don't know how to prove _that_ without Cauchy!
Comment: I don't know how Liouville proved it, but the theorem also holds for harmonic functions, a bounded entire (defined on all of $\mathbb{R}^n$) harmonic function is constant (follows from the Poisson integral). Also you have a removable singularity theorem for harmonic functions (in real dimensions two or larger), so you can do a lot with the theory of harmonic functions. But, once again, the route with the Cauchy theorems is easier and more instructive.
Comment: The Poisson integral representation for a harmonic function on a disk is all you need to know. Historically, this representation came well before the Cauchy integral representation (1823). I think that Cauchy started with this representation in the process of coming to his representation, but I may be wrong about that.
Answer: Alternatively there's a topological approach, all the results rely in a topological index which is used to obtain the winding number of a curve around a point.
The reference is Topological Analysis by Gordon Thomas Whyburn.


Can you break up a pairwise comparison into components, so that individual participants don't have to assess every possible comparison?

Question: I'm trying to do a pairwise comparison for 40 items. However, this results in $\frac{40 \times 39}{2} = 780$ combinations. Obviously no-one is going to sit through a survey that long. Is there a way of breaking down a pairwise comparison so that participants each only assess a subset of these possible comparisons?
For example, if I had 3 things being rated, this would give the following comparisons:

1 with 2
1 with 3
2 with 3

Could this be broken down between three participants $P_i$ so that....

$P_1$ assesses 1,2 and 1,3
$P_2$ assesses 2,3 and 1,2
$P_3$ assesses 1,3 and 2,3

or is this a statistical no-no?
Comment: I have attempted to clarify some of the wording and formatting. Feel free to revert any changes you don't agree with.
Comment: Are the pairwise comparisons just a correlation between, e.g., all of the ratings given to item 1 and all of those given to item 2? Is it a single rating that each person gives each item, or do they rate them on more than one scale?
Answer: What you've outlined can be described as a "full factorial" experimental design where participants are exposed to all of the possible combinations of the choice set. Obviously, this is an impossibly time-consuming research task, even for a relatively small set of options. I'm not an expert in this methodology, but "fractional factorial" designs offer one way out of the mess. This Wiki article describes their logic ...
https://en.wikipedia.org/wiki/Fractional_factorial_design


Laravel elixir calling task according to watch expression

Question: Say we have this (simplified) elixir in gulpfile.js:
mix
.bower()
.less('test.less');

The problem is then whenever any file is changed, both bower and less tasks are executed.
Is there a way to only execute less when a file in a certain watch path has changed and only execute bower when for example the bower.json file has changed?
The real problem is that with other tasks, my watch process is becoming extremely slow and bower is running everytime I update a less file (which is irrelevant).
Thanks!
Answer: As Jeffrey Way explained here
This is indeed a problem with the bower task. So I'm assuming it's also the case with imagemin.
I'll create a new task and open source it soon and update this answer.
Answer: You can define another task handler to run the bower related tasks. The author in the following blog post have illustrated how you can do that. http://laravelcoding.com/blog/laravel-5-beauty-using-bower#09-gulp-bs
Personally I copy the required files from bower_components to assets directory with copy task, since i want my assets directory to have only the ones that I use.
So I have my definitions like
mix.copy('bower_components/bootstrap-datepicker/dist/js/bootstrap-datepicker.min.js','public/assets/js/bootstrap-datepicker.min.js');
mix.copy('bower_components/bootstrap-datepicker/dist/css/bootstrap-datepicker.min.css','public/assets/css/bootstrap-datepicker.min.css');

Comment: Yeah but bower here is just an example.. if you had imagemin, it would do the same issue. It doesn't make sense to fire `bower` and `imagemin` everytime you change your `less` files. That's why I wanted to separate them
Comment: Well I know your problem here is those tasks running in a situation which is not mandatory, and it seems not only one tasks all tasks get fired on a change ha? Check if its same after running npm update
Comment: if those tasks are one time only, i would still recommend separating the tasks from elixir task.
Comment: yes but I'm still looking for an automated solution.. when you're working in a team you want these to be automated.. bower.json is changed? okay recompile vendor.js and vendor.css (for example)


How to pronounce "$\;\setminus\;$" (the symbol for set difference)

Question: A question for English speakers. When using (or reading) the symbol $\setminus$ to denote set difference —
$$A\setminus B=\{x\in A|x\notin B\}$$
— how do you pronounce it?
If you please, indicate in a comment on your answer what region you're from (what dialect you have).

This is a poll question. Please do not repeat answers! Rather, upvote an answer if you pronounce the symbol the same way the answerer does, and downvote it not at all. Please don't upvote answers for other reasons. Thanks!
Comment: I pronounce it "$\setminus$" of course. How else would I pronounce it?
Comment: How do *you* pronounce it?
Comment: @lhf, I upvoted [the relevant answer](http://math.stackexchange.com/a/114491). `:-)`
Comment: I tend to pronounce it "$A$ delete $B$."
Answer: $A$ minus $B$ seems to be the natural way.
Comment: This is how I say it, and I am a native speaker. This is also why the symbol is named "setminus".
Comment: I'm not a native English speaker.
Comment: As an English speaking programmer (not mathematician) that's how I'd say this
Answer: I usually say "A without B," but it depends on my mood that day
Comment: If someone said this out loud, without a visual cue, I would have no idea what it means.
Comment: @Carl even within the context of (mathematical) sets? Spoken language is *always* context sensitive and once you have the context I find this entirely understandable and natural.
Comment: @Konrad Rudolph: yes, even in the context of mathematics I would not recognize "without" to refer to a relative complement unless someone explicitly told me that it did.
Comment: This is the one I use. (US Midwest)
Answer: No one's mentioned "$A$ take away $B$" yet.
Comment: But is that the way you in practice pronounce it?
Comment: @Mitch Usually, yes. Especially   when the sets in question are collections of generic objects, such as people, chairs, etc...
Comment: That doesn't sound very mathematical. +1
Answer: I pronounce $A \setminus B$ just as the latex code: $A$ setminus $B$.
Comment: Reading letters with their latex command names usually puts me off... To hear someone seriously saying *varepsilon* makes me distrust him!
Comment: I should perhaps stop writing this as "A \backslash B"...
Answer: In situations such as the theory of automorphic forms/functions, the expression $A\backslash B$ means "$B$ left mod $A$", not set-complement. It is pronounced that way: "B left-mod A".
This is necessary because there are quotients on the right also, which interact with these. It amazes me how much trouble it is to get students to "recover from" thinking that "backslash" could only mean "set complement". I say to them "why not just write '-' for set complement?".
The literature on automorphic what's-its is full of left quotients, notated $A\backslash B$.
Answer: Complement of B in A  for A-B  or sometime A difference B
This is more self explaining way to say.. I feel!
Comment: This is what seems to me the right way!
Answer: A less B.............................
Comment: This is what I was taught (in Ireland).
Answer: I say any of the following (it varies):
"A not B"
"A slash B"
and when I am trying to be clear
"The complement of B in A"
Answer: Prologue: I usually try NOT to pronounce math symbols or expressions or say it loud in mind as (claimed in folklore) that internal reading seems to hinder 'speed reading' or comprehension. (Note: I am hugely influenced by Eastern arts so may be there is some truth to that. (Meta-note: Feel free to comment on it if you disagree))
Vote status: Upvoted for making me look up some links.
Answer:

Pronunciation guide for mathematical notation
Blog on "Pronunciation Guides for Mathematical Notation, Expressions, and Greek Letters"
Answer: I say "A slash B".${}{}{}{}{}{}{}{}$
Comment: I guess there's not too much confusion between this and $A/B$, A modulo B :P


Node.js, Express.js - unexpected token {

Question: My app crashes every time it reaches this line:
const {name, price} = req.query;
        ^

can't seem to locate the exact answer..here is the error log
SyntaxError: Unexpected token {
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:373:25)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:140:18)
    at node.js:1043:3

context:
app.get('/products/add' , (req, res) => {
  const {name, price} = req.query;

  const INSERT_PRODUCTS_QUERY = `INSERT INTO products (name, price) VALUES ('${ name }', ${ price })`;
  connection.query(INSERT_PRODUCTS_QUERY, (err,results) => {
      if(err)
      {
        return res.send(err);
      }
      else
      {
        return res.send('succesfully added product');
      }
  });
});

Comment: What version of node?
Comment: node version: 4.8.6
Answer: According to node.green, the object destructuring with primitives syntax works after Node.JS v6.4.0, and throws the Unexpected Token { on Node.js versions below that.
Also, the object rest/spread properties only works out of the box from Node v8.6.0. It works in v8.2.1 with the --harmony flag, and throws the Unexpected Token ... on Node.js versions below that.
Answer: You tried to use destructuring assignment. AFAIK its support by nodejs v.6+ from a box and from 4.2.2 with flag --harmony_destructuring


regex to filter all but whitelisted characters from a multi-language string

Question: I am trying to cleanup a string coming from a search box on a multi-language site.
Normally I would use a regex like:
<code>$allowed = "-+?!,.;:\w\s";
$txt_search = preg_replace("/[^" . $allowed . "]?(.*?)[^" . $allowed . "]?/iu", "$1", $_GET['txt_search']);
</code>
and that works fine for English texts.
However, now I need to do the same when the texts entered can be in any language (Russian now, Chinese in the future).
How can I clean up the string while preserving "normal texts" in the original language?
I though about switching to a blacklist (although I´d rather not...) but at this moment the regex just completely destroys all original input.
Answer: you can use \p{LN} <code>[\p{L}\p{N}]</code> instead of \w , see http://www.php.net/manual/en/regexp.reference.unicode.php
Comment: Thanks. I keep getting *invalid UTF-8 string at offset 5*. Is it possible that PCRE was not compiled with unicode properties and can I check that? At the moment I´m just testing on a ubuntu 9.10 desktop server.
Comment: I cannot get it to work but I think I´ll try it on another server, perhaps it´s a local problem.
Answer: It is common problem, that russian letters not recognised by \w pattern, so you can use
$allowed = "-+?!,.;:\w\sа-я";
Comment: Won't [а-я] be expanded to their UTF-8 numbers, so [50-60000] (numbers not accurate)? That would be slow for regex, I think?
Comment: Can you use a character class inside a character class? This would translate to [^-+?!,.;:\w\s[а-я]]
Comment: Jeroen,
I can, but you right, there is no "subclass", so it adds "[" and "]" symbols. Answer fixed :)
Comment: Hmmmm, that does not work, if I use that I get a warning: "Message: preg_replace() [function.preg-replace]: Compilation failed: invalid UTF-8 string at offset 5".
Comment: I think, that you saved a file not in utf8, but in something like cp1251 encoding.
Comment: Thanks for your help. It´s just a form that is getting sent in, how can I change it´s encoding?
Comment: You can change file encoding with iconv from unix shell (I assume, that it is standard windows 1251):
iconv -f cp1251 -t utf-8 < file.php > new_file.php; mv new_file.php file.php
Comment: I cannot get it to work but I think I´ll try it on another server, perhaps it´s a local problem.


What is the distance allowed for the goalkeeper to be off the line during penalty kick?

Question: Recently, I saw a video showing a penalty kick replayed twice because the goalkeeper advanced from the goal line.
My question is: what is the distance allowed for the goalkeeper to be off the line during a penalty kick? And generally, have these kind of incidences happened before?
Comment: Since 2016, [this offence is now a mandatory caution (yellow card)](http://theifab.com/laws/the-penalty-kick/chapters/penalty-kick-offences-and-sanctions). Since referees don't like giving soft cautions, you can now expect the enforcement of this rule to be even more lax than it already was.
Answer: According to the IFAB Laws of the Game, Law 14 - The Penalty Kick, 1. Procedure:

The defending goalkeeper must remain on the goal line, facing the
  kicker, between the goalposts until the ball has been kicked.

These kind of incidences actually happen quite rarely, the famous one being Bill Hamid of D.C. United saving 2 attempts from penalty kick in the game against Portland Timbers but both being overruled by the referee. Here's the video.


Tem um jeito mais reduzido para a resolução da questão abaixo? e saber se está correto

Question:
Crie um algoritmo que contenha um vetor de inteiros. Depois, seu algoritmo deve gerar mais dois vetores. O primeiro vetor deve armazenar os números do vetor inicial em ordem crescente e o segundo deve armazenar os valores  do vetor inicial em ordem decrescente.

código:
    #include <iostream>

      using namespace std;

      int main()
       {
    int vetor[5] = {8, 2, 7, 3, 6};
    int cresc[5] = {vetor[0], vetor[1], vetor[2], vetor[3], vetor[4]};
    int decre[5] = {vetor[0], vetor[1], vetor[2], vetor[3], vetor[4]};

        for(int i = 0; i < 5; i++){
            for(int j = 0; j < (5 - 1); j++){
                if(cresc[i] < cresc[j]){
                    int aux = cresc[i];
                    cresc[i] = cresc[j];
                    cresc[j] = aux;
                }
            }
        }
        for(int a = 0; a < 5; a++){
            for(int b = 0; b < (5 - 1); b++){
                if(decre[a] > decre[b]){
                    int aux2 = decre[b];
                    decre[b] = decre[a];
                    decre[a] = aux2;
                }
            }
        }

        for(int x = 0; x < 5; x++){
            cout << "vetor crescente[" << x << "]: " << cresc[x] << "" << endl;
        }
        cout << "--------------" << endl;
        for(int z = 0; z < 5; z++){
            cout << "vetor decrescente[" << z << "]:" << decre[z] << endl;
        }
      return 0;
     }

Comment: Precisa ver se tem algum enunciado que exija alguma coisa, tem algumas poucas que diminui um pouco, mas pode ser que a intenção não seja fazer assim.
Comment: Cuidado, c e c++ são linguagens completamente diferentes. Escrever em uma linguagem não quer dizer que vá funcionar na outra. A propósito, o seu programa não funciona com um compilador C
Answer: A sua resolução está correta sim, no entanto consegue reduzir, optimizar e melhorar um pouco:

Pode tornar os arrays sem ser de tamanho fixo, utilizando uma
variável ou uma constante.
Não precisa de fazer duas ordenações pois a ordenação crescente é inversa a decrescente. Com isto pode após a primeira ordenação copiar invertido para a segunda.
Usando até um algoritmo de ordenação semelhante, o insertion sort, pode construir ambos os arrays ao mesmo tempo.

Aplicando estas modificações ficaria assim:
#include <iostream>

using namespace std;

#define TAM 5 //constante para o tamanho aqui

int main()
{
    //agora cria os arrays com base num tamanho
    int vetor[TAM] = {8, 2, 7, 3, 6};
    int cresc[TAM], decre[TAM];

    //agora tem de se copiar do vetor para o cresc pois o tamanho pode variar
    for (int i = 0; i < TAM; ++i){
        cresc[i] = vetor[i];
    }

    for(int i = 0; i < TAM; i++)
    {
        for(int j = i + 1; j < TAM; j++)
        {
            if(cresc[j] < cresc[i])
            {
                int aux = cresc[i];
                cresc[i] = cresc[j];
                cresc[j] = aux;
            }
        }

        //ao mesmo tempo que constroi o cresc colocando os mais pequenos na parte inicial
        //coloca também os maiores na parte final do decre
        decre[TAM-i-1] = cresc[i];
    }

    //resto das escritas dos arrays igual
    ...

    return 0;
}

Poderia naturalmente utilizar algoritmos de ordenação mais eficientes como o quicksort ou mergesort que lhe garantem complexidade de O(nlogn) mas já são um pouco mais complexos e provavelmente não farão parte das soluções que esperam para o exercício.
Answer: Bom, como vc está usando a tag c++, pq não resolver o problema usando o que se tem de melhor no C++?
Para o vetor, use o std::vector. Para manusear os dados, o STL tem vários algoritmos que vc pode usar, como por exemplo, o std::sort. Abaixo tem um exemplo no qual faz o que vc pede:
#include <vector>
#include <algorithm>
#include <iostream>

template <typename T>
void imprime(const T& v)
{
    for(auto&& i : v)
        std::cout << i << " ";
    std::cout << std::endl;
}

int main()
{
    auto vetor = std::vector<int>({8, 2, 7, 3, 6});

    auto cresc = vetor;
    std::sort(cresc.begin(), cresc.end());

    auto decre = decltype(cresc)(cresc.rbegin(), cresc.rend());

    imprime(vetor);
    imprime(cresc);
    imprime(decre);
    return 0;
}

Veja funcionando no coliru
PS. Tem que se usar a versão C++11 ou superior
Answer: Em C++98 você pode fazer algo como:
#include <iostream>
#include <algorithm>
#include <vector>

using namespace std;

int nums[] = { 8, 2, 7, 3, 6 };

void exibir( vector<int> & v )
{
    for( vector<int>::iterator it = v.begin(); it != v.end(); ++it )
        cout << *it << " ";

    cout << endl;
}

int main( void )
{
    vector<int> vetor( nums, nums + (sizeof(nums)/sizeof(nums[0])) );

    vector<int> cresc( vetor );
    sort( cresc.begin(), cresc.end() );

    vector<int> decre( cresc );
    reverse( decre.begin(), decre.end() );

    exibir( vetor );
    exibir( cresc );
    exibir( decre );

    return 0;
}

Answer: Solução em C:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

int qsort_crescent( const void * a, const void * b ) { return *((int*)a) - *((int*)b); }
int qsort_decrescent( const void * a, const void * b ) { return *((int*)b) - *((int*)a); }

void exibir( int array[], int tam )
{
    int i = 0;

    for( i = 0; i < tam; i++ )
        printf( "%d ", array[i] );

    printf("\n");
}

int main( int argc, char * argv[] )
{
    int vetor[5] = { 8, 2, 7, 3, 6 };

    int cresc[5];
    int decre[5];

    memcpy( cresc, vetor, sizeof(vetor) );
    qsort( cresc, 5, sizeof(int), qsort_crescent );

    memcpy( decre, vetor, sizeof(vetor) );
    qsort( decre, 5, sizeof(int), qsort_decrescent );

    exibir( vetor, 5 );
    exibir( cresc, 5 );
    exibir( decre, 5 );

    return 0;
}


How do I retain user input in a form after submission?

Question: I am working on a simple form for searching a collection of marriage licenses. It consists of a select box where the user may choose to search last names, first names, or both, and a text field where they type the name they're searching for.
Everything is working fine so far, except that the user input is not retained after the form is submitted.  For example, if I choose "Last names" and enter "Olsen", after the form is submitted the search box will be empty and the select box will revert to its default of "Both first and last names".
Here is my buildForm() method:
public function buildForm(array $form, FormStateInterface $form_state){
    // The select box.
    $form['target'] = [
        '#title' => $this->t('Search'),
        '#type' => 'select',
        '#options' => [
            'both' => $this->t('Both first and last names'),
            'last' => $this->t('Last names'),
            'first' => $this->t('First names'),
        ],
        '#default_value' => $form_state->getValue('target'),
    ];

    // The search box.
    $form['q'] = [
        '#title' => $this->t('for'),
        '#type' => 'textfield',
        '#default_value' => $form_state->getValue('q'),
    ];

    $form['actions'] = ['#type' => 'actions'];

    $form['actions']['submit'] = [
        '#type' => 'submit',
        '#value' => $this->t('Go'),
    ];

    return $form;
}

As you can see, I've tried using #default_value to set a default value by calling the form state to get whatever the user picked/entered.  But alas, this does not seem to have done it.
How do I get it to display the user input in the form after submission?
Comment: what happens in your `submitForm()` ?
Comment: Currently, nothing; it's a stub containing a comment that says "Do the search and return the results".  I hadn't gotten that far yet. Is this something that gets set in that stage?
Answer: Try adding $form_state->setRebuild(TRUE); to your submitForm().  Quoting from here:

Normally, after the entire form processing is completed and submit handlers have run, a form is considered to be done and \Drupal\Core\Form\FormSubmitterInterface::redirectForm() will redirect the user to a new page using a GET request (so a browser refresh does not re-submit the form). However, if 'rebuild' has been set to TRUE, then a new copy of the form is immediately built and sent to the browser, instead of a redirect.

I think that as you have stubbed a submit handler, but not implemented anything (and so are presumably not calling parent::submitForm either), it is not considered "done", so it is not redirecting; but it is not rebuilding based on the new $form_state either, because you haven't told it to.  So you end up just seeing the original form.
As well as setting rebuild to true you may also need to $form_state->disableRedirect(TRUE) to get the desired behaviour of staying on the form, but with inputted values showing.
Comment: I added calls to `disableRedirect(true)` and `setRebuild(true)` on the `$form_state` in my `submitForm()`, and it now remembers the user's input correctly.

Thank you!


How to fixe repeated callbacks in jQuery

Question: I have a problem with my jQuery code, that happens when I scroll to bottom to load more content.
My code:
<code>$(window).scroll(function(){
if ($(window).scrollTop() >= ($(document).height() - $(window).height())*0.9)
{
MyFonction();
}
});
</code>
When I scroll past the 90% mark of the page, I get repeated callbacks from MyFonction -- it is called multiple times.
Is there is a solution to make only one callback per scrolling?
Answer: I think you are looking for the debounce and throttle functions - two similar (but different!) techniques to control how many times we allow a function to be executed.
The Debounce technique allow us to "group" multiple sequential calls in a single one.
The Throttle technique only allows our function to execute once every X milliseconds.
Here are some very good articles and examples:
How to change text element based on input text field in HTML using Javascript?
https://remysharp.com/2010/07/21/throttling-function-calls
https://css-tricks.com/debouncing-throttling-explained-examples/
https://davidwalsh.name/javascript-debounce-function

Another idea, that might be more suitable to this inquiry, is to also use a parent-scope variable to say whether you've scrolled past the 90% marker or not. Even using a throttler/debouncer, you will still get additional calls if the user stops and then starts again.
As a general rule, we usually use a debouncer whenever querying the <code>.scroll()</code> because there are so many events generated - but at 90%, you want to do something and not receive any further notifications until the next marker. So, use a variable for each marker and only check the scroll for that marker when that marker's variable is turned on.
Comment: can you please give me an example
Comment: Just use an ordinary variable - global vars if you must - that indicate if you are past 70% (turn on the check_90pct_var) and if it passes 90% then load in the new data and turn it off again, and have another one for the next point, etc. Old skool but it works. Dont forget about window.varname vars if you need them.


Mismanagement of dask future results slow down the performances

Question: I'm looking for any suggestion on how to solve the bottleneck below described.
Within a dask distributed infrastructure I map some futures and gain results whenever they are ready.  Once retrieved I've to invoke a time consuming, blocking "pandas" function and, unfortunately, this function can't be avoided.
The optimum would be to have something that let me create another process, detached from the for loop, that's able to ingest the flow of results. For other constraints, not present in the example, the output can't be serialized and sent to workers and must be processed on the master.
here a small mockup. Just grab the idea and not focus too much on the details of the code.
class pxldrl(object):
    def __init__(self, df):
        self.table = df

def simulation(list_param):
    time.sleep(random.random())
    val = sum(list_param)/4
    if val < 0.5:
        result = {'param_e': val}
    else:
        result = {'param_f': val}
    return pxldrl(result)

def costly_function(result, output):
    time.sleep(1)
    # blocking pandas function
    output = output.append(result.table, sort=False, ignore_index=True)

    return output

def main():
    client = Client(n_workers=4, threads_per_worker=1)

    output = pd.DataFrame(columns=['param_e', 'param_f'])

    input = pd.DataFrame(np.random.random(size=(100, 4)),
                                columns=['param_a', 'param_b', 'param_c', 'param_d'])

    for i in range(2):

        futures = client.map(simulation, input.values)

        for future, result in as_completed(futures, with_results=True):
            output = costly_function(result, output)

Answer: It sounds like you want to run costly_function in a separate thread.  Perhaps you could using the threading or concurrent.futures module to run your entire routine on a separate thread?
If you wanted to get fancy, you could even use Dask again and create a second client that ran within this process:
local_client = Client(processes=False)

and use that.  (although you'll have to be careful about mixing futures between clients, which won't work)
Comment: To speed up the system the idea would be to process the costly_function in a way that the client.map would be free to go ahead with the second loop.
I tried, without much success, to implement a second async Streamz process and that's why I'm looking for alternatives. Another attempt I did was to create an async process through the 'aiomultiprocess'. Unfortunately isn't so easy for me to deeply understand how to make all this process asynchronously collaborative.
Comment: how about a sort of cache that is then processed through a Fire and Forget on a local_client as you suggested? that would free the loop to the next one? indeed the cache needed to be passed as a deepcopy otherwise it would be compromised from the next loop.


Como enviar formulário sem refresh da página?

Question: Queria saber como que eu envio formulário sem refresh, eu consegui colocar para não dar refresh mas eu quero que apareça uma tela logo após dar o submit tipo "E-mail enviado com sucesso".
Answer: <!doctype html>

<meta charset="utf-8">
<title>Documento sem título</title>

<form id="forms">
<input type="text" name="name" placeholder="digite o nome" />
<input type="submit" value="enviar" />
</form>

enviado com sucesso!

<input type="button" value="download">

agora a pagina que recebe os dados
dados.php
<?php

$dados = $_POST['name'];

?>

Answer: Tente fazer dessa forma utilizando jquery e ajax:

</script>