Competing risk regression with CRR slow on large datasets?

Question: I'm doing a registry based study with almost 200 000 observations and I want to perform a competing risk analysis. My problem is that the crr() in the cmprsk package is exponentially increasing with increasing number of observations. I therefore wrote a simulation for trying different approaches; check how factors, data frames and matrixes affect the performance so that I could choose the most efficient combination.
I have a 1 year old computer with 8 Gb of RAM and still it didn't finish 70 000 observations when I left the computer overnight.
My main questions:

Is there a faster way of performing competing risk analysis?
Why does Win7 4 times perform better? Is it the 64-bit version that improves the performance?
Can I do something to speed things up?
Is the simulation similarly slow on your computer? (see simulation code at the end)

Thanks
Max

My output with some of the errors that some of the alternatives cause
* Start crr performance check *
Using 10000 observations and 3 - 10000 covariates
..
Error: NA/NaN/Inf in foreign function call (arg 4)
In addition: Warning message:
NAs introduced by coercion
...
Error in drop(.Call("La_dgesv", a, as.matrix(b), tol, PACKAGE = "base")) :
  Lapack routine dgesv: system is exactly singular
..

The output in Linux
- Linux – Ubuntu – R32-bit 2.13.1
 *** Test results ***

Test 1 passed - using no factors
It took: 4.2 minutes

Test 2 passed - using factored status
Benchmark test 2: 91 %

Test 3 failed - factored covariates fail

Test 4 passed - using factored status & data frame covariates
Benchmark test 4: 78 %

Test 5 passed - using factored status & twice the covariates
                where the new covariates differ
Benchmark test 5: 76 %

Test 6 failed - exact copies of the covariates fail

Test 7 passed - using regular status & twice the covariates
                where the new covariates differ
Benchmark test 7: 80 %

Test 8 passed - using regular status & thrice the covariates
                where the new covariates differ
Benchmark test 8: 83 %

The output in Windows
- Windows 7 – R64-bit 2.13.1

 *** Test results ***

 *** Test results ***

Test 1 passed - using no factors
It took: 1.1 minutes

Test 2 passed - using factored status
Benchmark test 2: 99 %

Test 3 failed - factored covariates fail

Test 4 passed - using factored status & data frame covariates
Benchmark test 4: 104 %

Test 5 passed - using factored status & twice the covariates
                where the new covariates differ
Benchmark test 5: 122 %

Test 6 failed - exact copies of the covariates fail

Test 7 passed - using regular status & twice the covariates
                where the new covariates differ
Benchmark test 7: 122 %

Test 8 passed - using regular status & thrice the covariates
                where the new covariates differ
Benchmark test 8: 144 %

Test with 20 000 (twice the data) on Windows 7 R64 2.13.1 (removed the failures)
*** Test results ***
Test 1 passed - using no factors
It took: 8.5 minutes

Test 2 passed - using factored status
Benchmark test 2: 100 %

Test 4 passed - using factored status & data frame covariates
Benchmark test 4: 100 %

Test 5 passed - using factored status & twice the covariates
                where the new covariates differ
Benchmark test 5: 123 %

Test 7 passed - using regular status & twice the covariates
                where the new covariates differ
Benchmark test 7: 122 %

Test 8 passed - using regular status & thrice the covariates
                where the new covariates differ
Benchmark test 8: 147 %

The performance test code
library(cmprsk)
# Define a set size
my_set_size <- 10000

# Create the covariables
cov <- cbind(rbinom(my_set_size, 1, .5),
  rbinom(my_set_size, 1, .05),
  rbinom(my_set_size, 1, .1))
dimnames(cov)[[2]] <- c('gender','risk factor 1','risk factor 2')
factored_cov <- data.frame(
  gender = factor(cov[,1],
  levels=c(0,1),
  labels=c("male", "female")),
  rf1 = factor(cov[,2],
  levels=c(0,1),
  labels=c("no", "yes")),
  rf2 = factor(cov[,3],
  levels=c(0,1),
  labels=c("no", "yes")))

framed_cov <- data.frame(
  gender = cov[,1],
  rf1 = cov[,2],
  rf2 = cov[,3])

cov_doubled <- cbind(cov[,1],
  cov[,2],
  cov[,3],
  rbinom(my_set_size, 1, .5),
  rbinom(my_set_size, 1, .05),
  rbinom(my_set_size, 1, .1))
dimnames(cov_doubled)[[2]] <-
  c('gender','risk factor 1','risk factor 2',
    'risk factor 3', 'risk factor 4', 'risk factor 5')

cov_doubled_exact <- cbind(cov[,1],
  cov[,2],
  cov[,3],
  cov[,1],
  cov[,2],
  cov[,3])
dimnames(cov_doubled_exact)[[2]] <-
  c('1: gender','1: risk factor 1','1: risk factor 2',
    '2: gender','2: risk factor 1','2: risk factor 2')

cov_tripled <- cbind(cov[,1],
  cov[,2],
  cov[,3],
  cov_doubled[,4],
  cov_doubled[,5],
  cov_doubled[,6],
  rbinom(my_set_size, 1, .5),
  rbinom(my_set_size, 1, .05),
  rbinom(my_set_size, 1, .1))
dimnames(cov_tripled)[[2]] <-
  c('gender','risk factor 1','risk factor 2',
    'risk factor 3', 'risk factor 4', 'risk factor 5',
    'risk factor 6', 'risk factor 7', 'risk factor 8')

# Create random time to failure/cens periods
ftime <- rexp(my_set_size)

# Create events
my_event1 <- rbinom(my_set_size, 1, .04)
my_event2 <- rbinom(my_set_size, 1, .20)
# The competing event can't happen if 1 has already occurred
my_event2[my_event1 > 0] <- 0
fstatus <- my_event1 + my_event2*2

# Factor the censor variable
factored_status <- factor(fstatus, levels=c(0,1,2), labels=c("censored", "re-operation", "death"))

# Check that it seems Ok
#table(fstatus)

# The Sys.time() is a crude but I found it efficient enough
# for my simple analysis
cat("\n * Start crr performance check *\nUsing", my_set_size, "observations and",
  length(cov[1,]), "-", length(cov_doubled[,1]), "covariates\n")

# Initialize vars for later if() checks
my_test1 <- my_test2 <- my_test3 <- NULL
my_test4 <- my_test5 <- my_test6 <- NULL
my_test7 <-NULL

my_t1.a <- Sys.time()
my_test1 <- crr(ftime, fstatus, cov)
my_t1.b <- Sys.time()
cat(".")
my_t2.a <- Sys.time()
my_test2 <- crr(ftime, factored_status, cov, failcode="re-operation", cencode="censored")
my_t2.b <- Sys.time()
cat(".")
my_t3.a <- Sys.time()
my_test3 <- crr(ftime, factored_status, factored_cov, failcode="re-operation", cencode="censored")
my_t3.b <- Sys.time()
cat(".")
my_t4.a <- Sys.time()
my_test4 <- crr(ftime, factored_status, framed_cov, failcode="re-operation", cencode="censored")
my_t4.b <- Sys.time()
cat(".")
my_t5.a <- Sys.time()
my_test5 <- crr(ftime, factored_status, cov_doubled, failcode="re-operation", cencode="censored")
my_t5.b <- Sys.time()
cat(".")
my_t6.a <- Sys.time()
my_test6 <- crr(ftime, factored_status, cov_doubled_exact, failcode="re-operation", cencode="censored")
my_t6.b <- Sys.time()
cat(".")
my_t7.a <- Sys.time()
my_test7 <- crr(ftime, fstatus, cov_doubled)
my_t7.b <- Sys.time()
cat(".")
my_t8.a <- Sys.time()
my_test8 <- crr(ftime, factored_status, cov_tripled, failcode="re-operation", cencode="censored")
my_t8.b <- Sys.time()
#summary(my_test)

cat("\n *** Test results ***")

# Don't check if test 1 passed since this is the benchmark
cat("\n\nTest 1 passed - using no factors")
cat("\nIt took:", round(difftime(my_t1.b, my_t1.a, units="mins"), 1), "minutes")
benchmark_time <- as.numeric(my_t1.b - my_t1.a)

if (length(my_test2)){
  cat("\n\nTest 2 passed - using factored status")
  bench.t2 <- as.numeric(my_t2.b - my_t2.a)/benchmark_time
  cat("\nBenchmark test 2:", round(bench.t2*100), "%")
}else{
  cat("\n\nTest 2 failed")
}

if (length(my_test3)){
  cat("\n\nTest 3 passed - using factored status & factored covariates")
  bench.t3 <- as.numeric(my_t3.b - my_t3.a)/benchmark_time
  cat("\nBenchmark test 3:", round(bench.t3*100), "%")
}else{
  cat("\n\nTest 3 failed - factored covariates fail")
}

if (length(my_test4)){
  cat("\n\nTest 4 passed - using factored status & data frame covariates")
  bench.t4 <- as.numeric(my_t4.b - my_t4.a)/benchmark_time
  cat("\nBenchmark test 4:", round(bench.t4*100), "%")
}else{
  cat("\n\nTest 4 failed")
}

if (length(my_test5)){
  cat("\n\nTest 5 passed - using factored status & twice the covariates",
  "\n                where the new covariates differ")
  bench.t5 <- as.numeric(my_t5.b - my_t5.a)/benchmark_time
  cat("\nBenchmark test 5:", round(bench.t5*100), "%")
}else{
  cat("\n\nTest 5 failed")
}

if (length(my_test6)){
  cat("\n\nTest 6 passed - using factored status & twice the covariates",
  "\n                where the new covariates are exactly a copy of first covariates")
  bench.t6 <- as.numeric(my_t6.b - my_t6.a)/benchmark_time
  cat("\nBenchmark test 6:", round(bench.t6*100), "%")
}else{
  cat("\n\nTest 6 failed - exact copies of the covariates fail")
}

if (length(my_test7)){
  cat("\n\nTest 7 passed - using regular status & twice the covariates",
  "\n                where the new covariates differ")
  bench.t7 <- as.numeric(my_t7.b - my_t7.a)/benchmark_time
  cat("\nBenchmark test 7:", round(bench.t7*100), "%")
}else{
  cat("\n\nTest 7 failed")
}

if (length(my_test8)){
  cat("\n\nTest 8 passed - using regular status & thrice the covariates",
  "\n                where the new covariates differ")
  bench.t8 <- as.numeric(my_t8.b - my_t8.a)/benchmark_time
  cat("\nBenchmark test 8:", round(bench.t8*100), "%")
}else{
  cat("\n\nTest 8 failed")
}

#summary(my_test1)
#summary(my_test2)
#summary(my_test4)
#summary(my_test5)

Answer: Variance estimation with the R crr() function from the cmprsk package accounts for slowing down the run time by perhaps an order of magnitude. If you call crr() and disable the variance calculation as follows:
crr(ftime, fstatus, cov, variance=FALSE)

then the run time will be dramatically reduced, but the returned model gives only the regression coefficients, and not the se(coef) or the p-values. It turns out that if you are doing stepwise competing risks regression, that you do not need these variance estimates for variable selection, only the log likelihoods, and can perform the variance calculation once at the end when you have finalized the variables in your model.
A single crr() run on ~270k observations without variance calculations can take <30 minutes, versus many hours with the default setting of variance=TRUE. We have been implementing a multicore version of the R crrstep package for stepwise competing risks regression, and discovered this variance bottleneck. We haven't been able to do anything about the slowness of the variance estimate, but have been able to run stepwise competing risks regression in a ~1 day run with ~270k observations and ~100 covariates on a 24-core machine, excluding the final variance estimates.
Some time ago I did some speed tests with the Stata competing risks regression and found it ran about an order of magnitude slower than the R cmprsk implementation. SAS also implements competing risks regression in its PROC PHREG, but I have no experience with its performance.
Answer: This is not an answer in terms of computational speed, but rather a practical solution based on my experience with other types of regression models: Have you tried with a much smaller sample? I'm guessing that even with n=10,000 you might get a decent model (in terms of precision). I'd also try estimating a few such small models and see how widely they vary to better capture precision.
Comment: Thank you for your answer. I guess that's my fall-back strategy, but right now I find it hard to believe that there is a regression that takes that long to compute. I also find it surprisingly hard to find examples on different competing risk analysis. I've read Melanie Pintilie's book on Competing Risks with a few helpful examples but not really the depth I was hoping for. I also stumbled upon Geskus article in Biometrics 2011 where he proposes weighting coxph() data but since I'm an MD I found the article a little hard to penetrate. There is a striking lack of examples...
Answer: Stata's stcrreg uses the method of Fine and Gray (1999), I don't know if CRR uses that methodology or not.  But if it does, there is an alternative methodology which does not estimate subhazard ratios but instead relies on stratifying by failure type and replicating records.  For a description see in Rosthoj S. Andersen P.K., Abildstrom S.Z. (2004) which uses SAS macros; or Putter, Fiocco, and Geskus (2007).  See also Stata's stcompadj.ado.  Interpreting affects is largely done graphically, however.  In Stata, stcrreg (Fine and Gray) is much slower than stcompadj (which uses additive hazards).


Chopin Nocturne in C sharp minor Scales RH and LF correspondence, does one choice make more sense musically than another?

Question: Take the longest scale (measure 56--in this previous discussion some alternatives were given but it does not address my question): The Henle edition I have (shown here) seems to group rather evenly (see blue vertical lines). My question is, would it make more "musical sense" if one plays a D on RH at the same time as a lower D on the LH, and a C# on RH with the C# on the LH, provided that the D or C# are nearby anyway? For example something like the yellow vertical lines? Or maybe it doesn't matter because it's played very fast?
Answer: TL;DR: Scroll down to the final sentence.

You're correct that, played literally and at speed, it's not going to matter. The difference will be so subtle and momentary so as to be indistinguishable. However, the notation is deceiving in that it suggests all 35 notes be played with equal length — a true 35-tuplet — which is not the musical intention.
However, for the purposes of learning the passage, it can be very helpful to have target correspondences, and there is clear logic behind matching like pitch classes (i.e., D to D and C# to C#).
Given the question, let's temporarily assume equal note lengths within each tuplet and look at the musical result.

This strategy gives an alignment against the eighth notes of 9-8-8-10, which is both reasonably balanced and has a slight "rollercoaster" effect: slower coming over the "hump" and then faster on the down-slope.
The example from Henle suggests a grouping of 9-9-8-9, which is more evenly distributed by the numbers, but negligibly different musically.
I tend to play the first 2–4 notes quite slowly, then rush up to the top, and then a ritardando on the way down. So even though the note-to-note distribution might be something like 9-8-8-10, in practice, the notes within those tuplets are not necessarily all the same duration as each other.
Comment: Nobody said the interpretation of Chopin pieces was going to be easy. Playing those two runs correctly and convincing is enough to break you.
Comment: I get it from both answers that unless it's for the purpose of *learning* to play it, it doesn't matter how one groups the RH with respect to LH. Regardless, is there such a thing as Which note "goes with" which note, from the point of view of music theory / harmony? Obviously the same note goes with the same note on different octaves (e.g. D with lower D). Other than the same note, is there another note that "goes with" D, while the rest of the notes do not "go with" D, musically speaking (i.e. don't sound in harmony)?
Comment: @GrandAdagio That depends a bit on the harmonic interpretation. Let's imagine that the passage is played slowly, such that it matters which notes are paired. A straightforward interpretation is that the underlying harmony is the `ii7` chord in C# minor (i.e., `D# half-diminished seventh = D# - F# - A - C#`). In that case, we'd want to match chord tones. By that interpretation, the left hand's `D#` could be paired with the right hand's highest `D#` or the `A` or `C#` leading to it or away from it.
Comment: Fun fact I tried making a guitar arrangement of this to sell on Hall Leonard, but had to give up because I could figure out how to do 34 in the time of 4 in musescore.
Comment: @NeilMeyer FWIW, it's easy to do in MuseScore 4.
Comment: Thanks Aaron, that's exactly what I wanted to know. I'll need to study music theory for chords and harmony. Thanks.
Answer: You don't have to over think it it is the 30 some notes in the time of the four notes in the bass. As long as all those notes are played in the time it takes to play the bass notes the rhythm is correct.
There is a crescendo as the passage ascends and a decrescendo as it descends and all should be played legato. I know this piece well and most interpretations I have heard speeds up and then slows down.
I'm reminded of what I read in the Oxford companion to music donkey years ago. After giving a detailed account of all the irregular note groupings (triplets and the like) it notes than composers are free to group any amount of notes in the time of something else.
Comment: This one by Wladyslaw Szpilman  seems to be starting slowly up then speeding up thereafter: https://www.youtube.com/watch?v=n9oQEa-d5rU  An interesting fact is that two Holocaust survivors both chose this piece when asked by Nazis to play piano for them. Here is Natalia Karp's 1949 recording (can't discern fast/slow in the scale part, all really fast): https://www.youtube.com/watch?v=cQAHnK7QWSI
Comment: @GrandAdagio did Adrian Brody not also play this in the movie The Pianist?
Comment: Neil Meyer, Brody played the role of Wladyslaw Szpilman in the movie and took piano lessons during the filming. From what I've read the movie used the recording of Wladyslaw Szpilman (the "pianist" in real life).


Sending email to hidden field using webform after calling hook_form_alter?

Question: Drupal 7.14, Webform 7.x-3.17 : I built a webform that has a hidden field "lt_email", and I'm assigning a session value to it using hook_form_alter -
function wbl_form_webform_client_form_1_alter(&$form, &$form_state) {
   $form['submitted']['lt_email']['#default_value'] = $_SESSION['Some']['Email'];
}

When I var_dump or dpm the ($form) I see the session value for lt_email, but after I hit submit on the webform, the value goes away, it doesn't get assigned.
Since %session[id] didn't handle multi-dimensional session arrays, I used the hook_form_alter workaround. I was under impression hook_form_alter would grab the session value, assign it to hidden field, and once submitted, the hidden field would retain the value, which I then use as the email component for the webform that gets triggered automatically. THe hidden field is blank upon submit.
I'm logged in as root, and all the other fields retain value upon submit (ie, name, address).
Comment: Try if using `#value` instead of `#default_value` gets you a better result.
Comment: Awesomesauce - that did it, changing #default_value to #value, please add as answer so you may earn proper credit!
Comment: Glad it worked :) Just added the comment at first since I wasn't sure whether it's Form API related or Webform does some weird magic there. Wrote up an answer now with added context that should shed some light on what was probably happening.
Answer: The problem is probably caused by the hidden field having the #value property set which overrides #default_value (even if the former is NULL or a empty string).
To get the desired result you have 2 options:
1) Override #value instead of adding a #default_value:
$form['submitted']['lt_email']['#value'] = $_SESSION['Some']['Email'];

2) Unset #value then add #default_value:
unset($form['submitted']['lt_email']['#value']);
$form['submitted']['lt_email']['#default_value'] = $_SESSION['Some']['Email'];

If you are wondering which option is the right one for you then #default_value means that Drupal will accept client-side changes to the field value (via javascript for example). If you don't need/want that possibility then #value should be used (Drupal will ignore any client-side changes in that case). It's mentioned in the documentation as well.


How to use wsdl2java 3.3.X version of Apache cxf java 11 using command line

Question: We were using Java 8 with apache 2.7.0 to convert wsdl to java and it was working fine. Now we want to upgrade to java 11 then getting error message.
How to use wsdl2java Apache cxf 3.3.X version with java 11 using command line.
Steps tried.
Apache 3.3.0 or Apache 3.3.1 or Apache 3.3.2
When trying to convert using below command
wsdl2java -client -autoNameResolution C:\SAMPLE.wsdl
Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/ws/Service
        at org.apache.cxf.tools.wsdlto.frontend.jaxws.JAXWSContainer.isJaxws22(JAXWSContainer.java:64)
        at org.apache.cxf.tools.wsdlto.frontend.jaxws.JAXWSContainer.getServiceTarget(JAXWSContainer.java:61)
        at org.apache.cxf.tools.wsdlto.frontend.jaxws.JAXWSContainer.validate(JAXWSContainer.java:68)
        at org.apache.cxf.tools.wsdlto.WSDLToJavaContainer.processWsdl(WSDLToJavaContainer.java:172)
        at org.apache.cxf.tools.wsdlto.WSDLToJavaContainer.execute(WSDLToJavaContainer.java:164)
        at org.apache.cxf.tools.wsdlto.WSDLToJavaContainer.execute(WSDLToJavaContainer.java:412)
        at org.apache.cxf.tools.common.toolspec.ToolRunner.runTool(ToolRunner.java:105)
        at org.apache.cxf.tools.wsdlto.WSDLToJava.run(WSDLToJava.java:113)
        at org.apache.cxf.tools.wsdlto.WSDLToJava.run(WSDLToJava.java:86)
        at org.apache.cxf.tools.wsdlto.WSDLToJava.main(WSDLToJava.java:185)
Caused by: java.lang.ClassNotFoundException: javax.xml.ws.Service
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 10 more

Also tried adding external jar file to CFX lib folder.

<dependency>
    <groupId>jakarta.xml.bind</groupId>
    <artifactId>jakarta.xml.bind-api</artifactId>
    <version>2.3.2</version>
</dependency>

<dependency>
    <groupId>org.glassfish.jaxb</groupId>
    <artifactId>jaxb-runtime</artifactId>
    <version>2.3.2</version>
</dependency>

or try this also
<dependencies>
    <dependency>
        <groupId>javax.xml.bind</groupId>
        <artifactId>jaxb-api</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>com.sun.xml.bind</groupId>
        <artifactId>jaxb-impl</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jaxb</groupId>
        <artifactId>jaxb-runtime</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>javax.activation</groupId>
        <artifactId>activation</artifactId>
        <version>1.1.1</version>
    </dependency>
</dependencies>

Java Version
openjdk version "11.0.16" 2022-07-19
OpenJDK Runtime Environment Temurin-11.0.16+8 (build 11.0.16+8)
OpenJDK 64-Bit Server VM Temurin-11.0.16+8 (build 11.0.16+8, mixed mode)

It didn't work with command line, instead I tried with cxf-codegen-plugin in pom.xml
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.cxf</groupId>
            <artifactId>cxf-codegen-plugin</artifactId>
            <version>3.3.0</version>
            <executions>
                <execution>
                    <id>generate-sources</id>
                    <phase>generate-sources</phase>
                    <configuration>
                        <wsdlOptions>
                            <wsdlOption>
                                <wsdl>src/main/resources/GIMA2Java.wsdl</wsdl>
                                <faultSerialVersionUID>1</faultSerialVersionUID>
                                <extraargs>
                                    <extraarg>-autoNameResolution</extraarg>
                                </extraargs>
                            </wsdlOption>
                        </wsdlOptions>
                    </configuration>
                    <goals>
                        <goal>wsdl2java</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

<dependencies>
    <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-frontend-jaxws</artifactId>
        <version>3.3.0</version>
    </dependency>
</dependencies>

Comment: Removed
1. -Djava.endorsed.dirs="${cxf_home}/lib/endorsed
2. set TOOLS_JAR=%JAVA_HOME%\lib\tools.jar;

Got below error but build path has jaxb-runtime-2.3.2 jar files.

 Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/ws/Service
        at org.apache.cxf.tools.wsdlto.frontend.jaxws.JAXWSContainer.isJaxws22(JAXWSContainer.java:64)
Comment: Please [don't upload text as image](https://meta.stackoverflow.com/a/285557/13447). Edit your question to contain all the information in text form - consider to use the editor's formatting options. Also see [ask].
Comment: will not add text image next time.
Comment: You can still edit this one...
Answer: Different versions of Apache CFX 3.3.x not contain all required libraries.
NOTE: These versions are archived and not supported by Apache. I didn't compare libraries for all versions. Probably some other libraries reuqired for these versions.
Missing libraries in Apache CXF 3.3.0

javax.xml.soap-api.1.4.0
jakarta.xml.ws-api.2.3.3

Missing libraries in Apache CXF 3.3.2

jakarta.xml.ws-api.2.3.3

The latest version of Apache CXF on javax.* namespace is 3.5.5.
Command line
Upgrade CXF version to 3.5.5 is strongly recommended.
If it isn't possible, then download mentioned jars under ./lib folder.
Maven usage
Upgrade cxf-codegen-plugin to 3.3.5
Additionally add this dependency to pom.xml
<dependency>
    <groupId>jakarta.xml.ws</groupId>
    <artifactId>jakarta.xml.ws-api</artifactId>
    <version>${jakarta.xml.ws-api.version}</version>
</dependency>

Choosing a proper ${jakarta.xml.ws-api.version} of this dependency is important due to breaking changes about namespace (package).

for javax.* namespace use 2.3.3 (based on the stacktrace this is the right choice in this case)
for jakarta.* namespce use 3.x or 4.0.0
Comment: Thanks for input, great help. From command line got below message
C:\apache-cxf-3.3.5\bin>wsdl2java -client -autoNameResolution C:\HelloWorld.wsdl
'f' is not recognized as an internal or external command,
operable program or batch file.  but it works fine Mavan
Comment: That's a kind of Windows issue. Fortunately, I haven't use Windows. Try to enclose wsdl parh with quotation marks.
Comment: Will it cause any issue if use javax.* namespace use 2.3.3 instead of akarta.* namespce use 3.x or 4.0.0.  Application works fine, but observed one case where its failing- https://stackoverflow.com/questions/76629729/migration-from-java-8-to-java-11-for-soap-web-service       any suggestion


returning multiple attribute values with jquery

Question: I have a web page that allows the user to scroll through answering questions. To show their answer they simply click an li that changes the class onclick via jquery to show that answer has been selected. On some of these questions I only allow a single answer to be chosen (a single li to be clicked) and on others I allow multiple to be clicked. Once the questions are complete the user clicks submit and jquery takes the value of the data-url attributes with the "answerPick" class and appends to the website url for navigating to.
My issue is that the method I use to select the attributes of the "answerPick" class only returns the first value (an issue when ppl choose multiple values).
$(document).ready(function() {

  $('#section-a .answer, #section-b .answer').on('click', function() {
    if ($(this).siblings().hasClass("answerPick") == false) {
      $(this).toggleClass('answer answerPick');
    }

  });

  $('#section-c .answer, #section-d .answer').on('click', function() {
    $(this).toggleClass('answer answerPick');
  });

  $('#submittah').click(function() {
    var q1 = $('#section-a').find('.answerPick').attr('data-url');
    var q2 = $('#section-b  .answerPick').attr('data-url');
    var q3 = $('#section-c  .answerPick').attr('data-url');
    var q4 = $('#section-d  .answerPick').attr('data-url');
    var finalURL = q1 + q2 + q3 + q4;
    window.location.href = finalURL;
  });

});

Here is my jsfiddle https://jsfiddle.net/gchis66/xorpyj82/
Any ideas?
Answer: This won't redirect the page, but will console log out the final query (open developer tools and look at console) or change console.log to alert() if you're so inclined.. Are you wanting to pass the values along with keys to the next page (I've updated the finalURL to contain keys for the value pairs). To answer your question about more than 1 answer, you want to loop through the multiple selected classes, and push them to an array and then join them as a comma delimited list. You may need to encode your URL though when sending over the answers to the location, depending how you are retrieving the key/value parameters from the URL.

$(document).ready(function() {

  $('#section-a .answer, #section-b .answer').on('click', function() {
    if ($(this).siblings().hasClass("answerPick") == false) {
      $(this).toggleClass('answer answerPick');
    }

  });

  $('#section-c .answer, #section-d .answer').on('click', function() {
    $(this).toggleClass('answer answerPick');
  });

  $('#submittah').click(function() {

    var q1 = $('#section-a').find('.answerPick').attr('data-url');
    var q2 = $('#section-b  .answerPick').attr('data-url');
    var q3Multi = [];
    var q3 = $('.answerPick', '#section-c').each(function() {
      q3Multi.push($(this).attr('data-url'));
    });
    var q4Multi = [];
    var q4 = $('#section-d  .answerPick').each(function() {
      q4Multi.push($(this).attr('data-url'));
    });
    var finalURL = '?q1=' + q1 + '&q2=' + q2 + '&q3=' + q3Multi.join(',') + '&q4=' + q4Multi.join(',');
    console.log(finalURL);
    // window.location.href = finalURL;
  });

});
/*  Core Styles */

body {
  margin: 0;
  text-align: center;
  line-height: 1.5;
}

h1,
h2,
h3 {
  padding: 1em 0 !important;
}

.btn {
  padding: 1em 2em;
  color: #000;
  text-decoration: none;
  background: #92c03c;
  border: 1px solid #494949;
  display: inline-block;
  margin: .5em 0;
}

/* Header Styles */

#showcase {
  min-height: 450px;
}

#showcase h1 {
  margin-top: 25px;
}

/* Section Styles */

#main section {
  display: block;
  min-height: 450px;
}

#submission {
  margin-top: 30px;
}

/* List Item Styles */

#main li {
  display: inline-block;
  cursor: pointer;
  text-align: center;
  border: solid 4px #fff;
  width: 140px;
  height: 140px;
  -moz-border-radius: 80px;
  -webkit-border-radius: 80px;
  border-radius: 80px;
  position: relative;
  color: #fff;
}

#main li>span {
  display: flex;
  justify-content: center;
  padding-top: 40%;
}

.answer {
  background: #1bb8d1;
}

.answerPick {
  background: #92c03c;
}

/* Media Queries */

@media(min-width:700px) {
  #showcase h1 {
    font-size: xx-large;
  }
  #showcase p {
    font-size: medium;
  }
  #main h2 {
    font-size: xx-large;
  }
  #main li {
    font-size: large;
  }
  .btn {
    padding: 2em 4em;
    font-size: medium;
  }
}

Example

 Example txt

  Get Started

Is this gift for a woman or man?

Man

Woman

What is the goal of the gift?

Relaxation

Skincare

Date Night

At home spa

Beard Care

Price Point

$5 or lower

$5-$10

$10-$20

$20-$50

Any other preferences?

Cruelty Free

Vegan

Gluten Free

Fragrance Free

        <button id="submittah" class="btn">Submit</button>

Comment: Thanks! A simple solution that works very well. Much appreciated
Answer: use jQuery each function:
http://api.jquery.com/jquery.each/
This will loop all the elements.


Biblioteca math

Question: Galera uma dúvida.
Estou dando uma estudada na biblioteca math. Qual a diferença pratica entre: ceil, floor e trunc??
A idéia é tirar a parte flutuante do número.
Abraço!
Answer: Embora esta resposta seja baseada em Microsoft .net ela está bem didática, traduzida via google translate:

 Siga estes links para obter as descrições do MSDN sobre:

Math.Floor, que arredonda para o infinito negativo.
Math.Ceiling, que  arredonda para o infinito positivo.
Math.Truncate, que arredonda para cima ou para baixo em direção a
  zero.
Math.Round, que arredonda para o inteiro mais próximo ou para o
  número especificado de casas decimais.

Você pode especificar o comportamento se ele for exatamente
  eqüidistante entre duas possibilidades, como arredondamento, de modo
  que o dígito final seja par (" Round(2.5,MidpointRounding.ToEven)" se
  tornando 2) ou que fique mais longe de zero ("
  Round(2.5,MidpointRounding.AwayFromZero)" tornando-se 3).
O diagrama e a tabela a seguir podem ajudar:
-3        -2        -1         0         1         2         3
 +--|------+---------+----|----+--|------+----|----+-------|-+
    a                     b       c           d            e

                       a=-2.7  b=-0.5  c=0.3  d=1.5  e=2.8
                       ======  ======  =====  =====  =====
Floor                    -3      -1      0      1      2
Ceiling                  -2       0      1      2      3
Truncate                 -2       0      0      1      2
Round (ToEven)           -3       0      0      2      3
Round (AwayFromZero)     -3      -1      0      2      3

Note que Round é muito mais poderoso do que parece, simplesmente porque
  pode arredondar para um número específico de casas decimais. Todos os
  outros arredondam para zero decimais sempre. Por exemplo:

n = 3.145;
>     a = System.Math.Round (n, 2, MidpointRounding.ToEven);       // 3.14
>     b = System.Math.Round (n, 2, MidpointRounding.AwayFromZero); // 3.15

Com as outras funções, você tem que usar o truque multiplicar /
  dividir para obter o mesmo efeito:

c = System.Math.Truncate (n * 100) / 100;                    // 3.14
d = System.Math.Ceiling (n * 100) / 100;                     // 3.15

*Tachado foi adicionado para partes não aplicáveis em Python
Aconselho ler esta resposta, mais precisamente na parte de Informações extras
Comment: Que bom. Achei bastante boa sua resposta. E ainda com esse link me ajudará bastante. Obrigado.
Comment: Eu usaria o que está na documentação do Python. Ex: "ceil - retorna o menor inteiro que é maior ou igual a x". Ou seja, `ceil(2.5)` retorna `3`, pois 3 é o menor inteiro que é maior ou igual a 2.5 - essa definição ajuda a entender também porque `ceil(-2.5)` resulta em `-2` :-)
Comment: Poderia coloca link de tirou a informação? +1
Comment: @MauryDeveloper claro! Foi no stackoverflow em inglês [Difference between Math.Floor() and Math.Truncate()](https://stackoverflow.com/questions/14/difference-between-math-floor-and-math-truncate) filtrando as respostas, dá pra fazer um ótimo estudo
Comment: "arredonda para o infinito" ficou uma tradução meio estranha, dá a impressão que o resultado vai dar infinito... "*towards*" seria algo como "em direção a", que é um jeito mais complicado de dizer que arredonda para um valor maior ou menor que o número atual ("em direção ao infinito positivo" quer dizer que o valor será maior que o atual). Enfim, acho que seria melhor indicar a [documentação do Python](https://docs.python.org/3.7/library/math.html), em vez de traduzir outra só para riscar parte do texto. E lá o texto é melhor, ex: ceil - "Return the smallest integer greater than or equal to x"
Comment: @hkotsubo entendo, por isso mencionei via google translate, pois sempre tive dúvidas na hora da tradução,  acredito que muitas vezes é melhor entender e usar o conceito em inglês. Mas uma dúvida, como ficaria a tradução, voltada para programação, de ceil e floor?
Answer: Fábio, todas tratam-se de funções que retornam apenas a parte inteira.

trunc = Retorna apenas a parte inteira do número, ignorando os decimais
ceil = Retorna apenas a parte inteira do número, arredondando para cima
floor = Retorna apenas a parte inteira do número, arredondado para baixo

Na prática, tendo como entrada o valor 1.9:

trunc = Retorna 1, pois ignora a parte decimal
ceil = Retorna 2, pois arredondou para cima
floor = Retorna 1, pois arredondou para baixo

Código:
import math

teste = 1.9

print(math.trunc(teste))
print(math.floor(teste))
print(math.ceil(teste))

Answer: Nesse código mostra os métodos trunc,ceil e o floor:
import math

print(math.floor(23.91));
print(math.ceil(23.91));
print(math.trunc(23.91));

No console mostra o resultado:
23,24,23
Floor: retorna o menor número inteiro dentre o número "x".
Ceil: retorna o menor número inteiro maior ou igual a "x"
Trunc: Remover as casas decimais do valor.
Mais informações:
https://www.geeksforgeeks.org/floor-ceil-function-python/
https://www.geeksforgeeks.org/g-fact-35-truncate-in-python/

Biblioteca math


Hide part of map

Question: Cont. on Distance between two locations

<form action="showmap1.php" method="post">
From: <input type="text" name="origin" id="origin" size="30" />
To:<input type="text" name="destination" id="destination" size="30" />
<input type="button" value="Search" onclick="showMap()" />
<input type="button" value="Reset" onclick="resetMap()" />

</form>

From above code, I have following questions:
(1) JavaScript
    When I run the above code, I get the output like the below image:
The gray part is the map displayed. I want to hide it when I first run the page. How should I modify it?
(2) JavaScript
    After I search a correct result, now I try to search a blank destination. It shows the below image:
However, the previous map and route are show together with alert. My expected output is only show the alert message "Cannot" with no map and route. How should I modify it?
(3) CSS
When I zoom my web page into 120%, my output is like below image:

Oh my god, i can't see the route. It is more even worst if I zoom my web page bigger and bigger. Why and how should I modify it in css?
Answer: (1)
you should put
var mapProp =
{
    zoom: 7,
    center: {lat: 62, lng: -110.0},
    mapTypeId : google.maps.MapTypeId.ROADMAP
};

map = new google.maps.Map(document.getElementById("googleMap"), mapProp);
directionsDisplay.setMap(map);

into showMap() function. Map would not show at the first time load page. It only show when you press Search button.
(2)
If show Cannot alert, hide the map and route as following:
if(status === google.maps.DirectionsStatus.OK)
{
     directionsDisplay.setDirections(response);
}
else
{
    document.getElementById("googleMap").style.display = 'none';
    document.getElementById("textDirection").style.display = 'none';
    window.alert('Cannot');
}

(3)
If you ues float:right, should not fix the width of element. If the width of elements is larger than screen, it will break your layout. I am not good at CSS, you could try the following code. Hope this help ^^
#textDirection{
    width: 40%;
    height: 60%;
    float: right;
    overflow-y: auto;
    margin-top: 1%;
}

#googleMap{
    width : 50%;
    height : 60%;
    top: 5%;
    position : absolute;
    margin-left: 5px;
}

Update:
To hide the map and only show when you need, please using event:
google.maps.event.trigger(map, "resize");

I had update my code, please check this. We only need is set zoom of map smaller to 1 and increase it immediately:
google.maps.event.addListener(map, "idle", function(){
    var center = map.getCenter();
    google.maps.event.trigger(map, "resize");
    map.setCenter(center);
});

map.setZoom( map.getZoom() - 1 );
map.setZoom( map.getZoom() + 1 );

Here is updated answer
Comment: Nice...I have a questions for Q2. I try your fiddle, and I still see the map and route together with alert ("Cannot").
Comment: Nice ^^...But this time the map cannot show the directions from your latest fiddle...
Comment: @KKLMichael: I had updated the answer, hope this help
Comment: Very very nice ^^... You solved my problem... I will understand your code and also learn jQuery as well... Next time if I have a problem I will find you...
Answer: If you're looking to use the google maps API, here is a function to use: Note: you must add &libraries=geometry to your script source
find in Google Maps - How to get the distance between two point in metre?
var p1 = new google.maps.LatLng(45.463688, 9.18814);
var p2 = new google.maps.LatLng(46.0438317, 9.75936230000002);

alert(calcDistance(p1, p2));

//calculates distance between two points in km's
function calcDistance(p1, p2){
  return (google.maps.geometry.spherical.computeDistanceBetween(p1, p2) / 1000).toFixed(2);
}

}

also you can use like
   var location1 = new google.maps.LatLng(42.3584308, -71.0597732);
      var location2 = new google.maps.LatLng(42.348805455204214, -71.07485794349975);

      var map;
      var mapOptions = { center: new google.maps.LatLng(0.0, 0.0), zoom: 2,
        mapTypeId: google.maps.MapTypeId.ROADMAP };

      function initialize() {
        map = new google.maps.Map(document.getElementById("map_canvas"), mapOptions);
      directionsService = new google.maps.DirectionsService();
      directionsDisplay = new google.maps.DirectionsRenderer(
      {
      suppressMarkers: true,
      suppressInfoWindows: true
      });
      directionsDisplay.setMap(map);
      var request = {
      origin: location1,
      destination:location2,
      travelMode: google.maps.DirectionsTravelMode.DRIVING
      };
      directionsService.route(request, function(response, status)
      {
      if (status == google.maps.DirectionsStatus.OK)
      {
      directionsDisplay.setDirections(response);
      //alert("the distance: " + response.routes[0].legs[0].distance.text);
      distance = "The distance between the two points on the chosen route is: "+response.routes[0].legs[0].distance.text;
      //distance += "
The aproximative driving time is: "+response.routes[0].legs[0].duration.text;
      document.getElementById("distance").innerHTML = distance;
      }
      });

      // show a line between the two points
      var line = new google.maps.Polyline({
      map: map,
      path: [location1, location2],
      strokeWeight: 7,
      strokeOpacity: 0.8,
      strokeColor: "#FFAA00"
      });
      }

      google.maps.event.addDomListener(window, 'load', initialize);

Fiddle link http://jsfiddle.net/39KKu/

Hide part of map


Bar Notation Problem

Question: everyone! I came across a problem in math that dealt with bar notation. Does anyone know how, for instance, 1.234(with a bar notation over the 34) is expressed as a fraction? I know already how 1.22(with a bar notation over the 22) is expressed as 1 and 2/9. Any answer would be helpful. Thanks!
Comment: Let $x=1.2\overline{34}$. Then $100x=123.4\overline{34}$. Subtracting the former from the latter gives you
$$100x-x=123.4-1.2=122.2$$ Can you take it from here?
Comment: Hint: Multiply $1.2\overline{34}$ with 100
Comment: @JyrkiLahtonen, "with a bar notation over the 34"
Comment: @JyrkiLahtonen, yes there is no automatic update on comments:)
Comment: @lab: sorry about the hassle here. +1 to your answer.
Answer: HINT:
Let $x=1.2\overline{34}$
$\implies 100x=123.4\overline{34}$
So, $100x-x=123.4\overline{34}-1.2\overline{34}=122.2$

Alternatively, $1.2\overline{34}=1.2+0.01\cdot 0.\overline{34}$
Now, let $S=0.\overline{34}=0.343434\cdots$
$100S=34.3434\cdots$
$\implies 100S-S=34.3434\cdots-0.343434\cdots=34$
$\implies 1.2\overline{34}=1.2+0.01\cdot\frac{34}{99}$
Comment: OK, thank you guys so much for your help! I get it now.
Comment: @user2516595, my pleasure. If there are $n$ digits inside the bar, we need to multiply by $10^n$  where integer $n\ge1$
Comment: wait so the answer cant be 122.2 though can it?
Comment: or would it be 122.2/99
Comment: yup never mind got confsed. i get it
Comment: @user2516595, we have $99x=\frac{1222}{10}=\frac{611}5,x=\frac{611}{5\cdot99}$
Comment: This is safely a comment already. And talk about volume?
Comment: @amWhy, you have probably missed the alternative method
Answer: $1.2\overline{34}= 1\frac{2\frac{34}{99}}{10}=1\frac{232}{990}=1\frac{116}{495}$
Answer: let x=1.23434..... eq.1
so eq.1 mulyiply by 10
10x = 12.3434... eq.2
so eq.2 multiply by 100
1000x =1234.34..... eq.3
subtract eq.2 from eq.3
1000x-10x = 1234.34-12.34
990x =1222
x=1222/990=611/495

Bar Notation Problem


How to insert onload="initialize()" class into the body tag for google maps

Question: I am trying to display a simple 'hello world' Google Map on my page via the Google Maps API v3.  I get it so far but what I'm cought up on is adding
 class attribute to my body tag.  What would be the right way to handle this?
Comment: Well, first of all, that's not a class, it is an attribute. Drupal 6 or 7?  Needed on every page or one particular page?
Comment: My apologies.  This is Drupal 7. And it was just on one page.  I found a functional answer to the question even though AFAK it doesn't actually add the attribute to ``.
Answer: I was able to find my own answer.  This line doesn't necessarily add the attribute to the body tag, but it does call up the map.
drupal_add_js('jQuery(document).ready(function () { initialize(); });','inline');


Socket.io: módulo não pode ser encontrado

Question: Estava aqui "brincando" com Socket.io, tudo funcionando legal, quando dei o comando node app.js de uma hora para outra o NodeJS me retorna um erro (ele estava funcionando antes)

Cannot find module 'socket.io'

Busquei outro exemplo na internet colei e ele sempre me diz a mesma coisa. Mas quando entro no node pelo cmd e executo var foo = require('socket.io'); ele encontra o módulo sem nenhum problema.
Comment: Já instalaste o módulo? Fizeste `npm install socket.io --save`?
Comment: Sim, instalei. Estou agora tentando instalar o módulo no dir da aplicação
Comment: Não @Sergio, mas instalei agora o módulo no dir do app
Comment: E agora ele já encontra?
Comment: Funcionou, mas estou meio "esperto" talvez volte a parar de funcionar
Comment: Se usaste `--save` ele fica gravado no `package.json` e vai ser instalado quando fizeres `npm install` da aplicação na próxima vez. Parece que o Walker respondeu com a resposta, se o problema era esse podes aceitar a resposta dele.
Comment: Sim, estava apenas esperando um pouco aqui, mas aparentemente está tudo nos conformes. Resposta certa a dele. Mas vou abrir outra pergunta pois me deparei com outro erro no meu app.
Comment: É boa ideia esperar mais tempo antes de aceitar é verdade. Neste caso era simples, em casos mais complexos podem vir respostas mais elaboradas mais tarde.
Answer: Tente instalar o pacote, no diretório do arquivo app.js, digite npm install socket.io
O NPM é um gerenciador de pacotes do NodeJS, para usá-lo é simples. Primeiro cria-se o manifesto da sua aplicação com o comando npm init, ele fará algumas perguntas e irá gerar o arquivo package.json onde ficará armazenado entre outras coisas, as dependências de sua aplicação.
As dependências instaladas com o comando npm install são salvas no diretório node_modules dentro do seu projeto. Se você adicionar a opção save (npm install --save pacote) a dependência é salva permanentemente no manifesto.
Para instalar todas as dependências de um pacote usando o manifesto, basta rodar o comando npm install no diretório de sua aplicação.
Comment: Obrigado e parabéns @WalkerLeite
Comment: @Gustavo podes também aproveitar para dar `+1` à resposta :)


Join three datasets with some specific columns in R

Question: I have three datasets and I would like to create a new dataset, containing the nclusters column, and the Rank columns from both databases.
#databases
df1<-structure(list(nclusters = c(7, 8, 9, 10, 11, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
32, 33, 34, 35), Distance = c(5248.99947051215, 64018.2500397613,
63884.2473586514, 63822.8276312568, 63797.0409132363, 63797.0409132363,
63794.4869445083, 63792.0363232768, 63792.0363232768, 63792.0363232768,
63792.0363232768, 63766.907179171, 63763.0472418499, 63772.2020429773,
63772.2020429773, 63772.2020429773, 63771.3399204139, 63766.8444809863,
63766.8444809863, 63760.4694766072, 63760.4694766072, 63760.4694766072,
63760.4694766072, 63760.4694766072, 63760.4694766072, 63760.4694766072,
63759.7505492485, 63758.58565, 63758.58565), Rank = c(1L, 29L,
28L, 27L, 25L, 26L, 24L, 20L, 21L, 22L, 23L, 15L, 12L, 17L, 18L,
19L, 16L, 13L, 14L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 4L, 2L, 3L)), row.names = c("1",
"2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13",
"14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24",
"25", "26", "27", "28", "29"), class = "data.frame")

df2<-structure(list(nclusters = c(7, 8, 9, 10, 11, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
32, 33, 34, 35), Coverage = c(0.375505373587409, 0.353979450349411,
0.440737487316408, 0.524365528918811, 0.587640812323337, 0.587640812323337,
0.595943762399558, 0.604428674828288, 0.604428674828288, 0.604428674828288,
0.604428674828288, 0.753926389756351, 0.807111474295117, 0.705458712214102,
0.705458712214102, 0.705458712214102, 0.712208937171404, 0.754627308916128,
0.754627308916128, 0.865583389298128, 0.865583389298128, 0.865583389298128,
0.865583389298128, 0.865583389298128, 0.865583389298128, 0.865583389298128,
0.891174812227392, 1, 1), Production = c(1, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228, 0.0146770379360228, 0.0146770379360228,
0.0146770379360228, 0.0146770379360228), Performance = c(0.687752686793705,
0.184328244142717, 0.227707262626215, 0.269521283427417, 0.30115892512968,
0.30115892512968, 0.30531040016779, 0.309552856382155, 0.309552856382155,
0.309552856382155, 0.309552856382155, 0.384301713846187, 0.41089425611557,
0.360067875075062, 0.360067875075062, 0.360067875075062, 0.363442987553713,
0.384652173426076, 0.384652173426076, 0.440130213617075, 0.440130213617075,
0.440130213617075, 0.440130213617075, 0.440130213617075, 0.440130213617075,
0.440130213617075, 0.452925925081707, 0.507338518968011, 0.507338518968011
), Rank = c(1, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 15, 12,
19, 18, 17, 16, 14, 13, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2)), class = "data.frame", row.names = c(NA,
-29L))

df3<-structure(list(nclusters = c("1", "2", "3", "4", "5", "6", "7",
"8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18",
"19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29"
), score = c(0.832550756796408, 0, 0.0623422939077367, 0.0994025225880894,
0.119244082570837, 0.119244082570837, 0.12148059799612, 0.123691601200231,
0.123691601200231, 0.123691601200231, 0.123691601200231, 0.153356141263302,
0.160910166815577, 0.145325231545442, 0.145325231545442, 0.145325231545442,
0.146519242848501, 0.153463626026167, 0.153463626026167, 0.168008878164586,
0.168008878164586, 0.168008878164586, 0.168008878164586, 0.168008878164586,
0.168008878164586, 0.168008878164586, 0.170786494229362, 0.180835139451362,
0.180835139451362), rank = c(1L, 29L, 28L, 27L, 25L, 26L, 24L,
20L, 21L, 22L, 23L, 15L, 12L, 17L, 18L, 19L, 16L, 13L, 14L, 5L,
6L, 7L, 8L, 9L, 10L, 11L, 4L, 2L, 3L)), row.names = c(NA, -29L
), class = "data.frame")

I did as follows:
   library(dplyr)
   df1$nclusters <- as.numeric(df1$nclusters)
   df4 <- dplyr::left_join(df1, df2, df3, by = "nclusters")
   df4%>%
   rename("df1" = Rank,"df2" = Rank,"df3" = rank) %>%
   select(nclusters, df1, df2, df3)

However, it didn't work, what am I doing wrong?
Expected output
  nclusters  df1  df2  df3
     7      1     1     1
     8      29    29    29
     9      28    28    28
    10      27    27    27
    11      25    26    25
    12      26    25    26
    13      24    24    24
    14      20    23    20
    15      21    22    21
    16      22    21    22
    17      23    20    23
    18      15    15    15
    19      12    12    12
    20      17    19    17
    21      18    18    18
    22      19    17    19
    23      16    16    16
    24      13    14    13
    25      14    13    14
    26      5     11    5
    27     6      10    6
    28     7      9     7
    29     8      8     8
    30     9      7     9
    31     10     6     10
    32     11     5     11
    33     4      4     4
    34     2      3     2
    35     3      2     3

Comment: It is unclear-  what do you want in your expected output?
Comment: I will add the output table to the question
Comment: I inserted the output in the question
Answer: Column df3 looks incorrect in your output. Do you want to join data frames by nclusters? If yes, see the solution below.
 df4 <- df1 %>%
      left_join(df2, "nclusters") %>%
      left_join(df3 %>%
                  mutate(nclusters = as.numeric(nclusters))) %>%
      select(nclusters, contains("rank")) %>%
      rename_at(vars(contains("rank")), ~c("df1", "df2", "df3"))

Answer: Try this:
> library(dplyr)

> df3$nclusters <- as.numeric(df3$nclusters)

> df4 <- df1 %>%
    left_join(df2, by = "nclusters") %>%
    left_join(df3, by = "nclusters")

> df4 %>%
    rename("df1" = Rank.x, "df2" = Rank.y, "df3" = rank) %>%
    select(nclusters, df1, df2, df3)

 nclusters df1 df2 df3
1          7   1   1  24
2          8  29  29  20
3          9  28  28  21
4         10  27  27  22
5         11  25  26  23
6         12  26  25  15


Computability: is there an alternative method to decide this language?

Question: For my computability revision I am trying to decide the language, $$L = \{  \text{all binary strings containing the pattern 001 (not necessarily in consecutive places)} \}.$$
I believe that I can do this, however my method is very long and difficult to keep track of due to the many different states required for my Turing Machine.
Anyway, I proceeded as follows (exact details omitted):
Given an input string look for the first 0. Upon finding a zero, check if 001 occurs there. If so, accept and if not move to the next 0 then do the same. If at some point we see a blank then it means no 001 was seen so do a carriage return whereby we plan to test if 0•0•1 appears (• is any element in the alphabet).
I did this step in generality: suppose we want 0•••...•••0•••...•••1 where there are n•s in each case. Then, upon seeing a zero move along n places in the tape by defining new states at each symbol that act as a counter (see attached image). I know that I need to include details here regarding if a blank is seen because then we have reached the end of string => carriage return.
I'm fairly sure this algorithm is correct (with a bit of polishing) and since there are a lot (finitely many however) of states, then I am ok, but is there an easier method? For this algorthim I have initial, accept, reject, carriage return, find-0, find-1, move-n spaces (2sets, before and after middle 0 in the string! They are distinct) for each value of n between 1 and the length of the string: this is a nightmare to write and keep track of!
Surely an easier method exists?!
The image is only rough work so please don't be too cruel. Also, $x$ is any element in the input alphabet, there are special cases if we get to a blank.
Answer: Your language seems to correspond to the regular expression $1^*01^*01[01]^*$
You only need 3 states:
S1: "No zero encountered yet." If you read a 1, stay here, if you read a 0, go to state S2.
S2: "1 zero encountered." If you read a 1, stay here, if you read a 0, go to state S3.
S3: "2 or more zeros encountered." If you read a 0, stay here, if you read a 1 accept.
Comment: Thanks for your answer. Firstly, what exactly do you mean by "the regular expression..."? Secondly, I understand how your states work (I have in fact written equivalent statements) but I fail to see how this deals with strings of 001 form but with separation between both zeros and the zero-one part.
Comment: A "regular expression" is a formal description of a language. A language that can be described like this is a "regular language" and can be accepted by a finite automaton, a less powerful computation model than a Turing machine.
Comment: If you encounter additional 1s before you have seen two zeros, or if you see more than two zeros before you reach the final 1, you just ignore these symbols and go on to read the next symbol.
Comment: I've just had my misinterpretation of the question clarified and now I understand what you are saying. Thanks for the help, it makes sense now!
Answer: Hint: are there at least two $0$'s to the left of the rightmost $1$?
Comment: I see how this would work for the standard $001$ string, however, I don't see how this would solve the problem of deciding that 0#0#1 is in the language (for an arbitrary symbol #) because you still have to count the number of symbols between the first and second zero, then the second zero and first one. Do you think perhaps I am misunderstanding the question? My interpretation of it is that 001, 0#0#1, 0##0##1 etc should all be accepted.
Comment: Yes, I think you are misunderstanding.  The numbers of symbols between the 0, 0 and 1 don't have to be the same.  0#0##1, for example, is acceptable.
Comment: I think I understand now, thanks. Now that my misunderstanding is clarified I think your hint is particularly helpful too.


php PDO for loop brakes code

Question: The following is part of my PDO script that gets AJAX POST requests from my webpage. So far I've got it working as intented, but now I would like to add a simple for-loop that iterates trough my $results array, and runs strtotime on a timestamp that's under the key of "posted".
Now every time I uncomments my for-loop the whole thing brakes, and I can't figure out why. All I get is HTTP/1.0 500 Internal Server Error.
Any explanation on what is going on and how to fix this is appreciated.
my code (I've commented out the irrelevant parts):
<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);

try {
    $hostname = "localhost";
    $username = "topdecka_admin";
    $password = "";

    $db = new PDO("mysql:host=$hostname;dbname=topdecka_PTC;charset=utf8",$username, $password);
    $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

    /*if (!empty($_POST["searchword"])) {
        $searchword = $_POST["searchword"];
        $query = $db->prepare(
            'SELECT articles.title, articles.posted, articles.extract, authors.name, GROUP_CONCAT(categories.cat_name) AS cat_name
            FROM articles, authors, categories, article_categories
            WHERE articles.author_id = authors.id
            AND articles.id = article_categories.article_id
            AND article_categories.category_id = categories.id
            AND ((title LIKE :searchword) OR (extract LIKE :searchword) OR (body LIKE :searchword) OR (name LIKE :searchword) OR (cat_name LIKE :searchword))'
            ); //end DB QUERY
        $query->execute(array(":searchword" => "%" . $searchword . "%"));
        $result = $query->fetchAll();
        //turns timestamp into integer
        for($i = 0; $i < count($result); ++$i) {
          $result[$i]['posted'] = strtotime($result[$i]['posted']);
        }
        echo json_encode($result);
        die();
    }
    else if (!empty($_POST["title"])) {
        $title = $_POST["title"];
        $query = $db->prepare(
            "SELECT articles.title, articles.posted, articles.body, authors.name, authors.img, authors.bio, GROUP_CONCAT(categories.cat_name) AS cat_name
            FROM articles INNER JOIN authors ON articles.author_id = authors.id
            INNER JOIN article_categories ON articles.id = article_categories.article_id
            INNER JOIN categories ON article_categories.category_id = categories.id
            WHERE title LIKE :title; SELECT comment.user_id, comment.text, comment.posted FROM articles RIGHT JOIN comment ON articles.id = comment.article_id
            WHERE title LIKE :title;"
            ); //end DB QUERY
        $query->execute(array(":title" => $title));
        $result = $query->fetchAll();
        $result[0]['posted'] = strtotime($result[0]['posted']);
        $query->nextRowset();
        $result2 = $query->fetchAll();

        for($x=0; $x < count($result2); $x++) {
            $result2[$x]['posted'] = strtotime($result2[$x]['posted']);
        }

        echo json_encode(array('article'=>$result, 'comments'=>$result2));
        die();
    } */

    else if (!empty($_POST["comment_load"])) {
        $comment_load = $_POST["comment_load"];
        $query = $db->prepare(
                "SELECT comment.user_id, comment.text, comment.posted FROM articles RIGHT JOIN comment ON articles.id = comment.article_id
                WHERE title LIKE :comment_load;"
            );
        $query->execute(array(":comment_load" => $comment_load));
        $result = $query->fetchAll();
        for($x=0; x<count($result); x++) {
            $result[$x]['posted'] = strtotime($result[$x]['posted']);
        }
        echo json_encode($result);
        die();
    }

    /*else if (!empty($_POST["comment-text"])) {
        $input_text = $_POST["comment-text"];
        $query = $db->prepare(
            'INSERT INTO comment (user_id, text, article_id)
            VALUES (101, :input_text, 4);'
            );
        $query->execute(array(":input_text" => $input_text));
        echo json_encode($result);
        die();
    }
    else {
        $query = $db->prepare(
            'SELECT articles.title, articles.posted, articles.extract, authors.name, GROUP_CONCAT(categories.cat_name) AS cat_name
            FROM articles, authors, categories, article_categories
            WHERE articles.author_id = authors.id
            AND articles.id = article_categories.article_id
            AND article_categories.category_id = categories.id'
            ); //end DB QUERY
        $query->execute();
        $result = $query->fetchAll();
        //turns timestamp into integer
        for($i = 0; $i < count($result); ++$i) {
          $result[$i]['posted'] = strtotime($result[$i]['posted']);
        }
        echo json_encode($result);
        die();
    }*/
}
catch (PDOException $e) {
    echo "Error!: " . $e->getMessage() . "
";
    die();
}
?>

Here's the var_dump($result) without the for-loop changing the timestamp:
"array(7) {
  [0]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(4) "miha"
    [1]=>
    string(4) "miha"
    ["posted"]=>
    string(19) "2015-06-15 10:18:44"
    [2]=>
    string(19) "2015-06-15 10:18:44"
  }
  [1]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(4) "miha"
    [1]=>
    string(4) "miha"
    ["posted"]=>
    string(19) "2015-06-15 10:19:21"
    [2]=>
    string(19) "2015-06-15 10:19:21"
  }
  [2]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(14) "miha miha miha"
    [1]=>
    string(14) "miha miha miha"
    ["posted"]=>
    string(19) "2015-06-15 15:57:40"
    [2]=>
    string(19) "2015-06-15 15:57:40"
  }
  [3]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(14) "miha miha miha"
    [1]=>
    string(14) "miha miha miha"
    ["posted"]=>
    string(19) "2015-06-15 15:59:05"
    [2]=>
    string(19) "2015-06-15 15:59:05"
  }
  [4]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(17) "miha miha miha..."
    [1]=>
    string(17) "miha miha miha..."
    ["posted"]=>
    string(19) "2015-06-15 15:59:35"
    [2]=>
    string(19) "2015-06-15 15:59:35"
  }
  [5]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(20) "miha miha miha......"
    [1]=>
    string(20) "miha miha miha......"
    ["posted"]=>
    string(19) "2015-06-15 16:00:15"
    [2]=>
    string(19) "2015-06-15 16:00:15"
  }
  [6]=>
  array(6) {
    ["user_id"]=>
    string(3) "101"
    [0]=>
    string(3) "101"
    ["text"]=>
    string(20) "miha miha miha......"
    [1]=>
    string(20) "miha miha miha......"
    ["posted"]=>
    string(19) "2015-06-15 16:11:12"
    [2]=>
    string(19) "2015-06-15 16:11:12"
  }
}

Thanks for the help.
Comment: this `for($x=0; x<count($result); x++) {` you're missing `$` signs for the `x`'s - The `x` is treated as a constant and error reporting would have told you that. Undefined constant x... - check your logs. Plus, you have it commented it out `for($i = 0; $i < count($result); ++$i) {` so do something similar to that, but using `x` rather than `i`.
Comment: yeh, the problem was with the missing $ signs when declaring x in the loop. Write it up as an anwser and I'll accept it. Thanks
Comment: check the server log for details about the 500. that should be your FIRST stop anytime you get a 500.
Comment: @MihaŠušteršič Right on, glad to hear it, *cheers*.
Answer:
"yeh, the problem was with the missing $ signs when declaring x in the loop. Write it up as an anwser and I'll accept it. Thanks"

As per OP's request, comment to answer:
this for($x=0; x<count($result); x++) { you're missing $ signs for the x's
The x is treated as a constant and error reporting would have told you that.

Undefined constant x...

Check your logs. Plus, you have something similar that is commented out in for($i = 0; $i < count($result); ++$i) { so do something similar to that, but using x rather than i

php PDO for loop brakes code


Why the derivative of the logarithm of a theta function is not an elliptic function?

Question: Let's consider the theta functions periodicity conditions
$$\vartheta\left(z+1\right) =\vartheta\left(z\right),$$
$$\vartheta\left(z+\tau\right) =e^{-\pi i\tau-2\pi iz}\vartheta\left(z\right),$$
Those imply conditions on the derivatives of theta functions
$$\vartheta'\left(z+1\right) =\vartheta'\left(z\right),$$
$$\vartheta'\left(z+\tau\right) =e^{-\pi i\tau-2\pi iz}\vartheta'\left(z\right),$$
Therefore if I take the derivative of the logarithm I should have an elliptic function since
$$f(z)=\frac{d}{dz}log\left( \vartheta\left(z\right)\right) =\frac{\vartheta'\left(z\right)} {\vartheta\left(z\right)}$$
then we have
$$f(z+\tau)=f(z)=f(z+1).$$
Anyway this can't be since it has only one pole in the fundamental period, so what am I missing?
Answer: From
$$\vartheta(z+\tau)
=e^{-\pi i\tau-2\pi iz}\vartheta(z)$$
you get
$$\vartheta'(z+\tau)
=e^{-\pi i\tau-2\pi iz}\vartheta'(z)
-2\pi i e^{-\pi i\tau-2\pi iz}\vartheta(z)$$
and so
$$\frac{\vartheta'(z+\tau)}{\vartheta(z+\tau)}
=\frac{\vartheta'(z)}{{\vartheta(z)}}
-2\pi i.$$
Therefore $\vartheta'/\vartheta$ is not an elliptic function
but $(\vartheta'/\vartheta)'$ is (essentially the Weierstrass $\wp$-function).
Comment: What does it change with $f\left(z\right)=\sum\lambda_{k}\frac{d}{dz}log\vartheta\left(z-p_{k}\right)+C$? I know in this case can be elliptic if $\sum\lambda_{k}=0$
Comment: @Dac0 In that case, the $-2\pi i$ becomes $-\sum2\pi i\lambda_k$....
Comment: Shark  You are definitely right :D :D :D


Compact subset of cantor set meeting all but one dense subset

Question: Let $C$ be the Cantor set, and let $\{D_\alpha:\alpha<\kappa\}$ be a collection of pairwise disjoint subsets of $C$ such that each $D_\alpha$ is countable and dense in $C$, and $$\bigcup \{D_\alpha:\alpha<\kappa\}=C.$$
Is there a closed set $A\subseteq C$ such that $A\cap D_0=\varnothing$ and $A\cap D_\alpha\neq \varnothing$ for all $\alpha>0$?
I thought about doing something with measure theory here, using a "fat" cantor set of positive measure, and covering $D_0$ with open sets of small measure.
Comment: But I want $A$ to be closed.  If it depends on the particular $D_\alpha$, I'd be interested in any positive example.
Comment: $A=C\setminus D_0$? Bigger is not possible. Rest may or may not depend on $D_\alpha$
Answer: You can construct a counterexample by transfinite induction.  Start by letting $D_0$ be your favorite countable dense subset of $C$.  Fix an enumeration $(U_\beta)_{\beta<\mathfrak{c}}$ of all the open subsets of $C$ which contain $D_0$.  Now just choose disjoint countable dense subsets $D_\alpha$ for $1\leq\alpha<\mathfrak{c}$ one by one so that for each $\beta<\mathfrak{c}$, there is some $\alpha>0$ such that $D_\alpha\subseteq U_\beta$.  We can do this since at any stage, we have defined fewer than $\mathfrak{c}$ of our sets $D_\alpha$, so we can choose a new countable dense subset which is disjoint from all the ones we've chosen so far and is contained in any particular $U_\beta$.  We can also construct some extra countable dense subsets along the way to make sure that the union of all of them is all of $C$.
Now, suppose the set $A$ which you ask for existed.  Then $C\setminus A$ would be an open set containing $D_0$, and so would be equal to $U_\beta$ for some $\beta$.  But then there is some $\alpha>0$ such that $D_\alpha$ is contained in $U_\beta$, and $A$ cannot intersect that $D_\alpha$.


How should I notate an intermediate dynamic between mp and mf?

Question: I know this is a question that's been asked before, but I haven't found an answer that suits my needs.
Consider the following example [edit: new excerpt, to include all swells and to remove other staves as requested]:

As you can see, there are several swells in this string passage (the rest of the string section uses identical dynamics). This is following an (unshown) intro in pp, hence why I want the first swell to be louder (p). Each swell is intended to sound louder than the last, with the final reaching f (but no higher, as there are louder climaxes later).
What should I write in the space of ? to indicate a dynamic between mp and mf? Is there something that means "louder than mp but not quite mf"? Until recently I used m, but many say that's too ambiguous.
Edit: I've received some informative answers and comments and I appreciate them all, and I intend to implement the things I've learned in my scores. But I don't think I've yet received an answer to the specific question of whether a middle-ground exists between mp and mf and how to notate it, in the event where this were necessary (assuming you couldn't upshift or downshift other dynamics for whatever reason).
For instance, would having a passage play mp and then adding a cresc. hairpin (that doesn't end with a written mf) work? (This assumes both mp and mf are used further along, and that you want gradual "steps" in volume.)
Comment: Much better -- now we can see the forest for the trees. // The natural way a string player does slow, legato hairpins is: up bow for the crescendo and down bow for the decrescendo.  The way you've written the first 6 bars will probably result in the player ignoring the bowing you've written.  The rest looks fine.
Comment: mp and mf ARE the middle ground. Something in between those two does not make sense. I know, on your computer, in the notation software it does. You need to try and separate the intent (which is what you write down) from the (computerized) "performance". Hence most people, myself included, use DAW's to turn the score into a "real" performance simulating mock-up.
Answer:
Each swell is intended to sound louder than the last, with the final reaching f

Ok, so express exactly that – not by hard-coding the overarching crescendo into discrete steps, but by describing it as what it is:
Answer: You might not like this idea, but from the point of view of practical performance I would simply replace your "mp" markings with "p" and your "?" with "mp".
That shows the general idea of "waves" of crescendo and diminuendo within an overall crescendo. Hoping that the same dynamic marking will have exactly the same dynamic level every time it is used just isn't going to happen with human performers.
Looking at the snippet of score, it seems like there are parts that are "theme" (e.g. the fourth staff for the first 4 bars) and others that are "accompaniment," and good human performers are not going to play those at the same dynamic level, unless you add some very explicit instructions like "senza expressivo".
Historically, "mp" was probably the most recent dynamic marking to be invented. Initially, only f and p were in common use. ff and pp were then added, and very occasionally mf as "something in between f and p". Mf gradually became more common, and mp was presumably invented by analogy with mf.
Also, originally f and p didn't even mean "loud" and "soft". In Italian "forte" and "piano" mean "bright" (or "strong") and "smooth". If you really want to micro manage dynamics, don't bother with the traditional markings and invent a graphical notation instead - but don't expect many people will want to play your music if your notation is hard to understand!
Comment: "I don't want it to be all that "quiet"." Don't worry about that. Playing very soft is harder than playing very loud!
Comment: There is no reason not to use a wider range than pp to ff if you want. fff and ppp are common enough, and Tchaikowsky once wrote pppppp. "niente" really means "inaudible" and is normally preceded or followed by a rest, not used in the middle of a pair of hairpins.
Comment: To indicate silence, you can use a break (looks like a big apostrophe) or a very small rest.  Probably the break (large apostrophe) is the easiest thing.  If you have the space for it, you could also use words, e.g. *fading to nothing* but a decrescendo to *ppp* followed by a break would do the trick, in my opinion.
Comment: The posted excerpt actually excludes (for space) a preceding swell, which itself goes from *niente* to ***p*** back down to the visible *niente* at the start of the excerpt. So would you suggest changing that original ***p*** to ***pp***? My fear then would be that it might be played *too* quietly, since it's meant to be, maybe not loud, but reasonably present, if that makes any sense – I don't want it to be all that "quiet".
Comment: Also, this segment is part of the intro, which builds up to the "main theme" that's played entirely in ***f*** – with later climactic swells of ***ff*** and ***fff*** – which is why I wouldn't want to make the last visible dynamic in that intro any louder than ***mf***.
Comment: ""niente" […] is normally preceded or followed by a rest, not used in the middle of a pair of hairpins." I wanted to mark clearly that the players should end the first swell with silence before the next cresc., and the second swell should end very quietly, but not quite as silently as the first. How would you notate that?
Comment: What about  something like "mp  poco cresc. mf" ?
Answer: Symbols indicating dynamics are all relative!  A f in one context might sound completely different than in another context.  This means that you can use the symbols in a flexible way to convey your intent.  If you have a sequence of crescendos, where each one lands at a louder level than the one before, then you could have them land at p, mp, mf or at mp, mf, f.  An experienced musician will interpret f according to the context.
An alternative approach would be to set up the bowing to elicit the dynamics you want.  The hairpin that contains the loudest sound needs more bow than the others.  So you might want to have the first two bars be a down bow with the long slur as you have written it; then no slur for bars 3 through 6, with a marking to stay in the upper half in bars 5 and 6.  It's a little "middle school" (i.e. "junior high school") to tell adults to use the upper half, but it's an option.
Comment: Responding to both this answer and your comment above: I'm not sure I understand your comments about bowing, though that may just be that I don't understand bowing that well. Why wouldn't a player just play & bow the instrument as written in the excerpt, if I want a smooth (legato) transition between the notes?
Comment: @Walter - When you play an up bow stroke (from tip to frog) there is a natural tendency to get louder.  When you play a "down bow," the sound naturally gets gradually softer as you get closer to the tip.  An experience string player works out the optimal bowing pattern on either the first reading or pretty soon thereafter.  You can generally tell whether the person who wrote out the part knew what he was doing.  If he didn't, then you just work out your own bowing according to your musical judgment.  Why don't you rent a student cello for a month and experiment?
Comment: Then taking bowing into consideration, how would you suggest I write it so that the notes in the swells all have smooth transitions, except from one swell to the next (as currently indicated by the slurs)? Also, renting is not feasible for me, I'm afraid.
Comment: Good string players can play legato even when changing bow.  Write "legato."  Alternatively you can have them change bow in a strange place.  Try this.  Get a dowel rod or fairly straight tree branch (peeled of bark), the diameter of a fat kindergarten pencil, with the length of a cello bow.  Use your left index finger (palm down) to simulate the string.  Practice bowing.  Sing some nice cello parts while you "bow" them.  When you go up bow, let your volume increase.  When you go down bow, let it decrease.  Let this become second nature.  Then you will be able to write bowings in your parts.
Comment: Do you have any questions about the suggestion I made in the second paragraph of my answer?
Comment: Your second paragraph (along with your bowing comment on my question) is what I was asking for more information about, yes. I'm still thinking about what to do with the information I've been given.
Comment: @Walter - If you look at a method book for beginning string players, at a music shop, you'll see that bowing instructions are given pretty precisely, with "upper half" for example.  The down - up natural dynamics are a fact of life for string players.  You try to use it to "sand with the grain" so to speak when you can, and when you can't, you have to work to go against it.  // Let me know later if you have any follow-up questions.  // General comment: I think that in general you may be assuming performers have more precise notches of volume than they actually have.  The truth is that ...
Comment: ... the subjective experience of dynamics, when one is playing, is very much *analog*, not *digital.*  You can't dial in a "*pp*" or an "*mf*."  What you can do is think, this part needs to be loud, how am I going to accomplish that, I will get closer to the bridge and allow the sound to get a little bit scratchy, remembering that the scratch will not be heard out in the hall, I will use arm weight (right arm), I will divide up the long notes and change bow a whole lot, I will adjust the angle of the bow so I have a lot of hair contacting the string.  And vice versa for soft.  For gradations,
Comment: ... I will think about *more than before* or *less than before*.  Also *a whole lot more than before* or *a whole lot less than before*.
Comment: (Final comment here since it says this discussion's getting too long) Yeah, thinking about dynamics more relatively is something I plan to implement in my scores. Though I still want to think that whilst the line may be blurry between neighboring dynamics (***ff*** and ***fff***), there should be a clear difference between, say, ***f*** and ***fff***, at least depending on how you define them (***f*** = loudly vs. ***fff*** = at the top of your voice, etc.).
Comment: @Walter - Try it.  Look at the score of some orchestral piece you're fond of.  Mark the places where there's *f* and then *fff*.  Listen to a recording.  Is that really where the precision lies?  Maybe sometimes, I would guess, but not always.  Once I had the good luck to have a composer, George Walker, come listen to a rehearsal of my youth orchestra's work on one of his pieces.  His comment each and every time the conductor asked him for feedback was "We need to hear more from the trombone section.  More trombone, please!"  To me, it all boils down to MORE! or LESS!
Answer: IMO perhaps something like "meno p" might work. It's still highly context dependent though. In fact if you have something that by its melodic contour, harmonic scheme, etc., naturally suggests a swell in dynamics with each climax gradually getting louder, you might be able to get away with writing just the first one or two dynamics and the cresc./dim. marks, or even just write 1 or 2 pair(s) of crest./dim. marks and write simile. after that, and then only write a crescendo just before you're getting to the big climax, followed by the dynamic marking for that climax (e.g. f, ff, etc.)
Answer: I would suggest poco piu forte or even poco piu forte ma non ancora mezzo forte.  More concisely, piu .


Reading twitter feed using api 1.1 from c#

Question: Ok. I thought I would fix this rather easy, but I'm not getting anything to work for some reason. What I want to do is simple; I want to create a web part displaying the latest tweets from a specific user. I'm coding C#.
I found a great post here but it's using php.
Can anyone help me to perform this simple task?
Answer: You can find alot examples over internet
Web:
http://www.microsoft.com/web/post/learning-the-basics-of-using-the-twitter-api-in-aspnet-web-pages-with-razor-syntax
Console:
http://www.d80.co.uk/post/2011/02/13/A-Simple-Twitter-Client-in-C-with-OAUTH-using-TweetSharp.aspx
:)
Comment: Thanks! Your links got me in the right direction. It turned out my code couldn't handle ssl, so I needed a little tweak to get around it.


Ansible conditionals - Wildcard match string

Question: I have the following conditional in an Ansible task:
when: ec2_tag_Name == 'testhost01'

It works fine, however I would like to match a wildcard on the ec2_tag_Name field.
So something like this
when: ec2_tag_Name == 'testhost*'

The goal is to match anything like testhostx testhost12 testhostABC etc etc just anything matching testhost at the start of the string.
Is this possible? Can't seem to get it working.
Answer: From Testing Strings:

To match strings against a substring or a regex, use the "match" or "search" filter

In your case:
when: ec2_tag_Name is match("testhost.*")

Comment: I had also found that `when: 'testhost' in ec2_tag_Name` can do a similar thing, it doesn't restrict it being at the start of the string - so yours is definitely the right way to do it. Thanks
Answer: This works as well.
when: "ec2_tag_Name.startswith('testhost')"

You can combine logical operators as well like and  and  or
Comment: Where is this documented? I could not find it


Difficulties with `agrep(..., fixed=F)`

Question: In ?agrep (grep with fuzzy matching) it mentions that I can set the argument fixed=FALSE to let my pattern be interpreted as a regular expression.
However, I can't get it to work!
agrep('(asdf|fdsa)', 'asdf', fixed=F)
# integer(0)

The above should match as the regular expression "(asdf|fdsa)" exactly matches the test string "asdf" in this case.
To confirm:
grep('(asdf|fdsa)', 'asdf', fixed=F)
# 1 : it does match with grep

And even more confusingly, adist correctly gives the distance between the pattern and string as 0, meaning that agrep should definitely return 1 rather than integer(0) (there's no possibility that 0 is greater than the default max.dist = 0.1).
adist('(asdf|fdsa)', 'asdf', fixed=F)
#      [,1]
# [1,]    0

Why is this not working? Is there something I don't understand? A workaround?
I'm happy to use adist, but am not entirely sure how to convert agrep's default max.distance=0.1 parameter to adist's corresponding parameter.
(yes, I'm stuck on an old computer that can't do better than R 2.15.2)
> sessionInfo()
R version 2.15.2 (2012-10-26)
Platform: i686-redhat-linux-gnu (32-bit)
locale:
 [1] LC_CTYPE=en_AU.utf8       LC_NUMERIC=C
 [3] LC_TIME=en_AU.utf8        LC_COLLATE=en_AU.utf8
 [5] LC_MONETARY=en_AU.utf8    LC_MESSAGES=en_AU.utf8
 [7] LC_PAPER=C                LC_NAME=C
 [9] LC_ADDRESS=C              LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_AU.utf8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

Comment: It's `aregexec`. Sorry for the typo.
Comment: Okay, I really need a coffee. It doesn't mention in `?agrep`. Once again sorry about that. I've now no idea where I saw it! :)
Comment: I am unable to explain this behaviour as well, but `aregexc` seem to work. `aregexc('asdf|fdsa', 'asdf')` (which takes default `fixed = FALSE`) and does regexp by default. But surprisingly `fixed=TRUE` here with `aregexc` gets back to the same issue.
Comment: Where did you get `aregexc` from? I'm guessing this is in a newer version of R than I have?
Comment: It mentions under the help page of `?agrep`, but yes I do have R 3.0.0. your problem seems to be with `|`. Because `agrep('la[sb]y', 'lazy', fixed=FALSE)` works.
Comment: Thanks for `aregexec` (my `?agrep` does not mention it).
Answer: tl;dr: agrep(..., fixed=F) does not seem to work with the '|' character. Use aregexec.
Upon further investigation, I think this is a bug, and that agrep(..., fixed=F) does not seem to work with '|' regexes (although adist(..., fixed=F) does).
To elaborate, note that
adist('(asdf|fdsa)', 'asdf', fixed=T) # 7
nchar('(asdf|fdsa)')                  # 11

If 'asdf' were agrep'd to the non-regular-expression string '(asdf|fdsa)', then it would have distance 7.
On that note:
agrep('(asdf|fdsa)', 'asdf', fixed=F, max.distance=7) # 1
agrep('(asdf|fdsa)', 'asdf', fixed=F, max.distance=6) # integer(0)

These are the results I'd expect if fixed=T. If fixed=F, my regex would match 'asdf' exactly and the distance would be 0, so I'd always get a result of '1' back out of agrep.
So it looks agrep(pattern, x, fixed=F) does not work, i.e. it actually regardes fixed as TRUE for this sort of pattern.
As @Arun mentions, it might just be '|' regexes that don't work. For example,  agrep('la[sb]y', 'lazy', fixed=FALSE) does work as expected.

EDIT: Workaround (thanks @Arun)
The function aregexec appears to work.
> aregexec('(asdf|fdsa)', 'asdf', fixed=F)
[[1]]
[1] 1 1
attr(,"match.length")
[1] 4 4

Answer: This has (finally) been fixed in the R sources "trunk" / R-devel") and R-patched which will become R 3.5.1  early July 2018.

Difficulties with `agrep(..., fixed=F)`


SSH tunnel over multi hops using putty (two hops)

Question: I have these config:
Host with putty -- > Gateway A (port 22 open) --> FinalMachine (port 22 with gateway A open and port 18080 with apache).
Like I don't have communication from Host with putty with FinalMachine I would like to tunneling the connection to can connect directly from Host to 18080 of FinalMachine.
I'm trying to do with putty but I can't find the solution.
Anyone could help me please? Thanks!
Answer: Method 1, using 2 instances of putty

Putty 1: Local forward 20022:GatewayB:22, Connect to GatewayA
Putty 2: Local forward 18080:FinalMachine:18080, Connect to localhost:20022

Method 2:

Putty Local forward 18080:127.0.0.1:18080,

From within session do next step ssh -L 18080:FinalMachine:18080 GatewayB
Comment: Sorry the Method 1 I can't understand. What is GatewayB??? Thanks
Comment: Problem is mine in the question redaction GatewayB is FinalMachine. I've edited the main question
Comment: Since you used "multi hops" and mentioned Gateway B I assumed there actually was a B machine. As long as `Gateway A` can access `FinalMachine:18080` you only need that one forward when connecting to `A`. On the other hand if `18080` is only accessible from `FinalMachine` itself, then use above, replace `B` with `Final` and possible change to localhost.


How do I play difficult ukulele chords (B major)?

Question: How do I play the harder chords on the ukulele? I'm self taught and am having a hard time with B.
Comment: How are you trying to play the B chord and what exactly is the problem?
Answer: B major is a chord that requires every note to be fretted, so I've seen two main ways to do it.

Barre the E and A strings at the 2nd fret with the index finger, then middle finger frets C string at 3rd fret, and ring finger frets G string at 4th fret (4-3-2-2).
Barre across all strings with index finger, middle frets C string at 3rd fret, ring frets G string at 4th fret (4-3-2-2).

If you like skipping strings, (x-3-2-2) works. Note that the B chord is like the B♭ chord, but a fet higher. Other voicings up the fretboard include (8-6-7-6), (11-11-11-9), and other B6 voicings also might work depending on context. I would advise against workarounds like those, however, as that chord shape is so useful and you'll probably need to learn it eventually.

I too am self-taught, and I can play most of the harder chords on ukulele, and from experience, I can say that it's extremely useful to work on using the pinky finger to fret notes. It takes practice, but if you can play 4-finger chords, you'll be able to play pretty much any kind of chord. Apart from that, I noticed that at first, weird shapes like A♭m (1-3-4-2) felt alien and I had to position and reposition my fingers over and over again. After a while (~a few hundred tries and a week or two), I noticed that I could suddenly do it just as well as any other shape I already knew. So don't despair; stick with the harder shapes, and eventually you'll be able to do them with your eyes shut and on command.

For more information, see this question. It covers a lot of what you've asked. How to play B major? What is the finger placement?
Answer: B is a difficult chord to learn on both ukelele and guitar because none of the notes in the chord are available on the open strings[1]. So you must fret all the strings or mute unwanted ones.
You can fret a B chord as 4-3-2-2. If barring the top two strings is difficult, you can try muting the top string and just playing the lower three.
Never be afraid to play just partial voicings, especially if you have other instruments to cover other parts of the sonic space. Even on a guitar with up to six strings sounding at once, letting just three ring out is very effective as the extra muting adds to the percussive attack.

There is a B string on the guitar of course, but the voicings which use it are somewhat more awkward than using a barre-chord form.
Comment: B is one of the open strings on guitar, if I'm not mistaken.
Comment: @user45266 -- probably the first **B7** that most are taught is `x-2-1-2-0-2`. Within easy reach are **B6**: `x-2-1-1-0-2`, **BMaj7**: `x-2-1-3-0-2`, or **B**: `x-2-1-x-0-2`, all using the open B string.
Comment: Fine points all around, gentlemen. I have added a footnote which hopefully somewhat mitigates my blunderous answer.
Comment: Although as I understand it, the B chord shape on guitar does not use the open B string, correct?
Comment: @user45266 it is absolutely possible to play a B major triad on guitar while using the open B string - it's just not in positions most commonly taught (going off an E-shape or A-shape barre chord on the 7th and 2nd frets respectively).
Comment: @user45266 I'm personally a fan of `X-X-9-11-0-11` because I'm currently going through a play-everything-high-on-the-neck phase.
Comment: @JamesWhiteley Well of course :) I meant the most commonly taught one; theoretically there's hunderds of voicings for B major, but most aren't playable by humans. As a side note, I have this one chord I like to do on ukulele that's similar (9-0-8-10) to your B major.
Comment: +1, @luserdroog. I'd hardly call that blunderous, and that's a great explanation of why those barre chords are preferred.
Comment: And thank you, @DavidBowling, that was quite illuminating.


Testing linear restriction in R

Question: I would like to test a linear restriction in R. Instead of the usual $\beta_i=0$, I want to test if $\beta_k=0.5$ and $\beta_j=-0.5$.
Is there a way to do this using lm command, and just writing a different formula for the models that lm command uses?
I was thinking of writing $y-0.5x_k+0.5x_j=\sum_{i\neq j,k} x_i\beta_i+(\beta_k-0.5)x_k+(\beta_j+0.5)x_j$. However, I'm not sure how to code this reparametrization...
P.S:Would writing $y−I(−0.5x_k+0.5x_j)$ ~ $\sum x_i$ as the formula to be run on lm, work as 2 t-tests?
Any help would be appreciated.
Answer: I think your current parameterization is sufficient to test either coefficient restriction (via the t-tests for each coefficient), but not both restrictions at the same time. To do that one can conduct an F-test between the unrestricted and the restricted model (since it is nested).
Example below in R using the car package.
library(car)
set.seed(10)
x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
y <- 5*x1 + 0.5*x2 - 0.5*x3 + rnorm(100)
m1 <- lm(y~x1+x2+x3)
linearHypothesis(m1,c("x2 = 0.5","x3 = -0.5"),test="F")

Which then spits out a table with the RSS for each model and the degrees of freedom, needed to calculate the F-statistic.
Linear hypothesis test

Hypothesis:
x2 = 0.5
x3 = - 0.5

Model 1: restricted model
Model 2: y ~ x1 + x2 + x3

  Res.Df    RSS Df Sum of Sq      F Pr(>F)
1     98 107.21
2     96 107.10  2   0.10752 0.0482  0.953

You answered you own question in terms of testing individual coefficients, you can transform the variable on the left hand side. To replicate the linearHypothesis function I trick R here by making the transformed variable:
#equivalent F test - can trick R by making y a new variable
y <- y -0.5*x2 +0.5*x3
m2 <- lm(y~x1)
anova(m2,m1)

Which reproduces the earlier table. You can use this same trick to get the default summary t-test's in the regression output to test against the alternative hypotheses:
#test for either coefficient restriction (with updated y)
m3 <- lm(y~x1+x2+x3)
summary(m3)
#you can see m3 is equivalent to m1 - just changes the location of the test
#anova(m1,m3)

Which produces for coefficient estimates:
Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept)  0.208282   0.107351   1.940   0.0553 .
x1           5.045149   0.112973  44.658   <2e-16 ***
x2           0.033042   0.110693   0.299   0.7660
x3          -0.004875   0.110122  -0.044   0.9648

To test with the original values, user2864849 answers that question. You just subtract out the null from the point estimate and use the same standard error. So using the coefficients from m1 you can see below t_est reproduces the t-statistic in model 3.
#to see how it is just the original t-test with the location changed for x2
pt_est <- summary(m1)$coefficients[3,1] #point estimate x2
se_est <- summary(m1)$coefficients[3,2] #standard error x2
t_est <- (pt_est - 0.5)/se_est          #t-stat for null of 0.5
t_est

Comment: Yep - you already answered your own question if that is the case.
Comment: Andy, thanks for your answer. If I was interested in doing the t-tests, how would I do it in R? I do not know how to code that part as a formula for the lm command. This is what I'm more interested about.
Comment: Andy, should I write '$y-I(-0.5x_k+0.5x_j) $ ~ $ \sum x_i$' as the formula to be run on lm?
Comment: Hi, is there a restriction test for time series models such as GARCH model? I cannot apply this 'linearHypothesis' onto my GARCH model at all.
Comment: I presume you can construct whatever hypothesis test manually using the parameter estimates and the variance/covariance matrix as the other answers have illustrated.
Answer: You can test the hypothesized value of $\beta_k$ much in the same way that it would be tested for $\beta_k = 0$. You can get a t-value on n-2 df with the formula:
$$T = \frac{\beta_i- hypothesezed~~ \beta_i}{Standard~~ Error~~ \beta_i}$$
In many cases this simplifies to beta over standard error because making the hypothesis be zero removes it from the equation.
Answer: Let's say you want to test the hypothesis of the form $R\beta = r$. In your case:
$$\underbrace{\left[ \begin{array}{cccc} \ldots & 1& 0 & \ldots \\ \ldots & 0& 1 & \ldots  \end{array}\right]}_R \underbrace{\left[ \begin{array}{c}\ldots \\ \beta_j \\ \beta_k \\ \ldots \end{array} \right]}_\beta = \underbrace{\left[ \begin{array}{c}-.5 \\ .5 \end{array} \right]}_r $$
Let $\hat{\beta}$ be your regression estimates of $\beta$. Under the condition that $\hat{\beta} \sim \mathcal{N}\left(\beta, \Sigma \right)$ conditional on data $X$ (eg. $\hat{\beta}$ asymptotically normal with mean $\beta$ and covariance matrix $\Sigma$) then the linear restrictions $R \beta = r$ can be tested with a $\chi^2$ test. Observe that $R\hat{\beta}-r$ would be normal with  $Var(R \hat{\beta} - r\mid X) = R \Sigma R'$. Then we would have:
$$(R\hat{\beta} - r)' \left( R \Sigma R' \right)^{-1} (R \hat{\beta} - r) \sim \chi^2_{\#r}$$
Where $\#r$ is the number of restrictions (in your case two).
The most classic way though is to do an F-test. This would take the form:
$$(R\hat{b} - r)' \left(R \Sigma R' \right)^{-1} (R\hat{b} - r) / (\#r)\sim F(\#r, n- k) $$
where $\Sigma$ is estimate of $Var(\hat{\beta} \mid X)$ and $\#r$ is the number of restrictions (note rank of R should be $\#r$.)
See Fumio Hayashi Econometrics p. 65-66

Testing linear restriction in R