How to include time in a mixed effects model

Question: I am trying to model the progress in proficiency of task completion over time. I would like to model specifically what is the effect of completing an evaluation and receiving feedback on the ability to sufficiently complete a task. Proficiency in that task is measured on an ordinal scale that I have categorized into "sufficient" or "insufficient".
At first, I used the following mixed effects model to model this outcome:
lme4::glmer(sufficient ~ time*scale(vol_evals) + complexity + (1|task) + (1|subjectID) + (1|raterID), data = df, family = binomial("logit"), control = lme4::glmerControl(optimizer= "bobyqa", optCtrl=list(maxfun=2e5)))

sufficient is a boolean value that is TRUE if the task is
completed sufficiently.
time is the day of the year (i.e. 1 to
365). This model is meant to model progress given feedback throughout
the duration of a year.
vol_evals is the number of
evaluations/feedback received by a given subjectID up to the time
point at which the task is being performed.
complexity is an ordinal factor (1-5) that denotes the difficulty of the task.
task is a given task and is modeled as a random effect (there are hundreds of unique tasks).
subjectID is a unique subject identifier. This is modeled as a random effect.
raterID is a unique rater identifier; this is the person providing the feedback. This is modeled as a random effect (the idea being different raters rate either more harshly or more easily.

My hypothesis is that given two otherwise identical individuals completing a task (same time of the year, same task, same complexity, same rater) the individual that has received more feedback up until that point in time (modeled by vol_evals) will be more likely to complete that task sufficiently.
The model listed above failed to converge. I therefore had to scale time:
lme4::glmer(sufficient ~ scale(time)*scale(vol_evals) + complexity + (1|task) + (1|subjectID) + (1|raterID), data = df, family = binomial("logit"), control = lme4::glmerControl(optimizer= "bobyqa", optCtrl=list(maxfun=2e5)))

The model output appears to make sense:
#> Scaled residuals:
#>     Min      1Q  Median      3Q     Max
#> -8.9518 -0.3503  0.2244  0.4313  3.7796
#>
#> Random effects:
#>  Groups    Name        Variance Std.Dev.
#>  task      (Intercept) 1.1871   1.0895
#>  raterID   (Intercept) 2.5451   1.5953
#>  subjectID (Intercept) 0.8894   0.9431
#> Number of obs: 9192, groups:  task, 871; raterID, 794; subjectID, 548
#>
#> Fixed effects:
#>                              Estimate Std. Error z value Pr(>|z|)
#> (Intercept)                   1.72910    0.13051  13.249  < 2e-16 ***
#> scale(time)                   0.24813    0.05201   4.771 1.84e-06 ***
#> scale(vol_evals)              0.32502    0.06046   5.376 7.61e-08 ***
#> complexity.L                 -0.90127    0.09190  -9.807  < 2e-16 ***
#> complexity.Q                 -0.13214    0.06343  -2.083   0.0372 *
#> scale(time):scale(vol_evals) -0.08059    0.05477  -1.471   0.1412
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Correlation of Fixed Effects:
#>             (Intr) scl(t) scl(_) cmpl.L cmpl.Q
#> scale(time) -0.075
#> scl(vl_vls)  0.253 -0.421
#> complexty.L -0.232 -0.031 -0.041
#> complexty.Q  0.203  0.006 -0.008 -0.418
#> scl(tm):(_) -0.067  0.323 -0.226 -0.001  0.008

The coefficients make sense - as time and number of evaluations increases, subjects are more likely to complete a task sufficiently. Likewise, as complexity increases, subjects are less likely to complete a task sufficiently.
However, I am not sure how to interpret scale(time). Nor am I certain this is the best way to deal with  the confounding influence of time (which is acquired by looking at the date an evaluation is filed). After all, I would expect people to improve over time regardless of if they are receiving feedback.
Is there a more appropriate way to control for the effect of time in this model?
Comment: What do you mean *"I am not sure how to interpret `scale(time)`"* ? You have literally interpreted it in the preceeding paragraph. Do you mean you are unsure how to interpret that it is *scaled*. Also note that since it is involved in an interation, it's interpretation is conditional on `vol_evals` being zero.
Comment: How would you describe a 1 unit increase in `scale(time)`? I'm struggling with how - given these results and time being scaled - to describe the effect of time on likelihood of sufficiently completing a task and how this relates to `vol_evals`.
Answer: scale(time) standardizes time so that it has a mean of zero and a standard deviation of 1.
This means that the regression coefficient for scale(time) is the association of a 1 standard deviation change in time (unscaled) with the change in the log odds of sufficient being true, and since it is involved in an interaction with scale(vol_evals) this is conditional on scale(vol_evals) being zero.

Is there a more appropriate way to control for the effect of time in this model?

Edit:  You might try to allow for a non-linear effect of time, for example by introducing a quadratic term


How to draw the Hasse diagram of the following poset?

Question: Which are the minimal and maximal elements?
The poset is: $(\{\emptyset,\{{\emptyset,1 \},\{1\}},\{1,2\}\},\subseteq) $
I assume, that it will look like a vertical chain, at the first stage from below with $\emptyset$, the second, $ \{\emptyset,1 \} $, the third $ \{1\}$, and the last one which is at the top of the chain will be $\{1,2\}$.
How could I draw Hasse-diagram with latex, is my solution correct in this way?
Answer: we have
$$(\{\emptyset, \{\emptyset, 1\}, \{1\}, \{1, 2\}\}, \subset)$$
At the very bottom we should have $\emptyset$ because
$$\emptyset \subset \{\emptyset, 1\}$$
$$\emptyset \subset \{1\}$$
$$\emptyset \subset \{1, 2\}$$
Then we have $\{1\}$ which should be straight up from there (nothing else at that level)
$$\{1\} \subset \{\emptyset, 1\}$$
$$\{1\} \subset \{1, 2\}$$
after that, we have two maximal elements $\{1, 2\}$ and $\{\emptyset, 1\}$
The lines should look like this:
\ /
|
Comment: yes, it is the minimal and the minimum (there's no maximum because there's two maximals and maximums must be unique).
Comment: I appreciate your fast answer, and can I say that $\emptyset$ is the minimal element?


How to compare a column with a cell id instead of value using google sheets conditional formatting option?

Question: I want to change the background color of a particular cell in a column if the value of that cell is greater than one cell in another column.
For example, the column that I want to compare all cells is Y and the cell that should be compared with is AA5
Right now I am manually comparing by adding the value of AA5 to the greater than format rule in google sheets conditional formatting.
I tried using =IF($Y:$Y>$AA5) and =$Y:$Y<$AA5 in the custom formula but didn't work.
Need help on this.
Comment: `=IF(Y1>AA$5, 1)` and `=Y1>AA$5` should work too
Comment: that worked, can you tell me how to add the background color to a row if the value in y column and z column is greather than AA5 for Y value and AA7 for Z value, I tried `=AND($Y:$Y>AA5,$Z:$Z>AA8)` this but didn't work?
Answer:
apply to range Y1:Y
select Greater than
custom formula: =AA$5
Comment: that worked, can you tell me how to add the background color to a row if the value in y column and z column is greather than AA5 for Y value and AA7 for Z value, I tried `=AND($Y:$Y>AA5,$Z:$Z>AA8)` this but didn't work.
Comment: @playero thanks, but I want the full row to be colored only if `Y` value is greater than `AA5` and `Z` is greater than AA8 in the same row. I mean it should satisfy both conditions.
can you please give me that rule?
Comment: @chijumelveettil `=AND(IF($Y1>$AA$5,1),IF($Z1>$AA$8,1))` https://i.stack.imgur.com/1PS2w.png
Comment: superb, thanks! one issue is that blank cells are also considered as greater than value, please check this screenshot, https://snag.gy/M9jhTk.jpg
Comment: is there a way to exclude blank columns?
Comment: @chijumelveettil try: `=AND(IF($Y1>$AA$5,1),IF($Z1>$AA$8,1),IF(y1=NOT(ISNUMBER($y1)),1))` https://i.stack.imgur.com/Qjy7H.png
Comment: Thanks a lot bro, I appreciate the help.


CSV file not recognised

Question: I am creating a CSV file in my system and MFT to another. When they receive it, their job does not pick up the file. When they open the file in excel, save it locally and reload the same file, the job picks up the records. I can't figure out what could be wrong with the file I create or something wrong with their job? Anyone experienced something similar?
Appreciate any ideas.
Thanks
Answer: With such a few details no one rather than you can find what's wrong.
My suggestion would be to get two files: one - the original file that the job does not want to deal with; second - the file that the job can consume (saved via Excel). Then open this two files in a notepad and try to find any differences.
Comment: Hi Sergey I have already done that and WinDiff does not pickup differences. CSV file is created by Java.
Comment: This sounds weird. If there's no difference in a files then try to manually "feed" the job with your original file.Check if that works.
Comment: I think it might be about line breaks of the file. When the file is opened with Excel, resaved and opened with NotePad, lines are formatted. When Java creates the file, when the output opened in NotePad, they are not line separated, even though I have /n. how can I make sure they are line formatted in Java?
Comment: I'm not really a Java guy but try "\r\n" or better System.lineSeparator()
Comment: Hi.. Issue solved. we were using /n and it wasn't creating new line.. replaced with /r/n and good to go

CSV file not recognised


Estimate for convolution

Question: Let $f \in L^p(\mathbb R)$ for $1 \leq p < \infty$, $\varphi \in L^1(\mathbb R)$ and $A \subseteq \mathbb R$ Borel-measurable with $\lambda(A) < \infty$. I want to show that
$$\int_{\mathbb R} \int_{\mathbb R} \vert \varphi(y) f(x-y) \chi_A(x)\vert\,dy\,dx < \infty \qquad (1)$$
to deduce that $g_x \in L^1(\mathbb R)$, $g_x(y) = \varphi(y) f(x-y)$ for $x \in \mathbb R$ almost everywhere.
But I don't know how to do that. I tried to use Fubini to get
\begin{align*}
\int_{\mathbb R} \int_{\mathbb R} \vert \varphi(y) f(x-y) \chi_A(x)\vert\,dy\,dx &= \int_{\mathbb R} \vert \varphi(y) \vert \int_{\mathbb R} \vert f(x-y) \chi_A(x)\vert\,dx\,dy \\
&= \int_{\mathbb R} \vert \varphi(y) \vert \int_{A} \vert f(x-y) \vert\,dx\,dy \\
&= \int_{\mathbb R} \vert \varphi(y) \vert \int_{A - y} \vert f(x) \vert\,dx\,dy
\end{align*}
Now I would like to bound $\int_{A - y} \vert f(x) \vert\,dx$ by a constant but I don't see how this works. Another idea of me was to approximate $f$ by elements from $C^\infty_c(\mathbb R)$, because I can show that $(1)$ holds for $f \in C^\infty_c(\mathbb R)$ but I can't deduce $(1)$ for $f \in L^p(\mathbb R)$ from that.
Also I have no idea how to deduce $g_x \in L^1(\mathbb R)$ from $(1)$. I would appreciate some hints :)
Answer: You have to use Minkowski's inequality
for integrals (see the bottom of this page Minkowski) and Fubini's theorem :
\begin{align*}
\left\Vert \int_{\mathbb{R}}\left\vert f\left(  \cdot-{y}%
\right)  \varphi\left(  {y}\right)  \right\vert \,d{y}%
\right\Vert _{L^{p}} &  \leq\int_{\mathbb{R}}\left\Vert f\left(
\cdot-{y}\right)  \varphi\left(  {y}\right)  \right\Vert
_{L^{p}}\,d{y}\\
&  =\int_{\mathbb{R}}\left\vert \varphi\left(  {y}\right)
\right\vert \left\Vert f\left(  \cdot-{y}\right)  \right\Vert
_{L^{p}}\,d{y}=\left\Vert f\right\Vert _{L^{p}}\int_{\mathbb{R}}\left\vert \varphi\left(  {y}\right)  \right\vert \,d{y},
\end{align*} where the last equality follows from the fact that the Lebesgue measure is
translation invariant. Since an $L^p$ function is finite $\mathcal{L}^1$ a.e. everywhere you get
$\int_{\mathbb{R}}\left\vert f\left(  {x}-{y}%
\right)  \varphi\left(  {y}\right)  \right\vert \,d{y}<\infty$
for $\mathcal{L}^{1}$-a.e. ${x}\in\mathbb{R}$.
Answer: Simply use Hölder's inequality to derive
\begin{align*}
\int_{A-y} |f(x)| dx
&= \int |f(x)| \cdot 1_{A-y} (x) dx \\
&\leq \| f \|_{L^p} \cdot \| 1_{A-y} \|_{L^{p'}} \\
&= \| f \|_{L^p} \cdot \| 1_{A} \|_{L^{p'}},
\end{align*}
where the right-hand side is independent of $y$.
To deduce that $g_x \in L^1$ for almost all  $x $, note that the expression in (1) can be written as
$$
\int \| g_x \|_{L^1} \cdot 1_A (x) dx.
$$
Now, shot that if $\int h (x) 1_A (x) dx < \infty $ for some $h \geq 0$ and  ever set $A $ of finite measure, then $h (x) < \infty $ almost everywhere.

 Hint:  Consider $B = \{x \, : \, h (x)=\infty\} $. It suffices to show that $B $ is a null set.

$ $

 Further hint: It suffices to show that each $A_\ell = B \cap [-\ell, \ell] $ is a null set.
Comment: But how can I deduce $g_x \in L^1(\mathbb R)$ from that? I don't see it.
Comment: @Yaddle: I updated my answer.

Estimate for convolution


getting format of JFormatTextField

Question: I've several text fields with certain formats: percent format, date format, etc:
     DateFormat dfm= new SimpleDateFormat("yyyy-MM-dd");
     textField = new JFormattedTextField(dfm);

Later I want to check what format is being used in a textField. I have checked if there is a method textField.getFormat() or similar. Didn't see anything relevant. Is there a way to know this?
Answer: As you can see in the source code, your Format is converted to a JFormattedTextField.AbstractFormatterFactory
public JFormattedTextField(java.text.Format format) {
  this();
  setFormatterFactory(getDefaultFormatterFactory(format));
}

There is a getter available on JFormattedTextField to retrieve this JFormattedTextField.AbstractFormatterFactory afterwards. This JFormattedTextField.AbstractFormatterFactory gives you access to a JFormattedTextField.AbstractFormatter.
However, you can no longer access your actual Format instance. Depending on why you need it, the JFormattedTextField.AbstractFormatter might be sufficient
Comment: If the given textfield uses date format then when submit button is pressed, text retrieved from textfield.getText(),which is String, needs to be formatted back to DateFormat:

   DateFormat dfm= new SimpleDateFormat("yyyy-MM-dd");
   Date a = dfm.parse(textfield.getText());
Comment: +1 @Nazerke: Also consider `InputVerifier`, seen [here](http://stackoverflow.com/a/2241997/230513).
Comment: interesting that official Oracle tutorial (as always) say about

getting format of JFormatTextField


Block access to C Drive Windows XP and 2003 server

Question: I have tried everything to prevent users from accessing the C: on the network through group policy but they always find a way around it (students).
They cant right click to create shortcuts but they have the shortcuts on USB and are able to run the shortcuts to access the c:.
There isnt a specific group policy that prevents access to the C drive.
Any thoughts?
Using windows xp clients with 2003 server for AD and GPolicy
Comment: NTFS perms, perhaps? Maybe I'm not understanding your objective here.
Comment: The objective is a false sense of "security" by having the Explorer GUI hide things. What the poster is really looking for is restricting the user's ability to execute arbitrary code, which could be satisfied with "Software Restriction Policies" in a real and substantive way. Hiding things is security-by-obscurity and is counter-productive (since it will make you feel secure while not actually being secure).
Answer: We do this all the time with our terminal servers, just in case someone somehow manages to figure out how to open a traditional browse dialogue, Windows simply pops up a message saying that the current policy is denying them access.
In your Group Policy, find the following:

User Configuration
Administrative Template
Windows Components
Windows Explorer

"Hide these specified drives in My Computer"
"Prevent access to drives from My Computer"

It's been a while since I did it but I'm fairly sure these are the settings you will require. There are ways to override the default options provided, so that you can modify the drives past the fairly restrictive options provided if you need them.
Comment: Perfect Worked very well, Cheers
Comment: Had an issue that forced me to disable it.  There was a computer lab that uses autocad and it needs access to the libraries on the C: which caused it not to work.  I will have to apply a loopback policy to all other devices to run this GP.
Comment: You could assign a GPO to just the machines in that lab that overrides this property. Gotta love those legacy apps!
Answer: Another alternative is to let them have access. We ended up curbing their behavior (well, destructive behavior) using Deep Freeze from Faronics; it lets you "freeze" the hard disk and then on reboot it goes back to the default state. It was cathartic when we tested it by wiping the Windows subdirectory until Windows kind of lurched and fell over...then rebooted and it came back up.
That allows the students to do whatever they want without destroying your configuration; we also found that it eliminated a lot of corruption issues with profiles and caches.
I think there are a couple other alternatives out there but I only have used DeepFreeze. There's a central console used to monitor workstations and can remotely thaw and freeze them, we also use it to get IP addresses for particular workstations.
I've worked in labs that were locked down to the point of being nearly unusable for anything. Even in class we couldn't get, for example, certain documents to open or use notepad to read source code because it wasn't allowed by the sysadmin who happened to be two hours away in a central site, and the local admin guy lamented not being able to fix certain issues without prearranging things with the central admins. And this was a college. Deep Freeze gave more freedom for getting work done without interference (it also allows at our site for certain people to get higher privileges on the local machine since any malware or alterations are erased at reboot; if they're going out of their way to install "personal" software they grow weary of having to constantly reinstall at each reboot; at least that's what we've seen).
We tried locking out things through policy when we ran 2000 terminal services; we found that the policy only locked out access from certain things like explorer. It looked like some programs (like an old copy of file manager from the 3.x days) was able to still browse the C: drive, along with some freeware file managers! Unless that was fixed, it looks like certain APIs still allowed users to get access to local drives that were supposed to be blocked out by policy. Plus we have systems that for reasons we never understood will sometimes not get the policy right away or act as if they wouldn't get any policy updates until after a reboot or two, so sometimes we had people that could bypass configuration lockouts and do things they weren't supposed to.
Comment: We do the same thing with DF.  The logged in lab user even has admin rights on the box! It's amazing the kinds of things you can find installed, so we don't limit users much at all, but we can always get back to a good state.
Comment: And, of course, Windows Steady State is a free alternative, if anyone is looking
Comment: Yeah We already use deep freeze but the kids put games on the C: and play them.
I have group policy to block MD5 of games they play but when a new version comes out it is a game of cat and mouse.
But Depp freeze saves the day.
Thanks for the tip about Windows Steady State.
Comment: If you really want to lock it down, you can whitelist the executables with another tool from Faronics. But like I said, I've tried working in a school where things were fanatically locked down and it was nigh impossible to actually get anything done. Many of the students opted not to use the computers unless they had to. All that equipment hardly utilized...


Manually disable a module

Question: In Drupal 7, I could manually edit the {system} table in the database to disable a stubbon module.  In my Drupal 8 site, this table is gone.
How can I manually disable a module in Drupal 8?
Answer: The Drupal 7 system table data is now stored in the config table in Drupal 8 against the core.extension parameter.
Solution 1: Update configuration
You can run the following code using drush eval or may be using the Devel module's provision to Execute PHP Code.
// Read the configuration.
$module_data = \Drupal::config('core.extension')->get('module');

// Unset the modules you do not need.
unset($module_data['MODULE_NAME']);

// Write the configuration.
\Drupal::configFactory()->getEditable('core.extension')->set('module', $module_data)->save();

You can do this all in a quick one-liner with drush.
drush eval "\$module_data = \Drupal::config('core.extension')->get('module'); unset(\$module_data['MODULE_NAME']); \Drupal::configFactory()->getEditable('core.extension')->set('module', \$module_data)->save();"

Solution 2: Edit config table if you cannot execute PHP
If the site is broken because of the problematic module and you can't even run PHP code, then may be you can edit the config table directly.
In the row in the config table where name = "core.extension" and edit the BLOB column data. The data is a serialized PHP array where you have to remove the module you want to get rid of from the module key of the configuration.
Solution 3: Fast and dirty solution

Remove the module from the file system
Truncate the table cache_config

However, this solution might result in messages saying the module does not exist in file system which mean something is wrong. But at least the broken module gets disabled and you can access your site in most of the cases.
Clearing Cache
Sometimes you might have to clear cache after following the above steps. Read this handy documentation on how to clear cache.
Comment: If you need to uninstall a module but can't because the tables for the module are missing, you can use the third option with `drush eval`. An example one-liner is: `drush eval "\$module_data = \Drupal::config('core.extension')->get('module'); unset(\$module_data['example_module']); \Drupal::configFactory()->getEditable('core.extension')->set('module', \$module_data)->save();` Note the escaped dollar sign so that the command line doesn't misinterpret `$module` as a Bash variable.
Comment: In terms of uninstalling modules I think this is the best solution.
Comment: After doing all the solutions listed there was still a system.schema entry in the key_value table for  the missing module. Deleting that cleared out the warning.
Comment: Cheers for the lazy one liner! Missing a quote at the end though
Comment: It's pretty much essential you do a cache clear, and then export config so the config doesn't try to re-install until you're ready.
Comment: I was getting a WSOD when going to the site's Status Report due to missing module (yet another D8 UX fail). I tried all 3 solutions above. Although they did fix the WSOD on the Status page; when going to update.php, i still see the error. After cr, cron and updb; the warning still shows. So solutions listed above are not complete.
Answer:
Manually edit the config table where name = 'core.extension' and
remove the module from the data blob which is a serialized array.
Be aware to decrement the module array length as well (...s:6:"module";a:HERE;{...)
Truncate the cache_config table from phpmyadmin or using command-line.
Comment: This answer helped solve a situation where a module could neither be uninstalled nor re-installed because a custom entity it defined was missing.
Comment: Thanks, this helped me. The answer should read "Manually *edit* the {config} table". The text portion to remove from the BLOB content is: `i:0;s:8:"name of the module";`
Comment: @hendrick this may have worked for you but the string to remove each module should be structured as `s:8:"name of the module";i:0;`. What @valli meant was that the array describing the number of modules should be decremented in number by the amount of modules removed. The begining of the blob in my setup is `a:4:{s:6:"module";a:59:{`, which is an array of 59 modules. If you delete two change that array value to 57.
Comment: If you don't want to delete the module from the serialized value manually, use this php snippet to remove it:
`$a = 'DATA_COLUMN_FOR_CORE_EXTENSION_ROW';

$b = unserialize($a);
unset($b['module']['MACHINE_MODULE_NAME']);
$c = serialize($b);
print $c;`
Answer: Do this:
rm -rf modules/your_stubborn_module
rm -rf sites/default/files/php

Comment: You have to run `drush cache-rebuild` too
Comment: Thanks for this: this worked on our Drupal version "8.9.13". We, of course, had to purge the cache tables (cache_) as well.
Comment: Depending on the module, you might have to delete some entries from the config table (see valli's answer) and some other tables.
Comment: This didn't work for me.
Answer: If you need to update anything related to Drupal configuration, in this case core.extension, use Drush:
[Drush 8.x in this example]
drush cedit core.extension

Answer: In Drupal 8, try to removing the module from your module folder and run rebuild.php.
Try drush pm-uninstall module-name as well.
Answer: Consider using Drush. Drupal 8 is still defining what "disabling modules" should be. There is an ongoing discussion if there should be that option or it should be removed.
Comment: I'm developing locally on a windows PC and last time I looked Drush wasn't ready for Drupal 8. I'll have to look again.
Comment: @bumpaw Drush should work on Windows as well. That will at least help you locally. When it comes to production servers, I know many shared hostings don't provide you with Drush or even SSH which usually makes the management way more cumbersome. http://www.drush.org/drush_windows_installer
Comment: And,  using drush,  how does one "manually remove" a module in a way that is different than the UI process?  You don't appear to have answered the question.
Answer: I tried all of the other answers but kept getting a drupal error message. To solve it I had to remove a row from the key_value table (look for the module name in the name column)
Answer: This is how, I manually removed a module named "better_messages" from my Drupal 8 instance. As soon as I installed the "better_messages" module, the site went down. So there was no way to uninstall the module from user interface. I dont have Drush installed. I have done many settings given in forums, but this is how it finally worked for me.
1 Renamed the module to old_better_messages in modules folder.

Through the url, ran http://IP:port/foldername/rebuild.php. This ensured that site is back, but only in read-only mode. I couldnt do the admin activities or edit articles.
Used following command to delete the entry from database

DELETE FROM key_value WHERE collection='system.schema' AND name='better_messages';
In my case, there was no entry in the database. I think it might have got deleted because of the various stunts that I did earlier.

Then using DB visualizer, I deleted entries from all the tables that starts with cache.

This resolved the issue.
This is based on my interpretation of https://www.drupal.org/node/2487215
Comment: your solution worked like a charm! Thank you for sharing.
Answer: There is a module for that. This module was posted in Aug-2013 on drupal.org. In case somebody needs.
Disable modules
As stated on the page of this module,

Drupal 8 has removed the ability to disable modules for many reasons.
  See #1199946: Disabled modules are broken beyond repair so the
  "disable" functionality needs to be
  removed and many other issues in
  the queue of various core and contributed modules.
This module brings back the ability to (temporarily) disable modules
  from the UI or with Drush. Note, that there's no guarantee for your
  content, config or even your site after you disable a module.
Answer: Jigarius' answer above, sort of worked...
I had to: // Read the configuration.
$module_data = \Drupal::config('core.extension')->get()['module'];

Which should do the same thing. Not sure why it didn't work as Jigarius wrote it...

Manually disable a module


Function/Code in R that is a class() variant of colnames()

Question: I am comparing multiple data frames in R, each with multiple columns that should have the same name and the same data type. I know how to simply code for a TRUE/FALSE result in checking for matching column names, and I have a method for checking matching column data types, but the data types method requires exponentially longer code than the column names method and I want a simpler solution.
I can verify column names as matching with (code_1):
all(c(
  all(colnames(df_1)==colnames(df_2)),
  all(colnames(df_1)==colnames(df_3))
))

But to verify data types as matching, this is the best I could come up with (code_2):
all(c(
  all(c(
    class(df_1$column_1)==class(df_2$column_1),
    class(df_1$column_2)==class(df_2$column_2)
  )),
  all(c(
    class(df_1$column_1)==class(df_3$column_1),
    class(df_1$column_2)==class(df_3$column_2)
  ))
))

Ideally, I would want something like this that can do the equivalent of code_2 (code_3):
all(c(
  all(xyzfunction(df_1)==xyzfunction(df_2)),
  all(xyzfunction(df_1)==xyzfunction(df_3))
))

I don't know if such an "xyzfunction" exists though and can't find such a function with my research. Do any of you know of such a function or a simpler way to write code_2 when you have many more data frames and columns per data frame?
Answer: You can't use == for comparing the return from class, since the length of its return is variable:
class(1)
# [1] "numeric"
class(Sys.time())
# [1] "POSIXct" "POSIXt"
class(ggplot2::diamonds)
# [1] "tbl_df"     "tbl"        "data.frame"

It should be safe-enough to use identical, though.
xyzfunction <- function(x, y) {
  if (!identical(sort(names(x)), sort(names(y)))) return(FALSE)
  nms <- names(x)
  return(identical(lapply(.subset(x, nms), class), lapply(.subset(y, nms), class)))
}

I use .subset instead of x[nms] since x[nms] works with data.frame and tbl_df but not with data.table, and subset itself is intended more for interactive use, not programmatic unsupervised use like this.


view_get_views returns null

Question: I have a module function which needs to get the raw results of a view, so I use this bit of code:
<code>$view = views_get_view('sluitingsuren_bakker');
  $view->preview();

  return $view->result;
</code>
The views machine name is sluitingsuren_bakker. I also tried some other views, but it seems the module can't find any of the views.
Also I set my permissions on this view to none, because I thought that was the problem.
Answer: You are missing the display name, which is the machine name of the block/page.
You can find the machine name under Advanced settings in the right side of the view.
try:
<code>   <?php
    $view = views_get_view(view_name);
    $view->set_display(machine_name);
    $view->preview();
    return $view->result
    ?>
</code>

view_get_views returns null


react-native run-android android-sdk does not exists

Question: I am doing the react-native Getting Started on a Windows 10 system.  I was able to do the react-native init AwesomeProject command to work and now I need to execute:
>react-native run-android

But that fails with this error:
FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':app'.
> The SDK directory 'C:\Users\plankton\AppData\Local\Android\android-sdk' does not exist.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 2 mins 0.64 secs
Could not install the app on the device, read the error above for details.
Make sure you have an Android emulator running or a device connected and have
set up your Android development environment:
https://facebook.github.io/react-native/docs/android-setup.html

I think the problem is that I have installed the latest Android SDK and now the path is this: C:\Users\plankton\AppData\Local\Android\sdk and NOT C:\Users\plankton\AppData\Local\Android-sdk.  If I where on a Unix like OS I would just make a syslink.  But I am using Windows 10.  What can I do? How do you make syslinks in Windows?
Answer: On Windows it looks at the ANDROID_HOME environment variable. You should be able to just update that, close out the command prompt or powershell and restart it then try again.
Answer: In Android local.properties Inside Change your
Sdk Location And solve this issue
1 react-native start --reset-cache
and go to android folder
cd android
and then clean gradle
2 gradlew clean
and Make Sure Uninstall the Previous app


$\vDash (\forall x \forall y \forall z (R(x,y) \land R(y,z)\rightarrow R(x,z)))\land (\forall x \exists y R(x,y)) \rightarrow \exists x R(x,x) $

Question: Prove that the following formula is true in every structure or construct  a structure as a counterexample where the formula is not true:
$(\forall x \forall y \forall z (R(x,y) \land R(y,z)\rightarrow R(x,z)))\land (\forall x \exists y R(x,y)) \rightarrow \exists x R(x,x)$
I have tried to show that for an arbitrary structure that this formula is true for every truth assignment. This got a mess very quickly and I'm not sure if it is correct what I did. My opinion so far is hat this formula is true in every structure because I didn't manage to construct a counterexample.
Does anyone has a hint or an advise to approach exercises of this kind?
Comment: Do you intuitively see what the antecedents say about the relation $R$? What are some examples that satisfy the antecedent?
Answer: First, some intuitions. The hypothesis $\forall x \forall y \forall z (R(x,y) \land R(y,z) \to R(x,z))$ means that $R$ represents a binary relation that is transitive. A typical example of a transitive relation is the strict order relation $<$.
The hypothesis $\forall x \exists y R(x,y)$ means that such a transitive relation $R$ is "unbounded upwards". If you keep in mind the intuition of $<$, this condition is satisfied by the strict order relation $<$ over an infinite totally ordered set without greatest element, for instance $\mathbb{N}$.
But the strict order relation $<$ is not reflexive, i.e. it does not satisfies the thesis $\exists x R(x,x)$. This should convince you that the formula $\big( \forall x \forall y \forall z (R(x,y) \land R(y,z) \to R(x,z)) \land \forall x \exists y R(x,y) \big) \to \exists x R(x,x)$ is not valid, i.e. there is a structure that does not satisfy this formula.
Formally, let $\mathcal{N} = (\mathbb{N}, <)$ where the set $\mathbb{N}$ of natural numbers is the domain of $\mathcal{N}$ and the usual strict order relation over $\mathbb{N}$ is the interpretation in $\mathcal{N}$ of the binary symbol $R$. As we have seen above, $\mathcal{N} \not \vDash  \big(\forall x \forall y \forall z (R(x,y) \land R(y,z) \to R(x,z)) \land \forall x \exists y R(x,y) \big) \to \exists x R(x,x)$.
Comment: Another way to phrase $\forall x \exists y.Rxy$ is "there are no sinks".


How do I use command-line options in Ruby?

Question: How do I use command options in scripts?
I have a script that connects to a device, runs commands and prints the output. I would like to use a file for the hosts, and one for the user and passwd info. When I try to run the script I get these errors:

ruby threat_detection.rb --host_file=hosts --config_file=config
threat_detection.rb:61:in '': needless argument: --host_file=hosts (OptionParser::NeedlessArgument)

ruby threat_detection.rb '--host_file=hosts' '--config_file=config'
threat_detection.rb:61:in '': needless argument: --host_file=hosts (OptionParser::NeedlessArgument)

ruby threat_detection.rb --host_file-hosts --config_file-config
threat_detection.rb:61:in '': invalid option: --host_file-hosts (OptionParser::InvalidOption)

This is the code:
options ={}

opt_parser = OptionParser.new do |opt|
  opt.banner = 'Usage: opt_parser COMMAND [OPTIONS]'
  opt.on('--host_file', 'I need hosts, put them here') do |host_file|
    options[:host_file] == host_file
  end
  opt.on('--config_file', 'I need config info, put it here') do |config_file|
    options[:config_file] == config_file
  end
  opt.on('-h', '--help', 'What your looking at') do |help|
    options[:help] == help
    puts opt
  end
end

opt_parser.parse!

How do I make these options get read? This is my guess (based off python experience). I'm just looking for it to print the lines right now:
if :config_file == true
  File.open(:config_file, r) do |params|
    puts params
  end
end

if :host_file == true
  File.open(:host_file, r) do |host|
    put host
  end
end

Answer: For taking arguments you need to use the following formats
"--switch=MANDATORY" or "--switch MANDATORY" # mandatory argument
"--switch[=OPTIONAL]"                        # optional argument
"--switch"                                   # no argument

You are currently using the third format, which is being interpreted as taking no argument. You should use the first or second.
Also worth noting you probably want to be doing an assignment instead of a comparison when a flag is on
You want this
options[:host_file] = host_file

not this
options[:host_file] == host_file


Bulk Update Meta Tags?

Question: I was wondering if it is possible to bulk update the meta tags for a particular content type? I have set the default to this: [node:field_description]. But all of the content is in the site. I'd prefer to not go through every node to paste that in the Meta tags Description field.
Is there a bulk operation to put that placeholder token into all the nodes of a particular content type?
Using this module: https://drupal.org/project/metatag The Metatag module on Drupal 7.
Comment: What meta tag module are you using? And what Drupal version?
Comment: Hello. Please add important things directly to a question. This is not a discussion forum, comments may disappear without notice.
Comment: Ah sorry! This one: https://drupal.org/project/metatag The Metatag module.
Answer: You may use combination of Metatag, Views and Editable Fileds.

Configure node types to use own fields for meta tags like
keywords, description, abstract, etc. and create it. use Field
Permission to restrict access to this fields.
Configure Metatag to    use newly added fields for filling metatags
on Metatag admin page    or node type page.
Create view with table    display, add    metatag-fields. Choose
formatter "Editable" on    each field    settings.
Set path for this    view like admin/config/search/metatags/massedit. Save view and now you    can mass    edit
any fields.

Also you can try Editable Views module.
Answer: Sadly there does not seem to be a way to bulk update the meta tags without some database or PHP work.
You're best bet would be to look at the structure in metatags table, and use PHP to join it with the node table on the type you want and manually update it using the database & serialize to put it back together. You may want to copy an already done serialized array from the database so you replicate the content.
But that may not be the best idea incase some have special parameters or something.

Bulk Update Meta Tags?


@Transaction annotation for integration testing in Spring Boot

Question: I have the following method in a JpaRepository that simply updates the parent_id column for every row with a specific value in that column. It works perfectly in pure SQL but it failed in Spring Data. I think it's due to just some problem with the transaction scope, because the update is done (first assert passes). It's just I can't see what code is changed in DB. I want to use the
@Transactional
@Rollback

because I guess this is best practice. Is there any way I can see what's don in the code from my test method?
@Modifying
@Query("update MovementCategory mc set mc.parentId = :newParentId where mc.parentId = :previousParentId and mc.userId = :userId")
int updateChildrenParentId(@Param("userId") long userId, @Param("previousParentId") long previousParentId, @Param("newParentId") long newParentId);

I've created a simple integration test that checks that changes are correctly set in DB, but it doesn't seem to work and I can't get to see why. I thought it could be because of transaction scope but I did a smaller test and discarded it, so no idea. The following is the integration test.
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = MySpringBootMavenApplication.class)
@WebAppConfiguration
@IntegrationTest("server.port:0")
@Transactional
@Rollback
public class MovementCategoryRepositoryIT {

private static final long USER_ID = -1L;

@Autowired MovementCategoryRepository repo;

@Test
public void testUpdateChildrenParentId() {

    long newParentId= -9723854;
    long previousParentId = -1239842;

    MovementCategory mc1 = getFilled(null, "DELETE ME");
    mc1.setParentId(previousParentId);
    mc1 = repo.saveAndFlush(mc1);

    int updates = repo.updateChildrenParentId(USER_ID, previousParentId, newParentId);

    // First assert passes, so there update is done
    assertEquals(1, updates);

    MovementCategory children1 = repo.findOneByIdAndUserId(mc1.getId(), USER_ID);

    // Second one fails
    assertEquals(newParentId, children1.getParentId().longValue());

}

Result:
java.lang.AssertionError: expected:<-9723854> but was:<-1239842>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:834)
    at org.junit.Assert.assertEquals(Assert.java:645)
    at org.junit.Assert.assertEquals(Assert.java:631)
    at com.ldepablo.repository.MovementCategoryRepositoryIT.testUpdateChildrenParentId(MovementCategoryRepositoryIT.java:171)
    ...

Answer: You need to start the transaction for reading explicitly and also roll back the transaction explicitly as the REST calls are in a different session than the test. Use TransactionTemplate as described in this answer.
(Answering too late, of course; but this was the first hit on the search engine)


Convergence in quotient space $X/Y$

Question: Let $Y$ is closed subspace of $X$ and $(x_n+Y)\rightarrow (x+Y)$ in $X/Y$ for some $x_n,x\in X$.
Question is to see that there exist $y_n\in Y$ such that $x_n+y_n\rightarrow x$.
This seems to be true but i could not give a proof.
Given $\epsilon>0$ there exists $N\in \mathbb{N}$ such that $||x_n+Y-(x+Y)||<\epsilon$ for all $n\geq N$
Fix $n\geq N$ then $||(x_n-x)+Y||<\epsilon$ i.e., $\inf\{||x_n-x+y||:y\in Y\}<\epsilon$  so, I can get $y_n\in Y$ such that $||x_n-x+y_n||<\epsilon$.
I thought this sequence $(y_n)_{n\geq N}$ adding some random $y_1,\cdots,y_{N-1}$ would work. But then i see that these $y_n$ even for $n\geq N$  depends on $\epsilon$.
Suggest some hints to get rid of this $\epsilon$.
Answer: For each $n$, you can find $y_n\in Y$ with
$$\Vert x_n-x+y_n\Vert\leq\Vert (x_n+Y)-(x+Y)\Vert+1/n$$
by definition of the norm on the quotient space.

Convergence in quotient space $X/Y$


Teste Unitário de ModelAndView Controller com Mockito e Spring

Question: Preciso montar o teste unitário para a controller abaixo (apenas para exemplo):
import javax.servlet.http.HttpServletRequest;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.ModelAndView;

@RestController(value = "/test")
public class TestController {

    @GetMapping(value = "/list")
    public ModelAndView list(ModelAndView model, HttpServletRequest request) {
        model.addObject("returnedAttribute", request.getSession().getAttribute("test"));
        return model;
    }
}

Recebi a sugestão abaixo do colega @nullptr em outro tópico:
@Test
public void testIniciaTela() {

    ModelAndView modelAndView = new ModelAndView();
    ModelAndView modelResultado = controller.iniciaTela(modelAndView, getMockServletRequest());
    assertEquals(
        "1234",
        modelResultado.getModelMap().get(AbstractConstantes.PERFIL_USUARIO_LOGADO)
    );

}

private HttpServletRequest getMockServletRequest() {
    MockHttpServletRequest mockRequest = new MockHttpServletRequest(); //aqui está dando erro...
    mockRequest.setSession(getMockSession());
    return mockRequest;
}

private HttpSession getMockSession() {

    PerfilUsuarioLogadoVO perfilUsuarioLogado = new PerfilUsuarioLogadoVO();
    perfilUsuarioLogado.setNuUnidade("1234");

    MockHttpSession mockSession = new MockHttpSession();
    mockSession.setAttribute(AbstractConstantes.PERFIL_USUARIO_LOGADO, perfilUsuarioLogado);

    return mockSession;

}

Mas estou recebendo esse erro:
java.lang.NoSuchMethodError: org.springframework.util.StreamUtils.emptyInput()Ljava/io/InputStream;
at org.springframework.mock.web.MockHttpServletRequest.<clinit>(MockHttpServletRequest.java:102)
at xxx.controller.ManterEnvioManualControllerTest.getMockServletRequest(ManterEnvioManualControllerTest.java:110)

ao tentar executar a linha abaixo:
MockHttpServletRequest mockRequest = new MockHttpServletRequest();

Meu arquivo pom.xml está assim:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
    <groupId>br.xxx.sigms</groupId>
    <artifactId>sigms</artifactId>
    <version>4.0.0-SNAPSHOT</version>
</parent>
<artifactId>sigms-web</artifactId>
<packaging>war</packaging>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<name>SIGMS - WEB</name>
<dependencies>
    <dependency>
        <groupId>br.xxx.sigms</groupId>
        <artifactId>sigms-common</artifactId>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>br.xxx.sigms</groupId>
        <artifactId>sigms-dao</artifactId>
        <type>ejb</type>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>br.xxx.sigms</groupId>
        <artifactId>sigms-ws</artifactId>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.jboss.logmanager</groupId>
        <artifactId>log4j-jboss-logmanager</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>commons-lang</groupId>
        <artifactId>commons-lang</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>commons-beanutils</groupId>
        <artifactId>commons-beanutils</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.codehaus.groovy</groupId>
        <artifactId>groovy-all</artifactId>
        <version>${version.groovy.all}</version>
    </dependency>
    <dependency>
        <groupId>org.jboss.spec.javax.ejb</groupId>
        <artifactId>jboss-ejb-api_3.2_spec</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-context</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-aop</artifactId>
        <version>${version.spring.aop}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-web</artifactId>
        <version>${version.spring.web}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-webmvc</artifactId>
        <version>${version.spring.webmvc}</version>
    </dependency>
    <dependency>
        <groupId>org.codehaus.jackson</groupId>
        <artifactId>jackson-mapper-asl</artifactId>
        <version>${version.jackson.mapper.asl}</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-core</artifactId>
        <version>${version.jackson.core}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>${version.jackson.databind}</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.springframework.ldap</groupId>
        <artifactId>spring-ldap-core</artifactId>
        <version>${version.spring.ldap.core}</version>
    </dependency>

    <dependency>
        <groupId>org.sitemesh</groupId>
        <artifactId>sitemesh</artifactId>
        <version>${version.sitmesh}</version>
    </dependency>
    <dependency>
        <groupId>commons-pool</groupId>
        <artifactId>commons-pool</artifactId>
        <version>${version.commons.pool}</version>
    </dependency>
    <dependency>
        <groupId>org.jboss.spec.javax.servlet</groupId>
        <artifactId>jboss-servlet-api_3.1_spec</artifactId>
    </dependency>
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>jstl</artifactId>
        <version>${version.jstl}</version>
    </dependency>
    <dependency>
        <groupId>javax.validation</groupId>
        <artifactId>validation-api</artifactId>
        <version>${version.javax.validation.api}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-validator</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>br.xxx.sigms</groupId>
        <artifactId>sigms-core</artifactId>
        <version>${project.version}</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>spring-security-web</artifactId>
        <version>${version.org.spring.framework.sucurity}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>spring-security-config</artifactId>
        <version>${version.org.spring.framework.config}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>spring-security-taglibs</artifactId>
        <version>${version.org.spring.framework.sucurity}</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>spring-security-ldap</artifactId>
        <version>${version.org.spring.framework.ldap}</version>
    </dependency>

    <dependency>
        <groupId>net.sf.jasperreports</groupId>
        <artifactId>jasperreports</artifactId>
        <version>${version.jasperreports}</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>com.thoughtworks.xstream</groupId>
        <artifactId>xstream</artifactId>
        <version>1.4.8</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <version>${version.org.spring.framework.boot}</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <scope>test</scope>
    </dependency>

</dependencies>

<build>
    <finalName>${project.artifactId}</finalName>
    <plugins>
        <plugin>
            <artifactId>maven-war-plugin</artifactId>
            <version>${version.war.plugin}</version>
            <configuration>
                <warSourceDirectory>src/main/webapp</warSourceDirectory>

                <failOnMissingWebXml>false</failOnMissingWebXml>
            </configuration>
        </plugin>
    </plugins>
</build>

As properties do pom.xml do projeto pai (onde estão definidas as versões das dependências) está assim:
    <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <jvm-version>1.7</jvm-version>

    <maven.compiler.target>1.7</maven.compiler.target>
    <maven.compiler.source>1.7</maven.compiler.source>

    <version.xxx.parent>4.0.0-SNAPSHOT</version.xxx.parent>
    <version.ejb.api>3.1</version.ejb.api>
    <version.commons.pool>1.6</version.commons.pool>
    <version.org.spring.framework.sucurity>3.2.3.RELEASE</version.org.spring.framework.sucurity>
    <version.org.spring.framework.web>3.2.3.RELEASE</version.org.spring.framework.web>
    <version.org.spring.framework.config>3.2.3.RELEASE</version.org.spring.framework.config>
    <version.org.spring.framework.ldap>3.2.8.RELEASE</version.org.spring.framework.ldap>
    <version.org.spring.framework.boot>2.2.2.RELEASE</version.org.spring.framework.boot>
    <version.javax.validation.api>1.1.0.Final-redhat-1</version.javax.validation.api>
    <version.jstl>1.2</version.jstl>
    <version.javax.servlet.servlet.api>2.3</version.javax.servlet.servlet.api>
    <version.javax.servlet.servlet.api>1.0.0.Final-redhat-1</version.javax.servlet.servlet.api>
    <version.sitmesh>3.0.0</version.sitmesh>
    <version.spring.ldap.core>2.0.4.RELEASE</version.spring.ldap.core>
    <version.spring.ldap>1.2.1</version.spring.ldap>
    <version.jackson.core>2.8.9.redhat-1</version.jackson.core>
    <version.jackson.databind>2.8.9.redhat-1</version.jackson.databind>
    <version.jackson.mapper.asl>1.9.13.redhat-4</version.jackson.mapper.asl>
    <version.spring.webmvc>4.1.4.RELEASE</version.spring.webmvc>
    <version.spring.web>4.1.4.RELEASE</version.spring.web>
    <version.spring.context>4.1.4.RELEASE</version.spring.context>
    <version.spring.aop>4.1.4.RELEASE</version.spring.aop>
    <version.groovy.all>1.7.5</version.groovy.all>
    <version.arquillian.protocol.servlet>1.1.5.Final</version.arquillian.protocol.servlet>
    <version.slf4j.simple>1.7.7</version.slf4j.simple>
    <version.weld.core>1.1.23.Final-redhat-1</version.weld.core>
    <version.arquillian.weld.ee.embedded>1.1.2.Final</version.arquillian.weld.ee.embedded>
    <version.jboss.as.arquillian.container.managed>7.1.1.Final</version.jboss.as.arquillian.container.managed>
    <version.jboss.javaee>3.0.2.Final-redhat-4</version.jboss.javaee>
    <version.arquillian.protocol.servlet>1.1.5.Final</version.arquillian.protocol.servlet>
    <version.arquillian.bom>1.1.7.Final</version.arquillian.bom>
    <version.maven.surefire.plugin>2.12</version.maven.surefire.plugin>
    <version.maven.compiler>2.3.2</version.maven.compiler>
    <version.sp1.rehat>4.2.14.SP1-redhat-1</version.sp1.rehat>
    <version.spring.context.support>3.1.2.RELEASE</version.spring.context.support>
    <version.jexelapi.jxl>2.6.10</version.jexelapi.jxl>
    <version.jasperreports>6.4.0</version.jasperreports>
    <version.ear.plugin>2.8</version.ear.plugin>
    <version.ejb.plugin>2.3</version.ejb.plugin>
    <version.surefire.plugin>2.10</version.surefire.plugin>
    <version.war.plugin>2.1.1</version.war.plugin>
    <version.junit>4.11</version.junit>
    <version.log4j>1.1.4.Final-redhat-1</version.log4j>
    <version.commons.beanutils>1.9.2.redhat-1</version.commons.beanutils>
    <version.commons.lang>2.6.0.redhat-6</version.commons.lang>
    <version.jboss.ejb.api_spec>1.0.0.Final-redhat-1</version.jboss.ejb.api_spec>
    <version.hibernate.jpa.api>1.0.0.Final-redhat-2</version.hibernate.jpa.api>
    <version.hibernate.core>5.1.10.Final-redhat-1</version.hibernate.core>
    <version.hibernate.entitymanager>5.1.10.Final-redhat-1</version.hibernate.entitymanager>
    <version.hibernate.infinispan>5.1.10.Final-redhat-1</version.hibernate.infinispan>
    <version.jboss.jsf.api>2.1.28.Final-redhat-1</version.jboss.jsf.api>
    <version.jboss.servlet.api>1.0.0.Final-redhat-1</version.jboss.servlet.api>
    <version.jboss.jms.api>1.0.1.Final-redhat-1</version.jboss.jms.api>
    <version.primefaces>5.0</version.primefaces>
    <version.commons.digester>2.1</version.commons.digester>
    <version.cdi.api>1.2.0.redhat-2</version.cdi.api>
    <version.javax.inject>1.0.0.redhat-6</version.javax.inject>
    <version.hibernate.validator>5.3.5.Final-redhat-2</version.hibernate.validator>
    <version.jboss.ejb3>2.2.0.Final-redhat-1</version.jboss.ejb3>
    <version.org.infinispan>5.2.11.Final-redhat-2</version.org.infinispan>

    <version.drive-oracle>184.108.40.206</version.drive-oracle>
    <version.h2>1.4.192</version.h2>
</properties>

Comment: Vamos vincular a [outra questão](https://pt.stackoverflow.com/questions/431682/teste-unit%c3%a1rio-controller-spring-com-mockito) aqui também para conhecimento
Comment: @nullptr aí está a continuação da nossa conversa.
Comment: Já sei o que pode ser, logo mais respondo se ninguém responder antes :)
Answer: O problema é a compatibilidade entre as versões de JARs que você está utilizando, seu projeto está baseado no Spring 4.1.4.RELEASE conforme suas dependências:
<version.spring.webmvc>4.1.4.RELEASE</version.spring.webmvc>
<version.spring.web>4.1.4.RELEASE</version.spring.web>
<version.spring.context>4.1.4.RELEASE</version.spring.context>
<version.spring.aop>4.1.4.RELEASE</version.spring.aop>

Mas para testes você está utilizando uma dependência do Spring Boot, o que causa problemas de compatibilidade entre as bibliotecas do seu projeto:
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <version>${version.org.spring.framework.boot}</version>
    <scope>test</scope>
</dependency>

Sugiro utilizar a dependência de testes compatível com a versão das bibliotecas do seu projeto, que provavelmente será esta:
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-test</artifactId>
    <version>4.1.4.RELEASE</version>
    <scope>test</scope>
</dependency>

O Spring Boot trouxe muitas novidades, mas infelizmente suas bibliotecas não são compatíveis com as versões mais antigas do Spring
Comment: Deve ser isso mesmo, cara. Tive que dar sequência em outras partes do projeto mas em breve vou retomar para testar sua solução. Muito obrigado !!!


How do I mock \Drupal::httpClient()?

Question: I'm unit testing a utility library (of our own making) that makes calls to an external REST API with \Drupal::httpClient()
So, I have a library class with static functions:
class myUtils {
  public static function getFromApi($path)
  {
    ...
    $response = \Drupal::httpClient()->request( ... );
    ...
  }

...

}

and I want to call that from a test class:
class myUtilsTest extends \Codeception\Test\Unit
{
  // Lots of other stuff ...
  public function testGetFromApi()
  {
     // Do some mocking magic
     ...
     myUtils::getFromApi('/some/test/path');
     ...
  }
}

I realize I can make a \GuzzleHttp\Handler\MockHandler and set it up to return whatever I want it to, but how do I set it to "overwrite" the call to \Drupal->httpClient()?
I've seen some examples which seem to assume you have an instance of a class that has its own httpClient member, and this is easily mocked - but in my use case, there's no reason at all to design the utilities like that. So how do I mock the global \Drupal::httpClient() in this case?
Thanks in advance for any response.
Comment: Easier to just include a test only module that has a middleware service to provide responses for any internal or external API. Otherwise you have to mock out the class that is injected to the service you are testing and mock each method that is in use in your implementation.
Comment: This is exactly the situation why you need dependency injection, so that the APIs you depend are loosely coupled to your code and can be mocked and swapped out during testing. Once you start depending on things like HTTP API, the database API, the entity API, and friends, you might want to consider putting your tools a service class instead. I would reserve static functions to standalone logic (e.g. core's Html.php, Xss.php, UrlHelper.php, etc are good examples).
Comment: Using the static container in classes is discouraged for several reasons, this is one of them. Fortunately, it is not too hard to do DI.
Comment: I agree with Patrick and trying to contribute to this in a positive way I'd say that you just found something to improve in your current implementation. And possibly the reason you are facing this issue is due to the fact that you didn't implement your solution with a T(B)DD approach in mind from the beginning. Next time this will not happen to you again :_)
Comment: OK, thanks, will consider it!

However, having to change a perfectly valid architecture & implementation solely for testing purposes rubs me the wrong way a bit. But yes, it might be a way.
Comment: I suppose you're all right! My own testing experience is mainly from Python where it's possible to mock somewhat more dynamically, so that the actual code design doesn't have to consider testing (since you can always test). This, of course, is Drupal and PHP and not Python and Django, where my main experience lies. :-)
Answer: Converting your class to use dependency injection would look something like this:
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use GuzzleHttp\Client

class myUtils implements ContainerFactoryPluginInterface {

  /**
   * The http service.
   *
   * @var \GuzzleHttp\Client
   */
  protected $httpClient;

  final public function __construct(array $configuration, $plugin_id, $plugin_definition, Client $httpClient) {
    $this->httpClient = $httpClient;
    parent::__construct($configuration, $plugin_id, $plugin_definition);
  }

  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $container->get('http_client')
    );
  }

  public function getFromApi($path)
  {
    ...
    $response = $this->httpClient->request( ... );
    ...
  }

...

}

How do I mock \Drupal::httpClient()?


Fourier transformation of complex exponential proof

Question: How do I prove that the Fourier transformation of the complex exponential
$$\exp\left[ i  \pi(( a^2  x^2) +(b^2 y^2))\right]$$
 is
$$\exp\left[i {\pi}\left( {fx^2 \over a^2} + {fy^2 \over b^2}\right)\right]$$
Comment: What is your $f$?
Comment: Please write your expression in Latex.
Answer: Careful analysis of the Cauchy principle value integral for the Fourier transform gives
$$
\begin{align}
   &\mbox{c.p.v.}\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{i\pi a^{2}x^{2}}e^{-ixs}dx \\
   & = \mbox{c.p.v.}\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{i\pi(ax-s/2\pi a)^{2}}dx\, e^{-is^{2}/4\pi a^{2}} \\
  & = \left(\mbox{c.p.v.}\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{i\pi a^{2}x^{2}}dx\right)e^{-ia^{2}s^{2}/4\pi} = Ce^{-is^{2}/4\pi a^{2}}.
\end{align}
$$
The constant $C$ comes from a Fresnel integral , at least after a linear change of variable.
$$
             C=\frac{2}{\sqrt{2\pi}}\int_{0}^{\infty}cos(\pi a^{2} x^{2})dx.
$$


Oracle11g - Problemas comando SELECT

Question: Estou com o seguinte erro:

java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly
  ended

Isto acontece após eu executar a seguinte linha de SQL:
/* Listar todas as reuniões do utilizador onde esteja convidado*/
String sql = "SELECT M.ID, M.TITLE, M.DIA, M.HORA FROM MEETING AS M, MEETING_LIST AS ML WHERE (M.ID=ML.ID_MEETING AND ML.ID_UTILIZADOR=( SELECT ID FROM UTILIZADOR WHERE NOMEUTILIZADOR='"+ parcel.getUsername()+"' ) AND ML.STATUS=1 AND ML.INVITE=1)";
toClient = new AbstractMessage(database.submitQuery(sql));

A tabela MEETING tem os seguintes atributos:

ID, TITLE, OUTCOME, DIA, HORA, LOCALIZACAO

A tabela MEETING_LIST tem os seguintes atributos:

ID, ID_MEETING, ID_UTILIZADOR, INVITE, STATUS

A tabela UTILIZADOR tem os seguintes atributos:

ID, NOMEUTILIZADOR, PALAVRACHAVE

A tabela MEETING_LIST é uma tabela que cruza os utilizadores e as reuniões (many to many).
Answer: A própria mensagem de erro está dizendo o que está acontecendo.

ORA-00933: SQL command not properly ended

Significa que algo na construção da sua SQL fez o ORACLE entender que chegou ao final do comando antes de realmente estar no final do comando.
Creio que o seu problema são os aliases que você colocou. Na verdade você não deve colocar "MEETING AS M" e sim apenas "MEETING M" a palavra reservada "AS" deve ser usada apenas para definir os alias das colunas e não redefinir o nome da tabela a ser usada no comando SQL.
Tente o seguinte.
String sql = "SELECT M.ID, M.TITLE, M.DIA, M.HORA FROM MEETING M, MEETING_LIST ML WHERE (M.ID=ML.ID_MEETING AND ML.ID_UTILIZADOR=( SELECT ID FROM UTILIZADOR WHERE NOMEUTILIZADOR='"+ parcel.getUsername()+"' ) AND ML.STATUS=1 AND ML.INVITE=1)";
toClient = new AbstractMessage(database.submitQuery(sql));

Tente ter sempre em mãos alguma IDE para gerenciamento de bancos de dados como por exemplo o PL/SQL Developer e sempre que ocorrer esses erros execute seus comandos SQL na IDE, ela ajuda muito na agilidade em encontrar os erros.
Answer: Solução é retirar o AS.
String sql = "SELECT M.ID, M.TITLE, M.DIA, M.HORA FROM MEETING M, MEETING_LIST ML WHERE (M.ID=ML.ID_MEETING AND ML.ID_UTILIZADOR=( SELECT ID FROM UTILIZADOR WHERE NOMEUTILIZADOR='"+ parcel.getUsername()+"' ) AND ML.STATUS=1 AND ML.INVITE=1)";
                toClient = new AbstractMessage(database.submitQuery(sql));

Answer: Tem dois parenteses a mais na clausula WHERE ocasionando o erro:
SELECT M.ID, M.TITLE, M.DIA, M.HORA
  FROM MEETING AS M, MEETING_LIST AS ML
WHERE (M.ID = ML.ID_MEETING AND
    ML.ID_UTILIZADOR =
       (SELECT ID

           FROM UTILIZADOR

          WHERE NOMEUTILIZADOR = '"+ parcel.getUsername()+"') AND
       ML.STATUS = 1 AND ML.INVITE = 1)

Tente este:
SELECT M.ID, M.TITLE, M.DIA, M.HORA
  FROM MEETING AS M, MEETING_LIST AS ML
 WHERE M.ID = ML.ID_MEETING
   AND ML.ID_UTILIZADOR =
       (SELECT ID
          FROM UTILIZADOR
         WHERE NOMEUTILIZADOR = '"+ parcel.getUsername()+"')
   AND ML.STATUS = 1
   AND ML.INVITE = 1

Comment: Use quatro espaço para formtar códigos ou selecione o texto e click no botão `{ }`
Answer: Quando vc executa o sql em um cliente oracle da certo ou errado?
se vc jogar o sql no squirrel ou outro cliente de sql da erro?

SELECT M.ID, M.TITLE, M.DIA, M.HORA FROM MEETING AS M, MEETING_LIST AS ML WHERE (M.ID=ML.ID_MEETING AND ML.ID_UTILIZADOR=( SELECT ID FROM UTILIZADOR WHERE NOMEUTILIZADOR='id_do_utilizador' ) AND ML.STATUS=1 AND ML.INVITE=1)

Se o sql estiver dando erro no cliente de sql entao uma estrategia de depuração,seria vc executar a query em partes, dessa forma vc vai saber onde é o ponto em que o erro ocorre.
Tenta ver se essa query aqui resolve:

SELECT
     M.ID,
     M.TITLE,
     M.DIA,
     M.HORA
  FROM
     MEETING AS M,
     MEETING_LIST AS ML
  WHERE
     M.ID=ML.ID_MEETING
     AND ML.ID_UTILIZADOR=(
        SELECT
           ID
        FROM
           UTILIZADOR
        WHERE
           NOMEUTILIZADOR='1'
     )
     AND ML.STATUS=1
     AND ML.INVITE=1

Oracle11g - Problemas comando SELECT


Wilcoxon rank-sum or t-test with a single subject in a group

Question: I'm just wondering if I have Group A (N=20) and group B (N=1), does it valid to do a statistical test, such as (independent) Wilcoxon rank sums or independent t-test? Note that group A and B are independent (different population).
Given that these tests works on class distribution, it seems that it wouldn't be correct to draw class distribution from only 1 sample. Any thoughts?
Thanks
Comment: @Dave2e no, they're completely independent (hence independent tests). Thanks, I added this info in question.
Answer: You can not use independent t-test for this because you can not compute variance when $n=1$. You can compute a Wilcoxon rank-sum test, but it will be uninformative : you could instead compute how many times "Group B" was over Group A.
Better, compute the z-score of "Group B" based on mean and variance of Group A, then see the probability of the value. It will tell you how likely is "Group B" compared to Group A.
Comment: So the essence of doing those tests is to check on the p-value of the variables and we're going to analyse on the significant variables. I will use your suggestion on z-score to compare "Group B" and Group A after getting those significant peaks. Given this, my next question is that - how valid is the p-value of wilcoxon rank-sum test in this case to get significant features? Also, another quick question, since what we do here is wilcoxon (median), what do you think of computing z-score on median and median absolute deviation?
Comment: Probably not very reliable. The wikipedia entry suggest that "for sample sizes above ~20, approximation using the normal distribution is fairly good". https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test (see calculations)
Comment: So any suggestion on how to select most significant features? Should I use the z-score method, and filter based on the "Group B"'s z-score (since it shows how much it deviates from mean/median, if median absolute deviation)? Using the z-score seems to be quiet subjective though, since there is no rule-of-thumb threshold (as far as I know) for a feature to be significant (vs p-value < 0.05 in stats tests).
Comment: Not sure to understand your comment. You can use usual critical z-scores like + or - 1.64 (90% confidence) or 1.96 (95% confidence).
Comment: Thanks! That's really helpful. Any thoughts on modified z-score (using median + median absolute dev) for data with some outliers?


Geometry of Rejection regions for tests (NP lemma, Karl-Rubin, UMPU, LMP, etc)?

Question: The Neyman-Pearson lemma, the Karlin-Rubin theorem, and the other for UMPU tests for the exponential family, etc. They all define a most powerful Rejection Region (RR), in a certain class of tests.
I was wondering how do these RR translate into graphs? I'm interested not only in those tests that are most powerful in a sense, but also the usual tests we use, even if we don't do a power study of them.
For example, when we use a Wald test of the form $n(\hat\phi-\phi)^TC^{-1}(\hat\phi-\phi)$ we get a quadratic. So, the RR we're interested in is the one defined just by a one-tailed test, i.e., an ellipsoid around the true value $\phi$, and whenever our estimates lie outside the ellipsoid, we reject the null.
It would be nice to gather as much examples of RR for different tests as possible.
Comment: In multiple testing (with e.g. Bonferroni correction) you get a rectangle in two dimensions, a box in three dimensions, ...
Comment: @fcop please write up an answer It would be great to be able to structure this. ;)
Comment: maybe you can help me with this @An old man in the sea ? http://stats.stackexchange.com/questions/229193/whats-wrong-with-multiple-testing-correction-compared-to-joint-tests
Comment: @fcop I'm not sure whether I'm that capable. I'll have a look. Meanwhile, I've put +1 on it. ;)
Comment: I am sure that, at least, you understood because your question on the geometric shape of the acceptance region made me think and led to this question
Comment: @fcop But I'm not myself familiarised with multiple testing corrections...
Answer: Refering to the Wald test that you mention and (for ease of understanding) limiting it to two dimensions, your test looks something like $H_0: \mu_1 = \mu_{10} \& \mu_2 = \mu_{20}$ versus $H_0: \mu_1 \ne \mu_{10} | \mu_2 \ne \mu_{20}$.
You can approach this problem in two ways; either you can use ''joint tests'' or you can use ''multiple testing procedures''.  Joint tests start from a multi-variate distribution for a test statistic (the Wald test you mention is an example) while multiple testing procedures ''adjust'' the p-value (or equivalently , adjust the signficinace level).  The Bonferroni correction is an example of a multiple testing procedure. For more details see this link.
The Bonferroni procedure states that, if you perform two tests, then you should perform two independent tests, but in order not to inflate the type I error, you should do each test with half the significance level (or you should double the p-values). (This also holds for more than two tests).
Obviously, if I perform each of the two tests in a ''univariate way'' with half the significance level, then I find an interval in each of the two dimensions or a reactankle in the two-dimensional plane.
You can see the different shapes in the figures 2,3,4 in the link supra. Figure 4 has a ''combined shape'', for details see the link.
Comment: I liked figure 6. :)
Comment: I agree, figure 6 is a fantastic picture :-)


How to setup a SPF record to help fight SPAM

Question: My little developer mind never really managed to understand how to do it, so maybe you can help me.
I'm using Google Apps for Domains and MyDomain as DNS external service.
Edit: Tough crowd, in Google apps says:

Change the SPF record to fight SPAM
  (optional)
You may define the SPF record to
  authorize only certain IP addresses to
  send email for your domain. This will
  prevent spammers from sending
  unauthorized email under a forged
  address from your domain.
Comment: No where did he say he way trying to prevent inbound spam...in fact the edit he added clearly shows "from your domain."  Anyway, SPF helps  reduce inbound backscatter and helps maintain the reputation of your domain - all of which are problems in the fight against spam.
Comment: SPF does fight spam by making it harder for people to impersonate you in spam.
Comment: Why the downvotes?  I hope its not because he is confused about the purpose of an SPF record....
Comment: @womble - Correct SPF helps makes it someone else's problem.  Reducing backscatter to your domain is a good thing.
Comment: SPF isn't an anti-spam technique.
Comment: Which does nothing to stop spam, because there's an infinite number of other domains which can be used in a From: address.
Comment: Womble: You're 'infinite' inspired me to look that up :-) So this site says 112.6 million domains http://www.domaintools.com/internet-statistics/ . Although I don't see any information about where those numbers come from. If I had to guess those are maybe the second level domains.
Comment: @Kyle: Investigate "domain tasting".  @DLux: You still receive exactly as much spam if you have SPF records as if you didn't.
Comment: @womble: Domain tasting is becoming harder with recent ICANN policy changes.
Answer: There is an article on Google Apps help about this.
http://www.google.com/support/a/bin/answer.py?hl=en&answer=33786
All you need to do is add that TXT record and you're set. I don't know what you mean by "fight SPAM", as all this does is authenticate that the e-mail you are sending is not spoofed, so all it will do is help you get your e-mail in someone else's inbox instead of their spambox, if that is currently the case.
Comment: I did find this by google, this post was the first hit :)
Comment: It will also refuse to authenticate spam that's pretending to come from you, and get that spam in someone else's spambox instead of their inbox. How is that not "fight SPAM"???
Comment: I probably understood this differently based on his initial post; I probably thought he wanted to stop spam from reaching *his* inbox. Since the post is from 2009, I can't really remember :).
Answer: You can use the Microsoft Sender ID Framework SPF Record Wizard  to generate SPF record.
Comment: Big +1 for this handy tool!
Answer: SPF doesn't prevent spam from being sent to you, it only helps ensure that no one sends spam as you.  Also, when bulk mailing, some mail services (hotmail, google, yahoo) pretty much require the sender has an SPF record or else the mail will likely end up in junk mail folders.  Also look into setting up a DomainKey record, as this is Microsoft's way of accomplishing almost the same thing.
Comment: Agreed. The Microsoft attempt you probably meant is Sender ID, which is similar to DomainKey, but was less popular. Recently DomainKey is being succeeded by DKIM, which is very similar to DomainKey but is standardized.


Highlight Entire row if any changes occur in Range("M7:Q500") or ("C7:C500") Excel VBA

Question: I have set up a code but I know it's completely wrong. My goal is to highlight entire row if a change is made in ("M7:Q500") or ("C7:C500") and highlight row based on where the change was made. Ex Cell M7 (Changed) Highlight entire 7 row
If Not Intersect(Target, Range("M7:Q500") Or ("C7:C500")) Is Nothing Then
    Cell.Interior.ColorIndex = 3
End If

Answer: The Or part is wrong.  Try this:
If Not Intersect(Target, Range("M7:Q500")) Is Nothing Or _
  Not Intersect(Target, Range("C7:C500")) Is Nothing Then
    Rows(Target.Row).Interior.ColorIndex = 3
End If

Comment: @Luis please mark as correct by clicking the grey/green check mark by the answer. It is something only you can do.
Comment: I'm getting a debug error on this Cell.Interior.ColorIndex = 3
Comment: @Chrismas007 got hung up on the first error.  I totally missed the second.  Thanks.
Comment: @ScottCraner So that eliminates the debug error, but that only highlights the individual cell that is changed. How do I highlight entire row based on that change?
Comment: Did you see my edit, it should do the entire row.  @Luis The comment by Christmas007 would only do that mine should do the entire row.


When to *say* "tensor" instead of "matrix"?

Question: The difference between a tensor and a matrix is a subtle but important and well discussed point.
In the context of a vector space $V$ over a field $F$ (and its dual $V^*$), briefly put, a $(p, q)$-tensor $T$ is a multilinear map $\ {V^*}^p \times {V}^q \to F\ $.
Any basis we choose for $V$ and $V^*$ can be inherited by $T$, and a matrix $M$ is a representation of $T$ in that particular basis. Had we chosen a different basis, the matrix representation of $T$ would be some different $M' \neq M$, while $T$ is just $T$. As a mundane example on $\mathbb{R}^2$ for a $(1, 0)$-tensor, the idea is just that $\ \begin{bmatrix}2 \\  3\end{bmatrix} \neq \begin{bmatrix}1 \\  -2\end{bmatrix}\ $ while $\ 2e_1 + 3e_2 = e'_1 - 2e'_2\ $. One consistent basis must be used to write a matrix equation, while a tensor equation (like the latter, usually shown in Einstein notation) shows equality of tensors, which holds true regardless of basis.
In practice however, I find literature commonly saying "matrix" where "tensor" would be more appropriate, making me doubt my understanding. Take for example the "covariance matrix" in probability theory. Covariance is conceptualized as the ellipse in the following geometric depiction:

This geometric object is invariant to our choice of basis (i.e., the ellipse exists regardless of how you draw your grid-lines), and thus is best modeled with a tensor. Everything I've ever seen done with a covariance "matrix" has been tensorial, from the $R^{-1} T R$ style transformations used in principal component analysis (typical of $(1, 1)$-tensors) to the contraction to $\mathbb{R}$ used in the exponent of the multivariate Gaussian distribution (where the covariance "matrix" is literally used as a bilinear map $V^* \times V \to \mathbb{R}$).
Meanwhile, an extremely similar object, the "inertia tensor" from rigid-body mechanics, which describes the spread of mass (instead of probability), is so purposefully called a tensor that many mechanics books include a section explaining the very difference I briefed above. Surely there is a naming inconsistency here?
Then, probably due to the popular software "TensorFlow", I see the matrices used in neural networks being called "tensors" by the machine learning community, even when there is no discernible basis-independent abstract object being described by these matrices. They're just linear mappings, and in most cases aren't even endomorphisms.
I think many people trying to understand tensors can benefit from clarifying these semantics. When should we really say "tensor"? Is it pretentious to say "covariance tensor"? Do any theories utilize a square matrix that isn't just an order-2 tensor in a particular basis? (Perhaps the Jacobian matrix?) Is there any kind of indicative mathematical expression or action that makes you go "yup, that matrix is just representing a tensor - the important object here is the tensor itself."?
Comment: I found a related answer: https://stats.stackexchange.com/a/198395/210281
Answer: There are several meanings of the term "tensor". For example, this context is close to your multilinear-map idea. But to a physicist, or differential geometer, a tensor satisfies a specific transformation law. Suppose we write it in two coordinate systems, one with lower case indices, the other with upper case; then a rank-$(p,\,q)$ tensor $T$ satisfies$$T^{a_1\cdots a_p}_{\quad\qquad b_1\cdots b_q}=\frac{\partial x^{a_1}}{\partial x^{A_1}}\cdots\frac{\partial x^{a_p}}{\partial x^{A_p}}\frac{\partial x^{B_1}}{\partial x^{b_1}}\cdots\frac{\partial x^{B_q}}{\partial x^{b_q}}T^{A_1\cdots A_p}_{\quad\qquad B_1\cdots B_q},$$with summation over repeated indices. For example, in a $2$-dimensional coordinate system a rank-$(1,\,0)$ tensor obeys$$T^a=\frac{\partial x^a}{\partial x^A}T^A=\frac{\partial x^a}{\partial x^\mathrm{I}}T^\mathrm{I}+\frac{\partial x^a}{\partial x^\mathrm{II}}T^\mathrm{II},$$if the Roman values $\mathrm{I},\,\mathrm{II}$ are the two options for $A$.
Answer: A few unorganized thoughts below. If this answer is not really appropriate, let me know and I will remove it.

saying "matrix" where "tensor" would be more appropriate, making me doubt my understanding

I think your understanding of the distinction is fine. One issue with math terminology/notation (that is characteristic of natural language) is that predominantly used terms tend to "stick," even if there are other more appropriate or less confusing choices that just didn't catch on. Another issue is that usage is also field-specific, and, in this case, if people in those fields did not focus much on basis-invariance, it may not have been worth dwelling on it and using, say, "covariance tensor" or stressing that it is a bilinear form, instead of saying "covariance matrix." In fields outside of mathematics, there is also the issue of alienating people by using terms different from the common usage; if thinking of linear transformations, bilinear forms, tensors etc. as matrices was sufficient for them, it may be too much overhead to try to upend the common usage in that area. Throw in cases where other fields like machine learning co-opt terms like "tensor" or give new names to existing concepts and you get a great mess of multiple terms referring to the same thing, or single terms referring to different things. But this is just how language works, and it is something we just have to deal with. I think the best you can do is to make sure you understand these distinctions (which you obviously do) and try to notice how different areas use (or "misuse") terms and become more "fluent" in usage. You can of course educate others to think about these objects in the way you do, but I think pushing entire fields to adopt more-consistent terminology is not worth one's effort.


Conversão String para Float

Question: Estou tentando plotar um gráfico, mas estou tendo o seguinte retorno "could not convert string to float" na última linha desse trecho do código. Alguém tem ideia do que consigo fazer?
from __future__ import unicode_literals
import numpy as np
import matplotlib.pyplot as plt
igs = 50*'='
desenho = plot.figure()
papel = desenho.add_subplot(111)
desenho.patch.set_color('white')

print('Grafico de f : [a,b] -> R/n', igs)
print('Intervalo a,b:')
a,b =np.float64(input('a,b ='))

Comment: Qual a entrada passada em `input('a,b =')` ? As linhas `from __future__ import unicode_literals`  e `import matplotlib.pyplot as plt`, de  `igs = 50*'='` até `print('Intervalo a,b:')` parecem não ter relevância alguma para a questão, as únicas linhas que importam são `import numpy as np` e `a,b =np.float64(input('a,b ='))`, a mensagem de erro na integra e uma amostra da entrada.
Answer: O input recebe valores no formato string. Sendo assim, é necessário usar o split().
a, b = np.float64(input('a,b =').split(","))

Exemplo
In [5]: a,b =np.float64(input('a,b =').split(","))
a,b =12.34,98.76

In [6]: a
Out[6]: 12.34

In [7]: b
Out[7]: 98.76

Conversão String para Float


MySQL Queries overlapping each other

Question: I'm having an issue with MySQL Queries, there are a couple queries that seem to be overlapping each other. Basically, I have some PHP code (being called via a REST API) using MySQL with PDO that inserts a value into a table if it doesn't already exist. If it exists, then it updates the record instead. Pretty simple stuff, and it seems to be working 99% of the time. But on certain occasions the same record gets inserted twice (instead of being updated the second time).
I looked into the MySQL log and found that on rare occasions the script is somehow being run twice at the same time (causing overlapping queries). Notice how the process IDs 95587 and 95588 seem to be overlapping.
95587 Connect     @localhost on ****
95588 Connect     @localhost on ****
95587 Query       SET NAMES utf8
95588 Query       SET NAMES utf8
95588 Query       SELECT * FROM `my_table` WHERE `data` = '89158268' LIMIT 1
95587 Query       SELECT * FROM `my_table` WHERE `data` = '89158268' LIMIT 1
95588 Query       INSERT INTO `my_table` (`data`, `date_created`, `date_updated`) VALUES ('89158268', '2011-08-21 17:11:10 ', '2011-08-21 17:11:10 ')
95587 Query       INSERT INTO `my_table` (`data`, `date_created`, `date_updated`) VALUES ('89158268', '2011-08-21 17:11:10 ', '2011-08-21 17:11:10 ')
95588 Quit
95587 Query       SELECT * FROM `another_table`
95587 ... more queries

This is the order of the code:

SELECT * FROM my_table WHERE data = '89158268' LIMIT 1
if record found, then update my_table
else if record not found, insert new record into my_table and then continue other queries

Can anyone think of a reason why these 2 mysql processes seem to be overlapping each other? Any help is greatly appreciated. Thank you.
Comment: By overlapping, do you mean running at the same time? And what exactly is the actual issue that you're having?
Comment: If data is a primary key, consider using INSERT ... ON DUPLICATE KEY UPDATE: http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
Comment: Two process IDs indicates two different connections to the server, indicating two separate PHP script invocations, suggesting two separate HTTP requests.
Comment: It means that there are 2 connections being made. It seems that the problem isn't in why are there 2 processes, your problem is the wrong approach to the problem. Using UNIQUE constraint and ON DUPLICATE KEY UPDATE would solve the issue with duplicate records. After you take care of duplication, you can proceed to researching why are there 2 connections being made instead of one. Since the info you provided isn't sufficient, it's hard to tell what's going on.
Answer: Try locking the table before the select:
LOCK TABLE my_table WRITE;
SELECT * FROM `my_table` WHERE `data` = '89158268' LIMIT 1;
INSERT INTO `my_table` (`data`, `date_created`, `date_updated`);
UNLOCK TABLES;

This will avoid any other MySQL session from using the my_table table, and avoid this race condition.
Comment: Why lock if there are more elegant and safer solutions? One of them is ON DUPLICATE KEY UPDATE as posted in comments.

MySQL Queries overlapping each other


How to get better at jazz quickly?

Question: I play the piano and guitar and I'm learning to read sheet music. All I can do to make my playing sound more jazz is occasionally add flat and sharp notes that are outside the major/minor scale, play minor/major/dominant seventh chords and that's it. Anything else I try like using modes when I'm playing over backing tracks sound awful. What are some techniques that I can learn quickly that'll make my playing better.
Comment: I'm voting to close this question because there are many, many threads here that discuss techniques for learning jazz on guitar and piano. Please search and read those. Then, if none of them provide what you need, edit your post to explain why, and it can be reopened.
Comment: Take a class and/or hire a teacher. There are no shortcuts. The fastest way is always private one-on-one instruction. Without that, you'll have to take the time to piece together all the various aspects of jazz from many different free sources on your own.
Comment: @Aaron can you please provide links?
Comment: There are too many to provide individual links, but [these search results](https://music.stackexchange.com/search?q=jazz+learn+is%3Aquestion) will get you started.
Answer: Check out the following concepts:

the altered scale
extended chords
four-note chord voicings
the 2-5-1 turnaround
tritone substitution and reharmonization

They may provide a quick way in if you just want a hack to sound jazzier without diving in too deep. These things are of course worth learning in-depth, and they will come up when you study jazz in a more formal way.
And if you want to put in more work, transcribe different versions of a few jazz standards, and try to find which details make them sound jazzy to you.
Comment: +1 for suggesting transcription. Study is the answer!
Answer: This sounds like my experience. I think the problem is first focusing on harmony as the principle aspect of jazz. But the rhythmic aspect is the more unique quality of jazz.
If you are working with modes along the lines of the "chord/scale" system, something like Jamey Abersold's methods, the first thing I would say is all the past jazz greats like Red Garland, Ahmed Jamal, Horace Silver, etc. didn't play that way. They played much more along the lines of chord tones with embellishments. If you play up and down modes in a scalar fashion, it just won't sound like those players. It's too scalar.
You didn't mention the blues scale or blue notes. Make sure you are working with the blues scale and blues style. You might think blues is separate from jazz, but they overlap and players like Horace Silver or Herbie Hancock will play bluesy riffs.
Be absolutely certain to focus on rhythm. Look up jazz swing feel and characteristic jazz syncopated rhythm. A lot of methods I have seen teach rhythm with simple rhythm notation (no pitch) and vocal syllables like "doo bee doo dit", or something similar. A lot of rhythm methods seems to approach this as a vocal exercise.
One thing in particular you might try to satisfy you desire to quicky start sounding like jazz is to stop working with the backing tracks and learn to play a walking bass in left hand while comping chords in the right hand. There are lots of method books to find for comping and it's more of a mechanical method than improvising "horn lines" on piano, so you can get up to speed in a fairly short time, at a basic level. It also provides the benefit of allowing you to develop a good sense of timing and phrasing.
Answer: Learn to 'play what you hear'.   No-one wants to listen to you play 'the scale of the chord'.  They might be interested in how you take the melody and rhythm (and, yes, the chords, but they aren't ALL that matter) of the original song and mix them up!

How to get better at jazz quickly?


Como buscar dentro de um arquivo em diferentes subpastas usando grep?

Question: Como posso fazer um grep que busque em um arquivo específico mas dentro de várias subpastas?
Algo como grep -R "busca" /home/apps/*/teste.php
O asterisco é a indicação de que quero buscar em todos os diretórios dentro de /home/apps mas somente dentro do arquivo teste.php que existe dentro dessas subpastas.
Comment: Questões desse tipo cabem no https://unix.stackexchange.com/ ou mesmo no https://ubuntu.stackexchange.com/  - Lembre-se de ler as regras específicas daquela comunidade antes de postar lá. Para futuras questões que estejam no [escopo do site](/help/on-topic) vale a pena entender [O que é o Stack Overflow](https://pt.meta.stackoverflow.com/q/7256/70) e ler o [Guia de sobrevivência (resumido) do Stack Overflow em Português](https://pt.meta.stackoverflow.com/q/8089/70).
Answer: O seu comando (grep -R "busca" /home/apps/*/teste.php) busca apenas o arquivo teste.php nas pastas que estão um nível abaixo de /home/apps/.
Ou seja, se o arquivo for /home/apps/pasta/teste.php, ele é encontrado. Mas se for /home/apps/teste.php ou /home/apps/pasta/subpasta/teste.php, estes serão ignorados.
Um jeito de buscar em todas as subpastas é usando a opção --include para especificar o nome do arquivo, e indicar a pasta a partir da qual será feita a busca:
grep -R "busca" --include="teste.php" /home/apps

Assim, ele busca pelos arquivos chamados teste.php que estão dentro de /home/apps (e a opção -R garante que buscará em todas as subpastas).
Outra opção é usar find para buscar os arquivos, e em seguida passá-los para grep:
find /home/apps/ -name "teste.php" | xargs grep "busca"

Só que ambas as soluções acima também fazem a busca em /home/apps/teste.php, caso exista. Mas se a ideia é pegar somente a partir de um determinado nível, você pode usar a opção -mindepth:
find /home/apps/ -mindepth 2 -name "teste.php" | xargs grep "busca"

No caso, -mindepth 2 faz com que arquivos um nível abaixo da pasta /home/apps sejam ignorados. Ou seja, /home/apps/teste.php será ignorado, mas /home/apps/pasta/teste.php e /home/apps/pasta/subpasta/teste.php não.
Comment: Perfeito, consegui, muito obrigado!


Detectar texto copiado

Question: Como executar uma ação sempre que alguém copiar algum texto no keyboard do Android?
E teria que usar um service para deixar rodando o tempo todo?
Comment: Que tipo de ação seria executada?
Comment: pegaria o texto copiado e registraria em um banco sqlite @DenerCarvalho
Answer: Você está procurando pelo método addPrimaryClipChangedListener do ClipboardManager. Ele recebe como parâmetro um ClipboardManager.OnPrimaryClipChangedListener.
Presumindo que você esteja familiar com Listeners, o código fica mais ou menos assim:
final ClipboardManager gerenciadorDeAreaDeTransferencia =
        (ClipboardManager) getSystemService(CLIPBOARD_SERVICE);

gerenciadorDeAreaDeTransferencia.
    addPrimaryClipChangedListener(new OnPrimaryClipChangedListener() {
                    @Override
                    public void onPrimaryClipChanged() {
                        // seu código
                    }
                }
    );

Vale notar que isso só funciona a partir da API 11.
Sobre fazer funcionar em segundo plano, você provavelmente vai depender de um service, sim, mas os detalhes disso são assunto pra outra pergunta.
Comment: É isso mesmo, eu já estava usando o ClipboardManager e ClipData, mas não vi os listeners, passou despercebido

Detectar texto copiado


How to interpret logistic regression with days of the week and months of the year as predictors together with continuous and binary predictors?

Question: I am using logistic regression to predict electricity spike prices (price that exceeds a certain threshold).
I directly use the following variables as my independent variables together:
(a) demand/load - a continuous IV
(b) is a binary IV
(c) days of the week
(d) months of the year.
I used the reference category for (c) and (d) and end up sept and tue for the reference. How do I interpret the results using marginal effects for all my predictors?
I am using Stata 9.
Answer: I think, first of all, you probably need some more complex model to account for time-dependence of the dependent variable. I think a non-linear mixed model is what you want.
But to answer your question, I would say the easiest way is to look at the odds ratio (OR) for each level of each categorical independent variable and note that that is the OR compared to the reference level (e.g. Tuesday)
Be sure to check whether Stata uses reference cell coding or effect coding or something else. These have different interpretations, and you may want to change the default.
Comment: Sir, I use those categorical IV( days and months) to account for the trend and seasonality of the prices. I will look unto the odds ratio, but, sir, I'm more concerned on the interpretion with regards to the probability(marginal effects/discrete change), Sir Peter. I'm using binary logistic regression for this. Thank you
Comment: Hi @Marc.  No need for "sir"! :-)

Unfortunately, days and months do not fully account for the dependence of the data - the value of the DV on (say) Jan 2 is going to be related to that on Jan 1, beyond the fact that both are January and whatever day of the week each is.

You can get the probability of each outcome directly in SAS and R, and probably in Stata as well, but since I don't use Stata, I don't know how in that program
Comment: Hi Mr. Flom :) hmm, I've ran the program(Stata) with a year's worth of data with months and days (my data is in hrs). I've seen a multiple linear regression used this dummy variables(days and months) as regressors that account to the trend and seasonality of the prices, so I did use it too with the same notion. Can I ask you're opinion on what I did? Thank you.
Comment: Months will account for seasonality, at least partly. But what about the dependence (or think of it as auto-correlation) in the dependent variable?


TYPO3 files in impexp

Question: When I export a t3d from one system (TYPO3 6.2) and import it in a second system (TYPO3 8.7) the content images changes. I've expected the image is also transferred from one system to the other, but TYPO3 seems to just say "the imported content element has sys_file with UID 123, so I still use this file", but this UID references to a totally dfferent image in the second TYPO3 system.
Comment: Problem is, the images are missing in the export. There's a bug report for it: https://forge.typo3.org/issues/82047
Answer: The images from standard content elements are not exported, if option "[ALL tables]" or options "sys_file*" are choosed for "Use static relations for tables". Don't select these options and your export will contain the images.
Also see forge.typo3.org/issues/82047. Thanks to Christian Kuhn for working this out.

TYPO3 files in impexp


Python kernprof line profiler wrong results

Question: I have two functions which i profile. If i run kernprof, it says that my first function runs in 75 seconds but the second which is python-cythonized in 15.
I already tried to use the time module in python and measure it by myself i see that it's more like 12 seconds for the first function and 9 for the second.
Why is that so?
Answer: We examined a bunch of profilers and found that for the benchmark we used for testing (from the Pyperformance suite), using kernprof causes roughly a 7x slowdown when executing pure Python code. This is remarkably close to your observed execution time dilation (6.25x), so I believe this is almost certainly the cause.
If you want to use a profiler that has less overhead (nearly none) and is thus more accurate, there are a handful of options. I personally recommend Scalene, a profiler that simultaneously profiles CPU, GPU, and memory, all with very low overhead and with no need to modify your code (i.e., adding @profile decorators is not required). Full disclosure: I am one of the primary authors of Scalene.