Form built by Ajax commands not sending values

Question: My problem has to do with Ajax commands and the form being rebuilt after calling ajax_command_append().
I have a form with two elements an option and a textfield. These are built-in with the form on page load. Then, in an Ajax callback function I add two other fields, i.e. another textfield and a text area.
The form runs and rebuilds perfectly fine, but when I try to read the values typed in or stored-from-beginning, nothing happens except that only the values for the first two fields that were originally built-in with the form are displayed.
I think the problem is either 1) I am not rebuilding the (same?) form properly, or 2) I am not calling the elements/fields properly when reading values.
Any hints will be greatly appreciated
Here is a simplified version of the test code I gathered from various examples:
function module_form($form, &$form_state) {
$form = array();
 $form['changethis'] = array(
 '#type' => 'select',
 '#options' => array(
  'one' => 'one',
  'two' => 'two',
  'three' => 'three',
  ),
   '#ajax' => array(
     'callback' => 'ajax_example_simplest_callback',
     'wrapper' => 'replace_textfield_div',
     'method' => 'replace',
     'effect' => 'fade',
    ),
  );
  $form['replace_textfield'] = array(
    '#type' => 'textfield',
    '#title' => t("The default value will be changed"),
    '#description' => t("Say something about why you chose") . "'" .     (!empty($form_state['values']['changethis']) ? $form_state['values']['changethis'] :     t("Not changed yet")) . "'",
    '#prefix' => '
',
    '#suffix' => '
',
   );
 $form['box'] = array(
   '#type' => 'markup',
   '#prefix' => '
',
   '#suffix' => '
',
   '#markup' => '
Initial markup for box
',
  );

  $form['submit'] = array(
    '#type' => 'submit',
    '#ajax' => array(
      'callback' => 'ajax_example_submit_driven_callback',
      'wrapper' => 'box',
     ),
    '#executes_submit_callback' => TRUE,
    '#value' => t('Submit'),

   );
  return $form;
}

function ajax_example_simplest_callback($form, $form_state) {

   $commands = array();

   $commands[] = ajax_command_append(NULL, '
<input name=replace_textfielddd id=replace_textfielddd value=79898>test data</input>

');

  $commands[] = ajax_command_append(NULL, '
<textarea rows=4 cols=30 name=replace_textarea id=replace_textarea>Changethis value: '. $form_state['values']['changethis'] .'</textarea>');
  $page = array('#type' => 'ajax', '#commands' => $commands);

  ajax_deliver($page);

  //print ajax_render($commands);       // not helping
  //drupal_exit();

 // return $form['replace_textfield'];  // not helping
 }

function ajax_example_submit_driven_callback($form, $form_state) {
   $form_state['rebuild'] = true;  // not helping
   $element = $form['box'];
   $element['#markup'] = "Clicked submit ({$form_state['values']['op']}): " .  $form_state['values']['replace_textfield']. " - ".  $form_state['values']['replace_textfielddd'] ." -.";
   return $element;
}

Thanks
EDIT
I also tried calling this function from the inside of command:
 function display_my_form(){
 $form = drupal_get_form('CPR_Calculator_form');

 $form['replace_textfielddd'] = array(
    '#type' => 'textfield',
    '#title' => t("bye"),
    '#description' => t("ummm"),
    '#prefix' => '
',
    '#suffix' => '
',
  );
  $display = '
';
  $display .= render($form);
  $display .= '
';

  return $display;
}

and then in the callback I replaced the ajax_command_append() with this line:
$commands[] = ajax_command_replace(NULL, display_my_form());

Answer: You need to build conditional elements in the form definition (mymodule_form) and not in the ajax callback. When the form is rebuilt using ajax, $form_state['values'] will be set, whereas it's not set on the initial form build.
Here is an example:
function mymodule_form($form, &$form_state)
{
  if(isset($form_state['values'])
  {
    // put your conditional form elements here
  }
}

Answer: Drupal will ignore any form data from elements not created with Form API, so you cannot expect elements created like this to work:
$commands[] = ajax_command_append(NULL, '
<textarea rows=4 cols=30 name=replace_textarea id=replace_textarea>Changethis value: '. $form_state['values']['changethis'] .'</textarea>');

You should create element using $form array instead, and then use callback function to return changed area of your form. See Form API example in Examples for Developers for correct way.
Comment: Many thanks, but when I try to add elements to the '$form'using a function and calling the function in the command, the same problem persists. the function I have is:'$form['replace_textfielddd'] = array(
  '#type' => 'textfield',
  '#prefix' => '',
  '#suffix' => '',
);
    $display = '';
    $display .= render($form);
    $display .= '';
    return $display;'
Comment: @ati you should create elements in form building function, using 'triggering_element' to know what to create. well, that's the most stable way for me so far
Comment: Thanks, but the problem is that my form is so dynamic that its elements depends on what the user inputs. With triggering_element I'd have to define my element at the beginning, no?
Comment: @ati your form building function is called **every time** AJAX needs it. I succesfully built form that creates fields based on "tags" in user input, and then creates configuration elements based on other configurations for each tag... Doable. And "already tried" code belongs to your question. Not to a comment.
Comment: Thanks for that, did you use triggering_element? I am not familiar with triggering_element. Ok, I am editing the question with more details. May be you could spot what I am missing in the function...
Comment: Sorry, I don't think I can provide full guidance. I'm active here on Q&A site and not on any Drupal forums specifically because I do not have time for discussions and mentoring.


Found an unexpected Mach-o Header code:0x72613c21 xcode 9.2

Question: First of all, I am new to xCode and have developed iOS app which has already been working in Android perfectly. But when I convert the same into Xcode, I found so many error that I removed myself gradually.
My problem is that I add the project in Xcode, add reference library
 etc and build the project. after so many error, finally my build start and end at BUILD SUCCESSFUL. Afterward, I start archiving the project and it completed but when I export it shows me Found an unexpected Mach-o header code. I google it and found so many solution but all in vain.

Critical.log 2018-06-18 17:15:43 +0000  Failed to generate
  distribution items with error: Error Domain=DVTMachOErrorDomain Code=0
  "Found an unexpected Mach-O header code: 0x72613c21"
  UserInfo={NSLocalizedDescription=Found an unexpected Mach-O header
  code: 0x72613c21, NSLocalizedRecoverySuggestion=}
Standard.log2018-06-18 17:15:37 +0000 [MT] Beginning distribution
  assistant for archive: kRide, task: Export 2018-06-18 17:15:42 +0000
  [MT] Proceeding to distribution step
  IDEDistributionAnalyzeArchiveStepViewController, context:
  ',
  distributionTask(resolved)='2',
  distributionMethod(resolved)='', team(resolved)='(null)'>    Chain (3, self inclusive):
    ',
  distributionMethod='', team='(null)'>  ',
  distributionMethod='', team='(null)'>  ',
  distributionMethod='(null)', team='(null)'>

Please suggest me where I am wrong and what I need to do to export the project to have .ipa file
Answer: try to DELETE BUILD folder from IOS directory and rebuild again with
react-native run-ios
Comment: I did so many things but all in vain. I did the same but same error.
Comment: ohh..remove the IOS folder and do react-native upgrade


.zshrc not working (accidentally deleted first line)

Question: I deleted the first line from my .zshrc and now nothing I put in .zshrc has any effect, I'm not sure what it was but it was there when I opened .zshrc the very first time.
All that is in my ~/.zshrc is the fallowing:
alias 'c=clear' 'cl=clear' 'hist=history' 'h=history' 'll=ls -l' 'v=cat'

PATH=$PATH:.
ZSH_THEME="fino"

None of the commands work, is there something else I need in the .zshrc to make it work?
Comment: While deleting that line *may* have caused the problems, the reason why it is not working now lies with what is still there. Without knowing the contents of your `~/.zshrc` it will be impossible to help.
Answer: Ah I found my answer… after too long.
All zsh commands that do not work in bash already need to be in-between
ZSH=$HOME/.oh-my-zsh

and
source $ZSH/oh-my-zsh.sh


How to make a workflow to stop and check a field's value and based on the value of the field proceed using SharePoint 2010 Designer

Question: i am badly stuck in one point and i am not finding any good alternative logic for it, let me explain it:
I have created a custom List WF using SP 2010 Designer.
The user submits a form request, including a date, that is routed to the handler and is either approved or rejected.
If the request is approved, then the workflow should pause until either:

the requester makes a change to the form (and it goes back for re-approval).
or the user won't do any changes and the WorkFlow should continue after checking whether the user brought changes or didn't

I used pause for a duration but the time after approval is not clear how long it should wait  to check whether the user brought changes or no? please help me out
Answer: Check if Date Created + 2 weeks (or whatever duration is when the WF should continue whether any changes have been made) <= (less than or equal to) Today's date, if yes, continue workflow, if not:
Compare date modified and date created to check if the item was modified after it's been added, if yes, reroute for approval.


Mediawiki create user error after migration

Question: So I had a mediawiki installed on windows with MySQL (running on AWS RDS). I've since moved it to a debian server for various reasons, but I think I've messed up the database because of the different versions of mediawiki I have used.
The windows install was v 1.20alpha (58f390e).
The new debian install is v 1.15.5-2squeeze4.
I've tried to update debian but it doesn't find an update, so is this the latest squeeze version?
Everything seems to work ok except adding users. It gives me a database error so I ran php maintenance/update.php which ran some stuff OK but didn't make a difference. I think I've not done the correct approach to this sort of move, does anyone know of a better way of doing it? I still have the old wiki running - but not used - on windows (using the same database) so I could always try this again.
Comment: I don't really see how this is off topic.
Answer: The simple fact is almost no web-apps are design to handle a down-grade.  Once a database has been updated, you usually can never go to an older version without causing problems.
I don't know much specifics about mediwiki, but my guess is that you have two choices.

Remove the packaged version of mediawiki you found on your Debian system, and you need to manually re-install, using the same version or newer version then what you were running.
Or simply, or not so simply, manually migrate/rebuild the entire site on the older version.
Comment: I think the best thing then is to wait for the debian version to be upgraded or to rebuild the whole thing.
Comment: Waiting for the Debian package is probably a bad idea.  Since it won't be updated until wheezy is released next year.
Comment: I ended up rebuilding the database. I've learnt not to switch os's with this kinda software!


How to apply css to all pages in custom module

Question: I have created one custom module in drupal-7. I want to apply same css to all pages belonging to that module.
I am able to apply css to one page by using drupal_add_css() but not able to do for all pages.
Answer: In module's info file, add following line:
stylesheets[all][] = folder/module.css

Substitute folder with a name of subfolder within your module you keep your CSS files in (or remove it). Repeat for all CSS files you need.
More about info files in the manual on Drupal.org.
Answer: 1- Place style.css in my_module folder
2-
in my_modulefolder/my_module.module
function my_module_preprocess(&$vars, $hook) {
  if($hook == 'page')
   drupal_add_css(drupal_get_path('module', 'my_module') .'/style.css');
}


Interpretation of Curvature formula for a parametric curve

Question: Given $S(t) = (x(t), y(t))$, the curvature at any point on S is given by below formula:
$$K = \dfrac{S'(t) \times S''(t)}{|S'(t)|^{3/2}}$$
Where $S'(t)$ is the first order derivative of $S(t)$
and $S''(t)$ is the second order derivative of $S(t).$

I know that the first order derivative gives the tangent vector function to the curve, but How do i interpret the second order derivative of a parametric curve?
In the above formula for curvature, how does more "perpendicular-ness" between $S'(t)$ and $S''(t)$ increases the curvature of the curve?
Comment: Aren't you missing the norm? In my book (wikipedia) it is: c(f) = | f' x f''| / |f'|^3. Note that | f' x f'' | measures the area of the parallelogram formed by f' and f'' which when f' and f'' are aligned is null, this makes sens since when velocity and acceleration are aligned their is no curvature.https://tutorial.math.lamar.edu/classes/calciii/curvature.aspx
Answer: Here's an intuitive explanation. The second derivative can be interpreted as acceleration. If you decompose acceleration in a tangential and orthogonal part with respect to the curve then the tangential part does not contribute to a change in direction (it only changes speed along the curve). So the curvature –how much you turn the steering wheel– depends only on the orthogonal part and your speed.
Comment: [+1] Good explanation... and this orthogonal part will be at its maximum when S' and S'' are orthogonal


Law of Large Numbers and Convergence in Probability of Quadratic Form

Question: Let $\{\mu_N\}_N$ be a sequence of $N$ dimensional vectors, and $\{\Lambda_N \}$ a sequence of $N \times N$ symmetric positive definite precision matrices. I drop the $N$ subscripts for simplicity of notation. Assume
\begin{equation}
\lim_{N \to \infty} \mu' \Lambda \mu = S
\end{equation}
where $S$ is a positive scalar. Assume $\bar X_T$ (which is the mean of $T$ iid random vectors) has a multivariate normal distribution
\begin{equation}
\bar X_T \sim N \left( \mu, \frac{1}{T} \Lambda^{-1} \right)
\end{equation}
The Question: How do you solve for the following probability limit?
\begin{equation}
\text{plim}_{\substack{N / T = c \\ N,T \to \infty}} \bar X_T' \Lambda \bar X_T
\end{equation}
for some positive scalar $c$.
Do the probability limit exist? If so, what is the answer?
Answer: I found an answer. Since $\Lambda$ is symmetric and positive definite, the square root $\Lambda^{1 / 2}$ exists and is unique. Define
\begin{equation}
\bar y_T = \sqrt{T} \Lambda^{1/2} \bar x_T \sim N \left( \sqrt{T} \Lambda^{1/2} \mu, I \right)
\end{equation}
Then we have
\begin{equation}
\bar x_T' \Lambda \bar x_T = \frac{1}{T} \bar y_T' \bar y_T
\end{equation}
Now $\bar y_T' \bar y_T$ actually has a noncentral chi-squared distribution with degrees of freedom $k$ and non-centrality parameter $\lambda$ equal to
\begin{equation}
k = N, \;\; \lambda = (\sqrt{T} \Lambda^{1/2} \mu)' (\sqrt{T} \Lambda^{1/2} \mu) = T \mu' \Lambda \mu
\end{equation}
Thus
\begin{equation}
E[x_T' \Lambda \bar x_T] = E \left[ \frac{1}{T} \bar y_T' \bar y_T \right] = \frac{1}{T} (k + \lambda) = \frac{1}{T} \left( N + T \mu' \Lambda \mu \right) = \frac{N}{T} + \mu' \Lambda \mu \to c + S
\end{equation}
\begin{equation}
\text{Var} [x_T' \Lambda \bar x_T] = \text{Var} \left[ \frac{1}{T} \bar y_T' \bar y_T \right] = \frac{1}{T^2} 2 (k + 2 \lambda) = \frac{2 N}{T^2} + \frac{4 \mu' \Lambda \mu}{T} \to 0
\end{equation}
Since the variance collapses to zero, it must be the case that
\begin{equation}
\text{plim}_{\substack{N / T = c \\ N,T \to \infty}} \bar X_T' \Lambda \bar X_T = S + c
\end{equation}


Accessing Network Share from Machine

Question: My coworker and I are attempting to setup a two node "cluster" in which each node needs to access a file on the first node. To make this simple (it's a dev environment), he created a network share in which the entire X drive on the first machine is shared (Everyone > All Rights). So, ultimately, I can get to the file by accessing \\xxx.xxx.xxx.xxx\X\filename.txt.
What's interesting, though, is that, while I can access the first machine from itself by using the IP address, or localhost, I cannot access it by using the name I gave it (which I resolve using the hosts file on both machines). \\my.machine.name\X\filename.txt gives me a login dialog (which I can't login with using the local account OR a domain account). However, the other machine can use \\my.machine.name\X\filename.txt and it can access the share fine. It's very confusing, and I haven't been able to figure out what's causing it.
Both boxes are Windows 2012. The account I am using is a local account (on both machines - same account name). The IPs are static. Both machines are running inside VMWare's vCloud and are in the same vapp. Any insight or even a good way to debug this would be much appreciated.
Comment: Are you using lmhost or host file? Netbios names resolution use lmhost file.
See http://support.microsoft.com/kb/105997/en
Comment: I am not, but adding one does not change anything. I am still getting a "Enter Network Password" box.
Comment: Are your servers in a domain? What are their host names? What is the domain name?
Comment: @Daniel - No, they are not in a domain. It's a cloud on a corporate network where you pretty much have free reign to build and destroy boxes as needed. I think my ultimate solution may be to setup DNS on the servers so everything resolves nicely. I just don't understand why the machines can see each other, but not the Windows share...
Comment: I reproduced your structure and it is working out of the box. You don't even need to change the hosts file as the servers resolve their hostnames via netbios. Installing a DNS server won't fix your problem.
Answer: This problem occurs if you access the SMB share via an alias that is not the servers hostname or FQDN.
You can disable strict name checking in the registry. Browser to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters and create a DWORD with the name DisableStrictNameChecking and value 1.
http://technet.microsoft.com/de-de/library/ff660057%28v=ws.10%29.aspx
Comment: Thanks for the response, but this didn't work either (unless I completely missed something).

Accessing Network Share from Machine


Linear approximation for an area between three circles

Question: I want to compute the area of a set bounded by three circles. Let $x,y \in [0,1]$, and $\theta \in [0,\pi]$, and consider the three disks $D_1, D_2, D_3$ with unit radius, and centers at the points $(0,0), (x,0), (y \cos \theta, y \sin \theta)$, respectively. What is $A_\theta(x,y) = area(D_1 \cap D_2^c \cap D_3^c)$? (The set $D_1 \cap D_2^c \cap D_3^c = D_1 \setminus (D_2 \cup D_3)$ is an intersection of two crescent shapes.)
It probably isn't possible to give a 'nice' formula for the area in terms of $x,y,$ and $\theta$: all I really care about is a linear approximation for $A_\theta(x,y)$ as a function of $x$ and $y$ as $x,y \to 0$, for fixed $\theta$. I suspect it should have the form
($\star$) $A_\theta(x,y) = c_1(\theta) x + c_2(\theta) y + O(xy, x^2, y^2).$
and I would like to know $c_1$ and $c_2$. Even if $c_1$ and $c_2$ can't be computed, I would still like to know if the Taylor series does have the form $(\star)$, or if not, determine what form it should have.
As an example, one can easily compute the area of a single 'crescent' $D_1 \cap D_2^c$: it essentially comes down to computing the area of intersection of two circles whose radii are $x$ apart. (The set $D_1 \cap D_2^c$ is the crescent shaped region outside the unit disk $D_1$ and inside the shifted disk $D_2$.) This yields
$area(D_1 \cap D_2^c) = \pi-2\arccos(x/2)+\frac{x}{2}\sqrt{4-x^2} = 2x - \frac{x^3}{12} + O(x^5)$ as $x \to 0$.
I can only handle the two trivial cases right now: when $\theta = \pi$, some simple estimates yield the bound $A_\pi(x,y) \leq 4 \sqrt{2} \max(x,y)^3$, so in that case $c_1 = c_2 = 0$, and $A$ decays faster than linearly as $x,y \to 0$. When $\theta = 0$, the intersection reduces to the case above (with just 2 circles), and the area is just $A_0(x,y) = 2 \min(x,y) + O(\min(x,y)^3)$.
I know there are formulas out there for intersections of multiple circles, but so far I've found them too general to apply here. Understanding the set analytically just seems too difficult right now: I can't even get a reasonable expression for the intersection point of the two circles $D_2$ and $D_3$ in terms of $x,y,$ and $\theta$. I think there may be hope in somehow dividing the intersection into two parts: one bounded by $D_1$ and $D_2$, and the other bounded by  $D_1$ and $D_3$, and analyzing these separately. But I haven't got that to work yet.
Edit: Added a picture for clarity. The shaded region is what I'm interested in.
Comment: Probably it is my lack of knowledge, but I lost you already when you define $A_\theta$. What does $D_2^2$ and $D_3^c$ mean?
Comment: Whoops, just a typo (now fixed): that should be $D_2^c$. $A_\theta$ is just the area of the set $D_1 \setminus (D_2 \cup D_3)$, as a function of $x,y,$ and $\theta$.
Comment: Oh thanks! The picture is very helpful. I will think about it and come back to you if I've an idea :)
Comment: BTW, I am sure that in your linear form, $c_1(\theta)$ and $c_2(\theta)$ will be zero, because you linearize around $x=0$ and $y=0$. This is easy to see: if $y$ stays zero, the area will always be zero, regardless of $x$. Vice versa, if $x$ stays zero, the area will always be zero, regardless of $y$. For an analytical expression, this paper might help: http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/4551/4/DSTO-TN-0722.PR.pdf
Answer: I found an analytical expression for $A_\theta(x, y)$. However, I could not get an easy expression. Define the following areas (for brevity, I omit the dependencies of $x$, $y$ and $\theta$):
\begin{align}
B &= D_1 \cap D_2^c \\
C &= D_1 \cap D_3 \\
D &= D_1 \cap D_2 \cap D_3
\end{align}
Then we have $A=B+D-C$. $B$ and $C$ are easy as you already provided an expression for $B$ and $C$ is almost similar:
\begin{align}
B &= \pi - 2 \arccos\left(\frac{x}{2}\right)+\frac{x}{2}\sqrt{4-x^2}, \\
C &= 2\arccos\left(\frac{y}{2}\right)-\frac{y}{2}\sqrt{4-y^2}.
\end{align}
For computing D, I used a paper as a reference. I tried to use the same symbols as much as possible. Let $(x_{ij},y_{ij})$ be the coordinate of the intersection between $D_i$ and $D_j$. Let $d_{23}$ be the distance between the centers of $D_2$ and $D_3$. Let $c_k$ with $k=1,2,3$ be one of the chords of D. Finally, let $\phi$ be the angle between the line going through the centers of $D_2$ and $D_3$ and the horizontal axis (the horizontal axis goes through $D_1$ and $D_2$. Now you can compute $D$ in the following way:
\begin{align}
x_{12} =& \frac{x}{2}, \\
y_{12} =& \frac12 \sqrt{4-x^2}, \\
x_{13} =& \frac12 \cos(\theta)y + \frac12 \sin(\theta)\sqrt{4-y^2}, \\
y_{13} =& \frac12 \sin(\theta)y - \frac12 \cos(\theta)\sqrt{4-y^2}, \\
d_{23} =& \sqrt{x^2 + y^2 - 2xy\cos(\theta)}, \\
\sin(\phi) =& \frac{y}{d_{23}} \sin(\theta), \\
\cos(\phi) =& \frac{d_{23}^2+x^2-y^2}{2d_{23}x}, \\
x_{23} =& x - \frac12\cos(\phi)d_{23} - \frac12\sin(\phi)\sqrt{4-d_{23}^2}, \\
y_{23} =& \frac12\sin(\phi)d_{23} - \frac12\cos(\phi)\sqrt{4-d_{23}^2}, \\
c_1 =& \sqrt{(x_{12} - x_{13})^2 + (y_{12} - y_{13})^2}, \\
c_2 =& \sqrt{(x_{12} - x_{23})^2 + (y_{12} - y_{23})^2}, \\
c_3 =& \sqrt{(x_{13} - x_{23})^2 + (y_{13} - y_{23})^2}, \\
D =& \frac14 \sqrt{(c_1+c_2+c_3)(c_2+c_3-c_1)(c_1+c_3-c_2)(c_1+c_2-c_3)} + \\
& \,\,\,\,\, \sum_{k=1}^3 \left(\arcsin\left(\frac{c_k}{2}\right) - \frac{c_k}{4}\sqrt{4-c_k^2} \right).
\end{align}
I checked the formulas by comparing the result with a numerical estimation (through Monte Carlo) and it seemed right. However, this is a far-from-easy expression. Perhaps one can simplify things. By taking the derivative with respect to $\theta$, you can compute the coefficients of your linear expression (you called them $c_1$ and $c_2$, but they should not be mixed with the chord length $c_k$).
Comment: Oh gosh. I guess I could try plugging this into Mathematica. I saw this paper, but was too afraid to write down the formulas: thanks for doing that in a clear way!
Comment: Why does $A = B + D - C$ hold? This seems incorrect to me.
Comment: I can see the following (trivial) equality: if $B = D_1 \cap D_2, C = D_1 \cap D_3$, and $D = D_1 \cap D_2 \cap D_3$, then $A = D_1 - B - C + D$.
Comment: Oops, I wrote $\cup$ instead of $\cap$... I changed it. Is it now clear for you?
Comment: BTW, the equality that you propose would also work :)
Comment: Aha, got it. Thanks.
Comment: Are you sure there are no more typos? I am trying to plug all this into mathematica, but right now the area is coming out negative for some values of $\theta$. I'm trying to cross reference everything with the paper you linked, and it looks OK. Just wanted to check with you @EdG, since you mentioned you got this to work in simulations...
Comment: Excuse me for my sloppiness. I forget to take the square root when computing $d_{23}$. Should be fixed now. I see now that there is another assumption I do, which is not always the case. I assume that $A$ is a single area, while if you have, e.g., $x=y=0.5$ and $\theta=\pi$ this is not the case!
Comment: OK, now it looks better. Still one strange thing I am noticing: swapping the values of $x$ and $y$ does not give the same result. There should be a symmetry under $x \longleftrightarrow y$, i.e. $A(\theta, x, y) = A(\theta, y, x)$, right? This seems to hold for $\theta$ larger than some threshold depending on $x$ and $y$, but I don't see why this would be.
Comment: Wow, you are very good at spotting bugs ;). I see what goes wrong. Especially when $y>x$ and small $\theta$, it might be that $\cos(\phi)$ becomes negative. However, the way I computed $\cos(\phi)$ always gives a positive value. This can be fixed by using the cosine rule instead of the $\cos(\phi)=\sqrt{1-\sin^2(\phi)}$
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/69786/discussion-between-j-richey-and-edg).


infinitely many integers such that ...

Question:
Let $a$ be an integer. Prove that there are infinitely many integers $b$ such that exists only a  prime number $p$ of the form $u^2+2au+b$ with $u$ integer.

I could only think that:
We can write the form of prime number $p$ as
$u^2+2au+b = (u + a)^2 - (a^2 - b)$
but I don't know how to continue.
Comment: There are no infinite integers, all integers are finite. Although there are *infinitely many* of them.
Comment: Something seems missing about the grammar(?) Does it mean "there are infinitely many integers $b$ such that..." and then end with "there exist only finitely  many prime numbers $p$ of the form ..."?
Comment: If you can prove there are infinitely many primes, then there are infinite integers $b$, no? And yeah, the grammar is incorrect.
Comment: ok, I apologize for the grammar. but the solution of the exercise?
Comment: "... such that exists only a prime number ..". this part of the text is correct.
Answer: ok. In both cases $y = \frac{p - 1}{2}$ and
we choose $(a^2 - b) = c^2$ and $c^2 = \frac{p - 1}{2}$.
Answer: Hint: $x^2-y^2=p$ (with $y>0$ fixed) is only prime if $x-y=1$ and $x+y=p$, or $x-y=-p$ and $x+y=-1$. In both cases, $y=2p+1$ determines $p$.
Now choose $b$ such that $a^2-b$ is of the form...

infinitely many integers such that ...


Showing by definition that a curve is a 1 dimensional differentiable manifold

Question:
Let $\gamma :(a,b) \to \mathbb{R}^d$ be a $C^1$ curve and $S = \left \{ x  |  \exists t\in(a,b) : \gamma(t)=x  \right \}$ its image.
Show that if $\gamma$ is injective and $\gamma' (t) \neq \vec{0}, \forall t \in (a,b)$, then $S$ is a one dimensional differentiable manifold in $\mathbb{R}^d$.

Just to clarify the definition we were taught: I need to show that $\forall x \in S$, there exists an open neighborhood $V \subset \mathbb{R}^d$ of $x$ s.t $S \cap V$ is the graph of a differentiable function $f: U \subset \mathbb{R} \to \mathbb{R}^{d-1}$.
I believe this should be a pretty basic exercise but I'm a little lost. Help appreciated.
*EDIT: The above question is false as mentioned, modifying the question this way:

Let $\gamma :(a,b) \to \mathbb{R}^d$ be a $C^1$ curve and $S = \left \{ x  |  \exists t\in(a+\varepsilon ,b-\varepsilon) : \gamma(t)=x  \right \}$ for some $\varepsilon>0$, the image of $\gamma |_{(a+\varepsilon ,b-\varepsilon)}$.
Show that if $\gamma$ is injective and $\gamma' (t) \neq \vec{0}, \forall t \in (a,b)$, then $S$ is a one dimensional differentiable manifold in $\mathbb{R}^d$.

how would we prove it?
Comment: This statement is false. For a discussion and a depiction of a counterexample, see the (somewhat poorly written) entry on wikipedia:  https://en.wikipedia.org/wiki/Submanifold#Immersed_submanifolds
Comment: Compactness is crucially a part of analysis. But here's the other way: You need to know that $\gamma^{-1}$  is continuous (as a map with domain $\gamma((a,b))$, of course). The point is that you'll need to know that each point of the image has a neighborhood in $\Bbb R^d$ that is just diffeomorphic to an interval in $\Bbb R$. Then you can apply the inverse/implicit function theorem. But yes, I think your instructor messed up.
Comment: There's a typo in the problem. The assumption should be $\gamma'(t)\ne 0$ for all $t$.
Comment: It's still false for the reason Lee Mosher has pointed out, though.
Comment: @TedShifrin Thanks for noticing, my bad
Comment: Yes, @Lee's reference to what I presume is the figure-eight is still a counterexample. You will need to add *proper* to make the problem correct.
Comment: @TedShifrin Doesn't being injective mean it's proper?
Comment: Absolutely not, @paxtibimarce. The definition of proper is that $\gamma^{-1}(K)$ is compact for every compact $K\subset\Bbb R^d$.
Comment: @TedShifrin Thanks for all the help, I have fixed the question, If you would like to take a look perhaps.
Comment: @LeeMosher what if $\gamma(a)$ and $\gamma(b)$ are explicitly not contained in the curve? I.e. the limit points at the two ends do not intersect the middle of the curve?
Comment: I doubt that your "graph definition" makes sense. The graph of $f : U \to \mathbb R^{d-1}$ is the set $\{(x,f(x)) \mid x \in U \} \subset \mathbb R^d$. But then $\{0\} \times \mathbb R$ would not be a one- dimensional differentiable manifold in $\mathbb R^2$.
Comment: @Paul Frost I don't see the problem, you just take the function which is identically zero in that case which is differentiable. (Notice we can choose which coordinates our function works on).
Comment: @paxtibimarce Then you get the $x$-axis as a graph. Or do you have an alternative definition of the graph of a function?
Comment: I'm not sure what you mean, this definition is for a differentiable manifold. It would be great if you could explain in more detail where is the problem.
Comment: @paxtibimarce Your  definition is that $S \subset \mathbb R^d$ is a $k$-dimensional manifold if it is locally the graph of some $f : U \to \mathbb R^{d-k}$ with $U \subset \mathbb R^k$ open. You use it only for $k=1$, but this is irrelevant. So what is the precise definition of the graph? If you agree that it is the set $G(f) = \{(x,f(x)) \in \mathbb R^d \mid x \in U\}$, then the  $y$-axis in $\mathbb R^2$ is not a $1$-manifold (which may be intended?) because in a graph we only have one point with a given $x$-coordinate. Alternatively you need a more general definition of what a graph is.
Answer: NOTE: we'll assume the function $\gamma$ proper, in order to avoid pathological cases (i.e. the curve getting infinitely close to itself - though without self-intersections). I thank Ted Shifrin for his observations and corrections on this proof.
I'll use the notation $\gamma(t)=(\gamma_1(t),...,\gamma_d(t))$.
For a fixed $t_0$ you can take one of the $d$ axis, say $\hat{x_1}$ (if not, $\hat{x_2}$...) such that $\gamma '(t_0)$ is not perpendicular to $\hat{x_1}$.
Then $\gamma_1'(t_0)\neq 0$ and I can take an interval on $\hat{x_1}$ where it remains $\neq0$. Over this interval, I reparametrise $Im\gamma$ as:
$\alpha(s)=(\gamma \circ \gamma_1^{-1})(s) = (\gamma_1 \circ \gamma_1^{-1},\gamma_2 \circ \gamma_1^{-1},...,\gamma_d \circ \gamma_1^{-1})(s) = (s, \alpha_2(s),...,\alpha_d(s)) $
by using that $\gamma_1' \neq 0 \Rightarrow \gamma_1$ bijective.
And there we have a local graph (if we restrict on $V\cap S$ with $V\subset\mathbb{R}^d$ a suitable open set)
Comment: Right. The question then is why shrinking to $S$ solves the problem that the proof would otherwise be wrong.
Comment: No, that isn't sufficient. Far-away points of the curve can approach your point ... or could before $S$ was introduced.
Comment: Think about the figure 8 example that we referred to in the original comments.
Comment: For the second line, I can't have that $\gamma'(t_0)$ is perpendicular to every axis. If it was, then $||\gamma'(t_0)||^2=(\gamma'(t_0)\cdot \hat{x_1})^2+...+ (\gamma'(t_0)\cdot \hat{x_d})^2=0$
Comment: Well, I skipped that because since I restrict $\gamma_1$, I should also restrict $\alpha$ and therefore $Im\alpha$
Comment: But why does that solve the problem? You need an open set in $\Bbb R^d$ whose intersection with the curve is *only* your graph.
Comment: You mean I should copy & paste the first two points of my other answer? I guess so.
Comment: Well I honestly thought that $\varepsilon$ had to be determined afterwards and the question was badly written.


2 player zero-sum-game rock paper scissors expected loss

Question: For the Rock-Paper-Scissors game, I am trying to determine the expected loss for P1.
The following matrix displays how much P1 has lost:
A:
| 0 1 -1 |
| -1 0 1 |
| 1 -1 0 |
I am trying to find the expected loss for P1 when P1 uses mixed strategy x = (x1, x2, x3) and P2 uses mixed strategy y (y1, y2, y3)
I am not sure how to do this?
EDIT:
My attempt is thinking that if I should always have a 1/3 chance of winning, right?
Comment: There are only $9$ things that can happen.  Figure out their probabilities and add up the expected gains and losses.
Comment: the solution proposed by saulspatz can be written elegantly in terms of $A$, $x$ and $y$
Comment: I will attempt this, and post an answer. Thank you for the feed back.
Comment: It's true that you can always break even by playing $x = (1/3,1/3,1/3)$. But you can possibly do better, depending on P2's strategy. If P2 always plays rock, then you can always play paper and win every time.
Comment: @Théophile thank you for the feedback. I know this is dumb question but I am trying so hard to understand it
Comment: You can take the payoffs from the matrix one by one and add them up, scaled by the corresponding probabilities. The probability that both players play rock is $x_1y_1$, so the loss for that is $0x_1y_1 = 0$. The probability that P1 plays rock but P2 plays paper is $x_1y_2$, with a corresponding loss of $1x_1y_2 = x_1y_2$. Do the same for the remaining elements and add everything together. You will end up with an expression involving all the $x_i$ and $y_j$.
Answer: The expected payoff for $P1$ when he plays the mixed strategy $x$ and his opponent plays the mixed strategy $y$ is $x A y^\top$.


what is the linear minimum mean squared estimator for y given x of the shaded region?

Question: A 2D random point (x,y) is uniformly distributed on the shaded region of the figure.

What is the linear MMSE estimator for y, given x?
This is what I have so far:
Since it's a linear estimator, I want my estimator to have a form as the following:
$$ \hat{y} = wx$$
Now I want to find $w$ with the least expected variance given x. So we want to find
$$\mathbb{E}[(y - \hat{y})^2 | x]$$
If I compute this I get:
$$\mathbb{E}[(y - \hat{y})^2 | x]= \int^{\infty}_{-\infty} (y - wx)^2 p(y|x) dy$$
After some algebra, I get the following:
$$
w = \left\{\begin{aligned}
&{3 \over 4x} &&: 0 \le x < {1 \over 2} \\
&{1 \over 2x} &&: {1 \over 2} \le x \le 1
\end{aligned}
\right.$$
and since we defined $ \hat{y} = wx$, we get
$$
\hat{y} = \left\{\begin{aligned}
&{3 \over 4} &&: 0 \le x < {1 \over 2} \\
&{1 \over 2} &&: {1 \over 2} \le x \le 1
\end{aligned}
\right.$$
This result bothers me because, first off, it's not linear and they are the same as the expected value of y given x, $\mathbb{E}[y | x]$.
Instead, if I take the unconditional expected value of it:
$$\mathbb{E}[(y - \hat{y})^2]= \int^{\infty}_{-\infty} \int^{\infty}_{-\infty}(y - wx)^2 p(x,y) dydx$$
I get
$$ w = {3 \over 4}$$
and
$$\hat{y} = {3 \over 4} x$$
Which one is the correct answer?
Comment: Piecewise linear does not qualify as linear and therefore cannot be the intended solution.
Comment: For each value $\alpha$ of $X$, $E[Y\mid X = \alpha]$ is the estimate of $Y$ that has minimum variance among all estimates of $Y$ when $X=\alpha$, that is, this estimate has least conditional variance given this specific condition that $X$ happens to have value $\alpha$.  $E[Y\mid X]$, which is a function of $X$, not $Y$, is the random variable that minimizes $E[(Y -g(X))^2]$ over all choices of measurable functions $g\cdot)$. In general as well as in this specific case, $E[Y\mid X]$ is a nonlinear function (a step function in this case) and that it is a _piecewise_ linear function ----
Comment: What's the exact problem statement? Your first estimator is piecewise linear if we consider functions of the form $\hat y=wx+b$.
Comment: (1) Often a linear estimator would be considered one of the form $y=wx + v$ for parameters $w$ and $v$.  (2) You haven't yet considered the distribution of $x$.
Comment: @whuber On (2), OP actually stated that $x,y$ is a uniformly distributed 2D variable.
Comment: @broncoAbierto I want to find a linear estimator $\hat{y} = wx$. The only reason I used piecewise is that we want to estimate least expected variance given $x$. So I thought I had to split it up into two different cases of $x$. Also wouldn't $\hat{y} = wx + b$ be an affine estimator and not linear estimator?
Comment: @bronco Yes, we can all see that.  The point is that a consideration of this fact will determine which answer is the right one.
Comment: @MoneyBall That is just a matter of convention. In analysis and other fields, a linear function is usually defined simply as $w^Tx+b$. Not in algebra, where it would violate the requisites for $b\neq 0$. In statistical inference (esp. machine learning), I've always seen the term "linear estimator" to refer to a function of the form $w^Tx+b$, so I would not worry. For that reason, I wanted to know how specific the problem statement was, to see if piecewise functions would be allowed.
Comment: @whuber Doesn't that automatically determine $p(x)$, considering that $p(x)=\int_{-\infty}^{\infty} p(x,y)dy$?
Comment: @broncoAbierto I guess my question boils down to this: Is linear minimum mean squared estimation a conditional expectation of least variance or unconditional expectation of least variarnce? Because if it's conditional, i think piecewise is the right solution, otherwise, the second solution is the correct solution.
Comment: @whuber It's not piecewise linear, it's a linear function that depends on $x$ which is given.
Comment: ---- doesn't make it a linear function in the accepted sense of the term: piecewise linear, Yes, linear No.
Answer: A complete solution to this problem can be found in the Solutions to Problem Set 13, ECE 313, Fall 2008, University of Illinois. The problem statement can be found here.  The linear MMSE estimate of $Y$ in terms of $X$ is
$$\hat{Y} = -\frac{4}{11}X + \frac{35}{44}.$$
As is generally accepted by statisticians and probabilists, a linear MMSE estimate is actually an affine estimate: the adjective "linear" means that the graph of the estimate is a straight line, and not "linear" in the linear algebra sense of $L(A+B) = L(A)+L(B)$ is the property that must hold for $L(\cdot)$ to be called a linear transformation.
Comment: @MoneyBall Draw the line and try to figure out if it is a good estimate.
Comment: Assuming that we are talking about "linear" in linear algebra sense, is the unconditional expectation of the variance(the second answer I wrote) the correct solution? If that is the case, we are not taking $x$(which is given) into consideration when we are computing our linear estimator
Comment: (+1).  MoneyBall, when you leave out the constant term, then indeed $\hat y = 3x/4$ is the least-squares solution.


Dynamic interaction between Rectangle Selector and a matplotlib figure

Question: I am working on a an interactive program which displays an image and lets the user select a part of this image through a RectangleSelector. The goal is that the program

first, displays in real time the minimum value of the three canals in the selected part of the image
save the coordinates of the rectangle when the user is happy with it, in order the program continues

Because my program project is big and covers much more than that, I have split it in several files. Here are they :

settings.py

import numpy as np

def init():

    global img # image
    global text_sig_red, text_sig_green, text_sig_blue

#    text_sig_red, text_sig_green, text_sig_blue = '', '', ''

    img = np.array([])

functions.py

import numpy as np
import settings

def line_select_callback(eclick, erelease):

#    'eclick and erelease are the press and release events'
    x1, y1 = int(eclick.xdata), int(eclick.ydata) # start position
    x2, y2 = int(erelease.xdata), int(erelease.ydata) # end position

    crop_img = settings.img[x1:x2,y1:y2]
    #print('shape film temoin', np.shape(film_temoin))

    sig_red = np.min(crop_img[:,:,0])
    sig_green = np.min(crop_img[:,:,1])
    sig_blue = np.min(crop_img[:,:,2])

    text_red = 'min red signal' + str(sig_red)
    text_green = 'min green signal' + str(sig_green)
    text_blue = 'min blue signal' + str(sig_blue)

    settings.text_sig_red.set_text(text_red)
    settings.text_sig_green.set_text(text_green)
    settings.text_sig_blue.set_text(text_blue)

def toggle_selector(event):
    if event.key in ['Q', 'q'] and toggle_selector.RS.active:
        toggle_selector.RS.set_active(False)
    if event.key in ['A', 'a'] and not toggle_selector.RS.active:
        toggle_selector.RS.set_active(True)

main.py

import numpy as np
import matplotlib.image as mpim
import matplotlib.animation as animation
import matplotlib.patches as patches
from matplotlib.widgets import RectangleSelector
from matplotlib import pyplot as plt

import functions as fct
import settings

settings.init()

settings.img = mpim.imread('pattern.png')

fig, ax = plt.subplots(1, 1)

ax_red = plt.axes([0.05, 0.5, 0.05, 0.02])
ax_green = plt.axes([0.05, 0.3, 0.05, 0.02])
ax_blue = plt.axes([0.05, 0.1, 0.05, 0.02])

settings.text_sig_red = ax_red.text(0.5,0.5,"",transform=ax_red.transAxes)
settings.text_sig_green = ax_green.text(0.5,0.5,"",transform=ax_green.transAxes)
settings.text_sig_blue = ax_blue.text(0.5,0.5,"",transform=ax_blue.transAxes)

ax.imshow(settings.img)

fct.toggle_selector.RS = RectangleSelector(ax, fct.line_select_callback,
                                           drawtype='box', useblit=True,
                                           button=[1, 3],  # don't use middle button
                                           minspanx=5, minspany=5,
                                           spancoords='pixels',
                                           interactive=True,
                                           rectprops=dict(facecolor='black',
                                                          edgecolor = 'black',
                                                          alpha=1.,
                                                          fill=None))

plt.connect('key_press_event', fct.toggle_selector)

plt.show()

Currently, I have trouble with the goal (1) of my program. It doesn't display dynamically the text text_sig_red.
And actually, I don't know how to save the coordinates of the rectangle when the user is happy with it. Maybe the user should stop touching the rectangle position and extents and click a button in order the program continue (This is linked to another problem I have)
PS : I have given here the random image I have used in this example for my question.
Answer: Hope your problem was solved by now, but if not:
The root cause of your problem with goal (1) is that the figure never explicitly gets updated after the initial show().
So one solution to this problem would be to add the following line to the end of the line_select_callback function in functions.py:
plt.gcf().canvas.draw()

(of course plt would need to be imported in this module as well)
Concerning the rest of your question, creating an obvious interaction such as a button is always a good choice.
Comment: Thanks a lot ! No this problem has not been solved before (like this other one : https://stackoverflow.com/questions/56312631/wait-till-the-user-click-a-button-in-a-matplotlib-figure-to-continue-the-program). Discussing about my project at my work, with my collegues, I have been told to re-write my program in a object-oriented way...


Storing URLs while Spidering

Question: I created a little web spider in Python which I'm using to collect URLs. I'm not interested in the content. Right now I'm keeping all the visited URLs in a set in memory, because I don't want my spider to visit URLs twice. Of course that's a very limited way of accomplishing this.
So what's the best way to keep track of my visited URLs?
Should I use a database?

which one? MySQL, SQLite, PostgreSQL?
how should I save the URLs? As a primary key trying to insert every URL before visiting it?

Or should I write them to a file?

one file?
multiple files? how should I design the file-structure?

I'm sure there are books and a lot of papers on this or similar topics. Can you give me some advice what I should read?
Comment: This is a pretty heavy duty problem, depends how heavy duty you want your solution :)  The biggest problem you'll run into is sites that put cookies and random numbers in their URLs, that leads to an infinite crawl space to get caught up in, at some point you'll need to start removing duplicates based on content.
Answer: I've written a lot of spiders. To me, a bigger problem than running out of memory is the potential of losing all the URLs you've spidered already if the code or machine crashes or you decide you need to tweak the code. If you run out of RAM most machines and OSes these days will page so you'll slow down but still function. Having to rebuild a set of URLs gathered over hours and hours of run-time because its no longer available can be a real blow to productivity.
Keeping information in RAM that you do NOT want to lose is bad. Obviously a database is the way to go at that point because you need fast random access to see if you've already found a URL. Of course in-memory lookups are faster but the trade-off of figuring out WHICH urls to keep in memory adds overhead. Rather than try writing code to determine which URLs I need/don't-need, I keep it in the database and concentrate on making my code clean and maintainable and my SQL queries and schemas sensible. Make your URL field a unique index and the DBM will be able to find them in no time while automatically avoiding redundant links.
Your connection to the internet and sites you're accessing will probably be a lot slower than your connection to a database on a machine on your internal network. A SQLite database on the same machine might be the fastest, though the DBM itself isn't as sophisticated as Postgres, which is my favorite. I found that putting the database on another machine on the same switch as my spidering machine to be extremely fast; Making one machine handle the spidering, parsing, and then the database reads/writes is pretty intensive so if you have an old box throw Linux on it, install Postgres, and go to town. Throw some extra RAM in the box if you need more speed. Having that separate box for database use can be very nice.
Answer: These seem to be the important aspects to me:

You can't keep the URLs in memory as RAM will get too high
You need fast existence lookups at least O(logn)
You need fast insertions

There are many ways to do this and it depends on how big your database will get.  I think an SQL database can provide a good model for your problem.
Probably all you need is an SQLite database.  Typically string lookups for existence check is a slow operation.  To speed this up you can create a CRC hash of the URL and store both the CRC and URL in your database.  You would have an index on that CRC field.

When you insert: You insert the URL and the hash
When you want to do an existance lookup: You take the CRC of the potentially new URL and check if it is in your database already.

There is of course a chance of collision on the URL hashes, but if 100% spanning is not important to you then you can take the hit of not having a URL in your DB when there is a collision.
You could also decrease collisions in many ways.  For example you can increase the size of your CRC (CRC8 instead of CRC4) and use a hashing algorithm with a bigger size.  Or use CRC as well as URL length.
Answer: It depends on the scale of the spidering you're going to be doing, and the kind of machine you're doing it on.  Suppose a typical URL is a string of 60 bytes or so, an in-memory set will take a bit more than 100 bytes per URL (sets and dicts in Python are never allowed to grow beyond 60% full, for speed reasons).  If you have a 64-bit machine (and Python distro) with about 16 GB of RAM available, you could surely devote over 10 GB to the crucial set in question, letting you easily spider about 100 million URLs or so; but at the other extreme, if you have a 32-bit machine with 3GB of RAM, you clearly can't devote much more than a GB to the crucial set, limiting you to about 10 million URLs.  Sqlite would help around the same range of sized where a 32-bit machine couldn't make it but a generously-endowed 64-bit one could -- say 100 or 200 million URLs.
Beyond those, I'd recommend PostgreSQL, which also has the advantage of being able to run on a different machine (on a fast LAN) with basically no problems, letting you devote your main machine to spidering.  I guess MySQL &c would be OK for that too, but I love PostgreSQL standard compliance and robustness;-).  This would allow, say, a few billion URLs with no problems (just a fast disk, or better a RAID arrangement, and as much RAM as you can afford to speed things up, of course).
Trying to save memory/storage by using a fixed-length hash in lieu of URLs that might be quite long is fine if you're OK with an occasional false positive that will stop you from crawling what's actually a new URL.  Such "collisions" need not be at all likely: even if you only use 8 bytes for the hash, you should only have a substantial risk of some collision when you're looking at billions of URLs (the "square root heuristic" for this well-known problem).
With 8-bytes strings to represent the URLs, the in-memory set architecture should easily support a billion URLs or more on a well-endowed machine as above outlined.
So, roughly how many URLs do you want to spider, and how much RAM can you spare?-)
Answer: Are you just storing URL's?  You should take a look at mongoDB.  It's a NoSQL database that's pretty easy to implement.
http://try.mongodb.org/
It's got python bindings, too:
http://api.mongodb.org/python/1.5.2%2B/index.html
Answer: Since it's likely that you'll see similar URLs at similar times (e.g., while spidering a website, you'll see lots of links to the main page of the website) I would advise that you keep the URLs in a dictionary until your memory becomes limited (just hardcode a reasonable number like 10M URLs or similar) and then flush the dictionary to a CDB database file when it becomes too large.
That way, the majority of your URL checks will be in memory (which is fast) while the ones that aren't in memory will still require only 1-2 reads from disk to verify that you've visited them.
Answer: Consider Pickling for now: Simple structured storage.
Mileage will vary of course because, as the other responders have said, you'll quickly exhaust your RAM.

Storing URLs while Spidering


Como ler um arquivo txt de apenas uma linha em VBA?

Question: Gostaria de ler um arquivo txt através do VBA.
Todos os códigos que encontrei na internet fazem um loop no arquivo(para pegar todas as linhas suponho eu), mas eu queria pegar apenas a primeira linha de forma mais simples e colocar numa variável.
Eis meu código atual, que pega o texto porém não estou achando legal:
Dim conexao As String

Dim ReadData As String

Open "C:\Sistema\Conexao.txt" For Input As #1

Do Until EOF(1)
   Line Input #1, ReadData
   conexao = ReadData
If Not Left(ReadData, 1) = "*" Then
End If

Loop

Close #1

Answer: O melhor que pude chegar foi da seguinte maneira:
Sub Buscar_Primeira_Linha()

Dim LINHA As Integer

    ' Configura o número da linha que deseja
    LINHA = 1 ' Primeira Linha, por exemplo

    Ler_Arquivo "C:\Arquivo.txt", LINHA

End Sub

Creio que assim funcione... segue a função Ler_Arquivo:
Function Ler_Arquivo(ByRef ARQUIVO As String, _
                 Optional ByRef LINHA As Integer) As String

Dim COUNT As Integer

    With CreateObject("Scripting.FileSystemObject").OpenTextFile(ARQUIVO)

        If LINHA < 1 Then LINHA = 1
        COUNT = 1

        Do While Not .AtEndOfStream

            If COUNT = LINHA Then
                Ler_Arquivo = .ReadLine
                Exit Do
            Else
                .SkipLine
            End If
            COUNT = COUNT + 1
        Loop
       .Close
    End With

End Function

Espero ter ajudado!
Answer: Para lêr apenas a primeira linha, não utilize o ciclo Do.
Contudo, parece-me que este código faz alguma validação para procurar a primeira linha começada (ou não) por "*". E sendo este o caso, o ciclo é necessário.
Answer: Function LerArquivo()
NomeArquivo = "C:\Dados\meuarquivo.txt"
Open NomeArquivo For Input As #1
LerAte = 2

Do While Not EOF(1)

    If Linha < LerAte Then Line Input #1, Texto: Linha = Linha + 1

    If Linha >= LerAte Then Exit Do

Loop

Close #1
End Function


Change tab color when there is a span present inside a div

Question: I have a two different tabs which has the following HTML code,

.tabs{
    margin-left: 10px;
    color: black;
    width: 100%;
}
.all-tab{
    margin-left: 15px;
}
.first-tab{
    margin-left: 0%;
}
.tab {
    position: relative;
    list-style: none;
    display: inline-block;
    width: 13%;
}
.tab-left {
    float: left;
    width: 17px;
    height: 30px;
    border-radius: 1px;
    margin-left: 5px;
    background-color: #686868;
    transform: skew(-20deg);
}
.tab-centre {
    float: left;
    margin-left: -12px;
    line-height: 30px;
    width: 90%;
    background-color: #686868;
    text-align: center;
    color:white;
}
.tab-right {
    position: absolute;
    float: left;
    margin-left: -7px;
    width: 20px;
    height: 30px;
    border-radius: 1px;
    background-color: #686868;
    transform: skew(20deg);
}

ALL

first

When I click on first tab it should change the color of both the text and the entire tab until the next tab is selected. I've tried differenty CSS combinations but it was of no use. Is there a way that span inside div can change color?
Comment: Have you considered using javascript/ jQuery to detect the click and add a class?
Answer:
You need to fix your Html structure, when wrapping the elements with 'ul' the immediate children of the element must be li.
The best solution is to wrap the content inside each li with 'a' element, then even if you click on span it will affect the wrapper DIV.
Look at the following example that shows you how to give a different design to each element inside 'a' when clicking on the element.
to keep the element mark you better use Javascript in order to mark the focused elements(Add a class to the clicked li).

<!DOCTYPE html>

  <meta charset="UTF-8" />
  <title>Document</title>

     first

        second


Authorization data saved in xmlhttprequest with VBA

Question: I am calling a Webservice with basic authentication with VBA. The problem I have is that the user/password is stored after the first successful call. That means I can put wrong user/password in the next calls and it works just the same.
Set objRequest = CreateObject("MSXML2.XMLHTTP")
blnAsync = True
With objRequest
    .Open "GET", strUrl, blnAsync
    .setRequestHeader "Authorization", "Basic " + EncodeBase64(user + ":" + passw)
    .Send
    'spin wheels whilst waiting for response
    While objRequest.readyState <> 4
        DoEvents
    Wend
    strResponse = .responseText
End With

If I call the Webservice from browser it is the same. First time I need to login, next time it works without login. After I delete cache/cookies/history I need to login again.
My questions:

Where is this (cache?) data stored if I call from VBA and how to delete this?
How to prevent VBA from saving the authorization data?
Answer: I found the crucial hint to put user/password in the .Open-method here: VBA XMLHTTP disable authentication pop-up
That's how it works for me. The only thing I can't do now is encoding the user & password.
Dim objRequest As MSXML2.XMLHTTP30
Set objRequest = New MSXML2.XMLHTTP30

With objRequest
    .Open "GET", strUrl, blnAsync, user, passw
    .setRequestHeader "Cache-Control", "no-cache"
    .setRequestHeader "Pragma", "no-cache"
    .setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
    While objRequest.readyState <> 4
        DoEvents
    Wend
    strResponse = .responseText
End With

Answer: To stop the caching, you could try adding:
.setRequestHeader("Cache-Control", "no-cache");
.setRequestHeader("Pragma", "no-cache");
.setRequestHeader("If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT");

See also VBA XMLHTTP clear authentication? for more information.
Comment: Thanks for your Answer. It is the same behavior as in your linked item. The solution works, if the authentication data is sent in the code. If the data is wrong or not sent, a login popup appears. If I use this popup the user/password is stored for the next webservice calls. I want to prevent that this popup comes up...


java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration

Question: I keep getting this error.I've included the hadoop commons and the core libs in the classpath but still i keep getting this error.Help would be highly appreciated
Comment: @Jens i got the **classNotFound** error  when i ran it from command line with the help of maven
Comment: Where do you run your code?
Comment: i run it in my localhost
Comment: Do run it in eclipse or in a tomcat or from command line?
Comment: @Jens ran it in eclipse   ..
Now it gives this error
WARN util.NativeCodeLoader:
 Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

**java.lang.UnsupportedOperationException: Not implemented by the DistributedFileSystem FileSystem implementation**
 at org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:214)
 at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2559)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2569) ....
Comment: I run it on apache tomcat? is there any issue with that?
Comment: Did you manage to get this working? There is no answer accepted here.
Answer: Here's how to troubleshoot:  Look inside the jar that you're executing to see if that class file is actually there:
jar tvf target/my-jar-with-dependencies.jar | grep hadoop/conf/Configuration.class

If it's not, you need to add it to your classpath or change the way your jar is packaged.
Are you using Maven or some similar build tool?  You may have a dependency with a 'scope', which means that it will only be compiled into your jar in certain circumstances.
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>${hadoop.version}</version>
        <scope>provided</scope>
    </dependency>

In this example, the scope tag tells Maven that you're using this dependency for building, but it indicates that the dependency will be provided during runtime, so you'll either need to remove this tag or add the hadoop jar using -cp=/path/to/jar.jar during runtime.  Another example of a scope like this is 'test', which indicates that the jar is only needed in the path during unit tests.
Answer: make sure the classpath in your jar. You can check it like mark said;
jar tvf target/my-jar-with-dependencies.jar | grep hadoop/conf/Configuration.class

Answer: add dependency to hadoop-core.
Comment: Read the post: the OP already has the core libs in the classpath.


How to get access to external libraries and header files VC10

Question: I'm going to get a book that uses Xerces C++. I'm currently using VS 2010 ultimate, so the program doesn't miss any features. I've downloaded the precompiled binaries for windows x86 VC10. My question is what do I have to do to my environmental variables and files to be able to have Visual studio import the header files i.e. #include . I have no experience linking external libraries, so I need a simple explanation.
Comment: Why don't you get a book that uses Visual C++?
Comment: I do have 1 book on Visual C++, but it only uses the standard library. Ivor Horton's Beginning Visual C++ 2010.
Comment: You have to set up Visual Studio's C++ project settings to configure the Include path for Xerces' headers, and the Lib path for its lib files. I am assuming you mean [Xerces XML Parser](http://xerces.apache.org/xerces-c/)
Comment: @birryree yes, I am referring to the XML parser, could you elaborate as to how I configure the include path and lib path?
Comment: @seljuq70 - I've answered this type of question before so I won't keep making repeat answers, so check out: http://stackoverflow.com/questions/4290249/mysql-mysql-h-file-not-found-in-vs2008-c-beginner-question/4290261#4290261
Comment: @birryree Was that VS08, because that menu doesn't appear at all in VS10. In the VS I'm using all I see is "VC++ Directories editing has been deprecated." and "VC++ Directories are now available as auser property sheet that is added by default to all projects."
Answer: Right click on your project -> Properties -> C/C++ -> General -> Additional Include Directories
Include the path of Xerces include files.
For Lib Properties Linker  -> General -> Additional Library Directories
Include the path of Xerces lib files.
In Linker  -> General -> Input -> Additional dependencies
Add the libs required for linking.
Comment: thanks, that worked completely. However, isn't there a way to grant all my programs access to the DLL files, without having to copy and paste them to my programs debug folder every time?
Comment: I've managed to add the .lib files, link the library directory, and added the path to the include files. My program compiles, but the program fails and says the dll files are missing. The files are in the bin folder of the xerces file I downloaded, what do I do about those?
Comment: @seljuq70 - For dll files put them in the folder, from where the exe is running.


Limit on number of features in Cox Proportional Hazards model with penalized spline

Question: I am trying to fit a Cox proportional hazards model with penalized splines using the survival R package with the following, but it seems like I am hitting a limit on the number of coefficients that can be estimated given the total number of observations in the dataset.
For example,
library(survival)

data(lung)
lung <- na.omit(lung)
lung <- data.frame(apply(lung, 2, as.numeric))
nrow(lung) # 167 rows
model <- coxph(Surv(time, status) ~ pspline(inst) + pspline(age)
               + pspline(ph.ecog) + pspline(ph.karno) + pspline(pat.karno)
               + pspline(meal.cal) + pspline(wt.loss) + sex, data=lung,
               method='breslow')

This yields Error in coxpenal.fit(X, Y, istrat, offset, init = init, control, weights = weights,  : NA/NaN/Inf in foreign function call (arg 5).
But when I double the number of observations:
lung.2x <- as.data.frame(lapply(lung, rep, 2))
nrow(lung.2x) # 334 rows
model.2x <- coxph(Surv(time, status) ~ pspline(inst) + pspline(age)
                  + pspline(ph.ecog) + pspline(ph.karno) + pspline(pat.karno)
                  + pspline(meal.cal) + pspline(wt.loss) + sex, data=lung.2x,
                  method='breslow')

This runs.
Could anyone provide some insight as to why the model won't fit on the smaller dataset? Looking at model.2x$coefficients, I see that there are 85 coefficients estimated which is still fewer than the number of observations in the smaller dataset.
(If it helps - the default values for pspline() are 4 degrees of freedom and degree = 3, i.e. a cubic spline.)
Comment: It's the number of events, not the number of total cases, that matters for these considerations in Cox regression. Even if a fit is possible with a low event/predictor ratio, once you start getting down toward a 10/1 ratio you are at risk of overfitting.
Answer: You are trying to apply splines to predictors that aren't continuous variables, while not recognizing that one of those variables had only a limited number of distinct values.
In the lung data "inst" is an institution code that should be treated as an unordered 17-level factor or, better, as a random effect; "ph.ecog" is a potentially 5-level ordered performance score that only takes on 4 levels in your complete-cases subset.
I have no problem with fitting the model when those 2 predictors are handled appropriately: frailty(inst) to treat institutions as a random effect, and ordered(ph.ecog) to use ECOG score as an ordered categorical predictor. The model:
coxph(formula = Surv(time, status) ~ frailty(inst) + pspline(age) +
    ordered(ph.ecog) + pspline(ph.karno) + pspline(pat.karno) +
    pspline(meal.cal) + pspline(wt.loss) + sex, data = lung,
    method = "breslow")

is probably terribly overfit with only 120 events, but there is no convergence problem.
Your convergence problem seems to have stemmed from the "ph.ecog" predictor. It only has 4 levels in your complete-cases subset, and the default for pspline()is thus too many at 4 df. Taking your original model and just changing the corresponding term to pspline(ph.ecog,2) led to no technical difficulty.


In the zabbix project, how to export historical data about CPU and memory in the past four hours from mariaDB?

Question: I can capture the real-time records of CPU and memory in the web interface of zabbix. The history table in the mariaDB database has data. How should I export the historical data of the past four hours?
enter image description here
Answer: Historical data is a history-related table that exists, and to get historical data from the CPU or memory, you need to get its itemid first.Find out the host's host from the host table, check its monitoring items according to the hostid, find out the data object you want to get, and query from the history table.
<code>select itemid,from_unixtime(clock) as time,value from history where itemid=29096 and clock >= unix_timestamp('2021/05/21 00:00:00') and clock <= unix_timestamp('2021/05/21 17:31:00');
</code>


What is wrong with my proof that, given two convergent sequences with $L\neq0$, that for all $n>max(N_1,N_2)$, $\frac{a_n}{L_a} + \frac{b_n}{L_b} = 2$

Question: The proof uses the fact that the limit of a convergent sequence is unique. I accidently found this when proving that the limit of the product of two convergent sequences is equal to the product of their limits.
Let $a_n$ and $b_n$ be convergent with limits $\neq 0$. Then, we have both
$n_a > N_a \Rightarrow |a_n - L_a| < \epsilon$ and $n_b > N_b \Rightarrow |b_n - L_b| < \epsilon$. Let $N = max(N_a, N_b)$.
Then, $n > N \Rightarrow |a_n - L_a||b_n - L_b| < \epsilon$. Expanding the product of the  two terms inside the absolute value signs, we obtain $a_n b_n -L_b a_n - L_a b_n +L_a L_b$. Keeping the $a_n b_n$ terms as they are, and factoring out a factor of $L_a L_b$ from the remaining terms, gives $a_n b_n -L_b L_a (\frac{a_n}{L_a} + \frac{b_n}{L_n} - 1)$. However, from the uniqueness of a sequence's limit, and the fact that we can prove the (intended) theorem (that the limit of the product of two convergent sequences has limit $L_a L_b$ without contradicting the expansion I have derived so far, it follows that $L_b L_a (\frac{a_n}{L_a} + \frac{b_n}{L_n} - 1) = L_b L_a \implies \frac{a_n}{L_a} + \frac{b_n}{L_n} - 1 = 1) \implies \frac{a_n}{L_a} + \frac{b_n}{L_n} = 2$.
From this, I would expect that $n > N \implies \frac{a_n}{L_a} + \frac{b_n}{L_n} = 2$ for all such $n$. However, when I have tried this with the functions $\frac{n+2}{2n+1}$ and $\frac{3-8n}{n+2}$, using both Excel and my Casio fx-991ES PLUS for n = 10000000, I almost obtain a value of 2, but it ends up being equal to about 1.9999999125. Is this due to the computer being unable to work to the required accuracy, or have I missed some steps in the proof?
Answer: There are a few issues with what you've written.
First it does not always hold that
$$|a_n - L_a|\cdot|b_n - L_b| = a_nb_n - L_ba_n - L_ab_n + L_aL_b$$
for example take $a_n = 1, L_a = 2, b_n=2,L_b=1$. Then
\begin{align}
   |a_n - L_a|\cdot|b_n - L_b| &= |-1|\cdot|1| = 1 \\
   a_nb_n - L_ba_n - L_ab_n + L_aL_b &= (1)(2)-(1)(1)-(2)(2) + (2)(1) = -1
\end{align}
what you should have is
$$|a_n - L_a|\cdot|b_n - L_b| = |a_nb_n - L_ba_n - L_ab_n + L_aL_b|.$$
Second you have written that
$$L_bL_a\left (\frac{a_n}{L_a} + \frac{b_n}{L_b} - 1 \right) = L_bL_a$$
when really you should have
$$
L_bL_a\left (\frac{a_n}{L_a} + \frac{b_n}{L_b} - 1 \right) = a_nb_n.
$$
and taking limits on both sides you get
$$
\lim_{n\rightarrow \infty}L_bL_a\left (\frac{a_n}{L_a} + \frac{b_n}{L_b} - 1 \right) = \lim_{n\rightarrow \infty}a_nb_n = L_aL_b.
$$
The thing about limits is that they don't say anything about equality for a fixed $n$, in particular there is no reason to expect that for any $n$ that
$$L_bL_a\left (\frac{a_n}{L_a} + \frac{b_n}{L_b} - 1 \right) = L_aL_b.$$
For example,
$$\lim_{n\rightarrow \infty} \frac{1}{n} = 0$$
but for all finite $n$ we have $1/n \neq 0$.
Overall you really should approach this limit in the following way
\begin{align}
   \lim_{n \rightarrow \infty} \frac{a_n}{L_a} + \frac{b_n}{L_b} &= \frac{1}{L_a} \lim_{n \rightarrow \infty} a_n + \frac{1}{L_b} \lim_{n \rightarrow \infty} b_n \\
&= \frac{1}{L_a}L_a + \frac{1}{L_b}L_b = 2,
\end{align}
but again there is no reason to expect an exact equality for any fixed $n$.


WSS admin panel is 404ing on fresh install

Question: I am banging my head against the wall here. I have basically a fresh install over server 08, TFS 2010 (don't get me started on TFS 2008), and WSS 3.0 which was packaged and installed with TFS. Unfortunately TFS is boned because WSS3.0 and WSS 3.0's Administrator Panel are unreachable. The app pools are running fine in IIS and previously, it would prompt for a username and password and then 404 (it is no longer doing so).
Has anyone ever experienced this? If so, what did you do to correct it / what should I look for? Thank you so much in advance!
Answer: Check your IIS logs. The 404 has a lot of subcategories.
It sounds like the handler may not be enabled, tho.
Comment: I'm sorry for this stupid question, but what handler?
Comment: In IIS manager, click the site, and choose "Handler mappings".
Answer: The Windows Event log might turn some details.
I would also look at Sharepoint logs (under the 12 hive).
Can you connect to your DB server from your web server?
Have you setup the correct account permsisions? link
Are you sure the right version of ASP.Net is mapped to the website?
Answer: Are you trying to access from the WFE server?
There is a security feature in WS2003 SP1 and WS2008 which blocks requests coming from the machine hosting the application. Here is a link to a good blog post on the details of the problem and best practice for overcoming it http://harbar.net/archive/2009/07/02/disableloopbackcheck-amp-sharepoint-what-every-admin-and-developer-should-know.aspx
A hard problem to diagnose as you won't see anything useful in logs etc. because nothing is actually failing!!
Comment: ok I didn't read the question correctly. This problem causes 401
Answer: Try re-running the SharePoint Products and Technologies Configuration Wizard to re-establish Centeral Administration.  Once you get that up and running you can make whatever other configuration changes are needed to get the rest of SharePoint going.


Projection of vector onto subspace

Question: I am studying MIT open courseware linear algebra course. This is the question posed in one of the quiz:

If $q_1$ and $q_2$ are any orthonormal vectors in $\mathbb{R}^5$, give a formula for the projection $p$ of any vector $b$ onto the plane spanned by $q_1$ and $q_2$ (write $p$ as a combination of $q_1$ and $q_2$).
Answer:
$$p=(q_1 T_b)q_1+(q_2 T_b)q_2$$
Can someone explain why is the answer this? I cannot seem to understand.
Comment: Google "orthonormal basis", "orthogonal projection" and/or "Gram-Schmidt Process"
Answer: Since the vectors $q_1$ and $q_2$ are orthonormal, you can picture them as direction vectors in the plane spanned by them. The component of the vector $b$ in the direction $q_i$ is given by the inner product $<b,q_i>$. So, you get that the projection $p$ of $b$ to the plane spanned by $q_i$ where $q_i\in{1,2}$ is:
$p=\sum_{i}<b,q_i>q_i=<b,q_1>q_1+<b,q_2>q_2$
Comment: Why multiplication by qi
Answer: Hints:
Denoting by $\;\langle,\rangle\;$ the usual inner (scalar) product in $\;\Bbb R^n\;$ , and using your notation, check that
$$V:=\text{Span}\,\{q_1,\,q_2\}\;,\;\;proj_Vb=\langle b,\,q_1\rangle\,q_1+\langle b,\,q_2\rangle\,q_2\implies V\perp(b-proj_vb)$$
Since $\;\{q_1,q_2\}\;$ is orthonormal, the above is simply the expression of a vector in $\;V\;$ which is the orthogonal projection of a general vector$\;b\in\Bbb R^n\;$ on the subspace $\;V\;$ , and such that $\;\left\|b-proj_Vb\right\|\;$ is the (minimal) distance (in the sense of the norm induced by the inner product) vector from $\;b\;$ to $\;V\;$ .

Projection of vector onto subspace


Unable to build the spring boot project with custom dependency using docker

Question: I am working on the spring boot project and I have multiple microservices. In order to keep the redundant code in all modules, I separated them and made a single project, and added this as a dependency in every module.
My scenario is, let's say
I have a commons spring boot project and A and B projects. Now I am able to add commons dependency in project A and project B as
<dependency>
 <groupId>com.example </groupId>
 <artifactId>commons</artifactId>
 <version>1.0.0</version>
</dependency>

and the commons pom.xml has the below
     <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>repackage</id>
                        <configuration>
                            <classifier>exec</classifier>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

If I run the below command on A and B projects, I can see build success
mvn clean install
Now the problem arises for me If I try to create the docker images for projects A and B. In this case, the commons module is not taking it as a dependency in both.
I have used the below sequence of docker command to create an image
#
# Build stage
#
FROM maven:3.6.0-jdk-8-slim AS build
COPY src /home/app/src
COPY pom.xml /home/app
RUN mvn -f /home/app/pom.xml clean package -Dmaven.test.skip=true

#
# Package stage
#
FROM openjdk:8-jre-slim
COPY --from=build /home/app/target/A.jar /usr/local/lib/A-0.0.1-SNAPSHOT.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/lib/A-0.0.1-SNAPSHOT.jar"]

Could anyone help me to add the commons dependency to project A?
Note: we are using AWS managed container image registry service to store the images.
Thanks in advance :)
Comment: If you build a spring boot application this will be names like your artifactId you have used in your project and the classifier you have specified in the configuration of the spring-boot-maven-plugin that means someting like `whateverartifactid-version-exec.jar` which is the one you have to copy within your docker container and also using to start your app...
Comment: Thanks for the comment @khmarbaise. Are you saying, the whateverartifactid-version-exec.jar, we have to move to docker container before building the project A. If so, could you please provide me the series of steps.
Comment: The thing is you are building inside the container but you have to copy that because that's the artifact which is being build by spring-boot-maven-plugin which contains all dependencies.. exception you want to build a multi layered spring boot app but for that the approach is wrong..
Answer: As i understand it, you haven't pushed your dependency to a remote maven artifact repository, hence it's only available in your local machine under ~/.m2/repository/com/example/commons/1.0.0/.
When you try to compile, maven will check your local machine first to see if the dependency is available, else it will reach out to maven central (or any other configured repo) to resolve and retrieve the artifact. Since you do this inside a docker container where the artifact isn't available, it will fail.
So my suggestion is to try:
Option 1
Add a row in your Dockerfile to copy this artifact into your docker container before running mvn clean package:
COPY ~/.m2/repository/com/example/commons/1.0.0/ ~/.m2/repository/com/example/commons/1.0.0/

Option 2
Setup an artifact repository and store your artifacts there
Answer: Finally, figure out the solution with the below set of commands
#
# Build stage
#
FROM maven:3.6.0-jdk-8-slim AS build

COPY /commons/src /home/commons/src
COPY /commons/pom.xml /home/commons
RUN mvn -f /home/commons/pom.xml clean install -Dmaven.test.skip=true

COPY A/src /home/A/src
COPY A/pom.xml /home/A/
RUN mvn -f /home/A/pom.xml clean package install:install-file -Dfile=/home/commons/target/commons-0.0.1-SNAPSHOT.jar -DgroupId=com.example -DartifactId=commons -Dversion=0.0.1-SNAPSHOT -Dpackaging=jar -DgeneratePom=true -Dmaven.test.skip=true
#
# Package stage
#
FROM openjdk:8-jre-slim
COPY --from=build /home/A/target/A.jar /usr/local/lib/A.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/lib/A.jar","--spring.profiles.active=test"]


Combinação de valores distintos

Question: Considerando um dataframe com 2 colunas pretende-se, para cada valor distinto da coluna origem:

selecionar os valores correspondentes presentes na coluna destino
formar combinações 2 a 2 de elementos distintos com base nos valores do ponto 1
registar resultado num novo dataframe.

Exemplo de aplicação:
df <- data.frame(
    origem = c("A","A", "A", "E", "E", "D", "D", "D", "D"),
    destino=c("B","A","C","C", "B", "A","A", "A","B"))

Para A (coluna origem) temos associados os valores: A, B e C. Formando combinações 2 a 2 dos elementos apurados teria A-B, A-C, B-A, B-C, C-A, C-B. O objetivo seria obter a tabela final:
   origem x y
1       A B A
2       A B C
3       A A B
4       A A C
5       A C B
6       A C A
7       E C B
8       E B C
9       D A B
10      D B A

Neste exemplo tenho 3 valores únicos na "origem": A, E e D. Cada um desses valores tem pelo menos um correspondente em "destino" (valores únicos com origem em A: A, B, C; valores únicos com origem em E: C e B; valores únicos com origem em D: A e B).
Consigo obter as combinações pretendidas quando filtro por um valor na "origem" no entanto, necessitava generalizar para todos os valores presentes nessa coluna. Alguém consegue ajudar?
Comment: Porque é que para origem D e E só há duas linhas no resultado final? Com origem E também temos 3 letras e 6 pares sem repetições. O mesmo para D.
Comment: Obg pela questão. Editei a pergunta colocando detalhe. O valor E na "origem" tem apenas 2 observações: (E, **C**) e (E, **B**), o que pretendo é combinar os valores "**destino**" 2 a 2 (C, B) i.é: C-B e B-C ( o valor E serve para split da df e apuramento dos destinos a combinar )
Answer: Uma solução, usando data.table:
library(data.table)

setDT(df)[, unique(expand.grid(destino, destino)), by = origem][Var1 != Var2]
#>     origem Var1 Var2
#>  1:      A    A    B
#>  2:      A    C    B
#>  3:      A    B    A
#>  4:      A    C    A
#>  5:      A    B    C
#>  6:      A    A    C
#>  7:      E    B    C
#>  8:      E    C    B
#>  9:      D    B    A
#> 10:      D    A    B

expand.grid gera todas as combinações dos pontos de destino; data.table facilita fazer a operação por grupo (ponto de origem, no caso). unique remove as combinações duplicadas, geradas porque para uma mesma origem há repetição de destinos. Como não quer as combinações em que as letras são as mesma, essa condição é filtrada do resultado final.

Combinação de valores distintos


How to get count of columns whose value greater than 40 from table

Question: I need to get column count whose value greater than 40
example :
id col_1 col_2 col_3

1   20     60  80

The output like this count=2
query:
select count(columns) from table where id =1 and col_values >40;

How to write query to get this please help me to write query.
Comment: I have the feeling this is a poorly designed schema
Comment: `mySQL` or `Oracle` please?
Comment: @MaheswaranRavisankar oracle
Answer: Premise:
Your database is not normlized! Please, normalize it.
Solution:
Try this:
SELECT SUM(
    CASE
        WHEN col1 >= 40 THEN 1 ELSE 0
    END +
    CASE
        WHEN col2 >= 40 THEN 1 ELSE 0
    END +
    CASE
        WHEN col3 >= 40 THEN 1 ELSE 0
    END)
FROM yourtable

With this query you summarize all columns and you'll get only sum trasversal by rows.
So, if you have this result set
id col_1 col_2 col_3

1   20     60  80
2   44     22  20

You'll get 3 (two from row with id 1 and 1 for row with id 2)
EDIT
If you want get this information only for one row you can integrate you query with id = value (as in your question) so the query becomes:
SELECT SUM(
    CASE
        WHEN col1 >= 40 THEN 1 ELSE 0
    END +
    CASE
        WHEN col2 >= 40 THEN 1 ELSE 0
    END +
    CASE
        WHEN col3 >= 40 THEN 1 ELSE 0
    END)
FROM yourtable
WHERE id = 1

Comment: I suppose the tile must be re-write, because in the body of question our friend wants a total number (he has accepted my answer)
Comment: The question title is "count OF COLUMNS" - what you provided is a count of individual values. See the request for clarification I put in a Comment to the OP's question.
Comment: Actually I looked at the question again and the OP has a WHERE condition. If id is primary key, then he is, in fact, looking at only one row at a time. You may want to edit your answer to reflect that; he wasn't looking to count all values across the entire table, but only across one row.


Possibility of an Eulerian Path or Cycle

Question:
Suppose that a connected graph $G$ has $11$ vertices and $53$ edges. Show that G is not Eulerian.

I can prove it for a simple graph by saying that the sum of degrees of all vertices can be maximum $100$ (max degree $10$ for each vertex) which corresponds to $50$ edges. But the given graph has $53$ edges.
How do I prove the same for a multigraph?
Answer: Two things: First, your argument for simple graphs is off. A simple graph on $11$ vertices can have up to $55$ edges (in general, a simple graph on $n$ vertices can have up to ${n\choose 2}$ edges).
Secondly, there do exist Eulerian multigraphs on 11 vertices with 53 edges: For example, take a cycle of length 11 (11 edges). Now between two consecutive vertices, place $42$ edges. Then each vertex has even degree (either $2$ or $44$) and so this graph is Eulerian.
Answer: If you take a complete graph on 11 nodes, it will have 55 edges, and every node will have degree 10. To construct a graph with 53 edges we must remove two edges. There are two cases to consider: either the two edges share a node or they don't.
Case 1: Call three of the nodes $A$, $B$, and $C$. Remove edges $AB$ and $BC$. Now $A$ and $C$ have degree 9, $B$ has degree 8 and all other nodes have degree 10. The graph remains connected, so there is an Eulerian path from $A$ to $C$ but there is no Eulerian cycle.
Case 2: Remove two disjoint edges $AB$ and $CD$ (where $D$ is a fourth node) Now there are four nodes of degree 9 so there is not even an Eulerian path.
In both cases there is no Eulerian cycle.


Inference on fixed effects in a mixed effects model

Question: I have correlated data and am using a logistic regression mixed effects model to estimate the individual level (conditional) effect for a predictor of interest. I know that for standard marginal models, inference on model parameters using the Wald test is consistent for the likelihood ratio and score tests. They are usually approximately the same. Because the Wald is easy to compute and available in R output, I use that 99% of the time.
However, with a mixed effects model, I was intrigued to see a huge difference between the Wald test for the fixed effects, as they're reported in the model output in R, and a "by hand" likelihood ratio test--which involves actually fitting the reduced model. Intuitively, I can see why this might make a huge difference, because in the reduced model, the variance of the random effect is re-estimated and can substantially affect the likelihood.
Can someone explain

How are the Wald test statistics computed in R for fixed effects?
What is the information matrix for the estimated model parameters in a mixed effects model? (and is the same mx from which the Wald test statistics are calculated?)
What are the differences in interpretation between the results from the two tests in the cases I described? which ones are generally motivated and used in the literature for inference?
Comment: wonder if [this](http://glmm.wikidot.com/faq) partly answers your question.
Answer: The traditional Wald statistic to test the hypothesis H0 Lt = l for given L, r x p, and l, r x 1, is given by W = (Lt - l)' [L (X'H-1 X)-1 L' ]-1 (Lt - l) and asymptotically, this statistic has a chi-square distribution on r degrees of freedom. These are marginal tests, so that there is an adjustment for all other terms in the fixed part of the model. R is open source

Do you have the source?
What is your model exactly? Mixed effects is a fairly broad category, in as far as it resonates into the Fisher information matrix.
You mean a likelihood ratio and a score test?


Menu doesn't link to alias URL

Question: I'm having difficulty on a fresh Drupal installation getting the menu to use an alias (let's say "foo") rather than "node/#" link. The alias works when I type it straight into my browser. When creating the menu, I specified the alias (foo) but it automatically applies the system path to it.
The message that comes up is:
<code>The menu system stores system paths only, but will use the URL alias for display. home has been stored as node/1
</code>
This doesn't come up as an error, rather just an information dialogue.
For this installation I did a minimal install as I wanted it to be a fairly simple website. I've never done this before so there were a few modules I had to enable to get it working how I wanted. I'm thinking maybe there's a module or menu setting that isn't enabled by default using the minimal install.
I'm not using any language packs, and none of the solutions on the following questions were successful (most revolve around languages.)

URL Alias does not work in Menu-Blocks with i18n (pathauto also installed)
How do I use URL aliases in menus?
Comment: This is normal behavior, does the menu link redirect you to foo or node/# when navigating ?
Comment: hmmm thats weird i guess you need pathauto module maybe?
Comment: I continued to work on the site figuring I'll fix it later. Turns out I think it's the way I implemented this menu caused it to use the non-alias link. All my other menus seem to work fine. I'll update this post later with how I did it and what to do instead in case anyone else comes across this problem (it's fairly unique though.) Thanks for your suggestions!
Comment: Hi Pan. If hover over the menu link it will show "node/#" etc. If I click on the link it will display "node/#" in the URL bar. However I can manually type the URL that uses the alias into my browser and it shows the same page, without redirecting (the URL in the browser still shows the alias.)
Comment: Interestingly, it started working properly on my "main menu" but a custom menu I have created still links to the system path not the alias.
Comment: +1 for pathauto - I don't know how I lived without it! The functionality it provides should almost be a core module I reckon.
Answer: You can use Pathauto and Global Redirect Module.
Answer: I guess you need pathauto module.
Answer: You could use the Pathauto module for this.

The Pathauto module automatically generates URL/path aliases for various kinds of content (nodes, taxonomy terms, users) without requiring the user to manually specify the path alias. This allows you to have URL aliases like /category/my-node-title instead of /node/123. The aliases are based upon a "pattern" system that uses tokens which the administrator can change.

For more detail about this module you could read the READMe.txt of this module.

Menu doesn't link to alias URL


Does it make sense to make a conditional Markovianity assumption?

Question: Please do excuse if the title is somewhat vague. As we know, a discrete-time Markov chain is a sequence of random variables $S_1,S_2,\cdots$ with the Markov property - i.e.
\begin{equation}
P[S_{t+1}=s_{t+1}\mid S_{t}=s_{t},\cdots,S_{1}=s_{1}]=P[S_{t+1}=s_{t+1}\mid S_{t}=s_{t}]
\end{equation}
Could the Markov property be extended by considering the same process conditional on, say, another variable $X$? - i.e.
\begin{equation}
P[S_{t+1}=s_{t+1}\mid S_{t}=s_{t},\cdots,S_{1}=s_{1},X]=P[S_{t+1}=s_{t+1}\mid S_{t}=s_{t},X]
\end{equation}
Thanks in advance.
Answer: Yes, you can form such a model. For example, $X$ can be the vector of probabilities in the transition matrix.


SQLite and Caching Application Block

Question: Has anyone used the Caching Application Block with SQLite to persist data?
I'd like to get some details.
Thanks
Comment: what are you looking for when you say "some details"? Would you be more specific? Perhaps the question is too general for folks who could contribute to do so.
Answer: <code>objCatch.Add("DsFromDb", ds, CacheItemPriority.Normal, null, new Microsoft.Practices.EnterpriseLibrary.Caching.Expirations.AbsoluteTime(TimeSpan.FromMinutes(5)));
</code>

SQLite and Caching Application Block


Correlation coefficient with X>2

Question: Obtain the correlation coefficient for the random variables $X$ and $Y$ having joint density $f(x,y) = \frac{16y}{x^3}$ for $x \gt 2$ and $0 \lt y \lt 1$.
I know this is an easy question but I was having trouble setting up my integrals.
For $E(X)$ what should my bounds be? I don't understand how to set up the $X>2$. It's different from the examples I'm used to.
Comment: Welcome to Cross Validated.  If this is a homework question, please add the `self-study` tag and read its wiki [here](http://stats.stackexchange.com/tags/self-study/info), which includes etiquette on asking such questions, including showing a "good faith" attempt at solving the problem yourself and then showing where you are stuck.
Answer: Notice that the joint density can be separated into the respective marginal densities. We now have $f(x,y) = \frac{16y}{x^3} = 16y \cdot \frac{1}{x^3} = f(x) \cdot f(y)$. Thus, $X$ and $Y$ are independent and have covariance of zero. Now the correlation must be zero.
Comment: This answer is correct, but its reasoning overlooks an important subtlety.  In fact, a complete expression of the density is $$f(x,y) = \frac{16 y}{x^3}\mathcal{I}_{(2,\infty)\times(0,1)}(x,y).$$ You cannot neglect the constraints (as represented by the indicator function $\mathcal I$).  Since it also happens to be the case that $$\mathcal{I}_{(2,\infty)\times(0,1)}(x,y)=\mathcal{I}_{(2,\infty)}(x) \mathcal{I}_{(0,1)}(y),$$ the conclusion of independence actually is true. This point seems directly to address the question itself, which is "how to set up [the constraint] $X\gt 2$."
Comment: So the way I'm use to doing these questions is to solve for E(X), E(Y), etc... and then plug these values in to the equations. My understanding of correlation isn't the best but are you saying because the bounds of X and Y do not intersect there is no correlation so the correlation coefficient is just 0? we do not need to solve for E(X) at all?
Comment: You can solve for $E(X)$ and $E(Y)$ and get the same answer but it is much longer. Any time random variables are independent they automatically have no correlation because they have no covariance. Solving the way you are used to would entail finding $f(x) = \int_{2}^{\infty} \frac{16y}{x^3} dy$ and $f(y) = \int_{0}^{1} \frac{16y}{x^3} dx$. You can then use these to find $E(Y)$ and $E(X)$.

Correlation coefficient with X>2


Optimization through 'alternating' descent

Question: While analysing a particular iterative method which reaches the fixed points of a multivariate function f(x, y) (x and y are N-dimensional vectors in this case), I was able to reformulate it in the following terms

that is, as some kind of 'alternating' descent, in which x goes in the direction which minimizes y, and vice-versa.
Has anyone ever seen something like that, e.g. some algorithm which performs this kind of update?
Actually the 2nd equation has a + instead of a -, so one equation would be performing a descent while the other one performs a ascent... But I am interested in any kind of algorithm which does this kind of alternation.
Comment: The question seems more suitable for [stats.se].
Comment: Looks like generalized expectation maximization.
Answer: Take any multivariate function. Doing alternating coordinate descent is a very natural choice and it is being used in many other problems (though not necessarily in the perceptron updates)
Answer: This sort of technique us commonly called coordinate descent in machine learning, and can be used to fit a variety of models, IIRC it is guaranteed to converge to the optimal solution provided the cost function is convex.  It has been used with SVMs, linear regression, logistic regression, with L2 or L1 penalty functions etc.
A related problem is optimising the hyper-parameters of a model (kernel parameters or regularisation parameters etc.) where the process can be viewed as alternate optimisation of model and hyper-parameters, see Gradient-Based Optimization of Hyperparameters by Yoshua Bengio