What's the 'right' number of parameters for an ARIMA model?

Question: I'm working on an unassessed course problem,

The file Pas-mile.txt contains the monthly numbers of passenger miles travelled on US airlines for each month between January 1960 and December 1977. Find an ARIMA model for the series, carrying out appropriate diagnostic checks.

I've put the data at the bottom of this post. Here's a plot of the time series.

I differenced the data seasonally and non-seasonally. I think I can pose my question using only one of these, say the seasonal one.
y <- ts(data)
x <- diff(y,lag=12,differences=1)

I visually inspected the acfs and pacfs as suggested by my course notes,
acf(x, lag.max=216)
acf(x, type='partial', lag.max=216)

and I also did a grid-search:
aic.df <- data.frame()
for (i in 1:12){
  for (j in 1:12){
    aic.df[i,j] <- AIC(arima(x, order=c(i-1,0,j-1)))
  }
}
aic.df

The lowest AIC (I think) is 324.5148, for $\text{ARIMA}(10,1,8)_{12}$. I tried searching a larger grid but stopped because the computation was taking too long. I also got quite a lot of messages like
Warning: possible convergence problem: optim gave code = 1 and Warning: NaNs produced.
This seems like quite a lot of parameters, and the non-seasonal differencing adds more. Writing the model out as $y_t=\dots$ would be quite cumbersome. If I had more computing power, perhaps I could have found a still better model with still more parameters. As I understand it, there's no risk of overfitting since the AIC takes parameter number into account. So what's the 'right' number of parameters?
Data
 2.42  2.14  2.28  2.50  2.44  2.72  2.71  2.74  2.55  2.49  2.13  2.28
 2.35  1.82  2.40  2.46  2.38  2.83  2.68  2.81  2.54  2.54  2.37  2.54
 2.62  2.34  2.68  2.75  2.66  2.96  2.66  2.93  2.70  2.65  2.46  2.59
 2.75  2.45  2.85  2.99  2.89  3.43  3.25  3.59  3.12  3.16  2.86  3.22
 3.24  2.95  3.32  3.29  3.32  3.91  3.80  4.02  3.53  3.61  3.22  3.67
 3.75  3.25  3.70  3.98  3.88  4.47  4.60  4.90  4.20  4.20  3.80  4.50
 4.40  4.00  4.70  5.10  4.90  5.70  3.90  4.20  5.10  5.00  4.70  5.50
 5.30  4.60  5.90  5.50  5.40  6.70  6.80  7.40  6.00  5.80  5.50  6.40
 6.20  5.70  6.40  6.70  6.30  7.80  7.60  8.60  6.60  6.50  6.00  7.60
 7.00  6.00  7.10  7.40  7.20  8.40  8.50  9.40  7.10  7.00  6.60  8.00
10.45  8.81 10.61  9.97 10.69 12.40 13.38 14.31 10.90  9.98  9.20 10.94
10.53  9.06 10.17 11.17 10.84 12.09 13.66 14.06 11.14 11.10 10.00 11.98
11.74 10.27 12.05 12.27 12.03 13.95 15.10 15.65 12.47 12.29 11.52 13.08
12.50 11.05 12.94 13.24 13.16 14.95 16.00 16.98 13.15 12.88 11.99 13.13
12.99 11.69 13.78 13.70 13.57 15.12 15.55 16.73 12.68 12.65 11.18 13.27
12.64 11.01 13.30 12.19 12.91 14.90 16.10 17.30 12.90 13.36 12.26 13.93
13.94 12.75 14.19 14.67 14.66 16.21 17.72 18.15 14.19 14.33 12.99 15.19
15.09 12.94 15.46 15.39 15.34 17.02 18.85 19.49 15.61 16.16 14.84 17.04

Answer: AIC does guard against overfitting, but it will not completely prevent it. No metric can do that. So it is still important to sanity check one's model.
I find AR or MA orders of 8 or 10 on seasonally differenced data rather questionable. I could not really imagine a data generating process that really looked back 10 years, rather than just 5. There are good reasons why automatic ARIMA modeling procedures do not consider orders above 5.
Also, you are only modeling the seasonal part: your model is ARIMA(0,0,0)(10,1,8)[12]. The seasonal part is extremely complex per above, but the nonseasonal part is a flat line. That looks strange. You may want to follow the seasonal ARIMA modeling workflow illustrated here or here to see how one can set both seasonal and non-seasonal orders.
I personally am not a big fan of interpreting ACF/PACF plots and differencing back and forth to decide on ARIMA models. On the one hand, it's extremely hard, requires a lot of statistical understanding and is likely not replicable between analysts (if you have classmates, ask them what model they got - I would be surprised if half the class agreed). And you can't do it if you have more than one or a handful of time series. The newer order selection process based on information criteria makes more sense to me. (However, your teacher likely won't accept "I threw the time series into auto.arima(), and this is what came out" as a solution.)
Comment: Stephen Kolassa and @Richard Hardy: thanks both for your answers. I find them both useful and have accepted the one that came first chronologically to close the question.
Answer:
I differenced the data seasonally and non-seasonally.

and

This seems like quite a lot of parameters, and the non-seasonal differencing adds more

characterizes a typical mistake known as overdifferencing. You would normally difference your time series if has a unit root. Simple differencing is for a simple unit root, and a seasonal differencing is for a seasonal unit root. Eyeballing your series, I do not see any of the two.
But what happens if you difference a series that does not have a unit root? You introduce a unit-root moving-average component. With both simple and seasonal differencing, you may have introduced two of them. They produce certain autocorrelations that were not in the original series, and so you end up trying to fit these using high-order ARMA terms.
What I would do instead is think twice before differencing and see e.g. what auto.arima suggests in two cases: allowing for arbitrary orders of differencing and forcing the orders of differencing to zero.
Comment: @mjc, you could generate some realizations of a random walk and plot them. E.g. run the following script a few times in R: `plot(cumsum(rnorm(100)),type="l")`. Meanwhile, your series looks like a smooth deterministic trend (not far away from linear, perhaps a slowly moving nonlinear one) coupled with heteroskedastic seasonality.
Comment: Can you say anything about how you visually identify unit roots?


Determine the Winding Numbers of the Chinese Unicom Symbol

Question: I'm practicing with Winding Numbers, and encountered an interesting example. You might be familiar with this liantong symbol, the logo of China Unicom:

Suppose we make this into a fully closed and connected curve, and try to determine the Winding Numbers of the various points in the symbol. For instance:
Find the winding numbers of the closed curve shown below at $z_1,z_2,z_3,z_4,z_5$

It seems to me that for each $z$, the winding number $W(z)$ is:

$W(z_1)=0$ (since it is outside the curve)
$W(z_2)=1$ (since it falls to the left of the curve in one loop)
$W(z_3)=-2$ (since it falls to the right of the curve in two loops)
$W(z_4)=0$ (since it falls to the right and to the left of the curve twice each, cancelling out)
$W(z_5)=-1$ (since it falls to the right of the curve in one loop)

Would you agree with these winding numbers (and given reasoning)? Thank you for your help!
Comment: Crossing the curve (at a point where the curve doesn't intersect itself) changes the winding number by $\pm 1$.
So the winding numbers of $z_1$ and $z_3$ must differ by $1$, not $4$.
Comment: Okay, I agree with you that $W(z_3) \ne -4$. But when I trace the curve around $z_3$ I keep getting $W(z_3)=-2$ now. This one is tricky.
Comment: $W(z_3)$ is definitely $-1$. But if you want to discuss it, you're going to have to label your intersection points.
Answer: If a curve crosses your path from left to right, the winding number increases.
If a curve crosses your path from right to left, the winding number decreases.

Going from $z_1$ to $z_2$, the curve crosses our path from left to right. Therefore, $\mathrm{W}(z_2)=1$.
Going from $z_1$ to $z_3$, the curve crosses our path from right to left. Therefore, $\mathrm{W}(z_3)=-1$.
Going from $z_3$ to $z_4$, the curve crosses our path from left to right. Therefore, $\mathrm{W}(z_4)=0$.
Going from $z_4$ to $z_5$, the curve crosses our path from right to left. Therefore, $\mathrm{W}(z_5)=-1$.
Comment: Great explanation! I didn't know it was this easy. I thought I had to trace the curve and keep track of the direction of every boundary around a point to see how the Winding number changes... This is a much simpler approach. Thank you!
Comment: In essence, this method does just that. By noting the direction of the curve at each intersection with a path, we are keeping track of the direction at each boundary along the path.
Answer: Note that the curve can be deformed to a union of clockwise and counterclockwise circles, which makes it much easier to find the winding numbers.


How to visualize 2- and 3-space matrix transformations in Python

Question: I want to visualize matrix transformations in 2- and 3-space. I've tried finding an existing tool to do this to no avail [1] so now I'm looking for Python libraries so I can do it (almost) myself.
So I'd like to be able to enter a vector or matrix, see it in 2-space or 3-space, enter a transformation vector or matrix, and see the result. For example, enter a 3x3 matrix, see the parallelepiped it represents, enter a rotation matrix, see the rotated parallelepiped.
Bonus points for ability to calculate area/volume, animate, change colours, visualize higher dimensions, wash my socks.
I'm guessing I'm looking for two libraries, one for matrix math and one for drawing them in 2D and 3D.
(I'm also willing to look into scripts for existing FLOSS MATLAB clones, as long as said clone can be easily installed on Windows/Ubuntu.)
[1] See https://math.stackexchange.com/questions/34032/floss-tool-to-visualize-2-and-3-space-matrix-transformations and https://superuser.com/questions/275539/floss-tool-to-visualize-2-and-3-space-matrix-transformations if you know of one :)
Answer: I don't know about mayavi, but with matplotlib you can definitely plot (almost) whatever you like. With the full power of python language you can define your vectors, matrices etc, multiply them any way you wish, and plot the results.
Comment: matplotlib doesn't have much in the way of 3D support, which is why I suggested mayavi instead.
Comment: it did not have it. right now it well has Axes3D object which, I guess, is enough for the OP's goals. I personally never used mayavi, so I cannot offer any sensible judgement here. It's the OP's call, anyway :-).
Comment: thanks to both of you, I stumbled on easyviz/scitools while following up on your answers, and I'm giving that a try
Answer: For reference, if you are looking to do something like plotTransforms in Matlab Example transformation matrix plot in Matlab
You could use pytransform3d Example transformation matrix plot in Python


Array cannot be resolved for sports_array

Question: Initially my array looks like this:
<string-array name="sports_array">
    <item>Shuttle Badminton</item>
    <item>Tennis</item>
    <item>Table Tennis</item>
    <item>Basket Ball</item>
    <item>Foot Ball</item>
    <item>Volley Ball</item>
    <item>Hockey</item>
    <item>Swimming</item>
</string-array>

It was located in string.xml but when I tried to use this code to retrieve it, it does not display the array. The error that I get is Array cannot be resolved or is not a field.
String[] sports = getResources().getStringArray(R.array.sports_array);

I cleaned my project twice and the error is still there but what I didn't realize what the array that I placed in the string.xml disappeared. So I went on reading a few questions here and one of the options is to create an array.xml and place my arrays there instead of in string.xml. I still get the same error. What am I doing wrong?
Comment: `R` java class is missing the `array` class since the resource was not loaded properly. Make sure you don't have hidden XML errors in your resource files.
Comment: Just checking did you accidentally moved the "values" folder outside of the "res" folder like this :  http://stackoverflow.com/questions/9252098/android-array-cannot-be-resolved-or-is-not-a-field  ?
Comment: does your class extend activity?
Comment: I solved it. I just put it back in string.xml and it works again.
Answer: You can try this solution.
create fetcharr.xml in res/values/fetcharr.xml
  <?xml version="1.0" encoding="utf-8"?>
    <resources>
        <string-array name="sports_array">
            <item>Tennis</item>
            <item>Table Tennis</item>
            <item>Basket Ball>
        </string-array>
    </resources>

now  in activity class
Resources res = getResources();
String[] planets = res.getStringArray(R.array.sports_array);

Answer: Be sure that you have defined this in strings.xml:
<string name="action_settings">Action Settings</string>


Special non-utf-8 characters appearing in feed imported from a windows based system?

Question: I have a custom feed importer that import content from an external feed. The import happens on cron and everything is working fine except that the content comes with special non-utf-8 characters. I looked on stackoverflow and there are a few suggestions about how to do this in code, I tried couple and it has not worked for me yet. I am hoping to see if there is a drupal way of handling it.
The character that I am having problem with is Â. It represent a space I believe and it only appear at end/and beginning of sentences.
Answer: You can use the Feeds Tamper module to do this with a custom plugin.
In your module:
/**
 * Implements hook_ctools_plugin_directory().
 */
function MYMODULE_ctools_plugin_directory($owner, $plugin_type){
  if ($owner == 'feeds_tamper' && $plugin_type == 'plugins') {
    return 'plugins';
  }
}

and in your module/plugins/utf8_encode.inc
$plugin = array(
  'form' => 'MYMODULE_utf8_encode_form',
  'callback' => 'MYMODULE_utf8_encode_callback',
  'name' => 'Convert ISO-8859-1 to UTF-8',
  'multi' => 'loop',
  'category' => 'Text',
);

function MYMODULE_utf8_encode_form($importer, $element_key, $settings) {
  $form = array();
  $form['html'] = array(
    '#markup' => t('This will convert all ISO-8859-1 characters to their UTF-8 equivalents.'
),
  );
  return $form;
}

function MYMODULE_utf8_encode_callback($result, $item_key, $element_key, &$field, $settings, $source) {
  $field = utf8_encode($field);
}

If that doesn't work, then you can use one of the strip-non-UTF8 regex functions to do it in the callback instead.  I use
$regex = '/((?:[\x00-\x7F]|[\xC0-\xDF][\x80-\xBF]|[\xE0-\xEF][\x80-\xBF]{2}[\xF0-\xF7][\x80-\xBF]{3}){1,100})|./x';

preg_replace($regex, '$1', $field);

Comment: @awm FWIW, I made this https://drupal.org/node/2263119
Comment: I thought I submitted both of these plugins to the Feeds Tamper issue queue, as UTF8 issues are rather common when getting data from Windows systems.  Apparently, I forgot, but I will do so.
Comment: Thank you. It is already a custom importer using feeds hooks (Classes/mehtods) I guess I will give your solution a try in one of the hooks/methods implemented and will report back. Also, I am going to try `drupal_convert_to_utf8`
Comment: @awm You learn something new every day.  I didn't realize that function existed.
Comment: the regex (at the bottom of your answer) worked better than anything else. I had already tried utf8_encode with success to get remove other non-utf8 chars but not the `Â`. Same result with `drupal_convert_to_utf8`. I will probably use a combination to be secure.
Answer: You can replace the special characters with empty string in
        hook_feeds_after_parse like so:
/**
 * Implements hook_feeds_after_parse().
 */
function mymodule_feeds_after_parse(FeedsSource $source, FeedsParserResult $result) {
  //Replace special characters with '' to prevent PDO exception
  foreach($result->items as $item_key => &$item) {
    foreach ($item as $sub_item_key => $sub_item) {
      $item[$sub_item_key] = preg_replace('/[\x00-\x08\x10\x0B\x0C\x0E-\x19\x7F]'
      . '|[\x00-\x7F][\x80-\xBF]+'
      . '|([\xC0\xC1]|[\xF0-\xFF])[\x80-\xBF]*'
      . '|[\xC2-\xDF]((?![\x80-\xBF])|[\x80-\xBF]{2,})'
      . '|[\xE0-\xEF](([\x80-\xBF](?![\x80-\xBF]))'
      . '|(?![\x80-\xBF]{2})|[\x80-\xBF]{3,})/S',
      '', $item[$sub_item_key]);
    }
  }


In Android how do I get the currently displayed notification for a given notification id?

Question: My intention is to display a notification with a personalised icon. However, if a notification with the same id is already displayed and then I need to add a new notification with the same id, my program will instead prepare a "stacked" version of the notification where the icon is changed to some generic icon and the notification content shows excerpts from the last and current notifications. Similar to how Gmail does when there are multiple emails.
To implement that I need to check if there are notifications of my app, currently displayed. I do not see any API to retrieve my own notifications.
I cannot simply cache the notification details that I have displayed till now, since in that case I need to know when they will be dismissed by the user, and update my cache accordingly. I also do not see any API to listen for dismiss events.
Answer: This is possible with android 4.3 upwards now
See http://developer.android.com/reference/android/service/notification/NotificationListenerService.html#getActiveNotifications()
Comment: I need to support minimum SDK 10. Do we have this API in support library?
Answer: If you observe the gmail app notification behaviour properly u will notice that even when u dismiss one notification gmail shows you the same notification again in the list when a new notification comes. It looks to me that gmail is relying on total unread/unopened messages rather than keeping a cache of notifications.
There is no direct api for ur suggested SDK version. You need to fallback to ur own implementation. However, there is a way to know if a already showed notification was dismissed - How to know when my notification is cleared via Clear button?


Understanding the Proof of a Theorem 1.9 and Theorem 2.5 from Chartrand's Graphs and Digraphs

Question: There are two Theorems in Chartrand's Graphs and Digraphs that follow the same line of argument and arrive at an inequality that I can't seem to understand where it is deduced from. The inequality being the following: $(deg(u)-1) + (deg(v)-1) \leq n-2$ and $deg(u) + deg(v) \leq n-2$ in Theorem $1.9$ and Theorem $2.5$ respectively.
The two Theorems are shown below:

Both proofs follow the same line of logic. I understand the proofs, however, how can we be certain that the sum of the degrees of vertices is always less than $(n-2)$?
For example, the deletion of $1$ from $deg(u)$ and $deg(v)$, respectively, from the LHS of the inequality: $(deg(u)-1) + (deg(v)-1) \leq n-2$ comes from there being no vertex $w$ adjacent to both vertex $u$ and $v$ (since the new graph does not contain a triangle), is that correct? Furthermore, how is it bounded above by $(n-2)$?
Also, in Theorem $2.5$, is it (implicitly) following a proof by contracdition method? Since it isn't stated, however comes to a contradiction.
Comment: I believe that by subtracting 1 from deg u and deg v, we are accounting for the 1 edge we know to be adjacent to those vertices, namely the one connecting them. We are counting how many *other* vertices are adjacent to those 2 vertices, and observing that if this number is too large, then among the n-2 remaining vertices, there must be one (by the pigeonhole principle) adjacent to both u and v. Taking the contrapositive of this implication gives the desired inequality (I think).
Answer: Let $N_x$ denote the neighbourhood of vertex $x$ so that:
$$
N_x = \{y \in V(G) \mid xy \in E(G)\}
$$

For the inequality from Theorem 1.9, observe that $N_u \cap N_v = \varnothing$ (otherwise, if there was some $w \in N_u \cap N_v$, then $u \to w \to v \to u$ would be a triangle). Hence, observe that:
\begin{align*}
n
&=|V(G)| \\
&\geq |\{u\} \cup \{v\} \cup N_u \cup N_v| \\
&= |N_u \cup N_v| &\text{since $u \in N_v$ and $v \in N_u$} \\
&= |N_u| + |N_v| &\text{since $N_u \cap N_v = \varnothing$} \\
&= \deg u + \deg v\\
\end{align*}

Likewise, for the inequality from Theorem 2.5, observe that $N_u \cap N_v = \varnothing$ (otherwise, if there was some $w \in N_u \cap N_v$, then $u \to w \to v$ would be a $(u, v)$-path). Hence, observe that:
\begin{align*}
n
&=|V(G)| \\
&\geq |\{u\} \cup \{v\} \cup N_u \cup N_v| \\
&= |\{u\}| + |\{v\}| + |N_u| + |N_v| &\text{since all four sets are pairwise disjoint} \\
&= 2 + \deg u + \deg v\\
\end{align*}


Web Scraping Wikipedia Table Using Beautiful Soup Getting 'None' Returned

Question: New to web scraping and coding in general. This is probably an easy problem for someone more experienced... maybe not... here it is:
Trying to web scrape a table from wikipedia. I've located the table in the html and added that info in my code. However when I run it I get 'none' returned instead of confirmation the table has been correctly located.
from bs4 import BeautifulSoup
from urllib.request import urlopen

url = 'https://en.wikipedia.org/wiki/List_of_songs_recorded_by_the_Beatles'
html = urlopen(url)
soup = BeautifulSoup(html, 'html.parser')

table = soup.find('table',{'class':'wikitable sortable plainrowheaders jquery-tablesorter'})
print(table)

Return: None
Answer: Remove the jquery-tablesorter from the "class" string - this class is added by javascript and beautifulsoup doesn't see it (note: always observe the real HTML document the servers sends you, that's what is beautifulsoup seeing - press ctrl-U in your browser):
from urllib.request import urlopen

from bs4 import BeautifulSoup

url = "https://en.wikipedia.org/wiki/List_of_songs_recorded_by_the_Beatles"
html = urlopen(url)
soup = BeautifulSoup(html, "html.parser")

table = soup.find("table", {"class": "wikitable sortable plainrowheaders"})
print(table)

Prints:

Name of song, core catalogue release, songwriter, lead vocalist and year of original release

Song

Core catalogue release(s)

Songwriter(s)

...


letrec in scheme with double brackets?

Question: I am failing to understand the ribbon01 code, confused with the brackets. Can somebody explain it to me I am new to scheme
(define ribbon01
  (lambda (lat)
    ((letrec
         ((di (lambda (lat)
                (display lat))))
       di) (cdr lat))))

(define ribbon02
  (lambda (lat)
    (letrec (( di (lambda (lat)
                  (display lat))))
    (di (cdr lat)))))

The ribbon02 is very easy to understand as there are 2 statements in letrec, declaration of di and then call to di.
Answer: Notice the double brackets in (1) and the returned value in (2):
(define ribbon01
  (lambda (lat)
    ((letrec ; (1)
         ((di (lambda (lat)
                (display lat))))
       di)   ; (2)
     (cdr lat))))

What's happenning is that we're returning di in (2) and then we apply it in (1) passing as parameter (cdr lat) - remember, a pair of () applies the value passed as a function, and in this case that value is di. It's the same as ribbon02, except that the function application in ribbon01 occurs outside letrec.
Answer: The only reason to wrap an expression in extra parenthesis is if you want to apply it as a procedure.
(define ribbon01
  (lambda (lat)
    ((letrec ((di (lambda (lat)
                    (display lat))))
       di)
     (cdr lat))))

Now the only variable in the letrec that is made is di and it is a procedure of one argument. Then in the body of the letrec di is evaluated, thus the result of the whole letrec is a procedure. Since di isn't used inside the lambda we can simplify the whole letrec with the lambda form.
(define ribbon01
  (lambda (lat)
    ((lambda (lat) (display lat)) (cdr lat))))

Now the lambda there is just using the one argument and supply display thus. we can substitute that as well:
(define ribbon01
  (lambda (lat)
    (display (cdr lat))))

So if you imagine you evaluate (ribbon01 '(1 2 3 4)) you know the cdr would amount to (2 3 4) and that is what is displayed.
The ribbon02 doesn't evaluate the procedure back and then call it but apply it from within the letrec. It's perhaps a more common way to use letrec but the result is the same. Again you replace di with what it represents:
(define ribbon02
  (lambda (lat)
    ((lambda (lat) (display lat)) (cdr lat))))

And you see that you are at the same spot as with ribbon01 and can do the same reductions from there.
So if you have an expression e and put parentheses around it (e something) you know the expression e needs to evaluate to a procedure and that something is to become it's one argument.
It is nothing different than (+ 1 2) since + is also an expression that becomes a procedure and since it has extra parentheses around it it is being applied just as with your letrec. Nothing different at all.
Comment: Thanks for the answer. I actually simplified the function before posting it on this forum. Thank you very much again for your time

letrec in scheme with double brackets?


Trying to make my jquery arrow img change to different arrow on Hide/Show content

Question: I have this very simple JQuery
$(document).ready(function() {
  $(".neverseen img").click(function() {
    $(".neverseen p").slideToggle("slow");
    return false;
  });
});

And this HTML

Title to always show

Text to toggle hide/show

So what I need is to add another image called arrow2.png that will replace the first arrow when hidden content activates or make the current arrow turn 180 degrees. I have tried different jquery methods but nothing seems to work so well.
Another thing I wish is to have a border-bottom that is visible at all times and moves with the hidden content so the border is below the title on hidden and below the content on visible, whatever I am doing here results in double borders.
Comment: Add this in your `click` function: `$(this).css('transform','rotate(180deg)')`
Answer: You have different solutions to achieve that, here is one of them:

$(document).ready(function() {
  $(".neverseen img").click(function() {
   $(this).parent().toggleClass("active");
   $(".neverseen p").slideToggle("slow");
     return false;
    });
});
img {
  width: 50px;
}

#secondImage {
  display: none;
}

.active #firstImg {
  display:none;
}

.active #secondImage {
  display: inline-block;
}

Title to always show

  Text to toggle hide/show

I would suggest you to use bootstrap collapse, it is really amazing!
Comment: Tested and it worked great, you don't have any idea regarding getting the bottom-border working as i described in the post?
Comment: Have you ever used bootstrap collapse? For your case, it really would help you! Take a look: http://www.w3schools.com/Bootstrap/bootstrap_collapse.asp @FilipPetersson
Comment: @TeautaKoraqi No i have never used it but it looks very promising will definetely take a hard look at that! thanks for the tip.
Comment: Just download bootstrap.min.js and bootsrap.min.css, and follow the instructions. I have used many times, so you can use it without no worries.  If you need any help, just comment here, and I will help you! :) @FilipPetersson
Comment: @FilipPetersson, I made a fiddle for you using bootstrap collapse: https://jsfiddle.net/teutakoraqi/xhjkv1j3/20/
Answer: Fairly simple procedure using jQuery's native .css() function. No need to substitute with a completely different image
Try this instead:
$(".neverseen img").click(function() {
    $(this).css("transform", "rotate(180deg)"); // rotate the image 180 degrees once it has been clicked
    $(".neverseen p").slideToggle("slow");

    return false;
});

EDIT: to achieve a continuous result & the bottom border, within your jQuery try something like this:
$(".neverseen img").click(function() {
    $(this).toggleClass("open");
    $(".neverseen p").slideToggle("slow");

    if($(this).hasClass("open")){
        $("h1").css("border-bottom", "none");
        $(this).css("border-bottom", "1px solid black");
    } else {
        $("h1").css("border-bottom", "1px solid black");
        $(this).css("border-bottom", "none");
    }

    return false;
});

and then, inside of your CSS:
h1{
    border-bottom: 1px solid black;
}
.neverseen img{
    transform: rotate(0deg);
    transition: transform 500ms; // rotate for 500 milliseconds
}
.open{
    transform: rotate(180deg);
}

Comment: This does indeed work to rotate the arrow once but if i toggle it again it wont rotate, i want it to rotate everytime i toggle show/hide.
Comment: @FilipPetersson one moment! :)
Comment: @FilipPetersson try it now :-)
Comment: This also worked great thanks. I also tested the bottom border solution but i also thought of that but the thing is that when the logo rotates the img  border turns upside down so it gets weird. You have any other ideas there? :)
Comment: @FilipPetersson think i might have fixed your problem :) try it now
Comment: 2 current issues with that solution. 1: the bottom border on the h1 is always there and its placed directly under the title, i would like some space between the h1 and the bottom border.  2: the bottom border doesnt go away when i show the hidden content.


ValueCangeListener work on second try in JSF 2.0

Question: I have two selectOneMenu elements which their id's make difference only.
 valueChangeListener works on second try.
At first try, page is refreshing, but does not go the valueChangeCode, thefore no value changes. At second try it goes  valueChangeCode and it gets new value and both of selectOneMenu show new value.
            <h:selectOneMenu id="changeCurrency" value="#dataBean.showCurrency}"                valueChangeListener="#myBean.changePaymentCurreny}"
valueChangeListener="#{myBean.changePaymentCurreny}"
                immediate="true"
                onchange="submit()">
                <f:selectItems value="#{my.currencyList}" />
            </h:selectOneMenu>
           <h:selectOneMenu id="changeCurrency2"
                value="#{dateBean.showCurrency}"
                valueChangeListener="#{myBean.changePaymentCurreny}"
                immediate="true"
                onchange="submit()">
                <f:selectItems value="#{myBean.currencyList}" />
          </h:selectOneMenu>

In ValueChangeListener code.
dataBean.setShowCurrency(newCurrency);
FacesContext.getCurrentInstance().renderResponse();
FacesContext.getCurrentInstance().getViewRoot().getChildren().clear();

When i remove the line which ends with getChidren.clear, and i change value of one them, changeListener called and changes the value at first try.
On view, only one selectmenu which i changed, shows new value. Other selectOnemenu still show the old value.
dataBean and myBean are in sessionScope. When i myBean put on requestscope, ajax request which works on page load, does not cath the mybean cause of being null.
Could anyone make clear this station.
Thanks very much.
Answer: try change the valueChangeListener for an ajax event tag like this.
<h:selectOneMenu id="changeCurrency" value="#dataBean.showCurrency}" valueChangeListener="#myBean.changePaymentCurreny}" onchange="submit()">
   <f:selectItems value="#{my.currencyList}" />
   <f:ajax event="change" listener="#{myBean.changePaymentCurreny}" process="@this" partialSubmit="true"  />
</h:selectOneMenu>

<h:selectOneMenu id="changeCurrency2" value="#{dateBean.showCurrency}" onchange="submit()">
   <f:selectItems value="#{myBean.currencyList}" />
   <f:ajax event="change" listener="#{myBean.changePaymentCurreny}" process="@this" partialSubmit="true"  />
</h:selectOneMenu>


Merge two layers with different shape reproducing SiamRPN

Question: I'm trying to reproduce this paper SiameseRPN using Keras.
The architecture is presented as below.

its give this in my code.

Template_input_shape = (127, 127, 3)
Detection_input_shape = (255, 255, 3)

// make the input
input_Template = tf.keras.Input(shape=Template_input_shape)
input_Detection = tf.keras.Input(shape=Detection_input_shape)

// make with alexNet model
processed_Template = alexNet(input_Template) // (6, 6, 256) output
processed_Detection = alexNet(input_Detection) // (22, 22, 256) output

// alexNet represente my model.

// after (if i've well understood) outputs are dispatched over a classification model and regression model

def classification_Branch(input_a, input_b, anchors=5, out=256):
    cls_x = keras.layers.Conv2D(256, kernel_size=3)(input_a.output)
    cls_z = keras.layers.Conv2D(out, kernel_size=3)(input_b.output)

cls = classification_Branch(processed_Template, processed_Detection)
rgr = regression_Branch(processed_Template, processed_Detection)

I'm falling to understand how can I merge both cnn (cls_x with cls_z etc) to give me the desired output.
Keras.layers.concatenate([cls_x, cls_z])

doesnt work here because the shape are not the same.
someone can explain me how in my situation can I merge both cnn into one cnn with the desired output.
Thank you
Answer: Yes, Keras.layers.concatenate requires both the Convolutional Layers passed to it to have the same shape.
In this case, you can use UpSampling Layer.
For Example, suppose the Shape of cls_x is (None, 80, 128, 8) and the Shape of cls_z is (None, 40, 64, 8), you can use
p1 = keras.layers.UpSampling2D(size = (2, 2))(c2)

before the step,
Keras.layers.concatenate([cls_x, cls_z]).
This will multiply the shape of the cls_z by 2 and as a result, Shape of cls_x and cls_z will become equal.
Note: We can pass decimal Sizes like (1.5, 1.5) as well, to UpSampling2D, based on our requirement.


How can I open MVC 3 project in Visual studio 2010 SP1 ?

Question: I just reinstalled my OS and installed everything again and found that whenever I open my project I got the error that.

The project type is not supported by this installation.

Can anyone explain why this happened whenever nothing goes wrong whenever I install.
The issue I found that is error code 1603 and I found later a alternate to install MVC3 you can read the whole post:
1603 installation Error in MVC 3.
Comment: Try creating a new MVC 3 app and inspect the ProjectTypeGuids in the `.proj` file of the app that is failing to open and compare them with those in the new MVC 3 app.
Answer: You need to go to the asp.net homepage, download and install the MVC3 package.
http://www.asp.net/mvc/mvc3
Answer: There are a few quirks with the web platform installer. Download the full mvc version by clicking on the download for "AspNetMvc3Setup.exe" on this page: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d2928bc1-f48c-4e95-a064-2a455a22c8f6
Then run the executable, which will get around having to use the web platform installer.
Comment: the provlme is already solved. i hope you read the post i attached in question. i not accept because it's matter that i not accept myself. great link but problem is solved.
Comment: It wasn't working after using the Web Platform Installer but this fixed it for me. Thanks.


Pre-requisite to analyze classical music

Question: First, I'm sorry for my poor English.
I enjoy listening to classical music.
According to Schiff's analysis of Beethoven's Sonata, It seems that classical music has a certain form, so if any note changes, there is a contradiction in the musical structure within a few bars.
I want to not only appreciate the works but also understand why they should be compose like that.
While majoring in physics mathematics in a non-English speaking country, I got a lot of help from Stack Exchange in book recommendation and detailed explanation about systematic studies.
Can you recommend books or open courseware for detailed analysis of classical music works?
Comment: Do you play an instrument?  Can you read music?
Comment: When I was young, I played Czeny, so I know how to read sheet music, but I don't know basic theories like theory of harmony
Answer: All music,and the 'classical' style in particular, is based on elements of repetition and of variation.  It's not really true than changing one note will bring the whole edifice tumbling down! But I sort of see what you're getting at.  If writing a fugue, for instance, where each voice states the theme in turn, it would be odd to capriciously vary one of the entries.
As you mention Beethoven, perhaps you would enjoy 'A companion to Beethoven's Pianoforte Sonatas' by Donald Tovey.  There seem to be plenty of second-hand copies on Amazon and eBay.
Comment: Thanks, I checked the book, but I can't understand the bar by bar analysis because I don't have prior knowledge on music theory. So I want to learn basic music theory systematically and read the book you recommended. Can you recommend a basic music theory book and curriculum?


Preferred way in Angular to trigger one API call (HTTP GET ) only when another API call has finished

Question: In Angular, if I have one part of the screen populated by an HTTP GET request and another part of the screen populated by a completely different GET request, but the second request should only get called after the first completely finishes, what is the preferred way (i.e. best practice) to accomplish this?
Comment: What have you tried, and what exactly is the problem with it?
Comment: @jonrsharpe I guess the issue is that there were a variety of ways to achieve this, but since I'm rather new to Angular I wanted to know if there was a "best practice" way. Seems like every post I find offers a slightly different take.
Answer: I would suggest creating a dependency on the first request while using switchMap to mutate the inner observable of the second request.
public firstResponse$ = this.http.get('api/endpoint').pipe(share());

public secondResponse$ = this.firstResponse$.pipe(switchMap(() => this.http.get('api/other-endpoint')));

This creates a stream that will ensure that the second request isn't triggered until the first request returns with a response.
Note the share() operator which will avoid making multiple requests when subscribed to multiple times.
Comment: as a bonus, switchMap also cancels the first stream when that part completes, which prevents inner subscription memory leaks. also, don't forget to subscribe to secondResponse$ in order to start the stream.
Answer: In your ngOnInIt:
ngOnInIt{
this.firstResponse() //assuming the name of the first method is firstResponse.
}

In your ts:
firstResponse(){
//Inside your subscription
this.secondResponse() //assuming the name of the second method is secondResponse.
}


Move a element in the view using css

Question: I need to move an element when hovering over it in the view using only css for an exercise. I found some options but they didn't work.

thanks in advance
EDIT: thanks for the answers.
I solved partially using the solution provided by Jaclyn Ciringione, with the :hover selector and using the solution provided in this post:
CSS3 animation translate3d not working
so I made a css like this:
(I had to move the class to the div to make it work as intended)

.hoverOver{
-webkit-transition:all 0.3s linear;
-moz-transition:all 0.3s linear;
-o-transition:all 0.3s linear;
-ms-transition:all 0.3s linear;
transition:all 0.3s linear;}

.hoverOver:hover{
-webkit-transform: translate(230px);
-moz-transform: translate(230px);
-o-transform: translate(230px);
-ms-transform: translate(230px);
transform: translate(230px);}

 I WANT TO MOVE

Comment: Hello and welcome to StackOverflow. We will need more informations about what you need. And please provide your attemps, SO is not a programming service, especially if this is an exercice, we are not here to make it in your place.
Comment: What movement do you expect? Move where? Can you explain what do you want to achieve?
Comment: you're moving the element you're hovering? But if it moves too much you'll lose the hover
Comment: For starters you could search for CSS3 transitions and animations.
Comment: Welcome to Stackoverflow. In addition to @Seblor'comment please read [how to ask questions](https://stackoverflow.com/help/how-to-ask) and [how to create a minimal example](https://stackoverflow.com/help/mcve) to improve your post
Comment: thanks edited with the updates
Answer: you can make it do anything you like by adding a hover class :hover
span.hoverOver:hover {
position: relative;
left: 20%;
}

Comment: The selector `div.hoverOver` will not match anything. The `span` has the class, not the `div`. Also you should wait for the OP to edit their question, because you cannot know if they want a solution like yours, or an animation or something else.
Comment: That is why in the explanation above the code snippet I said that you need to put the class on the div, not the span.... then I gave an example of how it would work when you do so.
Comment: Keeping the class on the span, without the `display` rule works well. Thanks for providing the 2 options though.

Move a element in the view using css


Sharp Architecture ignoring my Formula mappings

Question: I'm having a problem where Sharp Architecture will correctly map everything I have setup in my IAutoMappingOverride classes, except for Formula. These get simply ignored and thus I get SQL's invalid identifier when trying to query the database.
// NUnit setup
public virtual void SetUp()
{
    configuration = NHibernateSession.Init(
        new SimpleSessionStorage(),
        RepositoryTestsHelper.GetMappingAssemblies(),
        new AutoPersistenceModelGenerator().Generate(),
        null,
        null,
        null,
        FluentConfigurer.TestConfigurer.Contracts);

    new FluentConfigurer(configuration)
        .ConfigureNHibernateValidator()
        .ConfigureAuditListeners();
}

public AutoPersistenceModel Generate()
{
    return AutoMap.AssemblyOf<Contrato>(new AutomappingConfiguration())
        .Conventions.Setup(GetConventions())
        .IgnoreBase<Entity>()
        .IgnoreBase(typeof(EntityWithTypedId<>))
        .UseOverridesFromAssemblyOf<EmployeeMap>();
}

// My override.
public class EmployeeMap : IAutoMappingOverride<Employee>
{
    public void Override(AutoMapping<Employee> mapping)
    {
        // This works...
        mapping.Id(x => x.Id, "ID_EMPLOYEE");

        // This is ignored...
        mapping.Map(x => x.Name).Formula("UPPER(LTRIM(RTRIM(FIRST_NAME || ' ' || LAST_NAME)))");
    }
}

Any ideas?
Answer: This is not an issue with Sharp Architecture, it is an issue with Fluent Nhibernate. What version of FNH are you using?
Answer: I confirmed this is an issue with Fluent NHibernate (126.96.36.1994). Previously, a column-name mapping would give precedence to the FluentMappingOverrides, but the
latest would give precedence to the Convention. I modified my
convention to exclude the namespaces that contain the formula mappings
and now it's Ok.
    public class OracleUnderscoredNamingConvention : IPropertyConvention
    {
        public void Apply(IPropertyInstance instance)
        {
            // Previously worked without this condition.
            if
(Utils.WorkableDomainNamespaces.Contains(instance.Property.PropertyType.Nam espace))
            {
instance.Column(OracleConventionSetter.ApplyOracleNamingConventions(instanc e.Property.Name));
            }
        }
    }


Use variable from template.php in block

Question: I have a custom table in the Drupal DB, which I need to retrive and use on a few places on the site I'm creating. And I don't like to run soo much PHP in blocks, so I thought that I could get the table value I need, using PHP in template.php in the theme-dir and then just call the variable from a block, using PHP.
That didn't work.. Why and how should it be done?
template.php:
$sub_count = db_query("SELECT count FROM {sub_count}")->fetchField();

block:
<?php $sub_count; ?>

Answer: You need to use a preprocess function.  Take a look at template_preprocess_block().  In your template.php you can use:
function YOURTHEME_preprocess_block(&$variables) {
  $variables['sub_count'] = db_query("SELECT count FROM {sub_count}")->fetchField();
}

and can print into the .tpl.php file as you did above.
A few good articles to learn more about preprocess functions are

About overriding themable output
Setting up variables for use in a template (preprocess and process functions)

As a basic overview of preprocess functions, from the second article:

Preprocess functions only apply to theming hooks implemented as
  templates. The main role of the preprocessor is to set up variables to
  be placed within the template (.tpl.php) files. Plain theme functions
  do not interact with preprocessors.
Notes:

Preprocessors are also used for providing template suggestions.
In versions 5 and below, the function _phptemplate_variables served the
  same purpose. It has been deprecated in 6.
Prior to Drupal 6.7, for your theme to have its preprocessors recognized, the template
  associated with the hook had to exist inside the theme. When a default
  template exists, copy it to your theme and clear the registry (or you
  should really be upgrading to a later version of Drupal anyway for
  security reasons, at which point you don't have to worry about this).
Comment: Thank you, but this means that I can't use a variable inside a block?

Use variable from template.php in block


Return function after for loop and nested anonymous functions finish executing

Question: I am trying to execute some code AFTER I create a new array inside a function that takes an array of image files as input. This function loops over the files and has two nested anonymous functions inside of it. The main goal is to compress files uploaded by users before saving them to the database.
The compression part works great. The issue is that when I call the compressFiles function the function returns too early so the new result array is empty. If I add a setTimeout of about a second the array holds the newly compressed files just fine.
I know this is probably an async issue but I've read so many questions here on stackoverflow and I still can't wrap my head around it. So what would be the best way to return the compressFiles function AFTER the for loop and the anonymous function inside of it are done? Or am I just going about this the wrong way? I have tried nesting promises but have failed to arrive at a solution.
Any help or a push in the right direction would be greatly appreciated.
Calling and logging the compressFiles function:
handleFiles(files) {
  const compressedFiles = this.compressFiles(files)
  console.log(compressedFiles)
} // End handleFiles

Here is the compressFiles function:
compressFiles(files) {
// this function compresses image files
let result = []
  console.log('starting loop')
  for (var i = 0; i < files.length ; i++) {
      console.log('file' + i)
      let file = files[i]
      const reader = new FileReader()
      reader.readAsDataURL(file)
      reader.onload = (e) => {
        const imgElement = document.createElement('img')
        imgElement.src = e.target.result

        imgElement.onload = (e) => {
          const canvas = document.createElement('canvas')
          const MAX_WIDTH = 150

          const scale = MAX_WIDTH / e.target.width
          canvas.width = MAX_WIDTH
          canvas.height = e.target.height * scale

          const ctx = canvas.getContext('2d')
          ctx.drawImage(e.target, 0, 0, canvas.width, canvas.height)

          const srcEncoded = ctx.canvas.toDataURL(e.target, 'image/jpeg')
          ctx.canvas.toBlob( (blob) => {
            //A Blob() is almost a File() - it's just missing the two properties below
            var compressedFile = new File([blob], file.name);
            result.push(compressedFile)
            this.compressedFiles.push(compressedFile)
            console.log(i)
          }, e.target, 'image/jpeg')
        }
      }
  } // end for
return result
} // End compressFiles

Answer: I would simply use a counter and a callback function:
compressFiles(files, callback) {
   // this function compresses image files

   //define total files and counter
   const countTotal = files.length;
   let countLoaded = 0;

   let result = []

   console.log('starting loop')

   for (var i = 0; i < files.length ; i++) {
      console.log('file' + i)
      let file = files[i]
      const reader = new FileReader()
      reader.readAsDataURL(file)

      reader.onload = (e) => {
         const imgElement = document.createElement('img')
         imgElement.src = e.target.result

         imgElement.onload = (e) => {
            const canvas = document.createElement('canvas')
            const MAX_WIDTH = 150

            const scale = MAX_WIDTH / e.target.width
            canvas.width = MAX_WIDTH
            canvas.height = e.target.height * scale

            const ctx = canvas.getContext('2d')
            ctx.drawImage(e.target, 0, 0, canvas.width, canvas.height)

            const srcEncoded = ctx.canvas.toDataURL(e.target, 'image/jpeg')
            ctx.canvas.toBlob( (blob) => {
               //A Blob() is almost a File() - it's just missing the two properties below
               var compressedFile = new File([blob], file.name);
               result.push(compressedFile)
               this.compressedFiles.push(compressedFile)
               console.log(i)

               countLoaded++;

               //each time an image finishes loading, check if all images have now been loaded
               if(countLoaded === countTotal) {
                  //call a function with the result
                  callback(result);
               };
            }, e.target, 'image/jpeg')
         }
      }
   } // end for
} // End compressFiles

The above increments countLoaded at the end of the imgElement.onload function. If the number of images loaded is equal to the number of elements in the files array, then you are done and want to return the result. This is achieved using a callback function which returns the results array, because as you say, this is an async situation.
Comment: Fixed it! I've moved the `countLoaded++` and if statement above the `}, e.target, 'image/jpeg')` line so that they don't get executed until the end of that nested function. That way the `result` array is filled when calling the callback (`sendFiles`). Thanks again for the help @sbgib.
Comment: Thanks for answering. I added your suggestions, however the code is still not quite executing in the right order. I've added a `console.log(countLoaded)` right after `result.push(compressedFile)` and a `console.log(result)` inside the if statement that checks if `countLoaded === countTotal`. This is what the console shows:
`[]
4
4
4
4`
Comment: Logging the parameter in the callback also shows an empty array. Again, the code does append the new files to the result array but AFTER the callback is executed.
Comment: You're welcome! Well done! That makes sense - it seems the `ctx.canvas.toBlob()` function was also asynchronous, so the counter and callback needed to be inside there instead. I've updated the answer to reflect this. :)


A question on continuous random variable

Question: Let say, I have 2 continuous random variables X1 & X2. Both have same location parameters. Other parameters may be same or may not.
Now say, the q1-th quantile of X1 is less than the q1-th quantile of x2. But the q2-th quantile of x1 is more than the q2th quantile of x2.
My question is, is that possible? Is there any example of x1 & x2 which have that property?
I will be really grateful if someone can give me some pointer.

Edit: At this point, I realize the question I asked for was not correctly specified.
I'm particularly interested in the case where the two quantiles being considered are on the same side of the location parameter.
Comment: Two normal distributions centered around 0, one with larger variance (compare 0.1 and 0.9 quantile)?
Comment: @Michael Mayer is correct.  It might help to draw a picture of the two CDFs overlaid on one another.
Comment: I am not sure if this can happen. Quantiles for Normal distributions are basically the scaled up version of the corresponding SD. Therefore if for the 1st Normal, 0.1 quantile is higher then 0.9 quantile will also be higher. What I am looking for is opposite. I am looking for this scenario: 0.1 quantile is higher for 1st r.v. than 2nd r.v. but, 0.9 quantile is lower for 1st than 2nd.
Answer: Since you appear to doubt the example offered, I have included a diagram. As Michael Mayer said, two normal distributions centered around 0, one with larger variance, is sufficient.
In the diagram, we compare the 0.1 and the 0.9 quantiles for $\sigma=1$ (blue) and $\sigma=0.8$ (dark orange)

Michael Mayer's example fulfills the requirements of your question with $q_1=0.1$, $q_2=0.9$ and $X_1$ being the one with larger variance.

Edit:
For the case where $q_1$ and $q_2$ must both be on the same side of whatever the measure of location is, let's take two symmetric distributions, which share the same mean and median.
Let $X_1$ be $\sim \text{N}(0,1^2)$ and let $X_2$ be an equal mixture of a $\text{N}(-0.8,0.1^2)$ and a  $\text{N}(0.8,0.1^2)$, and let $q_1 = 0.6$ and $q_2 = 0.9$:

This example fulfills the new requirements of your question with $q_1=0.6$, $q_2=0.9$ and $X_1$ being the one with only a single normal component (shown in blue above).
Further, you should note that 'location parameter' isn't sufficiently specified. I could parameterize normal distributions by their 5th percentile and their standard deviation, and call the parameter based on the 5th percentile the location parameter (it's just a shift of the mean by $1.645\sigma$. and can work as a perfectly valid location parameter). Then Michael's original example suffices even under the new conditions. If that contradicts your intention, your intention needs to be stated specifically enough to exclude it.
Comment: Ron, I have added your change to your question and responded to it. Please note that the way you worded it in your comment doesn't make sense (what's the "fifth quantile"? What's the "first quantile"?), but with that part left out, we can get somewhere.
Comment: Thanks everyone for your quick pointer. However at this point, I realize the question I asked for is not properly specified. I should mention a caveat that, the quantiles should be of the same side of Origin (assuming location parameters for both r.v.s are zero). For example, let say 5th and 1st quantile of 2 random variables. My question is that, given 2 real valued r.v.s, if 5th quantile of a r.v. is smaller than 5th quantile of 2nd r.v. then 1st quantiles also will be of similar order? Is it a necessary property? Or there exist random variables for which such rule may not follow?


For polynomials $f, g$ and $\gcd(f(X),g(X))=d(X)$ then $\gcd(f(X+a),g(X+a))=d(X+a)$

Question: Suppose for polynomials f, g in $\mathbb{Q}[X]$ it holds that
$$\gcd(f(X),g(X))=d(X)$$
What we also want to now prove is that for $a \in \mathbb{Q}$: $$\gcd(f(X+a),g(X+a))=d(X+a)$$
So the question is asking that if we replace $X$ by $X+a$, the new gcd is the old one evaluated at $X+a$. I was thinking that we can always write:
 $$ \alpha(X) f(X)+ \beta(X)  G(X) =d(X)$$
For polynomials $\alpha$, $\beta$, but I don't see how I could use this. If I would just treat $d(X)$ as a function now and shift by $a$ to the left, the proof would immediately follow, but this feels a bit like cheating, is this statement valid?
I would then finish like:
If we now treat $d(X)$ as a function, we shift the function $a$ to the left and get:
$$d(X+a)=\alpha(X+a) f(X+a)+ \beta(X+a) g(X+a) .$$
Since we still have $\alpha(X+a), \beta(X+a) \in  \mathbb{Q}[X]$ this statement is equivalent to:
$$\gcd(f(X+a),g(X+a))=d(X+a) .$$
as desired $\square$.
Comment: Why do you think a valid inference is "cheating"?
Comment: Because I didn't really know why I was allowed to do it ;)
Answer: I think one problem might be
$$p(x+a)f(x+a) + q(x+a)g(x+a) = d(x+a)$$
does not imply
$$d(x+a)=\gcd(f(x+a),g(x+a)),$$
only that their $\gcd$ divides $d(x+a)$.
For example you can multiply arbitrarily large degree $k(x)$ to $p(x+a), q(x+a)$ and $d(x+a)$.
Perhaps arguing along with degree of $d(x), d(x+a)$ can work. I guess this also means $a$ must be defined as not a root.
But if $a$ is not a root perhaps another way is: we can find $p(x),q(x)$ such that
$$
p(x)(f(x)/d(x)) + q(x)(g(x)/d(x)) = 1
$$
Therefore
$$
p(x+a)(f(x+a)/d(x+a)) + q(x+a)(g(x+a)/d(x+a)) = 1
$$
This time round we can say something concrete about their $\gcd$:
$$
D(x) = \gcd(f(x+a)/d(x+a),g(x+a)/d(x+a)) = 1,
$$
as otherwise there is some non-constant polynomial $D(x)$ that divides $1$.
Therefore
$$
\gcd(f(x+a),g(x+a)) = d(x+a)
$$
Comment: Why does $p(x)(f(x)/d(x)) + q(x)(g(x)/d(x)) = 1
$
imply that
$
p(x+a)(f(x+a)/d(x+a)) + q(x+a)(g(x+a)/d(x+a)) = 1$ ?
Comment: Since the relations holds for all X, it must also hold for $X+a$?
Comment: @WesleyGroupshaveFeelingsToo Right, it really should have been written as $$f(x)/d(x) = u(x) \in \mathbb Q[x]$$  and similarly $v(x)$ for $g(x)/d(x)$. Then the change of variables hold and we can work with $u(x+a), v(x+a)$ first until $\gcd(u(x+a),v(x+a))=1$. Then $$\gcd(u(x+a)d(x+a),v(x+a)d(x+a)) = d(x+a)$$ and we can plug back $f,g$. Then translate $x+a$ back to $x$.
Answer: Hint $ $ shifts $\,f\mapsto \bar f$ are automorphisms so preserve divisibility $\ cd=a\iff \bar c \bar d = \bar a,\,$ so gcds
$$ c\mid (a,b)\iff c\mid a,b\iff \bar c\mid \bar a,\bar b\iff \bar c\mid (\bar a ,\bar b)$$


Specific values or asymptotic behaviour of ${}_{1}F_{1}$

Question: I'm interested in analysing the behaviour of the hypergeometric function ${}_{1}F_{1}$ for the following cases:
$${}_{1}F_{1}\Big[\frac{1}{2} - \frac{k}{2}, \frac{1}{2}; -z^2\Big]$$
and
$${}_{1}F_{1}\Big[1 - \frac{k}{2}, \frac{3}{2}; -z^2\Big]$$
for integer values, $k \in \mathbf{N}$. So far I've used some properties I found on NIST for small values of $k$ and uncovered error functions.
Can these functions be simplified more generally for this special case of integer or half-integer first arguments -- even getting some asymptotic form as $k \rightarrow \infty$ would be very welcome information!
EDIT:
It seems that I can use the identities
$$\begin{align}
{}_{1}F_{1}\Big[\frac{1}{2} - \frac{k}{2}, \frac{1}{2}; -z^2\Big] &= \textrm{e}^{-z^2}{}_{1}F_{1}\Big[\frac{k}{2}, \frac{1}{2}; z^2\Big]\\
{}_{1}F_{1}\Big[1 - \frac{k}{2}, \frac{3}{2}; -z^2\Big] &= \textrm{e}^{-z^2}{}_{1}F_{1}\Big[\frac{k}{2}+\frac{1}{2}, \frac{3}{2}; z^2\Big]
\end{align}
$$
to change sign of the first argument - does this help uncover any additional properties I can use?
Thanks for your suggestions!
Comment: For even $k$ it's a polynomial of degree $k−2$, related to Laguerre polynomials.
Comment: Well, it looks like you have the Hermite function $h_\lambda(x)={}_1F_1(-\lambda/2,1/2;x^2)$ with a sign difference.
Comment: The only reference I've found connecting these functions to Laguerre polynomials also contains awkward error functions - is there a relationship without these?
Comment: https://en.wikipedia.org/wiki/Laguerre_polynomials#Relation_to_hypergeometric_functions
Comment: Apply [this method](https://math.stackexchange.com/a/3288328) to
$$F = \sum_{j \geq 0} \frac {(n + a)_j} {(b)_j} \frac {x^j} {j!}.$$
If $j = \alpha \sqrt n$, the summand is asymptotic to $f(\alpha) e^{\phi(\alpha) \sqrt n}$ with
$$f(\alpha) = \frac {\Gamma(b) e^{\alpha^2/2}} {2 \pi \alpha^b n^{b/2}}, \\
\phi(\alpha) = \alpha (\ln x - 2 \ln \alpha + 2).$$
It suffices to approximate the sum for values of $\alpha$ close to $\sqrt x$, which gives
$$F \sim \sqrt n f(\sqrt x) \int_{\mathbb R}
 e^{(\phi(\sqrt x) + \phi''(\sqrt x) \alpha^2/2) \sqrt n} d\alpha,
\quad n \to \infty.$$
Comment: Great suggestion - thank you very much!
Answer: By http://dlmf.nist.gov/13.6.iv, we have
$$
e{}^{ - z^2 }\!{}_1F_1 \!\left( {\frac{k}{2},\frac{1}{2};z^2 } \right) \!= \frac{{2^{\frac{k}{2} - 1} }}{{\sqrt \pi  }}\Gamma\! \left( {\frac{{k + 1}}{2}} \right)\!e^{ - \frac{{z^2 }}{2}} \!\!\left( {U\!\left( {k - \frac{1}{2},\sqrt 2 z} \right) \!+ U\!\left( {k - \frac{1}{2}, - \sqrt 2 z} \right)} \right)
$$
and
$$
e{}^{ - z^2 }\!{}_1F_1 \!\left( {\frac{{k + 1}}{2},\frac{3}{2};z^2 } \right) \!= \frac{{2^{\frac{k}{2} - 2} }}{{z\sqrt {\pi } }}\Gamma\! \left( {\frac{k}{2}} \right)\!e^{ - \frac{{z^2 }}{2}} \!\!\left( {U\!\left( {k - \frac{1}{2}, - \sqrt 2 z} \right) \!- U\!\left( {k - \frac{1}{2},\sqrt 2 z} \right)} \right)\!.
$$
The parabolic cylinder functions then may be expressed in terms of the derivatives of the complementary error function (cf. http://dlmf.nist.gov/12.7.E6). There is probably no simpler "explicit" expression for these functions.
Comment: This is exactly the kind of thing I was looking for - certainly each function in these expressions admits an easy asymptotic expansion


How are missing data handled in Time series estimation?

Question: I am looking for most popular/theoretically sound methods for handling missing data in time series model (particularly ARMA class) estimation. Also what method is used in R (in arima and in forecast package)
My concern is that if there is enough data missing, the sample autocorrelation at longer lags may not be available and may influence model selection (in auto.arima). How is this handled?
Answer: ARIMA models in R are handled in a state space framework. See the help file for stats::arima which includes the following section:

Fitting methods
The exact likelihood is computed via a state-space representation of the ARIMA process, and the innovations and their variance found by a Kalman filter. The initialization of the differenced ARMA process uses stationarity and is based on Gardner et al (1980). For a differenced process the non-stationary components are given a diffuse prior (controlled by kappa). Observations which are still controlled by the diffuse prior (determined by having a Kalman gain of at least 1e4) are excluded from the likelihood calculations. (This gives comparable results to arima0 in the absence of missing values, when the observations excluded are precisely those dropped by the differencing.)
Missing values are allowed, and are handled exactly in method "ML".

The Kalman filter allows for the likelihood to be computed exactly when missing values are present.
Model selection via forecast::auto.arima does not use sample autocorrelations. It uses Akaike's Information Criterion which is based on the likelihood, so there is no problem in computing it when there are missing values present.
Answer: I have been working on Time Series data since past few weeks, my suggestions to your problem would be trying with basic operations like forward fill, backward fill and interpolation,
Examples
The other way would be using a R package called tsclean()
Read the Documentation here
Here is an example
Hope this helps!
Comment: This is not what I am looking for actually. I a looking for estimation methods applied when there is some missing data. Interpolation would first complete the series and then fit a model. There are methods which directly fit the model and estimate it. I am looking for those methods.


Limits as adjoint to constant functors

Question: Let us suppose that $C$ is a category where small limits are representable and $D$ is a small category.
Let $F:Hom(D,C)\rightarrow C$ be the functor defined by $F(f)=\varprojlim f$.
Is it true that $F$ has a right adjoint constant functor? Can someone please explain this to me? What about the unit and co-units here?
Also, could someone please suggest me some nice introductory books on this stuff? I am finding it very confusing.
Answer: No, $F$ has no right adjoint in general, but it has a left adjoint, which sends each object $c$ of $C$ to the constant functor in $\text{Hom}(D,C)$ that sends every object of $D$ to $c$.
I'm probably old-fashioned, but I still think the best introductory book on category theory is Saunders Mac Lane's "Categories for the Working Mathematician".
Comment: I like Mac Lane as well, though I also like Awodey's *Category Theory*, which doesn't cover as much ground, but is much friendlier to newcomers.
Answer: As Andreas already answered, $F$ has no right adjoint in general. Let's see why this is so on a simple example.
Take $C=\mathrm{Set}$, the category of sets and take $D$ a category with two objects, call them $*$ and $**$, and only identity morphisms. Then given an element $f \in \mathrm{Hom}(D, C)$ its limit is simply the product $f(*) \times f(**)$. Now limits and colimits in $\mathrm{Hom}(D, C)$ are given pointwise. Take $f, g : D \to C$ to be the constant $1$ (terminal object in $\mathrm{Set}$) functors. Then $F(f + g) = (1 + 1) \times (1+1)$ which is a set of cardinality $4$. On the other hand $F(f) + F(g) = 1 \times 1 + 1 \times 1$ which is a set of cardinality $2$, thus $F(f + g) \not\cong F(f) + F(g)$.
Now if $F$ were to have a right adjoint it would itself be a left adjoint and left adjoints preserve colimits. $F$ does not preserve colimits, thus it has no right adjoint.
Note however that in a quite a few cases $F$ will preserve colimits and will have a right adjoint.
Comment: Thank you @ Andreas and @Ales, this makes good sense. But just now I have found another question in this website ( titled Problem on Adjoint Functor by Mr Andy Flower) which is very similar to mine, except that Mr Flower wants to prove that $F$ has a right adjoint k such that k(X) is the constant functor under the same setting. What is your take on this one? Does it not imply that right adjoint exists? Or is that question false?
Comment: Well the question is wrong in the sense that what they want does not exist. In general, there will be no right adjoint. If this is homework then whomever stated it made a mistake.
Comment: Note however that if you were to take $C$ to have small colimits and $F$ to be the colimit functor, then it would have the constant functor functor as the right adjoint.

Limits as adjoint to constant functors


Need efficient way to perform database update from excel file for survey manager

Question: I have a tree with respondent-unit structure. Where respondents belong to units.
Lets assume that there are two departments in Bank, so I will have:
  + Accountants
     - Homer Simpson
     - Bart Simpson
     - Leny
  + It department
     - Charles Bartowski
     - Sarah Walker
     - Bill

Where accountans and IT are departments, and respondents inside them.
Each respondent has general properties:
 - Email
 - Language
 - Name
 - DepartmentID

Each department can provide it's own dynamic properties for all his respondents:
 - Salary
 - Respondent age
 - Gender

This properties can be added and deleted in runtime. They are stored as tables
Properties:
- Property name
- Property ID

Property values:
 - PropertyValueID
 - PropertyID
 - Value

I need to perform possibility to export them in Excel file, then user can edit smth in this excel and when he uploaded it to server, database should be updated.
I did export in Excel.
Now I am looking for good approach to perform allow upload.
Are there any articles or patterns for that to do it in efficient way?
Thanks in advance)
I am using ASP.NET and EPPlus for working with Excel.
Answer: I will start my answer with a strong recommendation to not do this!  =)
Exporting to Excel will be easy because your database is consistent and abides by certain rules and logic.
If you let a user modify an Excel file, let them upload it, and then try to parse it and update a database, you're asking for a lot of headaches.
Here is how you might think about doing it:

First I would let them upload the file.  Not sure what you're using for that, but it's the obvious start.
Next I would create a job to import the Excel file into the database.  If it's SQL Server, I would import it with an SSIS job as an example.  The goal is to bring it in to your database in a temporary table.
Write some code to read from the temporary table (which is your recently imported Excel data) and do validations.  Validate the field names are correct, the data types are correct, etc.
Write code to import the data from the temporary table to your master table(s).  You only do this after you've verified programatically the imported file is good.
Make sure you lock down Excel as much as possible.  Lock the sheet(s) so the user can't modify your data entry template.  Try to do validations in Excel so they don't use values that are wrong.  (For example, if you have an enumeration, make sure they stick to it instead of putting in free form text.)

The real problem with this approach is that users are crazy and they will not follow your template or guidelines.  I would expect to spend a lot of time debugging bad imports.
It's not an option to make a simple web application to let them add/edit data?  That way you would have more control over their input.  Excel potentially gives them too much power.
(Then again you can make forms with VBA in Excel to maybe restrict their input.)


Como retirar a borda das Colunas do Header

Question: Eu não to conseguindo retirar a borda branca do header da jTable
public static JTable CMtable(String[] colunas, int width, int height) {
    TableColumn coluna;
    JTable tbl = new JTable();
    tbl.setBounds(0,0,width,height);
    for (int i = 0; i < colunas.length; i++) {
        coluna = new TableColumn(i);
        coluna.setHeaderValue(colunas[i]);
        tbl.addColumn(coluna);
    }
    tbl.getTableHeader().getColumnModel().getColumn(0).setPreferredWidth(tbl.getWidth()*1/10);
    tbl.getTableHeader().getColumnModel().getColumn(1).setPreferredWidth(tbl.getWidth()*5/10);
    tbl.getTableHeader().getColumnModel().getColumn(2).setPreferredWidth(tbl.getWidth()*4/10);
    tbl.getTableHeader().setReorderingAllowed(false);
    tbl.getTableHeader().setResizingAllowed(false);
    tbl.setRowSelectionAllowed(false);
    tbl.setShowHorizontalLines(false);
    tbl.setShowVerticalLines(false);

    tbl.setFont(new Font("Tahoma", 1, 14));
    tbl.setBackground(BdoColor.CINZA);
    tbl.getTableHeader().setFont(new Font("Tahoma", 1, 14));
    tbl.getTableHeader().setOpaque(false);
    tbl.getTableHeader().setForeground(Color.WHITE);
    tbl.getTableHeader().setBackground(BdoColor.CINZA);
    tbl.setAlignmentX(SwingConstants.CENTER);
    tbl.getTableHeader().setAlignmentY(SwingConstants.CENTER);
    tbl.setRowHeight(25, 1);
    tbl.setBackground(BdoColor.CINZA);

    return tbl;
}

O resultado apos executar é o seguinte:

Só que eu queria que não tivesse as bordas brancas, já tentei mexer com o setBorder mas não obtive resultado.
Aguardo resposta!, qualquer dica de estudo seria bem vinda! Valeu
Comment: Possível duplicata de [Como mudar o background da header da JTable sem retirar a borda?](https://pt.stackoverflow.com/questions/163398/como-mudar-o-background-da-header-da-jtable-sem-retirar-a-borda)
Answer: Importe:
import javax.swing.BorderFactory

e tente:
tbl.setBorder(BorderFactory.createLineBorder(Color.GRAY,1));


Logistic Regression using two categorical variables

Question: I've been doing some work on regression, and a paper in particular caught my attention where they utilised two categorical variables in a logistic regression. From my understanding, if you're using two categorical variables, does that not just come down to conditional probabilities? Out of interest, I wrote some code, and was surprised to see that the method returned a p value (reproducible code below). I'm struggling to understand what the null and alternative hypotheses are in this context, so that I can interpret what the p value is telling me. Any assistance in dissecting the resulting model summary would be greatly appreciated.
Example interpretation:
P(Dep = Class1 | Pred = C) = 20/120 = 1/6
Is the model summary telling me this?
Code
foo <- data.frame(Pred   = c(rep("A",80),rep("B",20),
                             rep("C",40),rep("D",60)),
                  Dep    = c(rep("Class1",120),
                             rep("Class2",80)))
fit      <- glm(Dep ~ Pred, family=binomial(link='logit'), data = foo)
summary(fit)

Output
   Call:
glm(formula = Dep ~ Pred, family = binomial(link = "logit"),
    data = foo)

Deviance Residuals:
     Min        1Q    Median        3Q       Max
-1.17741  -0.00003  -0.00003   0.00003   1.17741

Coefficients:
              Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.157e+01  3.268e+03  -0.007    0.995
PredB       -7.168e-11  7.308e+03   0.000    1.000
PredC        2.157e+01  3.268e+03   0.007    0.995
PredD        4.313e+01  4.992e+03   0.009    0.993

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 269.205  on 199  degrees of freedom
Residual deviance:  55.452  on 196  degrees of freedom
AIC: 63.452

Number of Fisher Scoring iterations: 20

Comment: Your formula looks like it is backwards. Shouldn't it be `Dep ~ Pred`?
Comment: Good Point, I'll adjust that.
Comment: If you reverse the formula the resulting model should give P(Dep = Class2 | Pred = C) = 1 and P(Dep = Class2 | Pred = D) = 1.  Look at a cross table of Pred and Dep and you'll notice that Class1 is not observed when Pred is either C or D
Comment: This should give you the conditional probabilities that you're looking for: `1/(1+exp(fit$coefficients[1]));
1/(1+exp(fit$coefficients[2]));
1/(1+exp(fit$coefficients[3]));
1/(1+exp(fit$coefficients[4]))`
Comment: Ok, sorry to be a pain, but I'd changed the numbers slightly so that the example makes a bit more sense. So in the new example, I'd expect that `P(DepCClass1 | Pred = C) = 20/120 = 1/6`, however when applying your formulae, I see `1`.
Comment: Your results are a mess because you have quasi-complete sample separation.  $P\{Class2 | D \}=1$.  I'm surprised glm did not throw an error message.  It should have.
Comment: Per @Bill if there is separation, you should get a warning that some values were 0 or 1. Maybe not the clearest warning message.
Answer: Your intuition is right.
The logistic regression model gives identical inference to a Pearson chi-sq test of "independence" for categorical data (in the long run). In both cases, the null hypothesis is that the conditional probabilities are equal to the marginal probabilities. You can show with some algebra that this necessarily implies that the odds ratio is 1.
The minute differences in the actual logistic regression and Pearson test statistics owes to how they're computed. The $p$values you get in R from calling summary.glm come from a Wald test whereas the Pearson test is a related Score test.
The problem with your example is that you have a singular logistic model. One advantage of the score test is that it can provide test statistics for models like this one which "explode". For a more sane example, consider the following:
set.seed(123)
foo2 <- as.data.frame(sapply(foo, sample)) ## permute to avoid singularity
fit <- glm(Dep ~ Pred, data=foo2, family=binomial)
library(lmtest)
waldtest(fit, test='Chisq')
chisq.test(table(foo2))

Gives us:
> waldtest(fit, test='Chisq')
Wald test

Model 1: Dep ~ Pred
Model 2: Dep ~ 1
  Res.Df Df  Chisq Pr(>Chisq)
1    196
2    199 -3 1.7774     0.6199
> chisq.test(table(foo2))

    Pearson's Chi-squared test

data:  table(foo2)
X-squared = 1.7882, df = 3, p-value = 0.6175

Comment: +1. In terms of "exploding", it is possible with Bayesian logistic regression to keep the coefficients from exploding by imposing reasonable priors. Not sure that this is actually useful, but it might be.
Comment: @Wayne it's a good suggestion, I don't know how anything but an informative prior could prevent separation in logistic regression. Maybe we ought to rethink how "bad" informative priors are. When I've encountered singular logistic regression in practice, though, I simply comment on those findings and present the results from a score test, also inverting to obtain a CI which includes $\infty$.


Fitting a dataset into another dataset

Question: I had heard that there is a body of literature devoted to the following problem:
You have a dataset and you produced a good predictive model for it. Now you have a different dataset, derived from different instruments, or different data sources, but similar enough that you can hope to scale one to the other so that you can use the predictive model on it.
Alternately, you do not have enough data to produce a model and hope that combining different datasets by scaling one to the other will help you reach the quantity of data you need.
First off what is this problem called so that I can search it more effectively? And also does anyone know of a good recent survey of techniques for this, either in book or paper form? I currently have access to most academic journals so those links work for me as well.
Say I have dataset A that I have a model for, dataset B occupies the same database schema but is from a difference source with different factors that are not included in my feature set.
Initially my intuition was to construct a QQplot and fit lines (or curves) to features that I thought should be similar. If the difference in the way feature 1 from A increases is similar to the way feature 1 from B increases but with a constant factor then fitting a line can reveal this factor. If the difference was exponential or logarithmic then I could scale using a fitted function. In this way I could constrain the way one variable increased to fit how another variable increased.
However this is just my intuition. I can certainly test it for overfitting but when I had heard that there was a lot of literature devoted to this subject then it seemed as though I should learn a few ways in which I could question my assumptions. It would probably be good for me to review the literature.
Does anyone know what the tag for this literature might be?
Answer: My original source finally got back to me. The topic I want is called transfer learning. I'll share any survey resources I find here.
EDIT: As I promised, if I find a literature survey I'll include it here. I found this one.

Reference
PAN, S. J.; YANG, Q. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, v. 22, n. 10, p. 1345–1359, out. 2010.
Answer: Since you have a different dataset, which is derived from a different data source, but there are some commonalities among them, you can try to adapt the model that you have produced for the first dataset (where we assume that we have a sufficient number of data available) to solve the new problem of the second dataset. In the literature this topic of machine learning is called Transfer Learning or in some cases you can find it as Domain Adaptation. The most important in this kind of adaptation/transfer is the kind of difference among the two datasets. They can have different marginal distributions or different feature spaces. In the latter case, which is called heterogeneous Transfer Learning, you need to find a way to align the two different feature spaces.
If we assume that the source domain is the old dataset and the target domain is the newer one, then the Domain Adaptation process attempts to alter the source domain in a way to bring the distribution of the source closer to that of the target domain.
Transfer Learning in comparison with Domain Adaptation, is a broader topic. Apart from the above explained difference in the marginal distributions or in feature spaces, Transfer Learning can also include the case when you want to transfer knowledge from an old task to a new and different one.
Here are some surveys that analyzing the topic of Transfer Learning,
PAN, S. J.; YANG, Q. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, v. 22, n. 10, p. 1345–1359, out. 2010.
Weiss, K., Khoshgoftaar, T.M. & Wang: A Survey of Transfer Learning.
D. J Big Data, 2016.

Fitting a dataset into another dataset


Amazon s3 file download "TypeError: expected string or bytes-like object"

Question: I am trying to download a "csv" file using S3 with Python but it returns an error regarding line s3.download_file(bucket, origin, path):
TypeError: expected string or bytes-like object

Does anyone know why this happens?
from boto3.session import Session
import boto3
import sys

ACCESS_KEY="abc"
SECRET_KEY="xxx"
path="./ListadePrecios.csv"
origin="public/excel/Nombre.csv"

session = Session(aws_access_key_id=ACCESS_KEY,
                  aws_secret_access_key=SECRET_KEY)
s3 = session.resource("s3")
bucket = s3.Bucket("my_bucket")

def main(argv):
    #Descargar archivo
    s3 = boto3.client("s3", aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY)
    s3.download_file(bucket, origin, path)

if __name__ == "__main__":
    main(sys.argv[1:])

Error screenshot
Comment: Paste the error as text in the question so we can see it and it will stay forever with the question.
Comment: Try by replacing `bucket` with `"my_bucket"` in `download_file`
Comment: clever how simple, thanks, now in the ʻobj.delete () `part, is that correct?
Answer: The issue is that you are passing a Bucket object into:
s3.download_file(bucket, origin, path)

Instead, pass the bucket name:
s3.download_file("my_bucket", origin, path)

Amazon s3 Documentation

download_file(Bucket, Key, Filename, ExtraArgs=None, Callback=None, Config=None)
Download an S3 object to a file.
Usage:
import boto3
s3 = boto3.resource('s3')
s3.meta.client.download_file('mybucket', 'hello.txt', '/tmp/hello.txt')

Similar behavior as S3Transfer's download_file() method, except that parameters are capitalized. Detailed examples can be found at S3Transfer's Usage.
Parameters

Bucket(str) -- The name of the bucket to download from.
Key(str) -- The name of the key to download from.
Filename(str) -- The path to the file to download to.
ExtraArgs(dict) -- Extra arguments that may be passed to the client operation.
Callback(function) -- A method which takes a number of bytes transferred to be periodically called during the download.
Config(boto3.s3.transfer.TransferConfig) -- The transfer configuration to be used when performing the transfer.
Comment: ready problem solved, now I am solving the delete file
Answer: Your issue is with the first param of download_file

download_file(Bucket, Key, Filename, ExtraArgs=None, Callback=None, Config=None)

Parameters are:

Bucket (str) -- The name of the bucket to download from.
Key (str) -- The name of the key to download from.
Filename (str) -- The path to the file to download to.

So you should replace
s3.download_file(bucket, origin, path)

with
s3.download_file("my_bucket", origin, path)

Comment: ready problem solved, now I am solving the delete file


Weights versus roots

Question: I am not sure of the difference between weights and roots. Am I correct in thinking that the weights are the eigenvalues of the action of the maximal torus on a given representation, and the roots are the weights that occur when the maximal torus acts on the Lie algebra itself under the adjoint representation?
If you have time, what would this look like for $\mathfrak{sl}_3\Bbb C$?
Comment: Yes, that's right!
Answer: Certain representations of a (say, reductive) Lie algebra will decompose into eigenspaces for the action of a Cartan subalgebra $\mathfrak{h}$ (the Lie algebra analogue of a maximal torus).  Thus, $\mathfrak{h}$ acts on these subspaces, called "weight spaces" via a linear form $\mathfrak{h} \to \mathbb{C}$ (if you're working over the complex numbers), that is, an element of the dual space $\mathfrak{h}^*$.  These elements of $\mathfrak{h}^*$ (that correspond to nonzero eigenspaces) are called the "weights" of the representation.  Then you are exactly right when you say that the "roots" of a Lie algebra are the weights of its action on itself via the adjoint representation.  So roots are special types of weights.

Weights versus roots


How to display multiple density or distribution functions on a single plot?

Question: What would be the easiest way to plot in the same graph, the probability density function or the cumulative distribution function of a distribution, for 3-4 different parameter values.
Let's take for instance the Weibull distribution W(a,b). I would want to plot the PDF for a=1/2,b=20 , a=1/3,b=20, a=1/4,b=20, a=1/5,b=20 in the same graph and in different colors.
What is the easiest way to do this? You can recommend any kind of software. Also, the program should let me define custom distributions.
Answer: I love R, easy and free. Here's an example:
# The par removes the "padding" from the axis
par(xaxs="i", yaxs="i")

# Initiate the x, a small "by" is neat for a smooth curve
# Can't use 0 since it gives produces an integrate() error
x <- seq(0.0001, 3, by=.01)

# Just some vanity - adding a little color :-), heat.colors(5) could be an option
colors <- c("darkred", "red", "orange", "gold", "yellow")

plot(x, type="n", ylim=c(0,3), ylab="Density")
for(i in 1:5){
    lines(x, dweibull(x, shape=1/i), col=colors[i])
}
title("Weibull tests")

Gives this:

Update
I've played around with Peter Flom's suggestion with the integrate function. The prob. function, same as above:
plot(x, type="n", ylim=c(0,1), xlim=range(x), ylab="Prob")
for(i in 1:5){
  lines(x, pweibull(x, shape=1/i), col=colors[i])
}
title("Using the pweibull funciton")

Give this graph:

When using the integrate function to get the "same" graph the code looks like this:
plot(x, type="n", ylim=c(0,1), xlim=c(0, max(x)), ylab="Density")
for(i in 1:5){
  t <- apply(matrix(x), MARGIN=1, FUN=function(x)
    integrate(function(a) dweibull(a, shape=1/i), 0, x)$value)

  lines(x, t, col=colors[i])
}
title("Using the integrate funciton")

And this gives virtually an identical graph:
Comment: If you can write it as a function, you can probably write it in R. If (as seems likely) the pdf involves integration, have a look at [this](http://stat.ethz.ch/R-manual/R-devel/library/stats/html/integrate.html), or try `?integrate` from R.
Comment: Hi! That looks very nice, thank you very much. Is there a quick way to insert a custom function as a density function in your R code?
Comment: @PeterFlom: Thank you for your tip. Added an example based on this.
Answer: I like R too. Here is a more or less generic function to plot any probability distribution from the base R functions. It should not be difficult to extend the code with functions available in other packages, e.g. SuppDists.
plot.func <- function(distr=c("beta", "binom", "cauchy", "chisq",
                              "exp", "f","gamma", "geom", "hyper",
                              "logis", "lnorm", "nbinom", "norm",
                              "pois", "t", "unif", "weibull"),
                      what=c("pdf", "cdf"), params=list(), type="b",
                      xlim=c(0, 1), log=FALSE, n=101, add=FALSE, ...) {
  what <- match.arg(what)
  d <- match.fun(paste(switch(what, pdf = "d", cdf = "p"),
                       distr, sep=""))
  # Define x-values (because we won't use 'curve') as last parameter
  # (with pdf, it should be 'x', while for cdf it is 'q').
  len <- length(params)
  params[[len+1]] <- seq(xlim[1], xlim[2], length=n)
  if (add) lines(params[[len+1]], do.call(d, params), type, ...)
  else plot(params[[len+1]], do.call(d, params), type, ...)
}

It's a bit crappy and I haven't tested it a lot. The params list must obey R's conventions for naming {C|P}DF parameters (e.g., shape and scale for the Weibull distribution, and not a or b).
There's room for improvement, especially about the way it handles multiple plotting on the same graphic device (and, actually, passing vector of parameters only works as a side-effect when type="p"). Also, there's not much parameter checking!
Here are some examples of use:
# Normal CDF
xl <- c(-5, 5)
plot.func("norm", what="pdf",  params=list(mean=1, sd=1.2),
          xlim=xl, ylim=c(0,.5), cex=.8, type="l", xlab="x", ylab="F(x)")
plot.func("norm", what="pdf",  params=list(mean=3, sd=.8),
          xlim=xl, add=TRUE, pch=19, cex=.8)
plot.func("norm", what="pdf",  params=list(mean=.5, sd=1.3), n=201,
          xlim=xl, add=TRUE, pch=19, cex=.4, type="p", col="steelblue")
title(main="Some gaussian PDFs")

# Standard normal PDF
plot.func("norm", "cdf", xlab="Quantile (x)", ylab="P(X<x)", xlim=c(-3,3), type="l",
          main="Some gaussian CDFs")
plot.func("norm", "cdf", list(sd=c(0.5,1.5)), xlim=c(-3,3), add=TRUE,
          type="p", pch=c("o","+"), n=201, cex=.8)
legend("topleft", paste("N(0;", c(1,0.5,1.5), ")", sep=""),
       lty=c(1,NA,NA), pch=c(NA,"o","+"), bty="n")

# Weibull distribution
s <- c(.5,.75,1)
plot.func("weibull", what="pdf", xlim=c(0,1), params=list(shape=s),
          col=1:3, type="p", n=301, pch=19, cex=.6, xlab="", ylab="")
title(main="Weibull distribution", xlab="x", ylab="F(x)")
legend("topright", legend=as.character(s), title="Shape", col=1:3, pch=19)

Comment: I can't quite figure out why looking over the code at first glance, but the pdf's and the cdf's are switched.
Comment: Wow, impressive code - really interesting to see how to take my idea to the next level. Thanks @chl!
Comment: @Andy Sure! It looks like I swapped the type arguments at some point when cleaning up an earlier version of the code. (And good catch, btw)
Answer: The question asks for "easiest."  Interpreting that in terms of either (i) lines of code, (ii) naturality of expression, or (iii) raw capabilities, I find the Mathematica solutions to be well worth considering.
For example,
Plot[Evaluate[
  PDF[WeibullDistribution[#, 20]][x] & /@ {1/2, 1/3, 1/4, 1/5}], {x, 0, 1},
      AxesOrigin -> {0, 0}]

produces the example in the question

and
gMixture[x_, weights_, shapes_, scales_] :=
  MapThread[PDF[GammaDistribution[##]][x] &, {shapes, scales}] . weights / Total[weights];
Plot[gMixture[x, {1, 2, 3}, {2, 3, 10}, {1, 1, 1}], {x, 0, 20}, AxesOrigin -> {0, 0}]

shows what it takes to define and plot a new distribution (here, a mixture of gammas):

Need something more exotic?  It's likely already part of Mathematica.  E.g., here is a PDF obtained from a Jacobi theta function by normalizing its area to unity:
With[{c = NIntegrate[EllipticTheta[1, z, 1/2], {z, 0, Pi}]},
 Plot[EllipticTheta[1, z, 1/2] / c, {z, 0, Pi}, Filling -> Axis]]

Comment: This is very nice! Could you point me to a good documentation for the "Evaluate" function? The Mathematica "Help" doesn't tell me alot about all the options this function offers. What if I would want to use a vector of values for the second (instead of 20), as well as for the first parameter in the Weibull distribution? How do I do this? More specifically, how would I plot for instance the PDF of WeibullDistribution[1/5, 50], WeibullDistribution[1/4, 40], WeibullDistribution[1/3, 30], WeibullDistribution[1/2, 20] in the same graph?
Comment: (1) For the use of `Evaluate`, see the examples for `Plot`.  (2) Emulate this: `Plot[Evaluate[PDF[WeibullDistribution @@ #][t] & /@ {{1/5, 50}, {1/4, 40}, {1/3, 30}}], {t, 0, 1}]`  (3) For tougher questions, please visit our [Mathematica site](http://mathematica.stackexchange.com/).


Creating Custom TF.DataSet image dataset

Question: https://www.tensorflow.org/datasets/add_dataset
This Link shows how to create custom dataset. I have tried with this, but failed to create my own dataset
I have a folder 'Train' and 4 sub folders in that folder and have to create training data and label.
The code I used
cd path/to/my/project/datasets/
tfds new my_dataset  # Create `my_dataset/my_dataset.py` template files
# [...] Manually modify `my_dataset/my_dataset.py` to implement your dataset.
cd my_dataset/
tfds build  # Download and prepare the dataset to `~/tensorflow_datasets/`

one folder created by the program  my_dataset/ with three .py pgms
I didn't get the data and label.I don't know where the path of my dataset to specify.
NB: My purpose is to call my dataset using
ds = tfds.load('my_dataset')

Thank You
Answer: After this code
cd path/to/my/project/datasets/
tfds new my_dataset

You have to modify the python files generated to include your data. (modify all TODO lines)
Given tutorial is not very clear. One simple example for image_classification could have cleared all the issues. I am also stuck on registering dataset step.
I will notify you here If I find a solution.


comment_admin_overview is missing

Question: I am working on hook_theme_registry_alter. I have tried to dpm($theme_registry). But comment_admin_overview is missing, The version of drupal is 7. Any ideas??.
function hook_theme_registry_alter(&$theme_registry) {
  dpm($theme_registry);
  $theme_registry['comment_admin_overview']['function'] = 'theme_hook_comment_admin_overview';
}

Answer: comment_admin_overview() is a form function in Drupal 7 (it was a theme function in 4, 5 & 6).
You can use hook_form_alter() to make your changes instead.
Comment: Hi Clive, I wish to theme the entire comment listing, so it would be better to use hook_form_alter. rt?
Comment: You can theme the comment specifically by overriding comment.tpl.php in your theme (or via `hook_registry_alter()` in a module like you're already trying), but the form needs to be themed as a separate 'thing', as it's built differently and only included in the comment, rather than built as a part of it
Comment: One more question. What I am trying to do is exactly adding few more operation to VBO and listing the comments along with a collapsible form to edit the same with pre filled value in hook_form_formID_alter. Is there any thing wrong doing this way. I have a drupal 6 module which I'm trying to convert it into drupal 7. In form_alter, I'm checking the form id and assigning the #submit a particular function.

comment_admin_overview is missing


Slurm Config for limit the endtime for all new jobs on the cluster to a certain date

Question: Our cluster has to be shutdown for an update in two weeks. We would like to let users use the cluster until the last day, but we want to make sure, no job can be started, which would end after the shutdown date. Is there an easy way to limit the runtime of slurm jobs to a certain date instead of a fixed period of time (like the usual 5 days).
Is there a way to do this easily, or do have a change the slurm config each day and restart the slurm daemon?
Answer: SLURM has a feature to make this easier.  The feature is called Resource Reservation, and in particular, it's the Maintenance functionality of Resource Reservation.
The following example using the scontrol command is taken directly from the SLURM documentation for Reservation Creation :
<code>scontrol create reservation starttime=2009-02-06T16:00:00 \
   duration=120 user=root flags=maint,ignore_jobs nodes=ALL
</code>


Como fazer abas semelhantes às do WhatsApp?

Question: Alguém tem algum exemplo de como posso criar este estilo de abas no Android.
Na imagem o exemplo é do Whats App.

Me refiro:
1 - as abas CALL, Chats, Contacts
2 - A barra de Busca, que fica sempre, independente da aba que eu clicar
Alguém pode me ajudar?
Comment: A que imagem se refere?
Comment: Essa: https://fs02.androidpit.info/userfiles/4774964/image/AndroidPIT-WhatsApp-Material-Design-update-1-w782.jpg
Answer: No caso do WhatsApp eu imagino que seja usado um TabLayout e um ViewPager:
XML:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <android.support.design.widget.TabLayout
        android:id="@+id/tab_layout"
        app:tabGravity="fill"
        app:tabMode="fixed"
        android:background="@color/colorPrimary"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"/>

    <android.support.v4.view.ViewPager
        android:id="@+id/pager"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>

</LinearLayout>

FragmentPagerAdapter:
public class MyCustomPagerAdapter extends FragmentPagerAdapter {
    public MyCustomPagerAdapter(FragmentManager fm) {
        super(fm);
    }

//   Aqui você retorna uma instancia de fragment para cada aba dependendo da posição
    @Override
    public Fragment getItem(int position) {
//  Exemplo:
        switch (position) {
            case 0: return new CallFragment();
            case 1: return new ChatFragment();
        }
        return null;
    }

//   Aqui você retorna o número de páginas que o pager vai ter
    @Override
    public int getCount() {
//  Exemplo
        return 2;
    }

//  Aqui você retorna o titulo de acordo com a posição de cada aba (as que você definiu antes no getItem)
    @Override
    public CharSequence getPageTitle(int position) {
//  Exemplo
        switch (position) {
            case 0: return "Calls";
            case 1: return "Chats";
        }
        return super.getPageTitle(position);
    }
}

Activity
public class MyActivity extends AppCompatActivity {
    @Override
    public void onCreate(Bundle savedInstanceState, PersistableBundle persistentState) {
        super.onCreate(savedInstanceState, persistentState);
        setContentView(R.layout.my_pager_layouut);
        TabLayout tabLayout = (TabLayout) findViewById(R.id.tab_layout);
        ViewPager pager = (ViewPager) findViewById(R.id.pager);
        MyCustomPagerAdapter pagerAdapter = new MyCustomPagerAdapter(getFragmentManager());
        pager.setAdapter(pagerAdapter);
        tabLayout.setupWithPager(pager);
    }
}

Comment: acho que só faltou adicionar o menu item na toolbar, mas excelente resposta