Order statistics for non i.i.d. variables

Question: $u$ is a random variable with uniform distribution in $[a,b]$ and is not observed (the distribution is known though). At every period $t \in \{2, ..., T\}$, the researcher observes $Y_{t+1} = \alpha Y_t + (1 - \alpha) u_t$, where $u_t$ is an i.i.d. random draw from the uniform distribution in $[a,b]$. $T$, $\alpha$ and $Y_1$ are known.
How to find the distribution of the highest order statistic of $Y_t$ at every period $t$?
--- EDIT ---
I have the following derivations so far. Assume that we are at period 1:
(1) Rewrite $Y_t$ as:
\begin{aligned}
Y_t  &= \alpha Y_{t -1 } + (1 - \alpha) u_{t-1} \\
     &= \alpha^{(t-1)} Y_1 + (1 - \alpha) \sum_{j=1}^{t} u_{t-j} \alpha^{j-1}
\end{aligned}
Similarly for period 2
\begin{aligned}
Y_t  &= \alpha Y_{t -1 } + (1 - \alpha) u_{t-1} \\
     &= \alpha^{(t-2)} Y_2 + (1 - \alpha) \sum_{j=2}^{t} u_{t-j} \alpha^{j-2}
\end{aligned}
Comment: The phrase "highest order statistic of $Y_t$" makes no sense--$Y_t$ is just a single random variable--but your comment clarifies what you mean.  Please edit your post to include that clarification.
Comment: To be clear, you want the distribution of $M_t:=\max(Y_1,\cdots,Y_t)$? Also, is $\alpha>1$, $\alpha=1$ or $\alpha<1$?
Comment: If $t = 1$, I want the distribution of $\max(Y_2, ..., Y_T)$, note that $Y_2$ depends on $Y_1$. If $t = 2$, I want the distribution of $\max (Y_3, ..., Y_T)$, etc...
Comment: sorry, $0 < \alpha  < 1$
Answer: Your problem features not only non-identical distributions, but also dependence. It would be difficult enough to find a solution given numerical T (e.g. T = 10), never mind a general symbolic solution or dependence.
Nevertheless, there is always Monte Carlo ... here is the simulated pdf of the sample maximum when $T  = 10$ (top diagram) and $T = 50$ (bottom diagram). In all cases, $Y_1 = \frac12$, with $\alpha = \frac18, \frac12, \frac78$, as noted. The dashed line denotes a standard Uniform, for comparison.

(source: tri.org.au)
The limit distributions when $\alpha = 0$ and 1 are easy to derive, and make a helpful frame of reference.
Comment: I just found a paper which seems to solve the problem, and provides a closed form solution for the sum of uniform distribution with different support (note that each distribution is independent in my case) http://search.proquest.com/openview/522fabc2d91625ffc120a316ca310513/1.pdf?pq-origsite=gscholar&cbl=31177
Answer: Not really an answer to this quite difficult question, but too long
for a comment.
I doubt that a usable explicit form of the distribution can be found,
and the Monte-Carlo illustrated by @wolfies seems the only viable
approach. A related and less difficult question arises from the
stationary case, when the initial $Y_0$ is assumed to be drawn from
the stationary distribution: then for large $t$, how can we compute
the distribution of $M_t := \max \{Y_1, \, \dots, \,Y_t\}$?
Extreme Value Theory (EVT) is most known for the i.i.d. context, and the
Fisher-Tippet-Gnedencko theorem
gives the possible limit distributions for $M_t$.
When $\alpha = 0$ we know from EVT that $V_t:= t [M_t - 1]$ converges
in distribution to the reverse exponential with density $e^v 1_{v \leq
0}(v)$.
Extremes of stationary (dependent) sequences have also been studied
quite extensively, and form an important chapter of EVT. The book by
Embrechts P., Klüppelberg C. and Mikosch T. is a very good resource on EVT and
includes several sections (notably 4.4, 5.5 and 8.1) in relation with
the question. For a stationary sequence $Y_t$, the asymptotic
distribution (for large $t$) of the maximum $M_t$ can be related to
the marginal distribution in some cases. A number of results are
available under mixing conditions: then the distribution of the max
of the $t$ dependent observations is for large $t$ the same as the
distribution of $t \theta$ i.i.d. r.vs with the same distribution,
where $0 < \theta \leq 1$ is the extremal index and relates to the
(extremal) dependence between the r.vs $Y_t$.  For a gaussian AR(1)
as well as for stationary gaussian ARMA, if is known that $\theta =
1$, so these processes have (asymptotically) the same extremal
behaviour as an i.i.d. gaussian sequence.
In your case, for $0 < \alpha < 1$ the stationary (marginal)
distribution is not known explicitly but could be investigated since
an expression can be given for the characteristic function or Laplace
transform see Andel and
Hrach.  The
stationary distribution has support $[0,\,1]$ and has
a density which is infinitely derivable, including at its upper
end-point where all derivatives must vanish. This behaviour favours
the conjecture that the stationary distribution is in the Gumbel
domain of attraction, and not in the reverse Weibull domain as was the
case for $\alpha = 0$. I believe that the Von Mises conditions hold,
but did not yet succeed at proving it.
It could be a good approach to begin by finding the domain of
attraction of the stationary distribution, by investigating the
behaviour of the density near the finite upper end-point. Then see if
a mixing condition holds (which seems likely), and then find the
extremal index $\theta$. This might be a material for other
questions.
Comment: Thank you for the help, but I just found a paper providing a closed form density for the sum of uniformly distributed independent rv with different support. http://search.proquest.com/openview/522fabc2d91625ffc120a316ca310513/1.pdf?pq-origsite=gscholar&cbl=31177
Comment: @tony, I understand that you will get the *exact* marginal
distribution of $Y_t$ rather than an approximation of it, but how can
you find the distribution of the maximum?
Comment: By the order statistic, the distribution of the maximum is the product of all the distribution.
Comment: The distribution function of the max is the product of all the
distributions only when the r.vs are independent, so only when $\alpha
= 0$ here. In the general case, it is much more complicated.
Comment: Yes, the general case is complicated, but the formula exists. However, these RVs are independent, once we condition on `Y1`: you can compute the support using only information on `alpha` and `Y1`. It is just a sum of concentric uniform RVs with different support


How to validate and submit a form using AJAX?

Question: I've created a webform using the Form API. I'm using the #AJAX field option to add AJAX validation to each field.
Is it possible to validate and submit the form using AJAX without reloading the page. If validation fails I want to show an error message and if validation is successful then I want to show a message (ideally in a lightbox) and reset the form fields.
My code so far:
$form['name'] = array(
    '#type' => 'textfield',
    '#title' => t('Name'),
    '#default_value' => '',
    '#maxlength' => '128',
    '#required' => TRUE,
    '#ajax' => array(
        'callback' => '_validate_name',
        'wrapper' => 'name-error-icon-container',
        'method' => 'html',
        'effect' => 'none',
        'progress' => array(
            'message' => NULL,
        ),
    ),
);

$form['submit'] = array(
    '#type' => 'submit',
    '#value' => 'Submit',
    '#ajax' => array(
        'callback' => '_handle_form_submit',
        'effect' => 'fade',
    ),
);

callback functions look like this:
function _validate_name($form, $form_state) {

    if ($form_state['values']['name'] != '') {
    $output = 'OK';
    }
    else {
      $output = 'Enter a value';
    }

   return $output;

}

function _handle_form_submit($form, $form_state) {
}

But I'm not sure what should go in the _handle_form_submit function to validate and then either return a message or submit form and reset fields?
Answer: I know this is question has been lying around for a couple of years by now, but I don't feel any of the other answers really get the point of Drupal 7 ajax form submission, so I'll try to explain it.
Since your form should work as well without the ajax as per good practice, your ajax submit handler should do nothing but return the form. Everything else should be in your validation and submit functions.
function ex_form($form, $form_state) {
  $form['name'] = array(
    '#type' => 'textfield',
     '#title' => t('Name'),
     '#default_value' => '',
     '#maxlength' => '128',
     '#required' => TRUE,
  );

  $form['submit'] = array(
   '#type' => 'submit',
   '#value' => 'Submit',
   '#ajax' => array(
     'callback' => 'ex_form_ajax_handler',
     'effect' => 'fade',
   ),
  );
}

function ex_form_submit(&$form, &$form_state) {
  // Submit logic - runs when validation is ok
  // You can alter the form here to be just a markup field if you want.
}

function ex_form_validate(&$form, &$form_state) {
  // Validation logic - invalidate using form_set_error() or form_error()
}

function ex_form_ajax_handler(&$form, &$form_state) {
  return $form;
}

Comment: probably Best answer
Comment: Are you saying that the validate and submit methods will automatically be called for an AJAX callback on a submit button? In what order do the methods get called?
Answer: I think the post by maxtorete on October 17, 2011 seems to give a fuller example using both form_validate() and form_submit()
(I haven't tested it yet.)
Also Joshua Stewardson's answer over at stack overflow has a nice working example:
function dr_search_test_form($form, &$fstate) {

  $form['wrapper'] = [
    '#markup' => '

',
  ];

  $form['name'] = [
    '#type'     => 'textfield',
    '#required' => TRUE,
    '#title'    => 'Name',
  ];

  $form['submit'] = [
    '#type'  => 'submit',
    '#value' => 'Send',
    '#ajax'  => [
      'callback' => 'dr_search_test_form_callback',
      'wrapper'  => 'test-ajax',
      'effect'   => 'fade',
    ],
  ];

  return $form;
}

function dr_search_test_form_callback($form, &$fstate) {

  return '
Wrapper Div
';
}

function dr_search_test_form_validate($form, &$fstate) {

  form_set_error('name', 'Some error to display.');
}

Joshua makes the point that validation error messages replace the #ajax['wrapper'] element completely so your callback needs to resupply this element again as you replace it.
Comment: down vote is a bit harsh - even if it was only a link - it was a link to the answer - not just a link to a module that doesn't have the answer - unlike the accepted answer!! anyway I provided some working code while I was being downvoted.
Comment: when that link invalidates it's a sign the question is no longer relevant!
Answer: In general, validation and form submission handling should happen in the usual _validate() and _submit() callbacks. The idea is that forms should still work without ajax.
Pretty much the only thing the #ajax callbacks should do is return the part of the form that should be replaced, according to the defined wrapper.
Comment: This question is about Drupal 7...AHAH no longer exists in Drupal. In an [`#ajax`](http://api.drupal.org/api/drupal/developer!topics!forms_api_reference.html/7#ajax) callback function in Drupal 7 you only return the part of the form that you want to replace.
Comment: Be careful with your wording. The callback should return THE WHOLE FORM, rebuilt from the form_state. Returning PART of the form is the biggest Drupal/AHAH misconception. See http://katbailey.net/blog/ahah-drupal-may-it-one-day-live-its-acronym.
Comment: @ChrisD. As Clive mentioned, This is Drupal 7 and we have quite a nice ajax framework now that allows to do all kinds of fancy stuff like returning multiple separate form parts and so on.
Comment: @Clive I came across another ajax related issue http://drupal.stackexchange.com/a/142316/19205 where author mentioned "form needs to be build again after items_count is changed" which is also for d7. I am got confused with, which statement is right(without arguing who is right:-) ). Could you please share some info on this?
Answer: There are two ways
1)
function abc_form_validate(&$form, &$form_state) {
  // Validation logic - invalidate using form_set_error() or form_error()
}

function abc_form_ajax_handler(&$form, &$form_state) {
  return $form;
}

2)
     Example module
  http://cgit.drupalcode.org/examples/tree/ajax_example/ajax_example.module?id=refs/heads;id2=7.x-1.x#l402
Comment: I don't now how accurate your answer is, but the formatting of it definitely needs improvement (if you don't want it to be deleted by moderators). Check the help page if needed, or just use some of the wysiwyg editor buttons to at least mark your code as ... code.
Comment: Non correctly formatted code is not a reason to delete an answer. It just requires a user with edit privilege to fix it.
Comment: Here I come, desperate, looking for a solution, merely 2 years after this answer, clicked a link which leads me to a non-existing tree/branch.

Please don't use links on the answers, or if you do paste the content at the time of writing.
Answer: In my case, using a submit type caused the form to always submit, so I changed it to a button with #ajax specified. Then, I had to do my validation in the ajax callback.
I was making a ctools pane; I don't know if that plays into it.
function mymodule_form($form, &$form_state) {
  ...
  $form['button'] = array(
    '#type' => 'button',
    '#value' => t('Subscribe'),
    '#ajax' => array(
      'callback' => '_mymodule_ajax',
    ),
  );
  return $form;
}

function _mymodule_ajax($form, &$form_state) {
  if ( ! valid_email_address($form_state['values']['email']) ) {
    form_set_error($form, t('Please enter a valid email address.'));
  } else {
    $form = array(
      '#markup' => t('You have subscribed.'),
    );
  }
  return $form;
}

Answer: The examples module has an ajax form that you can refer to as well as many others. Here is a link to the ajax form example in the code repository, but I suggest downloading to look at the implementation yourself anyway.
Comment: How is this the accepted answer? It doesn't even say where to look in the link provided (and the link doesn't have validation example anyway).
Comment: sorry - I cant find any examples of ajaxy validataion in the example module
Comment: Yeah, that's a bloated answer right there. I actually don't see any validation there either. Each callback simply returns the form.
Comment: New link for the ajax form example: http://cgit.drupalcode.org/examples/tree/ajax_example/ajax_example.module?h=7.x-1.x
Comment: I made my answer a wiki, since it's accepted but poor quality


Monitor memory usage from process beginning to end

Question: I would like to monitor a processes' CPU and Memory usage (usage, heap allocations, locations, etc...) from the time the process begins until it exits.
For example:

monitorProcess.exe myProcess.exe --myProcessParameter 123

Are there any tools (preferably open source) that perform this?
Alternatively, what key API functions should I research to do this myself?
I'm not looking for Process Explorer or something like that. I want to attach it to a specific process and monitor only that one.
Comment: Why the immediate downvote with no explanation? I've googled various formulations of this question and I run into "process explorer" after "task manager" after "top" etc...
Comment: I can only guess, but probably because you haven't done much to define exactly what about the process you want to monitor, whether you want to display it real-time, or just collect a summary when it exits, etc. You probably also need to specify the operating system you want to target.
Answer: Valgrind has a tool for this, it is called Massif.


PHP: $_GET['url'] does not fetch the value from URL after Cleaning the URL with .htaccess and directing to index.php

Question: I have a problem not being able to access other parts of the website except the index page. I used htaccess to redirect all requests to the index.php but i get the error: Notice: Undefined index: url in /home4/amberrec/public_html/portal/app/core/app.php on line 52
This is my .htaccess code.
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond ${REQUEST_FILENAME} !-d
  RewriteRule ^(.*)$ /index.php?url=$1 [NC,L,QSA]
</IfModule>

This is my app.php code
class App
{
    protected $controller = 'home';
    protected $method = 'index';
    protected $params = [];

    function __construct()
    {
        // get the split url
        $url = $this->split_url(); // url[0]=controller, url[1]=method, the rest is params

        // create the path to the controllers folder
        $path_to_controller = __DIR__ . '../../controllers/';

        // check if the url[0] which is the controller exists/
        if (file_exists($path_to_controller . strtolower($url[0] . ".php")))
        {
            // set the controller if it exist
            $this->controller = strtolower($url[0]);

            //unset it
            unset($url[0]);
        }

        // require the class and create a new instance
        require $path_to_controller . $this->controller . '.php';
        $this->controller = new $this->controller();

        // check if the url[1] which is the controller's method exists/
        if (isset($url[1]))
        {
            if (method_exists($this->controller, $url[1]))
            {
                $this->method = $url[1];
                unset($url[1]);
            }
        }

        // store the rest of the params if available
        $this->params = array_values($url);

        // call the method of the contoller
        call_user_func_array([$this->controller, $this->method], $this->params);
    }

    // Gets the url link and splits into an array
    function split_url() : array
    {
        return explode('/', filter_var(trim($url,"/"), FILTER_SANITIZE_URL));
    }

    // test to get controller name
    function get_controller_name()
    {
        return $this->controller;
    }

    function showURI()
    {
        echo $this->url;
    }
}

This is my index.php
session_start();

require __DIR__ . '/app/init.php';

$app = new App();

This is my config code
/*root and asset paths*/

$path = str_replace("\\", "/",PROTOCAL ."://" . $_SERVER['SERVER_NAME'] . __DIR__  . "/");
$path = str_replace($_SERVER['DOCUMENT_ROOT'], "", $path);

define('ROOT', trim(str_replace("app/core", '', $path), '/'));
define('ASSETS', str_replace("app/core", "public/assets", $path));

This is my folder structure

app

controllers
models
views
core

app.php
config.php

public

assets

.htaccess
index.php
Comment: What I'm going to say doesn't solve your problem and I don't know why you're trying to do this, but I tried to build a basic MVC structure for a project, and over time I needed to add more functionality, spending more time with the framework than the project in itself. So unless you really need to do this from scratch, study Laravel which will be much more advantageous for you.
Comment: The error you are quoting does not appear to relate to the code you have posted? There is no `$_GET['url']` in any of the code you posted? "not being able to access other parts of the website except the index page" - by "index page" I assume you are referring to the document root? But it should be the other way round... when you request the doc root then the `url` URL param will not be set (yet you don't get the E_NOTICE for that request). You should be checking that that index/var exists before attempting to use it and defaulting otherwise, so you should never get that E_NOTICE.
Comment: The split_url() function references a non-existent variable $url.
Answer: you dont have $this->url. Try add:
protected $url =[];

then
$url = $this->split_url();
$this->url = $url;

or call the function with $url
showURI($url);
function showURI($url) {
 echo $url[0].$url[1];
 //var_dump($url);
}

IMPORTAN you cant echo of array like
echo $this->url;


Dominant 7th chord on the 4th degree of the minor scale?

Question: This song goes Fm9 - Bb13 - Bbm9 - C7b9(#5)
As far as I can tell this is a 1-4-5 in F minor... but what about the Bb13 ?
Note that this vamps repeats over and over ...the song is pretty fast ..maybe 2 seconds on each chord maximum...
Should it be considered just a passing chord ? What is the theory behind ? Thanks!
Wow , this generated more discussion than I thought ....here is the song :

 ...I`m analysing all the answers you folks kindly shared... Thank you !!
Comment: I don't understand the downvote. This is a perfectly valid question about harmonic theory.
Comment: The downvote(s) could easily be because we aren't given any opportunity to listen to the piece in question - for all we know, the chord names may even be inaccurate! Be nice to know though.
Comment: Chordify has an interesting (and inaccurate) take on the chords...
Answer: For starters, this is definitely an F minor progression. Do not interpret this as Eb in the absence of any Eb chords.
IV7 while in a minor key is a common use of modal interchange. It's borrowed from the dorian mode. It's common because it's a very strong suggestion of dorian (since it contains the nat6) and it is easy to return to the original key (by flatting the sixth again). This also creates a chromatic voice leading starting from the b7th of the scale and descending to the 5th if you then go to V7 or the root (in this case, Eb-D-Db-C).
By extension, it is useful to interpret the BbMi9 as the restatement of the key. You often see Db7#11 in this position, and in that sense it is somewhat unusual to see the ninth (C), but there is no need to overthink this. BbMi9 represents the subdominant of the key and the progression is a typical minor i-iv-V with a quick trip to the parallel dorian.
Answer: It's a kind of modal borrowing--in the context of F minor, Bb13 is arguably borrowed from the tonic major (F major). Think of it as IV13.
Comment: B♭13 isn't diatonic to Fmaj, so how can the chord B♭13 be borrowed from Fmaj?
Comment: @coconochao, the chord you spelled is Bb6, not Bb13. The chord Bb13 implies a flatted 7th (an Ab), whereas the Bb6 doesn't imply any 7th. For more on the conventions of chord symbols, check out this outstanding answer by Dom, one of the moderators: https://music.stackexchange.com/a/20561/40842
Comment: It's more of an incomplete borrowing. Given that the other answers often skip the added tones in the chords (Laurence Payne saying the 2nd chord is V7 instead of V13, your answer saying the 1st chord is Fm6 instead of Fm9), I figured the added tones don't quite count harmonic function-wise and it's the root, third, and fifth that really matter.
Comment: Fm6 & Fm9 are equivalent in way that B♭M7 and B♭7 are not. The reason I emphasized Fm6 is just to point out that the i chord can be thought of as containing a natural 13th. I could change it from Fm6 to Fm69 and my answer would still stand. Adding or removing the 6th or 9th does not change the chord quality. By contrast, removing the flat 7th from B♭13 *does* change the chord quality--significantly. Similarly, in Laurence's answer, adding or removing the 13th doesn't change the chord quality. He's just simplifying, which is great because then his answer applies to any V7 chord (V9, V13, etc.).
Comment: @jdjazz Considering Bb13 contains the notes Bb D F G, how can it not be diatonic to F major?
Comment: @coconochao, no problem! It's a great answer by Dom.
Answer: Overview
There is a good way to interpret the progression so that the B♭13 isn't out of place/being borrowed from a different mode. Think of the first two measures as being in F Dorian and the last two measures as being a iiø-V7alt in F min. The advantage of this approach is that there is more cohesion/continuity between the Fmin and B♭13 chords.
Think of B♭min as Gø7
The B♭min chord is serving a subdominant function: it preps the V7 chord, C7alt. In this way, it is interchangeable with Gø7. In fact, if we replaced B♭m9 with B♭m6 (which wouldn't change the function of the B♭m chord), then now the B♭m6 chord would be voiced B♭-D♭-F-G, which is also the exact spelling of a Gø7 chord. In general, ii-V-i progressions and iv-V-i progressions are both commonly found in music, and are often interchangeable.
Think of Fm9 in terms of F Dorian
Just as we could treat the B♭m9 chord as B♭m6, the Fm chord can be interpreted through many different scales (F melodic minor, F Dorian minor, F Aeolian minor, etc.). If you want the first two measures to be continuous, then you would choose F Dorian minor for measure 1, since F Dorian contains the D♮ from the B♭13 chord. In effect, you'd be treating the Fmin9 chord as Fmin6 (or more technically Fmin6/9). This interpretation/approach would create good cohesion in the first two measures, since both chords would now contain a D♮. When improvising, for example, an Fmin6 pentatonic scale would sound great (F-G-A♭-C-D) and would be seamless when passing from one chord to the next.
So ultimately, the way to think about this progression is as a i-IV7-iiø7-V7alt, with the first two measures using F Dorian and the last two measures being a turnaround:
<code>| Fm6/9 | B♭13 | Gø7 | C7alt |</code>
Side Note
Other interpretations are possible. For example, you could interpret the Fm9 chord as implying F Aeolian (i.e., F natural minor). Doing this would cause the B♭7 chord to stick out more, because it'll be the only chord in the four measures that contains a D♮. One downside of using F Aeolian is that the D♭ will be an avoid note over the Fm9 chord, whereas the D♮ wouldn't. If you use F Dorian, then now the D♭ from C7alt can resolve down a half step to C or up a half step to D♮.
So if you're trying to make the B♭7 chord be more cohesive with the other chords, I recommend the approach I've outlined above. (Of course, if you're comping/playing chords behind a soloist, you want to listen to what the soloist is playing and let that dictate your choices too. If the soloist is playing a D♭ over the Fm9 chord, then avoid a D♮.)
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/78745/discussion-between-jdjazz-and-fugu).
Answer: Seems it might be using F Dorian, Which sounds similar to F minor, but the IV chord becomes Bb major. The fact that it's a 13th chord, with at least a 6th and a m7th doesn't really make a lot of difference. Then maybe it reverts to Fm proper, before going to V on C. The Bb major could be called a borrowed chord from the parallel key of F major, if you want a bit of alternate theory.
Having listened to 'this song', it might make matters different...
Answer: Try hearing it as ii7, V7, v7, VI7 in Eb.  The fact that it isn't wrapped up with Fm7, Bb7, Eb doesn't matter.
Comment: I know for fact the song is in F minor. Even the composor said (it`s sort of a blues) The question still stands...
Comment: It isn't in in Eb at all, I don't see how thinking like this could help.
Comment: It's in Fm,  or more clearly, F Dorian - which is the Dorian of Eb, which will contain the same basic notes. However, since the chords have several extensions, thinking in Eb doesn't help much.
Comment: Thinking of it in E♭ would make sense if it were being used as a turnaround (basically like a ii-V7-iii-VI7 progression). Given that it's a vamp which never resolves to E♭, I think it makes more sense to think of Fm as the tonic, as the composer and poster has said. That way, the progression starts with the i chord and ends with the V7 chord.
Comment: It seems we agree,  F Dorian is a re-rooted Eb major.   If you're looking for a functional analysis,  it might be better to avoid the modal description.


Search and replace 'dh $@' with sed

Question: I want to replace this string
dh $@

with that one
dh $@ --foobar

I tried this
sed --expression "s/dh $@/dh $@ --foobar/" file

But of course there is the blank and the $ makeing "problems".
Answer: Use single quotes around the pattern to avoid shell variable expansion from expanding the $@ to the empty string (or whatever other contents the positional parameters contain.
sed --expression 's/dh $@/dh $@ --foobar/' file

You could also use & in the replacement to avoid needing to duplicate the pattern. (& in the replacement means the entirety of the match.)
sed --expression 's/dh $@/& --foobar/' file

Comment: @HuStmpHrrr You can do that but in this case you don't need to it would seem. The end-of-line meaning of `$` doesn't really make sense in the middle of the pattern and is presumably ignored (at least by the version of gnu sed I'm testing with).
Comment: To clarify (GNU sed): `$` as a metacharacter refers to the end of the pattern space except when the multiline flag is envoked (`/../M or s/.../.../M`) when it refers to the zero space before a `\n` and `\'` refers to end of pattern space.
Comment: i think you mean `\$` instead of `$`.
Comment: @EtanReisner well sed!
Comment: what is the use of `--expression`??? btw `$` means end-of-string, not end-of-line. It's just that in sed you're usually comparing with a string that ends at the end of the line so the meanings get mixed up which can bite you later with other tools or when using sed with a multi-line string in one of those "space" thingies (hold space?).
Comment: @EdMorton `-e`/`--expression`. Often you don't need it, as the first and only non-option argument is implicitly the script, but if you have more than script block or a file *and* a script block or a script block that starts with a `-` then you need it. And yeah, that line/string thing gets confused in sed because of how it works. I should be more careful how I talk about it.
Comment: Ah, `-e` I know, just never came across `--expression` before. Wonder what would inspire anyone to use THAT as an option... Thanks.

Search and replace 'dh $@' with sed


I can't seem to get the XPATH right for XSD UNIQUE Constraint

Question: I've been trying to get the UNIQUE constraint placed on some attributes I have in a fairly basic XSD Schema. I'm using XMLSpy and i'm wanting to put a unique constraint around TEMPLATE.ID for my nodes (i.e. it's important that they remain unique).
I've put the below in place and tried the following:
Secenario 1
<xs:unique name="uniqueviewid">
  <xs:selector xpath="./views"/>
  <xs:field xpath="@id"/>
</xs:unique>

Result:
XSD: Validates Ok. XML Validates but doesn't uphold the unique constraint (ie two or more elements with the same id aren't picked up
Scenario 2
<xs:unique name="uniqueviewid">
  <xs:selector xpath="views"/>
  <xs:field xpath="@id"/>
</xs:unique>

**Results* Same as Scenario 1. XSD Validates, XML Validates but ignores duplicate @id in view element
Scenario 3
<xs:unique name="uniqueviewid">
  <xs:selector xpath="*"/>
  <xs:field xpath="@id"/>
</xs:unique>

Results: XSD Validates and XML Validats and it respects the UNIQUE constrint (ie if duplicate views with @id it throws an invaldation as per design).
Problem is that the wild card is "*" which means all subnodes under VIEWPODS will be validated, which isnot what I want. I instead want the contraint to be focused on VIEWPOS/VIEWS/@ID exact path.
I think my XPATH is all wrong but i can't think of what i'm doing specifically wrong?

The example XML.
This is the example XML.
<config xmlns="http://tempuri.org/RIAGenicConfig.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://tempuri.org/RIAGenicConfig.xsd RIAGenicConfig.xsd">
  <view>
    <viewpods>
      <views id="view1"/>
      <views id="view1"/>
    </viewpods>
  </view>
</config>

The XSD in Question.
<xs:schema xmlns="http://tempuri.org/RIAGenicConfig.xsd" xmlns:mstns="http://tempuri.org/RIAGenicConfig.xsd" xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://tempuri.org/RIAGenicConfig.xsd" elementFormDefault="qualified" id="RIAGenicConfig">
  <xs:element name="config">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="view">
          <xs:complexType>
            <xs:choice>
              <xs:element name="viewpods">
                <xs:complexType>
                  <xs:choice>
                    <xs:element name="views" maxOccurs="unbounded">
                      <xs:complexType>
                        <xs:attribute name="id"/>
                      </xs:complexType>
                    </xs:element>
                    <xs:element name="blah"/>
                  </xs:choice>
                </xs:complexType>
                <xs:unique name="uniqueviewid">
                  <xs:selector xpath="*"/>
                  <xs:field xpath="@id"/>
                </xs:unique>
              </xs:element>
            </xs:choice>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

Answer: Qualify the xpath selection for mstns:views instead of *(all elements):
<xs:unique name="uniqueviewid">
  <xs:selector xpath="mstns:views"/>
   <xs:field xpath="@id"/>
</xs:unique>

Comment: Where did the mstns come from?
Comment: From the XSD.  It contains the namespace definition: xmlns:mstns="http://tempuri.org/RIAGenicConfig.xsd"
Comment: Thanks.  My xml schema had set xmlns to be the same as my target xml, which I thought would be enough.  I added a sub-namespace (is that what you'd call it?) to the schema and used that for my xpaths and now everything works. \o/


What kind of deep neural networks are (not) data-intensive?

Question: There are plenty of shapes and tastes of neural networks out there. Just like any machine learning model they require as much data you can get to deliver good performance, but it seems that some models are more data-intensive than others. I looked for an overview of which models require more/less data to work well, but failed to find anything. I was also wondering what kind of tricks can be used to make neural networks less data-intensive.
The things I learnt thus far from different sources:

Deep Belief Networks require less labelled data, because pre-training can be done with unlabelled data, which is often easier to get
Sigmoid activations suffer from the vanishing gradient problem and therefore require more data than newer activation functions like tanh, ReLU and ELU
Recurrent Neural Networks require more data than other models

Can these statements be confirmed? Does anybody have a good reference to an overview or the knowledge to provide one?
Comment: This is a very general question and will probably not get answer anytime soon for a number of reasons. While the three things you mentioned in your question are generally correct, model architecture is the MOST important factor in its data-intensivity.
Comment: The deeper the model, the more complex the optimization surface. That'll affect amount of data needed. Also, the higher the number of weight/paramters to tune, the higher the number of samples. This is similar to the previous point though.
Comment: You'd probably have a better chance of getting an informative answer if you breakdown your question to smaller more specific (model specific) ones.
Answer: If you take a careful look at the recent state-of-the-art results on some standard problems (language models, computer vision), you'll notice that the number of parameters in a network scales with the number of examples in the training data. (This is not the main thesis of this paper, but it is discussed in a few paragraphs of "mixup: Beyond Empirical Risk Minimization" by Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Paz.) Stated another way, you can model a small data set with a small model.
Or, looking at the problem from the other end, for any network which achieves state-of-the-art results for some problem, you could just remove any number of parameters and accept the drop in performance as the cost if training a smaller network.


Managing FCM device groups server side

Question: I am looking for a way to manage FCM device groups server side, hopefully with the use of the node.js firebase-admin library. As far as I can see the only way of achieving this is by using the old GCM based rest-endpoint POST: https://android.googleapis.com/gcm/notification
However I do not feel confident using that as using the gcm system is discouraged.
I have searched through the github repo and cound not seem to find anything on Device Groups anywhere. Is this feature something which isn't supposed to be used anymore? Or is it perhaps not documented, or been overlooked, for some reason?
I am aware that creating device groups is achievable on the client side, but it does seem to require the user to have a google account, which unfortunately in my case I cannot assume.
From https://firebase.google.com/docs/cloud-messaging/android/device-group#managing_device_groups

To create a device group on the client, the device must have at least one Google account.
Answer: I have been in contact with Google about this. They say that using the GCM api is the current preferred way. For my project I made a simple server-side wrapper library around their api so I could make things a bit easier for myself.

EDIT:
FCM has now added support for device groups; https://firebase.google.com/docs/cloud-messaging/android/device-group

Managing FCM device groups server side


Realized volatility vs close to close return and open to close return

Question: I am calculating daily volatility in 3 ways:

Realized variance=> sum of square of 5 minute returns for each trading day(from 09:30 to 16:00)
Close to close return=>(ln(close price at day i)-ln(close price at day i-1))^2
Open to close return=>(ln(close price at day i)-ln(open price at day i))^2

There are almost 2000 days at the data.
Mean of square root of above calculations are so different from each other:

volatility related realizations(5 min) is 0.0153
volatility related to close to close price is 0.0125
volatility related to open to close price is 0.0105

Whay are day so different from each other? Is it natural? Or do I have a mistake?
I will be very glad for any help.
Comment: Somewhat related: Molnar ["Properties of range-based volatility estimators"](https://www.sciencedirect.com/science/article/abs/pii/S1057521911000731). Or maybe not so relevant...
Comment: The first one is realized variance, not realized volatility.

The third one doesn't include volatility from opening gaps compared to the second. Part of the variance in the price of something is due only to gaps.
Comment: @javierazcoiti, why do you say first one isn't realized volatility? Isn't  it an estimate of it? I realize the diffrence between 2nd one and 3rd one. But I guess, 1st one shpuld close to 3rd one since, both include the trading time interval. I am very confused about the situation. Is one of them biased? Thansk.
Comment: The realized volatility is the squared root of the realized variance.
Comment: @javiaerazcoiti, I editid the content. Thanks.
Comment: @RichardHardy , thank your for the source recommendation.  Beside that source, https://www.researchgate.net/publication/46434088_Measuring_volatility_with_the_realized_range is also very related to my problem. I think, I will able to understand the problem with that source.
Answer:
...I am calculating daily volatility in 3 ways...

No, you are not computing the same quantity in 3 different ways.
You are computing volatility over different frequencies.
The log daily returns and log-5 minute returns need not have the same volatility. Similarly, you could also compute, say, quarterly returns or 10 second returns and get different volatility series.
Take a hypothetical scenario where the open price and close price are the same but there is intra-day price movement due to trading. Then that date contributes zero to daily volatility.
On the other hand, the realized volatility computed from a high-frequency intra-day return series would be non-zero.
These are volatility measures from the perspectives of investors with different holding periods and trading horizons. They are not the same.
Indeed, traders who hold their positions for 5 minutes and for 5 days face different types of risk.
The open-to-close and close-to-close measures should also not be the same.
They are not even the same returns. The close-to-close return accounts for off-hour, and then next day, trading.
Comment: @Micheal thanks for the explanation. You are right. They are different calculatons. But I was thinking that at the long run, they shoud close to each other. All are different ways of estimating unobserved daily volatility. Just the time interval included at the calculations may create biases. But for the 1st and 3rd ones, the time intervals are same.
Comment: "...the long run, they should close..."---that depends on different aspects of the instrument, e.g. nature of the instrument, liquidity, type of traders involved, etc. Your data also may not cover "long run", whatever that means in this specific case.
Comment: Also, you're implying that in the long run, traders who hold their positions for 5 minutes and for 3 months ("long run") face the same risk. That is doubtful.
Comment: @Micheal, I guess 2000 days enough for long run definition. But, I am not sure, I may be wrong. Ok. I wonder that, let I am forecasting one head day volatility everyday using previous open to close prices using a model. To compare the performance of the model; I compare the model forecasts and the realized obsered volatility for the day. Is an adaquate way to compare the model?. Because, I think realized volatility should be the best estimate for the unobserved volatility of the day.
Comment: "Is an adequate way to compare the model?" That is the general procedure to judge predictive models, not just for vol. Compute the MSFE --- (forecast - realized)^2.
Comment: @Micheal, I am confused with that: My model uses open to close returns for one head forecast. One day later, I can observe both 5 mins realized volatility or open to close volalitily. So, I have two volatility values(proxies) for comparing with the forecasted one. Which one should I use? 5 mins vol or open-close vol(where the model is constructed)?
Comment: As already discussed, 5 mins realized volatility and open to close volatility are not realizations of the same data generating process. They do not follow the same model. If your model input is open to close return series, then its vol forecast is a forecast of open to close vol.


Add Migration not working in Visual Studio 2017

Question: I am creating a Web API application using EF with a code-first approach in Visual Studio 2017. I tried to use enable-migration, initially I was not working so I installed these packages.
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 2.2.3
Install-Package Microsoft.EntityFrameworkCore.SqlServer

After installing those packages, enable-migration worked for me, but when I tried to add a migration using Add-Migration FirstMigration command, it throws an error

No DbContext was found in assembly 'UserRegistration'. Ensure that you're using the correct assembly and that the type is neither abstract nor generic.

Please help me how to resolve this issue in Visual Studio 2017.

PS : I have also defined the DBbConnection in my web.config file:
<connectionString>
    <add name="DefaultConnection"
         connectionString="Data Source=(local);Initial Catalog=WebAPIDB;Integrated Security=True"
         providerName="System.Data.SqlClient"></add>
</connectionString>

Answer:     public DbSet<ModelClassName> NameForModel {get; set;}

Please include the code above in your model class that has properties you want to migrate in the database
please refer to this link How to apply Entity Framework code first approach to a project to make it easier for you to understand.


accessing a json dictionary of an object from list of objects

Question: I am new to django and I am working on project where I am storing a key value pair dictionary as JSON in database.
Now later I want to show it on the html page all the list of packages but not able to access those key value pair as dictionary.
here is my
models.py
class Packages(models.Model):
    package_ID = models.AutoField("Package ID", primary_key=True)
    package_Name = models.CharField("Package Name", max_length=30, null=False)
    attribute_values = models.CharField("Item Details JSON", max_length=500, null=False)
    package_Price = models.IntegerField("Package Price", null=False, default=0.00)
    quantity = models.IntegerField("Quantity", null=False, default=00)
    prod_ID = models.ForeignKey(Product, on_delete=models.CASCADE, verbose_name="Product ID (FK)")

its data entry is something like this
package_ID = 1
package_Name = "basic card"
attribute_values = {"sizes": "8.5 in. x 11 in.", "Colour": "Full-Color Front - Unprinted Back",}
package_Price = 200
quantity = 400
prod_ID = 1

package_ID = 2
package_Name = "best card"
attribute_values = {"sizes": "8.5 in. x 11 in.", "Colour": "Full-Color Front - Unprinted Back",}
package_Price = 200
quantity = 500
prod_ID = 1

here the problem is that attribute_values fields stores a JSON string so I have to convert it in dictionary first and then have to access its dictionary.
the steps I want to do:

get all the packages having same product key:
now get the attribute_values of those packages
and convert that JSON into dictionary
then display the attribute_values of each package separately in html in particular block.

I want output something like this:

but I am getting this:
Comment: What version of Django do you use? Try upgrading to 3.1 or greater if lower than that (you can use JSONField in those versions), or if your database is PostgreSQL you can use it's JSONField.
Comment: I am using Django 3.1 but I am a newbie so I don't  have musch idea how to use it can you please give an example of this
Answer: With Django version >=3.1 one can simply use JSONField for storing JSON in databases.
It is supported for the following databases:

JSONField is supported on MariaDB 10.2.7+, MySQL 5.7.8+, Oracle,
PostgreSQL, and SQLite 3.9.0+ (with the JSON1 extension
enabled).

Using the JSONField should be pretty simple:
class Packages(models.Model):
    # other fields
    attribute_values = models.JSONField()

Let's say we have an instance of Packages we can simply set a dictionary / variable that can be valid JSON on to attribute_values:
package.attribute_values = {"sizes": "8.5 in. x 11 in.", "Colour": "Full-Color Front - Unprinted Back",}
package.save()

This will be stored as JSON in the database.
When we want to access this in python it would automatically be deserialized to their python value:
package.attribute_values['new_key'] = 'new value'
package.save()

Considering your current problem with the text field you need to simply convert your string to equivalent python objects. This can be done using json.loads:
import json

attribute_values_dictionary = json.loads(package.attribute_values)

You can also add a method to your model to do this for you:
import json

class Packages(models.Model):
    # Your fields

    def attribute_values_to_python(self):
        return json.loads(self.attribute_values)

Now in the view you can simply write:
attribute_values_dictionary = package.attribute_values_to_python()

In the template you can simply write:
{{ package.attribute_values_to_python }}


Retrieval of image from different storage using WHILE in PHP

Question: Goodday! I just want to know if there is a possible trick for this kind of problem. I want to fetch the image of my rows in php, i have 3 categories hotel, restaurants and Places To Go and each category has different storage(folder) i already fetch the other necessary data but the image is really hard for me to do that can anyone help me to fix this problem? Thank you in advance :)
//Mysql statement to get the data
$query = "

SELECT hotel as name, province, address as address, image as image FROM hotel
        WHERE CONCAT(hotel,province,address) LIKE '%".$valueToSearch."%'

       UNION ALL
      SELECT tourist_spot as name, province, address as address, image as image FROM users
        WHERE CONCAT(tourist_spot,province,address) LIKE '%".$valueToSearch."%'

       UNION ALL
        SELECT restau_name as name, province, restau_address as address, image as image FROM restauu
       WHERE CONCAT(restau_name,province,restau_address) LIKE '%".$valueToSearch."%'

";

<?php while($row = mysqli_fetch_array($search_result)):?>

 <?php

if($valueToSearch = $row['hotel']){
echo "";
}else if($valueToSearch = $row['tourist_spot']){
echo "";
 }else if($valueToSearch = $row['restau_name']){
 echo "";
}

  ?>

<?php echo $row['name'];?>

Located at  <?php echo $row['province']; ?>

<?php echo "
View Details
"; ?>

</form>

<?php endwhile;?>
Answer: You have issue in if condition use == for comparison = is assignment operator
if($valueToSearch == $row['hotel']){
 echo "";
}else if($valueToSearch == $row['tourist_spot']){
 echo "";
 }else if($valueToSearch == $row['restau_name']){
 echo "";
}


How to make a function which counts for 3 seconds

Question: I want to make a programm which counts how many computational steps the pc can make in 3 seconds it should work basicly like this:
var i = 0;
function count(){
  while(the3seconds are not over)
  {
    i++;
    div.innerHTML = i;
  }
}

Problem is ONE doesnt matter how I trie it does not stop after 3 seconds and TWO because its in a loop the variable i is not displyed while the process.
How to solve this two things?
EDIT:

var rechenschritte = 0;
var div = document.getElementById("rechenschritte");
function count(duration) {
    var end = new Date().getTime() + duration;

    while(new Date().getTime() < end) {
      rechenschritte++;
    }
    div.innerHTML = rechenschritte;
    rechenschritte = 0;
}
<!doctype html>

      <meta charset="UTF-8">

<input type="button" onclick="count(500)" value="Start"/>

How to see it counting?
Comment: Show us the code you have so far ....
Answer: If you want to have a blocking function than:
var end = new Date().getTime() + 3000;
while(new Date().getTime() < end) {}

FIDDLE #1
But this will block the whole UI and if you want an animation »loop«, than:
  (function loop (end) {

      //do stuff
      if (new Date().getTime() < end) {
          window.requestAnimationFrame(function(){
              loop(end);
          });
      }

  })(new Date().getTime() + 3000);

FIDDLE #2
If you compare the output of both versions, you can see that #1 will
start ...
many times sleeping
end ...

and #2
start...
1000
end
many times the delta

This is want blocking is actually referring to. The while loop will run with as much cycles as it can and blocks the thread, while #2 uses a timer, which will call loop as often as possible, but with a maximum of 60 times per second.
UPDATE
Referring to the comment: »How to see it counting« If you are using method #1, than you wont ever see it, because: The while loop will consume all available CPU cycles and runs until time is up. During that it updates the inner HTML, but since this loop block the UI, the Browser has no chance to actually render the result and you will end up with the result rendered. In other words: You simply cannot update the UI as fast as your CPU.
To see anything you need to use method #2, this way the UI is not blocked, because requestAnimationFrame asks the browser to run the callback as often as possible in between the render cycles of the browser, that is why there is a maximum of 60 Frames per Second, because this is the maximum update rate of the browser. If your callback takes longer than 1000/60ms the frame rate decreases accordingly.
If you want to display how many cycles the browser can run in a Interval, you need a different approach, which could be a Web Worker. With that, you can create a second thread, which wont interfere with the UI. Therein you can count the iterations and within the »UI« thread, you can pull that value from the worker and display it each animation Frame.
Please not that JS is single threaded and while this thread is processing anything, the UI hangs and wont be updated until the window »unresponsive script…« pop up and asks you to stop it.
With the code below you might get a count close to the »real« count of iterations. It counts for a fixed frame length, updates the html and than let the browser do the rendering, while counting the cycles is paused. So that will count less cycles than browser actually can.
function displayCount(duration){
    var end = new Date().getTime() + duration,
        cycles = 0,
        frame = 1000/30; //30fps

    (function loop(){
        var now = new Date().getTime(),
            countend = now + frame;

        while(new Date().getTime() < countend) {
           cycles++;
        }

        //UPDATE HTML here

        if(now < end) {
            window.requestAnimationFrame(loop);
        }

    })();
}

FIDDLE #3
Comment: thanks for that my code look now like this MY FIRST POST but how to solve problem TWO that you see it counting?
Comment: thanks for the update but i dont get it right i think. With that what your code. You mean it just works with 30fps or so dont get that? ;/
Comment: Yes I mean fps. The code above is a combination of #1 and #2. In principle it counts the iteration of the while loop for 1000/30ms and then prints that value. After the browser did its duties (rendering), than calls `loop()` again, this process is repeated for `duration`. The counting of the iteration is paused, to give the browser time to render.


MySQL - select first 10 bytes of a string

Question: Hello wise men & women,
How would you select the first x bytes of a string?
The use case: I'm optimizing product description texts for upload to Amazon, and Amazon measures field lengths by bytes in utf8 (not latin1 as I stated earlier), not by characters. MySQL on the other hand, seems to operate character-based. (e.g., the function left() is character-based, not byte-based). The difference (using English, French, Spanish & German) is roughly 10%, but it can vary widely.
Some tests concerning a field with #bytes < 250 (details: http://wiki.devliegendebrigade.nl/Format_inventarisbestanden_(Amazon)#Veldlengte):
OK, char_length: 248,   byte length latin1: 248,   byte length utf8: 248
OK, char_length: 249,   byte length latin1: 249,   byte length utf8: 249
OK, char_length: 249,   byte length latin1: 249,   byte length utf8: 249
OK, char_length: 249,   byte length latin1: 249,   byte length utf8: 249

Not OK, char_length: 250,   byte length latin1: 250,   byte length utf8: 250
Not OK, char_length: 249,   byte length latin1: 249,   byte length utf8: 252
Not OK, char_length: 248,   byte length latin1: 248,   byte length utf8: 252
Not OK, char_length: 249,   byte length latin1: 249,   byte length utf8: 252
Not OK, char_length: 249,   byte length latin1: 249,   byte length utf8: 257

Illustration:
set @tekst="Jantje zag € pruimen hangen";

select
   char_length(@tekst),   # 27 characters
   length(@tekst);        # 29 bytes

select left(@tekst, 15)   # Result: "Jantje zag € pr"

# Ideally, I'm looking for something like this:

select left_bytes_utf8(@tekst, 15)   # Result: "Jantje zag € "

One approach would maybe be through a sproc that iteratively calls itself, but I suspect there are more efficient solutions around.
Thanks already & regards, Jeroen
P.s.: Edited the question: Changed 2x "latin1" to "utf8". It's actually a bit more confusing: Uploads should be in Latin1, but field sizes are measured in bytes using utf8
P.p.s: Update: These uploads are for English, French, Spanish & German Amazon-sites. Characters don't get more exotic than 'ø' (diameter), '€', 'è', 'é', 'ü' and 'ö'. All within Latin1-encoding, but multibyte in utf8.
Comment: Why not then have latin1 encoding on the relevant column yourself ([docs](https://dev.mysql.com/doc/refman/8.0/en/charset-column.html))? Then you'll be byte-based too.
Comment: Functions like left() or substring() would still be character-based, and the problem would persist - Or I don't understand what you mean
Comment: Oh wait, you want 15 bytes _of UTF-8_? Your Amazon story threw me off :P
Comment: Doesn't really matter which character encoding. Just choose any
Comment: Well, 15 bytes of latin1 is exactly the same as 15 characters of latin1
Comment: Anyway, this should work as long as you don't cut a character: `SELECT CONVERT(LEFT(CONVERT(@tekst USING binary), 15) USING utf8);`. If you cut a character, (like if you take 13 bytes here instead of 15), you get NULL, because MySQL refuses to give you an invalid string.
Comment: You're right! Let's see how to fix the question...
Comment: @Amadan: Your comment from Jul 25 11:54 is the answer! If you post it as such, I can accept it as such. In any case: Thanks a lot!
Answer: SELECT CONVERT(LEFT(CONVERT(@tekst USING binary), 15) USING utf8);

will give you the UTF-8 string cut down to 15 bytes as long as it is still a valid UTF-8 string (MySQL will refuse to give you an invalid string, for example if you cut on a multibyte character, and give you NULL instead.) If that doesn't work, you can get the raw bytes by omitting that last re-conversion to UTF-8, but you will have to decode them to something useful yourself:
SELECT LEFT(CONVERT(@tekst USING binary), 15);

However, Rick James gives a lot of good advice; though only you can judge the degree it is relevant to you, and your particular situation.
Answer:
How would you select the first x bytes of a string?

Is that really what you want to do?  That could (as already pointed out) mangle the string by splitting a multi-byte character up into garbage.

Amazon calculates field lengths by bytes

Please provide evidence to this effect.

The difference is roughly 10%, but it can vary widely.

The max can be a factor of 4.  Emoji and certain Chinese characters need 4 bytes for UTF-8 (utf8mb4) encoding.
If Amazon is encoding things in latin1 (which is not the same as "by bytes"), then first you need to check whether the string can be encoded in latin1.  Western European text can be, but Asian text cannot be.  Sure, you can get "bytes", that that leads to mangled text, especially if you truncate to some byte, but not character, boundary.
SELECT CONVERT(CONVERT(@tekst USING latin1) USING utf8) = @tekst;

Will return 1 (true) if the conversion will work.
Then you can use CONVERT(@tekst USING latin1) with LEFT(..., 10) or whatever.
Better?
If Amazon is effectively using latin1, then you use latin1.  That is, declare your string:
 for_amazon VARCHAR(10) CHARACTER SET latin1

and/or connect with SET NAMES latin1
or you could have a bigger field, then do LEFT(..., 10)
Either will provide the conversion (before storing versus while fetching) so that the bytes you provide to Amazon will be latin1.
Caveat: If you store Chinese (or Russian or Greek, etc) in the column, it will be messed up.
Comment: Thank you! I clarified the question accordingly
Answer: Thank you @Amadan & @Rick James! Thanks to your input, I was able to come up with a multibyte-safe byte-wise left function:
CREATE DEFINER=`root`@`localhost` FUNCTION `left_byte`(
    input_string text,
    input_position integer
) RETURNS text CHARSET utf8
BEGIN

# Byte-wise left function
################################################################################
#
# * multibyte-safe for characters of up to 4 bytes (=max # bytes utf8)
# * utf8 Assumed to be the general encoding

return
ifnull
(
    ifnull
    (
        ifnull
        (
            convert(left(convert(input_string using binary), input_position) using utf8),
            convert(left(convert(input_string using binary), input_position-1) using utf8)
        ),
        convert(left(convert(input_string using binary), input_position-2) using utf8)
    ),
    convert(left(convert(input_string using binary), input_position-3) using utf8)
);
END


Esconder div após teclado ser aberto. HTML

Question: Boa noite!
Estou desenvolvendo um app utilizando Ionic / Angular, mas estou com um problema.
Ao clicar no teclado, os elementos do formulário, sobrescrevem o menu e o logotipo.
Desejo ocultar as divs responsáveis pela exibição do menu e do logotipo, estou usando no html:

Porém só está ocultando as divs, ao digitar algo usando o teclado e não ao clicar no input como era esperado.
Agradeço a atenção.
Comment: Quando abre o teclado a div não é escondida? Só quando começa a digitar algo?
Comment: Exatamente sam! O esperado era ao clicar, já esconder.
Comment: tenta setar uma boolean pra true no focus do input
Answer: seta um ngmodel nos seus forms field #formfield

Algo nessa linha de quando os formfield forem focados, sua div não aparece.
Comment: SEU CONTAINER
Comment: @RenanMartins na verdade responde sim, eu como usuário de Angular e de Ionic digo que é este um provavel caminho, claro que ele poderia ter escrito o que colocou no comentário, mas é provavel falta de experiencia em responder perguntas.
Comment: @GuilhermeNascimento Ok


How to sort a data table items base on ID in C#?

Question: I am new to C# and SQL and I am using windows forms.

As shown in the screenshot I am saving items to SQL table and then I read them to create buttons in flowLayoutPanel. The following code works fine but the problem is : when I read the items and add them to the flowLayoutPanel, they are added as they are in DataTable (unsorted).
Anyone knows how to add the buttons (Items) to flowLayoutPanel and re-order them based on their ID ? I mean I want to add them to flowLayoutPanel as : Item1 ,Item2, Item3, Item4 .
Thank you.
    SqlConnection MyConnection = new SqlConnection(ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString);
    SqlCommand MyCommand = new SqlCommand();
    DataTable DataTable = new DataTable();
    SqlDataAdapter Sql_Data_Adapter = new SqlDataAdapter();

                DataTable.Rows.Clear();
                DataTable.Columns.Clear();
                MyConnection.Open();
                MyCommand.CommandText = "SELECT * FROM Table_Items";
                MyCommand.Connection = MyConnection;
                Sql_Data_Adapter.SelectCommand = MyCommand;
                Sql_Data_Adapter.Fill(DataTable);

                MyCommand.Parameters.Clear();
                Sql_Data_Adapter.Dispose();
                MyConnection.Close();

                int RowIndex = DataTable.Rows.Count - 1;

                for (int i = 0; i <= RowIndex; i++)
                {

                    // create buttons

                    Button btn = new Button();

                    btn.Text = DataTable.Rows[i]"Item_Name"].ToString();
                    btn.Name = DataTable.Rows[i]["Item_Name"].ToString();

                    flowLayoutPanel1.Controls.Add(btn);

                }

Comment: The easiest way is probably: `SELECT * FROM Table_Items ORDER BY ID;`
Comment: Change query to `SELECT * FROM Table_Items ORDER BY Item_Name`
Comment: Yeah or as @Crowcoder said, sort in the DB query.
Comment: @ Crowcoder. worked like magic :) . thank you
Comment: @Chetan Ranpariya . thank you too.
Comment: Possible duplicate of [Sorting rows in a data table](https://stackoverflow.com/questions/9107916/sorting-rows-in-a-data-table)
Answer: Change your SQL Query to -
SELECT * FROM Table_Items ORDER BY Id


Funcionamento @classmethod

Question: Gostaria de entender melhor a anotação @classmethod do Python.
Resumidamente funciona como uma segunda alternativa para construtor. Porém, quais os benefícios? Quando utilizar?
@classmethod
def from_teste(cls, teste):

Qual o funcionamento do cls ?
Comment: "funciona como uma segunda alternativa para construtor" como assim? Métodos de classe (e métodos estáticos) não têm relação nenhuma com construtores, que eu saiba... (*um* método de classe - o `__new__` - tem envolvimento com a criação de objetos, mas o resto não tem nada a ver) P.S. Acabei de perceber que algumas pessoas *usam* métodos de classe como construtores alternativos (métodos de "fábrica", na verdade), e isto está ok, só que o conceito é bem mais amplo que isso.
Answer: Por padrão todo método criado em uma classe pertence ao objeto. Isso significa que é necessária uma instância do objeto para se chamar um método, e essa instância é normalmente associada ao primeiro parâmetro da função (que convenciona-se chamar de self). Mesmo o método __init__, chamado durante a construção do objeto, já assume que o objeto existe e está acessível através do parâmetro self.
class Foo(object):
    def __init__(self):
        pass # self é o objeto sendo criado
    def bar(self, a, b, d):
        pass # self é o objeto

foo = Foo()
foo.bar(1,2,3) # implicitamente passa "foo" como 1º argumento para "bar"
fn = foo.bar   # fn aqui é um bound method, ligado a "foo"
fn(1,2,3)      # Ainda passa "foo" como 1º argumento para "bar"

Para se criar métodos que pertençam à classe, e não ao objeto, pode-se usar os decoradores @staticmethod ou @classmethod. Ambos são semelhantes, embora o primeiro não faça referência alguma à classe e o segundo o faça (de forma semelhante ao método comum, embora nesse caso convenciona-se chamar de cls em vez de self).
class Foo(object):
    @staticmethod
    def bar(a, b, c):
        pass
    @classmethod
    def baz(cls, a, b, c):
        pass

Foo.bar(1,2,3) # Os argumentos são 1, 2 e 3
Foo.baz(1,2,3) # Os argumentos são Foo, 1, 2 e 3

x = Foo() # Também se pode chamar métodos de classe a partir de uma instância
x.bar(1,2,3) # Os argumentos são 1, 2 e 3
x.baz(1,2,3) # Os argumentos são Foo, 1, 2 e 3

A principal diferença entre @classmethod e @staticmethod é que o primeiro pode ser herdado de forma a fazer alguma coisa útil. Pois embora ambos sejam "herdáveis", o primeiro pode levar a classe em consideração ao exercer sua função, e o segundo não (pois ele não tem referência para a classe sendo usada).
class Foo(object):
    @staticmethod
    def bar():
        print Foo   # Não há referência à classe, então só se pode acessá-la pelo nome
    @classmethod
    def baz(cls):
        pprint cls  # Há referência explícita à classe

class Bar(Foo):
    pass

Bar.bar() # imprime Foo
Bar.baz() # imprime Bar

Um método de classe/estático pode ser usado para qualquer coisa, mas faz mais sentido usá-lo para coisas que têm relação com a classe sendo definida (do contrário, por que colocá-lo numa classe, e não diretamente no módulo?). Uma possibilidade é usá-lo como fábrica, customizando a criação de objetos não necessariamente da mesma classe (por exemplo usando subclasses quando apropriado), ou mesmo quando a classe é a mesma, variando os parâmetros recebidos. Mas há muitas outras. Pessoalmente, não me recordo de situações na qual um método de classe me foi útil [em Python], embora lembre vagamente de já tê-lo utilizado.

Funcionamento @classmethod


How to walk through disassembly from the beginning on Linux?

Question:    │0x565554fd                        lea    0x4(%esp),%ecx                      │
   │0x56555501                      and    $0xfffffff0,%esp                    │
   │0x56555504                      pushl  -0x4(%ecx)                          │
   │0x56555507                     push   %ebp                                │
   │0x56555508                     mov    %esp,%ebp                           │
   │0x5655550a                     push   %ebx                                │
   │0x5655550b                     push   %ecx                                │
   │0x5655550c                     call   0x56555529 <__x86.get_pc_thunk.ax>  │
   │0x56555511                     add    $0x1acb,%eax                        │
B+>│0x56555516                     mov    %eax,%ebx                           │
   │0x56555518                     call   0x5655552d <swap>                   │
   │0x5655551d                     mov    $0x0,%eax                           │
   │0x56555522                     pop    %ecx                                │
   │0x56555523                     pop    %ebx                                │
   │0x56555524                     pop    %ebp                                │
   │0x56555525                     lea    -0x4(%ecx),%esp                     │
   │0x56555528                     ret                                        │
   │0x56555529 <__x86.get_pc_thunk.ax>      mov    (%esp),%eax                         │
   │0x5655552c <__x86.get_pc_thunk.ax+3>    ret                                        │
   │0x5655552d <swap>                       push   %ebp                                │
   │0x5655552e <swap+1>                     mov    %esp,%ebp                           │
   │0x56555530 <swap+3>                     sub    $0x10,%esp                          │
   │0x56555533 <swap+6>                     call   0x56555529 <__x86.get_pc_thunk.ax>  │
   │0x56555538 <swap+11>                    add    $0x1aa4,%eax                        │
   │0x5655553d <swap+16>                    lea    0x3c(%eax),%edx                     │
   │0x56555543 <swap+22>                    lea    0x2c(%eax),%ecx                     │
   │0x56555549 <swap+28>                    lea    0x4(%ecx),%ecx

Here is the thing. Plz give me any clue how to use GDB walk through the assembly from the very beginning? Every time I start the program, it goes from the main() function, rather than the beginning of the __libc_start_main().
I would be hugely appreciate if anyone can do me this favour.
Comment: When you break at `__libc_start_main` I believe you use `si` to step the assembler.
Comment: Have you set a break point at `__libc_start_main`?
Comment: that will jump to __libc_start_main, and what I want should be walking through on the routine process
Comment: simple saying, how to start from 0x565554fd on my code snippet?
Answer: Use GDB's starti command to run, but stop before the first user-space instruction (even if that's inside the dynamic linker).  It's like setting a temporary breakpoint on the entry point before run.  With older GDB versions that don't support starti, there are hacks like b *0 to create an invalid breakpoint which you then remove after using run.  (Stopping at the first machine code instruction in GDB)
Or if you only want to start at __libc_start_main instead of your _start or the dynamic linker's _start, set a breakpoint there.
From there you can use si (aka stepi) to single-step by instructions.
You probably want layout reg or layout asm for this.
You might want to build a non-PIE executable (gcc -no-pie -fno-pie), although either way the dynamic linker will execute some code before jumping to your executable's _start.

how to start from 0x565554fd on my code snippet?

That's a totally different question.
That's just the top of main
GDB is partly designed for C debugging, and sometimes likes to save you the trouble of dealing with function prologues.  In this case, 32-bit main's prologue aligns the stack and gets a GOT pointer using PC-relative methods (because it's a PIE executable).
You could just b *0x565554fd and run your program again.
Or with a non-PIE, you can easily get the real address of main before running the first time.
Comment: You can use `b *main` to break at the very first instruction of `main`, rather than having to type out the address.


How to remove all nodes authored by users when their role changes?

Question: I have two roles on my site: Standard and Premium. Standard users are able to upgrade to Premium accounts. When this happens, I would like to deleted all content that the Standard users has authored.
Can this be achieved with Rules? Or, is there a different way?
Answer: You could use the Rules API to define a custom action that gets triggered whenever your users upgrade their accounts. In your custom action you could have a SQL query that searches all that users' content and deletes it.
For how to invoke a Rule programatically see this other DA answer: programmatically-execute-an-action-set-rules-module
Comment: Thanks for the reply amateur barista. I'm more of a front end guy, so this might be a little above my skill set, but I will look into this. Thanks!
Answer: Yes this can be done with the Rules module. Here is a summary of what you'll need to get it to work:

Keep track of all node IDs of a user that still need to be deleted later on (after the user upgrades). For this add an entity reference to the user's profile (multiple values allowed). And each time such a node get's authored (as in your question), you need to add the reference to that node as an extra value in this entity reference field (= Rule 1). So basically this is the "queue" of nodes for a user that need "post processing" after the user upgrades. It is will serve as your entry point starting from the user's id to "access (find) the exact node ids again later on".
Create a rules component (= Rule 2, though of type "component") with parameter just a "node id". All this Rules component has to do, is to "delete" the node indicated in the parameter. But you can add any Rules "Conditions" also if you'd need any (already now or later on if requirements change).
Create a rule (= Rule 3) containing a Loop that should be triggered after the user upgrades. The goal of this rule is to loop over each of the nodes in the reference field from Step 1, and in each iteration of that loop you use an action which is to invoke the Rule Component as detailed in Step 2 (and which uses the node id as a parameter).

Video tutorials
If you're not familiar (enough/yet) with using Loops in rules, then watch the video about List and loops.
For a video that is fairly similar to what's needed in step 2 and step 3, refer to the 2nd video (= Solution video) about Creating multiple back links. Some more details about how your scenario slightly differs from what is shown in that solution video:

You are creating an entity reference to nodes also, but from a user (not a node).
You only need the first parameter (= Node to link to) in your Rules component (you don't need the 2nd parameter).

Easy, no? ... and no custom code involved.


Netsuite Tax Item Script

Question: I have customers that have several ship to addresses in netsuite. My problem is this, each one of those ship to addresses needs a different tax item. So if I were to enter a sales order and select the customer, I need some way that when I select a ship to(from the drop down) the tax item is filled in with the correct one. I have yet to find a way to link a certain ship to address to a certain tax item, not to mention have it fill in once the ship to address is selected on a sales order. Any help is appreciated.
Comment: This question appears to be off-topic because it appears to be about how to use a program, rather than a programming question
Comment: This answer has to be with programming, there is no way to do this through the program. Has to be done with a script.
Comment: Then I suggest you put a bit more detail in here if possible.... I don't know anything about netsuite just trying to give you pointers to help you get an answer. Details about your form, anything you've attempted to script so far - people here are more likely to help if they can see what you've researched or attempted to do already.  Good luck.
Comment: Also, since netsuite seems rather 'specialist' rather than general purpose programming (which is the strength of StackOverflow) are there not any specialist forums or support groups you can find?  I'm not saying that no-one here will be able to help, but its certainly less likely than questions about more widely used platforms, there seem to be a question about it only every few days.
Comment: @iandotkelly instead of typing all those comments you could have answered someone else's question
Comment: @cja .... mmm so trying to help a new user, suggesting they add detail - such as the code - isn't constructive?  The purpose of one of the review queues is to help new users.
Answer: If these tax items are just for your reference, then you can create a custom record type and store the reference shipto & taxitem.
Then you can use client script to populate the taxitem based on the shipto by querying the custom record type.
Comment: I can't seem to think straight or something. I have an error I am trying to fix.
   object.setFieldValue('custbody_custom', 2);
I take the object out then no errors with internal ID
I put the object in then it says not a valid internal ID.
I am new but I can't seem to figure this one out. I have the code doing what I want except trying to populate it to a custom field.
Comment: If you can put the code then it will be more helpful to understand the problem

Netsuite Tax Item Script


Find an invertible non-diagonal $3 \times 3$ matrix $D$ such that $D^3 = D$.

Question:
Find an invertible non-diagonal $3 \times 3$ matrix $D$ such that $D^3 = D$.

I have forgotten how to solve this kind of question, can somebody give me some hint or idea how to start?
Comment: The title is not supposed to replace the first line of your question. The only possible candidate is $D_3$ and what is $D_3$ anyway?
Comment: Did you mean $D^3=d$?
Comment: I mean the cube of D equals D, D is just a symbol.
Comment: Have you tried taking a diagonal matrix with this property and conjugating it by something to make it non-diagonal?
Answer: Since $D^3-D=0$, the minimal polynomial $\mu_D(x)$ divides $x^3-x=x(x^2-1)$, but $D$ is invertible hence $\mu_D(x)$ divides $x^2-1=(x-1)(x+1)$. Since the matrix $D$ is not diagonal, $D\ne\pm I$ hence $\mu_D(x)\ne x\pm1$, hence $\mu_D(x)=(x-1)(x+1)$. No factor of $\mu_D(x)$ has degree greater than $1$ hence $D$ is diagonalizable, with eigenvalues $1$ and $-1$, that is:

The solutions $D$ are the non-diagonal matrices similar to the diagonal matrix with diagonal $(1,1,-1)$ or to the diagonal matrix with diagonal $(1,-1,-1)$.

Example: $D=\begin{pmatrix}1&0&0\\2&1&2\\-2&0&-1\end{pmatrix}$.
Answer: The equation $D^3 = D$ is invariant under similarity, i.e. if $D$ is a solution then so is $SDS^{-1}$ for any nonsingular $S$.  Find a diagonal matrix that satisfies the equation (and is not a multiple of the identity) and take some
random $S$.
Answer: An idempotent matrix ($A^2=A$) matrix also has $A^n=A$, so therefore this question reduces to the following question:
Constructing idempotent matrices
Comment: The matrix $A=-I$ solves $A^3=A$ but is not idempotent. (And I love the upvote.)
Comment: @Did - my mistake, that's not what I meant to write. Fixed.
Comment: But now you proved that every idempotent is solution, not that every solution is idempotent.
Comment: @Did, yes: The OP only needed a single instance.
Comment: Oh, I see. $ $ $ $


hide a view block if it has less than x results

Question: I would like to make a slideshow on author pages if they write more than 10 articles on my site, otherwise no.
I have created a views block containing the slideshow and i wonder how to automatically/programatically hide it if les than 10 articles were written .
Thanks in advance for any help
Yaz
Comment: Will authors be allowed to do anything else if they have more than 10 posts? I'd consider giving them a role and basing views of that.
Comment: not especially. I just want to have a better looking page for those who post many articles rather than havin a list.
Answer: You can use the template_preprocess_views_view() in your theme_name.theme (assuming D8) or in your template.php
function MY_THEME_NAME_preprocess_views_view(&$variables) {
  if ($variables['display_id'] == 'MY_DISPLAY_ID' && count($variables['view']->result) < 10) {
    $variables['view']->result = NULL;
  }
}

This is working for Drupal 7 and I assume it will for Drupal 8.
Be sure to check if your display is unique or you can also use the ID with :
$id = $variables['view']->storage->id(); // D8


How do I do stratified sampling on group-separated datasets in Python? Do packages for this exist?

Question: Say I have the following data:
<code>    Group_ID | Column_1 | Column_2 | Column_3 ...
==========================================
A        | 1        | 2        | 33
A        | 2        | 2        | 3765
A        | 3        | 6        | 3436
A        | 4        | 8        | 32
B        | 5        | 9        | 33
B        | 3        | 34       | 385
B        | 7        | 25       | 3
B        | 3        | 1        | 38
C        | 6        | 2        | 3
C        | 8        | 2        | 4
D        | 7        | 1        | 5
D        | 6        | 9        | 11
</code>
I want to:

First identify train-test splits that keep groups (Group_ID) separate between splits. I.e. no group can be in both train and test splits.
Out of all possible splits that have been identified, get splits which have the most similar distributions of Column_1, Column_2, Column_3 etc. across train and test splits.

In short, is there any way that I can split my data so that groups are separated, but that the other features are similar across the split?
Ideally, I would like to do this with a package in Python or the like, if it exists.
Comment: This sounds similar to [stratified group k-fold validation](https://www.kaggle.com/jakubwasikowski/stratified-group-k-fold-cross-validation)
Comment: Unfortunately, I do not think that works the way as intended. See my comment under the post, but essentially it ensures uniformity BETWEEN folds, but not WITHIN folds.
Answer: This problem is much more complicated than it may look. Try to construct a solution for your simple example. In general, this problem in general can be interpreted as multidimensional knapsack problem and thus even approximations are very hard to find - the one-dimensional knapsack problem is already NP-hard. In your toy example one can simply try all possibilities and choose the 'best' one. Which brings to the second problem: What is the 'best' solution? The goal is to have a proper split for all classes and thus this is a multi-critera optimization problem, so there may be solutions that are clearly worse than others, but one solution may be better for one class and a second solution may be better for another class. One possibility might consider the hypervolume created by the solutions with respect to the theoretical optimal solution. The theoretical optimal solution would create two sets such that both contain half of the samples of each class. For your toy example, I think the optimal solution (with respect to the product of the hypervolume of both splits) is to use:
split1 = Group A + Group B, and split2 = Group C + Group D
This yields:
| split1 | Column_1 | Column_2 | Column 3 |
==================================
A+B    | 28       | 87       | 7725
C+D    | 27       | 14       | 23

Column 1 is split really good: 27.5 would be optimal and it reaches 27 and 28.
Column 2 is split rather bad: optimal would be 50.5, and we get 87 and 14.
Column 3 is split really bad: optimal would be 3874, and we get 7725 and 23.
Therefore, I assume that there is no package to solve your problem. Good luck anyway!


Trying to delete records from a table that exist in another table

Question: I am trying to delete records from table ersr2 that exist in table ersr but I get a syntax error.
Here is my command:
MariaDB> delete from ersr2 e2 where e2.child_id in (select e.child_id from ersr e);

But I get the following syntax error:
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'e2 where e2.child_id in (select e.child_id from ersr e)' at line 1

I find this strange because the following command worked:
MariaDB> select e2.* from ersr2 e2 where e2.child_id in (select e.child_id from ersr e)

Any help will be appreciated.
Comment: Remove the alias: `delete from ersr2 where child_id in (select e.child_id from ersr e);`
Comment: or, use this form of the DELETE statement:  `delete e2.* from ersr2 e2 where ...`  (this is the form I generally use, because  I usually test the statement as a SELECT statement first, return the rows that I am going to delete. so I can review before accidentally deleting rows I didn't mean to delete.  I can also save off the results as a backup.  then changing `select e2.* from ersr2 e2 where` to a delete is a simple matter of replacing a single keyword
Comment: Thanks, that did the trick.
Answer: Declare the table for delete in delete clause  (delete e2)
delete  e2
from ersr2 e2
INNER JOIN  ersr e ON e.child_id = e2.child_id

and you could use inner join instead of a IN (sud query) clause


Swift compiler error on mapView declaration

Question: I upgraded to a newer Swift compiler and ran into this compiler error I can't figure out how to resolve.  I have a bunch of mapView functions declared from the MKMapViewDelegate.  They all seem to match up except this one which throws this error:
ViewController.swift:137:10: Objective-C method 'mapView:viewForAnnotation:' provided by method 'mapView(:viewForAnnotation:)' conflicts with optional requirement method 'mapView(:viewForAnnotation:)' in protocol 'MKMapViewDelegate'
ViewController.swift:12:7: Class 'ViewController' declares conformance to protocol 'MKMapViewDelegate' here
/ViewController.swift:137:10: Requirement 'mapView(_:viewForAnnotation:)' declared here (MapKit.MKMapViewDelegate)
I looked at MKMapViewDelegate method and I thought I was matching it up correctly so I am baffled as to what has changed that needs correction.  From there I see
optional func mapView(mapView: MKMapView!, viewForAnnotation annotation: MKAnnotation!) -> MKAnnotationView!
Here's my declaration below that throws the error.
    class ViewController: UIViewController, MKMapViewDelegate {

        func mapView(aMapView: MKMapView!, viewForAnnotation annotation: CustomMapPinAnnotation!) -> MKAnnotationView! {
         //The error is thrown here on the m in mapView
        }
    }

Answer:
I looked at MKMapViewDelegate method and I thought I was matching it up correctly so I am baffled as to what has changed that needs correction. From there I see

optional func mapView(mapView: MKMapView!,
    viewForAnnotation annotation: MKAnnotation!)
    -> MKAnnotationView!

Really? You think your declaration matches that? Look closer. You have:
func mapView(aMapView: MKMapView!,
    viewForAnnotation annotation: CustomMapPinAnnotation!)
    -> MKAnnotationView! {

Rewrite it like this:
func mapView(aMapView: MKMapView!,
    viewForAnnotation annotation: MKAnnotation!)
    -> MKAnnotationView! {

Do you see the difference? It is not your place to alter the types in the delegate method declaration. The declaration has MKAnnotation, you must declare MKAnnotation. This MKAnnotation may or not also be a CustomMapPinAnnotation, but that is for your code inside the body of the function to decide.
Comment: Ok then... what is intetresting is the compiler just ignored this previously and it just worked.  I suppose as the language tweaks were put to tighten up areas for further type enforcement.  My CustomMapPinAnnotation declaration was:
Comment: I understand that it just worked previously. But this, as you rightly say, was because the compiler was letting it slide - and now it doesn't.
Comment: finishing my comment from above just for reference to others, it's all back and working now.
    class CustomMapPinAnnotation : NSObject, MKAnnotation {  }

and now I just matched it 100% and declared a var as:

    var myannotation = annotation as? CustomMapPinAnnotation
Comment: Correct-a-mundo, well done! And this is much more correct code anyway, because now you are guarding against the possibility that this might _not_ be a CustomMapPinAnnotation.
Comment: I also had some code up top in mapView to check if my custom class was incoming and to bail out of not... was advised on this many moons ago in the beta days... :-)  Thanks for the "schooling".   Here is code to check on that within mapView              if !(annotation.isKindOfClass(CustomMapPinAnnotation))
            {
                println("Inside my mapview but was not passed Custom CustomMapPinAnnotation ")
                return nil
            }
Comment: "many moons ago in the beta days" I know just what you mean! The language has evolved a _lot_ since then.


How to override webform in template.php

Question: I've followed different tutorials but I cannot seem to override a field on my webform via a function in template.php.

Theme is called: 'myTheme' Id of the form is:
  'webform_client_form_76' Id of the input:
  'edit-submitted-henkilotiedot-etunimi' Name of the input:
  'submitted[henkilotiedot][etunimi]'

The function I think should work:
function myTheme_webform_form_76 (&$vars) {
    $vars['form']['submitted']['edit-submitted-henkilotiedot-etunimi']['#value'] = t('Name');
}

What is the correct format to override webform elements?
Answer: function [themename]_form_webform_client_form_76_alter(&$form, &$form_state, $form_id) {
// code of altering
}

Comment: Thanks, but I seem to also have the format of the code to alter wrong as well: `$form['form']['submitted']['edit-submitted-henkilotiedot-etunimi']['#value'] = t('Name');`
Comment: Setup devel module and then use `kpr($form);` to research $form structure. Looks like it should be `$form['submitted']['henkilotiedot_etunimi']['#value'] = t('Name');`
Comment: Thanks for the advice. It turns out I needed `$form['submitted']['henkilotiedot']['etunimi']['#default_value'] = t('Name');`

How to override webform in template.php


Testing for multicollinearity in logistic regression

Question: So far I have checked the tolerance value, VIF and condition indexes. But checking the variance of the regression coefficients I have to wonder: how little variance of the regression coefficient should be associated with the smallest eigenvalue and what is too much (indicating multicollinearity)?
Answer: The rule of thumb I've often read is VIF > 5 indicates a level of multicollinearity worth investigating (and Tolerance is just the reciprocal of VIF).
Comment: Yes, I've read similar information about the VIF. So if the VIF value doesn't give a reason for concern I don't have to check the other values like condition index?
Comment: No I'd look at both VIF and condition index - also see this similar question: http://stats.stackexchange.com/questions/4099/vif-condition-index-and-eigenvalues. Keep in mind if you're building a predicitve model, a little multicollinearity isn't necessarily bad since coefficient estimates are unbiased even if SEs are inflated.


PayPal IPN | Drop Down Questions

Question: On my PayPal IPN I had to change $_POST['item_name'] to $_POST['item_name1'] aswell as $_POST['item_number'] to $_POST['item_number1'] while testing in Sandbox IPN. I was wondering if I need to keep those or change them back when I go live.
Also I have a drop down list for the payment, I was wondering if the value of one of the list items is the item_name?
And one more question, can I access $_SESSION variables from my site in the IPN script? Or do I need to post them, and if so how do I post them?
Answer: Generally the test IPN mirrors the live IPN. And the IPN reference says it's going to be item_nameX (where X is an incremented number). Assuming you're using Payments Standard, then on0 will be item_name1 and os0 will be item_number1
You can't directly pass $_SESSION because PayPal will be POSTing data to you and would get a different session_id than your customer. You can use the custom field to pass the session_id and then use it in your IPN to set the session_id. This should be safe to do after you validate the IPN call.
session_id($_POST['custom']);
session_start();

PayPal IPN | Drop Down Questions


Troca rápida e contínua da cor de background de um Panel

Question: Estou tentando fazer com que uma panel fique trocando constantemente de cor com o seguinte código
while (true)
{
   panelColor.BackColor = Color.Blue;
   Thread.Sleep(500);
   panelColor.BackColor = Color.Red;
   Thread.Sleep(500);
}

O problema é que a aplicação trava todoa vez que chega nessa parte, alguem pode me dizer o por quê?
Comment: Isso é um loop infinito, a execução nunca sairá dele.
Comment: teoricamente não deveria ficar trocando as cores com o intervalo de 500 milisegundos nesse loop?
Comment: obrigado pela ajuda, era isso mesmo, esqueci do refresh
Answer: A aplicação trava porque está a executar um loop infinito.
Você também não vê o background a mudar de cor pela mesma razão, como a UIThread está bloqueada a executar esse código, o framework do windows não tem oportunidade de redesenhar os controles.
Altere o código da seguinte forma:
while (true)
{
   panelColor.BackColor = Color.Blue;
   panelColor.Refresh();
   Thread.Sleep(500);

   panelColor.BackColor = Color.Red;
   panelColor.Refresh();
   Thread.Sleep(500);
}

O método refresh() faz o controle auto redesenhar-se.


Integration by parts with polynomial formula proof

Question:

Calculate $$\int(x^3+x^2)(\sin x)\cdot\mathrm{d}{x}$$

There's an easy way to solve that kind of integrals:
$$\int (p\left(x\right))(f\left(x\right))\cdot\mathrm{d}{x}$$
Where $p\left(x\right)$ is a polynomial and $f\left(x\right)$ is a function.
The formula is $$\int (p\left(x\right))(f\left(x\right))\cdot\mathrm{d}{x}=\sum_{i=1} ^\infty ((-1)^{i+1}(p^{(i-1)})(f_{(i)}))+constant$$
where $a^{(n)}$ is $n$th derivative of $a$, $a_{(n)}$ is $n$th integral of $a$.
When we use the formula, we can see that the inegral $\int(x^3+x^2)(\sin x)\cdot\mathrm{d}{x}$ is equal to $$(x^3+x^2)(-\cos x)-(3x^2+2x)(-\sin x) + (6x+2)(\cos x)-(6)(\sin x)$$

I'm trying to prove that formula, but I don't know where to start from. I know that if this formula didn't exist, I should break it into parts, $x^3 \sin x$ and $x^2 \sin x$ then do $u = x^2$ or $u = x^3$ for both parts and $\sin x = \mathrm{d}{v}$ but I don't know how to use them to derive this formula.
Comment: If you solve $\int p(x) f(x)dx$ where $p$ is a polynomial and $f$ is any function then for any function $g$ you solve $\int g(x)dx=\int p(x)\dfrac{g(x)}{p(x)}dx.$
Answer: We prove it using induction over the degree of $p$. In all steps the constant of integration will be skipped.
When $p$ has degree $0$ it is just a constant, so
$$
\int p(x) f(x) \, dx
= p(x) \int f(x) \, dx
= p^{(0)}(x) f_{(1)}(x)
= \sum_{k=1}^{\infty} (-1)^{i+1} p^{(i-1)}(x) f_{(i)}(x)
$$
since $p^{(i-2)} = p^{(i-3)} = p^{(i-4)} = \cdots = 0.$
Now assume that the formula is valid for a polynomial of degree $n$. We will show that it is then valid for a polynomial of degree $n+1$. Thus, assume that $p$ is of degree $n+1$. Integrating by parts and applying the formula to $\int p^{(1)}(x) f_{(1)}(x) \, dx$ gives,
$$
\int p(x) f(x) \, dx
= p(x) f_{(1)}(x) - \int p^{(1)}(x) f_{(1)}(x) \, dx \\
= p^{(0)}(x) f_{(1)}(x) - \sum_{k=1}^{\infty} (-1)^{i+1} p^{(i)}(x) f_{(i+1)}(x) \\
= p^{(0)}(x) f_{(1)}(x) - \sum_{k=2}^{\infty} (-1)^{i} p^{(i-1)}(x) f_{(i)}(x) \\
= p^{(0)}(x) f_{(1)}(x) + \sum_{k=2}^{\infty} (-1)^{i+1} p^{(i-1)}(x) f_{(i)}(x) \\
= \sum_{k=1}^{\infty} (-1)^{i+1} p^{(i-1)}(x) f_{(i)}(x) \\
$$
By induction we can now conclude that the formula is valid for polynomials of any degree.
Answer: you can also write $$\int x^3\sin(x)dx$$ and $$\int x^2\sin(x)dx$$
Comment: I know how to solve this problem. I don't know how to prove that formula.
Comment: It is enough to work with $\int x^{k} f(x) dx$. It is reasonable to try it for the first smaller n values.