Embedded Database with take fast multiple writes

Question: I am creating an C# WPF application that intake RS232 data and store all data to an embedded database in relatively high speed (hundreds per second). The application does two things concurrently:

take RS232 data and store to the database
plot the data from the database.

I consider SQLite because my application required embedded database and it's a single user application. But storing and graphing the data (it could be storing current data and plotting data of yesterday's) requires concurrent Read and Write to the database. I heard SQLite get lock up on one query and canot process multiple query concurrently. Is that true? If so what other embedded database supports multiple writes?
Answer: Berkeley DB has non-blocking writes and has a reputation for being speedy.
Very interestingly, it can be used as a drop in replacement for SQLite.

applications written to the SQLite
  version 3 API can switch to using
  Oracle Berkeley DB with no code
  changes, by re-linking against the
  Berkeley DB SQLite library. The
  application-level behavior of the two
  products is identical, but the
  advanced features of Berkeley DB
  provide SQLite applications improved
  performance, concurrency, scalability,
  and reliability.

That sounds like it would fit the bill quite nicely.
Answer: Ok, a few things.
First, yes SQLite is thread safe, even though the author is not enthusiastic about it. Read about it here: http://www.sqlite.org/faq.html#q6.
Second, you mentioned "fast" as being hundreds of inserts per second. SQLite can easily reach 30,000 inserts per second if you wrap everything in a transaction. I say this not because it was really your question, but more because knowing this may help your decision (as it redefines what is considered "fast").
Comment: I am sure multiple write is not possible in SQLite. But does "thread safe" mean it can have one write and multiple read at least?
Comment: @KMC - Sqlite supports transactions so that is your means of concurrency control (which I suspect is what you mean by thread safety). Sqlite is thread-safe as long as you don't share connections across threads. Also note that that Sqlite only supports database-level locking, effectively limiting you to only one writer at a timer regardless of the number of connections.
Answer: Check out RavenDB.  http://www.ravendb.net/ it would be fairly easy to run some tests with it and see if it will handle your requirements.


Change checkbox state based on multiple conditions in AngularJS

Question: In my scenario I have a select dropdown with an ng-model assigned to it. Each item in the dropdown corresponds to a number of checkboxes that are checked when a selection is made. The difficulty is that I can't just check for a simple key:value because checkboxes aren't exclusive to any particular selection. For example:
<select>
  <option>Dog</option>
  <option>Cat</option>
  <option>Bird</option>
</select>

<input type="checkbox">Is an animal
<input type="checkbox">Can bark
<input type="checkbox">Can meow
<input type="checkbox">Can tweet
<input type="checkbox">Has four legs
<input type="checkbox">Has wings

So as you can, the checkboxes for "Is an animal" and "Has four legs" aren't exclusive. My current implementation uses a simple conditional expression to evaluate whether or not the checkbox is marked (ng-checked="animal=='dog'") but of course this excludes the other two possibilities. So I'm wondering if there's a native Angular way to handle OR statements or an array. Or if not, how can I go about this with JavaScript or jQuery?
Answer: See this fiddle: http://jsfiddle.net/QHza7/
Template:

    <select ng-model="opts" ng-options="a.opts as a.name for a in answers"></select>

    <input type="checkbox" ng-model="opts.animal" />Is an animal

    <input type="checkbox" ng-model="opts.bark" />Can bark

    <input type="checkbox" ng-model="opts.meow" />Can meow

    <input type="checkbox" ng-model="opts.tweet" />Can tweet

    <input type="checkbox" ng-model="opts.legs4" />Has four legs

    <input type="checkbox" ng-model="opts.wings" />Has wings

Code:
function Test($scope) {
    $scope.answers = [
        {name:"Dog", opts:{animal:true,bark:true,legs4:true}},
        {name:"Cat", opts:{animal:true,meow:true,legs4:true}},
        {name:"Bird", opts:{animal:true,tweet:true,wings:true}}
    ];
    $scope.opts = null;
}


java - Is it possible to check current used heap space at runtime?

Question: I'm writing an app the generates a big xlsx file using apache-POI.
At a certain time I get OutOfHeapSpace exception.
I want to  solve it by writing the content to the xlsx file when I determine that I'll soon be out of space, and thus freeing the memory, and then reading the file from the disk and continuing the writing.
A better solution might be to predefine the number of cells I'll write in each "block" , and then write the block to disk, but in any case this made me wonder if there is a way to determine the heap space that is left at runtime?
Comment: Does this help? http://www.exampledepot.com/egs/java.lang/GetHeapSize.html
Comment: yes it does with the question regarding heap space. bunting solved the greated issue.
thanks all.
Answer: The MemoryMXBean can give you a fair amount of information about the current memory usage.
<code>public class PrintMemory {
    public static void main(String[] args) {
        MemoryMXBean memoryMXBean = ManagementFactory.getMemoryMXBean();
        long[] array;
        for (int i = 0; i < 100; i++) {
            array = new long[100000];
            System.out.printf("%d%n", memoryMXBean.getHeapMemoryUsage());
        }
    }
}
</code>
Answer: Freeing unused memory at runtime is probably not very reliable since the JVM has considerable freedom in deciding when to garbage collect.
POI recently introduced the SXSSF API that uses streaming and thereby significantly reduces memory footprint for writing spreadsheets. This should help even with very big xsls files. There are a couple of downsides which are shown here. But if you can live with them, this should alleviate you of heap related problems.
Answer: You can get the available memory, but it only tells you the most you can allocate without triggering a GC.
Instead you can trigger a GC and see how much memory is free after wards.  The problem this is it has a perform overhead.
Another option is to monitor the GC cleanups via JMX and see how much is free after it naturally triggers a GC (if it doesn't run, you don't have a problem)
Answer: Using one of the coming with the JVM JMX MBeans is worth considering.
Like this one MemoryPoolMXBean


Esconder campo obrigatório e não obrigatório

Question: Eu tenho um formulário. Neste formulário eu tenho dois radiobutton com duas opções.
1) Questao 1 (Obrigatorio)
1 = Sim
2 = Nao

______________________

2) Questao 2(Obrigatorio)
1 = Vivo
2 = Morto

Se esta questão 1 for = Sim a Questão 2 perde a obrigatoriedade, não irá deixar de ser uma questão obrigatória a ser respondida, mas se for = Não, a questão 2 voltará com a obrigatoriedade.
Alguém tem alguma forma em poder me ajudar com isso?
Agradeço desde já.
Answer: Pessoal boa tarde eu fiz a seguinte validação
var value = $('input[name="CAMPO1"]').val();

if (value == '1') {
    $('[name="CAMPO2"]').rules('remove', 'required');
} else {
    $('[name="CAMPO2"]').rules('add', 'required');
}

e funcionou perfeitamente. Agradeço a ajuda!
Answer: Se você estiver usando a validação cliente no javascript, você pode adicionar o atributo data-val-required e removê-lo quando não quiser que o campo seja obrigatório, algo assim:
// Obrigatório
$(campo).attr("data-val-required", "*");

// Não obrigatório
$(campo).removeAttr("data-val-required");

Pode ser necessário informar ao form para fazer o parse dos controles novamente, o que você pode fazer usando esse comando:
$.validator
        .unobtrusive
        .parse("form");

Answer: Nesse caso precisaria criar um validator, implementando IValidatable e IClientValidatable. Uma sugestão seria criar algo igual [Required], mas apenas se outra propiedade ou condição fosse verdade.
Comment: Então, eu tenho um arquivo chamado validacao.js nele eu faço as validações das questões e mascaras etc
como exemplo.
$("#form-panel-AltaHospitalar").validate({
        errorClass: "input-error",
        rules: {
            dtahosp: "required",
},
Eu ja fiz diversos testes diversos metodos diferentes, só que não deu nada certo
Comment: Você está usando *unobtrusive-validation*? Ou simplesmente *jquery-validate*?
Comment: somente o  jquery-validate


pyinstaller Template not found

Question: I used pyinstaller to create an executable, but I am getting these errors when I run the executable:

> Traceback (most recent call last):
>       File "flask/app.py", line 1988, in wsgi_app
>       File "flask/app.py", line 1641, in full_dispatch_request
>       File "flask/app.py", line 1544, in handle_user_exception
>       File "flask/app.py", line 1639, in full_dispatch_request
>       File "flask/app.py", line 1625, in dispatch_request
>       File "testforms.py", line 13, in my_form
>       File "flask/templating.py", line 133, in render_template
>       File "jinja2/environment.py", line 851, in get_or_select_template
>       File "jinja2/environment.py", line 812, in get_template
>       File "jinja2/environment.py", line 774, in _load_template
>       File "flask/templating.py", line 57, in get_source
>       File "flask/templating.py", line 85, in _get_source_fast
>       TemplateNotFound: template.html

When I run this script locallly, it all works fine.
Here is the script:
from flask import Flask
from flask import request
from flask import render_template
app = Flask(__name__, template_folder='templates')

@app.route('/')
def my_form():
    return render_template('/template.html' )

    @app.route('/', methods=['POST'])
 def my_form_post():

    text = request.form['text']
    processed_text = text.upper()
    print processed_text
    return processed_text

if __name__ == '__main__':
    app.run()

To build the executable I used:
pyinstaller --onefile testforms.py

I also tried:
return render_template('template.html' )

Comment: I also tried:   return render_template('template.html' )
Comment: Possible duplicate of [Flask application built using pyinstaller not rendering index.html](https://stackoverflow.com/questions/32149892/flask-application-built-using-pyinstaller-not-rendering-index-html)
Answer: i encountered the same issue. But i didn't make one file. I just copy static and template folder and paste in "dist/main"  folder. Its works for me
Comment: Doesn't it defeat the purpose of bundling the app into a package (using PyInstaller) and distributing it as an executable?

pyinstaller Template not found


RemoteJMeterEngineImpl class in Jmeter, How to create instance and call rconfigure

Question: Agenda -  Invoke JMeter of Remote Area from localhost Java class
I found that RemoteJMeterEngineImpl class helps to invoke JMeter of remote area
 Problem - Not able to create instance of RemoteJMeterEngineImpl  class due private constructor
Please help if anybody knows how to play with RemoteJMeterEngineImpl  class
Comment: Welcome to StackOverflow, please take a moment to review the tour: https://stackoverflow.com/tour ,
How to create a Minimal, Complete, and Verifiable example: https://stackoverflow.com/help/mcve , and more specifically,
*How to Ask Good Questions* https://stackoverflow.com/help/how-to-ask
- you'll get better feedback and helpful answers if you put some thought in to your problem before coming to SO for help, and by including a specific problem with code *that you've tried* showing effort to solve the problem yourself. You can edit your post once you have an idea of the specific issue.
Answer: You are doing something which is weird and definitely not supported. Normally you should have remote slaves up and running and if you need to kick off your test programatically I believe DistributedRunner class is something you could use.
Example code:
List<JMeterEngine> engines = new LinkedList<>();
Properties remoteProps = new Properties();
//set properties you want to send to remote clients here
DistributedRunner distributedRunner=new DistributedRunner(remoteProps);

List<String> hosts = new LinkedList<>();
//add your JMeter slaves here
hosts.add("remote_host_1");
hosts.add("remote_host_2");
distributedRunner.setStdout(System.out);
distributedRunner.setStdErr(System.err);
distributedRunner.init(hosts, testPlanTree);
engines.addAll(distributedRunner.getEngines());
distributedRunner.start();

See Five Ways To Launch a JMeter Test without Using the JMeter GUI article and jmeter-from-code example project to learn more about programmatic creation and execution of JMeter tests.
Comment: Thank you, As i'm following mentioned steps i'm facing below exception.
Error in rconfigure() method java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
 java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
 java.io.InvalidClassException: org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase; local class incompatible: stream classdesc serialVersionUID = 240, local class serialVersionUID = 241
Remote engines have been started
Comment: You need to use the same versions of JMeter on master and slaves
Comment: Both Machine JMeter versions (3.1) and JDK (jdk1.8.0_131) are same, still facing the exception. I'm able invoke this from command line successfully.  I would like to send class file if i get your email id
Comment: Do we need to configure anything on remote machine other than rmiregistry?
Answer: you need to call
RemoteJMeterEngineImpl.startServer(rmiPort);

Code:
public static void startServer(int rmiPort) throws RemoteException {
        RemoteJMeterEngineImpl engine = new RemoteJMeterEngineImpl(DEFAULT_LOCAL_PORT, rmiPort == 0 ? DEFAULT_RMI_PORT : rmiPort);
        engine.init();
    }


How does one compare a date field with the non-UTC user/system time?

Question: Date field data won't always correspond to UTC. From a user perspective, these dates are assumed to be in the user's timezone.  However, it is currently not possible to store dates in timezones other than UTC, at least until Add ability to select a timezone for datetime field gets in.
So when comparing these fields to the current time, the timezones won't match because the current time is in the local timezone and the field data is in UTC. We therefore need to adjust the current time to match the incorrect date field timezone (or vice versa).
What's a good solution for this?
Comment: To whoever downvoted this: Please only downvote questions if you can provide feedback.  This will help us ask better questions.  Thanks.
Answer: I've developed a method to handle the conversion, thanks to help from another answer to another question.  It requires a patch from Add DateTimePlus::getPhpDateTime() for situations that require a DateTimeInterface to work, however.
<code>  /**
   * Fetches the current time adjusted for comparing to non-UTC date fields.
   *
   * Date field data won't always correspond to UTC. From a user perspective,
   * these dates are assumed to be in the user's timezone.  However, it is
   * currently not possible to store dates in timezones other than UTC. So when
   * comparing these fields to the current time, the timezones won't match
   * because one is the local timezone and the other is in UTC.  We therefore
   * need to adjust the current time to match the incorrect date field timezone.
   *
   * @param Drupal\Core\Datetime\DrupalDateTime $date_time
   *   The date-time object fetched from a Date field (e.g. field_date->date).
   */
  protected static function getCurrentTimeForComparingToDateFieldIncorrectlyStoredInUtc(DrupalDateTime $date_time) {
    $current_time = new DrupalDateTime('now', DATETIME_STORAGE_TIMEZONE);
    $userTimezone = new \DateTimeZone(drupal_get_user_timezone());
    $timezone_offset = $userTimezone->getOffset($date_time->getPhpDateTime());
    $time_interval = \DateInterval::createFromDateString($timezone_offset . ' seconds');
    $current_time->add($time_interval);
    return $current_time;
  }
</code>


Are there names for referring to the top or bottom numbers in a time signature?

Question: In a time signature of 3/4 (three by four), what is the name given to the 3 and the 4? I know it in my own language, but can't find the answer for the English equivalent.
Answer: The entire symbol that defines the meter of a measure of music is called a time signature.
To my knowledge, English does not have specific words to refer to the top and bottom numbers, though some people may use the words 'numerator' and 'denominator', respectively, as if it were a mathematical fraction. I do not recommend this (since time signatures have nothing to do with fractions).
If I needed to give names to the top and bottom numbers I would call them 'measure length' and 'subdivision quantifier,' but usually it's not necessary to do this--or at least, we have enough other ways in English to talk about musical meter and the time signature that it's not necessary to refer to those numbers directly.
Comment: I meant measure. I sometime confuse the music terminology names in English. That's very odd for me to know since I am used to have specific names for that. By curiosity, if I was to translate the names I am used to use to to English, the `numerator` would be `unit of tempo` and the `denominator` would be `unit of measure`. Thanks for your reply.
Comment: Thank you. I see your point and can appreciate those parallels. I usually think of a fraction as a ratio or a quotient, rather than the two separate attributes that describe that quotient. Both are valid interpretations, of course. My argument is mainly that applying concepts of elementary math to musical meter can easily cause confusion to a learning musician. 6/8 and 3/4 are mathematically _and_ durationally equivalent, but are _musically_ different, and should not be played _or_ notated in the same way (beaming, chiefly).
Comment: Time signatures have everything to do with fractions.
Comment: @GeneralNuisance I graciously invite you to support your argument instead of making an unhelpful contradiction.
Comment: I thought it was something that was generally accepted, but I will explain (a quick google search has brought my attention to that it's not, so it's worth my time). A fraction is made of a numerator and a denominator. Numerator, as my math text book defines it, is how many "things" are in a fraction. Denominator, it defines as "the type of things" in a fraction. A time signature describes a measure. It doesn't necessarily have to equal 1 to equal one whole measure, however, when you think of it this way, the parallels are strong. Fractions can be equivalent...
Comment: ...like 3/4 and 6/8, and while musically they are different rhythms with different beats, notationally, it is correct to write a song in 6/8 in 3/4. The time signature describes a whole measure. Even before I learned that, apparently, the connection between fractions and time sigs was not mainstream understanding, I would have (and did) flag you with the statement "time signatures have nothing to do with fractions." Except in extremely informal (and/or teaching) settings, I wouldn't refer to the numbers in a time sig as Numerator/Denominator, but I don't think it's fair to sever the concepts.
Comment: I regret the wording "worth my time" in the first comment. I'm not trying to come off as rude.
Comment: 6/8 and 3/4 are equivalent fractions in math, but those time signatures are NOT the same! You could write a 3/4 song in measures of four beats, but it would not be a 4/4 song...
Answer: If you really need single words, I'd recommend 'count' and 'quantum'. If it isn't obvious which is which, then this is a bad suggestion.
Answer: There are no specific names, although that would be nice. The numerator is beats per bar. The denominator is the note duration of a beat. So 3/4 means 3 beats per bar, where quarter note is a beat.
Also, although tempo is specificed in beats per minute, it actually means QUARTER notes per minute. Even if the beat is not a quarter note per the time signature.
Comment: Tempo can be specified with something other than quarter notes, and in the absence of specification I would be loath to assume that "40 bpm" for a piece in 3/8 means anything other than forty 8ths per minute (for example).
Comment: That last part isn't really true--tempo markings are either going to be unambiguously marked for a quarter-note-beat meter, or will include a note value to show what kind of note gets what BPM value. For example, 6/8 time will be marked q.=60 or e=180.
Comment: I'm pretttttty sure that if there's no q.=99 or e=99 specified, then the sTANDARD is quarter notes per minute regardless of time signature's beat duration...

and I'm absolutely positive that in a midi file, tempo values are ALWAYS quarter notes per minute regardless of time signature.
Comment: In musical notation, a metronome marking is always specified as "M.M." (for "Maelzel's Metronome") followed by a picture of a note, an equals sign and a number.  The note may or may not be a quarter note.  6/8 time is usually 2 beats per measure, not 6.  Dance music is almost always either 4/4 (quarter note beat), 3/4 (quarter note beat), 2/4 (quarter note beat) or 6/8 (dotted quarter note beat).
Comment: The MIDI spec should NOT be used as a guide to real music. Seriously. A piece of music in 5/16 with an ambiguous time signature is NEVER going refer to the quarter note, since it's practically useless for figuring out how to actually play the thing. It's either going to be the entire measure, an eighth note, or a dotted eighth.
Comment: I'll grant ya that.  All I'm tryin' to say is that sheet music CAN be ambiguous like that.  And if it's not spec'd it's safer to assume BPM=quarter note/min rather than "denominator" per min...  I'm positive I've seen ambiguous sheet music like that.
Comment: My point (and the last I'll say on this) is that while meter and beat may be ambiguous, a metronome marking should be interpreted as "beats per minute" if no other indication is given. _Then_ you can start figuring out which notational value is supposed to get the beat, and that part may or may not correspond with the bottom number in the time signature--you still have to go through the process of figuring that out. But the metronome marking should be considered _beats_ per measure, which may or may not be quarter notes. It's the _meter_ that's ambiguous.


How to add 2 string of code together for views in custom module?

Question: I have a view that needs these 2 lines of code:
 if($form_id == "views_exposed_form" && $form['#id'] == 'views-exposed-form-hide-tweaked-advanced')

and
if($form_id == "views_exposed_form" && $form['#id']=='views-exposed-form-hide-tweak-basic')

function:
<?php
 function dropdowninteger_form_alter(&$form, &$form_state, $form_id){
  if($form_id == "views_exposed_form" && $form['#id']=='views-exposed-form-hide-tweaked-advanced'){
     $options = array('1' => '1', '2'=>'2');
     $form['field_user_age_value']['#type'] ='select';
     $form['field_user_age_value']['#size'] = null;
     $form['field_user_age_value']['#default_value'] = '';
     $form['field_user_age_value']['#options'] = $options;
  }
}

How do I add them together for my custom module? Not sure if my question is clear enough though....
Comment: And please tell us why and where exactly that code is needed.
Comment: Please make your code more readable and highlight it using code tags.
Answer: This is how I usually set up my views functions:
function dropdowninteger_form_alter(&$form, &$form_state, $form_id){
  if($form_id == 'views_exposed_form') {
    switch ($form['#id']) {
      case 'views-exposed-form-hide-tweaked-advanced':
        <tweaked-advanced code here>;
        break;
      case 'views-exposed-form-hide-tweak-basic':
        <tweak-basic code here>;
        break;
    }
  }
}

Comment: If you don't think this is really a Drupal question, please flag as off topic, not attempt to answer. Answering questions you *know* are bad encourages people to ask more of them. So you are promoting degradation of quality, and you kinda know it...
Comment: @Molot I did both that way hopefully everyone gets something out of it :) If it gets closed it is gone, correct?
Comment: First, two weeks of being on hold. Then, if not fixed and reopened, and  if it's of positive or neutral score with upvoted answers, it stays. If not, or if moderator decides, it's deleted. Then, if moderator or few users over 10k decides it can be salvaged, it can be undeleted and edited. And so on.
Comment: The answer above worked, it was  a PHP question I guess, I looked for Stack Exchange PHP but only found programmers, where should I post stuff like this?
Comment: @BrunoVincent if question is not really about Drupal, but about PHP code, it's best suited for http://stackoverflow.com/ However, your question now have three "unclear what you're asking" votes. This means at least three people with pretty high reputation wasn't able to tell for sure what you need. That's not good. And sorry to say, but if you think your question was poorly received here, Stack Overflow can be even more strict.
Comment: Hi Molot, ok thanks, I'll keep working on my asking skills ;)


Given N independent random variables, when does independence stops holding?

Question: I have the following problem:
Given $X_1, X_2..X_N$ independent r.v.
$\{X_1,X_4 X_1\}$ are independent of $\{X_3,X_2X_1\}$, but $X_4$ is not independent from $X_4 X_1$
Is that statement true or false? Why?
I believe it has to be False since both sets contain $X_1$ so if I had information on the second one, then the probability of the first would be altered, therefore they are not independent, the same with the second part.
I am not entirely sure, and i do not know how to prove it in a formal way. I believe it has something to with the $\sigma$ algebra generated by the r.v., but honestly i dont know.
Thanks a lot in advance!
Answer: $$
P(\{X_1,X_1 X_4, X_3, X_2 X_1\} = (a,b,c,d))=\\
P(X_1 = a) P(X_3 = c) P({X_4, X_2} = (b/a,d/a)|X_1=a,X_3=c)\\
P(X_1 = a) P(X_3 = c) P(X_4 = b/a) P (X_2 = d/a)\\
P(\{X_1, X_1 X_4\}=(a,b)) P(\{X_3, X_1 X_2\}=(c,d))
$$
Therefore, $\{X_1, X_1 X_4\},\{X_3, X_1 X_2\}$ are independent $\forall a$ such that $P(X_2 = d/a) = P(X_1 X_2 = d)$.
Answer: Why should $\{X_1, X_4 X_1\}$ be independent of $\{X_2, X_2 X_1\}$?  In particular, there is no reason for $X_1$ and $X_1 X_2$ to be independent,
and it's easy to construct examples where they are dependent.  For example, suppose both $X_1$ and $X_2$ have possible values $1$ and $2$.  Then
$X_1 X_2 = 1$ implies $X_1 = 1$.
Comment: Well, @Robert Israel, I agree with you. But is that a formal proof? I mean, is the exercise just asking to find some counter-example? Or is there a more "profound" justification?
Comment: A specific counterexample should be "formal" enough.
Comment: Any systematic way of dealing with this? Because I actually have a bunch of similar excercises and the approach of finding a counter example works for proving that something does not hold.
I was thinking something about $P(\{X_1,X_1 X_4\} | \{X_3,X_2 X_1\})$ and working on it?


Existence of fractional moment of cauchy distribution

Question: For the standard Cauchy distribution, do fractional moments exist?  If $Y \sim C(1,0)$, is it possible to evaluate $E(Y^{1/3})$?
Comment: Note that the Cauchy takes negative values, so in effect almost all fractional moments are ruled out immediately - as they are for the normal, say (assuming you expect real-valued answers).
Comment: maybe complex moments
Answer: Let's apply the law of the unconscious statistician, so we want to be able to evaluate
$\int_{-a}^b \frac{x^k}{1+x^2} dx$
in the limit as $a,b$ each $\to\infty$.
As mentioned in comments, for most fractional $k$ there's problems for negative $x$.
Let's step aside from that issue, and just talk about convergence of the right side.
Consider then, the convergence of
$I(b)=\int_0^b \frac{x^k}{1+x^2} dx\,$, for $0<k<1$
First consider splitting into $\int_0^1$ and $\int_1^b$ for $b>1$
for $\int_0^1 \frac{x^k}{1+x^2} dx$ note that when $x$ and $k$ are between 0 and 1 and  that $x^k\leq 1$, so that integral is bounded.
Now note that for $x>0$, we have $\frac{1}{(1+x^2)}<x^{-2}$, so
$\int_1^b \frac{x^k}{1+x^2} dx<\int_1^b x^{k-2} dx = \left. \frac{x^{k-1}}{k-1}\right|_1^b = \frac{b^{k-1}-1}{k-1}=\frac{1-b^{k-1}}{1-k}$
which doesn't get any bigger than $\frac{1}{1-k}$ (when $b>1$ and $0<k<1$, as here).
So in the limit as $b\to\infty$, the integral converges.
As a result, $\int_{-\infty}^\infty \frac{|x|^k}{1+x^2} dx$ converges, and so when $x^k$ is real, $\int_{-\infty}^\infty \frac{x^k}{1+x^2} dx$ will converge to a real answer.
Specifically, the $\frac{1}{3}$ moment of a standard Cauchy is therefore 0 (as are all reciprocal-of-odd-positive-integer moments), by symmetry.

Note that according to Wolfram alpha (I didn't try to integrate it myself) $\int_{0}^\infty \frac{x^{1/3}}{1+x^2} dx=\frac{\pi}{\sqrt{3}}$.
Comment: The integral is easy to evaluate using the Calculus of Residues.  For the moment to be well-defined and real you need $k$ to be a rational number with odd denominator or even numerator. Only in the latter case will the moment be nonzero.
Comment: @whuber indeed, that's exactly how I'd have tackled it if I was going to try to evaluate it myself.
Answer: To complement @Glen_b's answer, I asked maple for help, and it gives the following answer, which is a complex number since we ask for fractional powers of negative numbers! If such a complex moment is of any use I do not know, probably one would have to take the moments of the positive and negative part separately, as is done in definition of The Mellin transform  https://en.wikipedia.org/wiki/Mellin_transform  As that page shows, these fractional moments is essentially the Mellin transform. The Mellin transform is used in studying products of random variables.
The result is (I did not check on maple ...)
$$
\int_{-\infty}^\infty x^\alpha \frac1{\pi (1+x^2)}\; dx = \frac{(-1)^\alpha + 1}{2 \cos (\pi\alpha/2)}, \quad \alpha < 1
$$
That's mostly curiosa, more interesting is the fractional moment of the absolute value
$$
  2\int_0^\infty |x|^\alpha \frac1{\pi (1+x^2)}\; dx = \sec(\pi\alpha/2), \quad \alpha<1
$$
Some papers which seems to have more details:  On Distribution of Product of Stable Laws   and  On the Use of Fractional Calculus for the Probabilistic
Characterization of Random Variables     and   MELLIN-BARNES INTEGRALS FOR STABLE
DISTRIBUTIONS AND THEIR CONVOLUTIONS   (I did'nt look much at them yet, but they seem to be relevant)
Answer: The fractional moments $E[x^k]$ do exist for $k<1$. You have to be careful with fractional moments of distributions with negative values, of course. The value $x^k$ may end up being a complex number.
The absolute moments $E[|x|^k]$ were studied in Goria, M. N. "Fractional absolute moments of the Cauchy distribution." Quaderni di Statistica e Matematica applicata alle scienze Economico-Sociali 1 (1992): 3-9.
Comment: As I said elsewhere, this is really the Mellin transform.
Answer: Let $\mu \in \mathbb{R}$ and $\sigma > 0$ be the location and scale parameters of a Cauchy distribution respectively.
Let $r > 0$ and $\theta \in (0, \pi)$ such that  $r\exp(i \theta) = \mu +
\sigma i$, where $i$ denotes the imaginary unit.
Let $-1 < \text{Re}(a) < 1$.
Assume that $x^a = |x|^a \text{sign}(x), \ x \ne  0$ and $0^a = 0$.
Then,  by (3.0.3) in V. M. Zolotarev, One-dimensional stable distributions, AMS,  1986
https://bookstore.ams.org/mmono-65,
$$ E[|X|^a] = \frac{\cos\left(a (\pi /2 - \theta)\right)}{\cos(a\pi /2)} $$
and
$$ E[X^a] = \frac{\sin\left(a (\pi/2 - \theta) \right)}{\sin(a \pi/2)}, \ a \ne 0, $$
and
$$ E[X^a] = 1 - \frac{2\theta}{\pi}, \  a = 0.  $$


Conditional Panel and Radio Button Selection not working

Question: I am trying to use conditional panel based on the input on a Radio button, which is as below
radioButtons("Type","Review Type:",c("Type1","Type2","Type3"),inline=T)

conditional panel statements are as below,
conditionalPanel(condition = "input.Type == 'Type1'",wellPanel(
radioButtons("Question1", "do you know R shiny?"choices = c("Y","N","NA"),selected = "Y",inline=T)

conditionalPanel(condition = "input.Type == 'Type2'",wellPanel(
radioButtons("Question1", "do you know Python?"choices = c("Y","N","NA"),selected = "Y",inline=T)

conditionalPanel(condition = "input.Type == 'Type3'",wellPanel(
radioButtons("Question1", "do you know Java?"choices = c("Y","N","NA"),selected = "Y",inline=T)

if you notice in the conditional panel the radioButton id ("Question1") is same across all the conditional panel, condition as such is working for me, but I have set selected = "Y"for all the radiobuttons which for some reason is not working as desired (it only work on the third conditionalpanel)
Answer: It's behaving this way because all your radioButtons have the same id, so they are considered as only one UI element. Give your radioButtons a unique name and this will work fine:
ui <- fluidPage(
  radioButtons(inputId = "Type",
               label = "Review Type:",
               choices = c("Type1", "Type2", "Type3"),
               inline = T),
  conditionalPanel(condition = "input.Type == 'Type1'",
                   wellPanel(
                     radioButtons(inputId = "Question1",
                                  label = "do you know R shiny?",
                                  choices = c("Y", "N", "NA"),
                                  selected = "Y",
                                  inline = T)
                   )
  ),
  conditionalPanel(condition = "input.Type == 'Type2'",
                   wellPanel(
                     radioButtons(inputId = "Question1",
                                  label = "do you know Python?",
                                  choices = c("Y", "N", "NA"),
                                  selected = "Y",
                                  inline = T)
                   )
  ),
  conditionalPanel(condition = "input.Type == 'Type3'",
                   wellPanel(
                     radioButtons(inputId = "Question1",
                                  label = "do you know Java?",
                                  choices = c("Y", "N", "NA"),
                                  selected = "Y",
                                  inline = T)
                   )
  )
)

Comment: Thank you , I am passing this response to the database, hence had kept the same id, is there any way I can make his work without changing the ID, like an  condition
Comment: I don't think you can. Using the same `id` won't allow you to to select more than one response for "Question1", meaning that if you answer to "do you know R shiny?" and to "do you know Python", you will only have one answer selected.


How can I fully utilise the GPU when training my model in FloyHub?

Question: I am training the below model on FloydHub using Jupyter Notebook. But, whenever I train the model it takes a lot of time(1 minute). The stats below the notebook displays that only 2% of the GPU is utilised.
I have tried running the command torch.cuda.is_available() and it returns True.
import torch
from torch import nn,optim
import torch.nn.functional as F
from torchvision import datasets,transforms

transform = transforms.Compose([transforms.ToTensor(),
                               transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5)),
                               ])
trainset = datasets.MNIST('~/.pytorch/MNIST_data/',download=True,train=True,transform=transform)
trainloader = torch.utils.data.DataLoader(trainset,batch_size=64,shuffle=True)

testset = datasets.MNIST('~/.pytorch/MNIST_data/',download=True,train=False,transform=transform)
testloader = torch.utils.data.DataLoader(testset,batch_size=64,shuffle=True)

class Classifier(nn.Module):

    def __init__(self):
        super().__init__()

        self.hidden = nn.Linear(784,256).cuda()
        self.output = nn.Linear(256,10).cuda()
        self.dropout = nn.Dropout(p=0.2).cuda()

    def forward(self,x):
        x = x.view(x.shape[0],-1).cuda()
        x = self.hidden(x).cuda()
        x = torch.sigmoid(x).cuda()
        x = self.dropout(x).cuda()
        x = self.output(x).cuda()
        x = F.log_softmax(x,dim=1).cuda()
        return x.cuda()

model = Classifier()
model.cuda()
criterion = nn.NLLLoss().cuda()
optimizer = optim.SGD(model.parameters(),lr=0.5)

epochs = 30
training_losses = []
test_losses = []

for e in range(epochs):
    train_loss = 0
    test_loss = 0
    accuracy = 0
    for images,labels in trainloader:
        optimizer.zero_grad()
        output = model(images)
        labels = labels.cuda()
        loss = criterion(output,labels)
        loss.backward()
        optimizer.step()
        train_loss+=loss.item()

    with torch.no_grad():
        # set the model to testing mode
        model.eval()
        for images,labels in testloader:
            output = model(images)
            labels = labels.cuda()
            test_loss+=criterion(output,labels)
            ps = torch.exp(output)
            # get the class with the highest probability
            _,top_class = ps.topk(1,dim=1)
            equals = top_class == labels.view(*top_class.shape)
            accuracy+=torch.mean(equals.type(torch.FloatTensor))

    model.train()

    training_losses.append(train_loss/len(trainloader))
    test_losses.append(test_loss/len(testloader))

    if((e+1)%5 == 0):
        print(f"Epoch:{e+1}\n",
                f"Training Loss:{train_loss/len(trainloader)}\n",
                f"Test Loss:{test_loss/len(testloader)}\n",
                f"Test Accuracy:{(accuracy/len(testloader)*100)}\n\n")

Comment: I'm surprised your code even works, considering everything is on GPU, except the input itself (notice you're never calling `images .cuda()`)
Answer: Three suggestions based on use of MNIST (small dataset):
- pre load your data: i.e. don't use the standard dataloader, but pre-load to .to(cuda), and iterate on this.
- increase your batch size.
- don't use MLPs (linear layers), try CNNs instead.
Comment: Does Pytorch automatically chooses not to fully utilise the GPU because MNIST is a small dataset ?
Comment: No not quite, rather _it doesn't automatically pre-load data into GPU memory_ . So if your dataset is small enough, you should do this manually.


Sampling efficiently in the space of strings?

Question: Let $L = \{a_1,a_2,...a_m\}$ be a set of characters. We have a large population of strings, $S$ ,with strings of size $n$, consisting of the characters in $L$ i.e $S \subseteq L^{n} $. Now given a string in $S$, I can map it to a vector $V_{s} = [|a_1|,...,|a_m|]$ which is the vector that counts occurences of each character in a given string $s$. For example, if $L = \{a_1,a_2,a_3\}$ (m=3) and $n= 5$, then one such $s$ could be $a_1a_1a_2a_3a_1$ and $V_s$ would be $[3,1,1]$.
My goal is that given an arbitrary population of $S$ (strings of a fixed length from a given language $L$), I would like to estimate the probability distribution over the $V_s$ by sampling some of the strings from $S$.
What is the best and most efficient way I can do this? Specially when $n\gg m$, can I sample $s\in S$only partially, i.e. taking only a part of a string?
PS: notice that if $S $= $L^n$  then this can be easily done using multinomial coefficients.
Answer: I'm assuming you have free access to the population of strings and so your only concerns are computational. The only real speed up you can get is by trading memory for speed, which might be relevant when your population is large.
When the strings are long it takes you $O(n) $ to convert from the string to the number you are interested in.
You can, before starting, construct a binary tree where all possible strings are ordered lexicographically. Each leaf of the tree would contain the $V_s$ associated with a string. Traversing the tree is $O(ln (n)) $ which gives you a speed up. Then, it remains only to count how many times you reach each leaf after sampling many many times.
EDIT:
Notice, that if you were interested in how to model the density there is no way to do so theoretically without having some further restriction on the distribution of letters. One such restriction might be to assume letters to be independently (though not necesserily identically) distributed which would greatly simplify the job, as in that case you can recover the $V_s$ density by estimating the probability of single letters.


Acessar item variável no JSON utilizando Node.js

Question: Estou acessando uma API usando Node.js, que me retorna o JSON a seguir.
Preciso acessar campo price, porém o item PETR4: varia de acordo com o symbol informado.
Consigo acessar até response.body.results.
Como posso acessar o próximo item (que nesse exemplo é PETR4)?
{
  by: 'symbol',
  valid_key: true,
  results: {
    PETR4: {
      symbol: 'PETR4',
      name: 'Petróleo Brasileiro S.A. - Petrobras',
      region: 'Brazil/Sao Paolo',
      currency: 'BRL',
      market_time: [Object],
      market_cap: 255916,
      price: 19.31,
      change_percent: -1.63,
      updated_at: '2020-09-29 20:47:36'
    }
  },
  execution_time: 0.01,
  from_cache: false
}

Comment: As vezes vem o campo `PETR4` e as vezes não, é isso?
Comment: O que pode vir além de `PETR4`?
Answer: Você pode usar a função Object.values() para obter os valores de um objeto:

    const resposta = {
      by: 'symbol',
      valid_key: true,
      results: {
        PETR4: {
          symbol: 'PETR4',
          name: 'Petróleo Brasileiro S.A. - Petrobras',
          region: 'Brazil/Sao Paolo',
          currency: 'BRL',
          market_time: [Object],
          market_cap: 255916,
          price: 19.31,
          change_percent: -1.63,
          updated_at: '2020-09-29 20:47:36'
        }
      },
      execution_time: 0.01,
      from_cache: false
    };

    const { price } = Object.values(resposta.results)[0];

    console.log(price);

Object.values()
O método Object.values() retorna um array com os valores das propriedades de um dado objeto, na mesma ordem provida pelo for...in laço (sendo a diferença que o laço for-in também enumera as propriedades na cadeia prototype).
Answer: Provavelmente não é a melhor forma de acessar tal propriedade (inclusive vou acompanhar o post para aprender), mas você pode utilizar um for in para iterar sobre a propriedade PETR4 ou qualquer outra que vier dentro do results.
Exemplo:

const json = {
  by: 'symbol',
  valid_key: true,
  results: {
    PETR4: {
      symbol: 'PETR4',
      name: 'Petróleo Brasileiro S.A. - Petrobras',
      region: 'Brazil/Sao Paolo',
      currency: 'BRL',
      market_time: [Object],
      market_cap: 255916,
      price: 19.31,
      change_percent: -1.63,
      updated_at: '2020-09-29 20:47:36'
    }
  },
  execution_time: 0.01,
  from_cache: false
}

const ativo = json.results
let objName,
    objProperties

for (let i in ativo) {
  objName = i // pega valor da propriedade PETR4
  objProperties = ativo[i] // pega o objeto interno da PETR4
}

console.log(objName) // 'PETR4'
console.log(objProperties) // imprime o objeto interno da PETR4
console.log(objProperties.price) // imprime a propriedade que você precisa


How can I relate the lengths in an analog watch and the angles in sexagesimal system?

Question: The problem is as follows:

In an analog watch; at what time immediately after 6 am, the number of
  minutes elapsed after 6 am is equal to the number of sexagesimal
  degrees the minute hand of the watch is ahead of the hour hand?

I'm puzzled on this situation as I don't know how can I relate the length of the arc and the minutes the minute hand advances. What I found were these relationships.
$1\,\textrm{min}=6 ^{\circ}$
The minutes the hour hand advances equates the following:
$12\,\textrm{min}=30 ^{\circ}$
I'm not sure if to solve this problem is required geometry or trigonometry, but since there is some connection between the arc lengths I'd like if somebody could help me with a drawing or diagram to clearly understand this situation better.
Comment: There are no arc lengths here, just angles.  At six o'clock the minute hand is at $0^{\circ}$ and the hour hand is at $180^{\circ}.$  The minute hand advances at $6^{\circ}$ per minute and the hour hand at $0.5^{\circ}$ per minute.  No trigonometry or geometry involved, really.
Comment: @g.kov Can you please use your expertise in drawing a sketch for this problem please?. That would be very appreciated!. :)
Answer: The angle of the hour hand (in degrees) is $180+0.5m$, where $m$ is the number of minutes elapsed. The angle of the minute hand is $6m$, so the number of minutes the minute hand is ahead of the hour hand is $5.5m - 180$.
Can you take it from here?
Spoiler:

Setting this equal to $m$ should give you your answer:

$$5.5m - 180 = m \\ 4.5m = 180 \\ m = 40.$$

Regarding your questions in the comments, here are some hints:

Can you make a drawing or a sketch so I can understand you better?

Draw an analog clock that shows six o'clock. Measure the angle clockwise starting at the 12. The minute hand is at $0^{\circ}$ and the hour hand is at $180^{\circ}$
Now draw a clock that shows $6:05$. Don't worry about exactly where the hour hand is. The minute hand, now pointing at the 1, makes an angle of $360^{\circ}/12 = 30^{\circ}.$

How do I know the hour hand of the clock advances $0.5^{\circ}$ per minute?

From above, we see that going from the 12 to the 1 is $30^{\circ}.$ The hour hand goes through this angle in one hour. So divide by $60$ to get how far the hour hand goes in a minute.
Comment: Can you make a drawing or a sketch so I can understand you better?. I made this request as part of the proposed answer.
Comment: How do I know the hour hand of the clock advances $0.5^{\circ}$ per minute?. Can you include this as part of your answer please?.
Comment: I edited my answer to address your questions.
Comment: The first expression I get from the problem as states the angle between the minute hand and the hour hand. But the second is not very clear. We differentiate so the total number of minutes that would be transformed into angles is $6m-0.5m=5.5m$ Am i understanding that correctly?. Then the 180 degrees minus that angle would be the same as the angle between the hour hand and the minute hand makes?.
Comment: What the $5.5m$ tells you is that the minute hand gains $5.5^{\circ}$ on the hour hand for each minute that passes. The minute hand starts $180^{\circ}$ behind the hour hand, so that's why it's subtracted. When $5.5m-180=0$, the two hands are on top of one another (zero degrees difference). When the difference equals $m$, then the minute hand is ahead by $m$ degrees.


What are the mean and stdev for a dataset that is approx. lognormally or Weibull-distributed?

Question: Given are particle sizes and corresponding percentages of the total volume. Data:
<code>    size    vol%
    0.594   0.03
    0.675   0.11
    0.872   0.25
    0.991   0.34
    1.28    0.55
    1.45    0.65
    1.88    0.89
    2.13    1.04
    2.75    1.44
    3.12    1.66
    4.03    2.16
    4.58    2.43
    5.92    3.03
    6.72    3.35
    8.68    4.05
    9.86    4.41
    12.7    5.09
    14.5    5.38
    18.7    5.7
    21.2    5.67
    27.4    5.15
    31.1    4.66
    40.1    3.38
    45.6    2.68
    58.9    1.42
    66.9    0.92
    86.4    0.18
    98.1    0.03
</code>
Plot using Excel:

Reading up on particle size distribution on WP, I found that this kind of data usually follow lognormal or Weibull-distributions. So I followed some YT tutorials on checking whether that is the case and arrived at this:

So it's not perfect (2nd is Weibull) but I want to follow through with it if possible. However, when I extract the $\mu$ and $\sigma$ from the equation of the regression line, I don't get the original distribution (plot using WA):

Is my approach/are $\mu$ and $\sigma$ correct?

Could I just take $\frac{1}{n}\sum_1^n x\cdot f(x)$ for $\mu$, and from that then calculate $\sigma$ as I would for a discrete distribution using $\sqrt{(\frac{1}{n}\sum_1^n (\mu-x)^2)}$?
Answer: It's difficult to say whether you are doing the estimation correctly based on what you have provided.  Lognormal model fitting is very sensitive to outliers.  The usual approach for model fitting is to transform the data to the log scale, then fit a normal distribution to the transformed data.  Then the values for $\mu$ and $\sigma$ are then used to calculate the lognormal mean and variance.  If $Y = e^X$ is lognormal where $X$ is normal with mean $\mu$ and variance $\sigma^2$, then
$$\operatorname{E}[Y] = e^{\mu + \sigma^2/2}, \quad \operatorname{Var}[Y] = (e^{\sigma^2} - 1)e^{2 \mu + \sigma^2}.$$
The fit based on a small amount of data is likely to be very poor.  Based on the regression plot you have included, I would say that the data is not lognormally distributed.  If you include a Weibull plot, I would be able to see whether the Weibull fit is actually better.
Comment: I tried to find the corresponding Weibull distribution and added some more data. What is weird is that both recreated distributions flatten out around 35, instead of around 100. So the particle sizes are not correct, while the overall shape looks ok.


What's a "chord register"?

Question: Someone said:

"When you're doing more than one person who does the rhythm part, do
  your chord in another register, maybe higher or maybe lower."

What's really the definition of the "register" on "chord register" and what makes the rhythm part become different/more complex?
Comment: It's part of the general rule - don't double up on what someone else is doing. Keep out of the other guy's way. No-one said "chord register", so don't waste time trying to analyse it!
Comment: On that note, I don't think "chord register" is a thing. The sentence you're quoting uses "chord" and "register" as two different ideas that are related to each other instead of one idea put together.
Comment: "Register" has almost the same meaning as "octave." In other words, play the same chords an octave higher or lower to avoid clashing with the lead part!
Answer: A register in the context in music is very close to and can be thought of as a different way of talking about the range of an instrument, group of notes, ect.  The definition of it is as follows:

In music, a register is the relative "height" or range of a note, set
  of pitches or pitch classes, melody, part, instrument, or group of
  instruments. A higher register indicates higher pitch. - Wikipieda

In the context of guitar and play chords it makes sense if you had two guitarist playing different voicings/shapes of the same chord to fill out the sound for a thicker texture. For example, if one guitarist is playing an open chord which would be in the lower register for a guitar the other guitarist can play up the voicing/shape up the neck to play in a higher register.
Let's look at a practical example. If someone instructed you to play an E chord, there are many ways to do it and just considering the 5 basic chord shapes we could play an E chord the following ways:
<code>%0/.2/.2/.1/.0/.0/[E]
%X/.X/.2/.4/.5/.4/[E]
%X/.7/.6/.4/.5/.4/[E]
%X/.7/.9/.9/.9/.7/[E]
%12/.11/.9/.9/.9/.12/[E]
</code>
As you can see these are all E chords, but the are all a different set of notes and are all in a slightly different register. Have two guitarist play two different chord shapes produces a much fuller sound than the two playing the exact same notes.
Answer: There is little point in two guitarists playing exactly the same chord shape,unless  the rhythm or tones are different. It's to do with the VOICING of chords. If, for example, one player uses an open C( at the head end of the guitar), then it increases the sound for another to play a barre C at the 8th fret. This means that some same notes are played by both, albeit with perhaps a different tone, but extra notes are included. They're all, E and G, but in different octaves, adding a bigger sonic dimension.
Answer: An other view, augmenting what was already said and illustrated.
Talking about "registers" makes a lot of sense for an organ. If you look at an old one in a church, switching registers simply means switching set of pipes.
Pipes can sound like flutes, like trumpets or horns and so on. Some sound loud as a mighty thunder, some silent, and each set has a different sound characteristic, i.e. frequency spectrum. Not all will be in harmony with other "instruments", like the choir or the singing community.
So changing register here means also adapting to the musical situation. Which, for an electric guitar, could be as simple as turning off an effect, switching to a less disturbing one or lowering the volume.
Was said already, viewed from perhaps its historical origin.
Answer: Register is a way of identifying octaves. From C to B the chromatic scale identifies an area of the piano. The guitar starts with E2 and a 22 fret guitar ends with D6. The register starts with C1 to B1 then goes C2-B2, etc.
Two groups of five chromatic pitches are duplicated 5 times. Some pitches are not duplicated some only two, three or four times.
Comment: A register isn't defined absolutely by octave numbering, not to my knowledge. Commonly, a register is just an inexact distinct-sounding region of the frequency spectrum *(I have heard non-musicians misuse "octave" for what musicians call a "register", though)*. Sometimes "register" for an instrument/voice is used more precisely to describe the range of tones created through a specific technique or mechanism, like a saxophone's altissimo register. If you have some information to support the idea that a register starts at C and ends at B as you describe, I'd be interested in reading that.
Comment: I think it's more the other way around.  Octaves can be used to describe the register.  Think of an analogy where register = height, octave = meter.  Saying the first octave is the first register is like saying the first meter is the first height.  But you could say the height is in the 1-2m range (the register is in the 1st octave).  There's probably a better analogy, but that's what I've got.

What's a "chord register"?


Zero Covariance vs Independence of Slope and Intercept Estimators in Linear Models with Least Squares

Question: $\newcommand{\Cov}{\operatorname{Cov}}$Problem Statement: Under the assumptions of Exercise 11.16, find
$\Cov\big(\hat\beta_0,\hat\beta_1\big).$ Use this answer to show that
$\hat\beta_0$ and $\hat\beta_1$ are independent if $\displaystyle\sum_{i=1}^n
x_i=0.$ [Hint: $\Cov\big(\hat\beta_0,\hat\beta_1\big)=
\Cov\big(\overline{Y}-\hat\beta_1\overline{x},\hat\beta_1\big).$ Use Theorem
5.12 and the results of this section.]
Note: This is Problem 11.17 in Mathematical Statistics with Applications, 5th Ed., by Wackerly, Mendenhall, and Scheaffer.
My Work So Far: The assumptions of Exercise 11.16 are that $Y_1, Y_2,\dots,Y_n$ are independent normal random variables with $E(Y_i)=\beta_0+\beta_1 x_i$ and $V(Y_i)=\sigma^2.$ The first part of this question is largely done for us in the book.
That is, it is derived that
$$\Cov\big(\hat\beta_0,\hat\beta_1\big)
=-\frac{\overline{x}\,\sigma^2}{\displaystyle\sum_{i=1}^n(x_i-\overline{x})^2},$$
where $\operatorname{Var}(Y_i)=\sigma^2.$
Now $\overline{x}=0$ if and only if $\sum_{i=1}^n x_i=0.$ So if the sum is
zero, the covariance is zero. However, just because $\hat\beta_0$ and
$\hat\beta_1$ are normally distributed and their covariance is zero does not
make them independent. That would only be true if they were bivariately
normally distributed.
My Questions: Is what I'm being asked to show even true? That is, is there something about $\hat\beta_0$ and $\hat\beta_1$ being OLS estimators that makes this result true? Or can I show that they are bivariate normal distributed? Zero covariance does not imply independence in general; why should it be so in this situation?
Note 1: in silverfish's answer to this question, it is mentioned in the paragraph beginning with "These two uncertainties apply independently..." that these two uncertainties "...should be technically independent." But it is not proven there, though it is intuitively explained and I could believe it.
Note 2: In this thread, Alecos simply makes the argument that I think the book wants here, but doesn't say anything about why zero covariance implies independence.
Note 3: I have reviewed a few other threads related to this, but none of them answers the main question of why zero covariance should imply independence in this situation, when it doesn't in general.
Comment: It seems to me that you have all ingredients together to answer your question: i) if $X$ and $Y$ have a bivariate normal distribution, zero covariance means independence, ii) if we assume normal errors, the least square coefficient estimates are jointly normally distributed (@jld mentions this in their answer). So the book is correct if the errors are assumed to be normal.
Comment: Ah! According to the solutions manual, the book just want's you to calculate the covariance (which you did) and show that it's zero under some circumstances. Yes, "jointly normal" extends to multiple coefficients not just two. It's probably better to say that they have a multivariate normal distribution.
Comment: Yeah I agree: The question probably shouldn't mention independence without making some further comments about the assumptions of the errors.
Comment: Joint distribution of OLS estimators is discussed at several posts, including https://stats.stackexchange.com/q/347628/119261.
Comment: @COOLSerdash Yes, but I don't understand jld's post, I'm afraid. This result is, apparently, supposed to be able to be proven without the linear algebra approach at all, since that comes later in this book. Question: does "jointly normally distributed" mean the same thing as "bivariate normally distributed"?
Comment: @COOLSerdash Right, that's evidently where I'm meant to go. And that's very straight-forward (I've shown how to do that in my question, here). So apparently, the question was poorly worded (for being in its location in the book)?
Comment: Well, actually the book did say that $\operatorname{Var}(Y_i)=\operatorname{Var}(\varepsilon)=\sigma^2.$ But it didn't say anything at all about showing the bivariate distribution.
Answer: $\newcommand{\one}{\mathbf 1}\newcommand{\e}{\varepsilon}$I would just go for a linear algebra approach since then we get joint normality easily. You have $y = X\beta + \e$ with $X = (\one \mid x)$ and $\e\sim\mathcal N(\mathbf 0, \sigma^2 I)$.
We know
$$
\hat\beta = (X^TX)^{-1}X^Ty  \sim \mathcal N(\beta, \sigma^2 (X^TX)^{-1})
$$
where
$$
(X^TX)^{-1} = \begin{bmatrix} n & n \bar x \\ n \bar x & x^Tx\end{bmatrix}^{-1} = \frac{1}{x^Tx - n\bar x^2}\begin{bmatrix} x^Tx/n & - \bar x \\ - \bar x & 1\end{bmatrix}.
$$
By assumption $X$ is full rank, which in this case means $x$ is not constant (since the only way to be low rank is for $x$ to be in the span of $\one$). This means $\det X^TX \neq 0$, so $\text{Cov}(\hat\beta_0, \hat\beta_1) = 0$ if and only if $\bar x = 0$ and we do indeed have bivariate normality so this is equivalent to independence.

Here's a different approach that avoids using the normal equations. We know
$$
\hat\beta_0 = \bar y - \hat\beta_1 \bar x \\
\hat\beta_1 = \frac{\text{Cov}(x,y)}{\text{Var}(x)}
$$
and we want to show $\bar x = 0 \implies \hat\beta_0 \perp \hat\beta_1$, where I'm using "$\perp$" to denote independence.
Without losing any generality I'll assume $x^Tx = 1$ (this preserves $\bar x = 0$). Then under the assumption of $\bar x = 0$ we have
$$
\hat\beta_0 = \bar y = n^{-1}\one^Ty \\
\hat\beta_1 = x^Ty - \bar y x^T\one = x^Ty.
$$
This means
$$
{\hat\beta_0 \choose \hat\beta_1} = (n^{-1}\one \mid x)^Ty
$$
so this is a linear transformation of a Gaussian and is in turn Gaussian, and the covariance matrix is proportional to
$$
(n^{-1}\one \mid x)^T(n^{-1}\one \mid x) = \begin{bmatrix} n^{-1} & 0 \\ 0 & 1\end{bmatrix}
$$
which gives us independence.

This result can be generalized by noting that $\bar x = 0$ is equivalent to having an orthogonal design matrix in this case.
Suppose now we have an $n\times p$ full column rank covariate matrix $X$ which is partitioned as $X = (Z\mid W)$ where $Z$ has orthonormal columns and $W$ is unconstrained.
If every column is orthogonal, i.e. $X=Z$, the result is easy as $X^TX = I$ so
$$
\hat\beta \sim \mathcal N(\beta, \sigma^2I).
$$
I'll prove the following more interesting result: letting $\hat\beta_A$ denote the vector of coefficients for block $A$ of $X$, the elements of $\hat\beta_Z$ are conditionally independent given $\hat\beta_W$.
This can be shown by directly computing the covariance matrix of $\hat\beta_Z \mid \hat\beta_W$ and since $\hat\beta_Z\mid\hat\beta_W$ is still multivariate Gaussian, this gives us independence. I'll take $\sigma^2 = 1$ without losing any generality.
I'll start with the full covariance matrix of $\hat\beta$, which is proportional to $(X^TX)^{-1}$. $X^TX$ is a $2\times 2$ block matrix so we can invert it as
$$
(X^TX)^{-1} = \begin{bmatrix}I & Z^TW \\ W^TZ & W^TW\end{bmatrix}^{-1} = \begin{bmatrix}
I + Z^TWA^{-1}W^TZ & -Z^TWA^{-1} \\
-A^{-1}W^TZ & A^{-1}
\end{bmatrix}
$$
where $A = W^TW - W^TZZ^TW = W^T(I-ZZ^T)W$ gives the covariance matrix of $W$ after projecting all columns into the space orthogonal to the column space of $Z$.
It is not true in general that $I + Z^TWA^{-1}W^TZ = I$, so marginally we are not guaranteed independence in the $\hat\beta_Z$. But now if we condition $\hat\beta_Z$ on $\hat\beta_W$ we obtain
$$
\text{Var}(\hat\beta_Z \mid \hat\beta_W) = I + Z^TWA^{-1}W^TZ - Z^TWA^{-1} \cdot A \cdot A^{-1}W^TZ = I
$$
so we do indeed have conditional independence.
$\square$
Comment: @AdrianKeister ok, I just updated with a slightly different approach. I did still use a tiny bit of linear algebra though. Does this help at all?
Comment: @AdrianKeister I just caught up on all of the comments. Do you feel like your question has been resolved at all or not really? If you really don't want to use any properties of multivariate Gaussians I think this might be an uphill battle...
Comment: Thank you for your answer, jld. That is helpful. However, I have not yet studied the linear algebra approach, so I cannot follow your line of reasoning at all, I'm afraid.
Comment: jld: That's only slightly better, in that I can follow a few more steps before I get lost. What I'm really looking for is an approach using the notation I've introduced (and not a whole lot of new notation), and concepts that I'm already familiar with. I have updated my question to include the book from which I'm learning - my apologies for not doing that earlier. While I am exceedingly familiar with linear algebra in general, its application to statistics is pretty much unknown to me altogether at this point (I hope to rectify that in the future.)
Comment: Yeah, I think so. You're absolutely right about the multivariate Gaussians: that's the crux of the matter. As it is, I think the book's problem statement was unwise for its position in the book. It should have simply said, "Show that the correlation is zero." Happy to mark your answer as the solution.
Comment: @AdrianKeister ok great! And yeah that does sound like a rather unclear problem statement


TimeZone Date Formatting Issue

Question: I'm trying to get a date from a string with multiple time zones, It works without any problems if the API returns zones with abbreviations like CST or UTC but it fails if it returns EET
let timeString = "17:32 (EET)"

let formatter = DateFormatter()
formatter.dateFormat = "HH:mm (zzz)"
let time = formatter.date(from: timeString) // return nil

Any idea what is the issue might be?!
Answer: Probably because there is more than one timezone that matches the timezone abbreviation or the date formatter's default date (January 1st) doesn't match the daylight savings of the timezone abbreviation used. Not all countries uses daylight savings time as well it might change at any time. Check this link. This will probably happen for all non US timezones abbreviation as well. For example CST it is used for "China Standard Time" and "Chicago Standard Time" as well. You can workaround that issue setting your date formatter property isLenient to true. Note that this will result in a date of January 1st 2000 with a probably incorrect timezone offset. If you have control of your string input you should use the timezone identifiers instead of its abbreviations to avoid ambiguity. You should also set your date formatter's locale to "en_US_POSIX" when parsing fixed date format to avoid date formatter's reflecting the users device locale and settings:
let timeString = "17:32 (EET)"
let formatter = DateFormatter()
formatter.locale = .init(identifier: "en_US_POSIX")
formatter.dateFormat = "HH:mm (zzz)"
formatter.isLenient = true
let date = formatter.date(from: timeString) // "Jan 1, 2000 at 1:32 PM" "BRT Brazilian Standard Time"

So you should use "GMT+2" or "GMT-3" to avoid ambiguity or as I have already suggested use its identifiers "HH:mm (VV)" i.e. "Europe/Athens" or "America/Sao_Paulo"
Comment: Perfect.. Thanks again
Comment: Thanks Leo, your answer is very clear. I think the issue I have is more related to the API as the response is different depending on the time zone of the user. So, i was trying to find a solution that works with any time zone without need of fixing the Local of the user time zone.

I will check the availability of sending the time zone of the user in the response if this might help, or if you have any suggestions regarding this point. Thanks again!
Comment: @AhmedM.Hassan You are welcome. As you can see in the link I posted the date formatter does not have enough information to infer the correct timezone as it depends on the country daylight savings time. There is multiple scenarios so the result would be unpredictable. That's why it returns nil. You might set the date formatter's defaultDate as well and a calendar as well to help it parse your date.

TimeZone Date Formatting Issue


Comparing two norms

Question: Let $x$ be a vector or $\mathbb{R}^n$ and let $n_p = \|x\|_p$ and $n_q=\|x\|_q$
denotes the $p$-norm and $q$-norms of $x$. I learn (many years ago and maybe I'm wrong...) that there exists two constants $c$ and $C$ such that whatever $x$, we have $cn_p \leq n_q \leq Cn_p$.

Is it true ?
If it's true, how can I compute $c$ and $C$ given $p$ and $q$ ?
What if $p=2$ and $q=\infty$ ?

Thanks !
Answer: Hint: draw a picture of the unit "circles" for these norms in $\Bbb{R}^2$. This will show you that the $p$-norms are all between the $1$-norm (with unit disc $D_1$: the square with vertices $(\pm 1, 0)$, $(0, \pm 1)$) and the $\infty$-norm (with unit disc $D_{\infty}$: the square with vertices $(\pm 1, \pm 1)$). If you choose the scale factors $c$ and $C$ so that $cD_{\infty} \subseteq D_1$ and $D_{\infty} \subseteq CD_1$, you will get $c n_p \le n_q \le Cn_p$ for any $p$ and $q$, including $p = 2$ and $p = \infty$. The reasoning works for any finite dimension.
Comment: Related, you can use $||x||_{\infty} \leq ||x||_p \leq n^{1/p} ||x||_{\infty}$ for all $p\geq 1$.
Comment: @Michael It's wrong. Take $x=(1,0,\ldots,0)^T$, then $\|x\|_p = 1$ for all $p\geq 1$
Comment: @Serge : I do not understand your comment "It's wrong."  Here I am using $||x||_{\infty} = \max\{|x_1|, |x_2|, ..., |x_n|\}$ and $||x||_p = (\sum_{i=1}^n |x_i|^p)^{1/p}$.  If $x=(1,0,...0)$ then $||x||_{\infty} = ||x||_p = 1$ for all $p \geq 1$. The inequality $||x||_{\infty} \leq ||x||_p \leq n^{1/p}||x||_{\infty}$ can be used to show $\lim_{p\rightarrow\infty} ||x||_p = ||x||_{\infty}$.
Comment: @Michael Yes you're right and I'm (definitively) wrong. Moreover, it is even a tight bound. I was distracted. Sorry
Answer: Yes - it is true. The condition you are asking for says that these two norms are equivalent. Because all norms in $\Bbb R^n$ are equivalent, then such constants do exist. The classical proof shows that an arbitrary norm is equivalent to the maximum norm, which is $\|\cdot\|_{\infty}.$
Comment: I suppose this holds for large values of "equivalent." (+1)
Comment: @Michael: "equivalence" here means that the two norms induce the same topology.
Comment: @RobArthan : Thanks.  I also googled "equivalent norm" and found google defined it in terms of differing by at most multiplicative constants.  Your explanation is better justification about why the word "equivalent" might be used.
Comment: @RobArthan The name *equivalent norms* or (more general) *equivalent metrics* is the folklore and it is taught on any anaalysis lecture. :) So I felt to be allowed to use this name and I hoped to be understood by everybody.
Comment: @szw1710: what did I do wrong?
Comment: @RobArthan Everything OK, I wanted to point out that this is rather standard name. :-) My best regards.

Comparing two norms


Fazer com que o VS Code encontre módulo instalado por comando Pip no linux

Question: Instalei o matplotlib por sudo apt-get e o VS Code conseguiu encontrar a biblioteca automaticamente, entretanto com o pyserial pelo comando pip, o VS Code não consegue encontrar, como devo proceder?
Comment: Não é necessário instalar o VirtualEnv não. Só perguntei porque isso costuma acontecer quando temos diferentes ambientes python com diferentes versões.

Verifique se você não está instalando a biblioteca referenciando a versão antiga(nativa do sistema) do Python instalada em sua máquina.

Instale utilizando o pip3:

pip3 install pyserial

E depois, import serial no interpretador para testar.
Comment: Você instalou o VirtualENV (para criar diversos ambientes de desenvolvimento) ou está usando apenas uma única versão do Python no seu único ambiente principal (linux)? Qual versão do Python está utilizando? *(Certifique-se de estar conectado ao ambiente corrente em que fez a instalação da lib.)* Você testou a chamada ao módulo? *Dentro do VScode, abra o terminal e o interpretador Python. Feito isso, digite ***import serial*** para testar a existência do módulo.*
Comment: Não instalei o VirtualEnv, pois nem sabia que era necessário, mas verificarei, o Elementary OS já me instalou o Python 3 e 2, eu já testei as chamadas de import, porém só a serial que falha.
Answer: Aparentemente o comando pip instalou em relação ao Python 2, então executei o
sudo apt-get install python3-pip

Em seguida executei pip específico para Python 3
pip3 install pyserial

Então VS Code reconheceu automaticamente.


Retirar os excessos de espaços em branco de uma string

Question: Preciso fazer um programa que leia um arquivo de texto, copie as informações e repasse para outro arquivo, mas sem as linhas em branco, sem comentarios de codigo (que inicia em "//") e remover excesso de espaços em branco entre palavras.
A funcao le_limpa_e_salva faz seu trabalho, mas a funcao limpa_espacos esta retornando strings vazias, e por isso, ta criando arquivo novo vazio. E eu nao sei pq ta fazendo isso. alguem ajuda por favor.
#include <iostream>
#include <fstream>

using namespace std;

void limpa_espacos(string &str)
{
    int p1= 0; // ponteiro para string de entrada
    int p2= 0; // para string de saida
    string temp; // guarda a string a medida que é limpa

    while(str[p1]!='\0') // varre a string ate o fim
    {
        temp[p2]= str[p1]; // copia uma posicao da string de entrada para temp

        if(str[p1]==' ') // se essa posicao é um espaco
        {
            while(str[p1+1]!='\0' && str[p1+1]==' ') // enquanto nao chegar no fim da string e as
                                                     // proximas posicoes continuam sendo espacos
            {
                p1++; // vai empurrando o ponteiro da string de entrada
            }
        }

        // senão, apenas incrementa ambos os ponteiros, ou incrementa de qualquer forma
        p1++;
        p2++;
    }

    temp[p2]= '\0'; // fecha a string
    str= temp;
}

void le_limpa_e_salva (ifstream &lod, ofstream &sav, string st)
{
    // enquanto tiver uma linha no arquivo com caracteres, esse "while" vai rodar
    while (getline(lod, st)) // faz leitura de uma linha do arquivo
    {
        if(!st.empty() && (st[0] != '/' && st[1] != '/')) // se a linha nao estiver vazia
        {
            cout << "\nantes de limpar espacos: " << st << "\n";
            limpa_espacos(st);
            cout << "\ndepois de limpar espacos: " << st << "\n";
            sav << st << "\n"; // grava essa linha no novo arquivo e pula para linha de baixo
        }
    }
    lod.close();
    sav.close();
    cout << "\n\nArquivo copiado com sucesso!\n\n";
}

int main()
{
    ifstream load;
    ofstream save;
    string str, apaga;

    // arq_orig: guarda o nome do arquivo digitado pelo usuario (arquivo fonte)
    // arq_dest: guarda o nome do arquivo digitado pelo usuario (arquivo de destino)
    char arq_orig[1023], arq_dest[1023];

    cout << "Digite o nome do arquivo a ser copiado, com sua extensao.\n";
    cout << "Digite: ";

    // muito melhor que colocar no codigo, o local do arquivo a ser lido
    // e o local do novo arquivo a ser criado, pois para cada execucao
    // em computador diferente, esse trecho do codigo deveria ser alterado.
    // Alem disso, ha diferenca de formatacao desse caminho para Windows e Linux.

    gets(arq_orig); // "pega" o nome digitado pelo usuario

    load.open(arq_orig); // lê o arquivo com esse nome

    if(load) // verificacao de erro
    {
        cout << "Digite o nome do arquivo que recebera o texto copiado, com sua extensao.\n";
        cout << "Digite: ";

        gets(arq_dest);

        save.open(arq_dest);

        if (save)
        {
            le_limpa_e_salva (load, save, str);

            load.close();
            save.close();
        }
        else
        {
            cout << "\n\nFalha ao criar arquivo de destino! Fechando programa.\n\n";
        }
    }
    else
    {
        cout << "\n\nFalha ao abrir arquivo fonte! Fechando programa.\n\n";
    }

    return 0;
}

Comment: Problema resolvido.
Answer: Olá!
Você escreveu algo como C e não C++. Para copiar os valores para a string de saída use '+' que é redefinido para strings e faz o que você quer.
       temp[p2]= str[p1]; // C, mas temp deve ter sido alocado antes

Em C já deveria haver memória alocada para a nova string e aí se pode copiar. Em C++ use
       temp += str[p1]; // C++, aloca conforme preciso

Mas considere essa função
string      trim(string& str)
{
    string nova{};
    char* p = &str[0];
    char branco = 0;
    while( *p!=0)
    {
        if ( *p == ' ')
            if ( branco == 0 )
            {
                branco = 1;
                nova += *p;
            }
            else
                ;
        else
        {
            branco = 0;
            nova += *p;
        }
        p++;
    };  // while()
    return nova;
}

que faz o que você quer e é mais flexível porque retorna uma string e pode usar em uma expressão como
    const string teste[] =
    {
        "  teste    t e s    te  ",
        "azul",
        "  azul",
        "azul  ",
        " azul",
        "azul "
    };

    for ( auto str : teste )
        std::cout << "\nAntes: '" << str <<
            "' Depois: '" << trim(str) << "'\n";

saida
PS C:\src\CPP\flt> g++ -o tst -Wall -std=c++17 tle.cpp
PS C:\src\CPP\flt> ./tst

Antes: '  teste    t e s    te  ' Depois: ' teste t e s te '

Antes: 'azul' Depois: 'azul'

Antes: '  azul' Depois: ' azul'

Antes: 'azul  ' Depois: 'azul '

Antes: ' azul' Depois: ' azul'

Antes: 'azul ' Depois: 'azul '
PS C:\src\CPP\flt>

deste exemplo
#include <iostream>
using namespace std;

string      trim(string& str);

int main(void)
{
    const string teste[] =
    {
        "  teste    t e s    te  ",
        "azul",
        "  azul",
        "azul  ",
        " azul",
        "azul "
    };

    for ( auto str : teste )
        std::cout << "\nAntes: '" << str <<
            "' Depois: '" << trim(str) << "'\n";

    return 0;
}

string      trim(string& str)
{
    string nova{};
    char* p = &str[0];
    char branco = 0;
    while( *p!=0)
    {
        if ( *p == ' ')
            if ( branco == 0 )
            {
                branco = 1;
                nova += *p;
            }
            else
                ;
        else
        {
            branco = 0;
            nova += *p;
        }
        p++;
    };  // while()
    return nova;
}
// main.cpp

Alternativa: use uma máquina de estados
Como seu programa é um filtro e passa uma vez só pela entrada, uma máquina de estados é algo muito mais simples. E leia o arquivo como uma sequência de letras. Eis os estados:

lendo um valor
depois de ler um espaço
depois de ler uma /
depois de ler um espaço adicional
depois de ler um segunda /
ler um \n
fim de arquivo

Não precisa de funções para nada se usar isso. Se precisar escreva e mostro um exemplo


Custom template for specific vocabulary term

Question: Hi guys i hope you can help me with this, i made a vocabulary called Colors and inside this i have the terms: red, green, blue.
I was wondering, how can i make custom templates for each term, for example, for mysite.com/colors/red i want a different template than for mysite.com/colors/blue
I saw that it's possible for nodes, but don't know if it's possible for taxonomies. How can i do this?
Edit: for Drupal 7
Comment: which drupal version?
Comment: Drupal 7 version
Answer: You can go with the following:
taxonomy-term--[term id].tpl.php You can get the term id number when you hover the edit button.
Comment: Thanks, i have to do anything more? add an extra function to template.php maybe?
Comment: thanks, it worked, just had to empty the cache. But there is a problem is not completely overrode, how can i make the template manually without displaying the text "No content for this term" ?


I'd like to find the formula for this triangle number sequence.

Question: This is a numeric triangle that came up in a problem I'm trying to solve. I got this sequence using the brute force approach (Javascript):
function f(s, b){
  if(s == 0 || b == 0) return 0;
  if(s > b) return 0;
  if(s == 1 || b == 1 || b == s) return 1;

  let sum = 0;
  for(let i=0; i<Math.min(s, b); i++){
    sum += (i + 1) * f(s-i, b-s);
  }

  return sum;
}

Result for the first few numbers (10 x 10).
0 0 0 0 0 0 0 0 0 0
0 1 1 1 1 1 1 1 1 1
0 0 1 2 3 4 5 6 7 8
0 0 0 1 3 5 8 12 16 21
0 0 0 0 1 4 7 12 20 30
0 0 0 0 0 1 5 9 16 28
0 0 0 0 0 0 1 6 11 20
0 0 0 0 0 0 0 1 7 13
0 0 0 0 0 0 0 0 1 8
0 0 0 0 0 0 0 0 0 1

I'm not very good at combinatorics (or math in general). I'd like to understand how I can change the brute force approach so that I can get the sequence without iterating all numbers from 0 to min(s, b). I've attempted building the sequence in many ways, but it's mostly by trying random multiplications and sums, so I don't think I'll end up having the correct formula soon with this approach.
Background (what I'm trying to achieve):
We are trying to build a structure that's made out of blocks. Each structure has s stacks of blocks, and the total number of blocks is b. The structure must be convex, not concave. The goal is to count how many different structures for each pair of s and b values can be built. For example, if the stack number is the same as the block number, we can only build one, because all blocks are being used for the base. If the block count is less than the stack count, we can build none. If s < b, then there are many ways to build a convex structure.
Comment: That looks like https://oeis.org/A072704.
Comment: You seem to have come across the sequence at https://oeis.org/A072704 , as @MartinR has observed. I suspect there is no non-recursive formula because if there were one it would be likely to appear there.
Comment: @MattiP. We are trying to build a structure that's made out of blocks. Each structure has `s` stacks of blocks, and the total number of blocks is `b`. The structure must be convex, not concave. The goal is to count how many different structures for each pair of `s` and `b` values can be built. For example, if the stack number is the same as the block number, we can only build one, because all blocks are being used for the base. If the block count is less than the stack count, we can build none. If `s < b`, then there are many ways to build a convex structure.
Comment: You don't sum them up!  You read your numbers (the non-zero ones) one column at a time.  (This isn't obvious if you haven't wasted too much time looking at OEIS.)
Comment: @MichaelLugo holy cow, now I see it! It's exactly the same. Thanks!
Comment: @ChrisVilches The illustration at https://oeis.org/A001523/a001523.gif might help clarify that OEIS actually refers to the same thing as you, and not just some other thing that happens to have the same count.  (In that image they have added up the columns, to get 1, 2, 4, 8, 15, 27, ...)
Comment: Would you have some context for the function $f$ ? What need does it fulfill? I'm looking for an answer along the lines of "with the function f I'm trying to solve a problem where ... "
Comment: @ChrisVilches Thank you for the clarification, this is actually a critical clarification in my opinion.
Comment: @MichaelLugo Thanks. I don't really understand how my sequence relates to the one you and Martin posted. I tried to sum up some values but I don't get the same numbers.
Answer: I was able to solve it. It requires two functions. I'm not sure about the function naming, and maybe it can be refactored into one function, but it works this way, and I was able to bring the complexity from $O(n^3)$ to $O(n^2)$, so I'm happy with it.
This is the C++ code:
ll dp(int s, int b) {
  if (s == 0 || b == 0) return 0;
  if (s > b) return 0;
  if (s == 1 || b == 1 || b == s) return 1;
  if (memo[s][b] != -1) return memo[s][b];
  ll result = dp(s, b - s) + 2 * dp(s - 1, b - 1) - dp(s - 2, b - 2);
  return memo[s][b] = result;
}

int calculate(int s, int b) {
  if (s == 1 || b == 1 || b == s) return 1;
  ll sum = 0;
  for (int i = 0; i < min(s, b); i++) {
    sum += (i + 1) * dp(s - i, b - s);
  }
  return (int)sum;
}

Then get the desired value (read the question statement to understand the problem) as calculate(3, 6) (i.e. ways to create a convex structure with 3 stacks and a total of 6 blocks).
Note: ll is typedef long long int ll;. The memo variable is for memoization.


How the embedding layer is trained in Keras Embedding layer

Question: How is the embedding layer trained in Keras Embedding layer?
(say using tensorflow backend, meaning is it similar to word2vec, glove or fasttext)
Assume we do not use a pretrained embedding.
Comment: http://colah.github.io/posts/2014-07-NLP-RNNs-Representations -> this blog post explains clearly about **How the embedding layer is trained in Keras Embedding layer**. Hope this helps.
Answer: Embedding layers in Keras are trained just like any other layer in your network architecture: they are tuned to minimize the loss function by using the selected optimization method. The major difference with other layers, is that their output is not a mathematical function of the input. Instead the input to the layer is used to index a table with the embedding vectors [1]. However, the underlying automatic differentiation engine has no problem to optimize these vectors to minimize the loss function...
So, you cannot say that the Embedding layer in Keras is doing the same as word2vec [2]. Remember that word2vec refers to a very specific network setup which tries to learn an embedding which captures the semantics of words. With Keras's embedding layer, you are just trying to minimize the loss function, so if for instance you are working with a sentiment classification problem, the learned embedding will probably not capture complete word semantics but just their emotional polarity...
For example, the following image taken from [3] shows the embedding of three sentences with a Keras Embedding layer trained from scratch as part of a supervised network designed to detect clickbait headlines (left) and pre-trained word2vec embeddings (right). As you can see, word2vec embeddings reflect the semantic similarity between phrases b) and c). Conversely, the embeddings generated by Keras's Embedding layer might be useful for classification, but do not capture the semantical similarity of b) and c).

This explains why when you have a limited amount of training samples, it might be a good idea to initialize your Embedding layer with word2vec weights, so at least your model recognizes that "Alps" and "Himalaya" are similar things, even if they don't both occur in sentences of your training dataset.
[1] How does Keras 'Embedding' layer work?
[2] https://www.tensorflow.org/tutorials/word2vec
[3] https://link.springer.com/article/10.1007/s10489-017-1109-7
NOTE: Actually, the image shows the activations of the layer after the Embedding layer, but for the purpose of this example it does not matter... See more details in [3]
Comment: This is exactly the explanation I was looking for! I think this sentence should be in bold: "Remember that word2vec refers to a very specific network setup which tries to learn an embedding which captures the semantics of words."
Comment: @nice answer, you answer here and answer [here](https://stats.stackexchange.com/questions/270546/how-does-keras-embedding-layer-work) helped me a lot!
Answer: The embedding layer is just a projection from discrete and sparse 1-hot-vector into a continuous and dense latent space. It is a matrix of size (n,m) where n is your vocabulary size and m is your desired latent space dimensions. Only in practice, there's no need to actually do the matrix multiplication, and instead you can save on computation by using the index. So in practice, it is a layer that maps positive integers (indices corresponding to words) into dense vectors of fixed size (the embedding vectors).
You could train it to create a Word2Vec embedding by using Skip-Gram or CBOW. Or you can train it on your specific problem to get an embedding suited for your specific task at hand. You could also load pre-trained embeddings (like Word2Vec, GloVe etc.) and then continue training on your specific problem (a form of transfer learning).
Answer: one way to envision the implementation (not sure this is how Keras/Pytorch or other frameworks implement it), such that the weight could be "learned" by back propagation, is to map the Embedding Matrix (n_vocab x n_emb_dim) as the weight between the input layer where word/token are one-hot encoded, and the embedded-layer where each token is represented as a n_emb_dim vector.
Given that the One-hot layer dimension is n_vocab, and the embedded layer dimension is n_emb_dim, the number of weights for the full connections between these two layers is n_vocab x n_emb_dim, which is the same as the "lookup table of weights".
This is only for one token, what if our input is a sequence of token, like a sentence? We could easily duplicate and concatenate the embedding layer vertically, with weight-sharing, so the weights count remain the same regardless of the sequence length.


What method should I use to study population level growth and individual level growth?

Question: I have a panel dataset which contains diameter measurement data for 88 trees obtained from a mapped stand in Duke Forest. The description of variables is:
Each tree is indexed by a unique ID (ID), and the diameter of a given year (yr) is the variable cm(cm). For each year, three variables of weather information are also available: annual precipitation (annualprec), average summer Palmer Drought Severity Index (summerpdsi), and average winter temperature (wintertemp).
Now I want to study both stand (population) level growth and individual level growth. I currently use fixed effect and random effects model. These two models allow me to investigate the effects of the 3 weather variables. But my main focus should be stand level growth and individual level growth. I don't know what method I should use to attain this goal.
The dataset can be seen here: http://www2.stat.duke.edu/~lm186/data/diamdata.txt
The description of this data set is:
Tree growth provides essential information about forest ecology. One common method to estimate tree growth is based on repeated tape measurements of the diameter of the same tree, and the diameter increment is the difference between the current and previous measurement. The dataset "diamdata.txt" contains diameter measurement data for 88 trees obtained from a mapped stand in Duke Forest. The stand was established in 1991 for the purpose of studying forest dynamics. The measurements are made at breast height marked by a nail that holds a tag indicating the identifying tree number. Diameter censuses were conducted at intervals of one to four years starting in 1993. Each year, some trees died and were removed from the census, and some new trees were planted and added to the census, resulting in different numbers of trees measured in each census and different numbers of measurements for each tree. Each tree is indexed by a unique ID (ID), and the diameter of a given year (year) is the variable cm. For each year, three variables of weather information are also available: annual precipitation (annualprec), average summer (Jun. - Sep.) Palmer Drought Severity Index (PDSI) (summerpdsi), and average winter (Jan. - Mar.) temperature (wintertemp). Explore and analyze the data to infer about the pattern of tree growth over time. In this regard, we may be interested in learning about both stand (population) level growth and individual level growth. Write a report of up to three pages regarding tree growth based on your analysis that is understandable and useful to ecologists. Details of key statistical methods or models should be given.
Comment: Welcome (if inadvertently) to Cross Validated! To get a helpful answer, please edit the question to show the details of the model(s) that you have used so far. In particular, what are you specifying as random effects? How many stands are there? Are there typically similar numbers of trees in each stand? How are you modeling time? How are you dealing with the large differences in baseline `cm` values for different trees when you want to estimate individual-level growth? Please provide that information by editing the question, as comments are easily overlooked and can be deleted.
Comment: @EdM I added the data description.
Comment: It seems that this is a course assignment or a self-study project. If so, please add the [`self-study` tag](https://stats.stackexchange.com/tags/self-study/info) to the question, read the policy about homework and similar questions on [this page](https://stats.stackexchange.com/help/on-topic), and look at the links for how we handle such questions on this site. It also would help if you could show in the question the actual model or models that you have tried so far. Use the `{}` code tool to enter code in a way that's easy to read and copy.
Answer: There are several ways to evaluate data on individuals (trees, here) over time, discussed for example in Chapter 7 of Frank Harrell's course notes. A simple fixed-effects model isn't appropriate, as that wouldn't take into account the correlations of changes within trees. This data set illustrates strengths, weaknesses, and limitations of different approaches to modeling longitudinal data. As you are studying these issues, consider trying several different approaches.
You evidently have chosen a mixed model with the 3 weather variables as fixed effects, and handled the intra-tree correlations by using tree ID as a random effect (I suspect just as random intercepts). To that extent, you've already started to evaluate the associations of those variables with growth at the stand level, because there is only 1 stand to evaluate. Coefficients associating growth with fixed-effect weather variables will thus represent associations at the stand level. There's an alternate approach that you might consider, generalized least squares, to modeling data over time like these. Here that would give marginal estimates of covariate effects for this stand, while ignoring tree-level effects. Harrell's notes illustrate that approach.
It's not clear how you have incorporated time into the model, if you've modeled the diameter values (or some transformation, like a log) directly as a function of time, or evaluated the change per year in diameter as a function of the weather variables. Harrell discusses different ways to incorporate time; a flexible spline fit for time might be a good choice. It's important to make sure that your combination of time and weather variables makes sense in a causal direction. You want to make sure that the data are formatted in a way that the weather variables are measured just before or during the periods of growth, not after. For example, make sure the wintertemp values are for the winter prior to measurements.
Evaluating individual-level growth can be tricky with data like these. Look at your tree-by-tree data first, with plots of diameter against year. You have no more than 7 observations on any one tree, which greatly limits your ability to evaluate the effects of any predictors on a tree-by-tree basis. You also have several small trees evidently planted in 2007 or 2008, while most have data dating back many more years. Think about whether those more recently planted trees should be modeled differently from the others.
You could consider using modeling the distribution of responses to fixed effects among trees by incorporating random slopes, not just intercepts, into your mixed model. Playing with your data suggest that incorporating some random slopes is possible here, without running into singular-fit problems, particularly if you log-transform your diameter values first.
You'll have to apply your understanding of the subject matter (which I don't have) to describe the results in a way "that is understandable and useful to ecologists."


add `DEBUG=1` commands to windows?

Question: I'm trying to use kaki
library with kivy and python but to use it you need to run
DEBUG=1 python main.py

but I'm facing this error
DEBUG=1 : The term 'DEBUG=1' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that
the path is correct and try again.
At line:1 char:1
+ DEBUG=1 python main.py
+ ~~~~~~~
    + CategoryInfo          : ObjectNotFound: (DEBUG=1:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

but I can find any source about how to add this command to my path
I'm using
python:3.7
windows 10
I will add any information if need it
Comment: Did you try with `set DEBUG=1` and then launching `python main.py`?
Comment: how I can do that any hint ?
Answer: In PowerShell, you can write to an environment variable which will be inherited by child processes, like this:
PS ~> $env:DEBUG = 1
PS ~> python main.py

Or as two statements on a single line:
$env:DEBUG = 1; python main.py

Python will now get the value 1 when resolving the DEBUG environment variable
In cmd.exe you can use the set keyword:
C:\> set "DEBUG=1" && python main.py

Comment: its possible to do this without using PowerShell because it difficult to use
Comment: actually PowerShell is much easier to use because it doesn't have the legacy quirks in cmd. @MathiasR.Jessen that won't work because spaces in cmd are important. You need `set "DEBUG=1" && python main.py`

add `DEBUG=1` commands to windows?


For every integer $n>1$ , does there exist a diagonal matrix $D \in M(n,\mathbb R)$ such that $AD=DA $ holds only if $A$ is diagonal?

Question: Is it true that for every integer  $n>1$ , there exist a diagonal matrix $D \in M(n,\mathbb R)$ such that $A \in M(n,\mathbb R)$ and $AD=DA \implies A$ is also a diagonal matrix ?
Answer: $\newcommand{\Set}[1]{\left\{ #1 \right\}}$Hint
Prove and use the following lemma

Let $D$ be a matrix, $\lambda$ an eigenvalue of $D$, and
  $$V_{\lambda} = \Set{ v : D v = \lambda v}$$
  the corresponding eigenspace. If $A D = DA$, then
  $$
A V_{\lambda} \subseteq V_{\lambda}.
$$
Comment: Because it sends each basis vector into a multiple of it.
Comment: Ok, I can easily prove that lemma , but how do I use it here ?
Comment: Take $D$ to have **distinct** values on the main diagonal. What are the eigenvalues and the eigenspaces of $D$ then?
Comment: the eigen values of $D$ are then just the diagonal entries of $D$ and for each eigenvalue the eigen-space is the span of one of the standard basis vectors ...
Comment: And then by the Lemma any matrix $A$ that commutes with $D$ has to send each of these eigenspaces to itself. Which means, $A$ is itself diagonal.
Comment: I don't get that why then $A$ has to be diagonal ..