What's the difference between Rao-Blackwell Theorem and Lehmann-Scheffé Theorem?

Question: I know that the Rao-Blackwell theorem states that an unbiased estimator given a sufficient statistic will yield the best unbiased estimator. Is the only difference between Lehmann-Scheffé and Rao-Blackwell that in Lehmann-Scheffé, you need an unbiased estimator that is based on a complete sufficient statistic? I am also having a hard time conceptually understanding the definition of a complete statistic.
Comment: from wiki (but maybe it wasn't on wiki at the time of this post): [Rao–Blackwell theorem - Completeness and Lehmann–Scheffé minimum variance](https://en.wikipedia.org/wiki/Rao%E2%80%93Blackwell_theorem#Completeness_and_Lehmann%E2%80%93Scheff%C3%A9_minimum_variance)
Answer: Rao–Blackwell says the conditional expected value of an unbiased estimator given a sufficient statistic is another unbiased estimator that's at least as good.  (I seem to recall that you can drop the assumption of unbiasedness and all you lose is the conclusion of unbiasedness; you still improve the estimator.  So you can apply it to MLEs and other possibly biased estimators.)  In examples that are commonly exhibited, the Rao–Blackwell estimator is immensely better than the estimator that you start with.  That's because you usually start with something really crude, because it's easy to find, and you know that the Rao–Blackwell estimator will be pretty good no matter how crude the thing you start with is.
The Lehmann–Scheffé theorem has an additional hypothesis that the sufficient statistic is complete, i.e. it admits no unbiased estimators of zero.  It also has an additional conclusion: the estimator you get is the unique best unbiased estimator.
So if an estimator is complete, unbiased, and sufficient, then it's the best possible unbiased estimator.  Lehmann–Scheffé gives you that conclusion, but Rao–Blackwell does not.  So the statement in the question about what Rao–Blackwell says is incorrect.
It should also be remembered that in some cases it's far better to use a biased estimator than an unbiased estimator.
Comment: Concerning my last comment above, that it's sometimes far better to use biased than unbiased estimators: This seems not to be too widely known among non-statisticians.  I wrote a paper about it, devoted largely to my own concrete example of that phenomenon: http://www.math.umn.edu/~hardy/An_Illuminating_Counterexample.pdf
Comment: The link above no longer works. Here's one that does: https://arxiv.org/abs/math/0206006
Comment: Thanks for the comment and answer. It cleared up some conceptual issues I had.
Comment: @Michael: (+1) Thanks for that link to your note. It's an interesting example.  I thought it curious that you opted to constrast the unbiased estimator of the variance with the MLE, when in the particular case you chose, the optimal choice would be to use a denominator of $n+1$. Also, what motivated the "light source" problem originally?
Comment: Actually I thought only that MLEs are sometimes biased, so I mentioned them as an example.  In cases often cited as examples, the MLE is usually a sufficient statistic, so in that sense it's not a good example (but in some cases the MLE is not sufficient and one could apply Rao-Blackwell).  I don't remember exactly what led me to the light-source problem, but it was while taking a graduate course that covered some related matters that I came up with that.
Comment: Related? [Is optimal solution to dual not unique if optimal solution to the primal is degenerate?](https://math.stackexchange.com/questions/1095760/is-optimal-solution-to-dual-not-unique-if-optimal-solution-to-the-primal-is-dege)
Answer: As Michael Hardy said, Rao-Blackwell only guarantees to improve (or not hurt) your original unbiased estimator. I.e. you start with some unbiased estimator $T(\underline x)$, then you improve it by taking the expected value conditioned on a sufficient statistic $T'(\underline x) := \mathbb{E}[T(\underline x) | S(\underline x)]$. Improve meaning that it's variance will be less or equal than the original estimator variance.
But something funny happens if you add the completeness to the statistic ($S(\underline x)$) - you get uniqueness. I.e. there will only be one unbiased estimator that will be be a function of $S$. And so if you start with some unbiased estimator, and use the Rao-Blackwell method with a sufficient & complete statistic, the result will be the only unbiased estimator for that parameter, and it will have a better variance than any of the original estimator. So it's (U)MVUE (Uniform-ly minimal variance unbiased estimator).
Completeness is a restriction for a statistic that if $\mathbb{E}(g(\underline x)) = 0$ then $g(\underline x) = 0$ always, with probability 1 for any $x$. Take a Bernoulli distribution - and $g(\underline x) = x_1 - x_2$. $\mathbb{E}(x_1 -x_2) = 0$ but $g(x) \neq 0$ when there are different results between the first and second experiment. It's only equal zero when they are the same. So it's not a complete statistic. $h(\underline x) = \sum x_i$ however is a complete statistic, as if it's expected value is equal to zero, it means that it itself is equal to zero with probability of 1.


Taking limits with respect to the Hausdorff metric does not preserve measure

Question: In this recent paper, in the introduction, the authors say:

"Disproportionate divisions are easily seen to exist when all the demands are rational...  However, this does notimmediately demonstrate existence for irrational demands, since taking limits with respect to the Hausdorff metric, for example, does not preserve measure".

The context is a partition of the unit interval $[0,1]$ into sub-intervals. I am trying to understand the emphasized sentence. Does it mean that there exists an infinite sequence of subsets of $[0,1]$, say $X_1,X_2,\ldots,$, and another subset $X\subseteq [0,1]$, and a measure $\mu$, such that:

The Hausdorff distance between $X_n$ and $X$ approaches $0$ when $n\to\infty$, but -
The measure $\mu(X_n)$ does not approach the measure $\mu(X)$ when $n\to\infty$?

If so, what is an example of such a sequence?
Comment: @TheoreticalEconomist sorry, I forgot to clean up, fixed.
Answer: In $[0,1]$ let $X_n$ be the finite set
$$
X_n = \left\{\frac{k}{n} : 0 \le k \le n\right\}
$$
Then $X_n$ converges to $[0,1]$ in the Haudsdorff metric, even though each $X_n$ has measure $0$ and $[0,1]$ has measure $1$.


Evaluate: $\int_0^{2a} y \sin^{-1}\sqrt{\frac{1}{2} -\frac{1}{4} y} dy$

Question: How to evaluate
$$\int_0^{2a} y \sin^{-1}\sqrt{\frac{1}{2} -\frac{1}{4} y} \ dy$$
Thanks for any help.
Comment: Try integration by parts
Answer: May be, you could make life a bit simpler.
Let
$$\sqrt{\frac{1}{2}-\frac{y}{4}}=t \implies y=2 \left(1-2 t^2\right)\implies dy=-8t\,dt$$
So,
$$\int y \sin ^{-1}\left(\sqrt{\frac{1}{2}-\frac{y}{4}}\right)\,dy=16\int  t \left(2 t^2-1\right) \sin ^{-1}(t)\,dt$$
Now, as @MathIsFun commented, "try integration by parts".


SSL Privacy error with Wpengine staging sites

Question: We have a staging site setup with Wpengine and since the Wordpress 4.0 update, all pages that require SSL are refusing to connect.
Error from Chrome:

Your connection is not private Attackers might be trying to steal your
  information from your-site.staging.wpengine.com (for example,
  passwords, messages, or credit cards).

Error from Firefox:

This Connection is Untrusted You have asked Firefox to connect
  securely to your-site.staging.wpengine.com, but we can't confirm that
  your connection is secure. Normally, when you try to connect securely,
  sites will present trusted identification to prove that you are going
  to the right place. However, this site's identity can't be verified.

Is anyone else experiencing the same issue or found a workaround? In Chrome, you can click Advanced > Proceed and still see the site just with SSL errors.
Comment: Unless you purchase a certificate for your-site.staging.wpengine.com then that's going to happen no matter what. It's using a self-signed certificate, which will always show that error since it's not issued by a "trusted authority"
Answer: Doyle Lewis has the correct answer, you are seeing the errors because the domain your staging site is on does not have a signed SSL certificate.
When you use WP Engine's staging service your site is mirrored on a temporary domain which is provided by WP Engine. Most likely what is going on here is that your site is requiring an SSL certificate and the one it's expecting would be for your normal domain (not 'your-site.staging.wpengine.com').
It's probably not a serious issue and should be safe to just click Advanced > Proceed to view the site.
If you're concerned about security then more info is needed. If I was in your position and concerned I would contact WP Engine support staff, but I imagine they will tell you the same thing.
Comment: Thank you for providing additional info on this. There was a temporary issue with the Wordpress 4.0 update that has since been resolved by WPEngine.
Answer: I've just seen similar.
It is not self signed.
It is part of the site.
They use "*.wpengine.com" wildcard certificate.
The URL is ".staging.wpengine.com"
Wildcards in certificates only match one level of sub-domain, so this is the wrong certificate to be using, they need to use "*.staging.wpengine.com" here.
Answer: The staging site isn't for testing your SSL because you're on a staging address, where you have zero control of the SSL as it belongs to WP Engine and you have no true way to force it properly.
Staging is for troubleshooting and theme development.
Answer: Your staging site works with https and being a wpengine.com subdomain there is already a valid certificate. However, by default, the staging gets created with http. If you try to access your login using https it will redirect, forcing you to login with unsecured form and yielding browser errors. Contact support and have them force staging to use https for all your future staging for this site.


Metrics in comparing the rank orders of elements from 2 vectors

Question: I am working on sampling a posterior distribution for parameters in my model using approximate Bayesian computation (ABC). I would like to come up with a summary statistic to compare the similarity/distance of ranked order between my simulated data and observed data. My observed data are a list of 10 species biomass. So I would like to compare the rank of species i between simulated and observed data, and summarize over 10 species.
My questions are:

Is there some metrics that can quantify the rank differences between 2 vectors? So far, what I did is to take the rank difference per species and calculate the sum of absolute difference across the 10 species.

Because the metric should provide a distance-like result: the more similar the 2 vector ranks, the smaller the value is. I am not sure if I can use the test statstics in Wilcoxon signed-rank test. Anyone has any comments?

Thanks very much
Answer: Can you use a simple Spearman correlation?  It will give you a rank-based correlations statistic with the benefit of having metric properties, I believe.
Comment: thanks very much. The Spearman rank correlation is exactly what I want
Answer: Although this question has been finished, I still want to mention the Kendall rank correlation coefficient. Personally, it accommodates the nature of ranks better and its intuition is very straightforward.


Option 82 and dhcpd. "No free leases" for second computer

Question: There is DHCP server in network (isc-dhcpd-server-3.0 on FreeBSD 7.2) than gives one IP per switch port to every user via Option 82
The problem appears when user disconnects one of his computers and connects another(i.e notebook with different MAC address) then DHCPD puts to log "...network net1: no free leases", because there is record in leases file that this IP is already owned by another MAC. That second computer will have his IP only after default-lease-time (that is IIRC minimum  10min, and after 3min he usually calling support) or after deletion of dhcpd.leases file and restart of dhcpd.
Is there a way to turn leases off at all, because we have strict binding between switch-port-ip?
Comment: How many ports do you have and how big is your IP address space? Could you just assign more than one IP to each switch-port?
Comment: IP pool is public so it's pretty limited and cost $$. So solution with assigning more than one IP is out of question, but thanks for the try.
Answer: At $former_employer, I modified the OpenBSD dhcpd to understand Option 82 and do address assignment directly on that.  As a matter of policy, "if you're coming in on connection X, you have the address assigned to X and if you use a switch to connect multiple computers, that's your own problem", and then just direct assignment based on that.
If you start seriously messing around with Option 82 assignment, it's probably worth doing this.  Wasn't too hard, but wasn't trivial either.
Comment: do you still have that patch around to share?
Comment: No, and it's not my property, it was written for and so belongs to that employer.
Answer: The only thing I can think of is to set the lease time excruciatingly low.  What you're trying to do is essentially the exact opposite of what DHCP is intended to do, so don't expect there to be any simple and painless answers that actually work.
Answer: One option would be to encourage your user to release the DHCP lease before disconnecting. I realise this is a workaround, rather than a solution, and might require some user training before it is effective!
On Windows, you can use the command "ipconfig /release" - although this requires elevated permissions, so a tool like CPAU might be useful.
Under FreeBSD or Linux, "dhclient -r" (must be running the ISC dhclient) is the release command. If you're using dhcpcd, "dhcpcd -k" should send the release packet. I'm yet to get this working while running NetworkManager, though. Again, these commands are privileged operations.
Answer: Solution found:

Setup switches to generate snmp
traps on port down.
On trap server catch them and call omshell
    script that deletes old lease.

Not trivial, but also not impossible.


Where to store my node-schedules

Question: I'm very new to Node/Express and I'm making an appointment system. I want my users to make an appointment for the day they want and my system sends them notification on that exact time. I found "node-schedule" module which is really good for this task but I don't know where to implemented this. Is there anyway to store all tasks in my app.js or is it enough to just create a node-schedule task every time I hit certain end point for example:
router.get('/', function(req, res, next) {
 var j = schedule.scheduleJob(date, function(){
   send notification();
});
res.send(200);
}

Note: I do not want to run a constant for loop on my sql table to check for dates
Answer: You'll need to persist your application data to some form of permanent storage by either writing to local files using something like SQLite, running your own database server (like MongoDB) or using a cloud-based storage service like Amazon SimpleDb.
Each of these options (and many others) have npm modules you can use to read/write/delete persistent data. For examples, see MongoDb, SQLite3, and SimpleDb, all available using npm on npmjs.com.
UPDATE
Per your comment below: Well, you did ask where you could store your scheduled events. ;)
To persist all scheduled events so they survive possible server failure, you'd need to create a storable data structure to represent them and for each event, create a new instance of your representation and store it to your persistent storage (MySQL).
Typically, you'd use something along the lines of:
{
  when:DateTime        -- timestamp when the event should fire
  what:Action          -- what this event should do
  args:Arguments       -- arguments to pass to Action
  pending:Boolean=true -- if false, this event has already fired
}

When you initialize your server, you would query your persistent storage for all events where pending===true and using the results, initialize instances of the node-schedule module.
When you need to schedule a new event while your server was running, you'd create a new event representation, write it into persistent storage and create a new instance of node-schedule using it.
Finally, and most importantly for customer happiness, when a scheduled event completes successfully, just before your event handler (the Action mentioned above) completes, it would need to mark the persistent version of the event it is handling as pending:false so you don't fire any event more than once.
So for example:
  'use strict';

  var scheduler = require('node-schedule');

  /**
   * Storable Representation of a Scheduled Event
   *
   * @param {string|Date} when
   * @param {string} what
   * @param {array.<string>} [args=[]]
   * @param {boolean} [pending=true]
   *
   * @property {Date} PersistentEvent.when       - the datetime this event should fire.
   * @property {string} PersistentEvent.what     - the name of the action to run (must match key of PersistentEvent.Actions)
   * @property {array} PersistentEvent.args      - args to pass to action event handler.
   * @property {boolean} PersistentEvent.pending - if true, this event has not yet fired.
   *
   * @constructor
   *
   * @example
   *
   * var PersistentEvent = require('PersistentEvent'),
   *     mysql = require('mysql'),
   *     conn = mysql.createConnection({ ... });
   *
   * conn.connect();
   *
   * // at some point when initializing your app...
   *
   * // assign your persistent storage connection...
   * PersistentEvent.setStore(conn);
   *
   * // load all pending event from persistent storage...
   * PersistentEvent.loadAll$(function (err) {
   *   if (err) {
   *     throw new Error('failed to load all PersistentEvents: ' + err);
   *   }
   *
   *   // from this point on, all persistent events are loaded and running.
   *
   * });
   */
  var PersistentEvent = function (when, what, args, pending) {
    // initialize
    PersistentEvent.Cache.push(this.init({
      when: when,
      what: what,
      args: args,
      pending: pending
    }));
  };

  // ==== PersistentEvent Static Methods ====

  /**
   * Pre-defined action event handlers.
   *

   * Where the property key will be used to match the PersistentEvent.what property,
   * and the property value is a event handler function that accepts an optional
   * array of args and a callback (provided by PersistentEvent.prototype.schedule)
   *

   *
   * @property {object}
   * @property {function} Actions.doSomething
   * @property {function} Actions.doSomethingElse
   *
   * @static
   */
  PersistentEvent.Actions = {
    doSomething: function (args, cb) {
      // defaults
      args = args || [];

      // TODO check specific args here ...

      var result = true,
          err = null;

      // do your action here, possibly with passed args

      cb(err, result);
    },
    doSomethingElse: function (args, cb) {
      // defaults
      args = args || [];

      // TODO check specific args here ...

      var result = true,
          err = null;

      // do your action here, possibly with passed args

      cb(err, result);
    }
  };

  /**
   * Cache of all PersistentEvents
   *
   * @type {Array.<PersistentEvent>}
   * @static
   */
  PersistentEvent.Cache = [];

  // Data Management

  /**
   * Connection to persistent storage.
   * TODO - This should be abstracted to handle other engines that MySQL.
   * @property {object}
   * @static
   */
  PersistentEvent.StorageConnection = null;

  /**
   * Sets the storage connection used to persist events.
   *
   * @param {object} storageConnection
   * @static
   */
  PersistentEvent.setStore = function (storageConnection) { // set the persistent storage connection
                                                            // TODO - check args here...

    // Note: this function isn't really needed unless you're using other kinds of storage engines
    // where you'd want to test what engine was used and mutate this interface accordingly.

    PersistentEvent.StorageConnection = storageConnection;
  };

  /**
   * Saves a PersistentEvent to StorageConnection.
   *
   * @param {PersistentEvent} event - event to save
   * @param {function} cb - callback on complete
   * @static
   */
  PersistentEvent.save$ = function (event, cb) {
    var conn = PersistentEvent.StorageConnection;

    if (null === conn) {
      throw new Error('requires a StorageConnection');
    }

    // TODO - check for active connection here...

    // TODO - check args here...

    conn.query('INSERT INTO TABLE when = :when, what = :what, args = :args, pending = :pending', event, cb);
  };

  /**
   * Loads all PersistentEvents from StorageConnection.
   * @param {function} cb -- callback on complete
   * @static
   */
  PersistentEvent.loadAll$ = function (cb) {
    var conn = PersistentEvent.StorageConnection;

    if (null === conn) {
      throw new Error('requires a StorageConnection');
    }

    // check for active connection here...

    // check args here...

    conn.query('QUERY * FROM TABLE WHERE pending = true', function (err, results) {
      if (err) {
        return cb(err);
      }
      results.forEach(function (result) {
        // TODO: check for existence of required fields here...
        var event = new PersistentEvent(result.when, result.what, result.args, true);
        event.schedule();
      });
      cb(null);
    });
  };

  // ==== PersistentEvent Methods ====

  /**
   * Initialize an instance of PersistentEvent.
   *
   * @param {object} opts
   * @return {PersistentEvent}
   */
  Event.prototype.init = function (opts) {
    // check args
    if ('object' !== typeof opts) {
      throw new Error('opts must be an object');
    }

    // set defaults
    opts.args = opts.args || [];
    opts.pending = opts.pending || true;

    // convert string to Date, if required
    if ('string' === typeof opts.when) {
      opts.when = new Date(opts.when);
    }

    // check that opts contains needed properties
    if (!opts.when instanceof Date) {
      throw new Error('when must be a string representation of a Date or a Date object');
    }

    if ('string' !== typeof opts.what) {
      throw new Error('what must be a string containing an action name');
    }

    if (!Array.isArray(opts.args)) {
      throw new Error('args must be an array');
    }

    if ('boolean' !== typeof opts.pending) {
      throw new Error('pending must be a boolean');
    }

    // set our properties
    var self = this;
    Object.keys(opts).forEach(function (key) {
      if (opts.hasOwnProperty(key)) {
        self = opts[key];
      }
    });

    return this;
  };

  /**
   * Override for Object.toString()
   * @returns {string}
   */
  PersistentEvent.prototype.toString = function () {
    return JSON.stringify(this);
  };

  /**
   * Schedule the event to run.

   * Side-effect: saves event to persistent storage.
   */
  PersistentEvent.prototype.schedule = function () {
    var self = this,
        handler = Actions[this.what];

    if ('function' !== typeof handler) {
      throw new Error('no handler found for action:' + this.what);
    }

    PersistentEvent.save$(self, function () {
      self._event = scheduler.scheduleJob(self.when, function () {
        handler(self.args, function (err, result) {
          if (err) {
            console.error('event ' + self + ' failed:' + err);
          }
          self.setComplete();
        });

      });
    });
  };

  /**
   * Sets this event complete.

   * Side-effect: saves event to persistent storage.
   */
  PersistentEvent.prototype.setComplete = function () {
    var self = this;
    delete this._event;
    this.pending = false;
    PersistentEvent.save$(this, function (err) {
      if (err) {
        console.error('failed to save event ' + self + ' :' + err);
      }
    });
  };

Note that is a first-pass boilerplate to show you one way of designing a solution to your problem. It will require further effort on your part to run.
Comment: @HiradRoshandel: You may wish to re-read the documentation for `node-schedule` as it handles "firing" your events at a scheduled datetime. You do not need to do so yourself, only to create an instance of node-schedule for each event you wish to occur. And while you could do this via `cron`, doing so will require a separate script.
Comment: Then you should look at [Mysql](https://www.npmjs.com/package/mysql) which is the nodejs connector to the Mysql database engine.
Comment: sorry Rob but my question is not about storage. I was asking how and where should I implement these node-schedules. I don't want to run a for loop on my sql table to check if appointment date is reached. Should I store it as a global variable in an array in my app.js?
Comment: This is awesome @rob-raisch , just what I was looking for to get me started
Answer: You can have a cron job every morning that will pick all the appointments for that day and schedule pushes for them. This way you'll have to query the DB once that too on the time when the server load is minimal.
Comment: I'll test it and see if it solves my problem but for now +1 thanks

Where to store my node-schedules


Erro ao importar xml.parsers.expat em Python 2.7 so funciona com Python3

Question: Bom dia, ha uns meses comecei com python estou usando-o para fazer testes em aplicações android. Eu tenho o seguinte problema Importando o xml.parsers.expat.
quando executo direto no terminal ele funciona. Mais se eu executo usando subprocess.call(import-expat.py, shell=True) não funciona  e da um erro.
Esse é o script (import-expat.py):
#! /usr/bin/env python
import xml.parsers.expat

Esse é o erro:
 File "/opt/u_script_files/import-expat.py", line 2, in <module> import xml.parsers.expat
  File "/usr/lib/python2.7/xml/parsers/expat.py", line 4, in <module> from pyexpat import *
ImportError: /usr/lib/python2.7/lib-dynload/pyexpat.x86_64-linux-gnu.so: undefined symbol: XML_SetHashSalt

Eu verifiquei as dependencias e no meu entender esta tudo certo:
$ ldd /usr/lib/python2.7/lib-dynload/pyexpat.x86_64-linux-gnu.so
linux-vdso.so.1 =>  (0x00007ffd9b392000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1608877000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f16084b2000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f1608288000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1608ca6000)

Pensei que o arquivo libexpat.so.1 não existia e procureri ele
$ apt-file search libexpat.so.1
libexpat1: /lib/x86_64-linux-gnu/libexpat.so.1
libexpat1: /lib/x86_64-linux-gnu/libexpat.so.1.6.0

Mais ele existe sim, e está no path correto, então não sei onde esta o erro,
pois executando direto no terminal funciona sim, mas usando o subprocess.call não funciona.
Dados adicionais:
S.O: Ubuntu 14.04 x86_64 x86_64 x86_64 GNU/Linux
Eu agradeço suas respostas.
EDIT:
Codigo onde é usado o subprocess.call:
user_path_script = '/opt/u_script_files/import-expat.py'
   for index, line in enumerate(listdevtotest):
     emulatorid = listdevtotest[index][0]
     subprocess.call(user_path_script + ' ' +  emulatorid, shell=True)

Eu fiz um teste local criei um script "call-import-expat.py"
#! /usr/bin/env python
import subprocess
subprocess.call('import-expat.py', shell=True)

e outro script "import-expat.py":
#! /usr/bin/env python
import xml.parsers.expat
print "imported expat"

Se é executado no terminal: python call-import-expat.py, ele funciona bem e mostra o texto "imported expat".
Mas se o script é chamado via web(php) ele não funciona. Na web e chamado da seguinte forma:
$cmd = 'python /path/script/call-import-expat.py';
$proc = popen($cmd, 'r');
pclose($proc);

NEW EDIT:
Acabei de editar essa linha
 $cmd = 'python /path/script/call-import-expat.py';

por esta:
 $cmd = 'python3 /path/script/call-import-expat.py';

Mudei python por python3. Mas o Python por default é 2.7. Não entendo porque funciona com Python3.
Comment: Posta a parte do código que chama o método `subprocess.call`.
Comment: @Dener Adicionei o código na pergunta. Obrigada
Answer: Izabel, você já tentou trocar o nome do seu arquivo?
As vezes o python 2.X reconhece como se fosse o mesmo arquivo da função!
Já a parte onde você diz que seu código roda normalmente com python 3 e não com o 2, pode ser por seu código Python ter sido escrito da forma que é padrão desta versão.
Caso queira testar, este código abaixo serve para você checar se seu código importa o módulo e caso ele não o faça, ele te responderá com um erro em sua tela!
try:
    import xml.parsers.expat
except:
    print(ImportError)


the definition of Riemann zeta function

Question: I've read that the Riemann zeta function for $0<s<1$ is defined :
$$\lim_{x\rightarrow\infty} \left(\sum_{n \leq x}\frac{1}{n^s}- \frac{x^{1-s}}{1-s}\right)$$
I don't know how to prove that this limit exists.
help me please.
thanks a lot.
Comment: @graydad: The (real) Euler zeta function is defined by just the basic series on the reals greater than $1$. The Riemann zeta function, defined by analytic continuation of the Euler zeta function into the complex plane with the point $1$ removed, is well defined on the real interval $(0\,,1)$.
Comment: The proof is in Apostol on the following pages. Essentially, you use the Euler Maclaurin formula (which is given earlier in the same chapter) to turn the sum into an integral, then you bound the pieces.
Comment: What have you done so far? And are you considering the complex plane or just the real numbers?
Comment: just real numbers.
Comment: Riemann-Zeta function is not defined on the real numbers for $01$
Comment: see the book "introduction to analytic number theory" by Apostol page 55.it is there.
Comment: Note that the given definition works also for $s>1$.
Comment: @Jonathan Taylor i,ve seen Apostol.there's no proof.help please.
Answer: Pg 55 of Apostol, theorem 3.2 (b) (halfway down page 56 he gives the proof) gives
$\sum_{n\leq x}\frac{1}{n^s}=\frac{x^{1-s}}{1-s} +\zeta(s) +O(x^{-s})$
if $1\neq s>0$
Subtract the first term of the right hand side and take $x$ to infinity.
(and the proof does consist of Euler Summation)

the definition of Riemann zeta function


How to insert html code in a scalatags atribute?

Question: I have a button that gets used as a popover with Twitter's bootstrap.
button(data.content := h1("an example of html").render, data.toggle="popover")("click here")

I want to have some html code in the content of the popover, so I need to pass html to the data.toggle atribute, however this prints the html code out plain, because of scalatags preventing XSS. How can I prevent this / how else can I get this effect?
Comment: Did you read http://www.lihaoyi.com/scalatags/#Auto-escapingandunsanitizedInput ?
Comment: @danielnixon I have yet to find a way to use that inside an attribute, as stated in my question.
Answer: This is gnarly but it'll do what you want. I'm not sure there's a nicer way.
import scalatags.Text.{Attr, TypedTag}
import scalatags.Text.all._
import scalatags.text.Builder

// Warning: this extends a "more-or-less internal trait" so may stop working after a ScalaTags update.
final case class RawValueSource(v: String) extends Builder.ValueSource {
  override def appendAttrValue(strb: StringBuilder): Unit = strb.append(v)
}

// We need an AttrValue for tags. This one just renders the tag *without* escaping the result.
class TagAttrValue extends AttrValue[TypedTag[String]] {
  override def apply(t: Builder, a: Attr, v: TypedTag[String]): Unit = {
    t.setAttr(a.name, RawValueSource(v.render))
  }
}

// We need this implicit in scope so that we can use tags on the rhs of the := operator.
implicit val tagAttr = new TagAttrValue()

button(data.content := h1("an example of html"), data.toggle := "popover")("click here")


Mangopay implementation for zf2

Question: I'm looking for a vendor/module that implements mangopay functionality into zf2.
Has anyone done this before or know someone who can do it?
I espially need e-wallet and adaptive payment functionality.
Answer: My team and I used https://github.com/MangoPay/mangopay2-php-sdk with success on a ZF2 project (I don't know if I can name it here without being perceived as advertisement and/or url bombing, but you can contact me for further details).
Pro tips:

Do not use the library directly in your business code, always wrap calls to this API into a dedicated service. Create a Wallet service with business-related methods (public function createUser(User $user), public function createCardRegistration(User $user, Card $creditCard), ...)
Be careful to properly read MangoPay api reference (https://docs.mangopay.com/api-references/) before you begin to use any feature, every detail is important and a missed sentence can rapidly become a headache
Don't be afraid to contact the support team (https://www.mangopay.com/contact/ with a technical case type) when you're blocked. They're fast and can unlock an apparently blocked situation in minutes.
Log everything, too much log is better than no log when it comes to payment. Play with log levels if you can't handle this much log :)

Mangopay implementation for zf2


How to Handle POST requests in Twisted

Question: I have a very simple twisted script where you can handle POST requests:
class FormPage(Resource):
    isLeaf = True
    def render_GET(self, request):
        return b"""<form method="POST"><input name="form-field" type="text" /></form>"""

    def render_POST(self, request):
        return 'You submitted: %s' % (cgi.escape(request.args["form-field"][0]),)

factory = Site(FormPage())
reactor.listenTCP(80, factory)
reactor.run()

But whenever I run this and fill out the box, I get and error:
builtins.KeyError: 'form-field'

Could anybody tell me why this is? thanks!!
Answer: I found a solution using request.content.read()
def render_POST(self, request):
    return 'You submitted: %s' % (request.content.read())

It might not be the best solution, but It worked for me. Please comment If you have a better solution, thanks!
Answer: Found the solution.  I got stuck here for a long time doing the "O'Reily Twisted" in Python3.  Here's what worked for me:
def render_POST(self, request):
    return_value = "You submitted: %s " % (cgi.escape(str(request.args[b"form-field"][0], 'utf-8')))
    return str.encode(return_value)

I guess the first reason I had trouble was that the form fields were that can be extracted in python code were in byte string.  Only after I checked the request args that I realized that.  I suppose in python2, it was regular string.

How to Handle POST requests in Twisted


Using the Blitline "burst_pdf" feature to convert large PDF documents to images

Question: I'm trying to use the burst_pdf feature of Blitline to convert a 30-page PDF document to images.
I'm posting the following to Blitline:
<code>"json" :  { "application_id": "MY APPLICATION ID",
          "src" : "http://mysite/mypdfdocument.pdf",
          "src_type" : {"name" : "burst_pdf"},
          "src_data" : {"dpi" : 200},
          "functions" :
          [{
            "name": "resize_to_fit",
            "params": { "width" : 500},
            "save" : {
                  "image_identifier" : "mypdfdocument.pdf.png"
              }
           }
          ]}
</code>
I get this response:
<code>{
  "results": {
    "images": [
    {
      "image_identifier": "mypdfdocument.pdf.png",
      "s3_url": "http://s3.amazonaws.com/bucket/7N6WmsVu1kHoMQYM79s-Kag.jpg"
    }],
    "job_id": "3zpQhNz6qGYBaPwaj47mpog"
  }
}
</code>
At first glance it looks OK, but according to the Blitline API docs (http://www.blitline.com/docs/pdf#burst_pdf) I should get another parameter back called group_completion_job_id which can be used for long-polling or postbacks once the PDF conversion is completed.
Has anyone used the "burst_pdf" feature of Blitline successfully? Any help appreciated! :)
Thanks!
Comment: There are some differences between different API versions as well, specifying the "v" parameter to use a specific version is advisable. Also, at the time of writing the API documentation was somewhat out of sync. This is reportedly being fixed.
Answer: Nevermind, I must have made a typo, or read some old documentation.
This argument...
"src_type" : {"name" : "burst_pdf"}
...should be:
"src_type" : "burst_pdf"
Doh! :)


Sending mail using netcat - getaddrinfo: Name or service not known

Question: i am trying to send a mail using netcat, and i am getting an error:
nc: getaddrinfo: Name or service not known

this is the command i am using
/usr/bin/nc smtp.xxxxx.yy 25 < test.txt

this is the content of test.txt
xxx.yyy.zz
MAIL FROM:<firstname.lastname@example.com>
RCPT TO:<firstname.lastname@example.com>
DATA
From: [xxx.yy] <firstname.lastname@example.com>
To: <firstname.lastname@example.com>
Date: Mon, 2 Nov 2015 20:32:26 -0400
Subject: Test Message

This is a test !

Have a good day!
Frogg

.

QUIT

the system is ESXI 5.5
Comment: To Womble : error message, command and system are present in the question / To Jeeny D & Ward : NetCat is a way to send mail, in this case used to send backup result cause Esxi doesn't have intregated mail system. + tags i used are available in serverfault + trouble was firewall linked, so what should i changed in this post ?
Answer: Netcat (nc) is telling you that it can't resolve the name "smtp.xxxxx.yy".  That's what nc: getaddrinfo: Name or service not known means.
To verify that this is the issue, I'd suggest entering the IP of your SMTP server manually in your nc command instead of the domain name.  If that works, then it's definitely a DNS issue.
To see what's going wrong in DNS, try running nslookup smtp.xxxxx.yy.  This will show you what the DNS server thinks the correct IP is.  I'm expecting that it will return one of the two following messages.
Scenario 1 - No DNS servers are configured, or are not reachable by the EXSi host:
nslookup foo.bar.baz
;; connection timed out; no servers could be reached

Scenario 2 - DNS server configured and reachable, but requested domain name is not valid
carbon:~ % nslookup foo.bar.baz
Server:     22.214.171.124
Address:    126.96.36.199#53

** server can't find foo.bar.baz: NXDOMAIN

Here's a VMWare KB article on the topic of troubleshooting DNS issues.
Comment: thanks, nslookup shown me smtp wasn't reachable, i had to change firewall rules to allow smtp request (after i got more trouble with anonymous request, but AUTH LOGIN solved the trouble ..but thats another story !)


woocommerce_before_calculate_totals hook stopped working after update to WC 3.0.1

Question: I have updated to WC 3.0.1 from 2.6.14.
My original code is as follows:
add_action( 'woocommerce_before_calculate_totals', 'add_custom_price' );

function add_custom_price( $cart_object ) {
    $custom_price = 10; // This will be your custome price
    foreach ( $cart_object->cart_contents as $key => $value ) {
        $value['data']->price = $custom_price;
    }
}

It is no longer updating the price in cart or minicart.
Comment: Look at `set_price()` in the [source](https://github.com/woocommerce/woocommerce/blob/3.0.0/includes/abstracts/abstract-wc-product.php#L804-L806)
Comment: You can no longer directly access `$product` properties. You should see a warning about this in the `debug.log` if you enabled `WP_DEBUG`. You must use setters and getters now for product, order, order item, and coupon objects.
Comment: Thanks helgatheviking,

Could you please provide an example for getting and setting price of product ?
Comment: Please write out a complete answer for future visitors.
Comment: Possible duplicate of [Dynamic cart item pricing not working on orders in WooCommerce 3.0+](https://stackoverflow.com/questions/43885585/dynamic-cart-item-pricing-not-working-on-orders-in-woocommerce-3-0)
Answer: For overriding product price on cart in the latest version of Woocommerce (3.0.1) try using set_price( $price ) function in woocommerce this will help. Source here
add_action( 'woocommerce_before_calculate_totals', 'woocommerce_pj_update_price', 99 );

function woocommerce_pj_update_price() {

    $custom_price = $_COOKIE["donation"]; // This will be your custom price
    $target_product_id = 413; //Product ID

    foreach ( WC()->cart->get_cart() as $cart_item_key => $cart_item ) {

        if($cart_item['data']->get_id() == $target_product_id){

            $cart_item['data']->set_price($custom_price);
        }

    }

}

Comment: 3 hours of my time wasted looking for a solution to this. Thanks to you is not 3 days...
Answer: Works with a little change:
//OLD:
$value['data']->price = $custom_price;

//NEW:
$value['data']->set_price( $custom_price );

function add_custom_price( $cart_object ) {
    $custom_price = 10; // This will be your custome price
    foreach ( $cart_object->cart_contents as $key => $value ) {
        $value['data']->set_price( $custom_price );
    }
}

Answer: add_action( 'woocommerce_before_calculate_totals', 'add_custom_price', 10, 1);
function add_custom_price( $cart_obj ) {

    //  This is necessary for WC 3.0+
    if ( is_admin() && ! defined( 'DOING_AJAX' ) )
        return;

    foreach ( $cart_obj->get_cart() as $key => $value ) {
        $value['data']->set_price( 40 );
    }
 }

if i set $value['data']->set_price( 40 ) work fine, but:
 foreach ( $cart_obj->get_cart() as $key => $value ) {
            $price = 50;
            $value['data']->set_price( $price );
 }

Answer: Well, the problem is you are calling price directly at $value['data']->price.
Make it:
$value['data']->get_price()

and I think you problem will be fixed


Equations for interior of Platonic solids

Question: It is well-known that for Platonic solids:

The interior of cube a.k.a. hexahedron can be described with inequality $\max\{|x|,|y|,|z|\}<a$.
The interior of octahedron is $|x|+|y|+|z|<a$.

But what are such equations for tetrahedron, icosahedron and dodecahedron?
I doubt that the equations are elegant.
Comment: check out coxeters polyhedra. has tons of nice information on all sorts.
Answer: Your equations are using "max" and "absolute value" to effectively compress many inequalities into one.
For example, the octahedron inequality you give could be expanded into 8 inequalities such as $-x + y + -z < a$.  Four of those eight have an even number of negations and four have an odd number of negations — each of these groups of four defines a tetrahedron.  Yes, an octohedron is the intersection of two tetrahedra.
In general, we can view your inequalities as being of the form
$$\vec{v}\cdot\vec{x} < 1$$
where $\vec{x}$ represents $(x,y,z)$, and $\vec{v}$ is a member of a specific set $V$ of vectors corresponding to the faces of the object.
For example, for the cube, $V$ is all cyclic permutations of $(1,0,0)$ and $(-1,0,0)$.
For the octahedron, $V$ is the eight vectors $(±1,±1,±1)$.
For the tetrahedron, $V$ is all cyclic permutations of $(1,-1,-1)$ and $(1,1,1)$.
For the dodecahedron, $V$ is all cyclic permutations of $(0,±1,±\phi)$, where $\phi=\frac{\sqrt{5}+1}{2}$ is the golden ratio.
For the icosohedron, $V$ is all cyclic permutations of $(±1,±1,±1)$ and $(0,±\phi^{-1},±\phi)$.
What these equations are really doing is expressing a relationship between a platonic solid and a standard 3-dimensional coordinate system of measurements along 3 orthogonal axes.  This works well when the group of symmetries matches well and the object is aligned with the axes, and less well in other cases.


Does it make sense to use PCA right after GBM?

Question: My Problem: I'm trying to classify a data into two groups as A and B based on 25 observations (data point) and 100 features. I used the Gradient Boosting Machine (GBM) to find out which feature has the most important contribution to classify the data points into group A or B. As a result, I extracted the top 4 most important features for this classification. Let's say they are X1, X2, X3, and X4. Right now, I'm interested to plot all combinations of these 4 features versus each other (i.e. scatter plots) and find out if I could find out that the data points that belong to category A or B are clustered together in certain regions or not. For example, I plot X2 versus X1 as a scatter plot and use KMeans for clustering identification. But the problem is KMeans did a pretty bad job on each those pair scatter plots (i.e. scatter plot of Xi vs Xj 1 <= i,j <= 4).
My Approach: Based on bad results of KMeans in order to cluster the points, I decided to use PCA on these 4 features and reduce the dimensionality furthermore to two variables as X'1 and X'2. Then again I used KMeans to cluster the point and it significantly improved the KMeans results.
My Question: Does it make sense to use PCA on those top 4 features extracted from GBM (i.e. X1, X2, X3, and X4)? Cause I'm wondering we already reduced the dimensionality from 100 to 4 by using GBM and extracting top 4 most important features in our data. Does it make sense to use PCA and reduce those 4 features furthermore into two variables that have highest variances? If your opinion is no can you give me a precise explanation or possibly some references and if your opinion is yes again explanation and references could help me a lot.
Comment: Have you tried using PCA on all of the features? If so, are the results the same?
Comment: @Sycorax Thanks for comment! Can you elaborate the similarity of my problem to the linked answer a bit more probably in an answer? Cause in the linked question the OP asked about use a linear regression after random forest but my problem is built based on GBM.
Comment: @Sycorax Really good point! I think it worth to be an answer to achieve better visibility for future users.
Answer: You're using so-called "feature importance" metrics derived from a tree-based model to determine the inputs to a linear method, just as in the linked post. Using a linear method doesn't make much sense when there are relevant nonlinear terms in the phenomenon you are studying; to a linear method, the nonlinearity is "invisible." By using GBM, your thesis is that the outcome is a nonlinear function of the inputs and is well-approximated by a tree structure. But by using PCA to tease out subpopulations, your thesis is that you just need to rotate the data. These are not consistent.
A cousin to this problem (replacing gradient-boosted trees with random forest, and replacing PCA with a GLM), with an answer elaborating about how linear and nonlinear models can give different answers to the same question, can be found in Can a random forest be used for feature selection in multiple linear regression?


Output the price of the fertilizer per pound and the cost of fertilizing per square

Question: During each summer John and Jessica grow vegetables in their back yard and buy seeds and fertilizer from a local nursery. The nursery carries different types of vegetable fertilizers in various bag sizes. When buying a particular fertilizer, they want to know the price of the fertilizer per pound and the cost of fertilizing per square foot. The following program prompts the user to enter the size of the fertilizer bag - in pounds, the cost of the bag, and the area - in square feet that can be covered by the bag. The program should output the desired result
This is what I've done so far:
import java.util.*;

public class Ch3_PrExercise4
{

static Scanner console = new Scanner (System.in);
public static void main (String [] args)
{

    double cost;
    double area;
    double bagSize;

    System.out.print("Enter the amount of fertilizer in pounds, in one bag: ");
    bagSize = console.nextDouble();
    System.out.println();
    System.out.println();

    System.out.print("Enter the cost of the, " + bagSize + " pound fertilizer bag: ");
    cost = console.nextDouble();
    System.out.println();

    System.out.print("Enter the area, in square feet, that can be fertilized by one bag: ");
    area = console.nextDouble();
    System.out.println();

    System.out.printf("The cost of the fertilizer per pound is: $%.2f%n", bagSize/cost);
    System.out.println();
    System.out.printf("The cost of fertilizing per square foot is: $%.4f%n", area/cost);

}
}

PS: I'm new to Java. I'm trying to figure it out. Any response will be appreciated. Thanks
Comment: What's your question?
Answer: I think the last few lines should use cost/bagSize (since you want dollars per pound, so you should have dollars on top and pounds on the bottom) and cost/area (since you want dollars per area, so you should have dollars on top and area on the bottom):
    System.out.printf("The cost of the fertilizer per pound is: $%.2f%n", cost/bagSize);
    System.out.println();
    System.out.printf("The cost of fertilizing per square foot is: $%.4f%n", cost/area);

Comment: Do you have any idea to improve this program? or any workaround in this? Thanks for response anyway @Chai T. Rex
Comment: I would remove some of the `System.out.println();` statements so that the user sees output text that looks better organized and nicer, but other than that, it seems to work nicely.
Comment: Thank you so much @Chai T. Rex. You are very helpful. Sorry for intruding you, sad to say im noob in this language it will take time practice to master it.


Calculate absence time in day for employee

Question: I have table to store attendance data of my employee as below:
EmployeeID|   Date   |EnterTime|ExitTime|
    1     |2017-01-01|07:11:00 |15:02:00|
    2     |2017-01-01|07:30:00 |12:00:00|
    2     |2017-01-02|07:00:00 |15:00:00|
    1     |2017-01-02|07:30:00 |10:00:00|
    1     |2017-01-02|11:20:00 |15:00:00|
    1     |2017-01-03|09:30:00 |10:00:00|
    1     |2017-01-03|11:20:00 |15:00:00|

Actually my working time is from 07:00:00 to 15:00:00 .
I want To sum up the absence time of each employee for example :
 in 2017-01-01 for employee ID 2 , is 03:00:00 hours as absence
and in date 2017-01-02 for employee ID 1 is 01:00:00 hour.
In Date 2017-01-03 , employee id 1 , is 02:30:00
and employee id 2 absence is 08:00:00 hour because there is no log in my table.
Finally need following report:
EmployeeID|TotalWorkingHour       |TotalAbsenceHour|
    1     |sum(EnterTime-ExitTime)|05:09:00        |
    2     |sum(EnterTime-ExitTime)|11:30:00        |

I get total working seconds by following select,but don't know how to calculate absence seconds:
select EmployeeID,
sum(datediff(second,Convert(DATETIME, EnterTime, 114), Convert(DATETIME, ExitTime, 114)) ) as TotalWorkingSeconds
from Attendance
where Attendance.Date between @FromDate and @ToDate
and (EnterTime is not null)
and (ExitTime is not null)
group by EmployeeID

Comment: Surely once you've got "presence time", "absence time" is just 8 hours - presence time, isn't it?
Comment: could you please post your current SQL stament? what have you tried so far? did you put any efford in solving your problem already?
Comment: sum(EnterTime-ExitTime) ? why would you want enter - exit?? 9:00-17:00 surely gives -8, youd want exittime-entertime?
Comment: @JonSkeet you are right but as mentioned in my post some days my employee didn't attend at all , so how to find those days?
Comment: @EstebanP. Alreaddy added to my post
Comment: @MahdiIBM: Well for those days, you'd have a present-time of 0, or no record at all, depending on whether you use a left join or not. It still feels simpler to only compute present time and use that as a basis for absent time, rather than trying to sum individual absences.
Comment: @JonSkeet please let me know how to find the days that employee don't have record also consider if all employee are not peresented!!
Comment: What do you mean "also consider if all employee are not represented" - how is that different from "that employee don't have record"? Basically this sound like you need a left join on the employee table, so that you can get the values for *all* employees, regardless.
Comment: @JonSkeet Imagine we have 5 employee and today is 2017-01-01 and only 4 employee attended to work  but for tomorrow no body attended to  work , so for tomorrow we should calculate absence for all of our employees. and for 2017-01-01 absence should be calculate for one person only .
Comment: Well you should presumably *calculate* the absent time (and it may well be 0) for all employees on every day. But yet again, this just sounds like applying a left join, and using a default of "If I don't have any records, that means a present_time of 0."
Comment: Is this related to C# and/or Visual Studio in any way? If not, please remove those labels. Also, please show us what you've tried thus far. You almost certainly need an "employees" table so you can use an outer join to show employees, regardless of whether they check in on a given day, and logic to define a working day - do weekends count? How about bank holidays?
Answer: It might be not the final and perfect solution, but it works out:
DECLARE @StartDate DATE, @EndDate DATE
SET @StartDate = '2016-12-31'
SET @EndDate =  '2017-01-05' -- GETDATE()

SELECT alldate_and_employee.EmployeeID
      ,alldate_and_employee.Date
      ,COALESCE(employee_workingdays.WorkingMinutes,0) as TimeWorked
      ,(alldate_and_employee.PlannedWorkingTime - COALESCE(employee_workingdays.WorkingMinutes,0)) as  WorkedTooLessMinutes
  FROM
  (
     -- returns a table with a combination of all Employees and all Dates
     SELECT DISTINCT EmployeeID, datelist.Date, 480 as PlannedWorkingTime
       FROM mytable
      CROSS JOIN
      (
        -- selects all dates between @StartDate and @Enddate
        SELECT DATEADD(DAY,number+1,@StartDate) [Date]
        FROM master..spt_values
        WHERE type = 'P'
        AND DATEADD(DAY,number+1,@StartDate) < @EndDate
      ) datelist
  ) alldate_and_employee
  LEFT OUTER JOIN
  (
      -- groups the working time of each employee for a working day
      SELECT EmployeeID
           ,Date
           ,SUM(DATEDIFF(minute, EnterTime, ExitTime)) as WorkingMinutes
        FROM mytable
    GROUP BY EmployeeID
            ,Date
  ) employee_workingdays
  ON employee_workingdays.Date       = alldate_and_employee.Date
 AND employee_workingdays.EmployeeID = alldate_and_employee.EmployeeID

ORDER BY alldate_and_employee.Date, alldate_and_employee.EmployeeID


Stream file through Struts2 Rest

Question: I need to stream file with Struts2 REST. Although it stream fine, Web Server throws error:
Exception processing the result.: java.io.IOException: UT010029: Stream is closed
Here is the Action code:
<code>    public HttpHeaders index() {

    try {

        // ...

        response.setContentType(vo.getContentType());
        response.setContentLength(bytes.length);
        response.addHeader("Content-disposition", "attachment;filename=\"" + fileName + "\"");
        response.getOutputStream().write(bytes);

    } catch( Exception e ) {
        log.error(e);
    }

    return new DefaultHttpHeaders("index");
}
</code>
Web Server log:
<code>13-04-2018 16:01:28,335 INFO  [org.apache.struts2.rest.RestActionInvocation] (default task-15) Executed action [//api/blablaAction!index!json!200] took 208 ms (execution: 16 ms, result: 192 ms)
13-04-2018 16:02:08,927 ERROR [org.apache.struts2.rest.RestActionInvocation] (default task-16) Exception processing the result.: java.io.IOException: UT010029: Stream is closed
    at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:136) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
    at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:128) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
    at org.apache.struts2.rest.DefaultContentTypeHandlerManager.handleResult(DefaultContentTypeHandlerManager.java:199) [struts2-rest-plugin-2.3.34.jar:2.3.34]
    at org.apache.struts2.rest.RestActionInvocation.executeResult(RestActionInvocation.java:232) [struts2-rest-plugin-2.3.34.jar:2.3.34]
    at org.apache.struts2.rest.RestActionInvocation.processResult(RestActionInvocation.java:197) [struts2-rest-plugin-2.3.34.jar:2.3.34]
    at org.apache.struts2.rest.RestActionInvocation.invoke(RestActionInvocation.java:145) [struts2-rest-plugin-2.3.34.jar:2.3.34]
    at com.opensymphony.xwork2.DefaultActionProxy.execute(DefaultActionProxy.java:147) [xwork-core-2.3.34.jar:2.3.34]
    at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:575) [struts2-core-2.3.34.jar:2.3.34]
    at org.apache.struts2.dispatcher.ng.ExecuteOperations.executeAction(ExecuteOperations.java:81) [struts2-core-2.3.34.jar:2.3.34]
    at org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:99) [struts2-core-2.3.34.jar:2.3.34]
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.8.Final.jar:1.1.8.Final]
</code>
Answer: I was able to resolve this issue by using custom ContentTypeHandler

Stream file through Struts2 Rest


Django CreateView: 'str' object is not callable

Question: I'm trying to use django's CreateView, but I get the following error:
File "/Users/PycharmProjects/gusta/venv/lib/python3.8/site-packages/django/views/generic/edit.py", line 33, in get_form
    return form_class(**self.get_form_kwargs())
TypeError: 'str' object is not callable

views.py:
from django.views.generic import ListView, CreateView, DeleteView, DetailView, UpdateView
from .models import Employee, Event, detailEvent, Department, Poste
from django.contrib.auth.mixins import LoginRequiredMixin
from django.urls import reverse, reverse_lazy

# -----------------------------------------------------  Views for postes.
class PostelistView(LoginRequiredMixin, ListView):
    template_name = "postes/post_list.html"
    context_object_name = "Poste"

    def get_queryset(self):
        # initial queryset of leads for the entire organisation
        queryset = Poste.objects.all()
        return queryset

class PosteCreateView(LoginRequiredMixin, CreateView):
    template_name = "postes/post_create.html"
    form_class = "Poste"

    def get_success_url(self):
        return reverse_lazy("employee:post_list")

    def form_valid(self, form):
        instance = form.save(commit=False)
        instance.save()
        self.Poste = instance
        return super(PosteCreateView, self).form_valid(form)

my models.py
from django.utils.translation import ugettext_lazy as _
class Poste(models.Model):
    intitule = models.TextField(_("Name"), max_length=250, default='')

Why is this happening?
Answer: In PosteCreateView, you set
form_class = "Poste"

This isn't a class, its a string. Django expects it to be a class and will later try to instantiate an object. You can see this in the error message, which gives you the line where the exception occurs:
return form_class(**self.get_form_kwargs())

To solve it, set form_class to the actual class:
form_class = Poste


How to render videos url in a template page using Video Embed field?

Question: I have done some research on how I can render on a node .tpl page a youtube video, but couldn't find a solution.
The module I am using is Video Embed field and Drupal version is Drupal 7.
When doing a print-r content I can see a list of values but I'm unsure on how I can use them to show my video.
Here is the code
 [field_video_url] => Array
                    (
                        [und] => Array
                            (
                                [0] => Array
                                    (
                                        [video_url] => http://www.youtube.com/watch?v=dS4JR0BbKO8
                                        [thumbnail_path] => public://video_embed_field_thumbnails/youtube/dS4JR0BbKO8.jpg
                                        [video_data] => a:21:{s:5:"xmlns";s:27:"http://www.w3.org/2005/Atom";s:11:"xmlns$media";s:29:"http://search.yahoo.com/mrss/";s:8:"xmlns$gd";s:32:
                                        [embed_code] =>
                                        [description] =>
                                    )

                            )

                    )
[#formatter] => video_embed_field
        [0] => Array
            (
                [0] => Array
                    (
                        [#theme] => video_embed_field_embed_code
                        [#url] => http://www.youtube.com/watch?v=dS4JR0BbKO8
                        [#style] => normal
                        [#video_data] => Array
                            (
                                [xmlns] => http://www.w3.org/2005/Atom
                                [xmlns$media] => http://search.yahoo.com/mrss/
                                [xmlns$gd] => http://schemas.google.com/g/2005
                                [xmlns$yt] => http://gdata.youtube.com/schemas/2007
                                [gd$etag] => W/"CUACRn47eCp7I2A9XRZbFU8."
                                [id] => tag:youtube.com,2008:video:dS4JR0BbKO8
                                [published] => 2011-12-12T08:23:43.000Z
                                [updated] => 2014-08-20T08:36:07.000Z
                                [category] => Array
                                    (
                                        [0] => Array
                                            (
                                                [scheme] => http://schemas.google.com/g/2005#kind
                                                [term] => http://gdata.youtube.com/schemas/2007#video
                                            )

                                        [1] => Array
                                            (
                                                [scheme] => http://gdata.youtube.com/schemas/2007/categories.cat
                                                [term] => Tech
                                                [label] => Science & Technology
                                            )

                                    )

                                [title] => Introduction to Theming Basics for Drupal 7
                                [content] => Array
                                    (
                                        [type] => application/x-shockwave-flash
                                        [src] => http://www.youtube.com/v/dS4JR0BbKO8?version=3&f=videos&app=youtube_gdata
                                    )

                                [link] => Array
                                    (
                                        [0] => Array
                                            (
                                                [rel] => alternate
                                                [type] => text/html
                                                [href] => http://www.youtube.com/watch?v=dS4JR0BbKO8&feature=youtube_gdata
                                            )

                                        [1] => Array
                                            (
                                                [rel] => http://gdata.youtube.com/schemas/2007#video.related
                                                [type] => application/atom+xml
                                                [href] => http://gdata.youtube.com/feeds/api/videos/dS4JR0BbKO8/related?v=2
                                            )

                                        [2] => Array
                                            (
                                                [rel] => http://gdata.youtube.com/schemas/2007#mobile
                                                [type] => text/html
                                                [href] => http://m.youtube.com/details?v=dS4JR0BbKO8
                                            )

                                        [3] => Array
                                            (
                                                [rel] => http://gdata.youtube.com/schemas/2007#uploader
                                                [type] => application/atom+xml
                                                [href] => http://gdata.youtube.com/feeds/api/users/yLF5zO6ghZcOXKHRWTipvw?v=2
                                            )

                                        [4] => Array
                                            (
                                                [rel] => self
                                                [type] => application/atom+xml
                                                [href] => http://gdata.youtube.com/feeds/api/videos/dS4JR0BbKO8?v=2
                                            )

                                    )

                                [author] => Array
                                    (
                                        [0] => Array
                                            (
                                                [name] => Drupalize.Me
                                                [uri] => http://gdata.youtube.com/feeds/api/users/Lullabot
                                                [yt$userId] => yLF5zO6ghZcOXKHRWTipvw
                                            )

                                    )

                                [yt$accessControl] => Array
                                    (
                                        [0] => Array
                                            (
                                                [action] => comment
                                                [permission] => allowed
                                            )

                                        [1] => Array
                                            (
                                                [action] => commentVote
                                                [permission] => allowed
                                            )

                                        [2] => Array
                                            (
                                                [action] => videoRespond
                                                [permission] => moderated
                                            )

                                        [3] => Array
                                            (
                                                [action] => rate
                                                [permission] => allowed
                                            )

                                        [4] => Array
                                            (
                                                [action] => embed
                                                [permission] => allowed
                                            )

                                        [5] => Array
                                            (
                                                [action] => list
                                                [permission] => allowed
                                            )

                                        [6] => Array
                                            (
                                                [action] => autoPlay
                                                [permission] => allowed
                                            )

                                        [7] => Array
                                            (
                                                [action] => syndicate
                                                [permission] => allowed
                                            )

                                    )

                                [gd$comments] => Array
                                    (
                                        [gd$feedLink] => Array
                                            (
                                                [rel] => http://gdata.youtube.com/schemas/2007#comments
                                                [href] => http://gdata.youtube.com/feeds/api/videos/dS4JR0BbKO8/comments?v=2
                                                [countHint] => 17
                                            )

                                    )

                                [media$group] => Array
                                    (
                                        [media$category] => Array
                                            (
                                                [0] => Tech
                                            )

                                        [media$content] => Array
                                            (
                                                [0] => Array
                                                    (
                                                        [url] => http://www.youtube.com/v/dS4JR0BbKO8?version=3&f=videos&app=youtube_gdata
                                                        [type] => application/x-shockwave-flash
                                                        [medium] => video
                                                        [isDefault] => true
                                                        [expression] => full
                                                        [duration] => 1153
                                                        [yt$format] => 5
                                                    )

                                                [1] => Array
                                                    (
                                                        [url] => rtsp://r3---sn-5hn7su7k.c.youtube.com/CiILENy73wIaGQnvKFtARwkudRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp
                                                        [type] => video/3gpp
                                                        [medium] => video
                                                        [expression] => full
                                                        [duration] => 1153
                                                        [yt$format] => 1
                                                    )

                                                [2] => Array
                                                    (
                                                        [url] => rtsp://r3---sn-5hn7su7k.c.youtube.com/CiILENy73wIaGQnvKFtARwkudRMYESARFEgGUgZ2aWRlb3MM/0/0/0/video.3gp
                                                        [type] => video/3gpp
                                                        [medium] => video
                                                        [expression] => full
                                                        [duration] => 1153
                                                        [yt$format] => 6
                                                    )

                                            )

                                        [media$credit] => Array
                                            (
                                                [0] => lullabot
                                            )

                                        [media$description] => Discover hundreds more Drupal tutorial videos at Drupalize.Me/Videos.

I tried the code below but it doesn't work.
    <?php print $content['field_video_url]']['#formatter'][0]['#url'];?>

I had looked at his link but I'm not really understanding how it is working for him https://stackoverflow.com/questions/21282194/drupal-7-calling-video-embed-field-from-template-file
Many thanks for your help.
Answer: If you are not trying to do anything fancy, and it is a module-provided video field, Drupal's render function should be enough:
<?php print render($content['field_video_url']);?>

Comment: @amara "doesn't work" is most useless report in IT. It caused syntax error? Runtime error? Watchdog report? Died silently? Does is thow your field if you directly `print render($content);`?
Comment: Update:I'm not sure how, maybe issue with the cache but after writting print render($content), I then could see the video while writing this code . So it works thank you for your help!
Comment: @amara cache issues are burden we all need live with :)
Comment: Thank you but unfortunately this doesn't work
Comment: sorry about the lack of details. It doesn't show anything on the front page when I render ($content['field_video_url']). When I go to the back-end this message error is coming "Notice: Undefined index: field_video_url] in include()" However it does show the video when I do print render($content);


Nginx check if parameters exists and if so only allow specific values

Question: I cant seem to find a way to limit some queries. Right now i have this
if ($args ~* "(.*&)?page="){
   set $wrongP W;
}
if ($args !~* (.*&)?page=(2|12|24)$){
   set $wrongP "${wrongP}R";
}
if ($wrongP = WR){
   rewrite ^(.*)$ $uri? permanent;
}

This does exactly what i want, the problem is if url has more than one parameter fox ex. /somefile.html?page=2&arg=file it removes ALL parameters. Im guessing its because of the second if.
How do check for urls that have multiple variables and removes all of them only if page variable is not equal to 2, 14 or 24 ?
Comment: There is no easy way to change only one parameter. May be you could use [Lua](https://www.nginx.com/resources/wiki/modules/lua/) or [njs](https://nginx.org/en/docs/njs/)
Comment: I dont want to change one paramter. I want to remove them all which is what rewrite ^(.*)$ $uri? permanent; does. But only if parameter page exists and its not equal to 2, 12 or 24
Answer: I guess that you need something like this (using map):
map $arg_page $need_redirect {
  default "1";
  ""      "0";
  "2"     "0";
  "14"    "0";
  "24"    "0";
}

server {
  ...
  if ($need_redirect) {
    rewrite ^ $uri? permanent;
  }
  ...
}


Given a linear code $C$, then the subset $D\subset C$ with $D:=\{v\in C : w(v)=2k \textrm{ for }k\in \mathbb N \}$ is a linear code

Question: I found the problem above in one of the previpous year exams for my discrete mathematics course but I think it might be wrong.
I've seen similar problems but they all add the hypothesis of $C$ being binary. This way, when analyzing the different cases for the addition of two elements of $D$ I get that  $w(x+y)=w(x)+w(y)-2\{i\in \{1,...,n\} : x_i=y_i=1\}$  . This way, I can make sure the weight of the sum of two elements in $D$ is even and therefore the sum is an internal operation in $D$. That added to the fact that the element $0$ has an even weight ensures that $D$ is a vector space and therefore a lineal code.
However, when analyzing a field different to $\mathbb  F_2^n$ I fail to find that $w(x,y)$ has to be even.
In fact, couldn't I just use $x=(1,2,0), y=(2,2,0) \in \mathbb F_3^3 $ such that $(x+y)=(0,1,0)$ where $w(x)=2, w(y)=2$ but $w(x+y)=1$ as a counter example?
I'd like someone to verify if the property is true or if my professor made a mistake in the exam.
I'd also apreciate it if someone checked the my reasoning on the binary part.
Thanks in advance
Comment: @DilipSarwate Thanks for the answer :)
Answer: You are correct; the statement you are asked to prove is true for linear binary codes but not for codes over larger fields (including extensions of $\mathbb F_2$). If you are asked the same question on an exam, I would suggest giving the proof for linear binary codes, and your counterexample to show the professor that the result to be proved is not true in general, and needs additional constraints to make it valid.


Limit # of messages in mailbox

Question: I have a situation where users have an average of 20 to 30,000 emails in their inbox. We are adding more employees and mailboxes and this is getting out of control.
I have limited each mailbox to 1GB total however most of the time because these emails are so small even with 50,000 messages it's less than 1GB mailbox size.
My question. Is there a way to limit the mailbox by # of messages. Similar to mailbox size restrictions ?
It's an Exchange 2007 server.
Comment: What issues are you running into with 30,000 small emails.  I have mailboxes with over 100,000 messages without issue
Comment: @sgmoore are you sure, i had a folder that regularly had 100k+ messages in it on an Exchange 2003 server (long story don't ask).
Comment: Exchange can handle 100,000 message in total, but not 100,000 messages in one folder (ie the inbox). It think it is recommended to keep the number down to below 3000.
Comment: My mistake. The Exchange 2007 'limit' is 20,000
Comment: @Zypher. See the article 'Understanding the Performance Impact of High Item Counts and Restricted Views' on technet  [link](http://technet.microsoft.com/en-us/library/cc535025(EXCHG.80).aspx)  .  On re-reading my message, I should have said, Exchange can **easily** handle 100,000 message in total, but not 100,000 messages in one folder (ie the inbox) ....  The original implied that Exchange would not handle that number, whereas I meant it could struggle. Also, it does make a **big** difference if the folder is regularly opened (eg inbox) as opposed to a folder which may rarely be accessed.
Answer: You can only limit mailboxes by size, not item count in Exchange.
Is there any reason you care how many items are there, as long as the quota is maintained?
Here is some light reading on Exchange quotas.
Answer: If you want Exchange to automatically enforce a limit, I think you have to resort to monitoring the size of the folders.
The Exchange Team Blog has an Article about Finding High Item Count Folders using this PowerShell command:
<code>Get-Mailbox | Get-MailboxFolderStatistics | Where {$_.ItemsInFolder -gt 5000} | Sort-Object -Property ItemsInFolder -Descending | fl Identity, ItemsInFolder
</code>
It's not an automated limit like you're looking for, but can help you monitor the situation and take action as you see is appropriate. There's a lot more information in the article too, including a downloadable version of the script.
Comment: Apologies. I didn't see anything in the original question that mentioned **how** he was monitoring the folders, so thought he may be doing it manually.  The link, which points to the exchange team blog (http://blogs.technet.com/b/exchange/archive/2009/12/07/3408973.aspx ) gives details of Powershell command which can be used to report only the folders which have more than a specified number of messages, plus a powershell script which can export the information to excel.
Answer: There's no way to do this built into Exchange.  With the right amount of gumption you could do this with Exchange Web Services.  The TotalCount property from the GetFolder operation could be performed on each folder within the users inbox and a total could be established.
See the reference for this here:
http://msdn.microsoft.com/en-us/library/aa580274%28v=EXCHG.140%29.aspx
There's a bunch of reasons to be careful when doing this.  You'd need an account with alot of access.  You'd also want to watch the performance hit this would generate by testing a mailbox or two at a time first.

Limit # of messages in mailbox


Rails: How do you use a search query to find a belongs_to item as well?

Question: What I need is to search for Story titles as well as who the story belongs to all in one search.
Obviously this isn't right but it gets my point across:
@stories = Story.where("user.display_name LIKE ? And title LIKE ?",
             "%#{@query}%", "%#{@query}%").limit(500).page(params[:page])

Stories belong to users and users have many stories.  Not sure what else I need to put here and I haven't really understood any answers that seem to be related.
Thanks.
UPDATE
<% if @stories.size == 0 %>
    Sorry, there were no stories found.
<% else %>

         Title
         User
         Created At
         Updated At

        <% @stories.each do |s| %>

<%= link_to s.title, s %>

<%= link_to s.user.display_name, s.user %>

<%= time_ago_in_words(s.created_at) %> ago

<%= time_ago_in_words(s.updated_at) %> ago

        <% end %>

<% end %>

Answer: If you don't want to eager load the user object associated with the story:
Story.joins(:user).where(
   "users.display_name LIKE :term And stories.title LIKE :term",
   :term =>  "%#{@query}%"
).limit(500).page(params[:page])

If you want to eager load the user object associated with the story:
Story.includes(:user).where(
   "users.display_name LIKE :term And stories.title LIKE :term",
   :term =>  "%#{@query}%"
).limit(500).page(params[:page])

Note:
I am using the placeholder condition in the where clause as the same query term is used in two places.
Comment: Thanks, tried it and it didn't seem to work.  I got this error: SQLite3::SQLException: no such column: user.display_name: SELECT COUNT(count_column) FROM (SELECT  1 AS count_column FROM "stories" INNER JOIN "users" ON "users"."id" = "stories"."user_id" WHERE (user.display_name LIKE '%%%%%' And title LIKE '%%%%%') LIMIT 20 OFFSET 0) subquery_for_count.  The codes you supplied were both the same as well, not sure if that was intentional.  Thanks!
Comment: Changed it to:  Story.joins(:user).where( "display_name LIKE :term And title LIKE :term", :term => "%#{@query}%").limit(500).page(params[:page])  and now it at least searches without errors, but still won't pull anything up that corresponds to a user's display name.
Comment: Updated the answer take a look.
Comment: I updated the question as well to show you the code I have to display results, thinking maybe there is an issue with it.  If I type in a search for a story I know exists, there is no problem, if I search for a user that I know exists, nothing comes up.
Comment: Is it an `AND` condition or an `OR` condition? From your description it looks like an OR condition to me, i.e., `users.display_name LIKE :term OR stories.title LIKE :term`.
Comment: And with that it is fixed!  Thank you good sir.  I have much to learn.


Dynamic SQL for updating a table from ASP .NET

Question: I have an ASP.NET 3.5 application that I want to allow the user to select a table and allow CRUD operations on that table. The user will be restricted to a number of tables to edit however the tables and even database won't be known until after deployment; the web.config will setup the connection and tables.
So I need to build a framework that will allow a generic table in a SQL database to be updated. Most of the ORMs I've seen require the schema to be known at compile time so I cannot use these. The best answer seems to be build the SQL query strings dynamically and use reg expressions to ensure valid input and protect against SQL injections. Is there a better way?
Answer: Yes. How about dataset and datatable objects? you can populate these via a dynamic sql statement, bind them directly to a good editor grid control, and the schema and column data type checking should be a snap. What did we ever do before ORM's and Linq?!?!
Answer: The Dynamic Linq Library is incredible.  You can specify everything as a string, except maybe the table names.

(source: scottgu.com)
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
However, look here for ideas on how to select tables at runtime by dynamically mapping the DataContext:
http://blogs.microsoft.co.il/blogs/bursteg/archive/2007/09/27/25294.aspx
Answer: You can use JJMasterData to dynamic generate SQL scripts for all CRUD operations from a table. We support both .NET 6 and .NET Framework 4.8.
https://www.github.com/jjconsulting/JJMasterData


Add fractions with exponents in the numerator

Question: How does one add these two terms with exponents in the numerator like  $h^2+\frac{h^2}{4}$ ?
According to my lesson on Khan Academy, one should get $h^2\left(1 + \frac{1}{4}\right)$.
However, intuitively, it would seem that one would get $\frac{4h^2}{4}+\frac{h^2}{4}$ having first taken a common denominator and then $5\frac{h^2}{4}$.
After having searched for clarification, none of the search results really helped me to derive the answer. Hopefully this will not add, as such, a redundant post. Please clarify.
Answer: note that $$\frac{h^2}{4}=\frac{1}{4}\cdot h^2$$ thus we have $$\frac{4}{4}\cdot h^2+\frac{1}{4}\cdot h^2=\frac{5}{4}\cdot h^2$$
Answer: Your answer and the Khan academy answer are the same.
Khan Academy: $$h^2\left(1+\frac14\right)$$
You: $$5\frac{h^2}4=\frac54h^2$$
But $$1+\frac14=\frac44+\frac14=\frac54$$So the Khan academy answer becomes $$h^2\cdot\frac54=\frac54h^2$$
which is the same as yours.


Why is V⁷ a seventh chord, while ii⁷ a minor seventh,

Question: I've been looking at chord progressions on HookTheory. But I'm a bit confused by the relative notation. For example here: https://www.hooktheory.com/trends#node=1.57&key=rel
It is possible to switch between a certain key and relative notation in the dropdown box.
Here the V⁷ corresponds to a seventh chord, while ii⁷ corresponds to a minor seventh, IV⁷ to a major seventh. And then I♭⁷ corresponds to a seventh chord. Is it dependent on the scale degree what kind of seventh it is? Or is the V chord a special case, so that when a 7 is given in superscript then it is a seventh chord, and not a major seventh?
Answer: The superscripts in Roman numeral analysis come from figured bass notation. This was a notation used in the Baroque period that allowed the continuo player (e.g. harpsichordist, organist, lutenist, ...) to improvise over the written bass note by using the given notes above the bass. Continuo parts were not written out, because they didn't always know what instrument would be playing them, and they wanted to allow maximum flexibility.
For example, a figured bass 53 means a root position triad, whereas 63 means a first inversion triad, and 753 means a 7th chord. But the 5 and the 3 became so common that they were omitted; thus root position now no longer has any numerals, whereas first inversion has only 6. Finally, 7th chords are now denoted 7.
All figures (not just 7ths) were understood to be in the current key, i.e. without introducing new accidentals. For example, in the key of C major, a ii6 is a ii chord in first inversion, i.e. (bottom to top) F, A, D, or F, D, A (figured bass does not distinguish between these).
By going through the different possible 7th chords (among which, in major, ii7, IV7, V7, and vii ø7 are the most common), you can easily see which 7th chord has which chord quality: ii7 is a minor seventh chord, IV7 is a major seventh chord, V7 is a dominant seventh, and vii ø7 is a half-diminished seventh.
You can alter the quality of the seventh chord by adding accidentals to the figured bass, e.g. ♭7 instead of 7. But in certain cases, it is more natural to write the harmony as an applied harmony instead; for example I ♭7 is really V7/IV, and II 7 ♯3 is really V7/V.
In minor, the situation is exactly the same, except with some conventions around the leading tone. Technically one should write V 7 ♯3 to indicate the dominant seventh chord, but the ♯3 became understood, so the notation simplified to V7. The situation is similar for chords based on ♯vii° (leading tone diminished chords): the ♯ in front gets dropped because it is understood. But beware that other chords (like III) can still have the lowered 7th scale degree in it.
Answer: For any given scale we can decide a set a chords that best go with it—that is the chord tones are picked directly from the scale—by harmonizing it in thirds starting from each scale degree.
So take C Major

note   C D E F G A B
degree 1 2 3 4 5 6 7

So the first chord starts with C then jumps a 3rd to get E then another get G. Notice that we didn't jump a major 3rd to G# we stayed within scale and used G (E to G is a minor third). Finally jump another 3rd to get B making our final chord: C, E, G, B (CMaj7).
Essentially all we're doing is taking every other note (diatonic thirds) from the scale we wrote out. That will give us a set of chords whose relative qualities will stay the same no matter what note you start from (the key you're in):

IMaj7   - CMaj7  - C E G B
iim7    - Dm7    - D F A C
iiim7   - Em7    - E G B D
IVMaj7  - FMaj7  - F A C E
V7      - G7     - G B D F
vim7    - Am7    - A C E G
viim7b5 - Bm7b5  - B D F A

So if you changed to a D major scale, the iim7 and V7 would now be Em7 (EGBD) and A7 (AC#EG) but ii is still a m7 and the V still a (dominant) 7. So, yes, it depends on the scale degree.
As for some of the confusion over 7ths, yes when it just "7" that's a regular 7th chord or a "dominant" 7th chord that has major 3rd but lowered 7th (C7=C,E,G,Bb). A Major 7th has the major 7th (CMaj7=C,E,G,B) and will be designated as either Maj7 or use a little triangle instead of the "Maj".
You also briefly mentioned a Ib7. While that's not the notation that I'd use, the "b" is probably a courtesy to let you know that the 7th is lowered in this particular I chord which would make it a dominant 7 or a I7. It's worth noting that just because we made a set of chords that are made up entirely of scale tones that doesn't make we can't sometimes use notes from outside of the scale. For instance in a lot of blues and pop you'll see I-IV-V progressions where they are all dominant 7ths.


Allocating memory for structure types in C

Question: Im having an issue in my program where I define a structure type but not a structure variable in a header as such.
typedef struct
{
    int a;
    int b;
    int c;
    Token d;
} Foo;

I then want to use this struct foo later on in a .c file that does infix to postfix
#include "header"
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
#include <stdlib.h>

int infix2postfix(char *infix, Arr arr)
{
    struct Foo foo;
    char szToken[MAX_TOKEN];
    Stack stack = newStack();
    infix = getToken(infix, szToken, MAX_TOKEN); //provides next token to be scanned by function.

    ... //push pop using switch case didn't post code for simplicity.
    case...
    push(stack, *foo.a);

     ...
    case...
    pop(stack);

    ...

    goOut(arr, *foo.d); //goOut(function that populates and "arr" Array from printing.

}

So when I compile here I get
error: storage size of 'foo' isn't known struct Foo foo;

I have tried struct Foo *foo = malloc(sizeof foo); to allocate memory but it messes up my push(stack, *foo.a); and goOut(arr, *foo.d); How do I go about fixing this? Do I have to allocate memory in the infix2postfix function first then declare a structure variable?
Comment: You have typedef it to `Foo`. So it need to be `Foo foo;`. That is, no `struct`.
Comment: No. Should be: `Foo *foo = malloc(sizeof *foo);` or `Foo *foo = malloc(sizeof Foo);`. I prefer the former as it requires less maintanence if the type changes.
Comment: yep typedef it to Foo
Comment: So when trying to allocate memory it should be `Foo *foo = malloc(sizeof foo);`? @Alan Au
Comment: So I did this and now my `push(stack,*foo.a); `and `goOut(arr, *foo.d); `functions are not working :( Messed up my pointers.
Comment: Hm, I don't see how it could have messed it up since you never had it working in the first place. You can't say it was working before if it wasn't compiling! You obviously have problems elsewhere. Post a new question with the details if you really can't work it out.
Comment: Yeah haha. My issue was that even though I declared the struct variable in the function infix2postfix(); I can't seem to figure out how to allocate memory for it without messing up the rest of my code :(. Im afraid to post it because it might be completely wrong too lol.
Answer: you have already defined Foo as a typedef struct, so you do not use struct Foo again to declare foo, just use
Foo foo; to declare not struct Foo foo;
Comment: Yeah I did this put now my `push(stack, *foo.a);` and `goOut(arr, *foo.d); `have messed up pointers. since *element isnt a structre :(.
Comment: either declare foo as a pointer to Foo.
Foo *foo;

then in push(stack, foo->a)

or declare foo as a struct
Foo foo;
then
push(stack, foo.a)
Answer: You defined a type Foo which is a tagless struct type.  You could have a separate struct Foo { int anonymous; char name[MAX_NAME]; }; which is wholly unrelated to the type Foo.  (It would be very confusing for humans, but the compiler would have no problem.)
In your function, you should write:
int infix2postfix(char *infix, Arr arr)
{
    Foo foo;

Comment: I had this in my function since I needed a foo variable, but it conflicts with the second argument in both push(); and goOut();. Compiler spits 'invalid uinary *(have int) for push and 'expected Foo but argument is type char for goOut();
Comment: You would need to write just `foo.a`, for example.  The variable `foo` is a structure, and one of its elements is an `int` identified by `foo.a`, and you can' tally the indirection operator `*` to a plain `int`.  If you had an appropriately initialized pointer, `Foo *bar = …;`, you could write `(*bar).a`, or more naturally, `bar->a`.
Comment: It might be that my code is just bad. I tried this aswell and in both `push(stack,foo.a);` and `goOut(out, foo.d);` with the `Foo foo;` declaration and I get `expected 'Foo' as argument 2 but argument is of type int` and `*char` respectively. So somewhere I managed to mess up the syntax of the arguments.