How to create line chart with JSON data using D3

Question: I am working with D3.js and react-hooks to create charts, So I tried creating one Line chart by searching around and got one.

But the one I am working with is using Sample data, here in my case I have JSON data.
I have made the charts responsive as well using resize-observer-polyfill this library.
Now I am struggling to implement it with JSON data, to renders it with dynamic data.

What I did
const svgRef = useRef();
const wrapperRef = useRef();
const dimensions = useResizeObserver(wrapperRef); // for responsive

// will be called initially and on every data change
useEffect(() => {
    const svg = select(svgRef.current);
    const { width, height } =
        dimensions || wrapperRef.current.getBoundingClientRect();

    // scales + line generator
    const xScale = scaleLinear()
        .domain([0, data.length - 1]) // here I need to pass the data
        .range([0, width]);

    const yScale = scaleLinear()
        .domain([0, max(data)])
        .range([height, 0]);

    const lineGenerator = line()
        .x((d, index) => xScale(index))
        .y((d) => yScale(d))
        .curve(curveCardinal);

    // render the line
    svg
        .selectAll('.myLine')
        .data([data])
        .join('path')
        .attr('class', 'myLine')
        .attr('stroke', 'black')
        .attr('fill', 'none')
        .attr('d', lineGenerator);

    svg
        .selectAll('.myDot')
        .data(data)
        .join('circle')
        .attr('class', 'myDot')
        .attr('stroke', 'black')
        .attr('r', (value, index) => 4)
        .attr('fill', (value, index) => 'red')
        .attr('cx', (value, index) => xScale(index))
        .attr('cy', yScale);

    // axes
    const xAxis = axisBottom(xScale);
    svg
        .select('.x-axis')
        .attr('transform', `translate(0, ${height})`)
        .call(xAxis);

    const yAxis = axisLeft(yScale);
    svg.select('.y-axis').call(yAxis);
}, [data, dimensions]);

    <React.Fragment>

            <svg ref={svgRef}>
                <g className="x-axis" />
                <g className="y-axis" />
            </svg>

    </React.Fragment>

Here I am not able to pass the data, my data is below
[
  { anno: 2014, consumo: 300, color: "#ff99e6" },
  { anno: 2015, consumo: 290, color: "blue" },
  { anno: 2016, consumo: 295, color: "green" },
  { anno: 2017, consumo: 287, color: "yellow" },
  { anno: 2018, consumo: 282, color: "red" },
  { anno: 2019, consumo: 195, color: "white" }
]

Here in my data I have color for each data, which I want to show in each dot generated.
working code sandbox of line chart
Similarly I tried doing bar chart and it is working fine
I did some dynamic rendering to labels, when we resize the window the labels gets adjusted automatically.
Here is the full working bar chart what I am trying to implement to line chart
I have also commented the lines where I am doing what.
Edit / Update
I ahve been following @MGO 's answer and it helped me Alot, but still I am facing issue to align labels and filling the color to dots.

actually it is obvious that it will overlap because of the text size, but just to overcome that In bar chart I have used below code
 const tickWidth = 40;
const width = xScaleLabels.range()[1];
const tickN = Math.floor(width / tickWidth);
const keepEveryNth = Math.ceil(xScaleLabels.domain().length / tickN);

const xScaleLabelDomain = xScaleLabels
  .domain()
  .filter((_, i) => i % keepEveryNth === 0);
xScaleLabels.domain(xScaleLabelDomain);

what it is doing is when the device size is small it will  filter some labels and will not show labels.
And also I am using below code to give color
.attr("fill", ({ color }) => color)

But is is not taking any color, but it is taking by default black color.
I have data to show as label as July.9.2021 11:18:28 but I only want to show time so what I am doing in my bar chart code is below
const xScaleLabels = scaleBand()
        .domain(
            data.map(
                ({ sensorValueAddedTime }) => sensorValueAddedTime.split(' ')[2]  // this I am using to show only time
            )
        )
        .range([0, dimensions.width])
        .padding(padding);

Same I am trying to do with Line chart, In a simple way, I do not want to change this to any time and all.
This is the second data, so basically the Answer I got is only working for 1st data not for second, I want that to be dynamic if data comes in any of these format I want to show.
sensorValueAddedTime I want to show on x-axis
sensorValue On y-axis
I have already added my bar chart full working code, Same I want to do with line chart.
    [
    {
      sensorValue: 32,
      sensorValueAddedTime: "July.9.2021 10:56:22",
      color_code: null,
      condition: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 32,
      sensorValueAddedTime: "July.9.2021 10:56:23",
      color_code: null,
      condition: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 35,
      sensorValueAddedTime: "July.9.2021 11:17:51",
      color_code: null,
      condition: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 35,
      sensorValueAddedTime: "July.9.2021 11:17:52",
      color_code: null,
      condition: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 36,
      sensorValueAddedTime: "July.9.2021 11:18:08",
      color_code: null,
      condition: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 36,
      sensorValueAddedTime: "July.9.2021 11:18:09",
      color_code: null,
      condition: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 38,
      sensorValueAddedTime: "July.9.2021 11:18:27",
      condition: null,
      color_code: null,
      __typename: "sensorData"
    },
    {
      sensorValue: 38,
      sensorValueAddedTime: "July.9.2021 11:18:28",
      condition: null,
      color_code: null,
      __typename: "sensorData"
    }
  ]

Comment: To clarify, do you want to produce a line chart with the array that has {anno: 2014, consumo: 300, color: "#ff99e6" }, or with the array in the Codesandbox? I've answered using the data provided in the Codesandbox. Is that what you're asking?
Answer: In the original example from  Muri's D3 with React Hooks video series data is a flat single-dimensional array: [10, 25, 30, 40, 25, 60].
In the new array you're providing (data1), we're asking D3 to render a chart based on an array of objects. So when we pass data1 in as a prop to our chart function, we need to do a little more work to access the values inside of our array of objects.
We'll need to scale the data values into the appropriate ranges. The original example looked up the values by their index in the array. We're going to need to access the values in the objects contained in the data1 array based on those objects' keys. Like this, to create our scales:
const yScale = scaleLinear()
                  .domain([0, max(data, (d) => d.sensorValue)])
                  .range([height, 0]);

// scale the values of sensorValueAddedTime into the range for our x axis values
// since it's a date you'll need to convert the strings to Date objects:

const xScale = scaleTime()
                 .domain([
                   new Date(min(data, (d) => d.sensorValueAddedTime)),
                   new Date(max(data, (d) => d.sensorValueAddedTime))
                  ])
                 .range([0, width]);

And like this, in our lineGenerator function:
const lineGenerator = line()
      .x((d) => xScale(new Date(d.sensorValueAddedTime))
      .y((d) => yScale(d.sensorValue));

You'll notice above that if we're going to use scaleTime(), we'll have to convert sensorValueAddedTime into a value that D3 can make sense of -- right now there's an extra space in the string in your object, but if you strip that space you can convert to a Date object and use d3.scaleTime().
There's an updated, working Sandbox that's rendering this data to a line chart with additional comments here.
Update: OP was asking to plot a different set of data and asked that the points "appear in that order".
There's a different working Sandbox rendering this different set of data to a line chart correctly here.
It seems like what you're asking for -- the "line graph version of the bar chart" may not produce what you're after. Just talking it through:
If you use the index position of each object on your xScale, like this:
const xScale = scaleLinear()
      .domain([0, data.length - 1])
      .range([0, width]);

that will produce a pretty boring chart -- because you're saying "put a dot at every whole number representing the index of the array". In other words, a dot at 1, at 2, at 3, and so on.
For this chart, we should choose meaningful values in our data -- like data1.anno -- and use those to communicate.
For example:
Scaling the values into the range according to the index positions of the array and plotting them according to their index position, like this:
const xScale = scaleLinear()
      .domain([0, data.length - 1])
      .range([0, width]);

....chart code....

.attr("cx", (value, index) => xScale(index))

Produces this plot:

Solution
Scaling the values of the data into the range and plotting the points according to their values. We can use d3's extent method, like this:
const xScale = scaleLinear()
      .domain(extent(data, (d) => d.anno))
      .range([0, width]);

    const yScale = scaleLinear()
      .domain([0, max(data, (d) => d.consumo)])
      .range([height, 0]);

....chart code...

.attr("cx", (value) => xScale(value.anno))
.attr("cy", (value) => yScale(value.consumo));

Produces this plot:

data1.anno is clearly a year. The option to parse the years as date obects is a good one. var parse = d3.timeParse("%Y"), then use this to set the domain of your time scale: .domain([parse(minDate), parse(maxDate)]), as shown here.
Otherwise, if you'd like to preserve that value on the x-axis and not treat it as a Date object, you can use Javascript's toFixed() method to remove the decimal places, remove the comma delimiter with d3.format and change the number of ticks that appear with axis.ticks().
That produces this chart:

To generate a line with these points, use the values in the data, not the indexes.
const lineGenerator = line()
      // .x((d, index) => xScale(index)) // old
      // .y((d) => yScale(d)) // old
      .x((d) => xScale(d.anno)) // new
      .y((d) => yScale(d.consumo)) // new
      .curve(curveCardinal);

That produces this chart:

Finally, if your goal is to assign the color value in each object to a point, access those values and assign their values to the fill, like this:
.attr("fill", (value) => value.color)

That produces this chart:

Here's the working sandbox.
Hope this helps! ✌
Comment: i have updated code sandbox with proper data, please check once
Comment: actually i do not want to change that ti date object, I want it to show how data is coming in that order
Comment: Got it, the answer is now updated. tl;dr, you will still need to access the values in each object in the array. Let me know if that answers the question and mark it if it does. Here's the working sandbox with the different dataset https://codesandbox.io/s/eloquent-vaughan-pg99v?file=/src/BrushChart.js:1550-1594
Comment: Hey i tried your code, now if i am trying to do with previous data that is  `July.9.2021 10:56:22` date is coming in this form and i want to show in same format so it is not working, actually i do not want to do any other thing just what ever data i am getting i want to show on that order so data is not same for every case as my data is dynamic so in one place i am getting like above, so `tofixed` is causing issue, when i removed that the chart is not populated clearly.
Comment: That's expected, since `sensorValueAddedTime` is a string. The value in the other object (`anno`) is a number.

The answer above correctly describes how to chart those values with D3 per the original question. Reading that value in as a Date object would solve the problem in both cases.

It seems like your question may actually not be about D3 and instead have more to do with processing and normalizing data from different sources -- "Given objects that have values of different types, how can I process the JSON so that the values can be represented in the same dimension?"
Comment: Here's the Sandbox that shows how you can turn the value you want to plot on the x-axis as a Date object. I will add it to the original answer.

https://codesandbox.io/s/lucid-haslett-7vobh?file=/src/BrushChart.js


Python Dataframe: Different RMSE Score Every Time I Run Random Forest Regressor

Question: I currently run a random forest model using the following code.  I set a random_state equal to 100.
from sklearn.cross_validation import train_test_split

X_train_RIA_INST_PWM, X_test_RIA_INST_PWM, y_train_RIA_INST_PWM, y_test_RIA_INST_PWM = train_test_split(X_RIA_INST_PWM, Y_RIA_INST_PWM, test_size=0.3, random_state = 100)

# Random Forest Regressor for RIA_INST_PWM accounts

import numpy as np
from sklearn.ensemble import RandomForestRegressor

regressor_RIA_INST_PWM = RandomForestRegressor(n_estimators=100, min_samples_split = 10)
regressor_RIA_INST_PWM.fit(X_RIA_INST_PWM, Y_RIA_INST_PWM)

print ("R^2 for training set:"),
print (regressor_RIA_INST_PWM.score(X_train_RIA_INST_PWM, y_train_RIA_INST_PWM))

print ('-'*50)

print ("R^2 for test set:"),
print (regressor_RIA_INST_PWM.score(X_test_RIA_INST_PWM, y_test_RIA_INST_PWM))

And then I use the following code to calculate the prediction values.
def predict_AUM(df, features, regressor):

    # Reset index for later merge of predicted target values with Account IDs
    df.reset_index();

    # Set predictor variables
    X_Predict = df[features]

    # Clean inputs
    X_Predict = X_Predict.replace([np.inf, -np.inf], np.nan)
    X_Predict = X_Predict.fillna(0)

    # Predict Current_AUM
    Y_AUM_Snapshot_1yr_Predict = regressor.predict(X_Predict)
    df['PREDICTED_SPAN'] = Y_AUM_Snapshot_1yr_Predict

    return df

df_EVENT5_20 = predict_AUM(df_EVENT5_19, dfzip_features_AUM_RIA_INST_PWM, regressor_RIA_INST_PWM)

Finally, I calculate the RMSE of my results:
from sklearn.metrics import mean_squared_error
from math import sqrt

rmse = sqrt(mean_squared_error(df_EVENT5_20['SPAN_DAYS'], df_EVENT5_20['PREDICTED_SPAN']))
rmse

Each time I run my code ... my RMSE changes.  It has varied from 7.75 to 16.4  Why is this happening? And how can I have the same RMSE each time I run the code?  Additionally, how do I optimize my model for RMSE?
Comment: Did you get back to trying my suggestion?
Answer: You only seeded the train_test_split which makes sure that the random allocation of the data to train and test set is reproducible.
As the name suggests RandomForestRegressor also contains parts in the algorithm that rely on random numbers (e.g., specifically different parts of data or different features for training individual decision trees). If you want reproducible results, you need to seed it as well. For that you need to initilize it with random_state like that:
regressor_RIA_INST_PWM = RandomForestRegressor(
                           n_estimators=100,
                           min_samples_split = 10,
                           random_state=100
                         )


How to prove the multiplicative cancellation of the orders on natural numbers

Question: I'm trying to prove the statement
$$m < n \iff mp < np$$
for all $m, n, p \in \mathbb{N}$ with $p$ non-zero.
The implication seems easy enough, with
$$m < n \implies n = m + c \implies np = (m + c)p = mp + cp \implies mp < np$$
for $c, cp$ non-zero (which is easy to prove) being a fairly simple deduction. However, whenever I try to tackle the converse I get stuck almost immediately.
All my attempts are halted after $mp < np \implies np = mp + c$ with no ideas where to go. Is there a simple proof that I'm just missing?
Comment: Divide both sides by $p$?
Comment: @Raffaele I assume OP wants a proof in PA, i.e. not leaving $\Bbb N$.
Comment: You can probably do it by contradiction. Assuming $m\ge n$ and deducing $mp\ge np$ is almost the same as the first direction and some additional case for equality.
Answer: Since, $n>m$ or $n-m>0$, we obtain:
$$np-mp=(n-m)p>0,$$
which gives $np>mp.$
Comment: This addresses the $\Rightarrow$ direction the OP found easy enough, not the $\Leftarrow$ direction they got stuck on.
Comment: @kimchi lover If $np-mp>0$ so $(n-m)p>0$ and since $p>0$, we obtain $n-m>0$ or $n>m.$
Answer: Without requiring division or subtraction (both may be problematic when working explicitly only in $\Bbb N$:
Assume $mp<np$. Exactly one of $m<n$, $m=n$, $m>n$ is true. Of course, $m=n$ would lead to $mp=np$, and you already showed, $m>n$ would lead to $mp>np$. Hence only $m<n$ remains.


How to use ForeignKey field in Django Model

Question: I am using this in my Model
bank  = models.ForeignKey(Bank)

But in my editable form I have the select box but the values which appear is like
Bank object
Is there any option so I can see field like Bank.name in select box
Answer: By overriding the __unicode__ method of your Bank class, you can provide alternative display values for your model instances.
class Bank(models.Model):

    ...

    def __unicode__(self):
        return unicode(self.name)

Answer: define a unicode method for Bank class like below:
def __unicode__(self):
    return self.name

Answer: I think you are looking for the Choicefield, try this link:

http://www.viksit.com/node/16


PHPUnit tests running too quickly, making no difference in MySQL timestamps

Question: I have a PHP class that create a line with a CURRENT_TIMESTAMPS in a MySQL database when I create a new instance of it. This timestamp is then used to sort theses records (and primarily getting the last one).
In this precise case, the timestamp is the automatically generated created_at column of an Eloquent (Laravel) model. If possible I'd like a solution that is not Laravel-specific given I could encounter the same problem with other projects.
I had written tests for this class on another PC and everything was working fine, the resulting oder was always correct.
Now I copied this project on my other PC that has an SSD and that is (but not much) more powerful than the other. And now, all my tests fail, because all records end up having the exact same timestamp.
In production it would not be a an issue if two records had the same timestamps (which is very unlikely), but that's not the situation I try to replicate in my test.
To quickly address the issue without digging in the code, I added some sleep(1) functions before instantiating a new object.
But now my question is, what is the best way to deal with that ?
Here are a few possibilities I thought of, but I'm not sure which one is the best:

Hard-coding the dates in the tests: no risk of failure, but my objects currently offer no way to define a date manually. I would need to add methods for that, and they won't be used for anything else.
Empty the database before doing the next operation. The problem there is that for certain tests I need to insert two objects one after the other, so it wouldn't solve all the problems
Make the code wait. But it sounds very stupid to me, because I lose all the benefits of having a powerful machine. And of course it takes much more time to run the tests
Manipulating the system or the database current time. I have no idea of what horrible things could happen if I try that ! Maybe it is possible to isolate a process that do that without impacting anything else on the PC ?

So which method do you recommend ? Is there a better one ?
More system informations: All my dev machines are running Ubuntu 14.04 with PHP 5.5 and MySQL 5.5. My current project is using the Laravel 4 framework and phpunit for the tests
Additional informations
I'd like to add there what I said in the comments below.
Ideally my test should do its assertions with dates that are spaced by many hours, but for the moment I relied on the fact that two records had at least one second difference, which is no more the case with this other PC config.
And, additional informations, in production, the older records get normally deleted before a new one is inserted or the db queried (say, 90% of the time) so even if two were created quickly, there would be a chance that one is removed before I do my logic on the database. But for my test, I need to have at least two records in database, spaced by some time, which is not important but greater than zero.
Comment: Smells like a design failure... Do you really need to relay on a timestamp to get the last record? Imho this will fail at some point. How about switching to a pk with an auto incremented value?
Comment: @aichingm And for me an auto-increment field should not be trusted for sorting. What if two synced servers use different incrementing starts, like 1 for the first and 10000 for the second one ? I could add another field in the db but again, this case (with identical times) is deliberately not treated in my application
Comment: @aichingm I agree with you, but my problem here is how to "simulate" the missing time between two records. What I'm testing here would normally be spaced by many hours in production, but I need a test that run in seconds.
Comment: does the timestamp field has a `UNIQUE` constraint? And what type of error handling do you use?
Comment: you can try current time for first insert and then increment that time stamp with some constant number , like $time = time()+10 , for all other insertions
Comment: @aichingm no there are no UNIQUE constraint, and no error handling for duplicates, given it's a legitimate situation for my app (but not the one I try to test)
Comment: @Deep123 yes it falls under the first possibility of my list. The thing is that if I do that I need to permit the used to manually set a time when using my class, which I would prefer not to do. (But for now it sounds the safer and easier method to achieve my goal)
Comment: For code testing, you could mock the results to see your code handles when the record is missing by your logic, etc...  You could simply sleep in your test for x seconds, to get a different time stamp.  I would watch though that depending on time stamps may have issues if you have to scale up to many servers.  If not, then you can do this, but as others have commented, it can be a risk.
Answer: I've found out a good solution that apply to Eloquent only.
This is the same that is used for the Database Seeds.
We can add Eloquent::unguard() before instantiating the object in the test:
Eloquent::unguard();

Model::create(array('created_at' => '2014-12-17'));

So we can manually set the created_at field of the model, without exposing it to "everybody" who use the class.
Answer: You can use Carbon::setTestNow($now) for mocking now time
foreach ($list as $index => $test) {
    Carbon::setTestNow(now()->addDays($index + 1));
    // functionCreateModel();
}


How to remove or hide the Subject field for comments?

Question: I have added a comment field to a content type. It is of the default comment type.
I don't want my commenters to fill in a subject, rather, just a comment body.
Trying to remove the Subject field, I went to admin/structure/comment/manage/comment/fields but there I only find comment_body, not something like comment_subject.
How to remove or hide the Subject field for comments?
In earlier phrasing, how to not require a subject from commenters?
Answer: You can't remove the subject, but you can hide it so that it will be auto-generated and not displayed.
Go to Admin -> Structure -> Comment types -> Default Comments -> Manage Form Display tab (/admin/structure/comment/manage/comment/form-display) and then drag Subject to Disabled.  Then save.
Note: If you only see the heading "Comments" but no name field and no comment field, and you are not logged in, then you need to give anonymous users permission to post comments.
Comment: On **Manage form display**, you should see **Author** and **Comment** (the comment body), and then below that will be **Disabled** with **Subject**.  This is working in Drupal 9.4.6; I wrote this answer based on a site configured as written.
Comment: Also check your theme.  Some non-standard themes have poor support for comments.


Difference between non-linear curve fitting and interpolation

Question: I understand the difference between linear curve fitting and interpolation. In interpolation, the targeted function should pass through all given data points whereas in linear curve fitting we find the general trend of dependent variable. The cost function could be the distance between them.
If we keep on going with same sense of cost function, are not in case of interpolation the difference between data points zero ?
Should we not call interpolation as non-linear curve fitting ?
If not then what is the difference
Answer:
In interpolation, the targeted function should pass through all given
  data points whereas in linear curve fitting we find the general trend
  of dependent variable.

Not at all! Interpolation is

a method of constructing new data points within the range of a
  discrete set of known data points.

You can use many different methods for interpolation including linear interpolation and polynomial, or spline curves.
When you are fitting curve to the data it is up to you to decide how close do you want it to fit the data. Nothing stops you from choosing the curve that perfectly fits to your data.
We are talking about interpolation when you use the fitted curve to re-create, or guess, the unobserved datapoints.
Comment: Are you saying that interpolation and non-linear curve fitting are one and the same? I would distinguish them by noting that interpolation tends towards the tautological where non-linear curve fitting involves fitting a model of some type, e.g., polynomial or spline curves, as you note. But that's just my POV.
Comment: @DJohnson Exactly the opposite: they are two different things :) Curves can be *used* for interpolation, as one of the many tools available. That's all.
Comment: @Tim Your are answering to this Question: 'What is interpolation'. You just explained interpolation with another way, probably with an Engineering perspective where we get data from sensor whose resolution does not fulfill our requirements.  I have implemented polynomial interpolation,
Comment: @hmluqman I do not understand what do you mean..?


Descomposition on temporal sobolev space

Question: Let $\Omega$ an open subset of $\mathbb{R}^2$ with Lipschitz boundary.
Can I descompose in a unique way any $u\in L^2(0,T;L^2(\Omega))$ such that for all $t\in [0,T]$,
$u(t)=u_1(t)+u_2(t)$
with $\dfrac{d}{dt}u_1(t)=0$ ?
I hope that the descomposition gives $\dfrac{d}{dt}u(t)=\dfrac{d}{dt}u_2(t)$
I think that it's possible using the Ortogonal descoposition theorem on $L^2(\Omega)$, but I'm not sure it is properly.
Answer: No, you can not. Let $x \in L^2(\Omega) \setminus \{0\}$ be arbitrary. Then, you have$$u(t) = 0 + u(t) = x + (u(t) - x)$$
and $$\frac{d}{dt} 0 = \frac{d}{dt}x = 0.$$
Note that $d u_1 / dt = 0$ is equivalent to $u_1$ being constant (in time) and there are many constant functions.
Comment: ok, thanks, your answer helps me to another idea that i'm developing


Nested Uniform Distributions in Monte Carlo Integration

Question: In terms of importance sampling for numerical Monte Carlo integration we can proceed as follows:
\begin{align}
\int_{\Omega} p(\mathbf{x}) d\mathbf{x} &= \int_{\Omega} p(\mathbf{x}) \frac{q(\mathbf{x})}{q(\mathbf{x})}d\mathbf{x} \\
& = E_{\mathbf{x} \sim q(\mathbf{x})}\left[\frac{p(\mathbf{x})}{q(\mathbf{x})}\right] \\
& \approx \frac{1}{N} \sum_{i=1}^N \left[\frac{p(\mathbf{x}_i)}{q(\mathbf{x}_i)}\right]
\end{align}
Therefore we can empirically estimate the integral by sampling $\mathbf{x}$ from some distribution $q(\cdot)$.
Now in my problem I know for a fact that my $\mathbf{x}$ is a finite vector ($\mathbf{x}=[x_1,x_2,x_3,...x_n]$), all values are contained in the interval $[0,1]$ (so $\Omega = [0,1]^d$), and there is a particular ordering to my $\mathbf{x}$, i.e.:
$x_1 \geq x_2$
$x_2\geq x_3$
$x_2 \geq x_3$ etc....

So given the structure of my problem (nested, all values in $[0,1]$), my questions are:
Question 1: Given this nested structure to my problem, is it better to choose certain $q(\cdot)$ functions over others?
I feel that because all my variables are in the interval $[0,1]$, and I have no reason to believe any particular weighting of the points (I am actually evaluating an integral to help find a volume), so should I choose a uniform distribution?
Question 2: If I opt for a uniform distribution for $q(\cdot)$, do I need to reflect the prior structure of my variables in the sampling procedure?
i.e. I cannot just sample $\mathbf{x} \sim U[0,1]^n$, as this will occasionally violate my required hierarchical/ordering/nested structure.
Question 3: In many importance sampling integration problems, they give the example of uniform 1D, in which case $q(\cdot) = \frac{1}{b-a}$, so this term can be removed from the expectation/sum to the front. How can I calculate this same volume, assuming I opt for $q(\cdot)$ to have a nested hierarchical structure? How should this scale and be dealt with

Thus ultimately I am unable to understand how to properly include such a prior knowledge of the nested nature of my variables into MC integration.
Answer:
Question 1: Given this nested structure to my problem, is it better to
  choose certain $q(⋅)$ functions over others?

A generic recommendation is to have the importance function $q(⋅)$ supported by the same support as the integrand $p(⋅)$. Therefore, if $\mathbf x$ belongs to the restricted unit hypercube, the function $q(⋅)$ should only be positive over this restricted unit hypercube.

Question 2: If I opt for a uniform distribution for $q(⋅)$, do I need to reflect the prior structure of my variables in the sampling
  procedure?

The constraint on the components of $\mathbf x\in(0,1)^n$ can be expressed by a change of variables:
\begin{align*}
x_1 &= y_1 &y_1\ge 0\\ x_2 &= x_1 + y_2 &y_2\ge 0\\ &\vdots \\ x_n &=x_{n-1}+y_n
&y_n\ge0\\ &\qquad\qquad y_1+\ldots+y_n\le 1
\end{align*}
which indicates that $\mathbf y=(y_1,\ldots,y_n)$ belongs to the simplex of $\mathbb R^n$ (or $\mathbb R^{n+1}$ depending on the convention). A natural family of distributions on the simplex of $\mathbb R^{n}$ is made of the Dirichlet $\mathcal D(\alpha_1,\ldots,\alpha_{n+1})$ distributions with density
$$f(\mathbf y)=\dfrac{\Gamma(\alpha_1)\cdots\Gamma(\alpha_{n+1})}{\Gamma(\alpha_1+\cdots+\alpha_{n+1})}y_1^{\alpha_1-1}\cdots y_{n+1}^{\alpha_{n+1}-1}$$
over this simplex of $\mathbb R^{n}$. The Uniform case corresponds to
$$\alpha_1=\cdots=\alpha_{n+1}=1$$

Question 3: In many importance sampling integration problems, they
  give the example of Uniform 1D, in which case $q(⋅)=1/(b−a)$, so this term
  can be removed from the expectation/sum to the front. How can I
  calculate this same volume, assuming I opt for $q(⋅)$ to have a nested
  hierarchical structure?

The volume of the simplex of $\mathbb R^{n}$ is given by the constant
$$\dfrac{\Gamma(\alpha_1+\cdots+\alpha_{n+1})}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_{n+1})}$$
Comment: never mind, I was being a bit silly, for some reason I thought $\frac{\partial }{\partial y_1}(y_1 + y_2 + y_3) = y_2 + y_3$ or something ridiculous.... when the answer should be 1, thereby creating the lower triangular matrix of completely 1s.
Comment: Thank you! :)

Just one follow up. Suppose I perform the proposed change of variables to the integral, I will require the determinant of the Jacobian. For a simple system ($x_1 = y_1, \quad x_2 = x_1 + y_2 = \sum_i^2 y_i, \quad  x_3 = x_2 + y_3 = \sum_i^3 y_i$), the jacbian matrix should be (I believe):

\begin{matrix}
\left[1 $0 $0 \\ y_2 $  y_1 $ 0\\ y_2 + y_3 & y_1 + y_3 & y_1 + y_2 \right]
\end{matrix}
Comment: The Jacobian is lower triangular with only 1's hence equal to 1.
Comment: Sorry, I didn't finish my equation before properly (the time to edit ran out!).

I thought the Jacobian should look something like this:

\begin{bmatrix}
1 &0 &0 \\
y_2 &  y_1 & 0\\
y_2 + y_3 & y_1 + y_3 & y_1 + y_2 \end{bmatrix}

I get the lower triangular part, but for each row we would have something like $\partial x_n / \partial y_m$ so wouldn't there need to be some sort of residual sum of the other terms left over (i.e. I don't get how it becomes only 1s (but if it is that is super nice!)


Large diaphragm condenser mic without pop filter

Question: I see some artists singing into a large diaphragm condenser mic without pop filter and can hear no 'popping sounds' even though they are singing quite closely to the mic (see

 for an example).
My question: are some large diaphgram condenser mics less sensitive to popping sounds than others (because I thought all mics of that type required a pop filter for completely eliminating popping sounds)? If so can you recommend a mic of this type that does well without popfilter (and that costs under 500 dollars)?
Thanks for your time!
Regards,
Bart
Comment: Having now *actually* listened to it - whatever mic they did use on her voice, or whatever multi band comp they put on afterwards ... I would avoid like the plague. It's crippling her esses.
Comment: I call "just for show", style over substance. If you want a pop-free live mic [with cool factor], use a good lav mic, off-axis. Google 'DPA 4060', you can get one of those for just inside your budget. [Radio gear extra]
Answer: I don't think she is really that close. And she is not singing directly into the mic. Remember that plosives are caused by a large puff of air from your mouth hitting the diaphragm. If you are above the mic a bit, like she is in the video, the air will go over the mic.
found this with a quick google search:

As always, prevention is infinitely better than cure, and stopping plosives from reaching the mic is really all about positioning. Ideally, the mic should be positioned well above and/or slightly to one side of the mouth. I find that raising the mic to around forehead height works well, as this keeps it away from the track of direct plosive blasts from the mouth, and also encourages the vocalist to stand up straight, which aids their breathing. If the recording environment is adequate, using an omnidirectional mic helps, because it is less sensitive to the pressure changes caused by plosives.

In addition, better singers develop both the singing techniques and the mic techniques to minimize plosives and sibilance. Slightly changing consonants (e.g., voicing "p"s slightly so they sound kind of like "b"s, d for t, z for s), and slightly turning off-axis to the mic for certain parts of the song are time-honored techniques for being mic-friendly. Lyricists will often look for ways to avoid words with plosives in them, since they literally stop the flow of the singing, and also cause the singer's airway to have to stop and restart.
With the right writers, singers, and engineers, great tracks can be recorded without pop filters. But it's rare, these days.
Comment: Learning how to *almost* completely eliminate consonants/plosives was a great part of how to do block BVs, back in the day. No-one wants 26 T's at the end of a word, all at slightly different times.
Answer: She wasn't actually performing live on the video of course!   Very likely there WAS a pop shield for the recording.
As it's so easy to use one when recording, there's no real need to design a recording mic with integral pop rejection.  'Live' mics like the SM4 or SM58 are another matter.  Even then, it's about designing a pop-sock into the construction rather than not having one at all.
Comment: A better reason not to build pop filtering into a mic is because hardly any mics are 100% dedicated to recording vocals. Live vocal mics obviously are more about vocals so integrated pop filtering is helpful there, but studio condensers are used for all kinds of sources.
Comment: @ToddWilcox - I'm trying to figure out how not having a pop filter would be beneficial, for other uses. Does it make much difference to the sound while recording other instruments?
Comment: @Tim It makes a difference when recording vocals, not just instruments! A pop filter prevents pops, but it also affects the sound. Even on the lowly SM-58, if you unscrew the pop filter/grille and (carefully) use the mic without it, you get a bit more high end.
Comment: I'd be tempted to say that a vocal mic with built in pop filter is already EQ'd to counteract it. Having said that, U87s used to come with pop filter "mickey mouse noses/condoms" though I've never actually used one with a pop filter other than the more modern 'pair of ladies tights over a coat hanger" variety
Comment: @ToddWilcox - I use them all the time for recording vocals, but meant with other instruments. I guess they may dull the sibilants a bit, and the very top end of some instruments' ranges.
Comment: @Tetsujin A 57 has no internal EQ at all. It's a capsule and a transformer. And you don't want any EQ that you don't need. You can't EQ without a phase shift so better to allow the pop filters be used when they are needed and not when they aren't.
Comment: @Tim If you're recording all the time and you haven't heard for yourself the difference between with and without a pop filter - I don't know what to say. Don't tell any of your clients that, I guess.
Comment: @ToddWilcox - no I just always use a pop filter for vox with condenser mics, and never for instruments. Not actually tried the acid test. When I've some spare time, maybe it's going to fill it !
Comment: A 57 doesn't have what I'd call a pop-shield at all, just a mesh. But have a look at what DPA/B&K do with their meshes - simply changing the mesh shape changes the EQ [they also do 'ball' structures that change the omni field, it's quite spectacular]. Not a clue how it's done, but it works a treat. I once worked with a guy who was part of the team that designed the 4006 family meshes & 'balls'. His explanation went right over my head ;-)
Comment: @Tetsujin Sorry, by "57" I meant "58". Which also is just a capsule and transformer and of course the ball grille/pop shield thing that screws on the end.
Comment: hokay - EQ doesn't have to be in the circuitry, though - hence my point about the B&K [though we go go off into esoterics far too easily on this topic ;-)
Comment: The mesh of a '58 has a thin layer of foam behind it.  Not the most effective pop filter, but a lot better than nothing.


Positive definite in the limit

Question: Consider the $n\times n$ matrix $F$ defined as
$$
F=A-\varepsilon B
$$
where $A$ is a constant matrix such that $a_{ij}=a>0$ for all $i,j$ and where $B$ is a symmetric matrix such that $b_{ij}\geq 0$ for all $i,j$.
I would like to find sufficient conditions on $B$ such that $F$ is positive definite when the scalar $\varepsilon>0$ is small enough.
Over at Mathoverflow Fedor Petrov shows that $F$ is positive semidefinite for $\varepsilon$ small enough if $B$ is positive definite on the subspace $H:\sum_i x_i=0$. It seems that a similar argument should works to show that $F$ is positive definite. I am probably missing something but can't figure out what.
Edit: Following Fedor's arguments, here is how I think the proof would go:
Since $B$ is negative definite on $H$, we have that $(Bx,x)\leqslant -c\|x\|^2$ for $x\in H$. We can write any vector $z$ as $z=x+y$ where $x\in H$ and $y \perp H$. Then
\begin{align}((A-\varepsilon B)z,z)&=n\|y\|^2-\varepsilon (Bx,x)-2\varepsilon (Bx,y)-\varepsilon (By,y)\\ &\geqslant (n-1/2)\|y\|^2+c\varepsilon \|x\|^2-2\cdot \varepsilon \cdot \|B\|\cdot\|x\|\cdot\|y\|\end{align}
for $\varepsilon>0$ small enough. This last expression is strictly convex in $\left(\|x\|,\|y\|\right)$ when $\varepsilon>0$ is small enough and reaches it minimum of 0 at $\left(\|x\|,\|y\|\right)=(0,0)$ or, equivalently, at $z=0$. Since $((A-\varepsilon B)z,z)>0$ for any $z\neq 0$, $((A-\varepsilon B)z,z)$ is positive definite.
Is there anything wrong with this proof?
Answer: The sufficient condition given in that MO answer is already one that makes $F$ positive definite for small $\varepsilon>0$.
In general, suppose $A$ and $B$ are two Hermitian matrices of the same sizes (whether they are entrywise positive/nonnegative is unimportant). If $A-\varepsilon B$ is positive semidefinite for every small $\varepsilon>0$, then by passing $\varepsilon$ to the limit, $A$ must be positive semidefinite. If $A$ is indeed positive semidefinite, denote its restriction on $(\ker A)^\perp$ as $A_1$. Then $A_1$ is positive definite and we may write $F=A-\varepsilon B$ in the form of
$$
F=\pmatrix{A_1-\varepsilon X&-\varepsilon Y^\ast\\ -\varepsilon Y&-\varepsilon Z}.
$$
Therefore, a necessary condition for $F$ to be positive semidefinite for every small $\varepsilon>0$ is that $A$ is positive semidefinite and $Z=(I-A^+A)B(I-A^+A)$ is negative semidefinite on $\ker A$.
However, if $A$ is positive semidefinite but $Z$ is negative definite on $\ker A$, we get a sufficient condition. This should be evident, because $F$ is congruent to $(-\varepsilon Z)\oplus S$, where
\begin{align}
S&=A_1-\varepsilon X-(-\varepsilon Y^\ast)(-\varepsilon Z)^{-1}(-\varepsilon Y)\\
&=A_1-\varepsilon (X-Y^\ast Z^{-1}Y)
\end{align}
is the Schur complement of $-\varepsilon Z$ in $F$. Since $A_1$ is positive definite and $Z$ is negative definite, $-\varepsilon Z,\ S,\ (-\varepsilon Z)\oplus S$ and in turn $F$ are positive definite when $\varepsilon>0$ is small.
In your case, $A$ is a positive multiple of the all-one matrix. So, $A$ is positive semidefinite, $\ker A=\{x\in\mathbb R^n:\sum_ix_i=0\}$ and $I-A^+A$, the orthogonal projection on $\ker A$, is the matrix $P$ in the MO answer you mentioned.
Comment: @user_lambda 1) This is just a change of basis. 2) No difference. I meant to write "**B** is negative definite on $\ker A$", but that was ambiguous, as $\ker A$ isn't an invariant subspace of $B$. So, I edited that $B$ to $Z$. 3) $Y$ doesn't play a role because $A_1$ is positive definite. When $\varepsilon$ is small, $A_1$, as a quadratic form, dominates $\varepsilon(X-Y^\ast Z^{-1}Y)$. In some sense this is similar to completing square, but that might take too long to explain here. 4) See the LDU decomposition in the linked Wikipedia entry for Schur complement.
Comment: Thanks a lot for your answer! 1) Why can we write $F$ in that matrix form? 2) Given how $Z$ is written, what is the difference between "$Z$ is negative definite" and "$Z$ is negative definite on ker $A$"? Because of the projection matrix $I-A^+A$ aren't we restricting any vector $x$ to ker $A$ anyway? 3) When discussing a necessary condition for $F$ to be positive semidefinite, why is $Y$ not playing a role? Is it because it shows up in $F/(A_1-\varepsilon X)$ with $\varepsilon^2$. 4) How do we know that $F$ is congruent to $(-\varepsilon Z) \bigoplus S$? Thanks!

Positive definite in the limit


Como organizar uma tabela MySQL de forma eficiente e escalável?

Question: Estou fazendo uma refatoração em um sistem que possui diversas com muitas tabelas que possuem muitas colunas, isto é, chega até 148 colunas em uma tabela, como refatorar tal tabela?
Essa é a tabela
CREATE TABLE `periodizacao12` (
  `periodizacao_id` int(50) NOT NULL,
  `user_id` int(50) NOT NULL,
  `weeks` int(50) NOT NULL DEFAULT '12',
  `startDate` date NOT NULL,
  `mesocicloNome1` varchar(100) NOT NULL,
  `mesocicloNome2` varchar(100) NOT NULL,
  `mesocicloNome3` varchar(100) NOT NULL,
  `mesocicloNome4` varchar(100) NOT NULL,
  `mesocicloNome5` varchar(100) NOT NULL,
  `mesocicloNome6` varchar(100) NOT NULL,
  `mesocicloNome7` varchar(100) NOT NULL,
  `mesocicloNome8` varchar(100) NOT NULL,
  `mesocicloNome9` varchar(100) NOT NULL,
  `mesocicloNome10` varchar(100) NOT NULL,
  `mesocicloNome11` varchar(100) NOT NULL,
  `mesocicloNome12` varchar(100) NOT NULL,
  `periodizacao_objetivo1` varchar(100) NOT NULL,
  `periodizacao_objetivo1_en` varchar(255) NOT NULL,
  `periodizacao_objetivo1_es` varchar(255) NOT NULL,
  `periodizacao_objetivo2` varchar(100) NOT NULL,
  `periodizacao_objetivo2_en` varchar(255) NOT NULL,
  `periodizacao_objetivo2_es` varchar(255) NOT NULL,
  `periodizacao_objetivo3` varchar(100) NOT NULL,
  `periodizacao_objetivo3_en` varchar(255) NOT NULL,
  `periodizacao_objetivo3_es` varchar(255) NOT NULL,
  `periodizacao_objetivo4` varchar(100) NOT NULL,
  `periodizacao_objetivo4_en` varchar(255) NOT NULL,
  `periodizacao_objetivo4_es` varchar(255) NOT NULL,
  `periodizacao_objetivo5` varchar(100) NOT NULL,
  `periodizacao_objetivo5_en` varchar(255) NOT NULL,
  `periodizacao_objetivo5_es` varchar(255) NOT NULL,
  `periodizacao_objetivo6` varchar(100) NOT NULL,
  `periodizacao_objetivo6_en` varchar(255) NOT NULL,
  `periodizacao_objetivo6_es` varchar(255) NOT NULL,
  `periodizacao_objetivo7` varchar(100) NOT NULL,
  `periodizacao_objetivo7_en` varchar(255) NOT NULL,
  `periodizacao_objetivo7_es` varchar(255) NOT NULL,
  `periodizacao_objetivo8` varchar(100) NOT NULL,
  `periodizacao_objetivo8_en` varchar(255) NOT NULL,
  `periodizacao_objetivo8_es` varchar(255) NOT NULL,
  `periodizacao_objetivo9` varchar(100) NOT NULL,
  `periodizacao_objetivo9_en` varchar(255) NOT NULL,
  `periodizacao_objetivo9_es` varchar(255) NOT NULL,
  `periodizacao_objetivo10` varchar(100) NOT NULL,
  `periodizacao_objetivo10_en` varchar(255) NOT NULL,
  `periodizacao_objetivo10_es` varchar(255) NOT NULL,
  `periodizacao_objetivo11` varchar(100) NOT NULL,
  `periodizacao_objetivo11_en` varchar(255) NOT NULL,
  `periodizacao_objetivo11_es` varchar(255) NOT NULL,
  `periodizacao_objetivo12` varchar(100) NOT NULL,
  `periodizacao_objetivo12_en` varchar(255) NOT NULL,
  `periodizacao_objetivo1s_es` varchar(255) NOT NULL,
  `periodizacao_treino1` varchar(100) NOT NULL,
  `periodizacao_treino2` varchar(100) NOT NULL,
  `periodizacao_treino3` varchar(100) NOT NULL,
  `periodizacao_treino4` varchar(100) NOT NULL,
  `periodizacao_treino5` varchar(100) NOT NULL,
  `periodizacao_treino6` varchar(100) NOT NULL,
  `periodizacao_treino7` varchar(100) NOT NULL,
  `periodizacao_treino8` varchar(100) NOT NULL,
  `periodizacao_treino9` varchar(100) NOT NULL,
  `periodizacao_treino10` varchar(100) NOT NULL,
  `periodizacao_treino11` varchar(100) NOT NULL,
  `periodizacao_treino12` varchar(100) NOT NULL,
  `listOfTreino1` varchar(100) NOT NULL,
  `listOfTreino2` varchar(100) NOT NULL,
  `listOfTreino3` varchar(100) NOT NULL,
  `listOfTreino4` varchar(100) NOT NULL,
  `listOfTreino5` varchar(100) NOT NULL,
  `listOfTreino6` varchar(100) NOT NULL,
  `listOfTreino7` varchar(100) NOT NULL,
  `listOfTreino8` varchar(100) NOT NULL,
  `listOfTreino9` varchar(100) NOT NULL,
  `listOfTreino10` varchar(100) NOT NULL,
  `listOfTreino11` varchar(100) NOT NULL,
  `listOfTreino12` varchar(100) NOT NULL,
  `listOfTreino13` varchar(100) NOT NULL,
  `listOfTreino14` varchar(100) NOT NULL,
  `listOfTreino15` varchar(100) NOT NULL,
  `listOfTreino16` varchar(100) NOT NULL,
  `listOfTreino17` varchar(100) NOT NULL,
  `listOfTreino18` varchar(100) NOT NULL,
  `listOfTreino19` varchar(100) NOT NULL,
  `listOfTreino20` varchar(100) NOT NULL,
  `listOfTreino21` varchar(100) NOT NULL,
  `listOfTreino22` varchar(100) NOT NULL,
  `listOfTreino23` varchar(100) NOT NULL,
  `listOfTreino24` varchar(100) NOT NULL,
  `listOfTreino25` varchar(100) NOT NULL,
  `listOfTreino26` varchar(100) NOT NULL,
  `listOfTreino27` varchar(100) NOT NULL,
  `listOfTreino28` varchar(100) NOT NULL,
  `listOfTreino29` varchar(100) NOT NULL,
  `listOfTreino30` varchar(100) NOT NULL,
  `listOfTreino31` varchar(100) NOT NULL,
  `listOfTreino32` varchar(100) NOT NULL,
  `listOfTreino33` varchar(100) NOT NULL,
  `listOfTreino34` varchar(100) NOT NULL,
  `listOfTreino35` varchar(100) NOT NULL,
  `listOfTreino36` varchar(100) NOT NULL,
  `listOfTreino37` varchar(100) NOT NULL,
  `listOfTreino38` varchar(100) NOT NULL,
  `listOfTreino39` varchar(100) NOT NULL,
  `listOfTreino40` varchar(100) NOT NULL,
  `listOfTreino41` varchar(100) NOT NULL,
  `listOfTreino42` varchar(100) NOT NULL,
  `listOfTreino43` varchar(100) NOT NULL,
  `listOfTreino44` varchar(100) NOT NULL,
  `listOfTreino45` varchar(100) NOT NULL,
  `listOfTreino46` varchar(100) NOT NULL,
  `listOfTreino47` varchar(100) NOT NULL,
  `listOfTreino48` varchar(100) NOT NULL,
  `listOfTreino49` varchar(100) NOT NULL,
  `listOfTreino50` varchar(100) NOT NULL,
  `listOfTreino51` varchar(100) NOT NULL,
  `listOfTreino52` varchar(100) NOT NULL,
  `listOfTreino53` varchar(100) NOT NULL,
  `listOfTreino54` varchar(100) NOT NULL,
  `listOfTreino55` varchar(100) NOT NULL,
  `listOfTreino56` varchar(100) NOT NULL,
  `listOfTreino57` varchar(100) NOT NULL,
  `listOfTreino58` varchar(100) NOT NULL,
  `listOfTreino59` varchar(100) NOT NULL,
  `listOfTreino60` varchar(100) NOT NULL,
  `listOfTreino61` varchar(100) NOT NULL,
  `listOfTreino62` varchar(100) NOT NULL,
  `listOfTreino63` varchar(100) NOT NULL,
  `listOfTreino64` varchar(100) NOT NULL,
  `listOfTreino65` varchar(100) NOT NULL,
  `listOfTreino66` varchar(100) NOT NULL,
  `listOfTreino67` varchar(100) NOT NULL,
  `listOfTreino68` varchar(100) NOT NULL,
  `listOfTreino69` varchar(100) NOT NULL,
  `listOfTreino70` varchar(100) NOT NULL,
  `listOfTreino71` varchar(100) NOT NULL,
  `listOfTreino72` varchar(100) NOT NULL,
  `listOfTreino73` varchar(100) NOT NULL,
  `listOfTreino74` varchar(100) NOT NULL,
  `listOfTreino75` varchar(100) NOT NULL,
  `listOfTreino76` varchar(100) NOT NULL,
  `listOfTreino77` varchar(100) NOT NULL,
  `listOfTreino78` varchar(100) NOT NULL,
  `listOfTreino79` varchar(100) NOT NULL,
  `listOfTreino80` varchar(100) NOT NULL,
  `listOfTreino81` varchar(100) NOT NULL,
  `listOfTreino82` varchar(100) NOT NULL,
  `listOfTreino83` varchar(100) NOT NULL,
  `listOfTreino84` varchar(100) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

A tabela é para uma lista de treinos de um usuário numa academia.
Comment: O que tem nos campos listOfTreino, periodizacao_treino etc...?
Comment: @Peter tem variações de treinos como A, B ,C ...
Comment: A criação ou modificação de uma base de dados, depende muito do que vc quer fazer. Se vc quer "ajuda", vc deve descrever o que tem nos campos, porque este tipo de dados, como os campos são usados etc...
Answer: Sem conheçer as intenções por detrás do código ou a forma como esta tabela se relaciona com as demais tabelas é dificil dar opiniões mas algo que me salta à vista são as multiplas colunas com o nome idêntico (a diferença é o Indice) - listOfTreino.
Estas colunas parecem-me fortes candidatas a serem colocadas numa tabela separada que se relaciona com a tabela do utilizador através do id do utilizador. Esta tabela em vez de terem 84 colunas tem apenas 3 (user_id, treino_id, value), ficando assim na versão "transposta" das colunas originais. O campo user_id é a chave para o utilizador, o treino_id é o numero que replica as 84 colunas originais e o campo valor registará o valor tal como inserido na tabela original.
CREATE TABLE `listOfTreinos` (
  `user_id` int(50) NOT NULL,
  `treino_id` int(50) NOT NULL,
  `value` varchar(100) NOT NULL

O mesmo raciocionio pode ser aplicado aos restantes campos, uma vez que parecem ser todos relacionados (partilham o mesmo nomes diferindo do indice de 1 a 12). Assim podem ser re-estruturados para uma tabela com os seguintes campos:
CREATE TABLE `listOfTreinos` (
 `user_id` int(50) NOT NULL,
 `mesocicloId` int(50) NOT NULL,
 `mesocicloNome` varchar(100) NOT NULL,
 `periodizacao_objetivo` varchar(100) NOT NULL,
 `periodizacao_objetivo_en` varchar(255) NOT NULL,
 `periodizacao_objetivo_es` varchar(255) NOT NULL,
 `periodizacao_treino` varchar(100) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Mas como disse antes, isto depende do que se pretende fazer e da forma como se quer fazer.


How does a resolver know a DNS response is authoritative?

Question: The title says everything about me question. How does a resolver know the response comes from an authoritative name server? An equivalent question: what flag or field is marked by a name server to say to the resolver that its response wasn't authoritative? And last related question, secondary DNS servers are always non-authoritative?
Comment: What do you mean by "secondary DNS servers"? If you mean servers with slave zones, those will answer authoritatively.
Answer: The DNS Response message header contains a bit indicating whether the resolved name came from an authoritative server, this would be the field called AA in the header reference table listed below:
<code>+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|                      ID                       |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|QR|   Opcode  |AA|TC|RD|RA|   Z    |   RCODE   |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|                    QDCOUNT                    |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|                    ANCOUNT                    |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|                    NSCOUNT                    |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|                    ARCOUNT                    |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
</code>
See RFC 1035 §4.1.1 for header format details


erro no envio de dados via ajax

Question: Não estou conseguindo enviar os dados para o mysql via ajax , não sei em que parte de meu código poderia está errado.
Meu index que na qual as categorias são listado do banco de dados via jquery.

Peixaria

                  <?php
                     categoriaas();
                  ?>

                <form method="post" action="relatorio.php">
                <input type="text" id="theinputsum">

                   <button type="submit" class="btn btn-lg btn-success btn-block"> Finalizar Pedido</button>
                </form>

o código ajax da qual estou querendo enviar para o mysql.

e o código que na qual envia e lista para o mysql
<?php
error_reporting(-1);
ini_set('display_errors', 'On');

//Criar a conexao
$link = new mysqli ("localhost", "root", "", "restaurant");
if($link->connect_errno){
     echo"Nossas falhas local experiência ..";
     exit();
}
$pedido = $_POST['products'];
$preco = $_POST['products'];

$sql = "INSERT INTO `pedido` (`pedido`,`preco`) VALUES ('{$pedido}','{$preco}')";

$link->query($sql);

         $sql= "SELECT id_pedido,numero_mesa,pedido_refeicao,num_refeicao,pedido_bebida,num_bebida,data FROM mpedido ORDER BY id_pedido DESC LIMIT 1";
        $consulta = mysqli_query($link,$sql);

?>

Comment: Qual está sendo o erro?
Comment: De novo? Você irá insistir em ficar recriando tópicos com a mesma pergunta?
Answer: Seu código tem alguns problemas:
Você esta usando o evento "submit" no componente errado.
Troque esta linha
<form method="post" action="relatorio.php">

Por esta
<form method="post" action="relatorio.php" id="formRel">

e esta
$('#theinputsum').submit(function(event){

por esta
$('#formRel').submit(function(event){

e esta
data: $("#theinputsum").serialize()

por esta
data: $("#formRel").serialize()

Você já começa a enviar o dados para o php. No arquivo php não há retorno nenhum se tudo focar ok, se você quiser retornar algo ao html , uma boa prática é escolher um tipo de retorno na sua requisição ajax assim:
$.ajax({
  method: "POST",
  url: "relatorio.php",
  data: $("#formRel").serialize(),
  dataType : "html"
})

No .done você retorna o conteúdo da página php em alguma div por exemplo:
.done(function( retorno) {
    alert( "muito bem" );
    $("#idDaDiv").html(retorno);
});

Por fim seu arquivo php deve imprimir (pode ser com echo mesmo), o resultado que vc quer que apareça na div.
Mais informações nos links http://api.jquery.com/jquery.ajax/ e http://api.jquery.com/html/
Comment: @RodrigoSartoriJabouche Obrigado . mas eu só quero mesmo é enviar para o mysql e depois faço em uma outra pagina para listar . Muito Obrigado
Comment: Ok, faça até a parte que envia então...
Comment: e a parte que inserir no mysql , poderia ficar igual como está ?
Comment: Pode, só não irá ter um retorno se deu certo ou errada a inserção.. para tanto se vc fizer tudo o que eu falei, você terá a resposta "Nossas falhas local experiência .."
Comment: olá passei a noite toda e não conseguir enviar nada para o mysql da maneira que você falou
Comment: Desenvolvimento as vezes é meio complicado, eu sugiro que você comece tentando coisas mais simples e vá avançando, dá uma olhada nesse tutorial http://www.w3schools.com/js/js_ajax_intro.asp.
Comment: Reparou nesta linha? $preco = $_POST['products']; Veja o tipo de dado do campo preço na tabela. tente dar um echo na sua query, você tem que debugar...
Comment: Pois é eu dei um echo e a o array parece vazio .
Comment: Olhe as propriedade name dos seus inputs, elas são iguais aos que vc busca no $_POST? Aconselho que você dê uma boa olha em conceitos básicos
Comment: bem , só um input que tem name"" o outro é o que vem via ajax do mysql, tipo a tela de categorias vem via javascript ai só tem essa linha   que faz isso . Como poderia fazer para colocar ela no post ?
Comment: Você criar os inputs...
Comment: Mas como poderia fazer outros inputs se os produtos são dinâmicos , feitos em jquery e via ajax do mysql , e quando clico no pedido ele é selecionado como mostra a figura .Posso lhe mandar o sistema todo por email .
Comment: Cara ai sua pergunta não tem como ser respondida ela tem mais de contexto, tente algo mais simples e vá avançando

erro no envio de dados via ajax


How to integrate Angular 9 with a camunda modeler?

Question: I searched for ways to get the camunda modeler into angular 9 application. What I found was using bpmn-js library and its other dependencies like bpmn-js-properties-panel, etc.
I was able to get the following output:

But, as in the online camunda modeler, I cannot see the connectors in my implementation. I know that there is a dependency called bpmn-js-connectors-extension but that is not working and I got below errors.
ERROR in ./node_modules/bpmn-js-connectors-extension/dist/index.es.js 775:11-16
export 'isAny' was not found in 'bpmn-js/lib/util/ModelUtil'
ERROR in ./node_modules/bpmn-js-connectors-extension/dist/index.es.js 977:11-16
export 'isAny' was not found in 'bpmn-js/lib/util/ModelUtil'
The dependencies that I'm using are below.

"bpmn-js": "6.3.4",
"bpmn-js-connectors-extension": "^0.4.6",
"bpmn-js-properties-panel": "0.37.5",
"camunda-bpmn-moddle": "^7.0.1",
"core-js": "3.6.4",
"diagram-js": "^11.13.0"

I need to know whether there is any way to integrate camunda modeler into Angular 9 and if using bpmn-js is okay, then what is the way to get the camunda connectors as below?
Answer: The question was answered in the bpmn-js forum https://forum.bpmn.io/t/bpmn-js-connector-extension/9276
The connectors extension is deprecated. You can use
GitHub - bpmn-io/bpmn-js-create-append-anything: A create/append anything bpmn-js extension instead.
Try and update your dependencies, too. bpmn-js v6 is 3 years old.


Insert value to df1 from df2 if df1 matches 2 columns in df2

Question: I have a dataframe df1 like this:
ID1 <- c("AAA" , "CCD" , "AAA", "ABC", "AAA", "ABC", "ZZZ")
year1 <- c(2009,2008,2012,2012,2000,2012,2005)
df1 <- data.frame(ID1,year1)
df1

  ID1 year1
1 AAA  2009
2 CCD  2008
3 AAA  2012
4 ABC  2012
5 AAA  2000
6 ABC  2012
7 ZZZ  2005

Then, I have a dataframe df2 like this:
ID2 <- c("CCC","AAA","AAA","CCD","AAA","ABC","AAA","ABC","ABC","CCD")
year2 <- c(2005,2010,2012,2008,2009,2012,2000,2012,2004,2008)
xp <- c(5,6,7,8,5,4,3,4,3,8)
df2 <- data.frame(ID2,year2,xp)
df2

   ID2 year2 xp
1  CCC  2005  5
2  AAA  2010  6
3  AAA  2012  7
4  CCD  2008  8
5  AAA  2009  5
6  ABC  2012  4
7  AAA  2000  3
8  ABC  2012  4
9  ABC  2004  3
10 CCD  2008  8

I want to add the value of the column xp to df1 if df1$ID1 == df2$ID2 & df1$year1 == df2$year2. I tried:
df1$xp <- df2$xp[match(df1$ID1, df2$ID2) & match(df1$year1, df2$year2)]

which won't work. The result should be:
  ID1 year1 xp
1 AAA  2009  5
2 CCD  2008  8
3 AAA  2012  7
4 ABC  2012  4
5 AAA  2000  3
6 ABC  2012  4
7 ZZZ  2005 NA

Answer: A possible solution, based on dplyr::left_join:
library(dplyr)

left_join(df1, df2, by = c("ID1" = "ID2", "year1" = "year2")) %>%
  distinct

#>   ID1 year1 xp
#> 1 AAA  2009  5
#> 2 CCD  2008  8
#> 3 AAA  2012  7
#> 4 ABC  2012  4
#> 5 AAA  2000  3
#> 6 ZZZ  2005 NA

Or using base R:
unique(merge(df1, df2, by.x = c("ID1", "year1"), by.y = c("ID2", "year2"),
    all.x = T))

#>    ID1 year1 xp
#> 1  AAA  2000  3
#> 2  AAA  2009  5
#> 3  AAA  2012  7
#> 4  ABC  2012  4
#> 8  CCD  2008  8
#> 10 ZZZ  2005 NA

Comment: You are missing a row in the expected output, which I think is because of the duplicates in `df2`. So, if you run `distinct` on `df2` in the join, then that should get to the OP's expected output. `left_join(df1, distinct(df2), by = c("ID1" = "ID2", "year1" = "year2"))`. Likewise, moving the `unique` in the base R should fix that as well (but didn't test that one).
Comment: Thanks, @AndrewGB, for having commented my answer. I had noticed that, but I am not sure the OP's expected output is really the wanted expected output, as `ABC  2012  4` repeats once (intentionally or by mistake?)
Comment: I added the duplicate in `df2` intentionally because such duplicates can appear in my data theoretically. Actually there were no duplicates of that kind in my data, so in this case  `left_join(df1, df2, by = c("ID1" = "ID2", "year1" = "year2")) %>% distinct` worked perfectly. Good to know @AndrewGB
Comment: @OK_4711 In your expected output above, you have a duplicate `ABC  2012  4`.  So, my comment above is working around that to keep that duplicate / return all rows from `df1`. However, if you don't want duplicate rows, then applying `distinct` or `unique` will remove those rows. I also updated my answer to remove the duplicate row, so that both are there, depending on your needs. But either way, glad that you have a working solution for your needs.
Answer: Here is another possible solution using data.table:
library(data.table)

setDT(df1)[setDT(df2), on = c("ID1" = "ID2", "year1" = "year2"), xp := i.xp]

Output
   ID1 year1 xp
1: AAA  2009  5
2: CCD  2008  8
3: AAA  2012  7
4: ABC  2012  4
5: AAA  2000  3
6: ABC  2012  4
7: ZZZ  2005 NA

Or if you only want to keep the unique rows, as mentioned above, then you can use unique:
df1 <-
  unique(setDT(df1)[setDT(df2), on = c("ID1" = "ID2", "year1" = "year2"), xp := i.xp])

Output
   ID1 year1 xp
1: AAA  2009  5
2: CCD  2008  8
3: AAA  2012  7
4: ABC  2012  4
5: AAA  2000  3
6: ZZZ  2005 NA


Precursor to Fubini's Theorem - Question on Proof

Question: I'm reading about Fubini's theorem. The book I'm reading considers the following theorem:

Suppose $(X, \mathcal{M}, \mu), (Y, \mathcal{N}, \nu)$ are $\sigma$-finite measure spaces. If $E \in \mathcal{M} \times \mathcal{N}$, then the functions $x \mapsto \nu(E_x)$ and $y \mapsto \mu(E^y)$ are measureable on $X$ and $Y$ respectively and $$\mu \times \nu (E) = \int \nu(E_x)\,d\mu(x) = \int \mu(E^y)\,d\nu(y)$$
  where $E_x = \{y \in Y:(x,y) \in E\}$ and $E^y = \{x \in X:(x,y) \in E\}$

They prove this by letting $\mathcal{C}$ be the set of all $E\in \mathcal{M} \times \mathcal{N}$ that satisfy the conclusions of the theorem and prove this is a monotone class. By a theorem proved earlier, this means $\mathcal{C}$ is a sigma algebra. They conclude the proof here.
My question: I'm confused why they are trying to prove $\mathcal{C}$ is a sigma algebra - how does the proposition follow from knowing this?
Answer:
The conclusion of the theorem holds for all $A \times B \in \cal{M} \times \cal{N}$.
The set of all $E \in \cal{M} \times \cal{N}$ for which the theorem holds is a $\sigma$-algebra $\cal{C} \subset  \cal{M} \times \cal{N}$.
Since $\cal{C}$ contains all sets of the form $A \times B$, it contains the $\sigma$-algebra generated by such sets.
That $\sigma$-algebra is exactly $\cal{M} \times \cal{N}$, hence $ \cal{M} \times \cal{N} \subset \cal{C}$.
Comment: I'm still a bit confused. How is this showing that 1. the function $x \mapsto \nu(E_x)$ is measurable, 2. the function $y \mapsto \mu(E^y)$ is measureable, and 3. The product measure $\mu \times \nu (E)$ evaluates to both of those integrals.
Comment: It doesn't prove step 1, but you should be able to with very little effort.
Comment: Could you elaborate a little? It's likely there's something obvious I'm not seeing


Cálculo para determinar se triângulo é retângulo não dá resultado esperado

Question: O programa que criei recebe três valores inteiros, calcula dentro do if esses valores e caso a condição for correta ele deveria mostrar na tela o SIM (o triangulo é retângulo), e caso a condição não seja satisfeita, deve mostrar NAO.
Digitando os valores 3, 4 e 5 a saída tem de ser SIM, digitando 3, 3 e 4, a saída deveria ser NAO, mas esta dando SIM em todos os casos.
Segue o código:
#include <stdio.h>
#include <math.h>

int main (void)

{
    int hip, cat1, cat2;

    scanf("%d", &cat1);

    scanf("%d", &cat2);

    scanf("%d", &hip);

    if (hip = pow(cat1,2) + pow(cat2,2))
    {
        printf("SIM");
    }

    else
    {
        printf("NAO");
    }

    return 0;
}

Como resolver?
Comment: Liga os warnings do teu compilador e corrige-os todos antes de aceitar o executavel.
Comment: use == no lugar de =, e também pra ser a formula do triangulo retangulo deve ter raiz.
Comment: Se suas entradas são inteiras, uma solução mais simples como as da resposta do bigown trará resultados corretos sempre. Apenas quando se admite entradas em ponto flutuante (o que extrapola a pergunta, mas estou citando por completeza) é que uma solução aproximada se faz necessária - caso em que a resposta do JJoao demonstra o modo mais usual de tratar do problema (estabelecer uma margem de erro para as comparações em PF). Aqui está [um exemplo](http://ideone.com/3NefCR) que ajuda a comprovar a correção dos métodos apresentados.
Answer: Há dois problemas:

Você usou = e não ==, para comparar é o segundo, você atribuiu uma novo valor para a hipotenusa.
Você não usou a raiz quadrada, portanto a fórmula está errada. Mas existe uma fórmula melhor.

Veja:
#include <stdio.h>
#include <math.h>

int main (void) {
    int hip, cat1, cat2;
    scanf("%d", &cat1);
    scanf("%d", &cat2);
    scanf("%d", &hip);
    if (hip * hip == cat1 * cat1 + cat2 * cat2) printf("SIM");
    else printf("NAO");
}

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura. Note que eu fiz um pouco diferente lá.
Comment: Sugestao: em vez de calcular a raiz quadrada (duas vezes), compara os quadrados: `if (hip*hip == cat1*cat1 + cat2*cat2)`
Comment: a solução do @pmg está correta. A pergunta é somente para inteiros. Exatamente como em http://ideone.com/E3Cxp0. Para valores quebrados utiliza-se a resposta abaixo.
Answer: Variante da versão anterior (@Maniero)
#include <stdio.h>
#include <math.h>
#define ERROMAX 0.0001

int main (void) {
    float hip, cat1, cat2;
    scanf("%f", &cat1);
    scanf("%f", &cat2);
    scanf("%f", &hip);
    if (fabsf(hip - sqrt(pow(cat1,2) + pow(cat2,2))) < ERROMAX ) {
        printf("SIM\n");
    } else {
        printf("NAO\n");
    }
    return 0;
}

de modo a dar
1 1 1 -> não

3 4 5 -> sim

Comment: @mgibsonbr, tens todissima a razão! vou corrigir imediatamente! (eu estava a pensar fabsf (para floats) mas o fabs (para doubles também dá).
Comment: `abs` é só para inteiros (e existe no `stdlib.h`, não no `math.h`), acho que você quis dizer `fabs`, certo?


How to Edit a contributed module

Question: I am using the Feedback Module, and I need to add an email field to the feedback form. How can I make the necessary edits to the Feedback Module in such a way that they stay when the module is updated?
Answer: You should never edit core or contributed modules if it can be helped (which it almost always can).  I haven't used Feedback, but the general Drupal way of extending forms is to use hook_form_alter from your own custom module.
Basically create a simple module.  All it needs is a mymodule.info file and a mymodule.php file inside of a folder in /sites/all/modules. Here is some more info on creating a module.
Then in your mymodule.php file, add a function that looks something like this:
<code>/**
 * Implements hook_form_alter().
 */
function mymodule_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'FEEDBACK_MODULE_FORMID') {
    $form['custom_email'] = array(
      '#type' => 'textfield',
      '#title' => 'Email',
      '#required' => TRUE,
    );

    $form['#submit'][] = '_mymodule_feedback_form_submit';
  }
}
</code>
This will add a new field to the form, and register a submit callback for the form.
Finally write the <code>_mymodule_feedback_form_submit</code> function to actually do something with the value when the form is submitted.
Answer: @Chances G is right but in cases that you need to make a new feature or make a change that you can't handle via Drupal's hook system you should create a patch and submit it to the community.
Comment: I think this is more of what I need. Since to make my email field work on the form I had to add code in in at least a dozen places.

So even though what I did might not be used by everyone using the module it is still worth it to submit it as a patch?
Comment: If it is something that you think is specific to you and won't work for everyone then it is better to create a custom hook in the contributed module and patch that. Now in your custom module you should be able to use your new hook.  If it something as simple as adding an email field I feel like @Chances G might be right in using hook_form_alter() to add an email field.
Comment: Ok, good to know.

How to Edit a contributed module


Open and closed complex sets

Question: was wondering if someone could shine some light on the highlighted half of this question? Any help would be greatly appreciated. Please excuse me for the poor format of the question, I'm new to this! Thanks in advance.

Determine whether the following set is closed:
  $$D = \{z\text{ is complex}: \Re(z)\ge\frac13 \text{ or } \Re(z)\le2\}$$
  Is the set $D$ open? Justify your answer.
Comment: Isn't that set all of $\mathbb{C}$?
Comment: http://math.stackexchange.com/help/notation should get you started on MathJax typesetting.
Comment: @Mike I hadn't seen that!
Comment: @Mike: It looks like all of$\mathbb C$ to me.
Comment: I would think the "or" should be "and" here.
Comment: @SuzuHirose I would wait for the OP's reactions. Maybe it is just a typo, maybe it was a trick question. We never know.
Comment: Or one of the real parts was meant to be imaginary?
Comment: I think maybe the point is that it is indeed open, but not "obviously" so.
Comment: @SuzuHirose: Either that orone of the $\Re$ should be a $\Im$. - Not that it would change the answer.
Comment: There is no point in thinking about what the question _might have been_. It may as well be intentional
Comment: @AlexR It's plausible that there's a pedagogical intention here. It's that it's possible to define a set in a way that doesn't make the fact that it's open immediately apparent, yet it can still be open.
Comment: I've already shown that the set is indeed closed by proving its compliment is open. However I was trying to work out how to show the set is or isn't open?
Answer: Hint
$D=\mathbb C$, so the question is:
Is $\mathbb C$ open?

Open and closed complex sets


Handle File->New in Word 2007

Question: I am writing a VSTO addin for Word 2007. When the user selects File->New, (or selects it from the quick access toolbar) I need to display a custom form instead of the standard new document dialog.  How do I do this?  I don't see an application event I can handle and I can't seem to find the buttont to add an event handler too.
Answer: Ok, just found it. You need to create a Ribbon xml and then add commands for those buttons. In this case the ribbon xml is
<commands>
    <command idMso="FileNew" onAction="FileNewOverride"/>
    <command idMso="FileNewDefault" onAction="FileNewOverride"/>
</commands>

and the code behind is
public void FileNewOverride(Office.IRibbonControl control, ref bool cancelDefault)
    {
        //do something
    }

This how-to on MSDN shows you how to do it  http://msdn.microsoft.com/en-us/office/dd361753.aspx
Comment: Do you have a link to a current copy of the page? Your link is now dead, and the archive.org page will redirect as well - https://web.archive.org/web/20120212042034/http://msdn.microsoft.com/en-us/office/dd361753.aspx
Comment: also adding if you want to Override File -> Save change "FileNew" to "FileSave"

Handle File->New in Word 2007


Replicating the results of a Poisson GLM wth log link function

Question: I'm working on an example from the book "An Introduction to Generalized Linear Models 3rd Edition" by Dobson. I'm trying the exercise on page 69 which starts:
"The data in Table 4.5 show the numbers of cases of AIDS in Australia
by date of diagnosis for successive 3-months periods from 1984 to 1988." Which looks like this:
year1984 <- c(1,6,16,23)
year1985 <- c(27,39,31,30)
year1986 <- c(43,51,63,70)
year1987 <- c(88,97,91,104)
year1988 <- c(110,113,149,159)

rbind(year1984,year1985,year1986,year1987,year1988)

The task is to fit a generalized linear model to these data using the Poisson distribution, the log-link function and the equation: $log \lambda_i=\beta_1+\beta_2x_i$ The solution given for the Poisson regression model is:
 $ log\hat{\lambda_i}= 0.996 + 1.327 log_i$
But my attempt doesn't give me this:
x<-c(1:20)
y<-c(1,6,16,23,
     27,39,31,30,
     43,51,63,70,
     88,97,91,104,
     110,113,149,159)

m1 <- glm (y~x,family=poisson)
m1

Where am I going wrong?
Comment: You are fitting `y~x`, but you should be fitting `y~log(x)` in order to match the model given for the solution; the log link transforms the left hand side of the equation, not the right hand side.  Try it and you'll see that you get the stated results.
Comment: Thanks for your response. Though the log given in the first equation $(log\lambda_i)$ for the task is just for the link function correct?
Comment: Right, the link function handles that part of it for you.
Answer: I was just looking for a solution to a similar problem and found out that R has a handy way of specifying this within the glm function (works also for glmer) that worked really well for me:
family=poisson(link="log")

maybe it helps someone else as well.
Comment: Note that the log link is the default for the Poisson GLM in R, because the log link is the _canonical_ link function for a Poisson GLM.


Should i learn double bass with an online teacher?

Question: I want to learn double bass but  i live in a country that there is no bass teachers at all, so i will have to learn online, is that feasible and effective? because if i want to learn double bass that is my only option.
Comment: If it's your only option, then it's your only option.
Comment: You should be able to determine whether it's working for you in two to four lessons or so. If it's not, you stop taking lessons from that person. Even better if you can rent a bass at the beginning. Try it and see if it works for you - we won't be able to tell you objectively whether it will or not. You'll have to give it a shot and determine your own answer.
Comment: Related to the votes to close and your edit - I'm on the fence about voting to close. Learning clarinet and classical guitar over zoom has been working great **for me**. So it **can** be effective, but that doesn't mean it always will be. And I managed to find excellent teachers. So I think every situation is potentially different. Will it work for you? Subjective. Can it work at all? Objectively yes.
Comment: By the way, there are several votes to close this question as opinion-based, but if you don't mind, I've replaced the word "advisable" with "feasible and effective," since I think that's what matters and can be answered more objectively.
Comment: @ToddWilcox Right, thus my edit to make it basically "can it work," which I think doesn't violate the heart of the OP's intent.
Answer: Yes, it is absolutely possible to learn effectively via telepresence (Zoom, FaceTime, Skype, etc.). Yes, it's not as efficient or easy as learning in person, but it's much more effective than trying to learn without a teacher.
Often, we get the question "Can I teach myself ___ without a teacher?" and the answer is often "Hm, I don't know, maybe kinda, I guess, but honestly maybe not, and it would definitely be really slow and you might learn some bad technique that will limit you." In comparison, the question "can I learn via telepresence" is "Yeah, it might be slower than in person, but if you're committed and the teacher is competent, you absolutely will make reasonable progress."
I'm skeptical that there really are no bass teachers at your disposal at all, though. Maybe none within a distance that lets you have regular weekly lessons, but if there's a major city within a day's travel, there's a strong chance that it has a few music teachers. If it's possible, it would be great to augment telepresence learning with an in-person lesson or two, even if it's just one or two per year.
Also, do you already have an instrument? An acoustic bass is a big object and a big purchase, and I have to imagine that there are places where it's almost as hard to obtain as a teacher.
Comment: I should add, throughout 2020 my daughter took Zoom lessons with a piano teacher in another country. They worked quite well for her—in fact, maybe better just because the teacher was such a good one and perfect fit for her. (Though in person *with that teacher* would still have been even better.)


Retornar apenas campos preenchidos no web service

Question: Fiz um REST com WCF. Em um método ele me retorna um objeto. Acontece que eu não preencho todos  os campos desse objeto e ele mostra todos no xml de resposta. Como eu faço para mostrar apenas os campos que têm valor, ou seja, os que eu preencho?
Comment: Acredito que seja a forma como ele serializa. Qual a motivação para isso? Apenas diminuir o tamanho da requisição/resposta? Caso seja isso aconselho usar JSON.
Comment: Json é o que o parceiro não quer. A programadora lá teve problemas com ele e pediu que fosse xml. A questão toda está no aprendizado e também no tamanho do xml, só isso. Como está dá pra trabalhar.
Comment: A soliução encontrada seria construir uma classe que tenha somente os atributos que serão retornados. Aí eu retorno esse objeto e pronto. A única coisa, mas não impacta, é que se houver atributos nulos, ele retornará como nil ou 0, dependendo do tipo.
Comment: Mas isso não retomaria o problema de "mostrar apenas os campos que têm valor, ou seja, os que eu preencho"?
Comment: Entendi, acontece que se o objeto tem muitos atributos e eu quero apenas alguns, essa parte eu elimino, pois não consegui resolver como eu gostaria, ou seja, mostrar tudo que é not null ou maior que zero(para Integer e Float)
Answer: Para evitar que os campos do objeto sejam serializados no XML (ou JSON), você pode usar a propriedade EmitDefaultValue do atributo [DataMember]. Se ela tem o valor false, números com valor 0, bool com valor false ou objetos com valor null não serão incluídos na resposta.
public class PT_StackOverflow_17297
{
    [DataContract(Name = "MinhaClasse", Namespace = "")]
    public class MinhaClasse
    {
        [DataMember(EmitDefaultValue = false)]
        public string Nome { get; set; }

        [DataMember(EmitDefaultValue = false)]
        public int Valor { get; set; }

        [DataMember(EmitDefaultValue = false)]
        public bool Flag { get; set; }
    }
    [ServiceContract]
    public class Service
    {
        [WebGet(ResponseFormat = WebMessageFormat.Xml)]
        public MinhaClasse Get()
        {
            var rndGen = new Random();
            var result = new MinhaClasse();
            result.Nome = rndGen.Next(2) == 0 ? "Alguma coisa" : null;
            result.Valor = rndGen.Next(2) == 0 ? 123 : 0;
            result.Flag = rndGen.Next(2) == 0;
            return result;
        }
    }
    public static void Test()
    {
        var baseAddress = "http://" + Environment.MachineName + ":8000/Service";
        var host = new WebServiceHost(typeof(Service), new Uri(baseAddress));
        host.Open();
        Console.WriteLine("Host opened");

        var c = new WebClient();
        for (var i = 0; i < 5; i++)
        {
            Console.WriteLine(c.DownloadString(baseAddress + "/Get"));
            Thread.Sleep(50);
        }

        Console.ReadLine();
        host.Close();
    }
}


How do I add the "cancel" button to a managed file

Question: I made a new component for webform that adds a file upload component. It has a "choose file" button, and an "upload" button. You can also choose to use a progress bar during the upload.
Now I want to include a "cancel" button to interrupt the upload process. How can I do this?
Answer: if you redirect your browser I'm 99.9% sure it will cancel the upload.
how you handle that redirection is up to you.  I'd probably just show a hidden div when the upload starts with a Cancel upload
hopefully what I wrote above helps, but I'd like to mention the other alternative:
don't put a cancel button in.  I would say that this is a useless "feature" and really why add things to your UI when you don't have to?
the user has to step through quite a few things, it'd be pretty odd if they selected the wrong file.


How to make a uibutton with touch down action and touch up inside action

Question: I need to make one button with two actions, touch down used to play a sound and then touch up inside to stop playing the sound when you let go of the button
Comment: A `UIButton` is a SubClass of `UIControl`. The first step you should do is to check out Apples Documentation about the Classes you use. You can simply alt+click on the classname to jump directly in the documentation.

Here the documentation about addTarget (http://developer.apple.com/library/ios/documentation/UIKit/Reference/UIControl_Class/Reference/Reference.html#//apple_ref/occ/instm/UIControl/addTarget:action:forControlEvents:)
Answer: Connect two methods to both these actions.
<code>[self.yourBtn addTarget:self action:@selector(touchDownMethod:) forControlEvents:UIControlEventTouchDown];
[self.yourBtn addTarget:self action:@selector(touchUpInsideMethod:) forControlEvents:UIControlEventTouchUpInside];
</code>


Imagem de fundo faz a página piscar ao ser carregada

Question: Estou criando um site, ao carregar qualquer uma das 6 páginas de uma categoria do site que possuem cada uma delas uma imagem de fundo com média de 40 a 70kb, a área onde se encontra a imagem pisca. A página pisca sempre ao clicar na página ou atualizá-la, o que posso fazer?
Comment: "o que posso fazer": não usar caps lock nem bold pra tentar chamar a atenção... Mas focando no problema, você tentou alguma coisa? Com CSS ou JS? Tipo `display:none` e fade-in quando carrega? . . . Ah, sim, **[edit]e** a pergunta pra adicionar detalhes. E bem vindo ao [pt.so] ;)
Comment: Você consegue compartilhar um exemplo para entendermos melhor o problema?
Comment: Não vi nada de errado... :) [Chrome no Ubuntu 14]
Comment: Rui repare que a página pisca na área onde esta a imagem de funda da categoria, ela pisca ao atualizar a página.
Comment: Acho que vou usar o sistema de compressão do base 64 vai ser o jeito !
Answer: A imagem em si não está a piscar, o que acontece é que devido ao elevado numero de requests que a tua página obriga a serem feitos, existe uma pequena demora entre o tempo em que a página é servida e os conteúdos chegam, mais propriamente a imagem.
A demora é pequena o suficiente para causar a impressão de estar a piscar, mas não é o caso.
Se tiveres uma ligação lenta vais poder observar que efectivamente a página é servida com uma zona a branco onde deveria estar a imagem e pouco depois aparece a mesma.
Solução
Reduz o número de pedidos que estás a fazer quer a nível de scripts, quer a nível de estilos quer a nível de imagens:

CSS
Todos os ficheiros de CSS podem ser descarregados num único pedido por parte do navegador:
Exemplo em PHP
<?php
header('Content-type: text/css; charset=UTF-8');

ob_start("compress");

function compress($buffer) {

  /* remove comentários */
  $buffer = preg_replace('!/\*[^*]*\*+([^/][^*]*\*+)*/!', '', $buffer);

  /* remove tabulaçõess, espaços, quebras de linhas */
  $buffer = str_replace(array("\r\n", "\r", "\n", "\t", '  ', '    ', '    '), '', $buffer);

  return $buffer;
}

$cssFiles = realpath(dirname(__FILE__)).'/*.css';

foreach(glob($cssFiles) as $file) {
  include($file);
}

ob_end_flush();
?>

Com o exemplo em cima, podes ter um ficheiro PHP dentro da tua pasta onde tens os diversos CSS e ao invés de chamares todos os .CSS no site, chamas apenas o ficheiro PHP:
<link type="text/css" media="screen" rel="stylesheet" href="http://meusite.com/css/css.php">

Para o teu caso em particular, passas 4 pedidos para 1 só.

JS
O exemplo dado para o CSS pode ser também aplicado aos ficheiros de script:
Exemplo em PHP
<?php
header('Content-Type: text/javascript; charset=UTF-8');

ob_start();

$files = realpath(dirname(__FILE__)).'/*.js';

foreach(glob($files) as $file) {
  include($file);
}

ob_end_flush();
?>

Com o exemplo em cima, podes ter um ficheiro PHP dentro da tua pasta onde tens os diversos scripts e ao invés de chamares todos os .JS no site, chamas apenas o ficheiro PHP:


Django ModelForm with foreign key

Question: I'm trying to create a ModelForm that updates a table with foreign keys.  What I have seems to work, but I was hoping someone could tell me if there's a better way to do this or if there's a problem with the way I'm doing it below.
Is it correct to use the queryset on the Author and Genres table?  It feels like I should be using a queryset on the Book model, and relate the foreign key to those tables.
models.py:
class Author(models.Model):
    name = models.CharField(max_length=200)

class Genre(models.Model):
    name = models.CharField(max_length=200)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey('Author')
    genre = models.ForeignKey('Genre')

forms.py:
class BookForm(ModelForm):
    title = forms.CharField(max_length=200)
    author = forms.ModelChoiceField(queryset=Author.objects.all())
    genre = forms.ModelChoiceField(queryset=Genre.objects.all())

    class Meta:
        model = Book
        fields = ['title', 'author', 'genre']

views.py:
def add_book(request):
    if request.method == 'POST':
        form = BookForm(request.POST)
        if form.is_valid():
            form.save(commit=True)
            return HttpResponseRedirect('/add/')
    else:
        form = BookForm()

Answer: The only thing that's wrong with this code is you are overriding the default behaviour of your model form.
Change it to look like:
class BookForm(ModelForm):

    class Meta:
        model = Book
        fields = ['title', 'author', 'genre']

And let django handle with the definition of those.
if you need to add labels or widgets, you can define them in the Meta class:
class BookForm(ModelForm):

     class Meta:
         model = Book
         fields = ['title', 'author', 'genre']
         labels = {'title': 'Book title', }

For example.
Comment: Thank you! (Filler text)
Answer: I'm not really sure what you're asking here, or why you would want a queryset on Book.
You haven't actually changed any of the defaults on any of the fields, so there is no need to redefine them: your form could be simply
class BookForm(ModelForm):
    class Meta:
        model = Book

and it would work exactly the same.

Django ModelForm with foreign key


Add external data to Orika MappingContext while mapping

Question: I want to add some external properties (something not in the object I'm mapping) to the MappingContext.
Here what I want to accomplish:
<code>    MapperFactory mapperFactory = new DefaultMapperFactory.Builder().build();
    mapperFactory.classMap(ObjA.class, ObjB.class).customize(new CustomMapper<ObjA, ObjB>() {
        @Override
        public void mapAtoB(ObjA objA, ObjB objB, MappingContext context) {
            objB.setName((String) context.getProperty("name"));
        }
    }).byDefault().register();
    MapperFacade mapper = mapperFactory.getMapperFacade();

    ObjA objA = new ObjA();
    ObjB objB = new ObjB();

    MappingContext context = new MappingContext(); //PROBLEME IS HERE: MappingContext is a abstract class
    context.setProperty("name", "Some information not in objA nor in objB");
    mapper.map(objA, objB, context);
</code>
How can I do that? Can I dynamically add data to the context? Or is it an other way to pass additional data to mapper?
Comment: [`MappingContext.Factory.getContext()`](http://javadox.com/ma.glasnost.orika/orika-core/1.4.0/ma/glasnost/orika/MappingContext.Factory.html#getContext())? Seems you should call `release()` once you're done with the context.
Comment: What's wrong with the solution you just posted? Looks like the right way to go.
Comment: @AdamMichalik MappingContext is abstract and cannot be instantiated. I didn't find a way to extract the actual mapper context nor to build a new one with a MappingContextFactory
Answer: The solution is simple:
<code>MappingContext context = new MappingContext.Factory().getContext();
context.setProperty("name", "value");
mapper.map(objA, objB, context);
</code>


AttributeError: 'str' object has no attribute 'num_points'

Question: Estou tentando criar um coletor de dados de um experimento que eu estou fazendo. Até o momento eu tenho o seguinte código:
em um arquivo experiments.py tem a classe:
import csv

class Experimento:

       def __init__(self, material: str, num_points: int, time: float=20.0, expType: str='varredura'):

            if expType not in ('varredura','ponto fixo'):
                raise ValueError('Tipo de experimento inválido. Selecione entre Varredura ou Ponto Fixo. \n NOTA: não utilize letras maiúsculas!')

            self.num_points = num_points
            self.time = time
            self.material = material
            self.expType = expType

E a subclasse:
class Ponto_Fixo(Experimento):

    def __init__(self, material: str, num_points: int, x: int, z: int, time: float=20.0):
        Experimento.__init__(material, num_points, time, 'ponto fixo')
        self.x = x
        self.z = z
        self.expID = material + ' x' + str(self.x) + ' z' + str(self.z) + ' pt=' + str(num_points)

        header = ['data','material','ponto','x','z','contagem','tempo(s)']

        csv_title = '\%s.csv' %(self.expID)
        with open('resultados'+ csv_title,'w') as file:
            writer = csv.writer(file)
            writer.writerow(header)

No main.py, eu executo esse código assim:
import experiments

if __name__ == "__main__":
    material = 'agua+CaCO3'
    expType = 'ponto fixo'
    tempo = 20.0
    num_points = 6
    x = 2
    z = 2

def new_experiment(material, num_points, tempo, x, z, expType):

    if expType == 'ponto fixo':

        experiments.Ponto_Fixo(material, num_points, x, z, tempo)

novo_exp = new_experiment(material, num_points, tempo, x, z, expType)

Porém, ao fazer essa execução eu obtenho o erro do título
  File "C:\Adam\Doutorado\Laboratorio\TARG\programs\data collection\main.py", line 46, in <module>
    novo_exp = new_experiment(material, num_points, tempo, x, z, expType)

  File "C:\Adam\Doutorado\Laboratorio\TARG\programs\data collection\main.py", line 37, in new_experiment
    experiments.Ponto_Fixo(material, num_points, x, z, tempo)

  File "C:\Adam\Doutorado\Laboratorio\TARG\programs\data collection\experiments.py", line 113, in __init__
    Experimento.__init__(material, num_points, time, 'ponto fixo')

  File "C:\Adam\Doutorado\Laboratorio\TARG\programs\data collection\experiments.py", line 37, in __init__
    self.num_points = num_points

AttributeError: 'str' object has no attribute 'num_points'

A linha que o programa acusa o erro é essa (que está dentro da classe Experimento):
self.num_points = num_points

Além disso, a linha que troca a tipagem da variável num_point de int para str está depois dessa linha do erro. Logo, não estou conseguindo enxergar onde e porque está ocorrendo a troca do tipo de int para str no num_point.
Alguém consegue dar uma luz?
Obrigado
Answer: Creio que o objetivo seja acessar a classe "pai", então troque:
Experimento.__init__(material, num_points, time, 'ponto fixo')

Por super(), assim:
super().__init__(material, num_points, time, 'ponto fixo')

Se for Python 2 vai ter que usar assim:
super(Ponto_Fixo, self).__init__(material, num_points, time, 'ponto fixo')

Documentação oficial: https://docs.python.org/3/library/functions.html#super, é por lá que é o ideal de começar a aprender, a sintaxe, recursos da linguagem e APIs nativas.
Comment: ao fazer o que você sugeriu, a princípio deu o seguinte erro ```ValueError: __init__() requires a code object with 0 free vars, not 1```

No entanto, insistindo um pouco na execução do código, o erro parou de acontecer e deu tudo certo. Continuo sem entender direito o porque disso, mas a sua ajuda resolveu o problema. Muito obrigado.
Comment: @AdamBasílio aqui seu código executou perfeitamente: https://ideone.com/Vg3MCh -- só removi a parte do CSV pq não tem como ler ou gravar arquivos nesse teste online.
Comment: @AdamBasílio deve ser pq vocÊ esta usando jupyter, ou qualquer coisa com autoreloader, as vezes ocorre.


How to get started recording music with Reaper?

Question: I would like to record my music and am completely new to recording (besides using the Windows OS inbuilt voice recorder) I downloaded Reaper to use as my DAW software and its completely overwhelming.
I would like to record my piano/guitar and my voice as individual tracks and put them together, and I recently purchased an MPK Mini MIDI keyboard for creating drum tracks (although now I'm not sure if that is possible with Reaper)
Are there any easy to follow beginner walkthroughs explaning what I would like to do? Alternatively, are there any inexpensive DAW softwares for Windows OS that would be suitable for a newbie like me? I'm willing to learn but would like to see results early on for what I'm trying to do (it seems simple enough) without getting bogged down by all the advanced options!
Comment: Just a word of 'reassurance': It is, in fact, very easy to work with. It just looks overwhelming a first because there's a lot of tools and buttons for things that you'll rarely end up using. Please do not feel demotivated -- once you work through one or two tutorials, you'll find out that it's quite an intuitive DAW to use. :)
Comment: Jon at http://reaperblog.net/about-contact/ is a great resource. I highly recommend.
Answer: Reaper is an awesome tool and can do A LOT, so it's a bit overwhelming but in its basic use, it's quite simple, once you know what you're doing. You can find a lot of tutorials and resources on the Internet about this software (because it is inexpensive, a lot of non-professional use it).
I like a lot the website Tutorials For Reaper, you can start with their first one, "Basic overview" and check the other ones.
I also follow The Reaper Blog, where you might find useful information (and their Youtube channel), and you'll find a few interesting links in the "more resources" menu. There is also a Reaper 101 category that might give you some additional tips on how to use the software.
If you understand french, Reno Mellow does some really great video tutorials.
Answer: I don't have time for a full-blown answer tonight, but as a user of Reaper myself, I can say that it is certainly capable of doing what you want, though I agree that it can be a tad overwhelming at first. Even after years of use, I certainly don't understand all of its features -- just the ones that I've come to use regularly.
I encourage you to stick with it, and check out the Reaper documentation wiki. It contains some excellent tutorials, including several that are oriented towards beginners (see the section headed Start Here If This is All New To You). There are also plenty of video tutorials to be found on YouTube. As a last resort, the Reaper forums might contain answers to more specific questions. And it never hurts to just get into the software and play around a bit.
Reaper is an excellent and powerful tool, but like any powerful tool, you do need to look at the documentation first.