Progressive Path Tracing with Explicit Light Sampling

Question: I understood the logic behind the importance sampling for the BRDF part. However, when it comes to sampling light sources explicitly, all becomes confusing. For example, if I have one point light source in my scene and if I directly sample it at each frame constantly, should I count it as one more sample for the monte carlo integration? That is, I take one sample from cosine-weighted distribution and other from the point light. Is it two samples in total or just one? Also, should I divide the radiance coming from the direct sample to any term?
Answer: There are multiple areas in path tracing that can be importance sampled. In addition, each of those areas can also use Multiple Importance Sampling, first proposed in Veach and Guibas's 1995 paper. To better explain, let's look at a backwards path tracer:
void RenderPixel(uint x, uint y, UniformSampler *sampler) {
    Ray ray = m_scene->Camera->CalculateRayFromPixel(x, y, sampler);

    float3 color(0.0f);
    float3 throughput(1.0f);
    SurfaceInteraction interaction;

    // Bounce the ray around the scene
    const uint maxBounces = 15;
    for (uint bounces = 0; bounces < maxBounces; ++bounces) {
        m_scene->Intersect(ray);

        // The ray missed. Return the background color
        if (ray.GeomID == INVALID_GEOMETRY_ID) {
            color += throughput * m_scene->BackgroundColor;
            break;
        }

        // Fetch the material
        Material *material = m_scene->GetMaterial(ray.GeomID);
        // The object might be emissive. If so, it will have a corresponding light
        // Otherwise, GetLight will return nullptr
        Light *light = m_scene->GetLight(ray.GeomID);

        // If we hit a light, add the emission
        if (light != nullptr) {
            color += throughput * light->Le();
        }

        interaction.Position = ray.Origin + ray.Direction * ray.TFar;
        interaction.Normal = normalize(m_scene->InterpolateNormal(ray.GeomID, ray.PrimID, ray.U, ray.V));
        interaction.OutputDirection = normalize(-ray.Direction);

        // Get the new ray direction
        // Choose the direction based on the bsdf
        material->bsdf->Sample(interaction, sampler);
        float pdf = material->bsdf->Pdf(interaction);

        // Accumulate the weight
        throughput = throughput * material->bsdf->Eval(interaction) / pdf;

        // Shoot a new ray

        // Set the origin at the intersection point
        ray.Origin = interaction.Position;

        // Reset the other ray properties
        ray.Direction = interaction.InputDirection;
        ray.TNear = 0.001f;
        ray.TFar = infinity;

        // Russian Roulette
        if (bounces > 3) {
            float p = std::max(throughput.x, std::max(throughput.y, throughput.z));
            if (sampler->NextFloat() > p) {
                break;
            }

            throughput *= 1 / p;
        }
    }

    m_scene->Camera->FrameBufferData.SplatPixel(x, y, color);
}

In English:

Shoot a ray through the scene
Check if we hit anything. If not we return the skybox color and break.
Check if we hit a light. If so, we add the light emission to our color accumulation
Choose a new direction for the next ray. We can do this uniformly, or importance sample based on the BRDF
Evaluate the BRDF and accumulate it. Here we have to divide by the pdf of our chosen direction, in order to follow the Monte Carlo Algorithm.
Create a new ray based on our chosen direction and where we just came from
[Optional] Use Russian Roulette to choose if we should terminate the ray
Goto 1

With this code, we only get color if the ray eventually hits a light. In addition, it doesn't support punctual light sources, since they have no area.
To fix this, we sample the lights directly at every bounce. We have to do a few small changes:
void RenderPixel(uint x, uint y, UniformSampler *sampler) {
    Ray ray = m_scene->Camera->CalculateRayFromPixel(x, y, sampler);

    float3 color(0.0f);
    float3 throughput(1.0f);
    SurfaceInteraction interaction;

    // Bounce the ray around the scene
    const uint maxBounces = 15;
    for (uint bounces = 0; bounces < maxBounces; ++bounces) {
        m_scene->Intersect(ray);

        // The ray missed. Return the background color
        if (ray.GeomID == INVALID_GEOMETRY_ID) {
            color += throughput * m_scene->BackgroundColor;
            break;
        }

        // Fetch the material
        Material *material = m_scene->GetMaterial(ray.GeomID);
        // The object might be emissive. If so, it will have a corresponding light
        // Otherwise, GetLight will return nullptr
        Light *light = m_scene->GetLight(ray.GeomID);

        // If this is the first bounce or if we just had a specular bounce,
        // we need to add the emmisive light
        if ((bounces == 0 || (interaction.SampledLobe & BSDFLobe::Specular) != 0) && light != nullptr) {
            color += throughput * light->Le();
        }

        interaction.Position = ray.Origin + ray.Direction * ray.TFar;
        interaction.Normal = normalize(m_scene->InterpolateNormal(ray.GeomID, ray.PrimID, ray.U, ray.V));
        interaction.OutputDirection = normalize(-ray.Direction);

        // Calculate the direct lighting
        color += throughput * SampleLights(sampler, interaction, material->bsdf, light);

        // Get the new ray direction
        // Choose the direction based on the bsdf
        material->bsdf->Sample(interaction, sampler);
        float pdf = material->bsdf->Pdf(interaction);

        // Accumulate the weight
        throughput = throughput * material->bsdf->Eval(interaction) / pdf;

        // Shoot a new ray

        // Set the origin at the intersection point
        ray.Origin = interaction.Position;

        // Reset the other ray properties
        ray.Direction = interaction.InputDirection;
        ray.TNear = 0.001f;
        ray.TFar = infinity;

        // Russian Roulette
        if (bounces > 3) {
            float p = std::max(throughput.x, std::max(throughput.y, throughput.z));
            if (sampler->NextFloat() > p) {
                break;
            }

            throughput *= 1 / p;
        }
    }

    m_scene->Camera->FrameBufferData.SplatPixel(x, y, color);
}

First, we add "color += throughput * SampleLights(...)". I'll go into detail about SampleLights() in a bit. But, essentially, it loops through all the lights, and returns their contribution to the color, attenuated by the BSDF.
This is great, but we need to make one more change in order to make it correct; specifically, what happens when we hit a light. In the old code, we added the light's emission to the color accumulation. But now we directly sample the light every bounce, so if we added the light's emission, we would "double dip". Therefore, the correct thing to do is... nothing; we skip accumulating the light's emission.
However, there are two corner cases:

The first ray
Perfectly specular bounces (aka mirrors)

If the first ray hits the light, you should see the light's emission directly. So if we skip it, all the lights will show up as black, even though the surfaces around them are lit.
When you hit a perfectly specular surfaces you can't directly sample a light, because an input ray has only one output. Well, technically, we could check if the input ray is going to hit a light, but there's no point; the main Path Tracing loop is going to do that anyway. Therefore, if we hit a light just after we hit a specular surface, we need to accumulate the color. If we don't, lights will be black in mirrors.
Now, let's delve into SampleLights():
float3 SampleLights(UniformSampler *sampler, SurfaceInteraction interaction, BSDF *bsdf, Light *hitLight) const {
    std::size_t numLights = m_scene->NumLights();

    float3 L(0.0f);
    for (uint i = 0; i < numLights; ++i) {
        Light *light = &m_scene->Lights[i];

        // Don't let a light contribute light to itself
        if (light == hitLight) {
            continue;
        }

        L = L + EstimateDirect(light, sampler, interaction, bsdf);
    }

    return L;
}

In English:

Loop through all the lights
Skip the light if we hit it

Don't double dip

Accumulate the direct lighting from all the lights
Return the direct lighting

Finally, EstimateDirect() is just evaluating $BSDF(p, \omega_{\text{i}}, \omega_{\text{o}}) L_{\text{i}}(p, \omega_{\text{i}})$
For punctual light sources, this is simple as:
float3 EstimateDirect(Light *light, UniformSampler *sampler, SurfaceInteraction &interaction, BSDF *bsdf) const {
    // Only sample if the BRDF is non-specular
    if ((bsdf->SupportedLobes & ~BSDFLobe::Specular) != 0) {
        return float3(0.0f);
    }

    interaction.InputDirection = normalize(light->Origin - interaction.Position);
    return bsdf->Eval(interaction) * light->Li;
}

However, if we want lights to have area, we first need to sample a point on the light. Therefore, the full definition is:
float3 EstimateDirect(Light *light, UniformSampler *sampler, SurfaceInteraction &interaction, BSDF *bsdf) const {
    float3 directLighting = float3(0.0f);

    // Only sample if the BRDF is non-specular
    if ((bsdf->SupportedLobes & ~BSDFLobe::Specular) != 0) {
        float pdf;
        float3 Li = light->SampleLi(sampler, m_scene, interaction, &pdf);

        // Make sure the pdf isn't zero and the radiance isn't black
        if (pdf != 0.0f && !all(Li)) {
            directLighting += bsdf->Eval(interaction) * Li / pdf;
        }
    }

    return directLighting;
}

We can implement light->SampleLi however we want; we can choose the point uniformly, or importance sample. In either case, we divide the radiosity by the pdf of choosing the point. Again, to satisfy the requirements of Monte Carlo.
If the BRDF is highly view dependent, it may be better to choose a point based on the BRDF, instead of a random point on the light. But how do we choose? Sample based on the light, or based on the BRDF?
Why not both? Enter Multiple Importance Sampling. In short, we evaluate $BSDF(p, \omega_{\text{i}}, \omega_{\text{o}}) L_{\text{i}}(p, \omega_{\text{i}})$ multiple times, using different sampling techniques, then average them together using weights based on their pdfs. In code this is:
float3 EstimateDirect(Light *light, UniformSampler *sampler, SurfaceInteraction &interaction, BSDF *bsdf) const {
    float3 directLighting = float3(0.0f);
    float3 f;
    float lightPdf, scatteringPdf;

    // Sample lighting with multiple importance sampling
    // Only sample if the BRDF is non-specular
    if ((bsdf->SupportedLobes & ~BSDFLobe::Specular) != 0) {
        float3 Li = light->SampleLi(sampler, m_scene, interaction, &lightPdf);

        // Make sure the pdf isn't zero and the radiance isn't black
        if (lightPdf != 0.0f && !all(Li)) {
            // Calculate the brdf value
            f = bsdf->Eval(interaction);
            scatteringPdf = bsdf->Pdf(interaction);

            if (scatteringPdf != 0.0f && !all(f)) {
                float weight = PowerHeuristic(1, lightPdf, 1, scatteringPdf);
                directLighting += f * Li * weight / lightPdf;
            }
        }
    }

    // Sample brdf with multiple importance sampling
    bsdf->Sample(interaction, sampler);
    f = bsdf->Eval(interaction);
    scatteringPdf = bsdf->Pdf(interaction);
    if (scatteringPdf != 0.0f && !all(f)) {
        lightPdf = light->PdfLi(m_scene, interaction);
        if (lightPdf == 0.0f) {
            // We didn't hit anything, so ignore the brdf sample
            return directLighting;
        }

        float weight = PowerHeuristic(1, scatteringPdf, 1, lightPdf);
        float3 Li = light->Le();
        directLighting += f * Li * weight / scatteringPdf;
    }

    return directLighting;
}

In English:

First, we sample the light

This updates interaction.InputDirection
Gives us the Li for the light
And the pdf of choosing that point on the light

Check that the pdf is valid and the radiance is non-zero
Evaluate the BSDF using the sampled InputDirection
Calculate the pdf for the BSDF given the sampled InputDirection

Essentially, how likely is this sample, if we were to sample using the BSDF, instead of the light

Calculate the weight, using the light pdf and the BSDF pdf

Veach and Guibas define a couple different ways to calculate the weight. Experimentally, they found the power heuristic with a power of 2 to work the best for most cases. I refer you to the paper for more details. The implementation is below

Multiply the weight with the direct lighting calculation and divide by the light pdf. (For Monte Carlo) And add to the direct light accumulation.
Then, we sample the BRDF

This updates interaction.InputDirection

Evaluate the BRDF
Get the pdf for choosing this direction based on the BRDF
Calculate the light pdf, given the sampled InputDirection

This is the mirror of before. How likely is this direction, if we were to sample the light

If lightPdf == 0.0f, then the ray missed the light, so just return the direct lighting from the light sample.
Otherwise, calculate the weight, and add the BSDF direct lighting to the accumulation
Finally, return the accumulated direct lighting

.
inline float PowerHeuristic(uint numf, float fPdf, uint numg, float gPdf) {
    float f = numf * fPdf;
    float g = numg * gPdf;

    return (f * f) / (f * f + g * g);
}

There are a number of optimizations / improvements you can do in these functions, but I've pared them down to try to make them easier to comprehend. If you would like, I can share some of these improvements.
Only Sampling One Light
In SampleLights() we loop through all the lights, and get their contribution. For a small number of lights, this is fine, but for hundreds or thousands of lights, this gets expensive. Fortunately, we can exploit the fact that Monte Carlo Integration is a giant average. Example:
Let's define
$$h(x) = f(x) + g(x)$$
Currently, we're estimating $h(x)$ by:
$$h(x) = \frac{1}{N} \sum_{i=1}^N f(x_i) + g(x_i)$$
But, calculating both $f(x)$ and $g(x)$ is expensive, so instead we do:
$$h(x) = \frac{1}{N} \sum_{i=1}^N \frac{r(\zeta, x)}{pdf}$$
Where $\zeta$ is a uniform random variable, and $r(\zeta, x)$ is defined as:
$$r(\zeta, x) =
\begin{cases}
  f(x), & 0.0 \leq \zeta \lt 0.5 \\
  g(x), & 0.5 \leq \zeta \lt 1.0
\end{cases}$$
In this case $pdf = \frac{1}{2}$ because the pdf must integrate to 1, and there are 2 functions to choose from.
In English:

Randomly choose either $f(x)$ or $g(x)$ to evaluate.
Divide the result by $\frac{1}{2}$ (since there are two items)
Average

As N gets large, the estimate will converge to the correct solution.
We can apply this same principle to light sampling. Instead of sampling every light, we randomly pick one, and multiply the result by the number of lights (This is the same as dividing by the fractional pdf):
float3 SampleOneLight(UniformSampler *sampler, SurfaceInteraction interaction, BSDF *bsdf, Light *hitLight) const {
    std::size_t numLights = m_scene->NumLights();

    // Return black if there are no lights
    // And don't let a light contribute light to itself
    // Aka, if we hit a light
    // This is the special case where there is only 1 light
    if (numLights == 0 || numLights == 1 && hitLight != nullptr) {
        return float3(0.0f);
    }

    // Don't let a light contribute light to itself
    // Choose another one
    Light *light;
    do {
        light = m_scene->RandomOneLight(sampler);
    } while (light == hitLight);

    return numLights * EstimateDirect(light, sampler, interaction, bsdf);
}

In this code, all the lights have an equal chance of being picked. However, we can importance sample, if we like. For example, we can give larger lights a higher chance of being picked, or lights closer to the hit surface. You just have to divide the result by the pdf, which would no longer be $\frac{1}{\text{numLights}}$.
Multiple Importance Sampling the "New Ray" Direction
The current code only importance samples the "New Ray" direction based on the BSDF. What if we want to also importance sample based on the location of lights?
Taking from what we learned above, one method would be to shoot two "new" rays and weight each based on their pdfs. However, this is both computationally expensive, and hard to implement without recursion.
To overcome this, we can apply the same principles we learned by sampling only one light. That is, randomly choose one to sample, and divide by the pdf of choosing it.
// Get the new ray direction

// Randomly (uniform) choose whether to sample based on the BSDF or the Lights
float p = sampler->NextFloat();

Light *light = m_scene->RandomLight();

if (p < 0.5f) {
    // Choose the direction based on the bsdf
    material->bsdf->Sample(interaction, sampler);
    float bsdfPdf = material->bsdf->Pdf(interaction);

    float lightPdf = light->PdfLi(m_scene, interaction);
    float weight = PowerHeuristic(1, bsdfPdf, 1, lightPdf);

    // Accumulate the throughput
    throughput = throughput * weight * material->bsdf->Eval(interaction) / bsdfPdf;

} else {
    // Choose the direction based on a light
    float lightPdf;
    light->SampleLi(sampler, m_scene, interaction, &lightPdf);

    float bsdfPdf = material->bsdf->Pdf(interaction);
    float weight = PowerHeuristic(1, lightPdf, 1, bsdfPdf);

    // Accumulate the throughput
    throughput = throughput * weight * material->bsdf->Eval(interaction) / lightPdf;
}

That all said, do we really want to importance sample the "New Ray" direction based on the light? For direct lighting, the radiosity is affected by both the BSDF of the surface, and the direction of the light. But for indirect lighting, the radiosity is almost exclusively defined by the BSDF of the surface hit before. So, adding light importance sampling doesn't give us anything.
Therefore, it is common to only importance sample the "New Direction" with the BSDF, but apply Multiple Importance Sampling to the direct lighting.
Comment: Multiple Importance Sampling can be applied anywhere you use importance sampling. The power of multiple importance sampling is that we can combine the benefits of multiple sampling techniques. For example, in some cases, light importance sampling will be better than BSDF sampling. In other cases, vice versa. MIS will combine the best of both worlds. However, if BSDF sampling will be better 100% of the time, there is no reason to add the complexity of MIS. I added some sections to the answer to expand upon this point
Comment: It seems we separated incoming radiance sources into two parts as direct and indirect. We sample lights explicitly for the direct part and while sampling this part, it is reasonable to importance sample the lights as well as BSDFs. For the indirect part, however, we have no idea about which direction may potentially give us higher radiance values since it is the problem itself that we want to solve. However, we can say which direction can contribute more according to the cosine term and BSDF. This is what I understand. Correct me if I'm wrong and thank you for your awesome answer.
Comment: Thank you for the clarifying answer! I understand that if we were to use a path tracer without explicit light sampling, we would never hit a point light source. So, we can basically add its contribution. On the other hand, if we sample an area light source, we have to make sure we should not hit it again with the indirect lighting in order to avoid double dip
Comment: Exactly! Is there any part that you need clarification on? Or there isn't enough detail?
Comment: Also, is multiple importance sampling used only for direct lighting calculation? Maybe I missed but I didn't see another example of it. If I shoot just one ray per bounce in my path tracer, it seems that I cannot do it for the indirect lighting calculation.
Comment: That's correct! Very good summary
Comment: Hi Richie, I enjoyed reading your answer here. While ago I posted a question on MIS but didn't get answer. I wonder if I could have your thoughts as well, thanks. https://computergraphics.stackexchange.com/questions/4868/efficiently-sampling-specific-surfaces-using-mis-in-path-tracing
Comment: @RichieSams Am I missing something or is your code sample not casting any shadow rays?
Comment: The first code block doesn't have shadow rays. However, the following code blocks add light sampling, which are a form of shadow rays
Comment: Hi, how to treat transparent object when testing with direct light?
Comment: The "correct" (but will need lots more SPP) approach is to treat all "transparent" objects as solid, for purposes of visibility for NEE. There is active research in how to get around that. For example: https://www.ics.uci.edu/~yug10/projects/translucent/report.html


proving $\cos (A+B)>0$, if given angles $A$ and $B$

Question: If $\displaystyle A=3\sin^{-1}\left(\frac{6}{11}\right)$ and $\displaystyle B = 3\cos^{-1}\left(\frac{4}{9}\right),$ then proving $\cos (A+B)>0$
Attempt: $$ 3\sin^{-1}\left(\frac{1}{2}\right)<3\sin^{-1}\left(\frac{6}{11}\right)<3\sin^{-1}\left(\frac{\sqrt{3}}{2}\right)\Rightarrow \frac{\pi}{2}<A<\pi$$
same way $$\displaystyle 3\cos^{-1}\left(0\right)<3\cos^{-1}\left(\frac{4}{9}\right)<3\cos^{-1}\left(\frac{1}{2}\right)\Rightarrow \frac{\pi}{2}<B<\frac{3\pi}{2}$$
so $$\pi<A+B<\frac{5\pi}{2}$$
could some help me how to prove $\cos (A+B)>0,$ thanks
Comment: Try using identity for $\cos (A+B)$
Comment: It is not necessary to actually find the value of $\cos(A+B)$ since $A+B$ can be shown to lie in either quadrant IV or quadrant I by the approach which you have taken. You just need tighter bounds on angles $A$ and $B$ than those you give. See my answer below.
Answer: Hint:
$\cos(3(A/3+B/3)) = 4 \cos^3 (A/3+B/3) - 3 \cos (A/3+B/3)  $
Now expand this and you will gets terms only in $\cos (A/3), \cos (B/3),\sin (A/3), \sin (B/3)$ which we can get from given hypothesis.
Answer: $A = 3\sin^{-1}(\frac{6}{11}), B = 3\cos^{-1}(\frac{4}{9})$
Let $C = \cos(\frac{A+B}{3})$
Note that $\cos (3x) = 4\cos^3x - 3\cos x$
Let $x=\frac{A+B}{3} \implies \cos x = \cos \frac{A}{3} \cos\frac{B}{3} -\sin\frac{A}{3}\sin\frac{B}{3}$
$\cos x = \cos (\sin^{-1}(\frac{6}{11}))\cos(\cos^{-1}(\frac{4}{9})) - \sin(\sin^{-1}(\frac{6}{11}))\sin(\cos^{-1}(\frac{4}{9}))$
Consider $y = \sin^{-1}(\frac{6}{11}) \implies \cos y = \frac{\sqrt{85}}{11}$
Consider $y = \cos^{-1}(\frac{4}{9}) \implies \sin y = \frac{\sqrt{65}}{9}$
Then $\cos x = \frac{\sqrt{85}}{11}\cdot \frac{4}{9} -\frac{6}{11}\cdot \frac{\sqrt{65}}{9} = \frac{4\sqrt{85}-6\sqrt{65}}{99}$
So then $\cos (3x) = 4\cdot (\frac{4\sqrt{85}-6\sqrt{65}}{99})^3-3\cdot (\frac{4\sqrt{85}-6\sqrt{65}}{99}) = 4\cdot (\frac{4\sqrt{85}-6\sqrt{65}}{99})^3+3\cdot (\frac{-4\sqrt{85}+6\sqrt{65}}{99})$
$\cos (3x) = \frac{134080\sqrt{85}-154080\sqrt{65})}{99^3} +\frac{-12\sqrt{85}+18\sqrt{65}}{99}$
$\cos (3x) = \cos(A+B) = \frac{(134080-117612)\sqrt{85} +(176418-154080)\sqrt{65}}{99^3} > 0$
and we're done
Answer: Let $\displaystyle \alpha=\sin^{-1}\left(\frac{6}{11}\right)=\frac{A}{3}$ and let $\displaystyle\beta=\cos^{-1}\left(\frac{4}{9}\right)=\frac{B}{3}$
Then since $\frac{1}{2}<\frac{6}{11}<\frac{\sqrt{2}}{2}$ we have $\frac{\pi}{6}<\alpha<\frac{\pi}{4}$.
And since $0<\frac{4}{9}<\frac{1}{2}$ we have $\frac{\pi}{3}<\beta<\frac{\pi}{2}$.
Addition of the two inequalities gives $\frac{\pi}{2}<\alpha+\beta<\frac{3\pi}{4}$.
Multiplication by $3$ gives $\frac{3\pi}{2}<3\alpha+3\beta<\frac{9\pi}{4}$.
Thus $\frac{3\pi}{2}<A+B<\frac{9\pi}{4}$ meaning that $A+B$ lies in either quadrant IV or quadrant I.
Thus $\cos(A+B)>0$.
What has been shown here is that, referring to the following diagram, for any standard angle $A$ such that the terminal side of $\dfrac{A}{3}$ lies in the blue region and for any standard angle $B$ such that the terminal side of $\dfrac{B}{3}$ lies in the green region it will be the case that $\cos(A+B)>0$. The "red" lines are the terminal sides of the particular $\dfrac{A}{3}$ and $\dfrac{B}{3}$ given in the question.
Comment: Your initial idea was good, you just needed better bounds on your angles.


Is there a way to flag user profile spam?

Question: I stumbled over this the user profile on Drupal SE. It belongs to somebody whose nick is "Taken 3 Online FREE Full HD".
This year old profile is clearly created just to promote an URL (I haven't clicked, since it may even link to malware), which has nothing to do with Drupal.
I know how to flag spam in Q&A, but can't find anything to flag a profile.  Am I missing something obvious?
Answer: There isn't a way to flag user profiles. If the users didn't post anything, you can just open a question on this very site, so the moderators can take the necessary actions.
I deleted the account, since the user didn't want to participate in our site, and it was only created to post those links you saw in their user profile.


django test runner do not see tests on py2 version, but see them on py3

Question: When I run a command on py3:
python manage.py test myproject.user_management --settings mysettings.py

I see all my tests working fine:

Ran 29 tests

But the same command on py2:

Ran 0 tests

Command on py2 is only working when I add tests folder to command, like:
python manage.py test myproject.user_management.test --settings mysettings.py

I already:
1. Double check my imports an __init__.py in all folders and sub folders
2. Settings etc.
My app structure looks like:
project_name
----user_management
--------__init__.py
--------tests
------------__init__.py
------------training
----------------__init__.py
----------------training.py

In python manage.py shell I can easy do:
from myproject.user_management.tests import TrainingViewSetTest
This behaviour happens only for few apps in my project.
Comment: Does the filenames of your tests start with `test*`?
Answer: Follow this structure for using the default Django test runner:
user_management/
   __init__.py
   tests/
       __init__.py
       test_training.py

It should be able to find your tests in both versions now.
With that said, over the years I've started to put all of my tests in a root tests/ directory, and use the following pytest configuration to find those tests (setup.cfg):
[tool:pytest]
addopts = --reuse-db --ds=settings.test
python_files =
    tests/integration/*.py
    tests/functional/*.py
test_paths =
    tests/

This has the benefit of quick test collection as well (not searching many directories and analyzing many files inside Docker on MacOSX!)


How to get the last not null value from previous rows

Question: I know that there are a lot of solutions for this but unfortunately I cannot use partition or keyword TOP. Nothing I tried on earlier posts works.
My table looks like this:

The result I want is when any completion percentage is NULL it should get the value from last non-value completion percentage, like this:
Answer: This is most easily done using outer apply:
select t.projectname, t.sequence,
       coalesce(t.completion_percentage, t2.completion_percentage) as completion_percentage
from t outer apply
     (select top 1 t2.*
      from t t2
      where t2.projectname = t.projectname and
            t2.completion_percentage is null and
            t2.sequence < t.sequence
      order by t2.sequence desc
     ) t2;

Answer: you can try this:
SELECT t2.project_name,t2.sequence,
case when percentage is null then (select percentage from table1  t1 where t1.sequence=t2.sequence-1 )
else t2.percentage end as percentage
FROM table t2


Javascript dataTransfer.setData is adding text to Clipboard in Internet Explorer

Question: I have a web page that has draggable UI elements. When using Internet Explorer, whenever I drag any UI element, some text gets added to the clipboard. When using Chrome this doesn't happen.
The text that is added is needed for other operations, but I don't want it to be added in the clipboard, so the behavior I have in Chrome is the one I want.
I did some troubleshooting and I was able to identify the code where this happens, bellow is a snippet of code where this can be replicated:

<!DOCTYPE html>

Test

Drag me

In the above example, if you run the snippet of code in Internet Explorer, if you drag the "Drag me" element, you will notice that some text gets added in the Clipboard, so next time you CTRL+V that text will be pasted.
Did anyone ever faced this problem? Is there a way for me to have the same behavior on both browsers? Or I will have to change my implementation?
Comment: Just for the sake of interest: Why did you chose to use the DataTransfer API in the first place? The method DataTransfer.setData is not supported in IE11: [caniuse DataTransfer.setData](https://caniuse.com/mdn-api_datatransfer_setdata)
Answer: The behavior in IE11 is a bit weird. I tested your snippet with jsbin.com and started with commenting out
ev.dataTransfer.effectAllowed = 'move';

The result was I was able to drag the element #me and no content was added into the clipboard via IE11.
Additionally I successfully retrieved the value of 'TEXT' via the DataTransfer api.
I cannot explain why the initial behavior described above appears in IE11.
Comment: When I remove the effectAllowed call (as suggested) I still have the same issue. It was the only thing you did?
Comment: No, took a look into caniuse.com. Please refer to the comment below your opening post


Datum object in caffe - unsupervised networks

Question: from a script on preparing data for a caffe network, the following piece of code turns an image (numpy array representing an image) into a datum object.
datum = caffe_pb2.Datum(
        channels=3,
        width=224,
        height=224,
        label=label,
        data=np.rollaxis(img, 2).tostring())

If the network were unsupervised, do you just create the object the same way but do not fill the label parameter, as shown below?
datum = caffe_pb2.Datum(
            channels=3,
            width=224,
            height=224,
            data=np.rollaxis(img, 2).tostring())

Answer: The label of Datum is optional:

optional int32 label = 5;

Meaning oyu do not have to provide it.
Side note:
Datum is a data structure used mainly for "Data" input layer and strictly speaking it is not part of the trained net.
Caffe uses N-D tensors Blobs to store both data and parameters of the net.
Comment: Shai, I wonder if you can take a stab at this https://stackoverflow.com/questions/45942883/bekeley-caffe-command-line-interface


Extract multiple IPv4 addresses from single field separated by ","

Question: I'm tryin to extract multiple IPv4 addresses from a single field that are separated by ","
Example string: "badips":["123.456.789.12","123.456.789.13","123.456.789.14"]
Expected output of field badips with values 123.456.789.12 123.456.789.13 123.456.789.14
^[^\[\n]*\["(?P<badips>\d+\.\d+\.\d+\.\d+) gets me the first IP
^[^\[\n]*\["(?P<badips>.*?)\"] gets me 123.456.789.12","123.456.789.13","123.456.789.14 with the "," in between.
I'm trying to come up with something dynamic because there will be atleast 1 IP in the field, but there could be more. ] will always follow the last IP
I'm new to regex and any help will be greatly appreciated. I have a regex cheat sheet and have been working at this all day, just can't seem to get it
Comment: If you can be specific about the regex flavor (ERE, PCRE, PCRE2, etc.) and/or your using language, you will have appropriate answers.
Comment: Does this answer your question? [RegEx for an IP Address](https://stackoverflow.com/questions/4890789/regex-for-an-ip-address) or one of the many other questions/answers with this search: [`[regex] "IP address"`](https://stackoverflow.com/search?q=%5Bregex%5D+%22IP+address%22)
Comment: Or perhaps one of these: https://stackoverflow.com/questions/52841454/regex-to-match-multiple-subnets-delimited-by-a-space, https://stackoverflow.com/questions/52972885/regex-to-match-multiple-ipaddress-domainnames-delimited-by-a-space, https://stackoverflow.com/questions/34268391/multiple-matches-with-delimiter
Comment: Start by creating a regex that matches a single valid IP (you can find this on Google _very_ easily). Then make a method to split the string on commas to separate all your IP addresses. Finally run through each IP address you separated and match it up against your regex to see if it's valid. If your IP's are actually inside JSON then use a JSON parser to get them out.
Comment: There is no way to do it with just regex. If you do it in Splunk, do it in two steps: `rex field=_raw "\["(?.*?)"]" | makemv delim="\x22,\x22" _raw`
Answer: You should use capture groups:
(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})+

demo here
Additional information about your implementation (language, regex flavor) will help us give you a better answer.


How to deploy my site?

Question: I am running PHPStorm, bought the Droplet on DigitalOcean.com and have the domain on 123.reg. Please help, I really need it I would really appreciate if you could talk to me in simple terms and guide me through the process, this is also my first site.
Comment: Please first try it yourself and come then back once you have issues and let us help you to find them
Comment: The best way to learn is by trial and error.
Comment: I have done. I am lost from the very star though  :(
Comment: I understand that but I cannot find the FTP Information etc
Answer: DO has really good tutorials on their own website, please refer to those and ask specific questions here. Stackoverflow is a specific problem solving forum, not a 'fix my stuff' service.
Guide to setup Apache and upload your site on a DigitalOcean droplet
Comment: Do yourself a favor and use GIT. If you have to use FTP, there is tutorials for that to: https://anotherguru.me/how-to-install-vsftpd-on-ubuntu-14-04-on-digitalocean-droplet/
Comment: On my first Google search I literally found more than 10 step-by-step tutorials you just can't mess up. Just try them out.
Comment: Okay so I got the FTP stuff and everything but I am not sure how to host it with DO and link PHPStorm to 123 reg?

How to deploy my site?


How to Add Extra Tab Using Template Language Twig In OpenCart 3.x

Question: I want an extra description tag termed "Overview" for the product page under the nav nav-tabs class.
This is the complete Twig template code:

{{ tab_description }}

        {% if attribute_groups %}

{{ tab_attribute }}

        {% endif %}
        {% if review_status %}

{{ tab_review }}

        {% endif %}

{{ description }}

        {% if attribute_groups %}

            {% for attribute_group in attribute_groups %}

                {{ attribute_group.name }}

            {% for attribute in attribute_group.attribute %}

              {{ attribute.name }}
              {{ attribute.text }}

            {% endfor %}

            {% endfor %}

        {% endif %}
        {% if review_status %}

          <form class="form-horizontal" id="form-review">

{{ text_write }}

            {% if review_guest %}

                {{ entry_name }}
                <input type="text" name="name" value="{{ customer_name }}" id="input-name" class="form-control" />

                {{ entry_review }}
                <textarea name="text" rows="5" id="input-review" class="form-control"></textarea>

{{ text_note }}

                {{ entry_rating }}
                    {{ entry_bad }}
                <input type="radio" name="rating" value="1" />

                <input type="radio" name="rating" value="2" />

                <input type="radio" name="rating" value="3" />

                <input type="radio" name="rating" value="4" />

                <input type="radio" name="rating" value="5" />
                 {{ entry_good }}

            {{ captcha }}

                <button type="button" id="button-review" data-loading-text="{{ text_loading }}" class="btn btn-primary">{{ button_continue }}</button>

            {% else %}
            {{ text_login }}
            {% endif %}
          </form>

        {% endif %}

With this code, the page looks like this:
I want one more tab there saying OVERVIEW with different content.
Can anyone help me out here with this?
P.S: I did read this: how to add extra tab without any extension in product page in opencart
But, I'm not sure where to find this piece of code (I've tried looking product TPL files as well).
Answer: You have just add two things
in ul tag add this li

My Tab

and add one div for tab-content

My Tab Content

Comment: from dashboard turn off cache both option its will not load cache again and clean cache also.
Comment: Thanks for the answer. Now I do realize that it's the correct procedure to add a tab there. But, even after adding those two things - I'm not able to see the new tab - even after clearing the cache.
Comment: clean cache from system/storage/cache delete all file instead of index.html
Comment: Given that I've already found the answer. I'll upvote the answers and mark the first one as the correct answer. I don't have file permissions as of now to delete the cache files - will contact my hosting and will see what can be done about it.
Answer: You Should try this its will add new tab name Overview

{{ tab_description }}

    {% if attribute_groups %}

{{ tab_attribute }}

    {% endif %}
    {% if review_status %}

{{ tab_review }}

    {% endif %}

Overview

{{ description }}

    {% if attribute_groups %}

        {% for attribute_group in attribute_groups %}

            {{ attribute_group.name }}

        {% for attribute in attribute_group.attribute %}

          {{ attribute.name }}
          {{ attribute.text }}

        {% endfor %}

        {% endfor %}

    {% endif %}
    {% if review_status %}

      <form class="form-horizontal" id="form-review">

{{ text_write }}

        {% if review_guest %}

            {{ entry_name }}
            <input type="text" name="name" value="{{ customer_name }}" id="input-name" class="form-control" />

            {{ entry_review }}
            <textarea name="text" rows="5" id="input-review" class="form-control"></textarea>

{{ text_note }}

            {{ entry_rating }}
                {{ entry_bad }}
            <input type="radio" name="rating" value="1" />

            <input type="radio" name="rating" value="2" />

            <input type="radio" name="rating" value="3" />

            <input type="radio" name="rating" value="4" />

            <input type="radio" name="rating" value="5" />
             {{ entry_good }}

        {{ captcha }}

            <button type="button" id="button-review" data-loading-text="{{ text_loading }}" class="btn btn-primary">{{ button_continue }}</button>

        {% else %}
        {{ text_login }}
        {% endif %}
      </form>

    {% endif %}

Overview

Output Image
Comment: Thanks for your prompt answer. Unfortunately, even after replacing that piece of code with yours, it did not show up. https://i.imgur.com/5kkFEAJ.jpg
Comment: clean your cache manually from system/storage/cache remove all file keep only index.php


Images not showing up in Safari, but will show up in any other browser

Question: My Images not showing up in Safari, but will show up in any other browser.
When I pull up the dev tools-- they are there, just not rendering to the page.
Hope someone can help me out:)
Answer: oh i see it's not https:// it must be https://cdn.img.facebook.org/ or else safari won't show it


Connecting a Honda 7000iS portable generator to a transfer switch

Question: I recently purchased a Honda EU7000iS generator (manual) to power our house when the power goes out. The house came with a transfer switch but I'm unsure which cord to buy to connect the two. Can someone point me in the right direction?
I've attached a picture of the instructions manual for the transfer switch.
Answer: Good news: you can use this generator with this transfer switch
The good news is that the EU7000is, luckily, is a floating neutral generator, so it can be used with simpler transfer switches, such as your "select circuit" switch, or breaker interlocks without violating the NEC's grounding and bonding provisions.  (Many jobsite generators have bonded neutrals as per the OSHA rules on worksite use, but this means that they need to be used as separately derived systems. This, in turn, requires a transfer switch which switches the neutral wire over so that you don't put the two neutral-ground bonds in parallel inadvertently, which'd cause merry havoc with stray neutral currents and even GFCI tripping.)
...but the cord you'll need is kind of odd
The problem is that you'll basically need an adapter in order to plug your generator into your house, as your transfer switch has a 50A, California Standard twistlock inlet which requires the correct connector (the inlet's a CS6375 and the mating cord cap's a CS6364), while your generator has a NEMA L14-30R on it.  Such a thing does exist, or you could use a length of 8/4 outdoor cordage (such as SOOW), a NEMA L14-30P, and a CS6364 cordcap to fabricate your own cord.
Comment: @MikeBuss yeah, you'd plug the adapter in at the transfer switch, and 40' should be fine, you don't want the generator right up against the house for fire and CO safety reasons
Comment: Thank you for your quick response! With the adapter you mentioned, can I buy a L14-30P cable like [this](https://www.amazon.com/L14-30P-L14-30R-Generator-Extension-Cords/dp/B07V4D5J6L/ref=sr_1_5?dchild=1&keywords=30+Amp+4-prong+NEMA+L14-30P&qid=1607131633&sr=8-5) to pair with it? Side question: would 40 ft length be OK, or should I try to stick to a shorter cable?
Answer: See where it says "NEMA 14-50"? That's the designation of the cord cap that you need for the generator. You also need cord rated for 50A, so depending on length, at least #6, 3 conductor + ground (so 4 "wires" in the cord). You can usually find those at RV supply stores.
Comment: Uh...that's not a NEMA 14-50... (I *wish* there was a NEMA L14-50 to be fair...anyone care to petition NEMA to adopt the California Standard 50A twistlocks?) (also, bad Generac!)


iOS - OAuth implementation how to create Framework ? (simple)

Question: Hi,
I need to include OAuth in my Project, to do that i found this simple "tuto".
But i don't understand how can i go from the provided files to a compiled framework file.
This is what they say :

Importing the Framework There are 3 steps for adding the pre-compiled
  framework to your app in Xcode:

Drag the framework into your Linked Frameworks group in Xcode,
  choosing to copy the framework instead of simply referencing it.
Create a new Copy Files Build Phase for your app's main target.
Drag the framework from the Linked Frameworks folder into the new Copy
  Files Build Phase and select "Frameworks" as its destination.

For now, please use this as a private embedded framework rather than copying it
  to the system's framework folder upon installation.

The framework : OAuthLibrary
When i compile it i get an .a file then, i don't understand what to do. (What is the Linked Frameworks group) ? Where i put this .a file ?
Thanks.

Edit : I'v just found this RSOAuthEngine - OAuth with ARC but it's for OAuth 1.
I'm looking for same think but for OAuth 2.
Answer: From the OAuthLibrary's web page, it appears to target Mac OS X only, not iOS.
Also, iOS doesn't support frameworks built by third parties. Static libraries are supported, however. So you would need to (1) build a static library with the OAuthLibrary source files, or (2) incorporate the source files directly into your app's project. But this doesn't alter the fact that the library appears to be designed for Mac OS X only.
Comment: Thanks for your answer, i didn't see that was only for OS X.
I'll use https://github.com/thekarladam/MPOAuth instead.
By the way, it's hard to find some source using ARC. If you have any link, thanks for sending.


How to change the size of the off-heap in neo4j?

Question: Does anyone know which parameters can affect the size of the off-heap memory?
In neo4j documentation, they said that the size of the off-heap memory can be modified with the dbms.pagecache.memory parameter.
I tried changing the size of this parameter, but when I check the off-heap memory with jconsole, the size is always the same.
PS: I'm working with the free version of neo4j.
Comment: which version are you using?
Comment: I'm using the 2.1.6 of the community edition
Answer: Neo4j <= 2.1.x uses the so called MMIO cache as first level cache. This basically uses mapped memory capabilities provided by the operating system. On Unix style OSes the file buffer cache is off heap on windows it's on heap. For a verbose description including the config settings refer to http://neo4j.com/docs/2.1.8/configuration-caches.html#_file_buffer_cache.
In Neo4j 2.2 this cache layer's implementation was changed to the page cache. Page cache is off heap on all OSes. It's configuration has been reduced to just on setting:
<code>dbms.pagecache.memory
</code>
So you've used a 2.2 config option on a 2.1 instance. So either use the set of options for 2.1 or do a upgrade.
Comment: Thanks for your answer.
So I tried to modify the file buffer cache with: `neostore.nodestore.db.mapped_memory`
`neostore.relationshipstore.db.mapped_memory`
`neostore.propertystore.db.mapped_memory`
`neostore.propertystore.db.strings.mapped_memory`
`neostore.propertystore.db.arrays.mapped_memory`
I put this parameters in the neo4j.properties.
Although I change the size of this parameters, the size of the off-heap memory remained the same.
Is it normal? My Neo4j servers are on debian virtual machines.


How can I improve solid, plastered walls with minimal mess and cost?

Question: The walls of my house, like many in this area, are pretty bad. Apparently a higher than normal amount of sand was used in the bricks and the mortar, and the plaster is very old and weak. Even putting in a nail to hang a picture on risks taking a large chunk out.
To make matters worse the previous owner covered the walls with woodchip/anaglyptic wallpaper which has been painted over half a dozen times. Woodchip is messy and difficult to remove at the best of times and I'm sure half the wall would come with it.
The obvious solution is to have the walls re-plastered, or have frames and plasterboard fitted. As much as I would love to do that (and get the place re-wired too) it's both expensive and extremely messy.
What other options are there? I'd like to end up with smooth walls suitable for painting and ideally some extra wiring for power and network. The skirting boards and ceiling coving really want replacing too but are less important.
Comment: Cheap, fast, good. Pick any two. :) You already have your options laid out.
Comment: Put enough nails in the walls in an attempt to hang things that you're left with no option but to rip it all out and start fresh?
Comment: Magic wands being difficult to come by, you're into mess and expense unless you want to try *sell this and buy another,* which is its own sort of mess and expense. Owning a house can be messy, and expensive.
Comment: What was used as the lath under the plaster coats? Wood slats? Gypsum lathboard? Expanded metal?
Answer: You could overlay with 1/4" gypsum, and finish as you would any other gypsum board wall. Still messy, but you would permanently fix the problem. 1/4" gypsum can even be applied with adhesives, and improves the sound transmission of the wall.
Comment: If the plaster is coming off I wouldn't want to stick anything to it. I suppose the adhesive might bridge the sections that are loose and distribute the load to sound plaster but it doesn't seem very safe.
Comment: Imagine a 6"x6"x1/4" piece adhered to the wall. Would that stay? It's extremely light weight. The loading is exactly identical to that if the adhesive is well distributed. Additionally, you can score and remove the weakest pieces before installing if it's falling off every time someone sneezes.
Comment: So you are saying stick plasterboard over it? Could be an issue for the coving and skirting I think but it might work.
Comment: All trim would have to be removed and reapplied to be done right.


How to Access the Source name in DragOver

Question: I cannot figure out how to access the name of the Source TObject being dragged over a panel.
procedure TfrmBodyPlan.pnlStepsDragOver(Sender, Source: TObject; X, Y: Integer;State: TDragState; var Accept: Boolean);

I know that the object is a TAdvShape.
Answer: Just cast the object :
if Sender is TComponent then
  LName := TComponent(Sender).Name;


Ubuntu 13.04 display problem (Samsung ATIV Book 9 Lite)

Question: I'm into trouble for few days and I really need help.
So, I've got a new laptop (Samsung ATIV Book 9 Lite) and installed Ubuntu 13.04 (64bit). Now everything look good but my graphical display is very bad (launcher doesn't take opacity and always displaying 1px border-right even it's hidden, windows are very slow etc..).
Actually I'm not prof-tech-man and I'm just stuck!
Q: What I need to do any more or am missing?
Thanks in advance.
Here is some infos;
kerem@kerem:~$ sudo lshw -c video
*-display UNCLAIMED
   description: VGA compatible controller
   product: Kabini [Radeon HD 8250]
   vendor: Advanced Micro Devices [AMD] nee ATI
   physical id: 1
   bus info: pci@0000:00:01.0
   version: 00
   width: 64 bits
   clock: 33MHz
   capabilities: pm pciexpress msi vga_controller bus_master cap_list
   configuration: latency=0
   resources: memory:c0000000-cfffffff memory:d0000000-d07fffff ioport:f000(size=256) memory:feb00000-feb3ffff memory:feb40000-feb5ffff

Seems not all OK;
kerem@kerem:~$ /usr/lib/nux/unity_support_test -p
OpenGL vendor string:   VMware, Inc.
OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.2, 128 bits)
OpenGL version string:  2.1 Mesa 9.1.3

Not software rendered:    no
Not blacklisted:          yes
GLX fbconfig:             yes
GLX texture from pixmap:  yes
GL npot or rect textures: yes
GL vertex program:        yes
GL fragment program:      yes
GL vertex buffer object:  yes
GL framebuffer object:    yes
GL version is 1.4+:       yes

Unity 3D supported:       no

Also I saw this and: https://wiki.ubuntu.com/X/HybridGraphics
kerem@kerem:~$ dpkg -l fglrx\*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
un  fglrx-glx      <none>                    (no description available)

Answer: You need to install Catalyst.
For your card and kernel, you want 13.6.
Follow these directions exactly, and you should be good to go.
Comment: @qeremy np!  was in the same predicament until i stumbled upon that answer via another q.  13.6 should be out of beta soon.
Comment: Hey! I used "AMD Catalyst™ 13.4 Proprietary Linux x86 Display Driver" not beta on this page (http://support.amd.com/us/gpudownload/linux/Pages/radeon_linux.aspx) and everything OK. Thank you again! :)
Comment: @qeremy oh good!  didn't see your card on the list.


How to migrate a complex project from Angular 1.5 to 2?

Question: I have a production site with realtime data.

The site is based on jHipster (angular 1.5 + java).
The site is state based.
Websockets are used for asynchronous communication.

Now, I want to migrate the site from angular v1.5 to v2. On reading the official angular migration guide, it seems I have to rewrite a lot of code. What is the best approach here for migration?

Write everything in angular 2 from scratch.
or
Follow the upgrade guide (using the upgrade module) in the official angular doc
Comment: write it again from scratch. that always makes for a better codebase, even if it's rewritten in the same environ as the first time, you learned from your mistakes so the 2nd version is better.
Comment: Only you can decide that...try both approaches
Comment: Angular 2 is **VERY** different from Angular 1. I'm surprised that it even has an upgrade guide
Comment: Build a sample ang2 app first to get a taste... then build your old app from the ground up in ang2, will be miles quicker that a conversion.
Answer: It depends.
Anyways, you have to rewrite a lot of code. That's for sure. And there is no easy way.
I will share with you two ideas (approaches), other then simply re-writing everything from the bottom up:

Using the ngForward approach.
In the ngForward approach, we are not actually upgrading existing Angular 1.x applications — we are creating an environment where these 1.x applications will behave like Angular 2 applications. In other words, we can gradually rewrite our Angular 1.x app to work like an Angular 2 app. We can then further augment new Angular 2 features to this 1.x application on the top of the existing app.
Find out more in this blog post.
Using the ngUpgrade approach.
By using the ngForward approach, we can write the code in the Angular 2.0 way on top of the existing Angular 1.x applications. However, this approach may not be comfortable for some as concerns over its maintainability and backward compatibility with the future Angular versions could be an issue. The ngForward approach will definitely cause problems for bulky Angular 1.x applications in the long run. This problem has been nailed down to a large extent with the use of the ngUpgrade approach. The ngUpgrade approach has the comprehensive documentation which can be obtained from the official Angular website.

You need to make a decision based on your specific use-case, project objectives and etc.
Credits: I learned that from this tutorial.
Answer: There's a third approach as well: create the parallel code base in Angular 2 and link it with the old Angular 1x code via url navigations. Here you will not need to fix or change any LOC of previous code base except for adding new navigation URLs. This will save your time. Your new code will have pure Angular 2, without any mixes of 1.5 and 2x at the same time on a page. The only challenge is to share user state and data in between requests since you cannot use shared services or factories anymore, but this is doable.
My history of similar migrations has been recently published in this blog post.


Creating a new Data.Frame from variable values

Question: I am currently working on a task that requires me to query a list of stocks from an sql db.
The problem is that it is a list where there are 1:n stocks traded per date. I want to calculate the the share of each stock int he portfolio on a given day (see example) and pass it to a new data frame. In other words date x occurs 2 times (once for stock A and once for stock B) and then pull it together that date x occurs only one time with the new values.
'data.frame':   1010 obs. of  5 variables:
 $ ID         : int  1 2 3 4 5 6 7 8 9 10 ...
 $ Date      : Date, format: "2019-11-22" "2019-11-21" "2019-11-20" "2019-11-19" ...
 $ Close: num  52 51 50.1 50.2 50.2 ...
 $ Volume     : num  5415 6196 3800 4784 6189 ...
 $ Stock_ID  : Factor w/ 2 levels "1","2": 1 1 1 1 1 1 1 1 1 1 ...

RawInput<-data.frame(Date=c("2017-22-11","2017-22-12","2017-22-13","2017-22-11","2017-22-12","2017-22-13","2017-22-11"), Close=c(50,55,56,10,11,12,200),Volume=c(100,110,150,60,70,80,30),Stock_ID=c(1,1,1,2,2,2,3))
RawInput$Stock_ID<-as.factor(RawInput$Stock_ID)

*cannot transfer the date to a date variable in this example
I would like to have a new dataframe that generates the Value traded per day, the weight of each stock, and the daily returns per day, while keeping the number of stocks variable.
I hope I translated the issue properly so that I can receive help.
Thank you!
Comment: Providing a snipped of your data via `dput(head(df, 20))` would make it significantly easier for people to help you. Where `df` is your `data.frame`.
Comment: What `Date` should `2017-22-13` be?
Comment: `> dput(head(RawInput))
structure(list(ID = 1:6, Date = structure(c(18222, 18221, 18220,
18219, 18218, 18215), class = "Date"), Close = c(52.03,
51.03, 50.1, 50.16, 50.23, 50.68), Volume = c(5415, 6196, 3800,
4784, 6189, 7753), Stock_ID = structure(c(1L, 1L, 1L, 1L, 1L,
1L), .Label = c("1", "2"), class = "factor")), row.names = c(NA,
6L), class = "data.frame")`
Comment: -And the date is of course meant to be `2017-12-23` or something comparable
Answer: I think the easiest way to do this would be to use the dplyr package. You may need to read some documentation but the mutate and group_by function may be able do what you want. This function will allow you to modify the current dataframe by either adding a new column or changing the existing data.
Lets start with a reproducible dataset
RawInput<-data.frame(Date=c("2017-22-11","2017-22-12","2017-22-13","2017-22-11","2017-22-12","2017-22-13","2017-22-11"),
                    Close=c(50,55,56,10,11,12,200),
                    Volume=c(100,110,150,60,70,80,30),
                    Stock_ID=c(1,1,1,2,2,2,3))
RawInput$Stock_ID<-as.factor(RawInput$Stock_ID)
library(magrittr)
library(dplyr)
dat2 <- RawInput %>%
    group_by(Date, Stock_ID) %>%  #this example only has one stock type but i imagine you want to group by stock
    mutate(CloseMean=mean(Close),
           CloseSum=sum(Close),
           VolumeMean=mean(Volume),
           VolumeSum=sum(Volume)) #what ever computation you need to do with
                                  #multiple stock values for a given date goes here
dat2 %>% select(Stock_ID, Date, CloseMean, CloseSum, VolumeMean,VolumeSum) %>% distinct() #dat2 will still be the same size as dat, thus use the distinct() function to reduce it to unique values
# A tibble: 7 x 6
# Groups:   Date, Stock_ID [7]
  Stock_ID Date       CloseMean CloseSum VolumeMean VolumeSum
  <fct>    <fct>          <dbl>    <dbl>      <dbl>     <dbl>
1 1        2017-22-11        50       50        100       100
2 1        2017-22-12        55       55        110       110
3 1        2017-22-13        56       56        150       150
4 2        2017-22-11        10       10         60        60
5 2        2017-22-12        11       11         70        70
6 2        2017-22-13        12       12         80        80
7 3        2017-22-11       200      200         30        30

This data set that you provided actually only has one unique Stock_ID and Date combinations so there was nothing actually done with the data. However if you remove Stock_ID where necessary you can see how this function would work
dat2 <- RawInput %>%
        group_by(Date) %>%
        mutate(CloseMean=mean(Close),
               CloseSum=sum(Close),
               VolumeMean=mean(Volume),
               VolumeSum=sum(Volume))
dat2 %>% select(Date, CloseMean, CloseSum, VolumeMean,VolumeSum) %>% distinct()

# A tibble: 3 x 5
# Groups:   Date [3]
  Date       CloseMean CloseSum VolumeMean VolumeSum
  <fct>          <dbl>    <dbl>      <dbl>     <dbl>
1 2017-22-11      86.7      260       63.3       190
2 2017-22-12      33         66       90         180
3 2017-22-13      34         68      115         230

After reading your first reply, You will have to be specific on how you are trying to calculate the weight. Also define your end result.
Im going to assume weight is just percentage by total cost. And the end result is for each date show the weight per stock. In other words a matrix of dates and stock Ids
library(tidyr)
RawInput %>%
     group_by(Date) %>%
     mutate(weight=Close/sum(Close)) %>%
     select(Date, weight, Stock_ID) %>%
     spread(key = "Stock_ID", value = "weight", fill = 0)

# A tibble: 3 x 4
# Groups:   Date [3]
  Date         `1`    `2`   `3`
  <fct>      <dbl>  <dbl> <dbl>
1 2017-22-11 0.192 0.0385 0.769
2 2017-22-12 0.833 0.167  0
3 2017-22-13 0.824 0.176  0

Comment: Actually the result you presented at last is kind of where this should be eventually. Still the hurdle I want to take is that I want to assign the unique date (like in your last table) a Portfolio dataframe, that contains the weight of each stock in the portfolio (for the first 2 days weight of stock 3 would be 0). These weights could translate to a weighted portfolio value on a given day. So if there is a long list from the database that has 1 stock on 1 day and 3 stocks on a 2nd day then I want to be able to filter that information and keep it flexible so to say. Thank you very much!
Comment: It sounds like you will need to write a function if you are looking for a custom summary tool. Good luck with your endeavor
Comment: Thank you very much you answered the question


Low water pressure from tub spout after new cartridge and spout

Question: I had very low water pressure coming out of my bathtub spout and the shower diverter was faulty so I removed the spout. The pressure was low coming out of the 1/2 inch copper pipe.
I then replaced the Moen 1222 single handle cartridge. It was extremely stuck and when I finally got it out the black silicone "plugs" on either side were stuck inside the supply lines on each side. I removed the silicone, used CLR and flushed the line. Water shot out like a fire hydrant!
I installed a new cartridge and again had almost no water pressure. I purchased another cartridge in case it was faulty, tried blowing out the pipe installed the new cartridge and was back to the initial low pressure out of the tub spout that I started with. After soaking and cleaning my tub spout I reinstalled it and could not get water to go up to the shower.
I have a new spout to install but am hesitant because my water pressure is still so low. Since it shoots out without the cartridge I feel something must be stuck between the valve and the drop pipe to the tub. As far as I can tell the valve does not look to have any corrosion. There is just one very small hole in it that could lead to the tub spout and I'm wondering if it is too small of a hike or if something is just stuck in there?
Comment: What method do you use to divert tub spout flow up to the shower, a valve integrated into your tub spout, or a lever or button integrated into the shower valve unit?
Comment: Did the water pressure drop suddenly or did it drop slowly over time?
Answer: I've dealt with a few very similar problems.
I'm not sure what you mean by very low water pressure but assuming it's not a low-flow valve the problem is a blockage between the valve and the faucet. If this happened slowly over time it's probably the result of corrosion/calcium buildup in the supply line from the valve which can be cleared with CLR or another decalcifying agent and flushing. However, I have the impression that this happened fairly suddenly. If so, what probably happened is that pieces of the black plugs you decscribed broke off and are lodged in the supply line between the valve and the faucet.
Usually when this happens the pieces flow through and are caught in the faucet strainer - but I'm assuming you checked that. Since that didn't happen your best bet is to remove the valve and the faucet and try to funnel some CLR or other cleaning agent down the supply line from the valve. I would then run a narrow flexible cable down the supply line from the top and up through the outlet at the spout end to try to break up and dislodge the debris. It will require some patience but 1/2 inch copper is large enough that it shouldn't be clogged too tightly.
I should also mention that the probable cause of the diverter not working was the low water pressure. Most diverters require a certain level of water pressure to engage and hold the gate diverter open.


Can't access the WEB API Cookie via document.cookie in Javascript

Question: I Implemented a AngularJS Project, Service Side implementation I used the .NET WEB API. For Authorization I'm using
FormsAuthentication.SetAuthCookie(token, false);

It's coming fine in browser Cookies.

But I can't able to get the cookies in JavaScript

Kindly assist me, why It's coming empty ?
Answer: The reason for this is that .MDAUTH cookie is set as HttpOnly. This means only the server side will be able to access the cookie and not the client side. You need to explicitly set HttpOnly to false while setting the cookie in the backend.


I defeated Skeletron and have 4 qualified houses and still have not obtained clotheir

Question: Not a Terraria genius, so sorry if these NPCS can't be obtained this way.
Mostly explained from title, I have four qualified  houses, and I defeated  Skeletron, and still have not received clothier or dryad. Again, sorry if I'm wrong.
Comment: Are the four valid houses *unoccupied*?
Comment: NPCs don't *immediately* come to town, you should learn to be a little more patient.
Comment: If I helped you, it would be nice to upvote or press the tick beside my answer :D @Kitkat3547
Comment: New NPCs won't move in until the morning - if it's still the night that you defeated Skeletron they won't move in yet
Comment: @JonK I have waited till morning and they have not moved in. I have now been waiting a day and still no NPC has moved in.
Comment: @Chippies yes. I have just built them.
Comment: Are your houses meeting the housing criteria? Have you made sure the housing is valid?
Comment: Yess. @user1337
Comment: The houses qualify as houses, I have four of em, and have defeated Skeletron, and have not received dryad or clothier.
Comment: Dryad just arrived.
Comment: One thing strikes me odd. The question asks "why hasn't X happened" and lists met criteria while the answers explains the criteria that are already met. Based on the comments, the proper answer would pretty much be "NPCs don't show up immediatly".
Comment: @Anton I have been playing for a few days now, almost playing the whole day.
Comment: Are the houses not 50 tiles or more close to corrution, have a table, chair, light, and player placed wall? If so, I have no idea. I hope this helps.
Answer: According to the wiki, these are the NPCs that will occupy your house a certian event.
Clothier
He is the Old Man guarding the Dungeon. The one that summmons Skeletron when activated. If you have a vacant house, the Clothier will move in.
Dryad
If any bosses are defeated in that world, the Dryad will occupy your house if it is vacant.
Comment: @ExplosionsPlus Probably because you blatantly quoted the wiki as opposed to telling the guy why it doesnt work in **his** case regarding **his** circumstances.
Comment: why is my answer downvoted?
Comment: No idea @ExplosionsPlus


Down Arrow Key is not working

Question: My Down Arrow Key in the keyboard is not working. Is there any way to map its functionality into another key in Ubuntu. How can I solve this problem without replacing my Keyboard?
Answer: Ubuntu has an Accessibility suite called Universal Access, of which there are keyboard controls.  Unfortunately, no custom mapping tools.
Another tool I found was Onboard which presents a keyboard onto your screen.  If you have a big screen, this may be the fastest solution.
Third, there may be some possibility with Keyboard tool by creating a Custom Shortcut.  Reading over the options, it does not appear there is an actual key remapping mechanism but something to explore.
Best solution was discovered on StackExchange.
Checkout this thread.  There is a command which you can use to redefine keys called xmodmap.
How can I remap keyboard keys?
Hope this helps.

Down Arrow Key is not working


can't find /etc/hostconfig file on OS X 10.10.3

Question: I'm trying to uninstall MySQL on OS X 10.10.3 and I found these commands to follow:
sudo rm /usr/local/mysql
sudo rm -rf /usr/local/mysql*
sudo rm -rf /Library/StartupItems/MySQLCOM
sudo rm -rf /Library/PreferencePanes/My*
edit /etc/hostconfig and remove the line MYSQLCOM=-YES-
rm -rf ~/Library/PreferencePanes/My*
sudo rm -rf /Library/Receipts/mysql*
sudo rm -rf /Library/Receipts/MySQL*
sudo rm -rf /private/var/db/receipts/*mysql*

It seems that /etc/hostconfig file doesn't exist.
Did I miss anything? What does this file do and what should I do to ensure MySQL is completely removed?
Comment: What does this have to do with [MySQL-Python](http://mysql-python.sourceforge.net/)?
Answer: As of OS X Yosemite, /etc/hostconfig is no longer installed as part of the OS.
Comment: Why? Replacement?
Comment: @powder366 as far as I can tell the replacement is `ect/hosts`


Reflection C# Activator returning a null value

Question: Please I don't know what I am doing wrong, when I run this code, I get an exception: Value cannot be null.... When I run it in debug mode, I see that "calculatorInstance" variable is null. Please help me out.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Reflection;

namespace ReflectionWithLateBinding
{
    public class Program
    {
        static void Main()
        {
            //load the current executing assembly
            Assembly executingAssembly = Assembly.GetExecutingAssembly();

            //load and instantiate the class dynamically at runtime - "Calculator class"
            Type calculatorType = executingAssembly.GetType("ReflectionWithLateBinding.Calculator");

            //Create an instance of the type --"Calculator class"
            object calculatorInstance = Activator.CreateInstance(calculatorType);

            //Get the info of the method to be executed in the class
            MethodInfo sumArrayMethod = calculatorType.GetMethod("SumNumbers");

            object[] arrayParams = new object[2];

            arrayParams[0] = 5;
            arrayParams[1] = 8;
            int sum;
            sum = (int)sumArrayMethod.Invoke(calculatorInstance, arrayParams);

            Console.WriteLine("Sum = {0}",  sum);

            Console.ReadLine();

        }

        public class Calculator
        {
            public int SumNumbers(int input1, int input2)
            {
                return input1 + input2;
            }
        }
    }
}

Comment: @BrianPayne: I'm pretty sure that `calculatorType` *is* null, for the reasons given in my answer :)
Comment: Have you made sure that `calculatorType` is not null?
Answer: I'm pretty sure it's actually the GetType method which is returning null - because there's no type with the fully-qualified name ReflectionWithLateBinding.Calculator. Your Calculator class is nested within your Program class.
It's the call to Activator.CreateInstance which is throwing an exception, therefore the assignment to calculatorInstance is never made - it's not that the variable has a value of null, but rather that its declaration statement (which includes the initializer) never completes.
Options (don't do both!):

Move the class so that it's not within the Program class (i.e. so it's declared directly in the namespace)
Change your GetType call to executingAssembly.GetType("ReflectionWithLateBinding.Program+Calculator")
Comment: thanks alot, I moved Calculator class to a new class --outside Program class and now it works fine.


VB program, looping for iterations and excel writing

Question: I'm writing a program for an internship and need some advice. I've done my research but have mostly returned fruitless... I need to loop the "buttonOneClick for one second iterations. The program will send a "P" character, wait one second, send a p, wait one second, etc... Also I need to write the information it receives to an excel spreadsheet. Any help/critiquing of existing code would be greatly appreciated.
Here's what I have:
Public Class Form2

Dim buttonOnePush As Boolean = False
Dim buttonTwoPush As Boolean = False

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
    ' Send strings to a serial port.

    Using com5 As IO.Ports.SerialPort =
            My.Computer.Ports.OpenSerialPort("COM5")
        com5.WriteLine("P")

    End Using

End Sub

Function ReceiveSerialData() As String
    ' Receive strings from a serial port.
    Dim returnStr As String = ""

    Dim com5 As IO.Ports.SerialPort = Nothing
    Try
        com5 = My.Computer.Ports.OpenSerialPort("COM5")
        com5.ReadTimeout = 10000
        Do
            Dim Incoming As String = com5.ReadLine()
            If Incoming Is Nothing Then
                Exit Do
            Else
                returnStr &= Incoming & vbCrLf
            End If
        Loop
    Catch ex As TimeoutException
        returnStr = "Error: Serial Port read timed out."
    Finally
        If com5 IsNot Nothing Then com5.Close()
    End Try

    Return returnStr
End Function

Private Sub TextBox1_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox1.TextChanged
    If IsNumeric(TextBox1.Text) AndAlso IsNumeric(TextBox2.Text) Then
        TextBox1.Text = CDec(TextBox2.Text)
    End If
End Sub

Private Sub TextBox6_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox6.TextChanged
    If IsNumeric(TextBox6.Text) AndAlso IsNumeric(TextBox3.Text) Then
        TextBox6.Text = CDec(TextBox3.Text)
    End If
End Sub

Private Sub TextBox7_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox7.TextChanged
    If IsNumeric(TextBox7.Text) AndAlso IsNumeric(TextBox4.Text) Then
        TextBox7.Text = CDec(TextBox4.Text)
    End If
End Sub

Private Sub TextBox8_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox8.TextChanged
    If IsNumeric(TextBox8.Text) AndAlso IsNumeric(TextBox5.Text) Then
        TextBox8.Text = CDec(TextBox5.Text)
    End If
End Sub

Private Sub TextBox15_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox15.TextChanged
    If IsNumeric(TextBox15.Text) AndAlso IsNumeric(TextBox16.Text) Then
        TextBox15.Text = Hex(TextBox16.Text)
    End If
End Sub

Private Sub TextBox14_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox14.TextChanged
    If IsNumeric(TextBox14.Text) AndAlso IsNumeric(TextBox11.Text) Then
        TextBox14.Text = Hex(TextBox11.Text)
    End If
End Sub

Private Sub TextBox13_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox13.TextChanged
    If IsNumeric(TextBox13.Text) AndAlso IsNumeric(TextBox10.Text) Then
        TextBox13.Text = Hex(TextBox10.Text)
    End If
End Sub

Private Sub TextBox12_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles TextBox12.TextChanged
    If IsNumeric(TextBox12.Text) AndAlso IsNumeric(TextBox9.Text) Then
        TextBox12.Text = Hex(TextBox9.Text)
    End If
End Sub

Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
    buttonTwoPush = True
    buttonOnePush = False

    Me.Close()
    Form1.Close()

End Sub

End Class
Answer: Use a Timer to issue that command at one second intervals.  Drag a Timer over to your form from the toolbox, and double click on it to get a _Tick method.
Set the .Interval member of the timer in your form's constructor, and use the .Start and .Stop methods to control it.
For the Excel piece, you'll need to add a reference to the project for the Microsoft Excel 12.0 (or 14.0 if you have Excel 2010) Object Library.  Find this under the COM tab of the Add Reference dialog which you get by right clicking on the project in the Solution Explorer.  See this page for an exhaustive reference (scroll down to the bottom of the page for a quick example in VB.NET).


Learning automated trading in X3: Albion Prelude

Question: So I finally managed to learn trading in X3: Albion Prelude and equipped my Mercury with the MK3 trading software.
Since then the function "Start sector trading" is available. But when I use it, the status of the ship switches to "buying 0 energy cells" and then switches back to "standby". Why does automated trading not work? Am I doing something wrong?
Any help is highly appreciated.
Answer: Your trader isn't doing any work because there are no good trades available in the sector. Find a sector where things that are produced also get consumed (Power Plant -> Wheat Farm -> Distillery, for example), or, if you believe the sector they are in fits that criteria, leave them alone and they will try again after a while (You can also reissue the command to get them to seek new traders immediately, as they will standby for some time before attempting to find another deal).
As your trader levels up, you will be able to increase the range in which they will seek deals, and when they reach level 8, they will be able to become Universe Traders (though their range for finding deals will still be limited, they will not feel obliged to stick around the same neighbourhood, and their trading may take them to any point in the galaxy where profitsss can be made).
You will need to reissue the sector trader command to get the option to increase the number of jumps in which they will seek deals.
It may be a good idea to leave them as sector traders with a big radius even after they've leveled up, as some parts of the universe are much more dangerous than the others (The area around Argon Prime, for example, is super safe, while Omicron Lyrae is close to many pirate and Xenon sectors).
P.S. the reason your automated traders sometimes loop through wares with "Buy x ware" orders is because that's how they assess available trades.
Comment: Keep at it, and don't forget to keep an eye on their level and reissue the command as necessary. Under ideal conditions, a sector trader with jump range of 2 will have access to 9 times the number of sectors (and therefore stations and deals) a 1st level trader would.
Comment: You can see the pilot's level in the ship information menu (I think the hotkey is U when targeting the ship). Are you sure there were profitable deals to be made in the sector (e.g. power plant selling e-cells for 12 credits, factory buying e-cells for 19 credits). Does the ship have enough free cargo space? Is the freighter fully shielded and equipped with a jumpdrive? Sometimes they will enter standby while awaiting delivery of shields and jumpdrive, though that should only happen with UTs, not STs.
Comment: Thanks, I will give that a try. I will just try out a various number of sectors. Hopefully it will work at some point...
Comment: @Valentin I recommend starting out your sector traders in empire's edge.  Its got almost a complete loop there and is perfect to train them up to become universe traders.  I wouldn't bother reissuing the order until they hit level 8 (when they turn into universe traders).  Empire's edge is the perfect training ground.
Comment: @kotekzot its level 8 to be universe traders
Comment: @yx. cheers, updated.
Comment: Thanks again for helping me with the game! I highly appreciate it. No matter where I send my ship to, in each sector it says "standby" instead of "trading" or something like that. I guess the pilot is still at minimum level (where do I see this level?) and I don't know what am I doing wrong. Should I start a new question? :)
Comment: +1 for the tip on reissuing the orders. I was wondering why my level 8 sector trader wasn't going further afield.


Disabling the checkbox in angular directive template

Question: I have the directive shown below that displays a checkbox in cells of a table:
app.directive('csSelect', function () {
    return {
        template: '<input ng-model="checked" ng-click="csFunc()" disabled="isDisabled" type="checkbox"/>',
        scope: {
            row: '=csSelect',
            csFunc: '&csFunc',
            isDisabled: '=csDisabled'
        },
        link: function (scope, element, attr, ctrl) {

            scope.$watch('checked', function (newValue, oldValue) {
                if (newValue === true) {
                    element.parent().addClass('st-selected');
                } else {
                    element.parent().removeClass('st-selected');
                }
            });
        }
    };
});

The problem is that the disabled attribute is not working for some reason. Whatever value I provide, the checkbox is always disabled. I have tried using @csDisabled as well but no success.
Any ideas on what I should do get the checkbox disabled whenever I want ?
Answer: As you are trying to assign scope value disabled attribute, for that you need to use ng-disabled directive instead of using only disabled.
template: '<input ng-model="checked" ng-click="csFunc()" ng-disabled="isDisabled" type="checkbox"/>',

Comment: Alright finally managed to get it working. Thank you so much for your time !
Comment: @Cemre Glad to help you..Thanks :)


Solving the indefinite integral of a trig function

Question: I'd like to ask for some feedback on my calculation.  Please let me know if you spot any mistakes in my technique:
$$\int{\frac{1}{\sqrt{x}}\sin^{-1}{\sqrt{x}}}\,\,dx$$
Using substitution:
$$u = \sqrt{x},\,\,\frac{du}{dx}=\frac{1}{2\sqrt{x}},\,\,du=\frac{1}{2\sqrt{x}}dx$$
So
$$2\int{\sin^{-1}{u}}\,\,du = u\sin^{-1}{u}+\sqrt{(1-u^2)}+c$$
$$=2\sqrt{x}\sin^{-1}{\sqrt{x}}+2\sqrt{1-x}+c$$
I'd greatly appreciate anyone's input.  Thank you!
Comment: Did you use IBP in the "before last" step?
Comment: Hi @PrasunBiswas, no we were 'taught' the identity of sin^-1x.  I suppose it would be more illustrative to show the steps, though?
Answer: Instead of memorizing some formula, you can use IBP to conclude your answer. That comes more intuitively to someone in my opinion.
Take $\arcsin(u)$ as the first function and $1$ as the second function. Now, using IBP,
$$I=\int\arcsin(u)\,\mathrm du=\left(\arcsin(u)\int\,\mathrm du\right)-\int\left(\frac{\mathrm d}{\mathrm dx}\left(\arcsin(u)\right)\int\,\mathrm du\right)\,\mathrm du\\ = u\arcsin(u)-\int\frac{u}{\sqrt{1-u^2}}$$
Now, make the substitution $1-u^2=t$ and $(-2u)\,\mathrm du = \mathrm dt$ to get,
$$I=u\arcsin(u)+\int\frac{\,\mathrm\,dt}{\sqrt{t}}=u\arcsin(u)+2\sqrt{t}+C$$
where $C$ is the constant of integration.
Now, completely rewrite $I$ in terms of $u$ to get the "identity" you were taught and then you proceed as you did in your own solution.
Comment: **Note:** $\arcsin(u)=\sin^{-1}u$
Comment: Thanks for that, I think it's probably more elegant to use IBP as it does show the entire process!
Comment: @ptikobj, Yes, also it shows how you got the "identity". Memorizing such results doesn't do good to anyone when you can calculate it yourself quite efficiently using elementary substitutions.
Answer: Your calculation and technique are correct. Checking with Wolfram Alpha, we get the same answer. You could integrate $sin^{-1}(x)$ by parts, but if you were taught it as an identity, then your technique works.
Comment: This seems more like a comment than an answer.
Comment: I don't have enough reputation to comment yet unfortunately
Comment: Ah, I see. No worries, bro! :)


Showing transitivity of a modulus

Question: Relation $V$ on $\mathbb{Z}$ is defined by setting $xVy$ iff $|x^2-y^2|\le2$. How do I show that this is transitive?
Comment: Transitivity needs to be proven in very few cases
Answer: Note that $x^2 - y^2 = (x+y)(x-y) $ and since both $x $ and $y $ are integers, so are $x+y $ and $x-y $. Now $xVy$ iff one of three things happen:
$(x-y) = 0$ or $(x+y) = 0$ or ($|x| \le 2$ AND $|y| \le 2$).
$(x-y) = 0 \iff x=y \rightarrow |(x^2 - y^2)| = 0 \le 2$;
Note that we get the same when $x = -y $;
(Note that both these cases imply $x^2 = y^2$)
Now how can we get $|(x+y)(x-y)| \le 2$ when $x \ne \pm y$?
$$|(x+y)(x-y)| \le 2 \iff |(x+y)||(x-y)| \le 2 \iff |(x+y)| \le \frac{2}{|(x-y)|}$$
If $|(x-y)| > 2 $ this is rendered impossible because the only integer less than 1 (in modulo) is 0, and we already excluded that possibility. So $|(x-y)| \le 2$. If we divide both sides by $|(x+y)|$ instead, we get the analogue result: $|(x+y)| \le 2$. Both conditions are only met when $x$ is at a distance of 2 or less of both $y$ and $-y$.
How far apart can $y$ and $-y$ be on the integer line so that my statement is true? If you take a pen and a piece of paper you quickly realise that $|y|$ must be $\le 2$ and that the same thing goes for $x$.
Now transitivity can be proven easily:
We assume that $xVy $ and that $yVz $. To prove $xVz $ just consider the three cases above (that are just essentially two) for $x $ and $y $.
First assume $x = \pm y$ and then assume that both $|x|$ and $|y|$ are $\le2$.
Answer: Note that $(a+1)^2-a^2=2a+1$. The difference between two consecutive squares is 1 only when it is $0$ and $1$. Otherwise it is at least 3. If the squares are not of consecutive numbers then the difference will be much higher.
So the condition $|x^2-y^2|\leq 2$ actually means $x=0,y\pm1$; (or the other way), the set has just these 3 elements. So any checking has to be with these 3 numbers which holds; so transitivity follows.

Showing transitivity of a modulus


Looping a carousel made with translate3d

Question: I have a carousel that has drag option made with translate3d. Taking into consideration that is use translate3d to go to the next slide, i end up translating to the end of the carousel. Can you help me finding a solution?
Next slide logic:
    goNext() {
    this.carousel.style.transform = `translate3d(${-(++this.elementIndex * this.album.clientWidth)}px,0,0);
    }

When i get to the end of the carousel:
I know that i can translate to 0 at the end but i want on the last slide to keep the drag option and to be able to drag to the first slide looking like is the next. Any ideas?
Answer: The trick is:

place a clone of last image to the beginnig of carousel and first image to the end of carousel (during initialization).
5 [1 2 3 4 5] 1
when user clicks "next" at the last slide you should switch it to the very first slide (cloned "5") without animation, and then you can transform it to "1" with animation
1)  5  1  2  3  4 [5] 1
2) [5] 1  2  3  4  5  1
3)  5 [1] 2  3  4  5  1


error: invalid operands of types 'double()' and 'double()' to binary 'operator+'

Question: Getting the error

"error: invalid operands of types 'double()' and 'double()' to binary
  'operator+'
         + getActualTitheOfferings + getActualLivingExpenses."

when attempting to compile.
Failing to understand what the problem is, including a small segment of code below but can add more if needed to help solve the problem.
Any help is appreciated!
<code>/**********************************************************************
* Get the difference
**********************************************************************/
double actualDifference()
{
    double actualDifference = getIncome - (getActualTaxesWithheld
       + getActualTitheOfferings + getActualLivingExpenses
                                           + getActualOtherExpenses);
}
/**********************************************************************
* Display all the variables
**********************************************************************/
int getDisplay()
{
    double income = getIncome();
    double budgetedLivingExpenses = getBudgetedLiving();
    double actualLivingExpenses = getActualLivingExpenses();
    double actualTaxesWithheld = getActualTaxesWithheld();
    double actualTitheOfferings = getActualTitheOfferings();
    double actualOtherExpenses = getActualOtherExpenses();
    double actualDifference;
    double zero = 0;
</code>
Comment: Looks like you are confusing a function name with a function call.  For example `getIncome` is an identifier which refers to the function (and I think technically, for portability, one should take the address of it `&getIncome` rather than rely on some sort of conversion to the address).  But `getIncome()` is a function call to that function, and will use the result of the function in the context of the call.
Comment: Your `actualDifference()` function does nothing. It creates a local variable and then forgets about it. Instead of `double actualDifference = ...` its body should be `return ...`.
Answer: Your code attempts to add together two functions. What you means to do was to call the functions and add their results.
The error message tells you this. It says that operator+ does not expect the types 'double()' and 'double()'. 'double()' means a function that takes no parameters and returns a double.
The corrected code would be:
<code>double actualDifference = getIncome() - (getActualTaxesWithheld()
       + getActualTitheOfferings() + getActualLivingExpenses()
                                           + getActualOtherExpenses());
</code>
Comment: I edited the code to what you suggested above and now get the error "error: invalid operands of types 'double()' and 'double' to binary 'operator-'
                                           + getActualOtherExpenses());"                                                The only change in the error I'm seeing is a (-) where there was a (+) after 'operator'
Comment: Right, getIncome() is a function call as well. Edited that in.
Comment: More importantly, is the explanation understandable? You understand that adding together the functions doesn't make sense, you need to call them and then add the results?

error: invalid operands of types ‘double()’ and ‘double()’ to binary ‘operator+’


java.lang.ClassNotFoundException in Maven project, while trying to load listener from web.xml

Question: I have converted existing web project into Maven, technologies used are : Struts 2, Hibernate. A listener extends  javax.servlet.ServletContextListener, which has a entry in web.xml as follows:
<listener>
    <listener-class>com.tcs.weserv.util.Listener</listener-class>
  </listener>

Maven project is built successfully, but when I try to start the server, it throws an exception:
java.lang.ClassNotFoundException: package.util.Listener

All dependencies are defined properly in pom.xml, listener is available at given location, still it throws ClassNotFoundException.
Comment: Post a stacktrace()
Comment: java.lang.ClassNotFoundException: com.tcs.weserv.util.Listener
 at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1714)
 at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
 at org.apache.catalina.core.DefaultInstanceManager.loadClass(DefaultInstanceManager.java:532)
Comment: Describe the steps you followed to "convert" your project. Please add the `...</build` fragment of your pom.xml file to your question
Comment: I have created a new maven web project, and then paste all java src to main/resources including struts.xml and hibernate.xml file. copied all webContent into webApp folder.  Here is the build in pom.xml
Workbench
Comment: To debug, I have created a demo project in maven , which will call the same listener(which is used to create hibernate session), at first time, it was giving the same ClassNotFoundException, but when I cleaned the project , It worked fine. same is not applicable in my actual project.
Comment: I am facing following error while starting the server : SEVERE: Error configuring application listener of class, followed by ClassNotFoundException
Answer: Thanks all for your valuable comments. It helped to debug the problem. The actual problem is, Maven Project structure was wrong. When I created a Maven project by selecting archtype as webapp, the project structure contains only src/main/resources.
Since I am new to Maven, I thought this is actual project structure, and I copied all resources as well as java files into this folder. As a result, It was able to build the project without any error, but while deploying it on server, It is showing ClassNotFoundException.
Comment: You can formally approve your answer to make evident that it solves the problem.
Answer: This is uknown listener. Cannot find any jar containing that listener. If you know how to create jar using some zip utility or jar executable then you can copy the class com.tcs.weserv.util.Listener.class to the jar file preserving the folder structure which is important to find the corresponding class.
Comment: this listener is defined in my project itself. n before converting the project to maven , it was working fine.
Comment: If it is in the project, why it cant find it , n says ClassNotFoundExceprtion.
Comment: the file should be on classpath and the server should be able to load it
Comment: see.. its a simple class which implements ServletContextListener. n It is in the project only. If i build the project , it should automatically in the war file. so.. while starting server, it is available on the specified path. I could not understand what do u mean by "file should be on classpath" ?
Comment: this is the thing that you don't understand. The class couldn't be loaded if it's not available. Verify your package or whatever you use to deploy your application to have consistent packaging that requires to build and start the web application.
Comment: Hi @Roman, I have cross checked war file, and it contains the class.   30217 Thu May 25 18:09:48 IST 2017 WEBINF/classes/com/tcs/weserv/util/Listener.java
Comment: In the exception message you say that it is `package.util.Listener`
Comment: last time, i haven't explained it in details, here package mean to com/tcs/weserv. The actual location of the listener is com.tcs.weserv.util
Comment: I didn't see where you explained it in details? You shouldn't substitute real values with imaginable. Nobody can understand this. I recommend you to check the `lib` folder to have the same jars as in working project. Maven is downloading jars and replaces them in the `lib` folder. You might have inconsistent project dependencies that failed on the server loading your class. See https://stackoverflow.com/q/34413/573032


Underlying space of fiber product of schemes

Question: Let $X,Y$ be schemes over a field $k$. I know that $X \times_k Y$ is not equal to the catesian product $X \times Y$. For example $\mathbb{A}_{k}^2 \neq \mathbb{A}_{k} \times \mathbb{A}_{k}$.
So, I don't understand example II.4.0.1 in Hartshorne's book.
This example explain that if $X$ is the affine line with double origin points, then
$X\times_k X$ is the affine plane with double axes and four origin points...
in this case, why $X\times_k X= X\times X$? and my additional question is that in general case, is there condition that $X \times_S Y = X\times Y$?, where $S$ is an arbitrary scheme?
Comment: See also this similar question: http://math.stackexchange.com/questions/322188/how-to-compute-the-topological-space-of-fibered-product-of-schemes/322256#322256
Answer: In this example, it is still not true that $X\times_k X = X\times X$ as topological spaces, since for example on the complement of the origin(s), the two are not equal. This follows for the same reason as your previous example: the Zariski topology on $\mathbb A^2$ is not equal to the product topology on $\mathbb A^1\times\mathbb A^1$.
What is true in both cases is that there is a natural correspondence between closed points of the fibre product and products of closed points. This follows because we know that maximal ideals of $K[x,y]$ have the form $(x-a,y-b)$ for $a,b\in K.$
Comment: thanks Andrew. then, hartshorne book is wrong?
Comment: Dear @SangCheolLee, no, Hartshorne is not wrong. You might prefer to look at Shafarevich Basic Algebraic Geometry 2, p. 44-5, which goes into a bit more detail on this example. Cheers


Find $\lim_{n\to\infty} I_n$ where $I_n=\int_{0}^{1} \frac{f(x)}{1+nx}dx$.

Question: Let $f$ be an integrable function on $[0,1]$.
Find  $\lim_{n\to\infty} I_n$ where $I_n=\int_{0}^{1} \frac{f(x)}{1+nx}dx$.
I have tried it over some specific functions and I think that  $\lim_{n\to\infty} I_n=0$.
I think also that we can use that the function is bounded since it is integrable.
I can use a hint (not a solution).
Note: I have just learned definite integrals till now.
Thanks in advance!
Comment: Hint: $|I_n|\leq\int_0^1\frac{|f(x)|}{1+nx} dx\leq \int_0^1 \frac{M}{1+nx} dx$ where $M$ is some bound of $|f|$.
Comment: $f$ is supposed to be continuous ?
Comment: No. It is not written in the question.
Answer: As you have noted, a Riemann integrable function is always bounded. Call this bound $M$. You can thus use $M$ to find an upper bound for $|I_n|$, and then use the squeeze theorem.
Answer: Hint: Lebesgue dominated convergence theorem.
Comment: This is the shortest solution, but I'm not sure OP is familiar with Lebesgue integration.


Are there certain colors we should use when highlighting selected items in a GUI?

Question: One part of my application allows users to select items in order to manipulate the items, and I want to put a border around the items with a color to show when it's selected.
Normally the users will select one item (i.e., delete item) or two items (i.e., move item one after item two) only.
I was thinking that one color would represent the first selected item, and a second color would represent the second item, so as to not confuse the two
items selected  This is especially important in the case of moving one item to the position after the another item, where it's important to know which item is moving after the other.
My questions are.

Should each item have their own selected color, or should the same color be used for both items?
Are there certain colors that are preferred, over other colors to use when selecting items?  i.e., green over red.

NOTE:  I wanted to mention that I'm not exactly highlighting the entire item, but merely adding a colored border around the item, so the items will still look the same, besides a 2px or so border around the item.
Comment: @Izhaki so what do you recommend in that case?
Answer: Use same color for selection that will reduce cognitive load unless you want to show a queue of selected items based on selected time. If your intention is to show a queue then choose the color that suits your application theme, but don't mix it with multiple colors - you can use shades of one color such as; dark for the first and little faint for the second and so on.
Don't  just rely on colors - think about color-blind users, they won't see any change if you only change colors.
Along with color change also consider changing the dimensions of the selected items - like making the selected items a little larger than the rest.
Comment: Good answer, color based on brand and shades for order. It is more important that the color creates enough contrast with the other element's color, than the color itself. The color could be in the background primarily but possibly in the border of the element or the text-color.
Comment: Thanks for the tips.  There is no queue, just 1 or 2 items being selected, but would like to show which is the first and second selection, even though the users should know which one they selected first or second.  The items will get a border colored around them, not the entire item being highlighted.  I should make that clear in the OP.  I like the dimension idea though, that might work as well.  Does the shade matter if it's the first or second one that's darker?  Is there an order that is usually followed i.e., darker first, then lighter?  Thanks.
Answer: Choice of colors depends on nature / industry of your application.
But pale yellow for first selection and light blue for other will be good in your case.
Comment: How did you get to that conclusion?