Laravel uploaded file not appears in linked "public/storage" folder

Question: When I run command: php artisan storage:link, this creates folder /public/storage.
then I have code, which handles uploaded file:
// get file original name and extension here, then generate file new name $fileNameToStore
// set file path
$path = $request->file('my_file')->storeAs('public/uploaded_imgs', $fileNameToStore);

Code works and uploaded file appears in /storage/app/public/uploaded_imgs/ folder, which is nice, though there is nothing in    /public/storage folder.
Why there is not uploaded_imgs folder in /public/storage directory?  What I'm doing wrong?
Comment: are you running this in a VM by any chance?
Comment: how did you create a /public/storage dir?
Comment: @mikrafizik - Laravel comes with `/public` folder, then wen I run `php artisan storage:link`, this creates `storage` folder into `/public`
Comment: @lagbox - No, this is not VM, (BTW I'm on ubuntu 16.04)
Comment: I think it might depends on your file expolerer and way of processing with sym links - in some caseses I've noticed results simply cache.
Comment: 1) try ls -la    and look where symlink goes, is it right way?
2) try to change CHOD to storage folder
3) try to remove symlink, and recreate it manually 'ln -s /path/to/laravel/storage/app/public /path/to/public/storage'
Answer: In config/filesystems.php, you could do this... change the root element in public
Note : instead of upload you can use your folder name
'disks' => [
   'public' => [
       'driver' => 'local',
       'root'   => public_path() . '/uploads',
       'url' => env('APP_URL').'/public',
       'visibility' => 'public',
    ]
]

and you can access it by
Storage::disk('public')->put('uploaded_imgs', $request->file('my_file'));

or
'disks' => [
    'local' => [
        'driver' => 'local',
        'root'   => storage_path(),
    ],
    'uploads' => [
        'driver' => 'local',
        'root'   => public_path() . '/uploads',
    ],
]

Then use it :
Storage::disk('uploads')->put('filename', $file_content);

Answer: Try this:
Storage::disk('public')->put('uploaded_imgs', $request->file('my_file'));

hopefully it will work.
Comment: they are uploading the file without issue and to a path that is what the `public` disk covers, assuming the default configuration for the `public `disk
Answer: When you run php artisan storage:link command, Laravel generates a "storage" symlink under "public" folder which directs to /storage/app/public so your code is correct.
There is no public/storage directory, its a symlink which directs to /storage/app/public which is where your uploaded_imgs folder has been generated.
public/storage => /storage/app/public
Answer: You can use the following code to upload a file:
$file = $request->file('my_file');
$fileNameWithoutExtension = pathinfo($file->getClientOriginalName(), PATHINFO_FILENAME);

$path = "public/uploaded_imgs/" + $fileNameWithoutExtension +"."+$file->getClientOriginalExtension();

Storage::disk('local')->put($path, file_get_contents($file), 'public');

Comment: they are uploading the file without issue and to a path that is what the `public` disk covers, assuming the default configuration for the `public `disk


weird URL deformation when changing src of : & -> &amp

Question: edit: It was Frontends fault. They made a copy of the img (for some mobile-version bs) including the id-attribute (facepalm) and didn't mentioned it. Thus my js worked correctly, but on the wrong element, which is hidden on desktop.
Sorry.
Original post:
At some point in my code, I want to set the "src" attributes of several pictures in a grid.
I have a const array of URLs
const pictureURLs = [
    'https://firebasestorage.googleapis.com/projectname/o/main%2Fgrid%2Fthumbnail%2Fstudents.webp?alt=media&token=12345678',
    'https...'
];

and a simple forEach() loop
pictureURLs.forEach((pic, i) => {
    // let img = $(`#picture-grid-${i}`)[0];
    let img = document.getElementById(`picture-grid-${i}`);
    img.src = pic;
    img.onclick = () => {
        // some code..
    }
})

For some reasons this method works for hundreds of images, except one.
When I log the element "img", every occurrence of the URL is correctly saved in several Nodes, like "src" and "currentSrc". But only at "outerHTML" its src attribute is changed from
"..media&token.."
to
"..media&token"
All other URLs are fine in all other pictures. It's just this one.
I tried to reupload, assign new tokens and even changed from jquery to vanilla js. But it's only this one picture.
Can somebody help me to understand where this & -> &amp conversion happens? I've only found traces to jquery, but as I already wrote, I ruled this out.
Tested in Firefox and Chromium, both on Linux.
Comment: Which console are you looking at?
Answer: The conversion happens because in HTML an ampersand (&) is used for HTML entities, so to have a literal & you have to escape it.
Comment: Thanks, but this doesn't make sense. Why are all other URLs working fine with exact the same symbols, but this particular one not? Do I have to check all URLs and escape the ones, that doesn't work, or all of them? As I said, it's a long list of URLs..
Comment: All of the URLs should have their ampersands in their element's outerHTML escaped. If only one URL is not working perhaps you have mistyped the URL or there is a server-side issue.


selecting delimiter in regex while selecting portion between lines in case of perl

Question: The message has detection report:
<detection_report>
Test 1
Test 2
Test 3
</detection_report>

---------------------------------------------
Have a nice day

I want to select portion between <detection_report> tags, including these two tags.
I have written following code.
The message has detection report\:((.|\n|\r)+)(\<\/detection_report\>)

but its not working. Can anyone help me with this.
Comment: That is not code, it looks like a regex. You should show the code that uses the regex.
Comment: Also, you should be more specific than "its not working".
Answer: Using a perl one-liner
perl -ane 'print $1 // $_ if m{(detection_report.*)} .. m{(.*?</detection_report>)}' file.txt

This is close to what M42 suggested, but it takes into account the possibility that the tags do not exist on their own lines.
Note, if this data is either HTML or XML, then you should use an actual HTML or XML Parser for extracting this data.
Answer: Here is a way to do the job:
perl -ane 'print if /<detection_report>/ .. /<\/detection_report>/' in.txt

output:
<detection_report>
Test 1
Test 2
Test 3
</detection_report>

Answer: There you go:
$ cat detection.txt
The message has detection report:

<detection_report>
Test 1
Test 2
Test 3
</detection_report>

---------------------------------------------
Have a nice day$ cat regex.pl
#!/usr/bin/perl

while(<>){
    $input.=$_
}

($body) = $input =~ m/(<.*>)/ms;

print $body
$ cat detection.txt  | perl regex.pl
<detection_report>
Test 1
Test 2
Test 3
</detection_report>$


Lambda no Python - Dúvida

Question: Em estudo sobre python 3, qual seria a função do lambda no Python, para que ele serve e quando é utilizado?
Comment: Relacionada: [O que são lambda expressions? E qual a sacada em usá-las?](https://pt.stackoverflow.com/q/2822/5878)
Answer: Lambda são funções de uma linha (geralmente, chamadas uma única vez). São conhecidas como funções anonimas em algumas linguagens. Vc pode querer utilizar lambdas quando vc não precisa usar uma função por mais de uma vez em um programa. Elas funcionam como uma função regular.
Sintaxe:
lambda arguments: manipulate(arguments)

Exemplo:
# Definindo uma função regular
deff add(x , y):
    return x+y

# A mesma função com lambda
add = lambda x, y: x+y

Para chama-la, use a mesma sintaxe para a chamada da função regular, por exemplo:
print(add(3, 7))
# output: 10

No exemplo acima, a função foi criada e atribuida a uma variável, nesse caso, poderia-se utiliza-la mais de uma vez, mas na maioria das vezes lambdas são executadas "inline", descartadas após o uso, como nos exemplos de map() e filter(), em seguida.
Funcoes lambdas são geralmente utilizadas em python como argumento para funcoes "de ordem superior" (funções que recebem outras funcoes como parte dos argumentos), bastante utilizadas como funções internas de filter() e  map().
Exemplo utilizando filter()
A funçõa filter() em python recebe em seus argumento um função e uma lista e retorna uma nova lista contendo os itens que a função avalia como True.
mylist = [1, 2, 3, 4, 5, 6]
newlist = list(filter(lambda x: (x%2 == 0) , mylist))

print(newlist)
[2, 4, 6]

Exemplo utilizando map()
A função map recebe uma lista e uma função e retorna uma nova lista na qual os elementos são resultantes da aplicação da função enviada em cada elemento da lista recebida.
mylist = [1, 2, 3, 4, 5, 6]
newlist = list(map(lambda x: x*2, mylist))

print(newlist)
[2, 4, 6, 8, 10, 12]

List Comprehensions
Sempre que sentir a necessidade de usar lambda em que o retorno seja uma lista, veja se não é possível usar as Lists comprehensions, são muito mais elegantes, intuitivas e pythonicas, veja os mesmos exemplos com elas:
# List comprehension like a filter()
mylist = [1, 2, 3, 4, 5, 6]
newlist1 = [n for n in mylist if (n%2==0)]
print(newlist1)
[2, 4, 6]

# List comprehension like a map()
newlist2 = [n*2 for n in mylist1]
print(newlist2)
[2, 4, 6, 8, 10, 12]

Answer: Lambda é uma função definida no runtime (tempo de execução) do programa. Ele pode ser utilizado para simplificar algumas operações, mas mais que isso lambdas trazem uma semântica funcional a linguagem, na qual você trabalha com objetos e funções, um exemplo de utilização do lambda seria pegar um vetor de números e multiplica-los por 2.
Com o lambda não é necessário realizar um loop explícito:
items = [1, 2, 3, 4, 5]
squared = []

for i in items:
    squared.append(i**2)

print ("squared =", squared)
vc pode usar a função map

def square(x):
    return x**2
items = [1, 2, 3, 4, 5]
squared = list(map(square, items))
print ("squared =", squared)
ou vc pode criar um lambda diretamente

items = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x**2, items))
print ("squared =", squared)

Comment: >Lambda é uma função definida no runtime (tempo de execução) do programa - o que isso significa exatamente? Não é o caso pra todas as funções?

Lambda no Python - Dúvida


Query optimization with number of tables join and order by with limit clause

Question: I have a query which has multiple tables joined using distincct - left join - order by - limit clause.
The query looks like this:-
Select DISTINCT a.col1, b.col2, c.col3, d.col4, e.col5, f.col6, g.col7, h.col8,
i.col9, j.col10
From test_a a
left join test_b b on a.col1 = b.col2
left join test_c c on c.col1 = d.col2
left join test_d d on d.col1 = c.col2
left join test_e e on e.col1 = d.col2
left join test_f f on f.col1 = e.col2
left join test_g g on g.col1 = f.col2
left join test_h h on h.col1 = a.col1
left join test_i i on i.col1 = f.col2
left join test_j j on j.col1 = i.col2
Where a.col2 = 'Y'
and c.col4  = 1
Order by h.col5 desc
limit 50;

All the column used the in coditions has index on it. And explan output of this query gives resultset where I can see it uses all the index properly and total rows it scanned from all the tables is 18000.
What I am wondering in this query is. It runs within seconds if I run it without order by clause. Something like:
Select DISTINCT a.col1, b.col2, c.col3, d.col4, e.col5, f.col6, g.col7, h.col8,
i.col9, j.col10
From test_a a
left join test_b b on a.col1 = b.col2
left join test_c c on c.col1 = d.col2
left join test_d d on d.col1 = c.col2
left join test_e e on e.col1 = d.col2
left join test_f f on f.col1 = e.col2
left join test_g g on g.col1 = f.col2
left join test_h h on h.col1 = a.col1
left join test_i i on i.col1 = f.col2
left join test_j j on j.col1 = i.col2
Where a.col2 = 'Y'
and c.col4  = 1
limit 50;

And if I run it with order by clause then it takes 30-40 seconds to execute.
I tried using the index hint functionality provided by mysql:- USE INDEX FOR ORDER BY (idx_h_col5), but I am getting syntax error while executing this query. The error message says incorrect syntax near
I have one composite index on the column used in order by clause. I also tried creating a single index on this column but nothing really works.
Comment: How many rows do the tables have? It would help if you provided the execution plan for the ordered query. And the tables' structure (PK, FKs) and existing indexes.
Comment: Most of the tables have million plus rows in it. I'll get the ouput of explain and put it here. Thanks
Answer: MySQL can use keys for sorting instead of sorting the result after fetching the data, but only if several conditions are met.
You can see a list of these conditions here: http://dev.mysql.com/doc/refman/5.0/en/order-by-optimization.html
In your case, I think that the multiple JOINs prevent the quick sorting. One of the mentioned cases in which MySQL can't use an index for sorting is:

You are joining many tables, and the
  columns in the ORDER BY are not all
  from the first nonconstant table that
  is used to retrieve rows. (This is the
  first table in the EXPLAIN output that
  does not have a const join type.)

I am not sure if there is a way around it. It depends on the tables structure and the actual query.
To get more help, try posting the explain output of the ordered query.
Answer: I would first try adding compound indices on:
Table a
    (col2, col1)

Table b
    (col4, col1)

and a simple index on
Table h
    (col5)


Getting different values for alpha using RidgeCV with normalize=True and using Pipeline

Question: I am trying to determine which alpha is the best in a Ridge Regression with scoring = 'neg_mean_squared_error'.
I have an array with some values for alpha ranging from 5e09 to 5e-03:
array([5.00000000e+09, 3.78231664e+09, 2.86118383e+09, 2.16438064e+09,
       1.63727458e+09, 1.23853818e+09, 9.36908711e+08, 7.08737081e+08,
       5.36133611e+08, 4.05565415e+08, 3.06795364e+08, 2.32079442e+08,
       1.75559587e+08, 1.32804389e+08, 1.00461650e+08, 7.59955541e+07,
       5.74878498e+07, 4.34874501e+07, 3.28966612e+07, 2.48851178e+07,
       1.88246790e+07, 1.42401793e+07, 1.07721735e+07, 8.14875417e+06,
       6.16423370e+06, 4.66301673e+06, 3.52740116e+06, 2.66834962e+06,
       2.01850863e+06, 1.52692775e+06, 1.15506485e+06, 8.73764200e+05,
       6.60970574e+05, 5.00000000e+05, 3.78231664e+05, 2.86118383e+05,
       2.16438064e+05, 1.63727458e+05, 1.23853818e+05, 9.36908711e+04,
       7.08737081e+04, 5.36133611e+04, 4.05565415e+04, 3.06795364e+04,
       2.32079442e+04, 1.75559587e+04, 1.32804389e+04, 1.00461650e+04,
       7.59955541e+03, 5.74878498e+03, 4.34874501e+03, 3.28966612e+03,
       2.48851178e+03, 1.88246790e+03, 1.42401793e+03, 1.07721735e+03,
       8.14875417e+02, 6.16423370e+02, 4.66301673e+02, 3.52740116e+02,
       2.66834962e+02, 2.01850863e+02, 1.52692775e+02, 1.15506485e+02,
       8.73764200e+01, 6.60970574e+01, 5.00000000e+01, 3.78231664e+01,
       2.86118383e+01, 2.16438064e+01, 1.63727458e+01, 1.23853818e+01,
       9.36908711e+00, 7.08737081e+00, 5.36133611e+00, 4.05565415e+00,
       3.06795364e+00, 2.32079442e+00, 1.75559587e+00, 1.32804389e+00,
       1.00461650e+00, 7.59955541e-01, 5.74878498e-01, 4.34874501e-01,
       3.28966612e-01, 2.48851178e-01, 1.88246790e-01, 1.42401793e-01,
       1.07721735e-01, 8.14875417e-02, 6.16423370e-02, 4.66301673e-02,
       3.52740116e-02, 2.66834962e-02, 2.01850863e-02, 1.52692775e-02,
       1.15506485e-02, 8.73764200e-03, 6.60970574e-03, 5.00000000e-03])

Then, I used RidgeCV to try and determine which of these values would be best:
ridgecv = RidgeCV(alphas = alphas, scoring = 'neg_mean_squared_error',
                  normalize = True, cv=KFold(10))
ridgecv.fit(X_train, y_train)
ridgecv.alpha_

and I got ridgecv.alpha_ = 0.006609705742330144
However, I received a warning that normalize = True is deprecated and will be removed in version 1.2. The warning advised me to use Pipeline and StandardScaler instead. Then, following instructions of how to do a Pipeline, I did:
steps = [
    ('scalar', StandardScaler(with_mean=False)),
    ('model',RidgeCV(alphas=alphas, scoring = 'neg_mean_squared_error', cv=KFold(10)))
]

ridge_pipe2 = Pipeline(steps)
ridge_pipe2.fit(X_train, y_train)

y_pred = ridge_pipe.predict(X_test)

ridge_pipe2.named_steps.model.alpha_

Doing this way, I got ridge_pipe2.named_steps.model.alpha_ = 1.328043891473342
For a last check, I also used GridSearchCV as follows:
steps = [
    ('scalar', StandardScaler()),
    ('model',Ridge())
]

ridge_pipe = Pipeline(steps)
ridge_pipe.fit(X_train, y_train)

parameters = [{'model__alpha':alphas}]

grid_search = GridSearchCV(estimator = ridge_pipe,
                          param_grid = parameters,
                          scoring = 'neg_mean_squared_error',
                          cv = 10,
                          n_jobs = -1)

grid_search = grid_search.fit(X_train, y_train)
grid_search.best_estimator_.get_params

where I got grid_search.best_estimator_.get_params = 1.328043891473342 (same as the other Pipeline approach).
Therefore, my question... why normalizing my dataset with normalize=True or with StandardScaler() yields different best alpha values?
Comment: A related recent question (about coefficients and `Ridge` rather than optimal alpha and `RidgeCV`: https://stackoverflow.com/q/73583687/10495893
Comment: @BenReiniger thanks for that. Should you maybe roll that one out as an answer? Or you reckon I should just delete this post as it wouldn't be helpful to have it?
Comment: I'll see how that answer fares/translates here when I have a chance (unless another answerer beats me to it).
Answer: The corresponding warning message for ordinary Ridge makes an additional mention:

Set parameter alpha to: original_alpha * n_samples.

(I don't entirely understand why this is, but for now I'm willing to leave it.  There should probably be a note added into the warning for RidgeCV along these lines.)  Changing your alphas parameter in the second approach to [alph * X.shape[0] for alph in alphas] should work.  The selected alpha_ will be different, but rescaling again ridge_pipe2.named_steps.model.alpha_ / X.shape[0] and I retrieve the same value as in the first approach (as well as the same rescaled coefficients).
(I've used the dataset shared in the linked question, and added the experiment to the notebook I created there.)
Answer: You need to ensure the same cross validation is used and scale without centering the data.
When you run with normalize=True, you get this as part of the warning :
If you wish to scale the data, use Pipeline with a StandardScaler in a preprocessing stage. To reproduce the previous behavior:

from sklearn.pipeline import make_pipeline

model = make_pipeline(StandardScaler(with_mean=False), Ridge())

Regarding the cv, if you check the documentation, RidgeCV by default performs leave-one-out cross validation :
Ridge regression with built-in cross-validation.
See glossary entry for cross-validation estimator.
By default, it performs efficient Leave-One-Out Cross-Validation.

So to get the same result, we can define a cross-validation to use :
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import Pipeline
from sklearn.model_selection import KFold

kf = KFold(10)

X_train, y_train = datasets.make_regression()

alphas = [0.001,0.005,0.01,0.05,0.1]

ridgecv = RidgeCV(alphas = alphas, scoring = 'neg_mean_squared_error', normalize = True, cv=KFold(10))
ridgecv.fit(X_train, y_train)
ridgecv.alpha_
0.001

And use it on pipeline :
steps = [
    ('scalar', StandardScaler(with_mean=False)),
    ('model',RidgeCV(alphas=alphas, scoring = 'neg_mean_squared_error',cv=kf))
]

ridge_pipe2 = Pipeline(steps)
ridge_pipe2.fit(X_train, y_train)

ridge_pipe2.named_steps.model.alpha_
0.001

Comment: I still get different values. If I add `cv=KFold(10)` and `with_mean=False`, I now get `ridgecv.alpha_ = 0.006609705742330144`, which is different than before but still not the same as `1.328043891473342` that I get with both `Pipeline` examples. But I believe it's what @Ben said, `normalize` and `StandardScaler` perform different calculations to the data, so it is expected them to yield different results.
Comment: did you feed the same KFold into Pipeline ?
Comment: Yes. I have edited my answer with the new code.


how does the absolute value of e to the power of a complex number only maintain the real part?

Question: how does $\lvert e^{2+i} \rvert $ equal to $e^2$??
im really confused as to how this is possible pls help. this is my first time here so if anything looks weird then sorry :/
Comment: If $\theta$ is any real number, then $|e^{(i\theta)}|$ is automatically equal to $1$.  This is a consequence of the definition of $|e^{(i\theta)}|$, the definition of $|z|$, when $z = x + iy, ~: ~x,y \in \Bbb{R}$, and the fact that for all $\theta, ~\sin^2(\theta) + \cos^2(\theta) = 1.$
Comment: For all complex $z,w, ~|z \times w| = |z| \times |w|$.  Also, $e^{(2+i)} = e^2 \times e^i$.
Answer: Recall that $e^{i\theta} = \cos{\theta} + i\sin{\theta}.$ So, when we take the modulus of $e^{i\theta}$ we get $\sqrt{\cos^2{\theta} + \sin^2{\theta}} = \sqrt{1} = 1.$
So, for any real $a$ and $b$ we have that $|e^{a + bi}| = |e^a|\cdot|e^{bi}|$ (by the multiplicative property of the modulus) which is in turn equal to $e^a \cdot 1.$ (by the positivity of $e^x$ and our earlier argument)
One way to think of this geometrically is that if we rewrite the number $e^{a + bi}$ as $Re^{i\theta},$ where $R = e^a, \theta = b,$ then $R$ gives the distance from the origin (the modulus) and $\theta$ gives the direction (argument) so changing $\theta$ maintains the same modulus while rotating about the origin.


Constructing high resolution images in Python

Question: Say I have some huge amount of data stored in an HDF5 data file (size: 20k x 20k, if not more) and I want to create an image from all of this data using Python. Obviously, this much data cannot be opened and stored in the memory without an error. Therefore, is there some other library or method that would not require all of the data to be dumped into the memory and then processed into an image (like how the libraries: Image, matplotlib, numpy, etc. handle it)?
Thanks.
This question comes from a similar question I asked: Generating pcolormesh images from very large data sets saved in H5 files with Python But I think that the question I posed here covers a broader range of applications.

EDIT (7.6.2013)
Allow me to clarify my question further: In the first question (the link), I was using the easiest method I could think of to generate an image from a large collection of data stored in multiple files. This method was to import the data, generate a pcolormesh plot using matplotlib, and then save a high resolution image from this plot. But there are obvious memory limitations to this approach. I can only import about 10 data sets from the files before I reach a memory error.
In that question, I was asking if there is a better method to patch together the data sets (that are saved in HDF5 files) into a single image without importing all of the data into the memory of the computer. (I will likely require 100s of these data sets to be patched together into a single image.) Also, I need to do everything in Python to make it automated (as this script will need to be run very often for different data sets).
The real question I discovered while trying to get this to work using various libraries is: How can I work with high resolution images in Python? For example, if I have a very high resolution PNG image, how can I manipulate it with Python (crop, split, run through an fft, etc.)? In my experience, I have always run into memory issues when trying to import high resolution images (think ridiculously high resolution pictures from a microscope or telescope (my application is a microscope)). Are there any libraries designed to handle such images?
Or, conversely, how can I generate a high resolution image from a massive amount of data saved in a file with Python? Again the data file could be arbitrarily large (5-6 Gigabytes if not larger).
But in my actual application, my question is: Is there a library or some kind of technique that would allow me to take all of the data sets that I receive from my device (which are saved in HDF5) and patch them together to generate an image from all of them? Or I could save all of the data sets in a single (very large) HDF5 file. Then how could I import this one file and then create an image from its data?
I do not care about displaying the data in some interactive plot. The resolution of the plot is not important. I can easily use a lower resolution for it, but I must be able to generate and save a high resolution image from the data.
Hope this clarifies my question. Feel free to ask any other questions about my question.
Comment: It can be done with GDAL, see for example:
http://stackoverflow.com/questions/14911590/can-you-loop-through-pixels-in-an-image-without-loading-the-whole-image/14912580#14912580
Comment: Can you use the hdf image api?
Comment: This question is not very clear. You don't seem to want an image, but a visualisation of your data in the form of `pcolormesh` or `imshow`. (Note that 20k x 20k in `Float32` is only 1.5 Gb, are you sure you don't have this?) Also, the approach in your other question can lead to a different intensity scaling in each sub-image. If you know the bounds of your image, I'd suggest creating an empty uint8 array (less memory usage) for the whole thing, and then populate it by scaling each sub-image to the min/max of the data. If that fails, I'd save each image as png and merge them with ImageMagick.
Comment: @tiago Please see edit to the question.
Answer: You say it "obviously can't be stored in memory", but the following calculations say otherwise.
<code>20,000 * 20,000 pixels * 4 channels = 1.6GB
</code>
Most reasonably modern computers have 8GB to 16GB of memory so handling 1.6GB shouldn't be a problem.
However, in order to handle the patchworking you need to do, you could stream each pixel from one file into the other. This assumes the format is a lossless bitmap using a linear encoding format like BMP or TIFF. Simply read each file and append to your result file.
You may need to get a bit clever if the files are different sizes or patched together in some type of grid. In that case, you'd need to calculate the total dimensions of the resulting image and offset the file writing pointer.


How to full outer join multiple csv files into a single data.table in R?

Question: I have been using python for a while now to work with pandas dataframes. I want to switch the same code I am using over to R. However, I don't have much experience with R and I'm not sure what options I have to do the same thing. I have a folder of many csv files, and I have a list of the file names which I would like to iterate over and do a full outer join of the files.
In pandas, I would run the following command,
import pandas as pd
filelist = pd.read_excel("/Users/XXX/Documents/test/data/list.xlsx") #contains a list of filenames in the File column around 8000 rows long

workingdf = pd.DataFrame() #create a blank data frame

for subdir in filelist.File:
    df = pd.read_csv(f"/Users/XXX/Documents/test/data/{subdir}",index_col=0) #open file 2 columns named DATE and VALUE
    df = df.rename(columns={"VALUE":subdir}) #rename the value column to the file name
    workingdf = pd.merge(workingdf,df,how='outer',left_index=True,right_index=True)

The last part of the code merges all the dataframes in the dictionary into one big data frame using a full outer join.
I want to take a bunch of csv files I have in a folder, iterate over a list of file names, create data tables out of the files, and append them one by one to make one big data table in R.
I am trying to switch to R because I am running into memory and efficiency problems and someone suggested I give it a try.
How can I run the same instruction set in R? I don't have much experience with it but I have the following code so far.
library(openxlsx)
library(data.table)

filelist <- openxlsx::read.xlsx("/Users/XXX/Documents/test/data/list.xlsx")

for (row in filelist[1]){

}

I know I need to use tempdf = read.csv(paste("/Users/XXX/Documents/test/data/",row,sep"")) to create the path to the file and make the frame.
The renaming function should be along the lines of names(tempdf)[2]<-row
I also know that I have to use workingdf <- merge(workingdf,tempdf,by="DATE",all=TRUE)
But I don't really know where to start on this because I can't create a blank dataframe like in pandas and just start merging them. Any suggestions?
Comment: Like this: https://stackoverflow.com/a/11433532/11437205
Comment: what if I want to use my own list of file names?
Comment: and i think this code appends row-wise, I would like to do a full outer join column wise
Answer: An example of full outer join on a list of csvs
library(data.table)
#get list of csv in current folder
l <- list.files(pattern="(.*).csv")

#use data.table::fread to read them and then merge with all=TRUE for full outer join
#Reduce is a recursive function takes prev output to be merged with next input
Reduce(function(x, y) merge(x, y, by="DATE", all=TRUE), lapply(l, fread))

Comment: there is a WIP PR https://github.com/Rdatatable/data.table/pull/4370 that proposes api `mergelist(l, on="DATE", how="full")`
Comment: What if I want to provide it my own list ?
Comment: then replace `l` with your list
Comment: If I use read.xlsx will the format be the same? the file names need to be concatenated with the directory
Comment: i do not know how your spreadsheet looks like. You can use `full.names=TRUE` argument for the full path. give it a try and see what other problems you have since we are unable to help if we dont have your data
Answer: One can use R functions such as paste0() or sprintf() to build the full path file name for a list of files to be read into R. Here is an example that downloads spreadsheets for the first seven generations of Pokémon, unzips the file, and uses sprintf() to build the path names to read each file.
download.file("https://raw.githubusercontent.com/lgreski/pokemonData/master/PokemonXLSX.zip",
               "PokemonXLSX.zip",
               method="curl",mode="wb")
unzip("PokemonXLSX.zip")
library(readxl)
# create a set of numbers to be used to generate file names
# that have leading zeros, e.g. gen01.xlsx, gen02.xlsx, etc.
generationIds <- 1:7
spreadsheets <- lapply(generationIds,function(x) {
     # use generation number to create individual file name
     aFile <- sprintf("./PokemonData/gen%02i.xlsx",x)
     data <- read_excel(aFile)
     })

In order to merge the files by an ID variable, one can use the Reduce() function as illustrated in another answer.
The following code illustrates how the file names are created.
theFiles <- lapply(generationIds,function(x) {
        # use generation number to create individual file name
        aFile <- sprintf("./PokemonData/gen%02i.xlsx",x)
        message(paste("current file is: ",aFile))
        aFile
})


In-app Billing promotion for users who have bought something in another app

Question: We have two apps who has premium features that are unlocked by In-app purchase. One app is mostly used in winter and the other in summer.
For billing we are using Google Play Services In-app Billing.
We would like to give 50% off (by allowing to buy cheeper product) for premium features in one app for users that have already bought premium in the other.
The question is: How to get information in application A from application B about purchases in B?
First idea
We've considered using hidden file stored in device which has info that indicates if user has premium in one app so we can read it form the other.
But there are few drawbacks in this solution:

this solution is not really secure due to file creation/edition possibility.
user has multiple devices and we can not get access to this file from all of them
if we put update for one app (eg winter app) users most likely will not open it until next season and the hidden file will not be created.

Second idea was to create ContentProviders in both apps to share info from internally (DB, SharedPrefs) stored data about purchases status.
This solution solves problems 1st and 3rd from first solution but the 2nd remains.
We don't have any sign in/server side system in our apps so we cannot store it in server backend.
We wondered if there is possibility to use In App Billing api to solve our problem? Or maybe there is some other solution for this?
Answer: If you want to use purchases for 'all devices' + 'sequre' + 'several apps' without your own server the only solution I would suggest is to use Firebase database. You can share Firebase data between several applications.
Nonetheless, in this case you have to use internet connection.
Comment: Thanks a lot for your answer.

It could be solution in some cases but unfortunately, we predict that most of the users of our 'winter app' will not open it until the next winter season. So even if we send the app update containing Firebase connection, we will have to wait until next season, when they start using it again, to reach the data.
Comment: You can silent update and send remote notification @mirekkomis
Comment: @mirekkomis you have to update your 'winter app' to make any connection between new and old apps


How can I get the first and last day of the week crossing months in javascript

Question: So for example, monday is sep 28, a week from monday, sunday is oct 4. How Can I get the first and last day of the week? So far the solutions cover only days belonging in the same month. Please help. Thanks
Answer: You can add a week to the starting date's time and construct a new date:

const DAY = 1000 * 60 * 60 * 24; // ms * seconds * minutes * hours
const WEEK = DAY * 7;

const today = new Date('2020-09-29');
const nextWeek = new Date(today.getTime() + WEEK);

console.log(nextWeek.toUTCString());

Then add or subtract from that date to get the first/last day of the week if necessary.

const DAY = 1000 * 60 * 60 * 24;
const WEEK = DAY * 7;

const today = new Date('2020-09-29');
const nextWeek = new Date(today.getTime() + WEEK);

const day = nextWeek.getDay(); // 0 sunday, 6 saturday

const firstDay = new Date(nextWeek.getTime() - (day * DAY));
const lastDay = new Date(nextWeek.getTime() + ((6 - day) * DAY));

console.log(firstDay.toUTCString()); // monday
console.log(lastDay.toUTCString()); // sunday
Comment: Thanks for taking Interest in my issue. I have resolved it now. Thanks again very much :D
Answer: You can use date-fns library for that:
const start = startOfWeek(date);
const end = endOfWeek(date);

Check out these threads for more solutions:

How to get first and last day of current week when days are in different months?
How to get first and last day of the week in JavaScript
Comment: The first thread link was very helpful
Answer: You want to look into the Date.getDay() method. This returns a number from 0-6 for the day of the week (Sunday is 0).
As a function, it could look like this:
function getMonday(date){
  const originalDay = date.getDay();
  const monday = 1;
  const newDate = new Date(date);
  const delta = originalDay === 0 ? 1 : originalDay - monday;
  newDate.setDate(newDate.getDate() - delta);
  return newDate;
}

Comment: Thanks for taking interest in my post. I have resolved the issue now. Thank you very much!:D


Invertible complex matrices

Question: I have 2 complex matrices:
$$A(p) = \pmatrix{
1  & 0 & i\\
p & i & p\\
1-i & p & 0\\
}$$
$$B(p) = \pmatrix{
p  & -1 & -1\\
p-i & 1 & 1\\
p+i & 1 & 0\\
}$$
I have to decide if they are invertible for
$A(p), $
$B(p),$
$(A(p)B(p)),$
$((A(p))^{10}(B(p))^3).$
What is the best method to solve it? Calculate theirs determinant? And how to deal with $(A(p)B(p))$ and $((A(p))^{10}(B(p))^3).$
Comment: Determinant is multiplicative.
Answer: The determinants are, respectively
$$\det\left(A(p)\right)=(-1+i)(p^2-1)\\
\det\left(B(p)\right)=-2p+i$$
Now

$A(p)$ is invertible iff $p^2\neq+1$
$B(p)$ is invertible iff $p\neq \frac i2$
$A(p)B(p)$ is invertible iff $0\neq\det(AB)=\det(A)\det(B)$. Thus, $AB$ is invertible iff $p\neq \frac i2\wedge p\neq \pm 1$
Similarly for $A^{10}B^3$
Answer: $\det(A)=(1-i)(1-p^2)$ and $\det(B)=i-2p$, so if $p$ is real but not equal to $\pm 1$.
Matrices $A$ and $B$ are non-singular so they are invertable. Since $\det (AB)= \det(A) \det(B)$, $AB$ is also invertible. Next, $\det A^m= (\det(A))^m$, so the other product is also invertable.
Comment: I think the factor $1-i$ should have opposite signs, but it does not change anything in the answer

Invertible complex matrices


How does git push heroku master save file/folder data in heroku server's db

Question: I was trying to understand and replicate git push myserver master in my Django project but didn't understand , how to integrate git push myserver master similar to git push Heroku master.
Can anyone help me out?
Thanks in advance.
Answer: I think they have a git repository on their side with a hook that catches whenever stuff gets pushed there.
See Server Side hooks here: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks


Anomaly Detection Without a Baseline

Question: I am attempting to find anomalies in accounting data (similar to this study: https://arxiv.org/pdf/1709.05254.pdf). I don't have any labeled data, so this attempt needs to be unsupervised. I am having trouble getting off the ground, because I have no baseline on what normal accounting data (i.e. journal entries) should look like.
I am primarily looking for local anomalies - anomalies that are dependent on the features (i.e. journal entry amount, GL account, number of lines, etc.) occurring in a specific combination that is different from the rest of the data.
I have thought of (and deployed some) clustering methods, but I have no way to know whether the clusters mean anything, because I don't know what normal behavior looks like. How do I determine the baseline of a new dataset without knowing what an anomaly looks like? Can anyone point me in the direction of an anomaly detection method that is unsupervised and works on existing data?
Answer: You can use OneClassSVM or IsolationForest. The purpose of these algorithms are to find what is normal by learning the distribution of the normal things. You need to fit them with normal data, then they will do a binary classification and say what is not normal.
This is also the idea behind auto encoder in anomaly detection (you point out in your article). You expect your anomaly to be rare, so by learning an auto encoder, you should detect the anomaly by getting the data which have bad reconstruction rate.
If you do clustering, you can try a DBSCAN.
Anyway, all those algorithms are not magic. They need parameters, and according to how you set them you will detect different things. For example, in DBSCAN you need to set the distance (or the similarity, to say which data are similar) and some parameters saying your tolerance to noise.
The best way in anomaly detection is to try out those algorithms and little by little get more and more insight on the data. When you will know well your data, you can do good feature extraction which will feed well your algorithm (in fact this is just data science life :D)
Comment: this was very helpful. I just stumbled upon sklearns implementation of the outlier detection algorithms, including IsolationForest. When you say "fit them with normal data", is it fine that there may be anomalies in the data? For instance, if I have 10,000 journal entries, and I have no idea which are anomalies, could IsolationForest or LocalOutlierFactor potentially identify the anomalies? (With the proper parameters/knowledge about the data, of course!) LocalOutlierFactor seems to address this by having implementations for novel detection and local detection.
Comment: Also, which outlier detections methods work well with more than two features?
Comment: It should be ok as long as you are not with only anomalies. There is a contamination rate for this purpose. They all works with more than two features. I don't know your dataset so it is hard to go further.
Answer: There is plenty of literature on unsupervised outlier detection.
For example the Local Outlier Factor (Wikipedia), and a simple k-nearest-neighbor-distance based approaches seem to work for many data sets.

Anomaly Detection Without a Baseline


SQL choose first of duplicate results

Question: Hello I have a query that returns computer names with their corresponding IP adress as follows:
SELECT DISTINCT TOP (100) PERCENT dbo.Inv_AeX_AC_TCPIP.[IP Address],
                                  dbo.Inv_AeX_AC_TCPIP.[Host Name]
FROM dbo.Inv_AeX_AC_Identification
INNER JOIN dbo.Inv_AeX_AC_TCPIP
   ON dbo.Inv_AeX_AC_Identification._ResourceGuid = dbo.Inv_AeX_AC_TCPIP._ResourceGuid

This is returning results such as:
Computer:               IP:

Frank's Computer        10.0.0.1
Frank's Computer        10.0.0.3
Nick's Computer         10.0.1.0

Is it possible to just return the first "Frank's Computer" row, and exclude the other one?
Thanks!
Comment: Why on earth are you using `TOP (100) PERCENT`?
Comment: `LIMIT 1` is what you need?
Comment: On one criteria do you want to put Frank first? Is it alfabetic? Without some criterion like that, you can't guarantee that records of a db are retrieved in the same order.
Comment: @AaronBertrand While I agree with you completely, the engine should just ignore it (doesn't it?)
Comment: @Matthew yes, but why clutter up the code with meaningless nonsense?
Answer: Yes, this is possible, but how depends on your DBMS of choice. The functionality you are looking for is GROUP BY, which puts similar results together in a same column, but obviously then it does need to know what to do with non-aggregated values, by putting them in an aggregate function like max, min or avg.
Your results would look fine like this:
SELECT min(dbo.Inv_AeX_AC_TCPIP.[IP Address]) AS IP, dbo.Inv_AeX_AC_TCPIP.[Host Name] AS Name
FROM dbo.Inv_AeX_AC_Identification
INNER JOIN dbo.Inv_AeX_AC_TCPIP
   ON dbo.Inv_AeX_AC_Identification._ResourceGuid = dbo.Inv_AeX_AC_TCPIP._ResourceGuid
GROUP BY dbo.Inv_AeX_AC_TCPIP.[Host Name]

However the choice of IP for this is rather arbitrary now (min will do an alphabetic sort). In SQL Server and Oracle you can define a user defined aggregate function to better select which to show, in MySQL you could use something like group_concat which would probably perfectly fit your needs.
Comment: No, you can't `GROUP BY Name` because this was an alias derived in the `SELECT` list, and SQL Server parses `GROUP BY` first. You need to repeat the expression or derive the alias in a CTE/derived table.
Comment: @AaronBertrand well spotted, fixed in post.
Answer: The problem is that you should stipulate which "Frank's Computer" row you want to include.
The one with the "minimum" IP? The first one inserted? The last one?
You likely want to use a GROUP BY clause, but will need to decide what sort of aggregation to use for the ip column. MIN(), MAX() ?
If you just want the "first" one in the return, that's coincidental... do you want to couple your result to the order of data on the page?
Comment: What about poor Nick?  He only wants to exclude the other computer from Frank.
Comment: @Blam what about him? Grouping by the user will certainly include him.
Comment: I'm not going to copy @Niels excellent answer above, but we have basically said the same thing. He provided a query.
Comment: But your answer did not group.  You later edited to include group.
Comment: @Blam you're right, I deleted that answer, updated it once I understood the question, then undeleted it. You deleted your initial comment, but apparently did not undo your downvote. We seem to be caught in some edit/page refresh loop. To me, it appears that the downvote occurred *after* I undeleted the edited answer.
Answer:     SELECT DISTINCT dbo.Inv_AeX_AC_TCPIP.[IP Address], dbo.Inv_AeX_AC_TCPIP.[Host Name]
FROM         dbo.Inv_AeX_AC_Identification INNER JOIN  dbo.Inv_AeX_AC_TCPIP ON                                              dbo.Inv_AeX_AC_Identification._ResourceGuid = dbo.Inv_AeX_AC_TCPIP._ResourceGuid
and dbo.Inv_AeX_AC_TCPIP.[Host Name] in (select top 1 dbo.Inv_AeX_AC_TCPIP from dbo.Inv_AeX_AC_Identification
    INNER JOIN  dbo.Inv_AeX_AC_TCPIP T ON dbo.Inv_AeX_AC_Identification._ResourceGuid = T._ResourceGuid and T.[IP Address] = dbo.Inv_AeX_AC_TCPIP.[IP Address])

Comment: The results are already distinct.  This answer does not remove duplicate IP.
Comment: @Pavel this does **not** do what was requested. The results posted are *already* distinct... the OP wants to ignore the *different* IP address values
Comment: Who ever down voted me, should explain his action, because this code does exactly what Richard asked for

SQL choose first of duplicate results


How to add activeDropDownList to form with jQuery? PHP - Yii2

Question: the question says it all, yesterday I asked
How to add fields to activeform with js/jQuery in Yii2?
well now I need to add a dropdown menu and is is not the same thing...
I have this:
use yii\helpers\Html;
use yii\widgets\ActiveForm;
use yii\helpers\ArrayHelper;
use app\models\User;
use app\models\ContactType;

$cntcttp = new ContactType;

$this->registerJs('$("#btnadd").on("click",function(){'
  . '$("#dynamicInput").append(\''
  .  Html::textInput("contacto","",['placeholder'=>"contacto"])
  . '\');'
  . '})');

?>

<?php $form = ActiveForm::begin(); ?>

    <?= Html::activeDropDownList($cntcttp, 'ContactTypeID',
    ArrayHelper::map($ctypes, 'ContactTypeID', 'Name') , ['prompt'=>'--contacto--']) ?>

    <input type='button' id='btnadd' value="add contact">

        <?= Html::submitInput('Submit', ['class' => 'btn-primary']) ?>

<?php  ActiveForm::end(); ?>

now I need to make the dorpdown appear when the user clicks the #btnadd instead of what I have now :P
Thank you in advance :)
Answer: It's almost the same thing. Just do like below:
$this->registerJs('$("#btnadd").on("click",function(){'
. '$("#dynamicInput").append(\''
.   Html::activeDropDownList($cntcttp, 'ContactTypeID',
    ArrayHelper::map($ctypes, 'ContactTypeID', 'Name') , ['prompt'=>'--contacto--'])
. '\');'
. '})');

That's all.

You probably get syntax error in javascript, since, Yii puts \n in dropdown. The most dirty way to evade that syntax error is to do like below:
$this->registerJs('$("#btnadd").on("click",function(){'
. '$("#dynamicInput").append(\''
.   str_replace("\n", "", Html::activeDropDownList($cntcttp, 'ContactTypeID',
    ArrayHelper::map($ctypes, 'ContactTypeID', 'Name') , ['prompt'=>'--contacto--']))
. '\');'
. '})');

Special thanks to Soju who mentioned in comment to use json_encode()
However, I wrote that the most dirty way would be using str_replace(), but, you can use json_encode() instead which is a more elegant solution:
$this->registerJs('$("#btnadd").on("click",function(){'
. '$("#dynamicInput").append('
.   json_encode(Html::activeDropDownList($cntcttp, 'ContactTypeID',
    ArrayHelper::map($ctypes, 'ContactTypeID', 'Name') , ['prompt'=>'--contacto--']))
. ');'
. '})');

Comment: You should simply use json encode... http://stackoverflow.com/questions/168214/pass-a-php-string-to-a-javascript-variable-and-escape-newlines
Comment: I tryed that, it guives me a syntax error :unterminated string literal
Comment: @FernandoAndrade Can you post the HTML that `Html::activeDropDownList()` produces (edit your question and add it)? Its just a matter of escaping things properly... You may be able to do it directly or you may need to overload `HTML` with a custom implementation of `activeDropDownList` but either way it should be doable.
Comment: I as about to put the code when you edited the comment

it worked :) once again you saved the day many thanks :)
Comment: @soju Hey buddy, Thank you for mentioning that. I have updated the answer. Thanks again


How do I convert "entity:node/9 in a link?

Question: I need to get some values from a custom block I've created, and I am using the following code.
<code>// Get the block
$block = \Drupal\block\Entity\Block::load('hightlight');

// Get the block content
$uuid = $block->getPlugin()->getDerivativeId();
$block_content = \Drupal::service('entity.repository')->loadEntityByUuid('block_content', $uuid);

// Get the field value
$field = $block_content->get('field_link')->getValue()[0];
</code>
The problem is that the field contains entity:node/9. Is it possible to convert that value in a link?
This is the content of the field.
Comment: Why not render the field? If you want to get the url to build your own link then use getUrl(), see https://drupal.stackexchange.com/questions/267497/what-is-the-correct-way-to-get-the-link-url
Comment: The problem is that it doesn't like a normal link field, I can't use the getUrl() function, I've added a screenshot of the dump of my variable
Comment: The properties you've just added to the question look like a normal link field and then you can apply the method getUrl() to compute the URL object.
Comment: When I use the getUrl() I got this error : `Error: Call to a member function getUrl()`
Answer: You have to rewrite
<code>$value = $block_content->get('field_link')->getValue()[0];
</code>
to something like this:
<code>$value = $block_content->get('field_link')->first()->getUrl()->toString();
</code>
See What is the correct way to get the link url?
Comment: Thanks it works !


OpenCV249 crashes after initializing camera capture on Windows 7

Question: I have got a problem with running my opencv app on windows 7. I used visual studio C++ 2012 on Windows 8.1 to compile it.
After compiling, everything is ok on Windows 8.1 but when i try to launch it on a laptop with Windows 7 it initializes the camera(blue led next to the camera lights up) then crashes. I havent got any idea what am i doing wrong.
Here is the code:
<code>#include <opencv2\highgui\highgui.hpp>
#include <opencv2\imgproc\imgproc.hpp>

int main( void )
{
    cvNamedWindow("TEST" ,cv::WINDOW_NORMAL);
    cv::VideoCapture capture;

    cv::Mat frame;

    capture.open( -1 );
    if ( ! capture.isOpened() )
    {
        printf("--(!)Error initializing the camera\n");
        system("pause");
    }
    while(capture.read(frame))
    {
        cv::imshow( "TEST", frame );
    }

    return 0;
}
</code>
Thank You for help.
Comment: -1 as id is the fallback to vfw, maybe it works with 0 or CAP_DSHOW+0 also, if you can get fresher drivers for the webcam, worth a try.
Answer: Ok guys. I figured out it may be the problem with the camera being not compatible with opencv249. Thats why it crashes while trying to initialize it. I replaced capture from camera with capture from file and everything went great.
I just wasnt smart enough to figure it out before spamming on forum, sorry.


Remove the image style from node--type.tpl.php

Question: Is there a way from a node--'type'.tpl.php template to remove the image style of an image field?
Comment: I think this is a duplicate of learning how to print out a field value for a field in a template (http://drupal.stackexchange.com/q/30063/42650). However more likely this should not even be done, and the image field should be configured with the appropriate formatter not not display as an image style instead.
Comment: I don't understand how this could possibly be a duplicate of "how to print a field", but w/e
Comment: Because you want to render the raw value of a field? Also, it looks like the question is missing details about the implementation. The question is too broad.
Comment: I never said I wanted to render the raw value of a field. My question only asks about "removing the image style" of an image field, so calling the function "render" will render it without applying the image style. Thought it would be clear, never mentioned "render" or anything related in my question.
Answer: I found it, in field.tpl.php.
<code>$items[n]['#display_settings']['colorbox_node_style'] = "";
</code>


How to disable the replica set monitor output

Question: I have a bash script which tries to monitor the number of documents in a collection by connecting to a replica set.
COUNT=`/bin/mongo --quiet --host $REPLICA_SET policy -u myuser -p mypwd --eval 'db.myColl.count()'`

I was expecting only a number as output, however I got:
Mon Apr 27 13:33:57.716 starting new replica set monitor for replica set xx wit                                                                             h seed of xxx
Mon Apr 27 13:33:57.719 succes                                                                             sfully connected to seed xx for replica set xx
Mon Apr 27 13:33:57.719 changing hosts to ...

...

Mon Apr 27 13:33:57.736 [ReplicaSetMonitorWatcher] starting 3043

Could anyone tell me how to disable this bunch of output? Thank you.
Answer: I could not figure how to disable this. As a workaround you could a) tail -1 your output, or b) output a token that's likely to be unique just before your output and then grep it (e.g. in js: print("__myAwesomeOutput__=" + x + ";").
Answer: You cannot right now: this is a known bug in the mongo shell: https://jira.mongodb.org/browse/SERVER-27159
That bug proposes some workarounds, none of which are completely satisfactory.


$Im(A+B) \subset ImA + ImB$

Question:
Consider linear maps from $\mathbb{R}^n$ to $\mathbb{R}^m$, where $m,n
 \geq 2$.
A. $Im(A+B) \subset ImA + ImB$
B. $\ker(A+B) \subset \ker A + \ker B$

I figured out a counterexample for B: take $A = I$ and $B = -I$. Could you help me to prove A?
My attempt: $Im (A + B) = \{(A+B)x\} = \{(Ax + Bx\} = Im A + Im B$.
Is it correct?
Answer: Your last inequality is incorrect. The correct statement is slightly weaker
$$\{Ax+Bx\}\subset \{Ax+By\} = \operatorname{Im} A + \operatorname{Im} B.$$
Comment: That explains why there is no equality sign in the statement of the problem. Thank you!
Comment: It should be $\{Ax+Bx\}\subset \{Ax\}+\{Bx\}$.
Comment: @mathcraze It's correct as written. The variables $x$ and $y$ are implicitly quantified over all elements of the vector space (separately).
Comment: It is better written in $\{Ax\}+\{Bx\}$, which is more clear.
Comment: @mathcraze Actually, that seems less clear to me. In any case, the asker had no trouble understanding what I meant.

$Im(A+B) \subset ImA + ImB$


in sources in browser show me old js file not the updated one

Question: so basically in source files in chrome browser, it is saved an old js file not the updated one . no matter what I change it does not update .
I just started with firebase and do not know if the problem is with that . So how can I update files in the browser?
Answer: What do you use to serve your app? Maybe it cached the js file and it is not updating the cache.
Comment: I use firebase serve.  If I change anything in html or use script tags in html it works fine
Comment: I'm glad I could help ;-)


i686-linux-gcc:error trying to exec 'cc1':execvp:No such file or directory

Question: When I run make otapackge on android4.4 ,it runs with the errori686-linux-gcc:error trying to exec 'cc1':execvp:No such file or directorywhen build fs_config,I am very confused about that
Answer: I fixed it by export PATH=../prebuilts/gcc/linux-x86/host/.../i686-linux/4.4.3/:$PATH


Mysql command for running .txt file

Question: I have a .txt file in which i have created database and tables. I want to run it on mysql command line window. I used the command source C:\Dbbut it failed to open it. Do i need to use some other command for it?
Comment: Can  you post a few lines of the text file so we can see what sort of file it is?
Answer: Your command has correct syntax and it should work if you really have file "c:\Db" and have permissions to read it. Maybe it's C:\Db.txt or C:\Db.sql ?
As you are using Windows, please check Folder Settings to disable hiding filename extensions for known filetypes.
Answer: do something like this:
1.create and save your sql script file (for example c:\mywork\foo.sql)
use foo_db;

drop table if exists users;

create table users
(
user_id int unsigned not null auto_increment primary key,
username varchar(32) unique not null
)
engine=innodb;

insert into users (username) values ('f00'),('bar');

select * from users order by user_id;

2.start mysql command and login (mysql.exe located in the mysql bin folder)
3.type the following command and hit enter to run foo.sql
\. c:\mywork\foo.sql

Mysql command for running .txt file


How can i find number of ways presenting number as sum of no more than 4 squares

Question: I am given a number and I have to find number of ways to present that number as sum of no more than 4 squares.For example $25$ can be presented as $1^2+2^2+2^2+4^2$, $3^2+4^2$ and $5^2$.
Comment: Jacobi's theorem does not refer to ordered positive representations. https://en.wikipedia.org/wiki/Lagrange's_four-square_theorem#Number_of_representations
Answer: I would start with $n_1$, which can be represented in exactly one way ($1=1^2$) and try to come up with a recurrence of how to compute $n_{k+1}$ if all of $n_1, n_2, \ldots, n_k$ are available.
Comment: I'm stuck at coming up with a solution of computing nk+1 when n1 is available.
Comment: https://en.wikipedia.org/wiki/Lagrange's_four-square_theorem#Number_of_representations


Update Custom Object within Array using a key

Question: The title can be a little unclear. But here is what i want to achieve:
I have an NSMutableArray with custom object e.g.
[
Custom Object 1,
Custom Object 2,
Custom Object 3,
..
..
..
Custom Object 10
]

The Custom object Consists of following variables:
id, name, date, seen_count;

I have the value of id e.g.10.
Now I want get the Index of the custom object where id=10 within NSMutableArray, so that i can update that object and reload a particular Row/Cell within UITableView to reflect the changes.
P.S: One way to do is to loop through the array. But is there any other way to achieve this?
Answer: Use NSPredicate::
CustomObject *object= [[[yourArray filteredArrayUsingPredicate:[NSPredicate predicateWithFormat:@"id == 10"]] lastObject];

// make changes in object.

Get Index of Object in array::
NSUInteger positionOfChange =  [yourArray indexOfObject:object];

// update your cell that is on position positionOfChange integer value

Hope this will help and sorry for any typo in code snippet.
Note :: if you are not confirm that object will always there for id than first check that filtered array has object and then go further otherwise it will crash. Don't check lastobject in filteredArrayUsingPredicate if you are not sure.
Answer: Since an array can only be accessed by index, iterating through the array is the only way.  You can loop explicitly, looking for the appropriate object or implicitly using the indexOfObjectPassingTest method of NSArray.  The advantage of this approach is that it only has to iterate as far as the first match, while filteredArrayUsingPredicate needs to iterate the whole array.
NSInteger objectIndex=[myArray indexOfObjectPassingTest:^BOOL(id  _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
    CustomObject *myObject=(CustomObject *)obj;
    if (obj.id==targetid) {   // Assumes id is an int, change the test as appropriate if it is a string or other object
        *stop=YES;
    }
}];

Another approach is to use an auxiliary NSDictionary that stores the index of the object.  This requires a single iteration of the array to build the dictionary but allows you to locate a given object more quickly.
For convenience I will assume that id in this case is a string; if it is an int then you need to box it in an NSNumber:
NSMutableDictionary auxDict=[NSMutableDictionary new];
for (i=0;i<myArray.count;i++) {
   CustomObject *obj=myArray[i];
   [auxDict addObject:[NSNumber numberWithInt:i] forKey:obj.id];
}

Then, to access an object, simply use the dictionary
NSNumber *indexNumber=auxDict[targetId];
int index=[indexNumber intValue];
CustomObject *object=myArray[index];

Answer:     NSString *search=@"5";
    NSInteger index=-1;
    if ([[arr valueForKey:@"id"] containsObject:search]) {
        index=[arr1 indexOfObject:search];

    }
    if (index!=-1) {
        //do whatever with your index
    }

Comment: This is a fairly inefficient solution - first it creates a new array that contains the value of the `id` property of the objects in the first array, then it checks to see if the desired `id` is in this new array (iterating the array in the process), then it obtains the index of the matching object (iterating the array again) making it O3n.  You could simply use `indexOfObject` to start with and check for the special result `NSNotFound` rather than using `containsObject` but that will still make it O2n
Comment: Would you mind explaining your answer?
Comment: You can simply do it with creating new array. This would also make it O2n.


Should I consider using nonlinear expression in logistic regression when the training set is just nonlinear?

Question: Conventional logistic regression expresses log odds as a linear expression of predictors, i.e., $\beta'x$. And the problem reduces to using linear classifier $\beta'x=0$ to classify. But sometimes the training set is apparently nonlinear (may be tested using linear programming), for example, if I even visualize two almost separable circles are the two classes in a two-dimensional training set, should I just use something like $\beta_1x_1^2+\beta_2x_2^2+\beta_3$ to express log odds? Is this arbitrary enough since the functional form is based on my visualization? But on the other hand, isn't conventional logistic regression also quite arbitrary since it visualizes linearity using techniques such as linear programming?

Do people in practice attempt to fit in any nonlinear functions for logistic regression?
Comment: See http://stats.stackexchange.com/a/64039/919 for an example.  That answers your last question.  Perhaps it indicates ways in which your first question might be answered.  For a more extensive answer to the last question, consult Frank Harrell's book on *Regression Modeling Strategies* and study his extensive remarks here on CV concerning splines.
Comment: Just finished reading the long post you hyperlinked. Is the idea here suggesting exploring statistical model (here the kernel method) by theory (correct me if I am wrong)? That said, does logistic regression without any kernel trick also need a theory to support its linear separability given certain data? And unfortunately, I can't use linear programming since it's just not theory, I guess. @whuber
Comment: You seem to be confusing logistic regression with something entirely different, such as SVMs.
Comment: Could you please explain a bit? It's really interesting but I don't even know SVM so far. BTW, seems I used the wrong term, by kernel method, I am actually meaning doing a nonlinear transformation on predictors. @whuber
Answer: The term "linear" here refers to the relationship between the features (predictors) and the parameters in the regression equation
$$\mathbb{E}[\,y\mid \mathbf{f}\,]=\mathbf{f}^T\boldsymbol{\beta}$$
Here $\boldsymbol{\beta}=[\beta_1,\ldots,\beta_m]$ is the vector of parameters to be estimated from the data, while for a given data point $(\mathbf{x},y)$, the vector $\mathbf{f}(\mathbf{x})=[f_1,\ldots,f_m]$ is a set of features computed from $\mathbf{x}$, and for logistic regression, $y=\mathrm{logit}[\Pr(\mathrm{class}=1)]$.
In your example, the feature-mapping
$$\mathbf{f}(\mathbf{x})=[x_1^2,x_2^2,1]$$
is nonlinear, but the regression is still linear in the parameters $\boldsymbol{\beta}$.

For your example problem of "separating circles", you can see a nice demonstration of linear vs. nonlinear approaches using the TensorFlow Playground site:

Using a linear model with linear features $(x,y)$, the data cannot be separated.
The data can be separated using a linear model with nonlinear features $(x^2,y^2)$.
The data can be separated using linear features, but only using a nonlinear model.

(For each example above, click on the link and then press "play" to train the classifier.)
The first example can do no better than chance (i.e. 50% error). The second example is the one from your question. The third example shows a neural network with a hidden layer that learns 3 new "hybrid features", which are then used to classify the data.

For your second question, I would say it is quite common to use nonlinear feature mappings. This is often done in the context of the so-called "kernel trick" (e.g. for SVMs). Classically, these feature mappings are pre-specified. As shown in the third example above, nonlinear feature mappings can also be learned, which is often done in the context of deep learning.
Comment: @Nicholas I am really not an expert in these areas. My understanding is that for many problem domains there are well developed (possibly nonlinear) feature mappings that are known to work. These may be very sophisticated, and commonly were honed over years of study by generations of specialists. For deep learning, the "top level" classifier will typically be an SVM/logistic regression/etc. The goal is for the hidden layers to *learn* appropriate nonlinear feature mappings in a generic way, that can apply across domains. (This is my broad understanding, at least.)
Comment: By the way the "kernel trick" is *not* a method to create complex/nonlinear feature mappings. It is purely a way to be able to efficiently *train* (and evaluate) a model which *uses* high dimensional feature mappings.
Comment: Appreciate this jargon clarification. But any comment on having a nonlinear feature-mapping?
Comment: Appreciate again. But one question, suppose your training set is not linearly separable, but there seems to be a pattern, maybe a circle, maybe a hyperbolic curve, then would you apply any kernel trick or would you prefer nonlinear models such as deep learning?


Appropriate filters' setting

Question: I've custom content-type with several fields, e.g. rozdzielczosc1, rozdzielczosc2 and rozdzielczosc3. There are references to the term.
I'm creating the search. Now, I've several filters

My question is about third group of filters. I created exposed filter for field_rozdzielczosc1 and filters for field_rozdzielczosc2 and field_rozdzielczosc3 inherit values from filter for field_rozdzielczosc_1. When user set the filter value for field_rozdzielczosc1, filters values for another filter (field_rozdzielczosc2 and field_rozdzielczosc3) is the same. It works.
My search

In my description I ignore the two first group of filters. They work.
When user set "Rodzielczosc" in the search I'd like to choose contents whose field_rozdzielczosc1 or field_rozdzielczosc2 or field_rozdzielczosc3 is equal choosen value.
When user set "Rodzielczosc" and fill in the "title", I'd like to choose what above but only if title of content includes given "title".
When user doesn't set anything I'd like to choose all contents.
As you've seen, I tried something, yet it doesn't work.
Comment: Look at your screenshots. A lot of blank space and letters way to small to read. Please improve them by taking them from site displayed in a smaller window, or paste this data in txt format
Comment: I think sreenshots are noticable, yet I've respected your opinion and I've tried to update my sreenshots. I don't know why, but I've difficulty in it. It's first time when I can't upload an image. I'll try to solve this problem.
Comment: What modules are you actually using? What URLs do these screens represent?
Comment: I use Views and Field group modules. I suppose that only these modules are used in that problem.
Answer: Create separate Group of filters and move a "title" filter there.
Maybe also check all OR and AND (but they seems fine for me...)
Comment: I suppose you haven't understood me. "title" was an example. My problem is referred to inappropriate filters in third group. Currently, for field_rozdzielczosc2 and field_rozdzielczosc3 I've got marked first term's value OR second term's value etc., so, it doesn't matter which value is given by user, because I choose all contents. Every content has the property that its field_rozdzielczosc2 has first term's value or second term's value or .. nth term's value. I don't know what should I change to have what I described.
Comment: That answer isn't good.
Answer: I think your content type is not properly organized. Have a look at your fields:
field_price
field_price_1
field_price_2

field_resolution
field_resolution_1
field_resolution_2

Why don't you use field_price for all items for sell? If you will have a thousand different products for sell, will you create a thousand fields like field_price_xxxx for them?
This is how I would do that:
Content type: Image
Fields:
field_image
field_price
field_category (term reference)

View page showing fields from Image content type.
Fields:
Content: Title
Content: Image
Content: Price

Filters criteria:
Content: Title (exposed)
Content: Published (Yes)
Content: Type (= image)
Content: Image:height (exposed)
Content: Image:width (exposed)
Content: category (exposed)

Done!
Notice: There is no fields like: field_price_1, field_price_2 and there is no fields with resolutions (you don't need them).

Appropriate filters' setting


Hdd for server, will it work?

Question: i have a server : IBM System x3250 M3
its have 2 HDD slots, but currently only have 1 HDD, that is : HDD 500 TB Hotplug, sata, with part number 9CA154-176
i want to add one more HDD, but i only found with part number 9FY156-176 or 39M4514, can i used it & will raid 1 work?
Comment: What does your IBM technical sales rep. say?
Comment: it is used server, so i dont have IBM technical sales rep. btw i understand hot swap with raid 1, is it simple swap with raid 1 too?
Answer: Michaelmore,
If the new drive is SATA 3.0 7200RPM then it should work. However if your goal is to configure Raid 1 then please plan on backing up your data before doing so (especially if this is an Production environment). I'm not sure how picky the controller is but there is always that risk of compatibility with a drive that is not "officially" supported by IBM. Personally I always go straight through the vendor when ordering replacements (ie. Dell, HP ect).
Here is a link to the manual for the x3250 M3. See page 11 for hard drive options: http://www.redbooks.ibm.com/technotes/tips0803.pdf
Hope that helps
Comment: No important data currently, i have time for the test.
thank you for the link & the page direction. GBU

Hdd for server, will it work?


Distinighsing Between Groups on a Bimodal Varaible

Question: I am working with the diamonds data set from the tidyverse package in R.
<code>library(tidyverse)
View(diamonds)
</code>
When I plot a histogram of the price variable with 300 bins there looks to be 3 groups.

When I take the log of price these groups become much more vivid. There might even be 4 groups instead of 3!

My question is, how do I make sure there are groups here? What are these groups? Lastly, if I do not have the data on what these groups actually are, let's say for our case the groups are continent from which the diamond was found.
Once I distinguish the groups (maybe through GMM) how do I incorporate it in a regression model, where I regress prices on carat?
Thank you kindly.
Edit** I am editing this comment due to Henry's comment. It's more of friendly rebuttal to the comment - try to explain the DV with features in the data set before trying to group the DV.
If we take a look at the iris data set (not knowing there are three different flowers) and regress Petal Width ~ Sepal Width we would conclude that Sepal Width is not a good predictor.

However, if we originally run our GMM on the Petal Width we get back 3 groups and can visually see that Sepal Width is a good predictor.
Comment: There is a curious gap in prices between $\$1455$ and $\$1545$, but the idea of groups just from the charts may be slightly spurious.  There is a more understandable marketing tendency for weights to cluster at $1$, $1.5$ or $2$ carats or just above rather than just below, but that is not enough to explain the price patterns.  You have the data on that and on other factors that affect price such as cut and clarity and color so you can investigate further
Comment: Hi Henry, I am a bit confused on what you think my next steps should be. Price is my DV.  Are you saying I should regress the price on other variables to see if I have a variable that explains the grouping? A concern I have is if on a new data set my dependent variables do not describe the grouping.  Thanks
Comment: What I am saying is that just looking at the distribution of price may be misleading.  If you have potential explanatory variables likely to affect price, as you do here, then you should investigate them before concluding there are a particular number of groups
Comment: Henry I added to my original post. Thank you for taking the time
Answer: Why are you grouping price at all?  Grouping continuous variables always entails loss of information and should only be done when you have a strong substantive reason for doing so, which you clearly do not.
What are you trying to find out?  If, for example, you want to see if price is related to continent, then I suggest doing quantile regression with price as the DV and continent as the IV.  (Or whatever IVs you are interested in). Quantile regression makes no assumptions about the residuals and lets you answer more questions than OLS regression.
Also, groups on price are very different from a group like "which continent".  Continent is, by its nature, a discrete variable, price is not. And your addition of the Iris data set isn't really relevant. We know, a priori, that there are different flowers. A rose is a rose and not a daffodil.
Comment: Thank you peter!!! It all clicked after I read that comment and ran a regression of price on carat with the grouping low, medium, high
Comment: Peter I also found your article on quantile regression which was very helpful. For those who want to give it a read https://towardsdatascience.com/an-introduction-to-quantile-regression-eca5e3e2036a
Comment: Sorry if I am being thick, I am just really trying to understand this. In the diamond case why can I not have 3 different coloured diamonds:red, yellow and blue? A blue diamond is a blue diamond not a yellow. How does knowing that a diamond is yellow (grouping) cause a loss in information? I would think that would increase information. Thank you for answer peter
Comment: You CAN have different color diamonds. No problem. But why are you grouping PRICE? What I am saying is that grouping price (say, into low, medium and high) loses information.


Can't load font woff

Question: I have relatively simple issue but I can't fix it myself.
My log is full warning with same message:
page not found
http://mywebsite.com/sites/all/themes/bigwig/font/entypo-icon-font.woff

I have looked around and most of suggestion is about .htaccess and I tried to add such block (but no luck).
<FilesMatch "\.(ttf|otf|eot|woff)$">
  <IfModule mod_headers.c>
    Header set Access-Control-Allow-Origin "*"
  </IfModule>
</FilesMatch>

I have tried to access other files in folder via browser and everything worked except the one with *.woff
http://mywebsite.com/sites/all/themes/bigwig/font/*

What else can I try?
Answer: It seems somewhere you have something referencing that file. First place I'd look is on the theme folder.
If you have access to a Linux or Mac machine, you can cd into the folder and run,
grep -rn 'entypo-icon-font.woff' . 2>/dev/null

This will tell you if there's a file that has a mention to that file.
To fix, you can either remove the reference to that file or see if you're able to find the file and add it to the path on the url.

Can't load font woff


The Laurent Series of $\sqrt{(z-1)(z-2)}$ at the point $z=\infty$?

Question: How can we find the Laurent Series of $\sqrt{(z-1)(z-2)}$ near point $z=\infty$? By definition, it's equivalent to asking the Laurent Seires of $\sqrt{\frac{(t-1)(t-2)}{t^2}}$ near the point $t=0$. The singularities are $t=0,1, 2$ but the square-root function is multivalued. So we have to define a branch cut $[-\infty,0]$, $[1,2]$. But all points on the branch cut is obviously non-analytic. So can we say the Laurent Series do not exist?
Comment: You don't *have* to choose the specific branch cuts you made; it is not mandatory to make the choice you did. And I don't think $\infty$ is really the primary problem for you. Let's take a simpler situation. Based on the concerns you have, what would you say is the Laurent series for $\sqrt{z}$ at $z = 1$?
Comment: @KCd, could you expand on what you mean?  It seems to me that since $\sqrt{(z-1)(z-2)}$ has a branch point at $\infty$ it can't have a Laurent series there; it's not analytic in any punctured neighborhood of $\infty$.
Comment: Strin is asking about "the" Laurent series, but that is not a well-defined concept even for $\sqrt{z}$ at $z=1$. Next I would ask what Strin would say about a Laurent series for $\sqrt{z}$ at $z = 0$, just to avoid involving $\infty$ as part of the issue.
Comment: Ah, I was mistaken.  The function does not have a branch point at $\infty$ after all.  It turns out that off-by-one errors are kind of important when talking about winding numbers!
Comment: @KCd I see, you mean it's not possible to define the Laurent Series at points like $z=0$ for $\sqrt{z}$?
Answer: You can define a branch of $\sqrt{(z-1)(z-2)}$ that has its branch cut on the interval $[1,2]$, and is thus analytic in a neighbourhood of $\infty$.  If $z = 1/t$, we write
$(z-1)(z-2) = (1-t)(1-2t)/t^2$ and take $\sqrt{(z-1)(z-2)} = \sqrt{(1-t)(1-2t)}/t$.  Note that $\sqrt{(1-t)(1-2t)}$ (using the principal branch of the square root) is analytic
in a neighbourhood of $t=0$.
Comment: But we have to define the branch cut [-$\infty$, 0] in addition to [1,2], for otherwise the function is multi-valued at 0 and discussion of analytic is not allowed. If we defined that branch cut, we cannot say the function is analytic at $\infty$.
Comment: Sorry........$\infty$ is not a branch point and you're right. By the way, it's not possible to define Laurent Series at $z=0$ for $\sqrt{\frac{(t-1)(t-2)}{t}}$ right?
Comment: Right, $\sqrt{(t-1)(t-2)/t}$ has a branch point at $0$ because $\sqrt{1/t}$ does.


Delete AWS RDS S3 Exported Snapshots

Question: So I have created S3 Exports from existing snapshots in RDS. I would like to delete duplicated ones. But I don't see any button to delete them.
I have already deleted the files from the bucket. But the entry still shows up.
Any idea how to delete them?
Comment: I have the same problem, and so does [this person](https://www.reddit.com/r/aws/comments/gb177q/rds_export_to_s3/). Also haven't found anything in the docs about this feature or when it's going to be available.
Answer: Try this...looks like there's a CLI that will do it.
https://docs.aws.amazon.com/cli/latest/reference/lexv2-models/delete-export.html
...well, this may be only for the Lex service.
Comment: After trying to use the delete CLI. This link seems not to be the solution for deleting RDS exported snapshots on S3.
Comment: While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference.  Link-only answers can become invalid if the linked page changes. - [From Review](/review/late-answers/545237)

Delete AWS RDS S3 Exported Snapshots


Is there a way to add new custom selection rules for panels?

Question: I find the pre-supplied list of selection rules to create panel variants to be quite lacking.  For instance, if I am using panels to override taxonomy term pages, and I have some arbitrary boolean field on each term (say, "Is Blog?"), then I would like to create a panel variant based on this field being equal to TRUE.
However there is no such flexibility when using Selection Rules to create variants.  Am I alone in thinking that this missing functionality greatly diminishes the power of Panels and panel variants?
How would a site builder, or developer, go about creating new "custom" selection rules?
Comment: Check out this post:
http://drupal.stackexchange.com/questions/33336/what-are-ctools-plugin-content-type-access-etc-and-how-does-one-create-them
Answer: Well, there is an inelegant workaround, suitable only for someone comfortable working with code.  There is a choice in the Selection Rules dropdown called: "PHP code" where the contexts are available in the $contexts variable.  This means all fields of the current taxonomy term are in there.
I ran this:
return @$contexts['argument_term_1']->data->field_is_blog['und'][0]['value'];
It will return 1 if the is_blog field exists and has a value of 1.  Otherwise (if the field does not exist or it otherwise evaluates to false), it fails silently.
I'm not totally happy with this solution, but it gets the job done.
Comment: Careful, Including php code in an open site could present a security problems allowing users undesirable access to your site.
Answer: We initially did it the way that you did. However, after a site security audit, we were told that we should not include PHP in that way (even though it's there and available). The solution is to create a custom rule using ctools access plugins. This page has an excellent breakdown of how to do it. We now have about 5 or so custom ones we use through custom modules.


AngularJS: How to handle keyboard shortcuts in routing controllers?

Question: How do you implement page-specific keyboard shortcuts in an AngularJS 1.5 app that uses routing?
This is what I would like to achieve:
index.html:
<!doctype html>

    ...

    ...

As I understand, you need to handle key-down events on the <body> tag, if you don't have elements in the scope that are focused (input fields for example, I don't have those).
app.js:
var myApp = angular.module('myApp', ['ngRoute', 'myControllers']);

myApp.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/', {
            templateUrl: 'pages/home.html',
            controller: 'HomeCtrl'
        })
        .when('/create', {
            templateUrl: 'pages/create.html',
            controller: 'CreateCtrl'
        });
}]);

controllers.js:
var myControllers = angular.module('myControllers', []);

myControllers.controller('HomeCtrl', ['$scope', function ($scope) {

    $scope.closeLandingPagePopUp = function () {
        ...
    };

    $scope.onKeyDown = function ($event) {
        switch ($event.keyCode) {
            case 27: // [Esc]
                $scope.closeLandingPagePopUp();
                break;
            ...
        }
    };
}]);

myControllers.controller('CreateCtrl', ['$scope', function ($scope) {

    $scope.cancelCreation = function () {
        ...
    };

    $scope.onKeyDown = function ($event) {
        switch ($event.keyCode) {
            case 27: // [Esc]
                $scope.cancelCreation();
                break;
            ...
        }
    };
}]);

So on both pages the [Esc] key should be handled differently.
Of course the HTML attribute data-ng-keydown="onKeyDown($event)" doesn't work, because it's outside the controllers' scopes.
Is there a way to do this while still keeping all page specific code within their respective controller?
Comment: you should use small directives not controllers for this.
Answer: georgeawg's answer was inspiring, but I ended up with this implementation:
myControllers.controller('HomeCtrl', ['$scope', '$document', function ($scope, $document) {

    var handleKeyDown = function(event) {
        switch (event.keyCode) {
            case 27: // [Esc]
                $scope.closeLandingPagePopUp();
                break;
            ...
        }
        $scope.$apply();
    };
    $document.on('keydown', handleKeyDown);

    $scope.$on('$destroy', function() {
        $document.unbind('keydown', handleKeyDown);
    });
}]);

There is actually nothing to do within the HTML parts.
Explanation:

Every controller has a handleKeyDown(event) function that implements the shortcuts
$document.on('keydown', handleKeyDown)binds the handler function to global keydown events when the controller is initialized. $document is Angular's jQuery wrapper.
Unbind the handler when the controller get's destroyed
Use $scope.$apply() to let Angular know when data has changed and the view needs to be refreshed
Answer: Global Keydown Directive
To get events outside of a directive, bind the event handler to $document.
angular.module('myApp').directive("globalKeydown",
  function($document) {
    return function linkFn(scope,elem,attrs) {
        var handlerUnbind = $document.on("keydown", function(e) {
            scope.$eval(attrs.globalKeydown, {$event: e});
            scope.$apply();
        });
        scope.$on('$destroy', function() {
             handlerUnbind();
        })
    }
});

This example directive puts a keydown event handler on the $document that invokes the expression defined by the global-event attribute. The event is as exposed as $event.
Example Usage

Click this window and press any key

      Global keydown = {{keycode}}

The DEMO on JSFiddle.
For more information on $event, see AngularJS Developer Guide -- $event.


Comparar os dados de um ArrayList para nao deixar cadastrar em duplicidade - Java

Question: Boa tarde a todos
Preciso comparar todos os indices desse arrayList com o indice que estiver sendo adicionado atualmente, é possivel?
//MENU - CADASTRAR PRODUTOS
       private void cadastrarProdutos() {
            String escolha;
            do {
                tituloPrincipal();

                System.out.println("**** INCLUSÃO DE PRODUTOS ****");
                Mercadorias mercadoria = setDadosDoProduto();
                escolha = confirmaOperacao();
                if (escolha.equalsIgnoreCase("S")) {
                    for(int i = 0; i < prodList.size(); i++) {
                            if(prodList.get(0).getNome() == prodList.get(i).getNome()) {
                                System.out.println("Esse produto já foi cadastrado!");
                            }
                            else {
                                prodList.add(mercadoria);
                            }
                        }
                }
                escolha = getRepetirOperacao();

            } while (escolha.equalsIgnoreCase("S"));
            adicionarProduto();
        }

Answer: Se você não quer que sua lista contenha elementos duplicados, a coleção correta a se usar é o Set. Como seu código não contempla a criação da variável prodList, darei a seguinte sugestão genérica para alteração do código:
Set<Mercadorias> prodList = new HashSet<>();

Para que a coleção verifique corretamente que o item já existe em seus elementos, você deve sobrescrever o método hashCode e equals da sua classe Mercadorias. Adicione o seguinte código a ela:
@Override
public int hashCode() {
  return this.getNome().hashCode();
}

@Override
boolean equals(Object obj) {
  if (obj == null) {
    return false;
  }

  if (!(obj instanceof Mercadoria)) {
    return false;
  }

  if (obj == this) {
    return true;
  }

  return this.getNome().equals(((Mercadoria) obj).getNome());
}

Assim depois dessas alterações você pode ajustar seu código para:
// ...
if (escolha.equalsIgnoreCase("S")) {
  if (!prodList.add(mercadoria)) {
    System.out.println("Esse produto já foi cadastrado!");
  }
}
// ...

Veja funcionando no ideone.

Set
A collection that contains no duplicate elements.

Em tradução livre:

Uma coleção que não contém elementos duplicados.
Comment: Se está sobrescrevendo `Object.hashCode()`, precisa sobrescrever também `Object.equals()`.
Comment: Feito, obrigado.
Answer: No trecho
if(prodList.get(0).getNome() == prodList.get(i).getNome()) {

Você está sempre comparando o primeiro item da prodList com cada item da mesma  prodList.
Imagino que você teria que comparar com o nome da nova mercadoria, por exemplo:
if(mercadoria.getNome() == prodList.get(i).getNome()) {

Outra observação:
Quando estamos comparando objetos, precisamos usar o método equals()
no lugar de ==
if(mercadoria.getNome().equals(prodList.get(i).getNome()))

https://morettic.com.br/wp2/poo/equals-em-java/
edit:
como o @Sorack bem comentou,
a função que adiciona a mercadoria deveria ser usada de fora do loop for, da forma que colocou, a mercadoria sera adicionada diversas vezes, sempre que o nome da mercadoria avaliada for diferente.
Comment: Sua solução corrige parcialmente o código, porém não considera que ele está percorrendo a lista, portanto a porção de código que faz a inserção está no local errado.


Confused with celeryd config file and daemon file for Django in Linux

Question: I am trying to integrate celeryd with djnago but i am stuck with its config file
There are three files they are talking about. I need to know what should be the contents of those files
My Celeryd executable is in /opt/python27/bin/celeryd
There are three file i don't know what should be their contents

celeryconfig.py
/etc/default/celeryd
/etc/init.d/celeryd

I am following this linl
http://celeryq.org/docs/cookbook/daemonizing.html
I am confused between 2 and 3 beacuse configuration is in celeryconfig.py
But in /etc/init.d/celeryd they have this code
if test -f /etc/default/celeryd; then
    . /etc/default/celeryd
fi

Answer: When using celery with django, you should not use the celeryd executable directly.
You have to install django-celery and run the celery daemon as

python manage.py celeryd

Also, you have to configure celery on the settings.py file opf your django project.
Get more information at http://packages.python.org/django-celery/
Comment: I tried that and its working if i use `python manage.py celeryd` but the problem is then it runs on foreground and i can't do anything else. How can i run that in background or as daemon. Also how cna i mention indjango-celeryy that i want to use MySQL as database backend with rabbitMQ. becasue i have put those in celeryconfig.py which is not used if use django-celery
Comment: All the settings that you put on celeryconfig.py go to the django settings.py file. Read the django-celery documentation to see how it is done. To put the process in the background you can use a standard init script or som tool like supervisor (http://supervisord.org/)


How to populate without using mongoose

Question: I'm trying to implement populate() function without using mongoose in the code below:
`
    course.students.forEach(async (student, i) => {
      const s = await Student.findById(student._id);

      console.log(s.toObject()); // gets student data properly

      course.students[i] = s; // does not work
    });

    console.log(course.json());

`
I just want to update the students array with the data fetched. Can anyone explain me with the assignation doesn't works?
Thank you all! :)
Comment: I think `s` here is a cursor. Need to use `findOne()` or get the results(s) you want from the cursor. Alternatively use `$lookup` to do all of this in the database and avoid a bunch of network round trips
Answer: forEach is not meant to be used with await, try to change to a for loop and use lean() to return a plain object from the query:
for (let i = 0; i < course.students.length; i++) {
  const student = course.students[i];
  course.students[i] = await Student.findById(student._id).lean();
}

console.log(course);

How to populate without using mongoose


How to pass dafault values to the NULL or undefined fields?

Question: I have small requirement, here I'm getting values to the table dynamically. Some times few fields returns some null/blank values. If it gives null values showing error message like
"worderId[i].childNodes[0] is undefined"

Please help me  how to assign some default values to the null values.
Here I tried some code,
if(worderId.length>0)
{

WOTableData= "

 	worderId	wostatus

";
var technologyImage="";
for(i=0;i<worderId.length;i++)
    {

        WOTableData=WOTableData+"<tr title='"+worderId[i].childNodes[0].nodeValue+"	"+ worderId[i].childNodes[0].nodeValue+"	"+wostatus[i].childNodes[0].nodeValue+"
";

    }
WOTableData=WOTableData+"

";

document.getElementById("WODataDiv").innerHTML = WOTableData;
}

Thanks.
Answer: You need to add a simple condition in your for loop to check for that value.
for(i=0;i<worderId.length;i++)
{

    if (!worderId[i].childNodes || !worderId[i].childNodes.length || !worderId[i].childNodes[0].nodeValue) {
        worderId[i].childNodes = [{nodeValue: "some_default_value"}];
    }

    WOTableData=WOTableData+"<tr title='"+worderId[i].childNodes[0].nodeValue+"	"+ worderId[i].childNodes[0].nodeValue+"	"+wostatus[i].childNodes[0].nodeValue+"
";

}

Comment: I added below code in my page, but still issue existing,                           if (!worderId[i].childNodes || !worderId[i].childNodes.length || !worderId[i].childNodes[0].nodeValue) {
        worderId[i].childNodes = "---";
    }
Comment: here is the issue "worderId[i].childNodes[0] is undefined"


If $P$ is a probability measure on $(\mathbb R,\mathcal B)$ , then does there exist a minimal closed set $S$ s.t. $P(S)=1$?

Question: Let $\mathcal B$ be the borel sigma algebra algebra on $\mathbb R$ , let $P$ be a probability measure on $(\mathbb R,\mathcal B)$ , then is it true that the partially ordered set (w.r.t. set inclusion )
$\mathcal A:=\{S$ is a closed subset of $\mathbb R : P(S)=1 \}$ has a minimal element ? It is trivial to see $\mathcal A$ is nonempty as $\mathbb R \in \mathcal A$ , to apply Zorn's Lemma the only place I am stuck at is the lower bound of any chain , say $\{S_{\lambda} \}$ is a chain in $\mathcal A$ , then obviously $\cap S_{\lambda}$ is a closed set , but how can I show that $P(\cap S_{\lambda})=1$ ? Please help . Thanks in advance
Answer: Let $G$ be the union of all the open sets $U\subset\Bbb R$ such that $P(U)=0$. Then $G$ is open and (exercise!) $P(G)=0$. The set $S:=\Bbb R\setminus G$ is closed and  $P(S) =1$. If $F$ is any other closed subset of $\Bbb R$ with $P(F)=1$, then $F^c$ is an open subset of $\Bbb R$ with $P(F^c)=0$; therefore $F^c\subset G$, and so $S\subset F$. [This $S$ is called the support of $P$.]
Comment: if $P(G)=0$ is exercise then what improvement have you done other than restating the problem .... ?
Comment: Sometimes the key to a problem is looking at it in the right way. As to the exercise: Let $\mathcal G$ be the class of open sets with $P$ measure $0$. Then (because $\Bbb R$ is second countable) there is a sequence $(G_n)_{n=1}^\infty$ of elements of $\mathcal G$ such that $G=\cup_{n=1}^\infty G_n$. Then $P(G)=P(\cup_nG_n)\le\sum_{n=1}^\infty P(G_n)=0$.
Answer: It is a known fact that for every non-empty subset $\mathcal{A}$ of a $\sigma$-algebra, there is a infimum $S'$. Further, if $\mathcal{A}$ is direted down-wards, there exists a sequence $\{S_n\}$ in $\mathcal{A}$ such that $S_n \searrow S'$.
In your case, it is not difficult to see that $\mathcal{A}$ is fact direted down-wards. Then, let us take  a sequence $\{S_n\}$ in $\mathcal{A}$ such that $S_n \searrow S'$, where $S'$ is the infimum.
We have that $S_n$ has probability $1$, we obtain that $S'=\bigcap S_n$ has probability $1$ too.


Why Baum-Welch algorithm is an instantiation of EM algorithm?

Question: $\newcommand{\E}{\mathrm{E}}$
I don't understand why Baum-Welch algorithm is an instantiation of EM algorithm.
Indeed, why computing  $\alpha_t(i)$ and $\beta_t(i)$ corresponds to Expectation step. Expectation step corresponds to compute expectation over the latent variable of log-likelihood of observed variable given the parameter: $\E_{Z|X,\theta^{t-1}}[L(X|\theta)]$.
I don't understand the link.
Answer: The EM algorithm computes the expectation of the complete data log-likelihood, not the observed data log-likelihood, conditional on the observed data and current estimate of the parameters.  The complete data log-likelihood can be written with indicators for the latent data.   The expectation of an indicator is its probability.  The expectation of the product of two indicators is the joint probability.
$\gamma_i(t) = E[ I(X_t=i) \vert Y, \theta] = P(X_t=i \vert Y, \theta)$
$\xi_i(t) = E[ I(X_t=i)I(X_{t+1}=j) \vert Y, \theta] = P(X_t=i, X_{t+1}=j \vert Y, \theta)$
Computing each of these conditional probabilities is the E-step, but computing them requires computing each of them requires computing all the $\alpha$'s and $\beta$'s.
See the Gaussian mixture example for a simpler illustration of how latent variables can be written as indicators in the complete data likelihood http://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm