How can I see tty during shutdown?

Question: The shutdown of my ubuntu 12.04 never terminates. (It's been running for one hour, showing the ubuntu word with the rotating dots below: it's still alive.)
Right after I initiate the shutdown, the screen briefly (very briefly) show a tty with information on it. I'd like to see it.
How can I see the information on tty1 during shutdown?
(I already tried Ctrl+Alt+F1 in vain.)
Comment: http://askubuntu.com/questions/33416/how-do-i-disable-the-boot-splash-screen-and-only-show-kernel-and-boot-text-inst
Comment: @Rinzwind had already seen 33416, but I'd like to see what is on the screen right now, not the screen of a future shutdown.
Answer: Do you mean you want to see shut down messages?
Usually all you need to do is to press Esc when the Ubuntu logo shows up.

How can I see tty during shutdown?


Can we further simplify this matrix equation?

Question: Let $X_{n \times p}$ (rank $\leq p)$, $Y_{n \times q}$ is of rank $q$, $\Sigma_{q \times q}$ is of rank $q$ and $B_{q \times p}$ such that $Rank(J + B^TU^T) = q$ where $J = [0_{p\times q} | I_p]$ of rank $p$ and $U^T = [I_q | -C_{q \times p}]$ of rank $q$.
Can we solve the following equation for $B$ under two cases:
\begin{align}
X B \Sigma^{-1} B^T = Y\Sigma^{-1}B^T
\end{align}
where $\Sigma = A + CB + B^TC^T + B^TDB$, where $A$ is symmetric and $D$ is symmetric and positive definite.
I can see two solutions here:

$B$ is a null matrix.
$B$ is full column rank.

Are there any other possibilities?
$X B \Sigma^{-1} B^T- Y\Sigma^{-1}B^T $ is the gradient of some complicated objective function. Is that mean the objective function is non-convex?
Because it is hard to compute the second derivative of the matrix to see whether it is convex or not.
Answer: A mistake: $B\in M_{p,q}$.
Clearly, if the equation (1) $XB=Y$ has at least one solution, then the complete equation $(*)$ has at least a solution.
$(1)$ has a solution iff $XX^+Y=Y$ where $X^+$ is the Moore- Penrose inverse. cf.
https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse#Obtaining_all_solutions_of_a_linear_system
Then the general solution of $(1)$ is $B=X^+Y+(I-X^+X)W$ where $W\in M_{p,q}$ is arbitrary.
Notice also that, if $rank(B)=q$, then $(*)$ is equivalent to $(1)$ but that not implies that $(1)$ has a solution.


Acquiring a gun early without murdering innocent shopkeepers

Question: One thing that I have noticed is that acquiring some form of gun, be it a pistol or shotgun, makes the early game in Spelunky far, far easier to handle.
I have random luck with finding pistols in boxes, however those are few and far between.  Having enough gold to purchase one in the randomly-generated weapons shops is also often difficult, as shotguns are quite expensive (though I have been able to purchase the random pistol).
I am curious if there is any surefire method for increasing my ability to acquire a weapon without murdering the shopkeeper, as dealing with his relatives at each successive exit becomes a pain, and often leads to my brutal, bloody death at the hands of a shotgun blast.
Comment: I don't think the shopkeeps are innocent. If you try any funny business, they'll go after ya!
Answer: Besides finding one randomly or from a shopkeeper, you can find shotguns beneath graves labeled "ASH" (a reference to the Evil Dead movies) on Restless Dead type Jungle levels. This can be as early as the fifth level, not sure what your criteria for "early" is.
Comment: Spelunky isn't in the habit of making things easy for you!
Comment: Honestly was hoping for 'the first few' levels, as Jungle is kind of a pain without the nice clearing blast of the shotgun
Comment: I've noticed!  Reminds me quite a bit of Nethack in that regard
Answer: You can also get them by going to a dice rolling place (i don't know what their actual name is) and get them, though that is more rare. Sometimes they are in crates though too.


Placement of romex

Question: Inside of my house on the ceiling there is exposed romex stapled to the ceiling which is about 6 feet long. The ceiling is 10 feet high. Is this okay?
Comment: This isn't code anywhere in North America.  It either must be in the wall or protected by armor.
Comment: @DMoore Sounds like an answer (and an important one).
Comment: Seconded. A reference about required code would be helpful/nice. I'd also describe the 'armament' that might be common (track, conduit, etc)
Comment: As @DMoore has said, this is not up to code.  But it also says that whoever did this, was either ignorant, cheap, lazy or all of the above.  Where this could be resolved with a surface mounted conduit, I'll be concerned about how it's connected to the rest of the electrical and any other work that the prior owner did.
Comment: @bib - We have a lot of electricians on here now.  Let them look up the codes behind this.  I just state the obvious.
Comment: Could be fine, since at 10' high it's likely not in danger of physical damage.
Comment: @DMoore Can you cite a code section?
Comment: @diceless Can you cite a code section?
Comment: @Tester101 - I answered... you can argue with it now.
Answer: No problem, as long as the Authority having Junction (AHJ) doesn't consider it subject to physical damage.

334.10 (A) Type NM. Type NM cable shall be permitted as follows:
(1) For both exposed and concealed work in normally dry locations except as prohibited in 334.10(3)
334.15 Exposed Work. In exposed work, except as provided in 300.11(A), cable shall be installed as specified in 334.15(A) through (C).
(A) To Follow Surface. Cable shall closely follow the surface of the building finish or of running boards.

Notice that 334.15 (A) says "Cable shall closely follow the surface of the building finish".
Though the AHJ could always fall back to 110.12, and say it's not installed in a workmanlike manner.

110.12 Mechanical Execution of Work. Electrical equipment shall be installed in a neat and workmanlike manner.
Comment: Growing up in NYC where nothing but armored was allowed, and then Long Island where *everyone knows* (?) you can't have exposed NM (and raceways were the only surface treatment allowed), this is an eye opener. I think I agree with @DMoore that the building inspectors I have met would insist on NM being behind the walls except in unfinished space. But, as you say, the code does not seem to explicitly require it.
Comment: @bib I agree that it's not a typical installation technique, but as far as code goes, I wasn't able to find anything that disallowed it.
Comment: @DMoore My answer is based on what the code says, not what *your* inspector says. I agree that it's not the best way to do it, but again, the code doesn't seem to disallow it.
Answer: I would be under the assumption that any inspector would cite:

334.15 Exposed Work. In exposed work, except as provided in 300.11(A), cable shall be installed as specified in 334.15(A) through (C). (A) To
  Follow Surface. Cable shall closely follow the surface of the building
  finish or of running boards. (B) Protection from Physical Damage.
  Cable shall be protected from physical damage where necessary by rigid
  metal conduit, intermediate metal conduit, electrical metallic tubing,
  Schedule 80 PVC conduit, or other approved means. Where passing
  through a floor, the cable shall be enclosed in rigid metal conduit,
  intermediate metal conduit, electrical metallic tubing, Schedule 80
  PVC conduit, or other approved means extending at least 150 mm (6 in.)
  above the floor. Type NMC cable installed in shallow chases or grooves
  in masonry, concrete, or adobe, shall be protected in accordance with
  the requirements in 300.4(F) and covered with plaster, adobe, or
  similar finish.

Or they could just say that the work wasn't done in a workman-like manner.
I have done work on homes all over the US.  In the past 15 years or so we haven't even been able to run romex in garages exposed.  There is no way in the world any inspector is going to pass this is in a normal living area.
Even more on point then the garage example are basements.  I have been told by many local inspectors that if we wanted to put up walls in basements, then the only exposed wiring we could have is in the utility room area.  Basically told me to keep the basement open and totally unfinished or made me rerun all of the stapled to the joists wiring (which I have done countless times).
I don't think height matters.  If your ceilings are 10 feet you probably need a ladder to change a light bulb.  Said metal ladder could hit circuit and make you go ouch.  In a practical sense, just ask the local inspector what he thinks.  It is his interpretation of the NEC that matters.  However I would just change it, since any future owner having and inspection will see this as a red flag.
Comment: I think that the height *does* matter as it does for all other situations. It's definitely not considered *readily* accessible, it's also not subject to physical damage. I don't think the code is vague, it's just generally applied instead of specifically as it was intended. Echoing what's been said as well, examples of unfinished basements or garages is a bit of counter argument to your post. IOW, those have issues which OP does not.
Comment: The walls of garages and unfinished basements are almost always considered subject to physical damage, which is why NM cable is not allowed.
Comment: @Tester101 - Ceilings too?  I am just going by interpretations in a few areas of midwest and west coast.  No way in the world would I get this passed.
Comment: If the ceiling is out of the reach of normally damaging activities, I'd say it's not subject to physical damage.
Comment: @Tester101 - not saying you are wrong.  But the code is vague.  And there is no way in hell would even think about showing that to an inspector.  If you think you can get this passed then we are in different worlds as far as inspections.

Placement of romex


PHP How to use IPN with a PayPal shopping cart?

Question: Okay so I know how to use PayPal's IPN system for Buy Now and Basic payments, although I have now created a basic shop with about 12 various items which could be ordered multiple times using a shopping cart system and it's getting a little complicated.
My question is, while the items selected POST in this type of code:
mc_gross=0.02

item_number1=0003
item_number2=0012

payment_date=06:07:02 Feb 23, 2015 PST
payment_status=Completed
first_name=Name
firstname.lastname@example.com
firstname.lastname@example.com
last_name=lastname
email@example.com

custom=

num_cart_items=2
item_name1=Professional Website Design
item_name2=Additional Website Page

quantity1=1
quantity2=1

txn_type=cart
mc_currency=GBP

mc_gross_1=0.01
mc_gross_2=0.01

How would I be able to do a MySQL update to add each different item, or multiple items & their prices?
For example, here's what I have for a single item posting to my database...
// assign posted variables to local variables
    $item_name = $_POST['item_name'];
    $item_number = $_POST['item_number'];
    $payment_status = $_POST['payment_status'];
    if ($_POST['mc_gross'] != NULL)
        $payment_amount = $_POST['mc_gross'];
    else
        $payment_amount = $_POST['mc_gross1'];
    $payment_currency = $_POST['mc_currency'];
    $txn_id = $_POST['txn_id'];
    $receiver_email = $_POST['receiver_email'];
    $payer_email = $_POST['payer_email'];
    $custom = $_POST['custom'];
    $first_name = $_POST['first_name'];
    $last_name = $_POST['last_name'];

    if($payment_status == 'Completed'){
        $txn_id_check = mysql_query("SELECT `txn_id` FROM `payment_logs` WHERE `txn_id`='".$txn_id."'");
        if(mysql_num_rows($txn_id_check)!= 1){

            if($receiver_email == 'firstname.lastname@example.com'){

                if($item_name == 'Personal Website Design'){

                    // Add TXN ID to LOG
                    mysql_query("INSERT INTO `payment_logs` VALUES ('','".$first_name."','".$last_name."','".$payer_email."','".$txn_id."','".$item_name."','£".$payment_amount."',NOW(),'0','','£0.00')");

If the user selected multiple items, how would I change the code & my database to incorporate the various different selections a buyer could use?
Answer: Here is the snippet I've always used to parse cart items in IPN.
$i = 1;
$cart_items = array();
while(isset($_POST['item_number' . $i]))
{
    $item_number = isset($_POST['item_number' . $i]) ? $_POST['item_number' . $i] : '';
    $item_name = isset($_POST['item_name' . $i]) ? $_POST['item_name' . $i] : '';
    $quantity = isset($_POST['quantity' . $i]) ? $_POST['quantity' . $i] : '';
    $mc_gross = isset($_POST['mc_gross_' . $i]) ? $_POST['mc_gross_' . $i] : 0;
    $mc_handling = isset($_POST['mc_handling' . $i]) ? $_POST['mc_handling' . $i] : 0;
    $mc_shipping = isset($_POST['mc_shipping' . $i]) ? $_POST['mc_shipping' . $i] : 0;
    $custom = isset($_POST['custom' . $i]) ? $_POST['custom' . $i] : '';
    $option_name1 = isset($_POST['option_name1_' . $i]) ? $_POST['option_name1_' . $i] : '';
    $option_selection1 = isset($_POST['option_selection1_' . $i]) ? $_POST['option_selection1_' . $i] : '';
    $option_name2 = isset($_POST['option_name2_' . $i]) ? $_POST['option_name2_' . $i] : '';
    $option_selection2 = isset($_POST['option_selection2_' . $i]) ? $_POST['option_selection2_' . $i] : '';
    $option_name3 = isset($_POST['option_name3_' . $i]) ? $_POST['option_name3_' . $i] : '';
    $option_selection3 = isset($_POST['option_selection3_' . $i]) ? $_POST['option_selection3_' . $i] : '';
    $option_name4 = isset($_POST['option_name4_' . $i]) ? $_POST['option_name4_' . $i] : '';
    $option_selection4 = isset($_POST['option_selection4_' . $i]) ? $_POST['option_selection4_' . $i] : '';
    $option_name5 = isset($_POST['option_name5_' . $i]) ? $_POST['option_name5_' . $i] : '';
    $option_selection5 = isset($_POST['option_selection5_' . $i]) ? $_POST['option_selection5_' . $i] : '';
    $option_name6 = isset($_POST['option_name6_' . $i]) ? $_POST['option_name6_' . $i] : '';
    $option_selection6 = isset($_POST['option_selection6_' . $i]) ? $_POST['option_selection6_' . $i] : '';
    $option_name7 = isset($_POST['option_name7_' . $i]) ? $_POST['option_name7_' . $i] : '';
    $option_selection7 = isset($_POST['option_selection7_' . $i]) ? $_POST['option_selection7_' . $i] : '';
    $option_name8 = isset($_POST['option_name8_' . $i]) ? $_POST['option_name8_' . $i] : '';
    $option_selection8 = isset($_POST['option_selection8_' . $i]) ? $_POST['option_selection8_' . $i] : '';
    $option_name9 = isset($_POST['option_name9_' . $i]) ? $_POST['option_name9_' . $i] : '';
    $option_selection9 = isset($_POST['option_selection9_' . $i]) ? $_POST['option_selection9_' . $i] : '';

    $btn_id = isset($_POST['btn_id' . $i]) ? $_POST['btn_id' . $i] : '';

    $current_item = array(
                           'item_number' => $item_number,
                           'item_name' => $item_name,
                           'quantity' => $quantity,
                           'mc_gross' => $mc_gross,
                           'mc_handling' => $mc_handling,
                           'mc_shipping' => $mc_shipping,
                           'custom' => $custom,
                           'option_name1' => $option_name1,
                           'option_selection1' => $option_selection1,
                           'option_name2' => $option_name2,
                           'option_selection2' => $option_selection2,
                           'option_name3' => $option_name3,
                           'option_selection3' => $option_selection3,
                           'option_name4' => $option_name4,
                           'option_selection4' => $option_selection4,
                           'option_name5' => $option_name5,
                           'option_selection5' => $option_selection5,
                           'option_name6' => $option_name6,
                           'option_selection6' => $option_selection6,
                           'option_name7' => $option_name7,
                           'option_selection7' => $option_selection7,
                           'option_name8' => $option_name8,
                           'option_selection8' => $option_selection8,
                           'option_name9' => $option_name9,
                           'option_selection9' => $option_selection9,
                           'btn_id' => $btn_id
                          );

    array_push($cart_items, $current_item);
    $i++;
}

Then you can add a loop to your database updates...
foreach($cart_items as $cart_item)
{
    /**
     * Update database with values like...
     * $cart_item['item_name']
     * $cart_item['item_number']
     * $cart_item['quantity']
     * $cart_item['mc_gross']
     */
}


Option to upload and replace a file in google colab?

Question: I'm trying to upload and replace a file using google colab.
Currently what I do is
from google.colab import files
upload = files.upload()

then if I need to modify the file, I do it locally on my computer. If I upload it again using the same cell, the new version of the file will be uploaded as "filename(1)". I would like the new version to replace the old one.
What I do then is
!rm "filename"

And then I run the first cell again. But it is not great.
Is there an option like what follows ?
upload = files.upload(replace=True)

Comment: Does this answer your question? [How can access Uploaded File in Google colab](https://stackoverflow.com/questions/48485255/how-can-access-uploaded-file-in-google-colab)
Answer: My approach is as follows.
lsdi = os.listdir('/content')

uploaded = files.upload()

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name = fn, length = len(uploaded[fn])))

if fn in lsdi:
    os.remove(fn)
    lsdi = os.listdir('/content') # list is in arbitrary order
    for k in sorted(lsdi,reverse=True): # sorted to get the most recent file name.
        fil_dados = re.match(fn[:fn.rfind('.')],k)
        if fil_dados:
            fn = k

Comment: When I entered the source code of "files.upload()" I noticed that the 'upload()' function had no argument, so I don't think there's a way to insert an option like you're looking for.

`
def upload():
  """Render a widget to upload local (to the browser) files to the kernel.

  Blocks until the files are available.

  Returns:
    A map of the form {: } for all uploaded files.
  """
`


Fix puppy chewing damage on corner of veneer cabinets

Question: Our puppy chewed on each corner of our IKEA veneer cabinets.  She chewed past to veneer to pressboard.
I would say it is light to medium damage.  The corners are still there.  She basically scraped the veneer off.
Is there a way to repair them without replacing them?
Comment: How about a few pictures so we can see what you see.
Comment: Boy, you have it  nice. We had a dog that ate my wife's shoes, potted plants, and chair legs. He ate so much of the chair legs that we had to replace the chairs and finaly the DOG.
Comment: Thankfully it's a small dog!
Answer: Not easy to make "invisible" joints to the surface.
So, what about adding brass corners to all corners to cover the damage.
As a bonus, they may dissuade the puppy as well.
Answer: Corner protector?

https://www.vintagehardware.com/proddetail.php?prod=27256&optn0x0y0=1&gclid=Cj0KCQiA6Or_BRC_ARIsAPzuer_H6EMRmJ6MCdaAm1r5Qqtxv2skJ4s32_sqBtyClLZLeCELK8HWGKoaAoQjEALw_wcB
Something like this will cover the damage and prevent new.  You can get them at any hardware store.  I see Ikea makes some too.


Pillow recognizes JPEG encoder on install, but not use

Question: I'm doing some work on a vagrant box running Ubuntu 13.04 and python 3.3.  I've installed Pillow and libjpeg-dev (installing the latter first, as recommended), and when I install (or re-install Pillow, I see this:
--------------------------------------------------------------------
PIL SETUP SUMMARY
--------------------------------------------------------------------
version      Pillow 2.3.0
platform     linux 3.3.1 (default, Sep 25 2013, 19:29:01)
             [GCC 4.7.3]
--------------------------------------------------------------------
*** TKINTER support not available
--- JPEG support available
--- ZLIB (PNG/ZIP) support available

Great!  JPEG encoder recognized.  Except, when I run Pillow's selftest.py script, it fails a bunch of tests and starts off with this:
--------------------------------------------------------------------
Pillow 2.3.0 TEST SUMMARY
--------------------------------------------------------------------
Python modules loaded from /usr/local/lib/python3.3/dist-packages/PIL
Binary modules loaded from /usr/local/lib/python3.3/dist-packages/PIL
--------------------------------------------------------------------
--- PIL CORE support ok
*** TKINTER support not installed
*** JPEG support not installed
--- ZLIB (PNG/ZIP) support ok

So now it's not recognizing the JPEG decoder.  To make sure this isn't just a problem with selftest.py, I did my testing and sure enough I can make and manipulate .pngs but not .jpgs.  I've searched around and tried some suggested solutions, including making a symbolic link to the jpg library, like so:
sudo ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib

No dice.  Anybody have an idea of something else I could try to get the jpeg decoder working?  It really isn't practical for me to restrict the workflow entirely to .png.
Answer: And, this is why I need to wait 24 hours before posting questions.  I needed to pay attention to the Pillow install path.  It works if I create the symlink like so:
sudo ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/local/lib/

/usr/lib wasn't in the Pillow search path, but the above is.


"This application has requested the Runtime to terminate it in an unusual way."

Question: My Qt program (g++ 4.4.0) is giving me the Microsoft Visual C++ Runtime Library error "This application has requested the Runtime to terminate it in an unusual way" when I close it. But when I run it in the debugger, I don't get the error message. Does anybody know a way to get some information about the crash? The message box just has an OK button.
Edited to add: Following Wimmel's advice, I attached to the debugger. There are two threads left alive, with ThreadID 1 and 3. The stacks look like this:
Level Function              File                           Line Address
0     VTagOutput            C:\Windows\syswow64\user32.dll 0    0x7529438d
1     VTagOutput            C:\Windows\syswow64\user32.dll 0    0x7529438d
2     USER32!EmptyClipboard C:\Windows\syswow64\user32.dll 0    0x752a2674
3     ??                                                   0

Level Function                                    File                          Line Address
0     ntdll!LdrFindResource_U                     C:\Windows\SysWOW64\ntdll.dll 0   0x76f5000d
1     ntdll!LdrSetAppCompatDllRedirectionCallback C:\Windows\SysWOW64\ntdll.dll 0   0x76fdfabe
2     ??                                                                        0    0x7b9609b5
3     ??                                                                        0

Perhaps the second thread is displaying the error message? The first is a little strange: the top two rows are identical. I have nothing in my code that has TagOutput in its name, and neither does Qt as far as I can tell.
Comment: Maybe you can attach the debugger before clicking OK, and watch the stack frame of all threads to see where the error comes from.
Comment: All the symbols you're seeing are from system DLLs. Have you included debug information in your build?
Comment: @Alan: Yes. I suspect the program has already gone, and this is just the system's error-reporting mechanism.
Answer: The only time I have ever seen that error message with Qt was when I first tried setting up a Qt development environment and had installed a number of different versions of MinGW. It seemed as though my executable was finding and loading a version of mingwm10.dll that didn't agree with everything else. I hunted down and eliminated all the versions of that dll except the one that I wanted and the problem disappeared.
Comment: I have hunted them down, and there are six(!) of them, in three different versions. How did you decide which one you wanted?
Comment: That would depend but try this. If you are using the Qt SDK, you probably want the one that is included with the SDK. One method that occurs to me is to copy each of the dlls you found into the same directory as the executable and try each of them, one at a time until you identify one that is compatible. If indeed, this is the cause.
Comment: Thanks for the tip. But there's only one mingwm10.dll in my path, and it came with Qt, so I don't think this can be the problem. Meanwhile the program has stopped crashing, after some seemingly unrelated changes were made. So it's lurking in there...
Answer: Use procmon.exe and filter for process name and file access to find missing Dependencies.
I had the same Problem with Qt 5.0.1.
The program was missing some dlls from the Qt framework.
I found them in the Directory:
<QTINSTALLDIR>\Qt5.0.1\5.0.1\mingw47_32\plugins


Images overlapping at loading

Question: I have been struggling with this for 2 days.
Since I added text labels on the list of thumbnails on that page : http://hixair.fr/galleries/indexcopy.php
The page seems to load fine until it gets at the middle and thumbnails are overlapping.
What is wierd is if you open the code inspector, or rotate the display on an iPhone/ipad, things get refreshed and are displayed perfectly.
I tested with Safari mostly, issue is also present in the Galaxy SII browser. Seems fine with Firefox.
Some help will be really appreciated!!
(page validates as XHTML 1.0 Strict)
Answer: Setting the width and height on the images or their containing div via CSS (not using the html height attribute) should fix this problem for you.
Comment: This is a good idea.  I personally would float the containing div (`.thumb`) left instead of making it inline and set an explicit width/height on it, and set `overflow` to `hidden` to cut off the text.
Comment: Issue solved! Setting the width and height fixed the issue. You guys are fantastic!

Images overlapping at loading


Asp.net Invalid viewstate - Maybe IE8 4k dropped bug?

Question: I'm getting 100+ errors per day on my website with System.Web.HttpException: Invalid viewstate. at System.Web.UI.Page.DecryptStringWithIV
The website is asp.net 3.5 running on iis6 , not running in a web-garden/web-farm , single server.
The website is using xhtml doctype.
viewstate size vary from 2kb-20kb
Here are a few sample errors.
<code>Machine: ML Framework Version: 2.0.50727.3603 Assembly Version: 6.5.3664.33889
Source: http://www.domain.com/WebResource.axd?d=z5VmXXoSLLpQHoPictureAlert
Exception: System.Web.HttpException: Invalid viewstate. at System.Web.UI.Page.DecryptStringWithIV(String s, IVType ivType) at System.Web.Handlers.AssemblyResourceLoader.System.Web.IHttpHandler.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Machine: MLFramework Version: 2.0.50727.3603 Assembly Version: 6.5.3664.33889
Source: http://www.mydomain.com/ScriptResource.axd?d=SE0Ej7OlEAx91j2Cjv_6KkRPplqT-5wB4M7CZPdGdGn3LahLwqlRPApUcdxBsbFXYHZ91Q76FHAHWgHs8SmOC4zemr7
siym0QY0rF3XtJTu%3C/a%3E%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%3Ca%20id=
Exception: System.Web.HttpException: Invalid viewstate. at System.Web.UI.Page.DecryptStringWithIV(String s, IVType ivType) at
System.Web.UI.Page.DecryptString(String s) at System.Web.Handlers.ScriptResourceHandler.DecryptParameter(NameValueCollection queryString) at
System.Web.Handlers.ScriptResourceHandler.ProcessRequestInternal(HttpResponse response, NameValueCollection queryString,
VirtualFileReader fileReader) at System.Web.Handlers.ScriptResourceHandler.ProcessRequest(HttpContext context) at
System.Web.Handlers.ScriptResourceHandler.System.Web.IHttpHandler.ProcessRequest(HttpContext context) at
System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
</code>
Solutions i already tried:

Wraping all inline javascript with //
Setting enableViewStateMac to false
Setting maxPageStateFieldLength to 5000 bytes.
Define a MachineKey in web.config using
http://www.developmentnow.com/articles/machinekey_generator.aspx

i cant figure out what causes the error or which page generates them.
help please!
thank you!
Answer: Is it possible that users are leaving the app open for long lengths of time before coming back to it? Are you pulling any information from the viewstate?
Session might be a better way to keep values across postbacks.
Comment: yes that is possible , i know for a fact some users open the website and leave the browser open for long period. but how will this causes these errors?
Comment: When you generated the error (you mentioned it in a comment below), did it just happen at random or did you have to leave the site open for a long time? Is there absolutely no way to reproduce the error except by random chance?
Comment: no when i generate the error its not when the site is open for a long time.
i just click on a few links/buttons on the page. sometimes i get an error (i dont see the error its just logged into log4net file) and sometimes nothing. (even doing the same steps over and over doesn't reproduce the error.)
Comment: Are you messing with the viewstate anywhere in the program?
Comment: yes , but nothing serious , i have a few place im storing a int/string variable in viewstate.
Comment: any smart way to trace the source of the errors? the log shows no useable information.
Comment: Please post the log. Are you checking to see if the ID exists? An easy way is to do
string id = (string)ViewState["id"];
if(id != null)
{
//Do stuff
}
else
{
//Put in code to send you an email
}

If you get the email then you know either you have a mismatched ID string or the viewstate has been invalidated due to inactivity
Comment: i'm not sure i understand what you mean , in my post above i posted some sample errors from the log. i can post more if it helps but the error keeps repeating itself with different garbage at the end of the "d" param like http://www.domain.com/ScriptResource.axd?d=JpGbpqus%20=%20$get('ctl00_cphContent_UserMenu1_LoginBox1_lblStatus');%20%20%20%20%20%20%20%20if%20(email.length%20==%200%20%7C%7C%20email.value%20==
Comment: its a long shot , Any chance this is related ?
scriptResourceHandler enableCompression="true" enableCaching="true"
Answer: In my experience, these errors have been generated by a security company, or in lesser cases, by a hacker trying to post invalid data to the page.  This is especially true if you cannot seem to find the source of the error yourself by clicking through the normal usage of the page.
We handle the error on the base page (which we use to inherit all our pages from) in the page_error routine.  We then check the error message for this, and for others such as: a potentially dangerous querystring value, invalid postback or callback argument, etc.   If one of those errors exist, we store it in an audit and have a process which handles those separately.
Also, is the site on a sesssion/web farm?   It is important that the machine keys are the same on both servers.  Jeff Atwood wrote an article about this as well: http://www.codinghorror.com/blog/archives/000132.html
Comment: i can generate the error when im browsing the website but i cant seem to link it to a page or a component it just happens randomly.
On the other side i just CANNOT generate the error on my DEV machine
Answer:
Adjust the settings on the application pools so that recycling is less likely to occur at peak periods.
Use a specific key in the machine.config to prevent automatic key generation on each process start
Only post to the same ASPX page
Turn off firewall and antivirus software

found those here: http://support.microsoft.com/kb/555353
Comment: my pool recycling is the default one : every 1960 minutes.


Upgade Parallels Plesk Panel from version 9.3.0 to 11.0.9 (Latest Version)

Question: I want to upgrade Parallels Plesk Panel on Server. My server details are:
<code>Parallels Plesk Panel version: 9.3.0
Operating system: Linux 2.6.18-028stab107.1
CPU: GenuineIntel, Intel(R) Xeon(R)CPU L5506 @ 2.13GHz
</code>
I want to know that can we upgrade Plesk Panel 11.0.9 (Latest Version) on this version of OS or should i need to upgrade OS also and what the risk to upgrade Plesk ?
Comment: This is completely off-topic for SO; belongs on [sf]
Answer: In-place upgrade it's always risky because of a lot of server customizations.
The most safest upgrade approach it is upgrade through transfer. It's when you install new panel instance(I'm not sure maybe you need new license or trial license for this new panel, ask Parallels sales) and use Migration manager to transfer you data from old panel instance to new one.
In case of any critical issues Parallels has program of free assistance for Parallels Plesk Panel 11 Transfer/Migration
I recommend you to upgrade to 11.5.30, it's stable now.


Does Marshal.ReleaseComObject call the garbage collector?

Question: I was asked this question today by a colleague, and couldn't find any clue on the Internet...
Can someone tell me if calling Marshall.ReleaseComObject()directly calls the garbage collector ?
As I understand it, it only removes COM references, and then the G.C. cleans memory on its next pass, but I can be mistaken...
Thanks in advance for your help!
Answer: Looking at the implementation of ReleaseComObject in Reflector shows that it doesn't. It eventually calls a native method, but I would be surprised if that could somehow force a garbage collect.
The documentation of RCW says that

The runtime performs garbage collection on the RCW.

I would interpret that as the RCW is handled by regular GC.


How to avoid code repeating on Git repository branches

Question: I am working on Android project with 4 developers.
and the Client order the following :

New design for the app (4 developers)
Keep updates and fixes till new design ready (1 developer)

My issue is :
How to apply the above on Git repository (branches)
Shall I create a new branch for re-design and another one for update and fixes ? and what is the best usage to avoid code moving and repeating ?
some updates and fixes may affect java classes that under design changing.
Advice, please.
Very thanks
Comment: so the logic will stay almost the same? only the UI has redesign?
Comment: @user1506104 ,
the UI will change (without showing)
and some updates and fixes will be applied on the old UI (for Google play updates)
Answer: Since you are using Git, this can be done easily via Basic Branching and Merging workflow.
Your team should know how to apply Git operations like branch switching, merging and how to solve branches conflicts.
A full article about this can be found in Git documentation here
is covers a case similar to yours.


Can we find a closed form for $ \int\nolimits_{- \infty}^{\infty} \frac{\exp\left(-(a+bx)^2\right)}{1+\exp(x)}\mathrm dx$?

Question: Can we find a closed form for this definite integral:
$ \int\nolimits_{- \infty}^{\infty} \frac{\exp\left(-(a+bx)^2\right)}{1+\exp(x)}\mathrm dx$?
Comment: Please be sure to put put the entire problem in the body of the message.  Also, if you right-click on the integral, and select "view source", you can see the $\TeX$ code.
Comment: Chao: Did you make this question up yourself or did you get it from somewhere?
Comment: I got this integral when I was trying to calculate the mean of a random variable.
Answer: The integral has a closed form when $a = 0$ and $b \neq 0$. (The integral diverges if $b=0$.) We have
$$
I = \int_{-\infty}^{\infty} \frac{e^{-b^2x^2}}{1+e^x} dx.
$$
Making the substitution $x \to -x$, we can rewrite it as
$$
I = \int_{-\infty}^{\infty} \frac{e^{-b^2x^2}}{1+e^{-x}} dx = \int_{-\infty}^{\infty} \frac{e^{-b^2x^2} \cdot e^x}{1+e^{x}} dx.
$$
Adding the two integrals and dividing by $2$,
$$
I = \frac{1}{2} \int_{-\infty}^{\infty} e^{-b^2x^2} dx = \frac{1}{2 |b|} \int_{-\infty}^{\infty} e^{-y^2} dy,
$$
through the substitution $y =|b|x$. Thus we end up with the familiar Gaussian integral, which evaluates to $\sqrt{\pi}$. Thus the given integral is equal to
$$
I = \frac{\sqrt{\pi}}{2|b|}.
$$
For general $a$, of course, this trick does not work. I think it should be difficult to compute the integral.

Note. The same idea can be used to show that if $f(x)$ is a continuous even function and $c > 0$, then
$$
\int_{-c}^c \frac{f(x)}{1+e^x} dx = \frac{1}{2} \int_{-c}^c f(x) dx = \int_0^c f(x) dx.
$$
Answer: Split integration region and expand $(1+\mathrm{e}^x)^{-1}$:
$$
  (1+\mathrm{e}^x)^{-1} = \left\{
    \begin{array}{ll}
          \sum_{k \ge 0} (-1)^k \mathrm{e}^{-(k+1)x} & x > 0 \\
          \sum_{k \ge 0} (-1)^k \mathrm{e}^{k x} & x \le 0
     \end{array} \right.
$$
and then integrate term-wise, assuming $b>0$:
$$
   \int_0^\infty \mathrm{e}^{-(a+b x)^2} \mathrm{e}^{-(k+1) x} \, \mathrm{d} x =
      \frac{\sqrt{\pi}}{2 b} \exp \left( \frac{(k+1)(k+1+4 a b)}{4 b^2} \right) \mathrm{erfc}\left( a + \frac{k + 1}{2 b} \right)
$$
and
$$
   \int_{-\infty}^0 \mathrm{e}^{-(a+b x)^2} \mathrm{e}^{k x} \, \mathrm{d} x =
     \frac{\sqrt{\pi}}{2 b} \exp\left( \frac{k(k-4 a b)}{4 b^2} \right) \mathrm{erfc}\left( \frac{k}{2b} -a\right)
$$
Comment: ...but there's no known form for an infinite sum of error functions, right? :(
Comment: Yes, I am afraid.
Answer: Wolfram Alpha gives a result with $a=0$ of $\frac{\sqrt{\pi}}{2b}$ (note my $a$ is your $b^2$) but chokes when $a$ is nonzero.
Answer: The result of Srivatsan
$$\frac{\sqrt{\pi}}{2|b|} = \int_{-\infty}^{\infty} \frac{e^{-b^2x^2}}{1+e^{x}} dx = \int_{-\infty}^{\infty} \frac{e^{-b^2x^2} \cdot e^x}{1+e^{x}} dx$$
gives the case $a=-\frac{1}{2b}$ because this is
$$
 =
 e^\frac{1}{4b^2}\int_{-\infty}^\infty\frac{e^{-(-\frac{1}{2b}+bx)^2}}{1+e^{x}} dx.
$$


Dynamically Update aframe Videosphere based on screen size

Question: I am having trouble updating the video sphere content based on device size. I am currently doing the code below.
Looking at the code in the dev tools it looks like everything gets updated with the right src files but the a-videosphere doesn't seem to update or reload with the new video source.
I have tried having two different video assets within aframe but it just eats the bandwidth and makes loading way to slow.
Any help is greatly appreciated
<code><head>
<script src="https://aframe.io/releases/0.7.0/aframe.min.js"></script>
<script>src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>

<!-- SCRIPT TO CHANGE VIDEO BASED ON SCREEN SIZE -->
<script>
$(function(){
if ($(window).width() < 769) {
$("#v1").attr("src", "img/Mobile.mp4");}
else if ($(window).width() > 768){
$("#v1").attr("src", "img/Full.mp4");}
});
</script>

</head>

<body>
<a-scene vr-mode-ui="enabled: false">

<a-assets>
    <video id="video" loop crossorigin="anonymous" playsinline>
         <source id="v1" src="" type="video/mp4">
    </video>
</a-assets>

 <a-videosphere id="videosphere" src="#video" rotation="0 -95 0"></a-videosphere>

 <a-entity id="camera" camera="fov: 75" look-controls></a-entity>

</a-scene>

</body>
</code>
Answer: The assets are for preloading/cache. Any change must be done in the entity and not in the assets.
<code>$("#videosphere").attr("src", "img/Mobile.mp4");}
</code>
Comment: Thanks Antony, I actually got this working in this exact way but totally forgot I posted this: appreciate coming back with the answer though :)
Comment: Hi Calum, I was looking for a solution to this problem and came across this question. I posted the answer as soon as I found the solution. This is my first answer on stack overflow. Can you make it the accepted answer as people looking for a solution can confirm to this answer. Thank you.


while installing ffmpeg giving error during make

Question: I want to install ffmpeg on ubuntu. and following this article http://pasindudps.blogspot.com/2010/12/compiling-ffmpeg-in-ubuntu-1010.html
but when m doing make after config
<code>Then go to the ffmpeg folder.
cd ffmpeg
Start the installation
./configure --enable-gpl --enable-version3 --enable-nonfree --enable-postproc \
--enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb \
--enable-libopencore-amrwb --enable-libtheora --enable-libvorbis \
--enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab
make
</code>
giving this error
<code>/home/reach121/ffmpeg/libavcodec/x86/cabac.h:93: undefined reference to `ff_h264_norm_shift'
/home/reach121/ffmpeg/libavcodec/x86/cabac.h:93: undefined reference to `ff_h264_lps_range'
/home/reach121/ffmpeg/libavcodec/x86/cabac.h:93: undefined reference to `ff_h264_norm_shift'
/home/reach121/ffmpeg/libavcodec/x86/cabac.h:93: undefined reference to `ff_h264_mlps_state'
/home/reach121/ffmpeg/libavcodec/x86/cabac.h:93: undefined reference to `ff_h264_norm_shift'
collect2: ld returned 1 exit status
make: *** [ffmpeg_g] Error 1
</code>
Please suggest why this error is coming and what to do to solve this problem
Answer: For licensing reasons, FFmpeg does not include all codec libraries. You have to download, compile and install them separately. This is the case for MP3 and H.264 codecs.
A great guide that deals with this can be found here:
Compilation Guide Ubuntu
Comment: sorry i didnt able to get you , please suggest what i need to do
Comment: You have to install libx264. Some indications can be found at http://www.saiweb.co.uk/linux/ffmpeg-install-with-libx264-h264


Laravel redirect()->with('status','My message') error

Question: I have created a email verification on Laravel 5.2. When a user fill the registration form and is validated they return the user to login form with:
return redirect('/login')->with('status','We send a email verification, please
confirm.');

And this works perfect. The problem is when the user click on link verification in his inbox they activated the account and redirect to admin page but without status message. Here is the code for that in AuthController.php:
public function activateUser(Request $request,$token)
{
    if ($user = $this->activationService->activateUser($token)) {
       return redirect('/login')->with('status','You account is now activated. Please login.');
    }
    abort(404);
}

On my login.blade.php:
@if (session('status'))

        {{ session('status') }}

@endif

Please, someone know why the session variable is not working?
Thanks.
Comment: Can you show the code where you want to show the message?
Comment: try changing if ($user = $this->activationService->activateUser($token)) to if ($user == $this->activationService->activateUser($token)). You have single = sign in your if statement.
Comment: @RuhulAmin I edited check my code again. Thanks.
Comment: @boroboris I change that and say's $user variable it's not defined.
Answer: it should work like this. in your AuthController, just change
  return \Redirect::to('/login')->with('status','You account is now activated. Please login.');

and in your login.blade
 @if (Session::has('status'))

{{ Session::get('status') }}

 @endif

Answer: Okey i don't what happend here and why redirect()->with() is not working. I solved doing a new view called activated.blade a copy of login.blade but with a message saying: "You account is now activated. Please login." so i do this in the controller:

return redirect('/activated');


How can I install a vanity over this unfortunate plumbing?

Question: I have recently removed a vanity from my bathroom and am facing an unfortunate plumbing footprint when trying to put a new vanity in place.
The cold water line runs right against a built in cabinet which I would like to keep in place and the drain for the sink is also the drain for the bathtub in the upstairs bathroom.  So the drain pipe actually comes out of the wall and then into the floor.
My question is will I be able to remove enough material from a new vanity to fit it in this space or will I compromise the structural integrity of the vanity I am trying to install.
The previous vanity that I removed was custom built to fit in the space, ie extra pieces of wood were attached to an old piece of furniture and a piece of marble was cut for the counter top.  Believe me it sounds nicer than it looked.
Here is an image of what I am dealing with.
Comment: Do I understand that the white pipe coming out of the wall is the drain for a second floor bathtub?  If so, wouldn't an obstruction below this point cause the tub to empty backwards through the sink drain onto the floor?
Comment: It all depends on what's inside the wall and why they didn't run the plumbing inside the wall to start with.
Comment: Well looking at it from the basement there is a hot air vent which runs up the wall right where those pipes come up so putting it in the wall there is unfortunately not an option.
Also, there is a bunch of ducts which runs right under this bathroom.  I had looked into trying to move the supply line on the right over about 2 to 4 inches to give me a better footprint but I dont think I can get a torch and solder in there.  Not sure how big of a job pulling out some of the ducts would be, but it looks like that would be required to reroute the pipe, unless I can do it from above.
Comment: Here is an image of what I have under the floor.  You can see the hot air duct that runs through that wall.  Moving the pipe over several inches would probably make this a non issue with Jack's suggestion but I am not sure how easy it would be to solder in that tight location.[under bath](http://www.flickr.com/photos/113527208@N03/11790182036/)
Comment: I'd suggest editing your post to include the additional photo rather than posting a link to flickr. The link can go bad and make this question more difficult to follow by future visitors. I'd add it myself, but you have it covered by a copyright on flickr.
Comment: If the pipe on the right is fed from a pipe under the floor from the left, you could cut the pipe high enough to still get a water stop with a compression fitting to connect a valve back to it. then cut the supply under the floor beyond the ductwork, maybe down by the X bracing, so that when you shorten the piece above the floor, you can now remove completely. Redrill a new hole in the floor closer to the other supply line and measure how much shorter to cut the line you just removed. Add a slip coupler and solder it up
Comment: You should really not be using the bathtub drain as a vent. Are you sure this comes from the bathtub? Ideally there would be a separate vent line (there should be one for the bathtub above). Any chance of tying into the vent line from the bathroom upstairs, assuming there is one?
Answer: Cut that pipe out in 2 places. One place is above the bottom shelf of the new cabinet and mid way between the 2 fittings on the upper pipe that goes at a 45 degree angle. Recouple it back together when the cabinet is installed. The supply lines need to be relocated too, but you may be able to move them around to get past their issues.
The cutting of the vent pipe is still taking a small chance it will be difficult to get the pipe back together. As it is you will need to shorten the pipe you cut  out at least by a 1/2" to make room for the stop built in as part of the coupler. You will also need to account for how much out of square the cut has too.
If the upper portion of the pipe has a little upward give to it, it will be a lot easier to rejoin. If not you will need to be really assertive on getting it back together again.
Comment: @Freiheit, I would prefer that everything be relocated into the wall and floor as it should be, but he asked how to get the cabinet around the pipes. I presumed there were issues to keep from reworking the plumbing.
Comment: This seems to be a great answer. It reduces the work from modifying the vanity to just one hole drilled out for the drain and it reduces the plumbing work to making two cuts and adding two couplers. The only objection I can raise is that its another non-standard job whereas relocating the plumbing to the wall makes it right.
Answer: I would probably just cut out section of subfloor around this mess, and the section of drywall, and then re-route everything (under the subfloor) so the pipes come out of the wall.  It's not that difficult, and it's probably less work than modifying cabinets and doing the creative plumbing necessary to get this to work.
Of course, this might also mean replacing the flooring (if you intended on keeping that), unless the new vanity can completely hide the patch.
This would also give you the option of using a pedestal sink, which might be appropriate because of the tight space between the wall and door. Personally I prefer the "euro style" vanities in this situation, but just throwing it out there anyway.
Comment: I had posted above, unfortunately it looks like they ran the supply lines up through the floor because a hot air duct goes up the wall right there next to those supply lines.
Comment: You can run the lines through the joists to get into the next bay, if necessary (or even to come up in through the floor in a better place). Just keep in mind the [guidelines for holes in joists](http://diy.stackexchange.com/a/10661/157), and that you might need a right-angle drill.
Comment: Based on your photo from underneath, you don't even have to go through any joists, so it should not be too bad to route the lines up next to the vent (where the drain line enters the wall now). I would still cut out a rectangular section of the floor boards in this space to do the work from above (from the wall to the joist underneath), then patch it back up with a piece of OSB/plywood.
Answer: Cutting three small holes out of the bottom of the vanity is in no way going to affect the integrity of the vanity nor is it bad practice.  I come across bottom plumbing from time to time.
The chances are there was a reason that they couldn't put the pipes in the wall - most of the time it is the wall sitting on a joist.

Cut your pvc out so that you have about 10 inches sticking out of floor.
Cut pvc so there is a couple inches stick out of wall.
Drill small hole for copper line.  For the one of the right make sure that you do not drill vanity base.
Drill out back hole for pvc.
Take off valves off copper lines so that you do not have to make huge holes for them.  If soddered on then just cut them off below.
Need one person holding vanity and the other guiding the pipes in.
Finish underneath plumbing.

Have never had an inspector say a word about plumbing straight to an unfinished area - basement.  It isn't normal but its certainly not helping your plumbing situation by moving everything.  It is just money and time spent for possibly more issues.
Comment: +1, the only problem I see is the supply on the right, in the corner. That may need to be moved; cutting out the corner of the cab risks instability. Cutting off  *almost* the entire back panel is fine.
Answer: Looks like it could be a project to reroute the pipes.  I would modify the new vanity.  Just attach it to the wall good.  Would be a good idea to add a nailer to the other side of the pipe


Pandas : Compare columns of dataframe on basis of position and irrespective of data

Question: I have the following dataframe:

I want to output the data into multiple sheets that will contain partial filled fields.
Partial Sheet 1:
First 3 are the key columns. I want to compare the columns F5,F6,F7,F8 and get the columns which have data in the same index and delete the rest. There can be N number of columns.
In this case, my output should be in the following format:

Partial Sheet 2: Will contains Key columns and one field column (F5) without null values.

Partial Sheet 3: Will contains Key columns and one field column (F7) without null values.

I tried researching a lot however could not find anything substantial.
Any ideas would be appreciated.
Comment: You could not find solution because you have not provided the logic. No one can tell what is happening to your data to be transformed to the output
Comment: Not understand output, what positions is compared? Is specify, how? Why is not returned `z3` row, if data in all columns `F5,F6,F7,F8` ?
Comment: OK, thanks for EDIT. `I want to compare the columns F5,F6,F7,F8 and get the columns which have data in the same index and delete the rest. There can be N number of columns.` - why are returned `F6, F8` ? And not `row3` with `F5,F6,F7,F8` ?
Comment: @jezrael - I've updated the post which explains it in a bit more detail. The output excel will contain multiple sheets and only if 2 or more fields match on the basis of position then they will be included in one sheet. Else, they will be included in a new sheet.
Comment: what is `the basis of position` ?
Comment: @jezrael - F6 & F8 are being returned because number of rows with non null values and their indexes match. In other fields it is not the case therefore, they are being outputted in the new sheet.
Comment: I start understand, Need compare `F5` with `F6`, then `F7`, `F8`? Still need comapre 2 columns? Or need compare, 2, 3, 4... N columns toegteher? What is `N`  in real data?
Comment: N will be the Nth column. We need to compare all the F columns F6, F7, F8....Fn to find where number of rows and their index will match and return them in the same sheet. Else return individual columns in a new sheet.
Comment: In real data `N` is `4` ? or `100` ?
Comment: Apologies, the fields will start from F1,F2,F3,F4...FN. Where N can range to 100.
Comment: @Deluxplanet345 - Not easy, answer was edited.
Answer: Use:
#if missing values are not NaNs
df = df.replace('', np.nan)

#columns from third column
cols = df.columns[3:]

#columns with keys
keycols = df.columns[:3].tolist()

out = cols.copy()
df1 = df.copy()

#in loop create combinations of columns with same index
final_cols = []

for c in out:
    if c in out:
        m = df1[c].notna()
        #test if all columns has not missing values like tested column
        #and also test if rows with missing values like tested column are missed too
        c1=out[df.loc[m, out].notna().all()].intersection(out[df.loc[~m, out].isna().all()])

        df1 = df1.drop(c1, axis=1)
        out = out.difference(c1)
        final_cols.append(c1.tolist())

#finally sorted list of columns by length
final_cols = sorted(final_cols, key=len, reverse=True)

#create excel with multiple sheets
with pd.ExcelWriter('output.xlsx', engine='xlsxwriter') as writer:
    for c in final_cols:
        df[keycols + c].dropna(subset=c).to_excel(writer, sheet_name=f'{"_".join(c)}')

Answer: Full code
cols = df.filter(like='F').replace('', pd.NA).stack().reset_index(0).groupby(level=0)['level_0'].agg(frozenset)
common = cols.reset_index().groupby('level_0')['index'].agg(list).to_list()
extra = ['Key1', 'Key2', 'Key3']
for c in common:
    print(f'data_cols_{"-".join(c)}.csv')
    print(df.loc[cols[c[0]], extra+c])

Details
step1
You can first use aggregation as frozensets to identify the non-NA indices per column:
cols = (df
 .filter(like='F').replace('', pd.NA)
 .stack().reset_index(0)
 .groupby(level=0)['level_0'].agg(frozenset)
)

output:
F5    (2, 3, 4, 5)
F6    (0, 1, 2, 3)
F7          (1, 2)
F8    (0, 1, 2, 3)
Name: level_0, dtype: object

step2
Then aggregate again to group the columns with the same indices:
common = cols.reset_index().groupby('level_0')['index'].agg(list).to_list()
# [['F5'], ['F7'], ['F6', 'F8']]

step3
Finally, slice and export:
extra = ['Key1', 'Key2', 'Key3']
for c in common:
    print(f'data_cols_{"-".join(c)}.csv')
    df.loc[cols[c[0]], extra+c].to_csv(f'data_cols_{"-".join(c)}.csv')

output:
data_cols_F5.csv
  Key1 Key2 Key3 F5
2   x3   y3   z3  a
3   x4   y4   z4  a
4   x5   y5   z5  a
5   x6   y6   z6  a

data_cols_F7.csv
  Key1 Key2 Key3 F7
1   x2   y2   z2  c
2   x3   y3   z3  c
data_cols_F6-F8.csv

  Key1 Key2 Key3 F6 F8
0   x1   y1   z1  b  d
1   x2   y2   z2  b  d
2   x3   y3   z3  b  d
3   x4   y4   z4  b  d

used input
df = pd.DataFrame({'Key1': ['x1', 'x2', 'x3', 'x4', 'x5', 'x6'],
                   'Key2': ['y1', 'y2', 'y3', 'y4', 'y5', 'y6'],
                   'Key3': ['z1', 'z2', 'z3', 'z4', 'z5', 'z6'],
                   'F5': ['', '', 'a', 'a', 'a', 'a'],
                   'F6': ['b', 'b', 'b', 'b', '', ''],
                   'F7': ['', 'c', 'c', '', '', ''],
                   'F8': ['d', 'd', 'd', 'd', '', '']})

comparison of the different answers:
on 60k rows
# mozway
69.3 ms ± 7.79 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

# jezrael
122 ms ± 9.87 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

on 600k rows:
# mozway
747 ms ± 69.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# jezrael
1.32 s ± 61.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Comment: Have you have a chance to test this answer? I believe it is not only simpler but more efficient than the alternative.


create this array

Question: i know this sounds silly, but can someone please post the arrays described by rfc2612:
Cm = 0x5A827999
Mm = 0x6ED9EBA1
Cr = 19
Mr = 17

for (i=0; i<24; i++)
{
    for (j=0; j<8; j++)
    {
        Tmj_(i) = Cm
        Cm = (Cm + Mm) mod 2**32
        Trj_(i) = Cr
        Cr = (Cr + Mr) mod 32
    }
}

i think im doing is wrong for some reason
i get this for Tr
[[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2],
[10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2]]

Answer: Tr0 = { 19, 27, 3, 11, 19, 27, 3, 11, 19, 27, 3, 11, 19, 27, 3, 11, 19, 27, 3, 11, 19, 27, 3, 11 }
Tr1 = { 4, 12, 20, 28, 4, 12, 20, 28, 4, 12, 20, 28, 4, 12, 20, 28, 4, 12, 20, 28, 4, 12, 20, 28 }
Tr2 = { 21, 29, 5, 13, 21, 29, 5, 13, 21, 29, 5, 13, 21, 29, 5, 13, 21, 29, 5, 13, 21, 29, 5, 13 }
Tr3 = { 6, 14, 22, 30, 6, 14, 22, 30, 6, 14, 22, 30, 6, 14, 22, 30, 6, 14, 22, 30, 6, 14, 22, 30 }
Tr4 = { 23, 31, 7, 15, 23, 31, 7, 15, 23, 31, 7, 15, 23, 31, 7, 15, 23, 31, 7, 15, 23, 31, 7, 15 }
Tr5 = { 8, 16, 24, 0, 8, 16, 24, 0, 8, 16, 24, 0, 8, 16, 24, 0, 8, 16, 24, 0, 8, 16, 24, 0 }
Tr6 = { 25, 1, 9, 17, 25, 1, 9, 17, 25, 1, 9, 17, 25, 1, 9, 17, 25, 1, 9, 17, 25, 1, 9, 17 }
Tr7 = { 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2, 10, 18, 26, 2 }

..and Tm (in hex):
Tm0 = { 5a827999, d151d6a1, 482133a9, bef090b1, 35bfedb9, ac8f4ac1, 235ea7c9, 9a2e04d1, 10fd61d9, 87ccbee1, fe9c1be9, 756b78f1, ec3ad5f9, 630a3301, d9d99009, 50a8ed11, c7784a19, 3e47a721, b5170429, 2be66131, a2b5be39, 19851b41, 90547849, 723d551 }
Tm1 = { c95c653a, 402bc242, b6fb1f4a, 2dca7c52, a499d95a, 1b693662, 9238936a, 907f072, 7fd74d7a, f6a6aa82, 6d76078a, e4456492, 5b14c19a, d1e41ea2, 48b37baa, bf82d8b2, 365235ba, ad2192c2, 23f0efca, 9ac04cd2, 118fa9da, 885f06e2, ff2e63ea, 75fdc0f2 }
Tm2 = { 383650db, af05ade3, 25d50aeb, 9ca467f3, 1373c4fb, 8a432203, 1127f0b, 77e1dc13, eeb1391b, 65809623, dc4ff32b, 531f5033, c9eead3b, 40be0a43, b78d674b, 2e5cc453, a52c215b, 1bfb7e63, 92cadb6b, 99a3873, 8069957b, f738f283, 6e084f8b, e4d7ac93 }
Tm3 = { a7103c7c, 1ddf9984, 94aef68c, b7e5394, 824db09c, f91d0da4, 6fec6aac, e6bbc7b4, 5d8b24bc, d45a81c4, 4b29decc, c1f93bd4, 38c898dc, af97f5e4, 266752ec, 9d36aff4, 14060cfc, 8ad56a04, 1a4c70c, 78742414, ef43811c, 6612de24, dce23b2c, 53b19834 }
Tm4 = { 15ea281d, 8cb98525, 388e22d, 7a583f35, f1279c3d, 67f6f945, dec6564d, 5595b355, cc65105d, 43346d65, ba03ca6d, 30d32775, a7a2847d, 1e71e185, 95413e8d, c109b95, 82dff89d, f9af55a5, 707eb2ad, e74e0fb5, 5e1d6cbd, d4ecc9c5, 4bbc26cd, c28b83d5 }
Tm5 = { 84c413be, fb9370c6, 7262cdce, e9322ad6, 600187de, d6d0e4e6, 4da041ee, c46f9ef6, 3b3efbfe, b20e5906, 28ddb60e, 9fad1316, 167c701e, 8d4bcd26, 41b2a2e, 7aea8736, f1b9e43e, 68894146, df589e4e, 5627fb56, ccf7585e, 43c6b566, ba96126e, 31656f76 }
Tm6 = { f39dff5f, 6a6d5c67, e13cb96f, 580c1677, cedb737f, 45aad087, bc7a2d8f, 33498a97, aa18e79f, 20e844a7, 97b7a1af, e86feb7, 85565bbf, fc25b8c7, 72f515cf, e9c472d7, 6093cfdf, d7632ce7, 4e3289ef, c501e6f7, 3bd143ff, b2a0a107, 296ffe0f, a03f5b17 }
Tm7 = { 6277eb00, d9474808, 5016a510, c6e60218, 3db55f20, b484bc28, 2b541930, a2237638, 18f2d340, 8fc23048, 6918d50, 7d60ea58, f4304760, 6affa468, e1cf0170, 589e5e78, cf6dbb80, 463d1888, bd0c7590, 33dbd298, aaab2fa0, 217a8ca8, 9849e9b0, f1946b8 }

(I'm not sure why they didn't just include these as tables in the spec).
Comment: thanks! i knew i was doing something wrong. can you make Tm as well?
Comment: me neither. those real crypto people can be really annoying at times. sigh. i still dont know why i cant get these values. i'll try again tomorrow

create this array


Accessing PHP Class Method in Joomla

Question: There is an example Hello Word component in Joomla.
In Models:
class HellowWorldModelHelloWorld extends JModel
{
Protected $message;

Public function getMsg()
{
if (!isset($this->message))
{
$this->message = 'Hello World';
}
return $this->message;
}
}

in Views:
class HelloWorldViewHelloWorld extends JViewLegacy
{

    function display($tpl = null)
    {
        $this->msg = $this->get('Msg');

        if (count($errors=$this->get('Errors')))
        {
            JLog::add(implode('
',$errors),JLog::WARNING, 'jerror');

            return false;
        }

        parent::display($tpl);
    }
}

Now my question is how this works, as far as I have learned, I can't make it out the below code
$this->msg = $this->get('Msg');

First $this->msg - msg property is not defined anywhere in the class or in the parent class. So how msg is get defined or set by directly accessing $this->msg
Second the method in the model is defined as function getMsg() . So I think it should be accessible as $this-getMsg()(but it doesn't work) but it is being accessed as $this-get('Msg') and it is working fine.
A detailed answer with example will help me properly understand what I am missing.
Answer: Not sure how detailed I can make this, but executing $this->msg = $this->get('Msg) amounts to executing function getMsg() that is located in the model.
So, in your View, $this->msg = $this->get('Msg') will populate the variable $this->msg with whatever is returned by the function getMsg() in the Model.

Accessing PHP Class Method in Joomla


R: read.table problems

Question: I might just be crazy but I really can't see where I am going wrong on my simple R program. I am trying to read a table from a file but every time I try to it comes back with this error:
./tmp.r: line 1: syntax error near unexpected token `('
./tmp.r: line 1: `tmp <- read.table("/home/Data/run1.DOC.sample_summary",header=FALSE)'

The file I am trying to read from looks something like this:
Aim  A_%_above_20  A_%_above_30  A_%_above_40
28         0.0          0.0          0.0
99         50           100.0        82.9
34         62.1         0.0          0.0

Here is my code:
tmp <- read.table("/home/Data/run1.DOC.sample_summary",header=FALSE)
names(tmp)
max_num <- max(tmp)
hist(tmp$'*_%_above_30',col=heat.colors(max_num), main='Percent in Test', xlab='Percent Covered')

Does anyone see what I am doing wrong here? I am just not seeing it.
Thanks
Comment: As a rule I try to make my column names "propper" from an R perspective before I read the data in.  In this case, R doesn't like th e `%` symbol.  I'd suggest using a command line tool like `sed` or going in manually and changing the header row and trying again.  You can also try `read.table(..., skip=1, header=FALSE)`.
Comment: When I copy+paste your example file into a text file, I am able to read it just fine (although I'm not sure why you would want to specify `header = FALSE`; that seems wrong to me). R will by default remove the `%` signs, but that shouldn't cause an error. You probably have some other strange character in your file.
Answer: it this because you used wrong path name? e.g., try changing "home..." to "/home..."
Comment: Unfortunately that isn't the cause. I tried that already. Thanks though
Comment: I do not know how to add comment in the above discussion. is R -f equivalent to Rscript?
Answer: Those tmp$'*_%_above_30' really work in your last line ?
Also, try to put comments on different part of your code to see which one is making your code crash.
Finally, maybe it's just a bad encoding of some characters in your code. Try to rewrite it from scratch.

And how do you launch your script ?
Comment: So try to remove all the lines except the first one, see if that solves your problem. Then to launch your code, what command are you using ? Try `R -f tmp.r`
Comment: I am attempting to do it from command line. Previously when I have used R it was with a gui but that isn't available on where I am working from here
Comment: Also, I have no idea if that line works or not. I was just giving it a go. I haven't managed to get past the first line even. Lol
Comment: Ahhhhhh. My command was wrong I guess? I didn't include the -f. Also just for future reference that line of code doesn't work, but that is something I haven't even tried to play with yet. So thank you.

R: read.table problems


fork() - multiple processes and system calls

Question: I am writing a mapreduce program that uses multiple I/O pipes (one pipe per process) to get some final results. I am having a problem with creating the processes. Specifically, I am getting the following error:
wait error: Interrupted system call

This is my code that spawns processes:
while (values[inc]!=NULL) //provided array of text lines
{
    if ((pid = fork()) == -1) {
        perror("fork error");
        exit(EXIT_FAILURE);
    }

    else if (pid == 0) {             /* start of child process      */
        printf("Child process...\n");
        /* pipes[inc][1] is a file descriptor to which myMap writes some data
           using the write() system call
           mr is a struct that holds other function pointers */
        mr->myMap(pipes[inc][1],values[inc]);
        exit(0);
    }
    else {                           /* start of parent process     */
        printf("Parent process...\n");

        if ((wpid = wait(&status)) == -1)
        /* Wait for child process.      */
            perror("wait error");
        else {                       /* Check status.                */
            if (WIFSIGNALED(status) != 0)
                printf("Child process ended because of signal %d\n",
                       WTERMSIG(status));
            else if (WIFEXITED(status) != 0)
                printf("Child process ended normally; status = %d\n",
                       WEXITSTATUS(status));
            else
                printf("Child process did not end normally\n");
        }
        //close(fd[1]);

        printf("Parent process ended\n");
    }
    inc++;
}

After this I am creating one thread
pthread_t newThread;
pthread_create(&newThread,NULL,threadFunction,values);
pthread_join(newThread,NULL);

The threadFunction uses select() function to find out which file descriptor is ready to be read and reads it and puts data in a dictionary.
When running form gdb debugger, the program output:
Parent process...
Child process...
wait error: Interrupted system call
Parent process ended
Parent process...
Child process ended normally; status = 0
Parent process ended
Parent process...
Child process...
Child process...
wait error: Interrupted system call
Parent process ended

I don't know how to resolve the issue. Any suggestions?
Thanks!
Answer: You need to put your wait() call into a loop and if it returns an error (-1) and errno == EINTR continue the loop.  Any other error is a real error and should be treated as such.
Things like profiling timers can cause signals to be sent to the process, however the signal probably causing the interruption is SIGCHLD, which as you know is called when a child process changes state.
EDIT:  OK, I'll write the answer in code:
do
{
    wpid = wait(&status);
}
while (wpid == -1 && errno == EINTR);
if (wpid == -1)
{
    perror("wait error");
    return -1;
}
else
{
    // we have wait status
    ...
}

Comment: I'm not sure what you mean. Please elaborate.


Deploying struts2 + hibernate application using google app engine

Question: This is quite cumbersome for me now...
I have developed an application using struts2, and hibernate (mysql) and it is working fine with apache tomcat.
Now i want to deploy it on Google App Engine. I reckon that I have to convert my java dynamic web project to google web project. After doing so, when i tested it on google supplied server for eclipse, it is not running. It's showing exceptions with hibernate. First exception that I encountered is jdbcException: Not able to open connection .
I heard that google app engine(GAE) does not support mysql. Is it true? What are the alternatives?
Answer: By default, GAE does not use MySQL or any kind of relational database for storing data, If you want to use relational database in your application then you should go for "Google Cloud SQL". By using this you can store your data in relational database on cloud and can access it from GAE based java application.
Both of the services (GAE and Cloud SQL) are freely available but there are some limitations also. Like free versions have limited storage.
For more details, have a look : https://cloud.google.com/products/cloud-sql


CORS config set successfully to google cloud bucket. But CORS header aint added to images uploaded BEFORE settings CORS config to bucket

Question: I just can't understand why this is, but as the title explains, i applied the following CORS config to my google cloud storage bucket:
//this filename: gcs_cors_config.json

[{
  "maxAgeSeconds": 3600,
  "method": ["*"],
  "origin": ["*"],
  "responseHeader": ["*"]
}]

using the gsutil command:
gsutil cors set config/gcs_cors_config.json gs://<bucket_name>
Now, images uploaded to the bucket AFTER i set this configuration, have the access-control-allow-origin = * header as expected

But, images uploaded BEFORE setting this configuration, DO NOT have the access-control-allow-origin = * header.
Now this is a problem for me, because I'm using HTML canvas to do some stuff, and without the CORS header on the images, the canvas is getting tainted and all sorts of issues are happening.
Once again to clarify, the CORS header does get attached to images uploaded AFTER settings the CORS config. But, does not get attached to images uploaded BEFORE setting the CORS config.
Any ideas & help would be greatly appreciated!
Comment: hey @jub0bs you are right! i just checked again, i guess the cache on cloud-storage side got invalidate (because max-age was 3600), so everything is working fine now!
Comment: Are those images cached? If so, could you flush that cache?
Comment: @jub0bs i flushed the cache on the browser side. No luck. But could it be that the images are also cached on the cloud storage server side? If so, any idea on how to invalidate that cache? thanks for the response!
Answer: Okay so this was a false alarm.
Turns out cloud-storage caches images on its server side as well:
https://cloud.google.com/storage/docs/caching
Which has an age of 1 hour.
So after an hour, the older images also had the access-control-allow-origin = * header, and so all works well!
thanks @jub0bs for the suggestion :)


C# Screen resolution and Form display

Question: I have a C# WinForms application and when I give the executable to different users the application displays in different sizes (based on their screen resolution). Some of the parts of the application can't be seen.
Is there anyway to auto-size the window based on the screen resolution, or is there another approach?
EDIT : furthermore it appears in different styles under different Operating systems, is there away to standardize its design ?
Comment: One thing I'd like to say is, if it is possible, use WPF. WPF applications are resolution independent, so an inch is an inch is an inch, regardless of monitor resolution or monitor DPI.
Comment: The different styles, if you are referring to scrollbars, etc, are just how each operating system displays those objects.  In Winforms, you have no practical way of changing this-- just live with it or go to WPF.
Answer: It sounds like you have specified your controls with absolute positioning and other layout defaults. In order to make a WinForms application that looks and feels the same and behaves correctly in various resizing scenarios, you need to utilize the Anchor and Dock properties. Arranging controls in WinForms can be a tiring process, but MSDN includes some nice How To's on the subject.
I would also suggest following along with this TechRepublic article, which covers the difference between Anchoring and Docking, and shows you visually what each property accomplishes:
Comment: So do i have to re-design my application according to this? isn't there any other way to handle this issue :(
Comment: @illep: unfortunately, these are the properties which govern the layout of controls during resizing. @Navid's answer covers a workaround for the interesting side effect of DPI changes. Basically, your program should be liberal in the resolutions/DPI it supports. I was certainly frustrated the first time I realized just how much effort was required to make my GUI look/feel/behave the same across computers. You're not alone!
Comment: Actually i am displaying a browser on the Form, so i need to resize the controls of the browser as well. How should i do this ?
Answer: You can use Control.ScaleControl and Control.Scale
private void MainForm_Load( object sender, EventArgs e )
{
    float width_ratio = (Screen.PrimaryScreen.Bounds.Width / 1280);
    float heigh_ratio = (Screen.PrimaryScreen.Bounds.Height / 800f);

    SizeF scale = new SizeF(width_ratio, heigh_ratio);

    this.Scale(scale);

   //And for font size
   foreach (Control control in this.Controls)
   {
      control.Font = new Font("Microsoft Sans Serif", c.Font.SizeInPoints * heigh_ratio * width_ratio);
   }
}

In the case when the development platform resolution is 1280x800
According to  @sixlettervariables 's answer Docking and anchoring will help of course.
Comment: How do i use it? is there any sample code where i could look at ?
Comment: @Navid Rahmani , Your way it only maintains the form size, but the buttons and links inside the form hasn't got re-sized :(
Comment: No `Control.Scale` scales all content
Comment: If you mean the font size check my answer again
Comment: Actually i am displaying a browser on the Form, so i need to resize the controls of the browser as well. How should i do this ?
Comment: @Navid My form has a few panels, and on that panels there are several textboxes, buttons and links buttons. I have placed these panels infront of a Webbroswer Controller. So when i am gonna resize the application i have to consider these facts as well. any clue ?
Answer: Try this
private void MainForm_Load( object sender, EventArgs e )
  {
     this.Size = Screen.PrimaryScreen.WorkingArea.Size
  }

C# Screen resolution and Form display


How to reduce the execution time for the following code?

Question: I tried to give some sketch effects to an image for that i used Gaussian modeling technique in opencv, but i face an issue that it takes more time for execution. Time is reduced when the size of the picture is small if the size is large it takes more time. Please any one tell how to reduce the time for execution without changing the actual size of an image for the following code

<code>#include "opencv2/opencv.hpp"
#include <iostream>
#include <vector>
#include "opencv2/ml/ml.hpp"
#include <list>
#include <iostream>
using namespace cv;
using namespace std;

void clustrize_colors(Mat& src,Mat& dst)
{
 // Number of clusters
 int NrGMMComponents = 96;

 cv::GaussianBlur(src,src,Size(3,3),1);

 int srcHeight = src.rows;
 int srcWidth  = src.cols;

 // Get datapoints
 vector<Vec3d> ListSamplePoints;

 for (int y=0; y<srcHeight; y++)
 {
  for (int x=0; x<srcWidth; x++)
  {
   // Collecting points from image
   Vec3b bgrPixel = src.at<Vec3b>(y, x);

   uchar b = bgrPixel.val[0];
   uchar g = bgrPixel.val[1];
   uchar r = bgrPixel.val[2];
   if(rand()%25==0) // peek every 25-th
   {
    ListSamplePoints.push_back(Vec3d(b,g,r));
   }
  } // for (x)
 } // for (y)

 // Form training matrix
 int NrSamples = ListSamplePoints.size();
 Mat samples( NrSamples, 3, CV_64FC1 );

 for (int s=0; s<NrSamples; s++)
 {
  Vec3d v = ListSamplePoints.at(s);
  samples.at<double>(s,0) = (float) v[0];
  samples.at<double>(s,1) = (float) v[1];
  samples.at<double>(s,2) = (float) v[2];
 }
 //
 cout << "Learning to represent the sample distributions with " << NrGMMComponents << " gaussians." << endl;
 cout << "Started GMM training" << endl;

 Ptr<cv::ml::EM> em_model;
 cv::ml::EM::Params params(NrGMMComponents,cv::ml::EM::COV_MAT_GENERIC);

 Mat labels(NrSamples,1,CV_32SC1);
 Mat logLikelihoods( NrSamples, 1, CV_64FC1 );

 // Train classifier
 em_model=cv::ml::EM::train(samples,logLikelihoods,labels,noArray(),params);
 cout << "Finished GMM training" << endl;

 // result image
 Mat img  = Mat::zeros( Size( srcWidth, srcHeight ), CV_8UC3 );

 // predict cluster
 Mat sample( 1, 3, CV_64FC1 );

 Mat means=em_model->getMeans();

 for(int i = 0; i < img.rows; i++ )
 {
  for(int j = 0; j < img.cols; j++ )
  {
   Vec3b v=src.at<Vec3b>(i,j);
   sample.at<double>(0,0) = (float) v[0];
   sample.at<double>(0,1) = (float) v[1];
   sample.at<double>(0,2) = (float) v[2];
   int response = cvRound(em_model->predict( sample ));
   img.at<Vec3b>(i,j)[0]=means.at<double>(response,0);
   img.at<Vec3b>(i,j)[1]=means.at<double>(response,1);
   img.at<Vec3b>(i,j)[2]=means.at<double>(response,2);
  }
 }

 img.convertTo(img,CV_8UC3);
        namedWindow("result",WINDOW_AUTOSIZE);
 imshow("result",img);
        imwrite("D:\\nfr.jpg",img);
 waitKey();
 dst=img;
}

void processLayer(Mat& src,Mat& dst)
{
 Mat tmp=src.clone();
 Mat gx,gy,mag,blurred;
 Sobel( src, gx, -1, 1, 0, 3);
 Sobel( src, gy, -1, 0, 1, 3);
 magnitude(gx,gy,mag);
 //GaussianBlur(mag,blurred,Size(3,3),2);
 //mag+=blurred;
 normalize(mag,mag,0,1,cv::NORM_MINMAX);
 //sqrt(mag,dst);
 dst=mag.clone();
 normalize(dst,dst,0,1,cv::NORM_MINMAX);
}

int main(int ac, char** av)
{
 Mat clusterized;
 Mat frame=imread("image path"); ////load an image//////
        //resize(frame,frame,Size(256,256),0,0,INTER_LINEAR);
 clustrize_colors(frame,clusterized);
 clusterized.convertTo(clusterized,CV_32FC3,1.0/255.0);
 frame.convertTo(frame,CV_32FC3,1.0/255.0);
 Mat result1;
 vector<Mat> ch;
 split(frame, ch);

 processLayer(ch[0],ch[0]);
 processLayer(ch[1],ch[1]);
 processLayer(ch[2],ch[2]);

 merge(ch,result1);

 result1=(0.5*frame-0.9*result1+0.3*clusterized)*2.0;
        namedWindow("result1",WINDOW_AUTOSIZE);
 imshow("result1",result1);
        //cout<<result1;
        imwrite("D:\\finalresult.jpg",result1);
 waitKey(0);
 //destroyAllWindows();
 return 0;
}</code>
Comment: You might consider putting in some timing instrumentation to identify which parts are slow.
Comment: Everything looks slow. But in the first loop, toughest part should be rand(), then %25, which are both expensive operations. Maybe per-generate a sampling map can help? In the 3rd loop, inlining em_model->predict might help, and it looks stupid to convert byte to float then to double then back to byte....
Comment: Yes; optimizing code is much easier and much more fruitful when you know what needs to be optimized.
Comment: in the third loop , inlining em_model->predict . I found the time consumption is more. But am unable to proceed with that so please can u alternative method for that loop
Answer: In all likelihood the bottleneck is opencv's cv::ml::EM::train method. Training a classifier is not an easy or simple task. The classification problem has not been solved conclusively. That's why there are large tradeoffs and differences in between algorithms, let alone across different problem spaces.
As for performance, if you insist on using EM, check the EM class documentation and possibly its parent classes to modify:

Max number of iterations for training and/or
Term criteria to stop training.

Due to using a third-party library, there is not much you can do that will increase speed but not sacrifice accuracy. On the other hand, the library is open-source, and it is likely reasonably well optimized. I don't recommend trying to optimize the actual library code.


Converting a char array into an structure similar to int argc and char** argv

Question: I have read a text line from a file, and I need to convert it to an structure similar to main function arguments. For example if the char array is char* text="There are books in the library." and I have below structure definition:
struct mystruct{
   int argc;
   char** argv;
};

if I have struct mystruct a, with the function foo(a, text) I end up with a.argc equal to 6, a.argv[0] equal to There, a.argv[1] equal to are, ... .
Is there any function or library in C I can use for this purpose? Since this conversion is done automatically for input arguments when we execute C main function.
Comment: The shell does the splitting and passes the split arguments to the `exec*()` functions which relay the split arguments to the called main program.  So, your best bet may be looking at the code of someone else's shell.  It's not dreadfully hard to write; if you record an `int argmax;` in the structure, you can even make it reasonably efficient.
Comment: If you make it `char[] text = ...` then you could go through the string with `strtok` and save the resulting pointers. (Maybe this is the first time I ever recommended strtok...)
Comment: You can also use `sscanf` to read strings delimited by whitespace in the input and then use `strdup` to duplicate the strings.
Comment: I think *argv[] will work
Comment: The conversion from command line arguments to `argc` and `argv` is done by the execution environment. There are no library functions to create that. However, as @MattMcNabb suggested, you can write a function to construct a `mystruct` given `text` using `strtok` and `malloc`. Shouldn't be too hard.
Answer: You could use strtok to do this split, like this:
struct split_result {
    int cnt;
    char *buf;
    char **strs;
};

int
split(const char *str, struct split_result *rst)
{
    int idx, str_num;
    char *buf, *sep, **strs;

    buf = strdup(str);
    if (buf == NULL) {
        perror("strdup");
        return -1;
    }

    str_num = 1;
    strs = malloc(str_num * sizeof(char *));
    if (strs == NULL) {
        perror("malloc");
        return -1;
    }

    sep = " \t";
    idx = 0;
    for (strs[idx] = strtok(buf, sep);
         strs[idx];
         strs[idx] = strtok(NULL, sep))
    {
        idx++;
        if (idx >= str_num) {
            str_num += 10;
            strs = realloc(strs, str_num * sizeof(char *));
            if (strs == NULL) {
                perror("realloc");
                return -1;
            }
        }
    }

    rst->cnt = idx;
    rst->strs = strs;
    rst->buf = buf;

    return 0;
}

Here is a test program and its output:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct split_result {
    int cnt;
    char *buf;
    char **strs;
};

int
split(const char *str, struct split_result *rst)
{
    int idx, str_num;
    char *buf, *sep, **strs;

    buf = strdup(str);
    if (buf == NULL) {
        perror("strdup");
        return -1;
    }

    str_num = 1;
    strs = malloc(str_num * sizeof(char *));
    if (strs == NULL) {
        perror("malloc");
        return -1;
    }

    sep = " \t";
    idx = 0;
    for (strs[idx] = strtok(buf, sep);
         strs[idx];
         strs[idx] = strtok(NULL, sep))
    {
        idx++;
        if (idx >= str_num) {
            str_num += 10;
            strs = realloc(strs, str_num * sizeof(char *));
            if (strs == NULL) {
                perror("realloc");
                return -1;
            }
        }
    }

    rst->cnt = idx;
    rst->strs = strs;
    rst->buf = buf;

    return 0;
}

int
main(void)
{
    int i, j;
    struct split_result rst;
    const char *msg[] = {
        "",
        " One",
        " One  Two",
        " One  Two Tree  ",
        "There are books in the library.",
        NULL,
    };

    for (i = 0; msg[i]; i++) {
        if (split(msg[i], &rst) < 0) {
            exit(EXIT_FAILURE);
        }

        printf("msg[%d] = >>%s<<\n", i, msg[i]);
        for (j = 0; j < rst.cnt; j++) {
            printf("cnt = %d: |%s|\n", j, rst.strs[j]);
        }

        free(rst.strs);
        free(rst.buf);
    }

    exit(EXIT_SUCCESS);
}

Output:
$ ./a.out
msg[0] = >><<
msg[1] = >> One<<
cnt = 0: |One|
msg[2] = >> One  Two<<
cnt = 0: |One|
cnt = 1: |Two|
msg[3] = >> One  Two Tree  <<
cnt = 0: |One|
cnt = 1: |Two|
cnt = 2: |Tree|
msg[4] = >>There are books in the library.<<
cnt = 0: |There|
cnt = 1: |are|
cnt = 2: |books|
cnt = 3: |in|
cnt = 4: |the|
cnt = 5: |library.|

Comment: If `realloc` fails, then it returns `NULL` which is assigned to `strs`. This will lose handle on the previously allocated memory causing memory leak. You should use a temporary variable to save `strs` before calling `realloc`.
Comment: @ajay If `realloc()` failed, then `split()` will return `-1` and the program will terminate. However, it is possible to add proper free statements before all those `return -1;` in this `split()` so when it returns `-1`, any memory regions it dynamically allocated are freed.
Answer: I presume the file have a number that specifies the number of arguments. if that is the case then create a loop and read each string until you encounter a delimiter ( again, assuming that you were told there is one ). If you don't have "the" number of arguments then add a member of type unsigned, in your struct, to keep track of the number of words read.
You really don't need to create a split_function, but it doesn't hurt if you decided to create one :)


Generating PHPUnit tests from array with endpoints

Question: For an application we use configuration files in which a large number of endpoint characteristics are determined (relations, fillables, visibles, roles, etc.) We would like to loop through these files and conduct automatic tests with PHPUnit, simply to see if we receive a response, if validation errors are being triggered, if the response is in line with the files, etc.
We load the configuration and perform the tests for each endpoint configuration:
<code>public function testConfigurationFiles()
{
    $config = resolve('App\Contracts\ConfigInterface');

    foreach ($config->resources as $resource=>$configuration) {
        foreach ($configuration->endpoints() as $method=>$rules) {
            $this->endpoint($method, $resource, $configuration);
        }
    }
}
</code>
After which we use a switch, to test each type of method differently (index, show, create, update, delete). In total this comes down to dozens of tests with hundreds of assertions.
However, if even one of these endpoints fails, the entire tests fails without showing explicit information what went wrong. Is there a way to automatically generate a "test{$resource}{$method}" method for each endpoint, so they will be handled like individual tests?
Besides these tests we also conduct units tests & e2e tests, so we are fully aware of the disadvantages of this way of testing.
Answer: After studying PHPUnit some more, I found my answer in dataProviders:
https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers
This way you can indicate a data provider for a method, which should return an array with all cases you want to iterate over.


Pixilated display, clicks not working on Toshiba Satellite

Question: I have an old toshiba satellite (A135-S2326) laptop I downloaded Ubuntu 14.04.03 and then booted it through a Usb. The problem is if it decides to actually boot and I get passed the log in screen. What happens is my screen is heavily pixelated and their is a system error. I can move my mouse but anytime I try to click something, nothing happens. So the only thing I can do is go to the terminal. The problem with the terminal is I'm not connected to my wi-if and I don't know how to, so I think that's why it's also not allowing me to install any package. If I type "Sudo apt-get install " it will say already installed and below that line will say could not find package. I've been trying to fix this and research this for the last 5 hours. I'm at the verge of breaking my laptop, this is my last resort so please help.....I'm new to this btw(if you can't tell)
Answer: I have a Toshiba Satellite A105-S4004 and it runs Lubuntu 14.04 just fine.  Ubuntu 14.04 may be too much for that old of a system (I used to run 12.04 2D Desktop on it and that was OK, but 3D desktop was problematic.) Try Lubuntu or Xubuntu.
Comment: What's the difference?
Comment: http://askubuntu.com/questions/690/what-is-the-difference-between-ubuntu-and-its-derivatives


Android - Intent in BroadcastReceiver lost String

Question: I want to use BroadcastReceiver to send notification with specific String. Here is my function in MainActivity:`
private void createAlarm (int i) throws ParseException {
        AlarmManager alarmManager = (AlarmManager) getSystemService(Context.ALARM_SERVICE);
        Intent intent = new Intent(this, AlarmReceiver.class);
        intent.putExtra("name", names.get(i));

        PendingIntent pendingIntent = PendingIntent.getBroadcast(this,
                (int) timeInMillis / 1000),
                intent,
                PendingIntent.FLAG_ONE_SHOT
        );

        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
            alarmManager.setExact(AlarmManager.RTC_WAKEUP, timeInMillis, pendingIntent);
        }
    }

BroadcastReceiver code:
public class AlarmReceiver extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
        String name = intent.getExtras().getString("name", "deafultValue");

        if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
            Intent notificationIntent = new Intent(context, MainActivity.class);
            TaskStackBuilder taskStackBuilder = TaskStackBuilder.create(context);
            taskStackBuilder.addNextIntentWithParentStack(notificationIntent);
            PendingIntent pendingIntent = taskStackBuilder.getPendingIntent(50, PendingIntent.FLAG_UPDATE_CURRENT);

            NotificationCompat.Builder builder = new NotificationCompat.Builder(context, "EventStartChannel")
                    .setSmallIcon(R.drawable.ic_stat_name)
                    .setContentTitle(name)
                    .setContentText(name)
                    .setColor(Color.argb(255, 255, 0, 0))
                    .setAutoCancel(true)
                    .setContentIntent(pendingIntent)
                    .setPriority(NotificationCompat.PRIORITY_HIGH)
                    .setSound(RingtoneManager.getDefaultUri(RingtoneManager.TYPE_NOTIFICATION));

            NotificationManagerCompat notificationManager = NotificationManagerCompat.from(context);
            notificationManager.notify((int) System.currentTimeMillis() / 1000, builder.build());
        }
    }
}

And manifest code:
        <receiver android:name=".OtherClasses.AlarmReceiver"
            android:enabled="true"
            android:exported="false" >

            <intent-filter>
                <action android:name="android.intent.action.BOOT_COMPLETED"/>
                <action android:name="android.intent.action.SEND" />
            </intent-filter>

        </receiver>

But sometimes, when alarm is set on one day later or more, extras in onReceive are empty and name get deafult value. I've tried passing string with bundle in extras and then was the same problem.
Answer: To send
Change this:
intent.putExtra("name", names.get(i));

To:
intent.putExtra(android.content.Intent.EXTRA_TEXT, names.get(i));

To get
Change this:
String name = intent.getExtras().getString("name", "deafultValue");

To:
String name = intent.getStringExtra(android.content.Intent.EXTRA_TEXT);

Comment: Sadly still same poblem. But thanks for trying to help.
Comment: `new Intent(context, MainActivity.class);` - I think this was the problem. Try using the same intent instead of creating a new one. `intent.setClass(context, MainActivity.class);`


How to get dynamic array's size in C?

Question: I'm trying to get size of dynamic array in C with sizeof.
I have structure and and make it array with 10 elements.
struct date{
int day;
int month;
int year;
};

void main(){
struct date *table;

table = (struct date*) malloc(sizeof(struct date) * 10);
}

So when I try to get size with sizeof I get 8 instead of 120 (since size of structure is 12).
printf("%d, sizeof(table));
output is 8

Instead of making it dynamic array i changed it to static array to see what happens and it gave 120. So I don't understand what is the problem. I know I cannot say sizeof(*table) because it will give size of first element. For the context purposes need to use dynamic array for expanding it's size for every new data.
Comment: A `sizeof` expression is evaluated during compilation time. It gives you the size of the type of the operand. In the case of a dynamic array, it gives you the size of a pointer, which is typically 4 or 8 bytes (depending on the underlying platform).
Answer: In your example table has type struct date *.  So when you use the sizeof operator on it you get the size of the pointer.
There is no portable way to determine how much available space a pointer to allocated memory points to.  You need to keep track of that yourself.
Comment: @kimboi Arrays are converted to a pointer to their first element when passed to a function, so you would also get the size of a pointer in that case.  You would need to pass the size to the function in a separate parameter.
Comment: will this apply to passing static arrays to functions? just for an example, if I have static array of struct date array[10] and want to print size of the array in another function, printSize(struct date a[]), will this case happen?
Answer: The only way is to store the size information somewhere like in this example:
typedef struct
{
    size_t size;
    int arr[];
}int_array_t;

int_array_t *alloc(size_t nelem)
{
    int_array_t *arr = malloc(sizeof(*arr) + nelem * sizoef(arr -> arr[0]));

    if(arr)
    {
        arr -> size = nelem;
    }
    return arr;
}

How to get dynamic array's size in C?