vbscript output to console

Question: What is the command or the quickest way to output results to console using vbscript?
Answer: You mean:
WScript.Echo "Like this?"

If you run that under wscript.exe (the default handler for the .vbs extension, so what you'll get if you double-click the script) you'll get a "MessageBox" dialog with your text in it. If you run that under cscript.exe you'll get output in your console window.
Comment: Unintuitively for me, `WScript.Echo` *must* be used for whether you're running via `WScript` or `CScript`. That is, there *is not* a `CScript.Echo`, in case future googlers wonder. (*Very* happy the msgboxes are gone [when run with `cscript`], however; thanks.)
Comment: Actually, I just found this `popup` method. Very similar to `echo` but allows you to specify a timeout after which it will automatically close the popup box. Very convenient and easy to use: https://technet.microsoft.com/en-us/library/ee156593.aspx
Comment: You can directly use on the wscript.exe the function `MsgBox("text")` or `MsgBox(object.property)` but `Wscript.Echo` is easier to write. Thanks.
Comment: @GabrielStaples - Not w/ the stock `WScript.Echo`. I suppose, if you wanted to stay totally within WScript you could do something horrifyingly dodgy like Exec'ing off another process to do a "SendKeys" to the parent process to close the MessageBox.
Answer: This was found on Dragon-IT Scripts and Code Repository.
You can do this with the following and stay away from the cscript/wscript differences and allows you to get the same console output that a batch file would have.  This can help if your calling VBS from a batch file and need to make it look seamless.
Set fso = CreateObject ("Scripting.FileSystemObject")
Set stdout = fso.GetStandardStream (1)
Set stderr = fso.GetStandardStream (2)
stdout.WriteLine "This will go to standard output."
stderr.WriteLine "This will go to error output."

Comment: @BernhardHiller has a valid point. The thrust of this answer is that using stdout directly would avoid the CScript/WScript differences. That is incorrect. This solution still only works under CScript.exe, so there doesn't seem to be much benefit over just using `WScript.Echo`. In fact, the difference is magnified, because the script will no longer run at all under WScript. Its a valid technique that has its uses, for example if one needs to write to StdErr, but in the context of this answer, it is misleading.
Comment: @Bernhard: You are getting this error if you run the script using wscript.exe. Wscript is windows-oriented and has no console streams. Use cscript.exe instead: http://technet.microsoft.com/en-us/library/bb490816.aspx
Comment: If the script is started by double-click and thus opened with wscript, the script results in an error message: "Invalid Handle".
Comment: I just want to light up the benefit of this method over `WScript.Echo`:

`cscript //b foobar.vbs`

Runs `foobar.vbs` without any console output, but by Rob's method you can have output even when passing `\\b` to `cscript.exe`
Answer: You only need to force cscript instead wscript. I always use this template. The function ForceConsole() will execute your vbs into cscript, also you have nice alias to print and scan text.
 Set oWSH = CreateObject("WScript.Shell")
 vbsInterpreter = "cscript.exe"

 Call ForceConsole()

 Function printf(txt)
    WScript.StdOut.WriteLine txt
 End Function

 Function printl(txt)
    WScript.StdOut.Write txt
 End Function

 Function scanf()
    scanf = LCase(WScript.StdIn.ReadLine)
 End Function

 Function wait(n)
    WScript.Sleep Int(n * 1000)
 End Function

 Function ForceConsole()
    If InStr(LCase(WScript.FullName), vbsInterpreter) = 0 Then
        oWSH.Run vbsInterpreter & " //NoLogo " & Chr(34) & WScript.ScriptFullName & Chr(34)
        WScript.Quit
    End If
 End Function

 Function cls()
    For i = 1 To 50
        printf ""
    Next
 End Function

 printf " _____ _ _           _____         _    _____         _     _   "
 printf "|  _  |_| |_ ___ ___|     |_ _ _ _| |  |   __|___ ___|_|___| |_ "
 printf "|     | | '_| . |   |   --| | | | . |  |__   |  _|  _| | . |  _|"
 printf "|__|__|_|_,_|___|_|_|_____|_____|___|  |_____|___|_| |_|  _|_|  "
 printf "                                                       |_|     v1.0"
 printl " Enter your name:"
 MyVar = scanf
 cls
 printf "Your name is: " & MyVar
 wait(5)

Comment: Best solution ever, thank you, but only the "ForceConsole" function matters, the "printf" and the rest are totally unnecessary since if you force to close the current script on Wscript.exe instance and then run a new cscript.exe instance of the current script, then Wscript.Echo will output to that console instance...
Comment: Are you sure that answers the [actual question](http://stackoverflow.com/questions/4388879/vbscript-output-to-console)?
Comment: Yes, only call the ForceConsole() and then use printf() to print text in the output console. Also you have other alias to clear screen, scan text and wait (sleep)
Answer: There are five ways to output text to the console:
Dim StdOut : Set StdOut = CreateObject("Scripting.FileSystemObject").GetStandardStream(1)

WScript.Echo "Hello"
WScript.StdOut.Write "Hello"
WScript.StdOut.WriteLine "Hello"
Stdout.WriteLine "Hello"
Stdout.Write "Hello"

WScript.Echo will output to console but only if the script is started using cscript.exe. It will output to message boxes if started using wscript.exe.
WScript.StdOut.Write and WScript.StdOut.WriteLine will always output to console.
StdOut.Write and StdOut.WriteLine will also always output to console. It requires extra object creation but it is about 10% faster than WScript.Echo.
Comment: ... and as said in a comment to a previous answers, this doesn't work when executing with wscript.exe: https://stackoverflow.com/questions/4388879/vbscript-output-to-console#comment26762213_12865458
Comment: Also found an explanation about GetStandardStream() vs WScript.StdIn/.StdOut/.StdErr : "VBScript in a Nutshell: A Desktop Quick Reference (2nd Edition)" https://books.google.fr/books?id=NLpuZSatG3QC page 298 says it's "functionnaly equivalent".
Comment: When you say "this doesn't work", can you specify which of the five method does not work and what you mean by not working?
Answer: I came across this post and went back to an approach that I used some time ago which is similar to @MadAntrax's.
The main difference is that it uses a VBScript user-defined class to wrap all the logic for switching to CScript and outputting text to the console, so it makes the main script a bit cleaner.
This assumes that your objective is to stream output to the console, rather than having output go to message boxes.
The cCONSOLE class is below. To use it, include the complete class at the end of your script, and then instantiate it right at the beginning of the script. Here is an example:
    Option Explicit

'// Instantiate the console object, this automatically switches to CSCript if required
Dim CONS: Set CONS = New cCONSOLE

'// Now we can use the Consol object to write to and read from the console
With CONS

    '// Simply write a line
     .print "CSCRIPT Console demo script"

     '// Arguments are passed through correctly, if present
     .Print "Arg count=" & wscript.arguments.count

     '// List all the arguments on the console log
     dim ix
     for ix = 0 to wscript.arguments.count -1
        .print "Arg(" & ix & ")=" & wscript.arguments(ix)
     next

     '// Prompt for some text from the user
     dim sMsg : sMsg = .prompt( "Enter any text:" )

     '// Write out the text in a box
     .Box sMsg

     '// Pause with the message "Hit enter to continue"
     .Pause

End With

'= =========== End of script - the cCONSOLE class code follows here

Here is the code for the cCONSOLE class
     CLASS cCONSOLE
 '= =================================================================
 '=
 '=    This class provides automatic switch to CScript and has methods
 '=    to write to and read from the CSCript console. It transparently
 '=    switches to CScript if the script has been started in WScript.
 '=
 '= =================================================================

    Private oOUT
    Private oIN

    Private Sub Class_Initialize()
    '= Run on creation of the cCONSOLE object, checks for cScript operation

        '= Check to make sure we are running under CScript, if not restart
        '= then run using CScript and terminate this instance.
        dim oShell
        set oShell = CreateObject("WScript.Shell")

        If InStr( LCase( WScript.FullName ), "cscript.exe" ) = 0 Then
            '= Not running under CSCRIPT

            '= Get the arguments on the command line and build an argument list
            dim ArgList, IX
            ArgList = ""

            For IX = 0 to wscript.arguments.count - 1
                '= Add the argument to the list, enclosing it in quotes
                argList = argList & " """ & wscript.arguments.item(IX) & """"
            next

            '= Now restart with CScript and terminate this instance
            oShell.Run "cscript.exe //NoLogo """ & WScript.ScriptName & """ " & arglist
            WScript.Quit

        End If

        '= Running under CScript so OK to continue
        set oShell = Nothing

        '= Save references to stdout and stdin for use with Print, Read and Prompt
        set oOUT = WScript.StdOut
        set oIN = WScript.StdIn

        '= Print out the startup box
            StartBox
            BoxLine Wscript.ScriptName
            BoxLine "Started at " & Now()
            EndBox

    End Sub

    '= Utility methods for writing a box to the console with text in it

            Public Sub StartBox()

                Print "  " & String(73, "_")
                Print " |" & Space(73) & "|"
            End Sub

            Public Sub BoxLine(sText)

                Print Left(" |" & Centre( sText, 74) , 75) & "|"
            End Sub

            Public Sub EndBox()
                Print " |" & String(73, "_") & "|"
                Print ""
            End Sub

            Public Sub Box(sMsg)
                StartBox
                BoxLine sMsg
                EndBox
            End Sub

    '= END OF Box utility methods

            '= Utility to center given text padded out to a certain width of text
            '= assuming font is monospaced
            Public Function Centre(sText, nWidth)
                dim iLen
                iLen = len(sText)

                '= Check for overflow
                if ilen > nwidth then Centre = sText : exit Function

                '= Calculate padding either side
                iLen = ( nWidth - iLen ) / 2

                '= Generate text with padding
                Centre = left( space(iLen) & sText & space(ilen), nWidth )
            End Function

    '= Method to write a line of text to the console
    Public Sub Print( sText )

        oOUT.WriteLine sText
    End Sub

    '= Method to prompt user input from the console with a message
    Public Function Prompt( sText )
        oOUT.Write sText
        Prompt = Read()
    End Function

    '= Method to read input from the console with no prompting
    Public Function Read()
        Read = oIN.ReadLine
    End Function

    '= Method to provide wait for n seconds
    Public Sub Wait(nSeconds)
        WScript.Sleep  nSeconds * 1000
    End Sub

    '= Method to pause for user to continue
    Public Sub Pause
        Prompt "Hit enter to continue..."
    End Sub

 END CLASS

Comment: "_include the complete class at the end of your script_" isn't necessary when using a [WSF](https://www.vbsedit.com/html/c999e0fa-3986-41be-90a9-3f8ae3a38253.asp). See also [Etalon's answer to _How do I include a common file in VBScript (similar to C #include)?_](https://stackoverflow.com/a/12374208/1744774)
Answer: Create a .vbs with the following code, which will open your main .vbs:
Set objShell = WScript.CreateObject("WScript.shell")
objShell.Run "cscript.exe ""C:\QuickTestb.vbs"""

Here is my main .vbs
Option Explicit
Dim i
for i = 1 To 5
     Wscript.Echo i
     Wscript.Sleep 5000
Next

Comment: Isn't `Sleep` a bit counter-intuitive if OP asks for the *quickest way*?
Comment: @Wolf Perhaps OP meant _quickest to write_. OK, agreed, not even that applies. :)
Answer: You can run this script to echo in the command line
set wshShell = createObject("wscript.shell")
    wshShell.run"cmd.exe /c echo something",1

Answer: For console output I use a snippet that quits elegantly if run in the wrong run-time. (And print is shorter to type...)
Sub print(s)
  On Error Resume Next
  WScript.stdout.WriteLine (s)
  If  err= &h80070006& Then WScript.Echo " Please run this script with CScript": WScript.quit
End Sub

print "hello"

vbscript output to console


get access_token for a Twitter application which does not have a callback URL using Python

Question: I have created an application with its "callback URL" field as blank. I want to programatically use access_token function of oauth endpoint (https://dev.twitter.com/docs/api/1/post/oauth/access_token) and fetch oauth_token and oauth_token_secret.
I am able to do this from my browser my going to my applications and then generating the access token by click of a button. However, I want to do this automatically via a python program. https://dev.twitter.com/docs/api/1/post/oauth/access_token states that "callback URL" is a mandatory flied. Keeping it blank is not working as results in "401 Unauthorized Failed to validate oauth signature and token" response.
I do not want to use any twitter API wrappers or oauth library.
I have tried the following code -
<code>    import httplib
    import uuid
    from time import time

    host = 'api.twitter.com'
    url = "/oauth/request_token"
    req = httplib.HTTPSConnection(host)
    req.putrequest("GET", url)
    req.putheader("Host", host)

    uid = uuid.uuid4()
    req.putheader("oauth_consumer_key" , consumer_key)
    req.putheader("OAuth oauth_nonce" , uid.hex)
    req.putheader("oauth_signature_method" , "HMAC-SHA1")
    req.putheader("oauth_timestamp" , str(time.time()))
    req.putheader("oauth_version" , "1.0")
    req.putheader("oauth_callback" , "")
    req.putheader("oauth_signature" , consumer_secret)

    req.endheaders()
    resp = req.getresponse()
    data = resp.read()
    print resp.status, resp.reason
    print data
</code>
This returns "401 Unauthorized. Failed to validate oauth signature and token"
How do I access the my oauth_token and oauth_token_secret for the application which I have created?
Answer: Check out this thread - if you want to implement the entire oAuth2 handshake.
Otherwise I am using tweepy, which will take care of that for you.


How do i add social buttons to my github page using the automatic page generator?

Question: I'd like to add social buttons (share on twitter, like on Facebook, share on google+) to a github page, which was created using the automatic page generator. Is there an easy way to do that?
Answer: as far as i know there's no automated way. but you can do it by editing the html of the github page.
the automatic page generator creates a 'gh-pages' branch with the html, css & js files for the github page you created. you can checkout this branch, edit it as normal and push it back to github.


How to order a dictionary by its values without changing it to a list of tuples

Question: I am currently working on a program for my robotics team the prints out a team key(a string) and a value that ranks how good it is. So the current output I have is a dictionary with the team key as the key and the value as the value. However the dictionary is not sorted. I have read that you can change it to a list of tuples which I have tried and worked, but I have just 1 issue. Currently the team key has 'frc' in front of the team number, and I remove it by doing [3:]. However when I do the same for the list of tuples, it doesn't work. So I was wondering if there was a way to sort the dictionary by its values without changing it to a list of tuples?
Comment: If you actual have a reference to the string, the slicing with `[3:]` would work too. I'm guessing you aren't unpacking the `tuple`s as you go, so you're trying to slice the wrong thing.
Comment: Can you break the problem down with example?
Comment: Did you reject OrderedDict? (https://docs.python.org/3/library/collections.html#ordereddict-examples-and-recipes)
Answer: You could add items to an ordered dict after sorting: https://docs.python.org/2/library/collections.html#collections.OrderedDict
But if you wanted to use a tuple, changing a value in one won't work as they're immutable. However, you could slice before creating it.
(An extension of the top answer here: Python 3 sort a dict by its values)
s = [(k[3:], d[k]) for k in sorted(d, key=d.get, reverse=True)]
Answer: mydict = {'frc1':40,'frc2':2,'frc3':1,'frc4':3}

for key, value in sorted(mydict.iteritems(), key=lambda (k,v): (k[3:], v)):
    print "%s: %s" % (key, value)

Output:
frc1: 40
frc2: 2
frc3: 1
frc4: 3

This should work, if I undestand your problem correctly :)
lambda functions are quite magical


Dented can of iced tea fizzed when opened

Question: I just opened a can of iced tea that got dented about a week ago I think from hitting the ground. It fizzed a bit when opened, like soda sort of and sprayed on my hand, after that it was flat. It's apparently not carbonated (Brisk). This is not a botulism risk is it? I put it down the drain but not sure how much cleaning up I need to do. I would guess that it has enough sugar but not sure. Or could it be fermentation from a possible tiny break in the lid? Thanks!
Answer:
It fizzed a bit when opened, like soda sort of and sprayed on my hand, after that it was flat.

I see this happen all the time with cans of uncarbonated drinks, e.g. Arizona iced tea. Even though the drink isn't carbonated, the cans seem to be pressurized to some degree: an unopened can is quite firm and hard to squeeze, while cans that have been opened even just a tiny bit yield easily. Also, you can hear the hiss of escaping gas as you open the can.
I've also seen these cans spray somewhat when they're opened, mostly after they've been shaken. My guess here is that shaking the can mixes the gas and liquid, and when you open the can the bubbles of gas in the liquid expand quickly as the pressure drops. I've never seen a can of tea spray nearly as much as a vigorously shaken can of soda would, but it enough to wet your hand a bit or spread a few drops on nearby surfaces.

This is not a botulism risk is it?

I doubt it -- botulism does not like acidic environments, and one source I found puts the pH of Lemon Brisk at 2.86, which is pretty acidic.

I put it down the drain but not sure how much cleaning up I need to do.

I'd wipe the area with a damp towel to remove the sugar, which will make things sticky as it dries. Other than that, I think you're pretty safe.
Comment: I was a bit thrown by "liking low-acid" and then a reference to an acidic, which is low pH, so I edited to "does not like acidic" instead of likes low-acid, just seemed a bit clearer. I understand how pH works, but thought this seemed a bit more straightforward.  Feel free to roll-back.
Comment: @PoloHoleSet Either way works for me -- discussing pH is like discussing f-numbers in photography... the fact that a lower numbers mean a larger quantity of the thing you're talking about is always bound to confuse someone.
Comment: Thank you! So not a problem putting it down the drain I hope.
Comment: I just thought the two "lows" meaning opposite things might make people have to stop and re-read.


LED bulb won't work in a standard outside porch light fixture

Question: I can't get LEDs to work together in twin wired Port Charlotte lights. I can put in an incandescent light with an LED bulb and just the led works. I put both LEDs in and neither works. I put both incandescent lights in and they both work. Power shows present on voltage meter. Could it be the switch? I have a standard light switch from 50 years ago technology
Comment: What does "twin wired Port Charlotte lights" mean? Is "Port Charlotte" a brand or a style or a location? What is "twin wired"? Maybe include a picture, too.
Comment: Are the LED bulbs of the same brand?
Comment: Is this a dusk to dawn controlled fixture? Or an electronic switch ?
Comment: *Power shows present on voltage meter.* What voltage are you reading at either socket with the meter?
Comment: Try a small value (25, 40  watt) incandescent, and a large-value (75, 100W) incandescent as the two bulbs.  Is the small-value one brighter? That would indicate series wiring.
Answer: Given the symptoms, I have to suspect that this fixture is wired with the bulbs in series, rather than with the bulbs in parallel.
So, two incandescents - operate happily if somewhat dim and yellow on half-voltage. Bulbs would last a long time and veer towards "bug light" yellow without the ugly yellow paintjob, so it's not unreasonable for a "porch light" fixture.
One LED one incandescent - adequate voltage for the LED, so little current you don't even notice the glow on the incandescent.
Two LEDs - half voltage, ain't nobody happy.
Diagnostic - one bulb, one empty socket - what happens?
Comment: I've never seen that in a fixture. Would you assume that it's a manufacturing flaw?
Comment: Not necessarily. Could have been designed that way, with the long life and bug-lite/cheerful yellow intent; could have (depending on fixture design) been field-wired that way (which could mean it's correctable for "not 60 volt" LEDs to work in it.)

LED bulb won’t work in a standard outside porch light fixture


How can I browse through Homebrew's formulae according to popularity (# downloads all time)?

Question: Though I can easily find an alphabetical list of all available formulae, there seems to be no resource online, including at Brew's Github, or seemingly anywhere where I'm able to browse through Homebrew formulae non-alphabetically, using other sort methods (# downloads all time, for example). Neither does any resource exist out there that provides categorizations of them (compiling tools, essential formulae, for fun, etc.). Is anyone aware of any resources that exist out there? Thanks in advance.
Answer: There are some vague or basic categorizations available under the "Alternative Repositories" section of http://braumeister.org/ - though certainly not nearly as comprehensive as what you're suggesting.
While certain categorization and sort methods you mention sound extremely useful, I have a feeling they are a quite challenging to implement in a very automated approach. Sounds like a great challenge for someone.


Remove commas and quote characters between quotes using csv_xs perl

Question: I would like suggestions on how to remove comas and quotes within the quoted filed
Ex:
"11","a"bc","s"am,","t"om,"","15"
In the example above for the value a"bc, "" should be removed, for s"am, " and , should be removed and for t"om," " , " should be removed. I am using the following code with CSV_XS module but it is not working.
#!/usr/bin/perl -w
#
use strict;
use Text::CSV_XS;

# Read the input filename from the command line
my $file = shift or die "Usage: $0 <csv_filename>\n";

# instantiate the CSV parser
my $csv = Text::CSV_XS->new ({ binary => 1 }, escape_char         => '"',allow_loose_quotes  => 1);

# open the input file for read
open my $inputFH,  "<", $file or die "$file: $!";

# open the output file for write
open my $outputFH, ">", "$file.out" or die "$file.out: $!";

my @out;             # declare variables outside the loops for better performance
my $outputRow;
my $inputRow;
while ($inputRow = $csv->getline($inputFH)) { # iterate over each row in the input file
  @out=();           # empty out the array which will hold our corrected fields
  foreach (@$inputRow) {   # iterate over each field in the input row
    s/[\0\|\n\r]//g;         # get rid NUL, pipe, CR, and LF characters
    s/\s+/ /g;               # change multi-whitespace to single
    push(@out,$_);         # push the corrected field on to an array
    print "Loop"
  }
  $outputRow = join('|',@out);  # create a pipe-delimited line from the corrected array

  $outputRow =~ s/^\s+//;  # trim leading whitespace
  $outputRow =~ s/\s+$//;  # trim trailing whitespace

  print $outputFH "$outputRow\n";
}

Answer: The docs say it should be able to handle your severly broken CSV by using allow_loose_quotes set to 1 and escape_char set to anything but ".


Given historical states and action/reward data, is reinforcement learning a reasonable approach?

Question: To summarize my problem:

I want to maximize my total reward over all timesteps
I have 3 discrete actions at each time step.
The state vector for each time step has 5 features. The features are numeric and continuous.
The rewards for each action change at each timestep.
The reward values are not known before the action is made.

But we will know what the rewards would have been for all actions before the next action needs to be taken. (I believe this means we can do on-policy training).

Edit: Selecting an action, shouldn't affect the next state or the future rewards.
I believe that selecting the best action is quite hard.

In my experiments, it is hard to beat a very simple strategy of picking an action based on some threshold of the first state/input feature, i.e;

$$
\text{action}_t = \begin{cases}
1 & \text{if } x_{1,t} < 0.1 \\
2 & 0.1 \leq x_{1,t} < 2.3 \\
3 & x_{1,t} \geq 2.3
\end{cases}
$$
Here, $x_1$ is probably the most important feature for deciding which action is superior. The other features are important for estimating the size of the reward.
I think it is very hard to predict the best action in any individual step. I want to find a (simple) policy that has good performance over many steps in reward space, not classification rate/accuracy space.
I have N>10,000 historical data points. My state data is a numpy array with shape (N, 5) and looks like this:
array([[ 0.091     , -0.2189    ,  0.97724431,  3.908     ,  3.999     ],
       [ 0.22      ,  0.091     ,  0.94358974,  3.68      ,  3.9       ],
       [-0.22      ,  0.22      ,  1.05978261,  3.9       ,  3.68      ],
       ...,
       [ 0.34      ,  0.3       ,  0.96617286,  9.7111    , 10.0511    ],
       [-0.6876    ,  0.34      ,  1.06875175, 10.6888    , 10.0012    ],
       [ 0.3988    , -0.6876    ,  0.96051485,  9.7012    , 10.1       ]])

My reward data is an array with shape (N, 3) and looks like this:
array([[ 35055,  35631,  27413],
       [ 37475,  36726,  24968],
       [ 36908,  36315,  24974],
       ...,
       [112152, 109845,  93747],
       [101289, 102263,  94943],
       [ 96325,  97717,  91522]])

I have tried posing this as a regular multiclass classification problem and whilst it may achieve better classification performance than a simple lag-1 strategy, it doesn't achieve as good cumulative reward. I believe this is because of the non-linearity between the (multiclass) logloss/accuracy metrics that the models optimise for and the actual reward space which is what we actually care about (I think Taleb describes this well here).
I have also tried posting this as a multi output regression problem where I try to predict the reward value of each of the actions. This doesn't work that well either, it has no concept of maximising reward over time instead of each action. It fails to take the uncertainty of the estimates/predicted rewards into account.
I have constructed a gym using the Gymnasium Python library that captures the problem.
class CustomEnv(gym.Env):
    def __init__(self, state_data, reward_data):
        super(CustomEnv, self).__init__()

        # Define the action and observation space
        self.action_space = spaces.Discrete(3)
        self.observation_space = spaces.Box(low=-np.inf, high=np.inf, shape=(5,), dtype=np.float32)

        # Store the state and reward data
        self.state_data = state_data
        self.reward_data = reward_data

        # Ensure the state and reward data have the same length
        assert self.state_data.shape[0] == self.reward_data.shape[0]

        # Initialize the current step
        self.current_step = 0

    def step(self, action):
        # Ensure action is valid
        assert self.action_space.contains(action)

        # Get the reward for the action at the current step
        reward = self.reward_data[self.current_step, action]

        # Move to the next step
        self.current_step += 1

        # Check if the episode is done
        done = self.current_step == self.state_data.shape[0]

        # Get the next state
        next_state = self.state_data[self.current_step] if not done else np.zeros(self.state_data.shape[1])

        return next_state, reward, done, {}

    def reset(self):
        # Reset the environment to the beginning
        self.current_step = 0
        return self.state_data[self.current_step]

    def render(self, mode='human', close=False):
        # This environment doesn't have a visual representation, but you could add one if you want.
        pass

My questions:
I believe that that must be a better, more systematic way to find a 'good policy' than the ad-hoc policy I can come up with myself and described above...

Is reinforcement learning the best way to approach this problem?
If so, what methods or algorithms would be the best to try first?

As far as I can tell, tabular Q learning won't work because I have continuous inputs.
Given the noisy reward behaviour, and from my own experiments, I imagine a very simple model would work best.

Have I set up my gym environment correctly?
What other tools/methods should be considered? The simpler the better!
Answer: If you can visit all states, dynamic programming (DP) will often find the optimtal solution.
A slightly more complex optional is value function approximation (VFA). It can be impossible to perform tabular Q-learning on continuous data. VFA replaces a look-up table with a function. If you have enough data, it is possible to use deep learning within VFA.
Comment: How does dynamic programming make use of the continuous state information and the variability of the reward? Can you provide any links or examples?

With VFA, do things essentially boil down to a regular supervised learning problem? Especially if actions do not influence future rewards and states?


How to protect server from simple DoS attack

Question: Please note I am NOT asking how to protect server from DDoS. What I mean is that if someone runs siege -c 1000 www.example.com on his VPS. That will result in my server serving thousands of requests and slowing down. If I run siege targeted to say, shared hosting, I don't see the extreme drop in responsivness (from 60ms to 2000ms/request) I do on my machine. I don't expect to be able to mitigate big DDoS, but I'd like to be safe from the first script-kiddie discovering siege :) On the other hand I'm OK with the fact that when big amounts of traffic comes from legitimate sources, the response may be slower.
Comment: You're not considering differences in latency between a "local" DoS on your own machine and one across the internet on shared hosting.
Comment: I used a VPS from different provider to issue the "attack" - so, yes I am... I used it both on my server and on shared server...
Answer: You can use a firewall to limit the number of concurrent connections and the rate of new connections coming from a network (e.g. a /32 for IPv4 and a /64 for IPv6). Example of what it may look like using iptables:
# Limit number of concurrent connections
-A INPUT -i eth0 -p tcp --syn -m connlimit --connlimit-above 50 -j DROP
# Limit rate of new connections
-A INPUT -i eth0 -p tcp --syn -m hashlimit --hashlimit-name tcp --hashlimit-mode srcip --hashlimit-above 3/sec --hashlimit-burst 7 --hashlimit-srcmask 32 -j DROP

(Same thing for ip6tables except adding --connlimit-mask 64 to the first and changing --hashlimit-srcmask to 64 in the second.)
You can also limit the rate of HTTP requests, for example with the limit_req module of nginx.
Comment: Just to make it absolutely clear, *please make sure* to use the `DROP` target and not `REJECT` for this. DROP will drop the packets and a sane TCP/IP implementation on a legitimate host will retry after some short delay, whereas REJECT will cause a connection reset on the remote end and cause a large "the web server refused the connection" to be displayed to the user (almost certainly not what you want).
Answer: Aside from some basic firewalling, we rely on haproxy to do the heavy TCP lifting. We have found that it runs circles around an out of the box Apache HTTPd instance as far as any kind of slowdos attack goes.
-A INPUT -p tcp -m tcp -m state --state RELATED,ESTABLISHED -j ACCEPT

-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -j SYNFLOOD

-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j HTTPDGUARD
-A INPUT -p tcp -m tcp --dport 443 -m state --state NEW -j HTTPDGUARD

-A HTTPDGUARD -m connlimit --connlimit-above 25 --connlimit-mask 32 -j HTTPDENY
-A HTTPDENY -j LOG --log-prefix "HTTP Flood: "
-A HTTPDENY -p tcp -m tcp -j REJECT --reject-with tcp-reset

-A SYNFLOOD -m state --state NEW -m recent --set --name SYNRATE --rsource
-A SYNFLOOD ! -s 18.104.22.168/24 -m state --state NEW -m recent --update \
   --seconds 5 --hitcount 200 --name SYNRATE --rsource -j DROP

-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -m state --state NEW -j ACCEPT

But basically, you either need to block the attack, fix the exploit, or do both.
Answer: fail2ban is one of the many tools that are designed to handle such situation. fail2ban worked for me in stopping couple of brute force attack on ssh. I assume that your server is a linux.


Do any tracking networks or ad networks not use JavaScript and only track based on IP address? If yes, how can they be prevented?

Question: Are there any websites that collectively track one computer based on its IP address alone? Browser extensions such as Ad block plus, Do Not Track Plus and Ghostery modify the client side script but is it not conceivable that a server would just associate your IP address with the page you requested and share it with some network? For example if you search for a suit on a site hosted by server A and server A passes along your search information and IP to other servers in its tracking network, such as server B, then you go to a page hosted by server B and you start seeing targeted advertisements for suits. Even with scripts completely disabled this wouldn't be prevented against, is there any way to prevent against this? Is this scenario, or any similar, happening?
Answer: It is most likely that some ad network uses this method. To be honest I don't know, but I'm going to answer your second question.
How can they be prevented?
1) Change your IP: You can use a vpn or a proxy or TOR to change your ip. But you have to trust your data with the VPN service provider or host.
2) Confuse them: You can confuse them with an automated bot that browses random pages of the sites that use the ad network, giving them false information about your interests.


What is the viscosity of Sun surface?

Question: It should be possible to compute value from hydrodynamics of Solar prominence. What is the current estimate of Solar surface viscosity?
Comment: Please be more precise. What do you mean by "surface"?
Comment: This is an astrophysics question, not a physics question; it would be better asked on http://astronomy.stackexchange.com .
Comment: Yes, I am asking about the surface of photosphere.
Answer:
What is the viscosity of Sun surface?

By "solar surface" I am going to assume you mean the photosphere.  In that case, then the question is whether you are asking about dynamic ($\mu$) or kinematic ($\nu$) viscosity.
According to a paper by Cowley [1990], the relevant parameters for the photospheric surface are: T ~ 4700 K, $\rho$ ~ 4.9 x 10-9 g/cm3, and ne ~ 1.1 x 1011 cm3.
We know that the following is an okay approximation $\mu \propto \rho \ \lambda \ V_{T}$, where $\lambda$ is the mean free path and $V_{T}$ is the thermal speed.  One can approximate $\lambda \propto (n_{e} \ \sigma)^{-1}$, where $\sigma$ is the collisional cross-section (here Coulomb collisions).
From Table 2 in Cowley [1990], we find that $\mu$ ~ 4.1 x 10-4 g cm-1 s-1 and since we know $\rho$ we can see that $\nu$ = $\mu/\rho$ ~ 8.4 x 104 cm2 s-1.

What is the current estimate of Solar surface viscosity?

I imagine things have not changed much other than more accurate density and temperature constraints for various regions.  However, any model that attempts to make things more accurate will quickly become nearly impossible without simulations (e.g., adding multiple ion species with energy-dependent cross-sections is not a trivial task when they have different temperatures and densities).
Comment: "ρ ~ 4.9 x 10^-9 kG/L " Didn't realize that photosphere is so thin. Seems like ideal gas would give good approximation. Thank you.
Comment: @Stepan - To be fair to the photosphere, it gets denser as one goes deeper rather quickly (i.e., exponentially).  ;)

What is the viscosity of Sun surface?


Export multiple tierprices of single product

Question: I'm export/importing products including the tier price to a CSV file:
<code>foreach($products as $productid) {
    $existingTierPrice = $product ->load($productid)->tier_price;

    foreach($existingTierPrice as $key=>$value) {
        $tierarray = array("all","all",$value['price_qty'],$value['price']);
        print stripslashes(implode('^',$tierarray)) . "\n";
    }
}
</code>
This works great but what to do when a product has multiple tier prices?
So in other words, how can i use the above code and make sure multiple tierprices of a product get exported and the tierprices can be edited and imported again into Magento.
Answer: Please try bellow code
<code>$productId = 1;
$product = Mage::getModel('catalog/product')->load($productId);
$insetTierPrice[] = array(
    'website_id' => 1,
    'cust_group' => Mage_Customer_Model_Group::CUST_GROUP_ALL,
    'price_qty'  => 2,
    'price'      => 2
);
$insetTierPrice[] = array(
    'website_id' => 1,
    'cust_group' => Mage_Customer_Model_Group::CUST_GROUP_ALL,
    'price_qty'  => 3,
    'price'      => 3
);
$product->setData('tier_price', $insetTierPrice);
try {
    $product->save();
} catch (Mage_Core_Exception $e) {
    $e->getMessage();
}
</code>
Comment: Thanks for this, but what i mean is how can i use my code and make sure multiple tierprices of 1 product get exported and the tierprices can ben edited and imported again into Magento


Closed form of the sequence $a_{n+1}=a_n^2+1$

Question: If $$a_{n+1}=a_n^2+1,$$ with initial $a_1=\frac{1}{2}$. How to solve this sequence problem, i.e., how to represent $a_n$ in closed form?
Comment: **Let's not jump into voting to close, before we ask OP for clarifications. Especially if OP is willing to promptly reply and edit the question within few minutes of "clarification request" comments. It's annoying.**
Comment: It is unlikely that there is a closed form. Recursions of the form $a_{n+1}=a_n^2+c$ are very complicated. The Mandelbrot set reflects this.
Comment: Why do you think there is a closed form? What would you do if you had one? Perhaps you're trying to solve a problem related to this sequence that does not need a closed form.
Comment: Yes, a closed form for $a_n$.
Comment: My first though is to represent $a_n=\frac{b_n}{c_n}$ in lowest terms.  It becomes $c_n=2^{2^{(n-1)}}$, $b_{n+1}=b_n^2+2^{2^{(n-1)}}$, $b_1=1$, which gets you into $\mathbb N$.  But I'm not sure it helps any.
Comment: What is the question?
Comment: By "solve," I assume you want to find a closed form for $a_n.$ Right?
Comment: @RossMillikan I have tried that, it does help much. I also tried to use sinh and cosh functions to replace $a_n$, but it did not provide any convenience either.
Comment: See http://applied-discrete-structures.wiki.uml.edu/Chapter+8
Comment: @J.D.Thanks for the correction.  I immediately regretted my vote to close, but the software does not allow me to retract it. Fortunately there is no harm done.
Answer:
EDIT: The answer below assumes $a_1$ is an integer, and as such is not directly applicable to your question, but I am leaving
  it here in case people can make use of the results mentioned in the
  paper below.

Aho and Sloan proved that for sequences like yours, there is a constant $k$ such that
$$a_n = \lfloor k^{2^n} \rfloor$$
for sufficiently large $n$. $k$ can be defined as a limit of a sequence using $a_n$ itself. If you include $k$ as one of your closed form constants, you are done!
See their paper for details: http://www.fq.math.ca/Scanned/11-4/aho-a.pdf
Of course, for your special case, one might still be able to find a different "closed form" which might be more appealing to you.
Comment: But $a_n$ is never an integer, is it?
Comment: @Didier: Yes, it isn't. The paper deals with that too. I read the question too quickly and thought $a_1 = 1$. Will update.
Comment: @anon: No, it is my fault.
Comment: @Didier: I added a disclaimer! It might need some work to get those results to work for this problem (if at all they work). But I have left the answer, in case someone wants to try.
Answer: The recurrence $a_{n+1} = a_n^2+c$ has a (known) closed form if and only if $c=0$ or $c=-2$.  See this answer for more explanation.
Answer: Perhaps let $a_n=\tan \theta_n$.  Then $\tan \theta_{n+1}=\sec^2 \theta_n$.   So, $\tan \theta_2=\sec^2 \theta_1$,  $\tan \theta_3=1+\sec^4 \theta_1$, etc. I am not sure if this procedure will produce a closed form.


Writing selenium tests, should I just get it done or get it right?

Question: I'm attempting to drive my user interface (heavy on javascript) through selenium.  I've already tested the rest of my ajax interaction with selenium successfully.  However, this one particular method seems to be eluding me because I can't seem to fake the correct click event.  I could solve this problem by simply waiting in the test for the user to click a point and then continuing with the test but this seems like a cop out.  But I'm really running out of time on my deadline to have this done and working.  Should I just get this done and move on or should I spend the extra (unknown) amount of time to fix this problem and be able to have my selenium tests 100% automated?
Answer: If the tests aren't 100% automated, they lose a tremendous amount of value. They won't be run reliably, and are a lot more error prone because of the human interaction. If you can't get it 100% automated, remove that test completely, even if it means not testing that piece of functionality for now.
As an aside, have you asked for help on stackoverflow? They may be able to give you a fairly quick solution.
Comment: +1 for removing automated tests that aren't automated.  Also, the fine folks over at sta.stackexchange have a little (well, a lot of experience) with Selenium (and most other test automation libraries).
Comment: Also, the people over at the [User Experience](http://ux.stackexchange.com/) site might be able to help. I often see Selenium related posts there.
Comment: Note there's a special site in SE (although still in beta) that is related to this in particular: [Software Quality Assurance and testing](http://sqa.stackexchange.com/)
Answer: I've seen that "wait for the user to do something" technique done.  It's a bad idea, for a lot of different reasons, but it can be made to work in a limited context.  If you do it, obviously you don't have an automated test, but rather one that requires active human observation and interaction.  Which means you can't run it as part of a Continuous Integration process, you probably won't want to run it as part of an in-the-background unit-test collection, etc.
If you decide to do this anyway, the correct way to do it is to alert the user that their action is required, and to then wait (looping with pauses as necessary) for the correct result of their action.  As opposed to waiting for a few minutes and hoping they did their piece of the job.
Answer: Try to find all possible workarounds.
And if nothing works classify them as non-automatable tests with proper blocker documentation. Maybe in future versions of selenium (or you application) you get some workaround for the step.


In Canada for 6 months, going back to US, how long until I can come back?

Question: I'm a US citizen and online business owner. I've been in Canada for 141 days, and have a permitted 180 day stay. I want to leave to the USA to basically reset the 180 days. Can I do that? I don't think I can apply for the visa extension because I'm not "working" in their country by their standards (since it's online) and I'm not a student.
So my questions are:
1.) Can I leave and come back?
2.) If so, how long do I need to be gone?
3.) CAN I apply for an extended visa? Although I never needed one to begin with as a US citizen. (<--cant find this answer on the CIC website)
Comment: "I'm not "working" in their country by their standards (since it's online)" yes you are. Please pay your taxes in the country where you live.
Answer: The very next day, if you can convince the border agent that you're using your visa for the appropriate means, and aren't trying to 'live' there instead.
They're going to wonder what took 141 days and now needs more.  If you're backpacking across the country, say, they can see evidence of your travels and you can show that say, you were now heading up to Churchill to see the polar bears and therefore need another few weeks.  They're going to want to know how long you're going for this time, and what you're planning on doing.
It gets a bit more unclear if you're a digital nomad like yourself.  Technically you're working for a US company from Canada. Some countries and guards can be very iffy on that.  Essentially, they may see it as you living and earning money there, benefiting from their roads and so on, without paying and Canadian taxes. And that's not approved.
However, if you provide them with your business address and show that you're travelling as a tourist as well, it might be deemed more acceptable.
Short version: You're going to have to convince the border agent that you're not living there tax-free, but are a bona-fide visitor, there for normal visitor reasons, who plans on returning to the US where you live.
EDIT: Indeed, from the Canadian Immigration website:

To visit Canada, you must:

have a valid travel document, such as a passport,
be in good health,
convince an immigration officer that you have ties—such as a job, home, financial assets or family—that will take you back to your home
  country,
convince an immigration officer that you will leave Canada at the end of your visit, and
have enough money for your stay. (The amount of money you will need can vary. it depends on things like how long you will stay, and
  whether you will stay in a hotel or with friends or relatives.)

I've highlighted the key bit. If it's apparent or seems that your plan is just to keep re-entering Canada, they may well disqualify you.  Indeed, when I lived in Vancouver I had a friend from Utah who went to renew her visa, and was seen to be 'living' there (She was actually trying to move to Vancouver long term, just did things in the wrong order) and was given 2 days to pack up and leave.  So basically, that's what your entry request will be tested against.
Comment: @Cupcake no worries. Please do report back once you've spoken to the border, and with any extra information that may help future travellers like yourself!
Comment: Thank you so much! I think I also, may be going about things the wrong way, which is why I'm extra nervous. I also applied for permanent residency, but after further reading, I think I have to leave Canada (and stay out) until that application is approved. So I'm not sure whether to extend my visa for the rest of my vacation rental time (until october) or to just leave and re-apply for my permanent residency.
Comment: It might be worth going to the border without ID, and asking them for advice. My British flatmate did that when applying for the long term visa (I forget the name), and got some good info (he had a complicated set up in multiple provinces).
Comment: Ahhh, thank you! That might be wise, seeing as I have a short amount of time to figure out what to do haha. Thank you for being so helpful!


How long does it take to set up a web server on a Linode?

Question: I've always used shared hosting in the past but now I'm thinking about moving to a Linode for my new web sites. However my knowledge of Linux and Apache is quite limited. I wonder how long it takes to set up a web server + MySQL db server?
Answer: Entirely depends...it's subjective. How fast a learner are you? How much is there to configure? How much custom work do you need done? You have limited Linux knowledge, but how much do you know? How much help do you have available?
Could take an hour. Could take a couple days. There is no way to answer this...
Comment: You wouldn't need to read and read to become a linux guru first; you end up reading and googling to solve problems as they arise, generally. You can start by looking for a howto on installing apache and mysql on a particular Linux distro you're aiming to use.
Comment: I see. The fact that it "could" take an hour is somewhat encouraging :). I love to learn but I don't want to read 30 books and become a Linux guru before being able to go live with my sites. That's why I'm still hesitant. Anyway, thanks for the answer.
Comment: This should be the accepted answer. People don't understand that this isn't formatting and reinstalling your windows machine. If you want to build a secure and robust server with even a moderate amount of clients, you're not going to be counting your hours on your fingers and toes.
Answer: If you don't know the answer in advance it might be best to leave a week part-time free. The first thing you'll want to do is secure the server (you need a set of iptables rules that will only permit those services you want, and probably re-assign the SSH port to something other than 22).
To be honest why use Linode if your knowledge of Linux and Apache is limited? I use Linode because my knowledge is deep and thus I can fully flex the capabilities of both on an always-on server when I cannot do that at home.
Otherwise a hosting provider will offer you scripting and MySQL database services (such as iPowerWeb which I've used in the past).
Comment: Ah. I would suggest trying to set all that up at home (or at work) first on a cheap (maybe second-hand) PC using debian or ubuntu linux. When you are comfortable with that then setting it up online should be a lot easier. The biggest issue with self-hosting is security. Your server _will_ come under attack every hour of the day. Port 25 (smtp) will be busy, as will port 80 (http) with false requests. Port 445 (microsoft-smb) will be probed continually. It is essential that you install an iptables firewall ruleset as the very first thing you do with an online Linux host. And use chkrootkit too.
Comment: I know it seems a bit silly. My decision to go for a Linode is based on some technical requirements: MySQL InnoDB storage engine, cron jobs, host multiple domains, use PHP memcache, PHP Data Objects (PDO) extensions. It seems to be hard to get all that at a decent price with regular hosting providers.
Comment: Good suggestion, I think I'll do that. As for the Linux distribution do you have any recommendation?
Comment: Debian is good if you want long-term stability. If you're playing around and want to use the latest-and-greatest then Ubuntu is popular.


SQLite LEFT JOIN Usage is Not Working Correctly

Question: I have 3 TABLES. Costumer, Product and Stats. When user select a user and a product, i increase Stats->count by 1.
Stats table has id, cid, pid and count values.
My purpose is sort products by count and they should be alphabetic. This is my sql request;
<code>SELECT * FROM product AS p
         LEFT JOIN stat AS s
         ON p._id = s.stats_product_id
         WHERE s.stats_customer_id = 'customerId'
         ORDER BY s.stats_count desc , p.product_name COLLATE NOCASE;
</code>
The problem is; because i create Stats values after i create some order, at the beginning i can't see all of products. Because some of products that i've never used are not in my Stats table. I know that LEFT JOIN should give me all of products values but it doesn't... Please help me. Thanks in advice.
Ex: I have a,b,c,d,e,f products. Let's think that a,b bought 3 times (their counts are 3), e,f bought 2 times (their counts are 2) and c,d never bought (they are not in stats table because they never bought by anyone and that's the problem tho!). Output order should be:
a
b
e
f
c
d
HANDLED: Solution on answer #1.
Answer: OK I got the answer. Query should be like this;
<code>"SELECT * FROM " +
                        "(product LEFT JOIN stat " +
                        "ON product._id = stat.stats_product_id " +
                        "AND stat.stats_customer_id = '" + customerId + "' ) " +
                        " ORDER BY " +
                        "stat.stats_count desc , product.product_name COLLATE UNICODE";
</code>
I hope this helps anyone.
Comment: You don't need to remove the aliases.


Setting an image as post header (not featured image)

Question: I am not a WordPress expert and need your support. Is there a way to create a button just like Set featured image but that button will allow us to attach an image to a post (from gallery) as a meta value? I am aware of custom meta boxes and manual upload of media but is there any possibility to avoid custom upload? I expect to duplicate the Set featured image functionality for a different purpose, i.e. to set a post header image, is there any possibility? Any help will be highly appreciated. Please ensure that featured image itself is being used for a different purpose and I am not willing to use it as the post header image.
Answer: The simplest way is to use the built-in custom fields. If you don't see them by default, while you're editing a Post/Page/CPT, choose "Screen Options" at the top right of the screen and check the "Custom Fields" box. Then scroll down through the editor until you see the Custom Fields box. Decide what you want to call your field, something unique with perhaps a prefix - like wpse_second_image - that you'll remember to use each time in the "name" field. You can then go to the media library, find the image you want, and copy and paste the URL into the "value" field.
Once your images are set, you'll probably want a fallback image in case you forget to set it or mis-type the field name. Something simple like this will do:
<?php
// make sure the Post object is available
global $post;
// get the "wpse_second_image" custom field for this Post
$second_image = get_post_meta($post->ID, 'wpse_second_image', true);
// if we found an image
if($second_image) {
    echo "";
// if we didn't find an image, use a fallback
} else {
    echo "";
}
?>

You'll put this wherever you want the image to appear in your theme. Sounds like header.php in a child theme may be appropriate for your needs.


Attempting to install Linux in BIOS mode but there's no option whether to boot in UEFI or BIOS mode

Question: I am trying to install Solus or Ubuntu onto an Acer Aspire V5-531 as Windows is painfully slow but I don't want any data to be deleted. The Setup menu is painfully simple and the system came with Win 7 installed in BIOS mode which is where the issue is coming from. I've tried using Live DVDs and USB devices but there isn't any option to not boot into BIOS mode. Is it possible to force Solus or Ubuntu to boot into BIOS mode? I would also like to note that I am aware of the Ubuntu Live CD but I would prefer to install Solus if possible. I can't seem to upload images but I do have some of the Setup menus if anyone would like to see them
Comment: Maybe the firmware has CSM disabled? I'd instead look into converting Windows to UEFI mode.
Answer: First of all, no operating system will let you choose whether to boot into BIOS or UEFI mode. But you can choose between those options by entering BIOS when your laptop boots up.
This thread implies that both modes should be available for your machine. Since it's an Acer laptop, I guess you can enter BIOS by pressing the F2 key a few seconds after pressing the power button. If that key does not work, you can try FN+F2, Shift+F2, ESC, DEL keys, but I'm pretty sure F2 will work for you.
Also, most Linux distributions will allow you to install the operation system on both BIOS or UEFI mode, as you prefer. Ergo, your laptop being in BIOS mode is no problem if your goal is just to install Linux. In order to receive further assistance with your setup, you need to tell us, which installation mode you prefer, and if you plan to change the Win7 installation from BIOS to UEFI, and if you want to be able to quickly switch between those two operating systems without entering BIOS.


What does it mean to "start from scratch"?

Question: This question speaks to the "start from the beginning" meaning, but in yesterday's USA Today, there was a headline about Obama and Romney starting from scratch because they were even in a poll.  Have you heard it mean to be at a tie?
Comment: The headline is, ["In 12 swing states, they're starting from scratch"](http://www.usatoday.com/NEWS/usaedition/2012-05-07-swingstate06_CV_U.htm), and "to start from scratch" does not mean "to be at a tie" there or elsewhere. Much rather, *because* they are at a tie, their previous effort might as well have not existed, so it's as if they are starting from scratch.
Comment: It still means the same. They are at the same point (tied) so it's as if they never started.
Comment: After all the efforts put in by Obama and Romney to win the poll, nobody came out a winner. So, that means all their efforts went in vain and because the decision has to be made, they both need to start again from where they started at first. All new efforts are needed again (maybe with a different approach this time).
Answer: Per other answers, USA Today's usage here is at the very least "questionable". In a sports context where the scores are equal some way through a match, you could reasonably say "It's as if they're starting from scratch". But in the context of electioneering, the "metaphor" doesn't work well. It gets mixed up with the idea that preceding campaign strategies might be replaced, since the expression start from scratch is...

...usually used to mean 'start again from the beginning' - where an initial attempt has failed and a new attempt is made with nothing of value carried forward from the first attempt.

In this and the related up to scratch (good enough to meet a particular standard), the word "scratch" comes from a boundary or starting point scratched on the ground in a sporting context. Originally it signified having no advantage or handicap (weaker contestants might be allowed to start beyond the scratch as a "head start"), from whence the meaning has shifted to the modern sense highlighted above (effectively, "gaining no advantage from preceding activities/knowledge").
Comment: I picked this answer because it explained the origin of "scratch" and the implication that it signifies having no advantage.  I had never heard it used this way before until seeing it in the headline. Thanks!
Comment: @David W: Yes, well I suppose it's always possible the USA Today headline writer was aware of that original meaning, and intentionally evoked it because what *he* meant was that neither Obama nor Romney had the advantage of "campaign impetus" by virtue of being ahead in the polls. But I must be honest and say I think it was just clumsy thoughtless phrasing (either that, or an Americanism on which I'm not qualified to have an opinion! :)
Answer: The article doesn't appear to explain what they mean by the headline. As others have noted, the phrase means "start from nothing", but clearly the two men are not starting from nothing now: the article explicitly points out that each has a substantial base and that they are fighting over the undecideds.
After seeing the headline but before reading the article, I thought they were going to say something like, Both men's campaign strategies have proven ineffective, and so they must start over with new ideas or new plans. An argument like that would be consistent with saying that they are starting from scrath. But the article doesn't say anything like that.
In short, I think the headline has nothing to do with the article. It's bad editing.
Answer: A "real" start from scratch for a contest is 0-0.
But if it's a tie, say 45-45, the impact of the contest is basically the same as if you were starting from 0-0.
I consider this headline misleading because one person was actually ahead, 47-45 (that is, within the 3-point margin of error). I consider this a "stretch" on both counts, but the paper was using an analogy of "starting from scratch." I call it "journalistic license."


Correcting pixelated hillshade using QGIS

Question: I'm running QGIS 3.20.2. I created a hillshade from lidar data for a property I live on, but the hillshade is quite pixelated. Is there a way to unpixelate it, so it looks smooth and clear? See attached pic for what it looks like now.
Comment: Reduce the pixel size when creating the hillshade to increase the resolution. Your hillshade is constrained by the resolution of your input data (DEM), so if the resolutions match there may not be anything you can do. See here https://gis.stackexchange.com/questions/169616/increase-resolution-of-generated-hill-shade-on-qgis
Answer: Completely agree with Kartograaf.
If you have the underlying point cloud, you can probably use kriging (or some other non-deterministic interpolation) to get a smoother, higher-resolution surface and hillshade.
If you don't, you can probably convert the DTM to a point polygon and then do some interpolation on it. I wouldn't do this if you're looking to run some analysis however, because it assumes an equal lidar sampling density throughout your property and disregards the positional component of observations, but if you're looking for a nice visual, it's a trick you could try.


Why isn't Rick's family bothered by his nudity?

Question: I have seen a couple of episodes like Pickle Rick, Rest and Ricklaxation etc. where Rick is completely nude but it doesn't seem to bother anyone. Why is that?
Comment: Morty and Summer are bothered by his nudity at the end of Big Trouble in Little Sanchez (Tiny Rick espisode). Perhaps they are simply used to it by this point
Comment: Are you asking about Rick's family in real life or reel life?
Comment: in rest and ricklaxation after the fight between rick and toxic rick.Rick stands nude while talking to beth & summer
Answer: This is a running theme in season 3 (and in the series as a whole): The family is becoming jaded by Rick's shenanigans. Remember how Morty was frightened and hysterical when Rick told him he's going to nuke the world in the pilot? Well, now he's nonchalantly defusing the Neutrino bomb that Rick has built in Vindicators 3, commenting angrily that this happened way too many times. And the Summer that was scarred by her experience with Rick's ship in The Ricks Must Be Crazy? She's willingly joining his adventures and unflinchingly murders people in Rickmancing the Stone.
Morty and Summer were startled by their grandpa's nudity in the past (in M. Night Shaym-Aliens! and Big Trouble In Little Sanchez, respectively), but this has happened so many times that they're simply accustomed to it by now. Note that even Beth doesn't really seem to be bothered by it when she returns to the house in Rest and Ricklaxation, and barely even cares that the house is in shambles. Rick's misadventures are gradually turning his family indifferent to all the strange occurences around them - and shaping them to be more and more like him.


How to detect decaying oscillations in a signal

Question: I have a sampled signal which contains (among other things) some damped oscillations. From the underlying theory I have reasons to believe that it can be characterized by some specific frequencies of oscillation $\omega$ and rates of decay $-\sigma$. I believe the signal contains components of the form $f(t)=e^{\sigma t+i \omega t}$. It might be that these are "there" only in some part of the signal, because at late times other components win over (due to exponential damping of these oscillations). I am interested in finding these dominating timescales. I need a hint how to proceed. There are plenty of tools available, like Fourier analysis, wavelets etc. However, I don't know which is the most appropriate for this task, as I have no experience with any of them, except maybe the Fourier transform. Below I include a graph of the signal that I want to analyze.

https://i.stack.imgur.com/tUQxo.png
Comment: What do you mean by dominating timescales? Do you mean the time points at which the peaks occur?
Comment: Welcome to DSP.SE! Interesting question. You will probably also need to include a phase term in the signal model: $f(t) = e^{\sigma t + i\omega t + i \phi}$.
Comment: Ok, to be more precise: What I am looking for are complex frequencies $\sigma + i \omega t$ which are somehow "dominant" in the signal. In the language of Fourier transform, I suspect there should be some visible peak in the spectrum of the signal. But this is only analogy, because I am interested in complex frequencies and traditional Fourier analysis doesn't apply.
Comment: For what I can see in the signal the frequency of the dumped oscillation is not constant, it is slightly decrescent. Which kind of analysis do you need? Batch or online?
Comment: I'm sorry, I don't understand the question "batch or online analysis?".
Comment: @Blazej From afar, your signal seems chirp-like: higher frequencies at the beginning, lower at the end. Moreover, there might be a dacaying baseline.
Answer: Your signal model lends itself particularly well for so called parametric frequency analysis. The basic idea is that you find the autocovariance matrix of your signal and identify its eigenstructure. The eigenvalues correspond to the exponents of a sum-of-exponentials model. In your case you will find conjugate pairs of eigenvalues with non-zero imaginary part describing the exponential decay.
This thesis looks like a good introduction to the topic:
http://zet10.ipee.pwr.wroc.pl/record/11/files/Leon_mon.pdf
Answer: If you take fourier transform $f_t=e^{\sigma t}\times e^{i\omega_0 t}$ it will be convolution of the two individual signal spectrums. $e^{i\omega_0 t}$ will result into impulse at $\omega_0$ and $e^{\sigma t}$ will give decaying spectrum. You can get some insight about these parameters from simple Fourier transform.
Comment: Sorry, I don't understand what do you suggest I do.
Comment: Take FFT of $ft$, it should peak at $\omega0$. And it should fall on both sides of the peak by factor the $\frac{1}{\sigma+j2\pi f}$.


Conjugate gradient 2D image reconstruction Matlab

Question: I'm working on a project where I should use the conjugate gradient method to reconstruct an SPECT image given the sinogram of the image with the help of matlab. I have realized that the linear equation Ax=b needs to be solved, where b is an N-dimensional vector containing the known sinogram values wrapped into a vector and x is a M-dimensional vector containing the unknown image to be reconstructed and that A is a system vector. I can easily construct b with the sinogram I have, but I don't really know how I should get A.
The image I want is 128X128 and the sinogram is 128X32 (i.e. 32 projections).
Does anybody have any Ideas of how i can use the help of matlabs PCG function to reconstruct my image?
Answer: The n_th column of the projection matrix A is the radon transformation of the n_th pixel, that is its n_th element is 1 and all other elements are zero. This is doable with the matlab radon function, but I doubt that it is very efficient because A will be sparse and I guess the loop will take quite some time, although 128x128x128x32 should still be manageable and the (sparse) matrix can be cached in a .mat file as long as your geometry doesn't change.
You can try other matlab scripts, for example http://www2.imm.dtu.dk/~pch/AIRtools/
where you get A by calling
<code>N=128;               % 128x128 pixels
p=128;               % Number of rays per projection
alpha=(0:32)*180/32; % List of projection angles in degrees
A=paralleltomo(N, alpha, p);
</code>
I hope that helps.


Is 集まろう! a common way to say "Let's get together?"

Question: I'm pretty sure 集{あつ}まろう (atsumarou) technically means "Let's get together" or "Let's gather together" i.e. as a group, but I'm not sure if this would be a commonly used phrase in real-world Japanese.  Are there any particular contexts in which this would make more sense or be more common?
Comment: More common than what?
Comment: More common than some other situation e.g. maybe this would be ok, but not common, for one friend to say to another but maybe it would be common to say in an office setting to a group of people talking about an after work get together.
Answer: It seems to me that in real-world Japanese, 集まろう is a very popular and commonly used phrase in a group of people, in the context such as いつもの飲み屋に集まろう or あの公園に集まろう or 名古屋で集まろう or ホテルで集まろう. In other words, 集まろう is usually preceded by the phrase (場所)に or (場所)で in daily conversation. Therefore the phrase 集まろう without the reference to the gathering place is used in very restricted situations, e.g. in advertisement, or chorus of shouts (you put ! after 集まろう), in an office setting to a group of people, with the mutual knowledge of the gathering place, talking about an after work get together. Last of all, I have to make one additional but important remark about this expression, 集まろう is an expression almost for males, and more polite form of 集まりましょう is preferred especially by females.
Comment: I'd be careful about saying 集まろう is for men and 集まりましょう is for women, though. 集まりましょう is just a more polite way of saying 集まろう. Men would use it too when speaking to their bosses or seniors at work, it's more a matter of familiarity/closeness between the speaker and the listener than a gender thing.
Comment: Thanks so much for the great answer! I was asking this question because I'm working on a project to make a multi-lingual (English and Japanese initially) social networking platform so regular people can create small social networking sites, focused on a specific topic, without knowing how to code. I am brainstorming names for the platform / site and thought Atsumarou.com might be kind of cool since, well, social networking sites are about getting together. Based on your answer, it seems like it might indeed be a good name for my project. Thanks again!
Comment: Deeeeekun, thank you very much for your comment. I should have been more careful about  asserting this or that from a gender point of view.
Comment: I know language evolves, but it still makes me mad when I learn that "men say it this way, be careful, only men say it this way. Women say it like this, which is also the more polite way to say it." Screw that! Seriously. This 'men have the freedom to say it this way not women' B. S. is, I know, part of the culture, but perhaps equality could start here. (This is the comment I'm referring to, and in no way an criticizing what they said, it's very helpful and necessary to point out BUT I'm asking, shouldn't we do our part to help society evolve? These are barriers imposed upon humans that need
Comment: The plain volitional is used by women *all the time*, in both real life and fiction
Answer: 集まろう does mean "Let's get together". But in the context you posted ("after work get-together"), I more commonly hear "集合しよう". Also, I think it's more common to actually indicate what will be done (i.e. "飲みに行こう", "ご飯行こう", "遊ぼう"). "集合しよう" would be used when indicating a specific time, date, or place.
Comment: `Also, I think it's more common to actually indicate what will be done` I agree, this seems like a better answer pertaining to a social scenario.

Is 集まろう！ a common way to say "Let's get together?"


Software to burn CDs and DVDs

Question: I have been trying to use Brasero to copy CDs , but it asks for toc2cue and cdrdao installation, I tried K3b and it says K3b can not copy multi-tracks. Can anyone suggest a software which would not give me hard times (something like Nero but free). Thanks!
Comment: There is nothing wrong with installing additional packages. No way to get everything on a single installation CD.
Answer: I recommend sticking with Brasero. I presume you're trying to burn a non-standard disk image format and that's why Brasero is asking for cue2toc and cdrdao. You could just install those packages and be on your way:
sudo apt-get install cue2toc cdrdao

Although I admit that this is a paper cut in usability and you should report it as a bug here:
https://bugs.edge.launchpad.net/hundredpapercuts/+filebug
Please explain exactly what you're trying to do, with what kind of files and what makes it different from burning normal iso files. Make it clear what you expect the computer to do (i.e. work without asking you to install extra things)
Comment: looks like it's spelled wrong in the warning for Bresaro, it asks for toc2cue, but the package is called cue2toc. A real issue and I noticed it when I was answering, but I assumed you had made the error and not Bresaro.
Comment: well , I had tried to install those two softwares but it gave me error saying that one of them could not be found ( I do not remember which one ) but surprisingly it worked this time with no problem. I googled this problem and it turns out that lots of other people had to download those two softwares and most of them had my problem : one of those two softwares could not be found ( I think it was cue2toc ) . anyway , it is working now , thanks

Software to burn CDs and DVDs


Matlab question: saving the path

Question: I am working with Matlab R2011a. I want to set a path at c:\matlab (say). But it seems that the path setting is not saved and I need to reset the path every time I start Matlab. Please advise.
Answer: If you want to start Matlab in a specific path, and have that path automatically added to your search path, you modify the userpath setting:
userpath('C:\matlab')

If you want to start Matlab in a specific path without having that path added to your search path, you have two options:
1) Create a file startup.m in the folder C:\Program Files\MATLAB\R2011a\toolbox\local and write in the file cd('C:\matlab'). This way, whenever Matlab is started, it changes the path (you can also add other commands that should be executed upon startup).
2) Right-click onto the Matlab icon you normally use to start up the program, and in the properties, write C:\matlab in the Start in line. Thus, whenever you start Matlab from the shortcut (but not otherwise), Matlab will change path.
Comment: I am told that I do not have a privilege to save a file at C:\Program Files\MATLAB\R2011a\toolbox\local. Please advise.
Comment: @user12586: Type "userpath('C:\matlab')" in the Matlab Command Window (no double quotes), then hit the return key.
Comment: @user12586: If you don't know the administrator password to your computer, either talk to the person who has the administrator password, or try modifying the properties of the icon you click on in order to start Matlab (solution #2 of starting up in a user-defined location without adding said location to the Matlab path).
Comment: Thank you. Do I have to type  "userpath('C:\matlab')" every time I start the matlab?
Comment: I got the following: Warning: Executing startup failed in matlabrc.
This indicates a potentially serious problem in your MATLAB setup,
which should be resolved as soon as possible.  Error detected was:
MATLAB:cd:NonExistentDirectory
Comment: @user12586: Once you do the `userpath` setting, you don't need to do anything - the setting is saved (as you can check by typing `userpath`). As to your second problem: What exactly did you write into startup.m? Did you perchance save it in Rich Text Format? Anyway, once you do the userpath thing, you don't need `startup.m`, so you could delete it to fix the second issue.
Comment: @user12586: Note that from the error thrown by `matlabrc`, which is the function that calls startup, it looks like you made a typo when writing the path in the cd command - Matlab cannot change to the directory you specified because it doesn't exist.
Answer: Make sure that you have access to all matlab files. I had the same problem, but when I gave myself full control of the C:\Program Files\MATLAB\R2012b folder (and all it's subfiles and folders) I no longer had this problem.
To do this, right click on the R2012b folder (or whichever version you have), and open properties.
Under the security tab, look for your username in the list. If it's not there, click "Edit..." and "Add..." your name. Then click the full control in the allow column. You may need an admin password for this.
Click "ok", "ok" (it may take a couple of minutes). Set the path again, close Matlab, reopen, and test if your setpath settings are still there.
Answer: To add to Jonas' answer (because changing the user path didn't work for me):
You can also use the start.m method "If you want to start Matlab in a specific path, and have that path automatically added to your search path". For Linux:
start.m
cd /path/to/my/folder % sets current working directory
addpath(genpath('/path/to/my/folder')) % adds directory (and its subfolders) to the path list

or for Window I suppose it would look like this:
start.m
cd c:\path\to\my\folder % sets current working directory
addpath(genpath('c:\path\to\my\folder')) % adds directory (and its subfolders) to the path list

Matlab question: saving the path


Calculating/estimating volatiles based on other representative samples

Question: I am currently working on a set of geochemical data representing calc-alkaline igneous intrusions from central Canada.  The samples were analyzed in ~2013-2014.  These samples have been tested for LOI wt%, but not H2O, CO2...etc.  Historical sampling from the same area representing the same suite of rocks (but different sample locations) have results for H2O and CO2 wt%.  I need H2O wt% values for my samples in order to complete a calculation for viscosity of the intrusions, but I won't be doing any more testing.  Am I able to estimate H2O in my samples using the historical data, and if so, how would I go about doing that?  Also, how would I correct for (if possible) the difference in quality of the measurements due to more advanced and accurate/precise instrumentation?
Answer: If you are sure that the samples for which you have H2O data are from the same intrusion than yours, then I don't see a problem for using these values for viscosity modelling, as long as it is clearly mentioned in the paper. After all, a sample is supposed to representative of the rock unit, and science is done by standing on the shoulders of giants!
If you have several historical samples with different H2O contents, this gives you a range of water contents. Do you intend to use a model like Giordano et al. (2008) ? If so, you also need a temperature estimate, which you may have by geothermometry. These temperature estimates have an uncertainty, see Putirka (2008) for a review. For example, if you use the two-feldspar thermometer, it gives you $T$ to $\pm$30 °C.
So, what you could do is to model two scenarios: one "cool and dry", with the lowest $T$ and H2O content, which gives you the highest viscosity; and one "hot and wet", with the highest $T$ and H2O content, which gives you the lowest viscosity. This way you can be quite confident that the viscosity was somewhere between these two values. Usually a $\pm$30 °C interval will change your result by half a log unit, and a $\pm$0.2 wt% H2O interval will change it by one log unit, so you'd get something like $\log \eta = X \pm 0.75$ Pa s, which is not that bad.
Answer: There are several problems with what you're trying to do.
First, you are looking at the H2O and CO2 contents of the rock as they are today. It is ignoring the fact that H2O and CO2 are lost during rising and cooling of the magma. They can also be gained during subsolidus alteration, which I suspect happened in your case because the solubility of CO2 in granitic magma is extremely low, and it should not contain any carbonate minerals. Therefore, late addition of alteration minerals such as calcite would change the values.
Second, the viscosity depends on the ratio between the crystal and melt load in the magma mush. Figuring out what was solid and what was liquid and at what stage, after it has all solidified, isn't easy. What if your H2O was all in wet amphibole phenocrysts? Or was it all in the melt? These are things you need to take into consideration.


How can I decode a message that was encrypted with a one-time pad?

Question: How can I decrypt a message that was encrypted with a one-time pad? Would I need to get that pad, and then reverse the process? I'm confused here.
Comment: One time pads can be "decrypted" into anything, making them unbreakable. They are used by sovereign head of states (including USA) to communicate. Their one "vulnerability" is distribution of the key.
Comment: If your question is not how to "break" an OTP, but just how, in the normal order of things as an authorized user, you would go about the process of decryption, then yes, you would need the pad, and then XOR the PAD again, with the cipher text, to recover the plain text. That would be legitimate decryption.
Answer: One-Time Pad is unbreakable, assuming the pad is perfectly random, kept secret, used only once, and no plaintext is known. This is due to the properties of the exclusive-or (xor) operation.
Here's its truth table:
A xor B = X

A | B | X
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0

Number of 0s in column A = 2
Number of 1s in column A = 2
Number of 0s in column B = 2
Number of 1s in column B = 2
Number of 0s in column X = 2
Number of 1s in column X = 2

Note that it introduces no bit-skew - the number of 0s and 1s in the inputs are equal to the number of 0s and 1s in the output, i.e. two of each. Furthermore, if you know only one element from a row, you cannot predict the values of the other two, since they are equally probable.
For example, let's say we know that X is 0. There's an equal probability that A = 0 and B = 0, or A = 1 and B = 1. Now let's say we know that X is 1. There's an equal probability that A = 0 and B = 1, or A = 1 and B = 0. It's impossible to predict. So, if you only know one element, you cannot possibly determine any information about A or B.
The next interesting property is that it is reversible, i.e.
A xor A = 0
B xor B = 0

A xor 0 = A
B xor 0 = B

A xor B xor B  =  A xor 0  =  A
A xor B xor A  =  B xor 0  =  B

So, if we take any value and xor it with itself, the result is cancelled out and it always results in 0. This means that, if we xor a value A with a value B, then later xor that result with either A or B, we get B or A respectively. The operation is reversible.
This lends well to cryptography, because:

xor introduces no bitskew
xor has equally probable inputs for any given output
given any two of A, B, X we can compute the third

As such, the following is perfectly secure:
ciphertext = message xor key

but only if message is the same length as key, key is perfectly random, key is only used once, and only one element is known to an attacker. If they know the ciphertext, but not the key or message, it's useless to them. They cannot possibly break it. In order to decrypt the message, you must know the entire key and the ciphertext.
Keep in mind that the key must be completely random, i.e. every bit must have an equal probability of being 1 or 0, and be completely independent of all other bits in the key.
This actually turns out to be rather impractical, for a few reasons:

Generating perfectly random keys is hard. Software generators (and many hardware generators) often have minuscule biases and odd repeating properties. It's almost impossible to gain truly random data in anything but tiny amounts.
If the attacker knows the ciphertext and can correctly guess parts of the message (e.g. he knows it's a Windows executable, and therefore must start with MZ) he can get the corresponding key bits for the known range. These bits are useless for decrypting other parts of the message, but can reveal patterns in the key if it's poorly generated.
You must be able to distribute the key, and your key must be equally as long as your message. If you can keep your key 100% secret between those of you who are authorised to read the message, why not just keep your message 100% secret instead?

The weak link here is your random number generator. The security of the one time pad is entirely limited by the security of your generator. Since a perfect generator is almost impossible, a perfect one-time pad is almost impossible too.
The final problem is that the key can only be used once. If you use it for two different messages, and the attacker knows both ciphertexts, he can xor them together to get the xor of the two plaintexts. This leaks all sorts of information (e.g. which bits are equal) and completely breaks the cipher.
So, in conclusion, in a perfect one-time pad you need to know the ciphertext and key in order to decrypt it, but perfect one-time pads are almost impossible.
Answer: One-time pads are extremely hard to break, in fact they are still used in some situations as if they are done correctly then they are essentially unbreakable. In a one-time pad system every character is changed by a stream of random data which is shared by both sides, without a copy of the pad you will not be able to break the code.
One of the few weaknesses in the system is the random data source. In WWII British one-time pads were being broken and they traced it to a worker whose job it was to pull random numbered balls out of a drum. The way it was supposed to work was that the worker would spin the drum, pull out a ball at random without looking at it, spin again, pick again, etc, etc. The worker started taking shortcuts by pulling out more than one ball after each spin and looking at the numbers, picking out favorites. It introduced patterns which enabled the opposition to break the pads, and lives were lost as a result.
The same is true today with pseudo-random number sources. Encryption protocols that should take millennia to break will really only last for years or decades without a true random data source.
Comment: Do you have a reference for that interesting WWII story?
Comment: Neal Stephenson's _Cryptonomicon_ mentions a similar (fictional) situation. An operator drawing numbered balls from a bingo drum keeps forgetting to close her eyes, thus rendering one-time pads insecure.
Comment: That wasn't fictional at all, it happened in ww2
Comment: @GdD I meant that this particular event in the story was fictional. RNGs were certainly vulnerable to human biases.
Answer: Assuming the encryption was competently done, the only way to decrypt is to get the pad.
Answer: While one-time-pad encryption is provably impossible to break, note that it is also extraordinarily rare.
Part of the definition of OTP is that the pad must contain truly random data, and truly random data can be hard to come by for computers. Instead, often the pad is composed of pseudo-random data, which is generally what you get when you ask a computer for a random number.
In this case, you no longer have unbreakable encryption. Instead, you have a message XORed with the output of a deterministic and often reversible algorithm, which can be attacked by reproducing the same pseudo-random string. This attack is particularly devastating if you know what algorithm is used and even more so if you know how the seed is generated. In such a case, the code can be cracked in a matter of seconds. But even without knowing the PRNG seed, often the pattern can be derived from the message itself.
Also, OTP encryption is particularly unwieldy because the key is as long as the encrypted message, and must somehow be transmitted to the recipient without being intercepted. Any software that claims to use OTP encryption that does not require you to exchange a key block as large as your message (or larger) is not using truly random data, and is therefore not using OTP encryption.
Also, one of the key features of OTP is that you cannot reuse old pads. If you reuse an old one even once, that can be enough to allow the code to be broken.
Answer: The question was, how can you decrypt a message that was created from one time pad, pretty sure that hasn't been answered yet.
Originally the one time pad was used with just characters, and is very basic. Let's say you have a message 'killtheking' (taken from one response), and you want to encrypt it so you need a key. For the key you need at least the same number of characters as the message, so you roll a dice which has 26 sides (one for each letter) 11 times, because there are 11 characters in the message.
Assume the result is aqheivlekrw
now we subtract the position in the alphabet of the letter in they key, from the position in the alphabet of the letter in the message, and the resulting number is the position in the alphabet of the cypher text. Wrap any letters that create a minus result as below (like if the result is 30, then use 26 - 30 = 4 or 'd'.
eg:
k - a = (11 - 1) = 10 = j

'j' is the 10th letter in the alphabet so we use that.
for the rest of the message we get:
i - g = (9 - 7) = 2 =   b
l - h = (12 - 8) = 4 =  d
l - e = (12 - 5) = 7 =  g
t - i = (16 - 9) = 7 =  g
h - v = (8 - 22) = -14 (26 - 14 = 12) = l
e - l = (5 - 12) = - 7 (26 - 7 = 19) = s
k - e = (11 - 5) = 6 = f
i - k = (9 - 11) = -2 (26 - 2 = 24) = x
n - r = (14 - 18) = -4 (26 - 4 = 22) = v
g - w = (7 - 23) = -16 (26 - 16 = 10) = j

so in the end
your message    = killtheking
you key         = agheivlekrw
your cypher     = jbdgglsfxvj

pass the cypher text to whomever you want to recieve it and they do the opposite of what you have done to encrypt it, they have to add the key to the cypher text:
j + a = (10 + 1 = 11) k
b + g = (2 + 7 = 9)   i
d + h = (4 + 8 = 12)  l
g + e = (7 + 5 = 12)  l
...

and so on.
Computers use XOR because it does the same thing and is a lot quicker and easier to implement, but to understand how it works, I found that it is good to see the 'human' usable example. So on a computer you can encrypt the message by doing:
Cypher = Message XOR Key
and decrypt it again by using:
Message = Cypher XOR Key
It is as simple as that.
Comment: This does, of course, assume the die is perfectly random. For the example, we can probably assume this safely. But in reality getting a perfectly unpredictable sequence is not that easy.
Comment: yes, using a loaded dice would kind of defeat the purpose, however, the encryption/decryption method is the same regardless of how effectively your random source is.
Answer: If you don't have the key ...
Brute force the ciphertext (permute it) and analyze the output for potential cleartext in the expected language.
Statistical analysis can assist in the detection of potential human-generated language in the output (maybe chi-squared?)
Comment: You can't brute-force a one-time pad. If you generate every possible pad, you'd find that it also decrypts the message into every possible output text, so there's no way to tell which one is correct.
Comment: But in your answer you say that you could permute the key and analyse the output but **you can't**. For example, say I sent the message `kill the king` and you intercept the 13 character (104 bit) ciphertext. You then somehow generate all 2^104 possible one-time pads (good luck doing that on a normal computer). Unfortunately, you find that along with `kill the king` as a possible output, there's also `save the king`, `save the duck`, `feed me ducks`, (consider padding/compression), etc., and there's absolutely no way to tell which one is correct without knowing something about the key.
Comment: +1 again. You are doing a much better job of explaining why this is unlikely to succeed than I did, cheers.
Answer: one time pad could de easily cracked if you know the output format. Such as audio headers, gifs images, etc. By reverse engineering the stream back. It could be good for some sort of characters but not for some other fields where the output format could be easily identify by other means. Nothing is 100% secure nowadays.
Comment: This is simply incorrect.


Is there an algorithm which can compute every algorithm's time complexity?

Question: I think of an algorithm which computes the time complexity. It would be great if a code editor could compute the time complexity of the selected lines and even compare two pieces of codes in order to help developer to select one of them. I heard something related with halting problem but couldn't figure out.
Answer: It would be great, wouldn't it?
And like most problems regarding Turing Machines (or "algorithms"), there is no such algorithm, and there never will be:
Consider for example the language
$$\{M: M\text{ is a TM that runs in time }O(n^3)\}$$
It is quite easy to reduce the halting problem to this language. Indeed, given input $M,w$ for the halting problem, the reduction outputs a machine $K$ that given input $x$, runs $M$ on $w$, and accepts $x$ iff $M$ halts on $w$, and does not halt otherwise. Since running $M$ on $w$ takes constant time (if it halts at all), then $K$ runs in time $O(n^3)$ iff $M$ halts on $w$.
Thus, it is undecidable (not in $coRE$, to be exact).
There is nothing special about $n^3$, the same reduction works easily for every function greater than $n\log n$ (below $n\log n$ things get a bit more involved).
Also, the Halting problem itself can be regarded as a sort of "run-time problem", in the sense that you don't even ask what the runtime of an algorithm is, but just whether it halts. Even that is undecidable.
It is interesting to note that the language, as formulated in this answer, is also not in $RE$, it is an easy exercise to show it. However, if you replace the $O(n^3)$ with exactly $n^3$, then the problem is in $coRE$. Then, the former reduction needs certain adjustments, but the problem is still undecidable.
Comment: @KutluhanMetin: How would it find the worst case? How would it make a statement about behaviour for $n \to \infty$?
Comment: The idea I became here is, let's say I have 100 lines of code and I put a breakpoint at line 10. It executes the code line by line from line 10 to 100. Maybe it would execute it for the worst case and tell us the complexity between that lines.
Answer: What you are proposing can be used to implement an algorithm to solve this problem:

Given a runtime bound $\Theta(f)$ and a procedure proc, decide whether proc runs in that time (in the worst case).

There may be special cases where you can decide such things (e.g. certain primitive recursive programs that can be identified as such), but in general, no, you can't decide runtime of programs.
As a matter of fact, you can't even decide between two given runtime bounds.


Magento 2 : Issues with sub categories

Question: I have an issue while adding sub categories.Its showing 1 in addition to the category name i.e subcategory 1 and the products are not reflecting at the front end. please help me on that as I am new to magento.
Comment: Please add more info.
Answer: It seems to be indexing problem. Run following commands to reset and reindex your indexes.
<code>php bin/magento indexer:reset
php bin/magento indexer:reindex
php bin/magento cache:flush
chmod 777 -R var pub
</code>

Magento 2 : Issues with sub categories


401K funds withdrawal at age 60 for non US resident

Question: I have funds in a 401K plan from a period of time I worked in the US in the late 80s.
I am not a US resident and I wonder if at the age of 60 I can start monthly withdrawals and how are these withdrawals going to be taxed in the US.
Answer: Yes, you can take monthly withdrawals (they are called "distributions") when you're 60.
If you're a US resident these funds are taxed as if they were wage income.
At your death, many years from now, any remaining money is considered part of your estate.
If you live outside the US the situation is complex. It depends, among other things, on treaties between the US and your country.  You should call or write to the distribution department of the financial institution holding your 401K. It is their job to help you sort this out.
Comment: No, it is not their job and they will not be helping in any way. No financial institution is required to provide tax advice, nor will they take such a liability upon themselves.
Answer: You will be able to withdraw them, and they will be US-sourced. I.e.: you'll need to file a non-resident tax return and potentially pay taxes on the income (if it is substantial enough). Your home country may have a tax treaty with the US which may affect how this is treated by the US or your home country. Your home country may also have foreign tax credit to allow reduction of local taxes on the account of the taxes you paid on this income in the US.
You should talk to a tax adviser familiar with your home country rules and the US rules, and especially if there's a tax treaty between the two.
No, the 401k custodian will not help you, and will not know what and how you should be doing. They'll just know to send you a check if you ask them to.


Display email address field next to author in wp_dropdown_users

Question: the post_author_meta_box uses wp_dropdown_users showing the user name of authors.
How do I display the email address in brackets next the author's name?
Can one filter the wp_dropdown users do to this?
Answer: the only filter hook called in wp_dropdown_users() function is wp_dropdown_users which pass a string of the dropdown in an html form so you can manipulate this dropdown at that hook with some major REGEX.
A better solution would be to use get_users() and create the dropdown your self, something like this:
$selected = 1; //just for example, you can get that by get_post_meta()
$siteusers = get_users(); // you can pass filters and option
$re = '';
if (count($siteusers) > 0){
    $re = '<select name="users_with_email">';
    foreach ($siteusers as $user) {
        $re .= '<option value="' . $user->ID . '">'.$user->user_nicename . ' ('.$user->user_email .')</option>';
    }
    $re .= '</select>';
    $re = str_replace('value="' . $selected . '"','value="' . $selected . '" selected="selected"', $re );
}
echo $re;

Comment: I dont want to edit the core, how would one do this with a plugin? Is it possible to replace the wp_dropdown_users() function with my own and not break wp admin elsewhere?
Comment: ok nice.. figured this out thanks to you: http://pastebin.com/RD58LANm