Monte carlo simulation in R

Question: I am trying to solve the following exercise but I actually have no clue on how to start doing this. I've found some code in my book that looks like it but it's a completely different exercise and I don't know how to relate them to eachother. How can I start simulating arrivals and how do I know when they are finished? I know how to store them and calculate a,b,c,d according to that. But I don't know how I actually need to simulate the monte carlo simulation.
Could someone please help me get started? I know this isn't a place where your questions get answered for you but only solved instead. But the problem is I don't know how to start.
An IT support help desk represents a queuing system with five
assistants taking calls from customers. The calls occur according to a Poisson process with
the average rate of one call every 45 seconds. The service times for the 1st, 2nd, 3rd, 4th, and
5th assistants are all Exponential random variables with parameters λ1 = 0.1, λ2 = 0.2,
λ3 = 0.3, λ4 = 0.4, and λ5 = 0.5 min−1, respectively (the jth help desk assistant has
λk = k/10 min−1). Besides the customers who are being assisted, up to ten other customers
can be placed on hold. At times when this capacity is reached, the new callers receive a
busy signal.
Use the Monte Carlo methods to estimate the following performance characteristics,
(a) the fraction of customers who receive a busy signal;
(b) the expected response time;
(c) the average waiting time;
(d) the portion of customers served by each help desk assistant;
EDIT : what I have so far is (not much) :
pa = 1/45sec-1

jobs = rep(1,5); onHold = rep(1,10);

jobsIndex = 0;

onHoldIndex = 0;

u = runif(1)
for (i in 1:1000) {

    if(u  <= pa){ # new arrival

        if(jobsIndex < 5) # assistant is free, #give job to assistant

            jobsIndex++;

        else #add to onHold array

            onHoldIndex++;
    }
}

Comment: It is not exactly about "how to do MC", but are you familiar with this package: http://www.r-bloggers.com/introducing-simmer-discrete-event-simulation-for-r/ ? It seems to fit perfectly for the kind of problems you describe (although using different model).
Comment: I am actually trying to solve this without external libraries, but if I can't do so I will use yours for sure :)
Comment: Show what have you done so far. You can't simply come here and ask for solution of a home work.
Comment: For those using Python, there is a solution provided in [Ciw's documentation](https://ciw.readthedocs.io/en/latest/Guides/behaviour/server_dependent_dist.html#with-desired-behaviour).
Answer: This is one of the most instructive and fun kinds of simulation to perform: you create independent agents in the computer, let them interact, keep track of what they do, and study what happens.  It is a marvelous way to learn about complex systems, especially (but not limited to) those that cannot be understood with purely mathematical analysis.
The best way to construct such simulations is with top-down design.
At the very highest level the code should look something like
initialize(...)
while (process(get.next.event())) {}

(This and all subsequent examples are executable R code, not just pseudo-code.)  The loop is an event-driven simulation: get.next.event() finds any "event" of interest and passes a description of it to process, which does something with it (including logging any information about it).  It returns TRUE for as long as things are running well; upon identifying an error or the end of the simulation, it returns FALSE, ending the loop.
If we imagine a physical implementation of this queue, such as people waiting for a marriage license in New York City or for a driver's license or train ticket almost anywhere, we think of two kinds of agents: customers and "assistants" (or servers).  Customers announce themselves by showing up; assistants announce their availability by turning on a light or sign or opening a window.  These are the two kinds of events to process.
The ideal environment for such a simulation is a true object-oriented one in which objects are mutable: they can change state to respond independently to things around them.  R is absolutely terrible for this (even Fortran would be better!).  However, we can still use it if we take some care.  The trick is to maintain all the information in a common set of data structures that can be accessed (and modified) by many separate, interacting procedures.  I will adopt the convention of using variable names IN ALL CAPS for such data.
The next level of the top-down design is to code process.  It responds to a single event descriptor e:
process <- function(e) {
  if (is.null(e)) return(FALSE)
  if (e$type == "Customer") {
    i <- find.assistant(e$time)
    if (is.null(i)) put.on.hold(e$x, e$time) else serve(i, e$x, e$time)
  } else {
    release.hold(e$time)
  }
  return(TRUE)
}

It has to respond to a null event when get.next.event has no events to report.  Otherwise, process implements the "business rules" of the system.  It practically writes itself from the description in the question. How it works should require little comment, except to point out that eventually we will need to code subroutines put.on.hold and release.hold (implementing a customer-holding queue) and serve (implementing the customer-assistant interactions).
What is an "event"?  It must contain information about who is acting, what kind of action they are taking, and when it is occurring.  My code therefore uses a list containing these three kinds of information.  However, get.next.event only needs to inspect the times.  It is responsible only for maintaining a queue of events in which

Any event can be put into the queue when it is received and
The earliest event in the queue can easily be extracted and passed to the caller.

The best implementation of this priority queue would be a heap, but that's too fussy in R.  Following a suggestion in Norman Matloff's The Art of R Programming (who offers a more flexible, abstract, but limited queue simulator), I have used a data frame to hold the events and simply search it for the minimum time among its records.
get.next.event <- function() {
  if (length(EVENTS$time) <= 0) new.customer()               # Wait for a customer$
  if (length(EVENTS$time) <= 0) return(NULL)                 # Nothing's going on!$
  if (min(EVENTS$time) > next.customer.time()) new.customer()# See text
  i <- which.min(EVENTS$time)
  e <- EVENTS[i, ]; EVENTS <<- EVENTS[-i, ]
  return (e)
}

There are many ways this could have been coded.  The final version shown here reflects a choice I made in coding how process reacts to an "Assistant" event and how new.customer works: get.next.event merely takes a customer out of the hold queue, then sits back and waits for another event.  It will sometimes be necessary to look for a new customer in two ways: first, to see whether one is waiting at the door (as it were) and second, whether one has come in when we weren't looking.
Clearly, new.customer and next.customer.time are important routines, so let's take care of them next.
new.customer <- function() {
  if (CUSTOMER.COUNT < dim(CUSTOMERS)[2]) {
    CUSTOMER.COUNT <<- CUSTOMER.COUNT + 1
    insert.event(CUSTOMER.COUNT, "Customer",
                 CUSTOMERS["Arrived", CUSTOMER.COUNT])
  }
  return(CUSTOMER.COUNT)
}
next.customer.time <- function() {
  if (CUSTOMER.COUNT < dim(CUSTOMERS)[2]) {
    x <- CUSTOMERS["Arrived", CUSTOMER.COUNT]
  } else {x <- Inf}
  return(x) # Time when the next customer will arrive
}

CUSTOMERS is a 2D array, with data for each customer in columns.  It has four rows (acting as fields) that describe customers and record their experiences during the simulation: "Arrived", "Served", "Duration", and "Assistant" (a positive numerical identifier of the assistant, if any, who served them, and otherwise -1 for busy signals).  In a highly flexible simulation these columns would be dynamically generated, but due to how R likes to work it's convenient to generate all customers at the outset, in a single large matrix, with their arrival times already generated at random.  next.customer.time can peek at the next column of this matrix to see who's coming next.  The global variable CUSTOMER.COUNT indicates the last customer to arrive.  Customers are managed very simply by means of this pointer, advancing it to obtain a new customer and looking beyond it (without advancing) to peek at the next customer.
serve implements the business rules in the simulation.
serve <- function(i, x, time.now) {
  #
  # Serve customer `x` with assistant `i`.
  #
  a <- ASSISTANTS[i, ]
  r <- rexp(1, a$rate)                       # Simulate the duration of service
  r <- round(r, 2)                           # (Make simple numbers)
  ASSISTANTS[i, ]$available <<- time.now + r # Update availability
  #
  # Log this successful service event for later analysis.
  #
  CUSTOMERS["Assistant", x] <<- i
  CUSTOMERS["Served", x] <<- time.now
  CUSTOMERS["Duration", x] <<- r
  #
  # Queue the moment the assistant becomes free, so they can check for
  # any customers on hold.
  #
  insert.event(i, "Assistant", time.now + r)
  if (VERBOSE) cat(time.now, ": Assistant", i, "is now serving customer",
                   x, "until", time.now + r, "\n")
  return (TRUE)
}

This is straightforward.  ASSISTANTS is a dataframe with two fields: capabilities (giving their service rate) and available, which flags the next time at which the assistant will be free.  A customer is served by generating a random service duration according to the assistant's capabilities, updating the time when the assistant next becomes available, and logging the service interval in the CUSTOMERS data structure.  The VERBOSE flag is handy for testing and debugging: when true, it emits a stream of English sentences describing the key processing points.
How assistants are assigned to customers is important and interesting.  One can imagine several procedures: assignment at random, by some fixed ordering, or according to who has been free the longest (or shortest) time.  Many of these are illustrated in commented-out code:
find.assistant <- function(time.now) {
  j <- which(ASSISTANTS$available <= time.now)
  #if (length(j) > 0) {
  #  i <- j[ceiling(runif(1) * length(j))]
  #} else i <- NULL                                    # Random selection
  #if (length(j) > 0) i <- j[1] else i <- NULL         # Pick first assistant
  #if (length(j) > 0) i <- j[length(j)] else i <- NULL # Pick last assistant
  if (length(j) > 0) {
    i <- j[which.min(ASSISTANTS[j, ]$available)]
  } else i <- NULL                                     # Pick most-rested assistant
  return (i)
}

The rest of the simulation is really just a routine exercise in persuading R to implement standard data structures, principally a circular buffer for the on-hold queue.  Because you don't want to run amok with globals, I put all of these into a single procedure sim.  Its arguments describe the problem: the number of customers to simulate (n.events), the customer arrival rate, the assistants' capabilities, and the size of the hold queue (which can be set to zero to eliminate the queue altogether).
r <- sim(n.events=250, arrival.rate=60/45, capabilities=1:5/10, hold.queue.size=10)

It returns a list of the data structures maintained during the simulation; the one of greatest interest is the CUSTOMERS array.  R makes it fairly easy to plot the essential information in this array in an interesting way.  Here is one output showing the last $50$ customers in a longer simulation of $250$ customers.

Each customer's experience is plotted as a horizontal time line, with a circular symbol at the time of arrival, a solid black line for any waiting on hold, and a colored line for the duration of their interaction with an assistant (the color and line type differentiate among the assistants).  Beneath this Customers plot is one showing the experiences of the assistants, marking the times when they were and were not engaged with a customer.  The endpoints of each interval of activity are delimited by vertical bars.
When run with verbose=TRUE, the simulation's text output looks like this:
...
160.71 : Customer 211 put on hold at position 1
161.88 : Customer 212 put on hold at position 2
161.91 : Assistant 3 is now serving customer 213 until 163.24
161.91 : Customer 211 put on hold at position 2
162.68 : Assistant 4 is now serving customer 212 until 164.79
162.71 : Assistant 5 is now serving customer 211 until 162.9
163.51 : Assistant 5 is now serving customer 214 until 164.05
...

(Numbers at the left are the times each message was emitted.)  You can match these descriptions to the portions of the Customers plot lying between times $160$ and $165$.
We can study the customers' experience on hold by plotting on-hold durations by customer identifier, using a special (red) symbol to show customers receiving a busy signal.

(Wouldn't all these plots make a wonderful real-time dashboard for anybody managing this service queue!)
It is fascinating to compare the plots and statistics that you get upon varying the parameters passed to sim.  What happens when customers arrive too quickly to be processed?  What happens when the hold queue is made smaller or eliminated?  What changes when assistants are selected in different manners?  How do the numbers and capabilities of the assistants influence the customer experience?  What are the critical points where some customers start getting turned away or start getting put on hold for long times?

Normally, for evident self-study questions like this one, we would stop here and leave the remaining details as an exercise.  However, I do not want to disappoint readers who may have gotten this far and are interested in trying this out for themselves (and maybe modifying it and building on it for other purposes), so appended below is the full working code.
(The $\TeX$ processing on this site will mess up the indentation at any lines containing a $\$$ symbol, but readable indentation should be restored when the code is pasted into a text file.)
sim <- function(n.events, verbose=FALSE, ...) {
  #
  # Simulate service for `n.events` customers.
  #
  # Variables global to this simulation (but local to the function):
  #
  VERBOSE <- verbose         # When TRUE, issues informative message
  ASSISTANTS <- list()       # List of assistant data structures
  CUSTOMERS <- numeric(0)    # Array of customers that arrived
  CUSTOMER.COUNT <- 0        # Number of customers processed
  EVENTS <- list()           # Dynamic event queue
  HOLD <- list()             # Customer on-hold queue
  #............................................................................#
  #
  # Start.
  #
  initialize <- function(arrival.rate, capabilities, hold.queue.size) {
    #
    # Create common data structures.
    #
    ASSISTANTS <<- data.frame(rate=capabilities,     # Service rate
                              available=0            # Next available time
    )
    CUSTOMERS <<- matrix(NA, nrow=4, ncol=n.events,
                         dimnames=list(c("Arrived",  # Time arrived
                                         "Served",   # Time served
                                         "Duration", # Duration of service
                                         "Assistant" # Assistant id
                         )))
    EVENTS <<- data.frame(x=integer(0),              # Assistant or customer id
                          type=character(0),         # Assistant or customer
                          time=numeric(0)            # Start of event
    )
    HOLD <<- list(first=1,                           # Index of first in queue
                  last=1,                            # Next available slot
                  customers=rep(NA, hold.queue.size+1))
    #
    # Generate all customer arrival times in advance.
    #
    CUSTOMERS["Arrived", ] <<- cumsum(round(rexp(n.events, arrival.rate), 2))
    CUSTOMER.COUNT <<- 0
    if (VERBOSE) cat("Started.\n")
    return(TRUE)
  }
  #............................................................................#
  #
  # Dispatching.
  #
  # Argument `e` represents an event, consisting of an assistant/customer
  # identifier `x`, an event type `type`, and its time of occurrence `time`.
  #
  # Depending on the event, a customer is either served or an attempt is made
  # to put them on hold.
  #
  # Returns TRUE until no more events occur.
  #
  process <- function(e) {
    if (is.null(e)) return(FALSE)
    if (e$type == "Customer") {
      i <- find.assistant(e$time)
      if (is.null(i)) put.on.hold(e$x, e$time) else serve(i, e$x, e$time)
    } else {
      release.hold(e$time)
    }
    return(TRUE)
  }#$
  #............................................................................#
  #
  # Event queuing.
  #
  get.next.event <- function() {
    if (length(EVENTS$time) <= 0) new.customer()
    if (length(EVENTS$time) <= 0) return(NULL)
    if (min(EVENTS$time) > next.customer.time()) new.customer()
    i <- which.min(EVENTS$time)
    e <- EVENTS[i, ]; EVENTS <<- EVENTS[-i, ]
    return (e)
  }
  insert.event <- function(x, type, time.occurs) {
    EVENTS <<- rbind(EVENTS, data.frame(x=x, type=type, time=time.occurs))
    return (NULL)
  }
  #
  # Customer arrivals (called by `get.next.event`).
  #
  # Updates the customers pointer `CUSTOMER.COUNT` and returns the customer
  # it newly points to.
  #
  new.customer <- function() {
    if (CUSTOMER.COUNT < dim(CUSTOMERS)[2]) {
      CUSTOMER.COUNT <<- CUSTOMER.COUNT + 1
      insert.event(CUSTOMER.COUNT, "Customer",
                   CUSTOMERS["Arrived", CUSTOMER.COUNT])
    }
    return(CUSTOMER.COUNT)
  }
  next.customer.time <- function() {
    if (CUSTOMER.COUNT < dim(CUSTOMERS)[2]) {
      x <- CUSTOMERS["Arrived", CUSTOMER.COUNT]
    } else {x <- Inf}
    return(x) # Time when the next customer will arrive
  }
  #............................................................................#
  #
  # Service.
  #
  find.assistant <- function(time.now) {
    #
    # Select among available assistants.
    #
    j <- which(ASSISTANTS$available <= time.now)
    #if (length(j) > 0) {
    #  i <- j[ceiling(runif(1) * length(j))]
    #} else i <- NULL                                    # Random selection
    #if (length(j) > 0) i <- j[1] else i <- NULL         # Pick first assistant
    #if (length(j) > 0) i <- j[length(j)] else i <- NULL # Pick last assistant
    if (length(j) > 0) {
      i <- j[which.min(ASSISTANTS[j, ]$available)]
    } else i <- NULL # Pick most-rested assistant
    return (i)
  }#$
  serve <- function(i, x, time.now) {
    #
    # Serve customer `x` with assistant `i`.
    #
    a <- ASSISTANTS[i, ]
    r <- rexp(1, a$rate)                       # Simulate the duration of service
    r <- round(r, 2)                           # (Make simple numbers)
    ASSISTANTS[i, ]$available <<- time.now + r # Update availability
    #
    # Log this successful service event for later analysis.
    #
    CUSTOMERS["Assistant", x] <<- i
    CUSTOMERS["Served", x] <<- time.now
    CUSTOMERS["Duration", x] <<- r
    #
    # Queue the moment the assistant becomes free, so they can check for
    # any customers on hold.
    #
    insert.event(i, "Assistant", time.now + r)
    if (VERBOSE) cat(time.now, ": Assistant", i, "is now serving customer",
                     x, "until", time.now + r, "\n")
    return (TRUE)
  }
  #............................................................................#
  #
  # The on-hold queue.
  #
  # This is a cicular buffer implemented by an array and two pointers,
  # one to its head and the other to the next available slot.
  #
  put.on.hold <- function(x, time.now) {
    #
    # Try to put customer `x` on hold.
    #
    if (length(HOLD$customers) < 1 ||
          (HOLD$first - HOLD$last %% length(HOLD$customers) == 1)) {
      # Hold queue is full, alas.  Log this occurrence for later analysis.
      CUSTOMERS["Assistant", x] <<- -1 # Busy signal
      if (VERBOSE) cat(time.now, ": Customer", x, "got a busy signal.\n")
      return(FALSE)
    }
    #
    # Add the customer to the hold queue.
    #
    HOLD$customers[HOLD$last] <<- x
    HOLD$last <<- HOLD$last %% length(HOLD$customers) + 1
    if (VERBOSE) cat(time.now, ": Customer", x, "put on hold at position",
                 (HOLD$last - HOLD$first - 1) %% length(HOLD$customers) + 1, "\n")
    return (TRUE)
  }
  release.hold <- function(time.now) {
    #
    # Pick up the next customer from the hold queue and place them into
    # the event queue.
    #
    if (HOLD$first != HOLD$last) {
      x <- HOLD$customers[HOLD$first]   # Take the first customer
      HOLD$customers[HOLD$first] <<- NA # Update the hold queue
      HOLD$first <<- HOLD$first %% length(HOLD$customers) + 1
      insert.event(x, "Customer", time.now)
    }
  }$
  #............................................................................#
  #
  # Summaries.
  #
  # The CUSTOMERS array contains full information about the customer experiences:
  # when they arrived, when they were served, how long the service took, and
  # which assistant served them.
  #
  summarize <- function() return (list(c=CUSTOMERS, a=ASSISTANTS, e=EVENTS,
                                       h=HOLD))
  #............................................................................#
  #
  # The main event loop.
  #
  initialize(...)
  while (process(get.next.event())) {}
  #
  # Return the results.
  #
  return (summarize())
}
#------------------------------------------------------------------------------#
#
# Specify and run a simulation.
#
set.seed(17)
n.skip <- 200  # Number of initial events to skip in subsequent summaries
system.time({
  r <- sim(n.events=50+n.skip, verbose=TRUE,
           arrival.rate=60/45, capabilities=1:5/10, hold.queue.size=10)
})
#------------------------------------------------------------------------------#
#
# Post processing.
#
# Skip the initial phase before equilibrium.
#
results <- r$c
ids <- (n.skip+1):(dim(results)[2])
arrived <- results["Arrived", ]
served <- results["Served", ]
duration <- results["Duration", ]
assistant <- results["Assistant", ]
assistant[is.na(assistant)] <- 0   # Was on hold forever
ended <- served + duration
#
# A detailed plot of customer experiences.
#
n.events <- length(ids)
n.assistants <- max(assistant, na.rm=TRUE)
colors <- rainbow(n.assistants + 2)
assistant.color <- colors[assistant + 2]
x.max <- max(results["Served", ids] + results["Duration", ids], na.rm=TRUE)
x.min <- max(min(results["Arrived", ids], na.rm=TRUE) - 2, 0)
#
# Lay out the graphics.
#
layout(matrix(c(1,1,2,2), 2, 2, byrow=TRUE), heights=c(2,1))
#
# Set up the customers plot.
#
plot(c(x.min, x.max), range(ids), type="n",
     xlab="Time", ylab="Customer Id", main="Customers")
#
# Place points at customer arrival times.
#
points(arrived[ids], ids, pch=21, bg=assistant.color[ids], col="#00000070")
#
# Show wait times on hold.
#
invisible(sapply(ids, function(i) {
  if (!is.na(served[i])) lines(x=c(arrived[i], served[i]), y=c(i,i))
}))
#
# More clearly show customers getting a busy signal.
#
ids.not.served <- ids[is.na(served[ids])]
ids.served <- ids[!is.na(served[ids])]
points(arrived[ids.not.served], ids.not.served, pch=4, cex=1.2)
#
# Show times of service, colored by assistant id.
#
invisible(sapply(ids.served, function(i) {
  lines(x=c(served[i], ended[i]), y=c(i,i), col=assistant.color[i], lty=assistant[i])
}))
#
# Plot the histories of the assistants.
#
plot(c(x.min, x.max), c(1, n.assistants)+c(-1,1)/2, type="n", bty="n",
     xlab="", ylab="Assistant Id", main="Assistants")
abline(h=1:n.assistants, col="#808080", lwd=1)
invisible(sapply(1:(dim(results)[2]), function(i) {
  a <- assistant[i]
  if (a > 0) {
    lines(x=c(served[i], ended[i]), y=c(a, a), lwd=3, col=colors[a+2])
    points(x=c(served[i], ended[i]), y=c(a, a), pch="|", col=colors[a+2])
  }
}))
#
# Plot the customer waiting statistics.
#
par(mfrow=c(1,1))
i <- is.na(served)
plot(served - arrived, xlab="Customer Id", ylab="Minutes",
     main="Service Wait Durations")
lines(served - arrived, col="Gray")
points(which(i), rep(0, sum(i)), pch=16, col="Red")
#
# Summary statistics.
#
mean(!is.na(served)) # Proportion of customers served
table(assistant)

Comment: +1 Amazing! Could you answer all questions with that level of comprehensiveness and attention to detail? Dreams, just dreams...
Comment: @mugen I mentioned the Matloff book in the text. It might be appropriate for those new to `R` who want another (but fairly similar) perspective on queue simulations.  While writing this little simulator I found myself thinking a lot about how much I learned by studying the code in (the first edition of) Andrew Tanenbaum's text *Operating Systems / Design and Implementation.*  I also learned about practical data structures, such as heaps, from Jon Bentley's articles in CACM and his *Programming Pearls* series of books. Tanenbaum and Bentley are great authors everyone should read.
Comment: +1 What can I say? Today I learnt so many many interesting things! Care to add any book for further reading, please?
Comment: @mugen, there's a free online textbook on queuing theory by Moshe [here](http://www.ee.cityu.edu.hk/~zukerman/classnotes.pdf). Also Prof. Gallager's discrete stochastoc processes course covers these topic on MIT [OCW](http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-262-discrete-stochastic-processes-spring-2011/). The video lectures are real good.
Comment: @whuber, a great answer. Though I don't think you can make kids these days to read Tanenbaum and Bentley :)
Comment: @Aksakal The Moshe reference is a good one, thank you.  His (brief) chapter 4 discusses a simple, practical way to simulate queues like this one.

Monte carlo simulation in R


cloudstack No suitable hosts found under this Cluster

Question: When I try to start an instance through template,I get the following error messages:
<code>2013-11-10 19:44:28,716 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] (Job-Executor-5:job-19 = [ d070b5ba-f342-4252-9137-4d2c1b19eca6 ]) No suitable hosts found under this Cluster: 2
2013-11-10 19:44:28,718 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] (Job-Executor-5:job-19 = [ d070b5ba-f342-4252-9137-4d2c1b19eca6 ]) Could not find suitable Deployment Destination for this VM under any clusters, returning.
2013-11-10 19:44:28,718 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-5:job-19 = [ d070b5ba-f342-4252-9137-4d2c1b19eca6 ]) Searching all possible resources under this Zone: 1
2013-11-10 19:44:28,718 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-5:job-19 = [ d070b5ba-f342-4252-9137-4d2c1b19eca6 ]) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough CPU and RAM capacity under this Zone: 1
</code>
I feel confused because I already have a host in cluster 2.
Can anyone give me some suggestions?Any reply will be appreciated!
Answer: You need to have a closer look at the log file to understand why the CloudStack is unable to place a VM on the host.  The information will appear above the entries you have provided.  There are a lot of issues that can cause this problem.
E.g. this blog entry walks through a configuration problem with XenServer
Another common issue is when using local storage, you need to create a new compute offering that uses local storage disks.  The default compute offerings do not support local storage.
Updated:  changed answer to take into account that your sample is from the log file.
Answer: Most probably you are running out of capacity or due to some other error cloudstack has added you host in the "Avoid List". It does that when the management server finds an error while deploying the instance and from the next time till the problem gets resolved, the host and the cluster will be part of avoid list and will be avoided in the subsequent deployments.
You need to find out the exact reason by monitoring the management server logs. Login to your management server and go to folder /var/log/cloudstack/management/.
Now run the command tail -f management-server.log
This will give you continuous output of the management server log, so that you know what exactly is happening at the moment.
Now do the operation on UI (e.g. try to add instance) and quickly monitor the running logs.
Abort the command when you find an exception in the log and monitor the log statements just above the exception.
Also as a standard practice, always have a habit of monitoring the management server logs and agent logs (on host - /var/log/cloudstack/agent/agent.log)


Robustness in panel regression analysis

Question: I am running a fixed effects panel regression as under
$y=a+bx+e,$
where y=dependant variable, $x$=control variables, $a$=individual effects, $b$=slope, $e$=error term.
I first ran the FE panel regression with variables in levels (that is without taking logarithm). Then I ran the FE panel regression with variables expressed in logs. Ideally since taking logs is just scaling of the variable, we should not expect any change in the sign and statistical significance of the coefficients, b. But I find that for some control variables, the coefficients have changed when I take the logs. Am I going wrong somewhere? Which version gives correct results?
Comment: Can you show us a subset of the values for your control/outcome variables? Are some of the values between 0 and 1?
Comment: @ThomasBilach Yes That is true. Here are the some of the values
Comment: @ThomasBilach Yes That is true.But I have not taken logs of dummy variables or categorical variables. Here are the some of the values
country_id 1
year             2006
crar           0.0844
tier1_cap_rwa 0.0935
RoA 0.1183
RoE 0.1139
z_score 0.1238
reg ranges between 0 and 1
accept_dep ranges between 0 and 1
country ranges between 0-1
ofc 0-1
country_type  1-3
real_st_rate 0.16234
ef1_gdp 0.4900399
ef2_gdp 0.0595650
ef3_gdp 0.0000000
ef4_gdp 0.0035492
ef5_gdp 0.2157386
act_restrict 8
entry_req 7
cap_reg 6
ff 0.077
rol_index 1.77
rol_rank 0.10
Comment: Your comment is somewhat inscrutable. Could you post your output? Also, which variable are you logging?
Answer: Log-transforming a value less than 1 results in a negative number. Here is a quick demonstration in R:
<code># Generating a sequence of values between 0 and 1 (inclusive), incrementing by 0.1

x <- seq(0, 1, by = 0.1)
x
[1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

# Taking the log() of each value in this sequence

round(log(x), 2)
[1]  -Inf -2.30 -1.61 -1.20 -0.92 -0.69 -0.51 -0.36 -0.22 -0.11  0.00
</code>
It is entirely possible for the sign of a Pearson's correlation to flip as a consequence of logging observations between 0 and 1. In such contexts, it's possible for $\rho(X,Y)$ to flip sign if we $\log X$ or $\log Y$, or both. See below for a quick-and-dirty simulation. In each iteration, I assessed the correlation between 10 pairs of values. The correlation was preserved in 7 out of the 10 trials. Note, upon each draw of 10 observations, a subset fall between 0 and 1, which reverse in sign after the log transformation; this has the potential to influence the relationship between variables.
<code>set.seed(13)

sims <- 10
levels <- vector(mode = "numeric", length = 10)  # storage for the numeric values in levels
logged <- vector(mode = "numeric", length = 10)  # storage for the "logged" numeric values

for (i in 1:sims) {
  x <- exp(rnorm(10)); y <- exp(rnorm(10))       # drawing 'positive' random deviates

  levels[i] <- cor(x, y)                         # correlation in levels
  logged[i] <- cor(log(x), log(y))               # correlation of the logged values
}

flipped <- (levels > 0 & logged < 0) | (levels < 0 & logged > 0)
preserved <- !flipped

# Were the correlations between pairs preserved?

cbind(levels, logged, flipped, preserved)

           levels      logged flipped preserved
 [1,]  0.10728307 -0.01369591       1         0
 [2,]  0.93652958  0.62956976       0         1
 [3,]  0.01300703  0.07601658       0         1
 [4,] -0.06794333  0.37656387       1         0
 [5,]  0.17978986  0.27654877       0         1
 [6,]  0.19476326  0.54571601       0         1
 [7,] -0.09462134 -0.04706490       0         1
 [8,] -0.25225396 -0.40215868       0         1
 [9,] -0.20694668 -0.05695933       0         1
[10,] -0.04839709  0.08447948       1         0
</code>
Another example is when a transformation dampens an outlier that was exerting major leverage. Review the top answer here for a very simple and clear demonstration of this. If you're working with per capita rates (e.g., limited number of incidents per unit population) or proportions, then a log transformation can influence or even reverse the sign of a coefficient.
Comment: If the matter is statistically argued, we can never establish robustness in our regressions.  The whole objective of a doing an regression with economic variables is to establish causality. If it lacks robustness just by log transformation, no referee will ever accept a paper.
Comment: Your question was focused on your coefficients and how a log transformation might affect the *sign* of the observed effect. Why did you log transform your control variables in the first place? Was it to facilitate interpretation? Did the levels have a discernible skew?
Comment: I think it may have to be statistically argued, especially when you're log-transforming values less than 1.
Comment: I have already mentioned I am not log transforming values less than 1. It is against econometrics principle to do that.
Comment: @indumann I would update your post and show us the distributions of the variables you logged. This way we can diagnose the problem more precisely. I will adjust (remove) my answer if it doesn't apply to your situation.

Robustness in panel regression analysis


How to remove "Show row weights" option on File Upload field?

Question: I have created a file field with multiple uploads allowed. Is there any way(except CSS) I can remove "Show row weights" link from file upload field? Or is there any way to remove weight drop down entirely from file field?
Comment: It is possible bot sure that the whole thing work. try `hook_form_alter`
Comment: Do you mean the extra top-right button with the four arrows so you can change the weight by integers? From a usability and cross-platform point-of-view, you should never remove that functionality, since it's the alternative for when you've got JS disabled or is not working as expected.
Comment: My issue is that I want the extra top right button as it is but just want to remove the link. I have done is using locale module for now. Thanks for the input.
Comment: I found a great article that explains how to do this in a way you can indicate to which fields apply it. enjoy it!
https://www.commercialprogression.com/post/how-remove-tabledrag-rearranging-multiple-value-field-widgets
Comment: Double up! All one has to do is pasting the code inside a .module, adding an .info and insert a field name within the last function. Put it on github here: https://github.com/leymannx/drupal-disable-weight-on-multi-value-fields
Answer: The first part of your question is regarding removal of link "Show row weights" (or "Hide row weights" if dragging functionality is disabled) in a form where table rows are draggable, so I'm going to answer that.
"Show row weights" string appears in Drupal.tableDrag.prototype.showColumns() method and "Hide row weights" appers in Drupal.tableDrag.prototype.hideColumns() method that is included in tabledrag.js file (see head in Drupal 7.x branch) so you need to override these methods in order to replace or remove these strings without losing row dragging functionality.
Place this code in a .js file that is loaded along with the form that uses tabledrag.js:
(function ($) {
  // Copy hideColumns() method
  var hideColumns = Drupal.tableDrag.prototype.hideColumns;
  Drupal.tableDrag.prototype.hideColumns = function() {
    // Call the original hideColumns() method
    hideColumns.call(this);
    // Remove the 'Show row weights' string
    $('.tabledrag-toggle-weight').text('');
  }

  // Copy showColumns() method
  var showColumns = Drupal.tableDrag.prototype.showColumns;
  Drupal.tableDrag.prototype.showColumns = function () {
    // Call the original showColumns() method
    showColumns.call(this);
    // Remove the 'Hide row weights' string
    $('.tabledrag-toggle-weight').text('');
  }
})(jQuery);

Comment: It works but it's much easier to simply add to your css: .tabledrag-toggle-weight-wrapper,
.tabledrag-toggle-weight,
.tableresponsive-toggle-columns,
.tableresponsive-toggle {
  height: 0 !important;
  font-size: 0 !important;
  color: transparent !important;
  display: none !important;
}
Yes, it seems a bit overkill but, depending on various themes, some might not be enough.
Comment: will it any effect on default misc/tabledrag.js ?
Comment: @Fawwad, the only side effect of the provided code is that toggle element text is hidden. Default `hideColumns()` and `showColumns()` methods are still invoked.
Answer: You need to disable/alter settings.tableDrag function.
<?php
/**
 * Implements hook_js_alter().
 */
function MODULENAME_js_alter(&$javascript) {
  unset($javascript['misc/tabledrag.js']);
}
?>

Comment: This removes the "Show row weights" link but when enabled it displays the Weight column.  It's a good answer but a complete solution would include removal of the Weight column.
Comment: This will remove the ability to reorder the rows entirely, not just remove the link at the top of the page. Adding this code will break the ability to order items. USE AT YOUR OWN RISK.
Answer: Here is a solution for Drupal 8 using hook_preprocess_field_multiple_value_form()
First find the field you want to remove re-ordering ability from:
/**
 * Implements hook_preprocess_field_multiple_value_form().
 */
function MY_MODULE_preprocess_field_multiple_value_form(&$variables) {
  if($variables['element']['#field_name'] == 'MY_FIELD_NAME') {
    MY_MODULE_remove_table_reordering($variables['table']);
  }
}

And then call a helper function to actually remove the tables re-ordering functionality:
/**
 * Removes the ability for a table to be re-ordered.
 */
function MY_MODULE_remove_table_reordering(&$table) {
  unset($table['#tabledrag']);

  // Remove re-order header column
  foreach($table['#header'] as $header_i => $header) {
    if(is_object($header) && (string) $header == 'Order') {
      unset($table['#header'][$header_i]);
    }
  }

  // Loop table rows
  foreach($table['#rows'] as &$row) {

    // Remove draggable class from tr
    $i = array_search('draggable', $row['class']);
    if($i !== FALSE) {
      unset($row['class'][$i]);
    }

    // Loop columns in row
    foreach($row['data'] as $col_i => &$col) {

      if(empty($col['class'])) {
        continue;
      }

      // Remove td with drag handle
      $i = array_search('field-multiple-drag', $col['class']);
      if($i !== FALSE) {
        unset($row['data'][$col_i]);
      }

      // Remove td with re-ordering select
      $i = array_search('delta-order', $col['class']);
      if($i !== FALSE) {
        unset($row['data'][$col_i]);
      }
    }
  }

}

Answer: You can also try a theme-override, as in https://api.drupal.org/api/drupal/modules!field!field.form.inc/function/theme_field_multiple_value_form/7
Paste the function to your theme's template.php, rename 'theme_' in the function name to your theme's name and clear the cache.
Find the line saying
drupal_add_tabledrag($table_id, 'order', 'sibling', $order_class);

and comment it out.
Answer: I'm not sure about disabling for a single field, but I found myself wanting to disable the row dragging for an entire form.
monymirza suggested an answer to disable it entirely. However in D7, that approach is not reliable, because in some forms drupal_add_tabledrag() will be called after your hook_js_alter().  This is typical Drupal - everything is theoretically customizable, but in practice it's error prone and sensitive.
In the end I used the following hook_form_alter() to disable table drag for just one specific form:
<?php

/**
 * Implements hook_form_alter()
 */
function sunflower_custom_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'field_ui_field_overview_form' && $form['#bundle'] == 'sunflower_habitat') {
    // The habitat form is wizardized based on row weights.  This is a sensitive thing, and drupal's row weight javascript is not at all sensitive, completely changing all values when it feels like it.  So we don't want that "feature" on this form.

    // Disable it, if already enabled.
    $javascript = &drupal_static('drupal_add_js', array());
    unset($javascript['misc/tabledrag.js']);

    // Prevent it from being enabled later.
    $tabledrag_added = &drupal_static('drupal_add_tabledrag');
    $tabledrag_added = TRUE;
  }
}
?>

Answer: Adding on to moneymirza's answer to remove the weights and add a qualifyer, since we don't want to disable tabledrag.js from the entire site.
<?php
/**
 * Implements hook_js_alter().
 */
function MODULENAME_js_alter(&$javascript) {
  // Use a more specific page condition as desired.
  if(arg(0) == "node" && arg(2) == "edit") {
    unset($javascript['misc/tabledrag.js']);
    drupal_add_js(drupal_get_path('module', 'MODULENAME') . '/MODULENAME.js');
  }
}

Then make a MODULENAME.js file:
(function ($) {
  Drupal.behaviors.NoTabledragWeights = {
    attach: function (context, settings) {
      // Again, get more specific if desired.
      $('td:nth-child(2),th:nth-child(2)').hide();
    }
  };
}(jQuery));


Maximum Likelihood Estimation for Normal Distribution with parameters $\theta$ and $2\theta$ with $\theta>0$

Question: I'm using log-likelihood and eventually I get to the following quadratic equation: $n{\theta}^{2} + 2n\theta - \sum_{i=1}^{n}X_i^2 = 0$. After solving the equation I get a solution ($\frac{-n + \sqrt{n^2 + n\sum_{i=1}^{n}X_i^2}}{n}$) that seems rather cumbersome and it's hard to check weather the given solution is indeed a maximum of the log-likelihood function. Is the solution correct and there simply isn't an easier way to do it or am I missing something here?
Comment: It would be natural to divide through by $n$ so ${\theta}^{2} + 2\theta - \frac1n\sum_{i=1}^{n}X_i^2 = 0$ and $\sqrt{1+\frac1n\sum_{i=1}^{n}X_i^2}-1$
Comment: What was your expression for the log-likelihood?
Comment: It's not surprising to see some kind of peculiar thing because the mean and variance are coupled, so $\theta_{MLE}$ is somehow a weighted average of the sample mean and half the sample standard deviation (I assume you use the $N(\mu,\sigma)$ convention not the $N(\mu,\sigma^2)$ convention). The weights being nonlinear is not a surprise.
Comment: Actually it's $N(\mu,\sigma^2)$, that's at least the one we've been using in our classes. The book might actually be using the other one, I'll need to check this
Comment: If it is the variance then something is definitely fishy about the scaling in your formula for $\theta_{MLE}$. (Something is also fishy about the model formulation in this case because then $2$ actually has units, which would be a very strange way to do things.)
Comment: $\frac{-n}{2}ln(4\pi\theta) - \frac{1}{4\theta}\sum_{i=1}^{n}(X_i - \theta)^2$
Comment: Did you mean the variance is $2\theta,$ or the standard deviation is $2\theta \text{ ?} \qquad$
Comment: BTW standard notation is $\ln(4\pi\theta),$ not $ln(4\pi\theta). \qquad$
Answer: You shoud inspect your log-likelihood function (which you should include in the question):
$$\ell(\theta)=-\frac{n}{2}\log (4\pi\theta) - \frac{1}{4\theta}\sum_{i=1}^{n}(X_i - \theta)^2$$
Its (natural [*]) domain is $(0,+\infty)$. The function is continuous and differentiable in all its domain. The behaviour at the extremes is: $$\lim_{\theta \to 0^+}\ell(\theta)=-\infty$$ (the second term wins) and $$\lim_{\theta \to +\infty}\ell(\theta)=-\infty$$
You've found a single critical point, inside the domain. Then, it must be a local maximum, and also a global maximum.
The fact that the critical point is a local maximum can also be seen by computing the second derivative.

[*] It should be stressed that the domain of the parameter should be part of the likelihood definition (as with any function). I'm just assuming here the "natural" one (the largest subset of the reals), but if your problem statement somehow implies a more restricted domain, the answer could differ.
Comment: This seems correct. Thank you!
Comment: Using a capital $L$ rather than a lower-case $\ell$ for the log-likelihood seems a bit like letting $n$ approach $0$ as $\varepsilon$ approahces $\infty. \qquad$
Comment: @MichaelHardy Haha, you're right, (nobody has ever seen an $\epsilon$ approaching $\infty$!). Fixing
Answer: $$
L(\theta) \propto \frac 1 {\sqrt\theta^n} \exp\left( \frac {-1}{2(2\theta)} \cdot \sum_{i=1}^n(x_i-\theta)^2 \right)
$$
Let $\overline x = \frac 1 n \sum_{i=1}^n x_i$ and $s^2 = \frac 1 n \sum_{i=1}^n
(x_i-\overline x)^2.$ Then algebra tells us that
$$
\sum_{i=1}^n (x_i-\theta)^2 = ns^2 + n(\overline x - \theta)^2.
$$
Hence
$$
L(\theta) \propto \frac 1 {\sqrt\theta^n} \exp\left( \frac{-1}{4\theta} \left( ns^2 + n(\overline x -\theta)^2 \right) \right).
$$
\begin{align}
\ell(\theta) & = \log L(\theta) = \text{constant} - \frac n 2 \log\theta - \frac {ns^2} {4\theta} - \frac{n(\overline x-\theta)^2}{4\theta} \\[10pt]
& = \text{constant} - \frac n 2 \log\theta - \frac{ns^2}{4\theta} - \frac{n\overline x^2}{4\theta} + \text{constant} - \frac{n\theta} 4 \\[10pt]
\ell\,'(\theta) & = - \frac n {2\theta} + \frac{ns^2}{4\theta^2} +\frac{n\overline x^2}{4\theta^2} - \frac n 4 \\[10pt]
& = \frac {-n} {4\theta^2} \left( 2\theta - s^2 - \overline x^2 + \theta^2 \right). \\[10pt]
& = (\text{negative constant}) \cdot \left( (\theta+1)^2 -s^2-\overline x^2 - 1 \right)
\end{align}
Since $\theta$ cannot be negative, the above expression is $0$ only if
$$
\theta = -1 +\sqrt{s^2+\overline x^2 + 1}.
$$


Node body not preserving HTML formatting?

Question: The body area of my node is set to WYSIWYG. When I enter in HTML formatting (using the WYSIWYG buttons), it doesn't make it to the content type template:
<code>print $node->content['body']['#value'];
</code>
Why doesn't the HTML come through? How can I print the node body to the page and preserve the HTML formatting? What am I doing wrong here?
Thanks!
Answer: Your input format is likely "Filtered HTML", or something other than "Full HTML". Check the input format (below the body textarea when editing the node).
Comment: Hmm... I have a custom editor profile that I'm using. I just checked its settings though. I don't see a filtered/full HTML toggle anywhere. Where is that setting? Thanks!
Comment: You are most likely looking at the wrong place. You have to check the input filter. Make sure it's not stripping the html code. I always create a Full HTML filter with no filters attached to get this working.
Comment: I see Input Formats... but no Input Filter. Where should I be looking?!
Comment: Wait. Nevermind. You're right it was a filter! Turned that bad boy off and voila!


How to give permission for an IAM service account to run a docker container within a GCP VM?

Question: I am trying to run a docker image on startup of a Google Cloud VM. I have selected a fresh service account that I created as the Service Account under VM Instance Details through the console. For some reason the docker run command within the startup script is not working. I suspect this is because the service account is not authorized to run the "docker" command within the VM - which was installed via a yum install. Can anyone tell me how this can be done i.e. to give this service account the permission to run docker command?
Edit.
Inside the startup script I am running docker login command to login to Google Container Registry followed by a docker run to run an image.
Comment: What docker command are you running? Are you trying to pull an image? Depending on the docker command, it could be due to missing Cloud IAM roles or it could be related to something else.
Comment: In addition to what Jason said, look at the logfiles for the actual error message running docker.
Comment: I have updated the question with command details. Will check the log files as well.
Answer: I have found a solution and want to share it here so it helps someone else looking to do the same thing. The user running the docker command (without sudo) needs to have the docker group. So I tried adding the service account as a user and gave it the docker group and that's it. docker login to gcr worked and so did docker run. So the problem is solved but this raises a couple of additional questions.
First, is this the correct way to do it? If it is not, then what is? If this is indeed the correct way, then perhaps a service account selected while creating a VM must be added as a user when it (the VM) is created. I can understand this leads to some complications such as what happens when the service account is changed. Does the old service account user gets deleted or should it be retained? But I think at least an option can be given to add the service account user to the VM - something like a checkbox in the console - so the end user can take a call. Hope someone from GCP reads this.
Answer: As stated in this article, the steps you taken are the correct way to do it. Adding users to the "docker" group will allow the users to run docker commands as non root. If you create a new service account and would like to have that service account run docker commands within a VM instance, then you will have to add that service account to the docker group as well.
If you change the service account on a VM instance, then the old service account should still be able to run docker commands as long as the older service account is not removed from the docker group and has not been deleted from Cloud IAM; however, you will still need to add the new service account to the docker group to allow it to run docker commands as non root.
Update: automating the creation of a service account when at VM instance creation manually would be tedious. Within your startup script, you would have to first create the Service Account using the gcloud commands and then add the appropriate IAM roles. Once that is done, you would have to still add the service account to the docker groupadd directory.
It would be much easier to create the service account from the Console when the VM instance is being created. Once the VM instance is created, you can add the service account to the docker groupadd directory.
If you would like to request for a new feature within GCE, you can submit a Public Issue Tracker by visiting this site.
Comment: Thanks for confirming @Jason
Comment: if the approach in my answer is correct, does it make sense to automate creation of a service account userid in a VM (automatically) when a VM is created? Is it possible to provide this option for users in case they need it?
Comment: Updated my original answer.
Comment: Thank you again Jason. Will check the public issue tracker. To be clear again, what I meant to be automated is not creation of the service account (and IAM permission assignment). I meant to automate just the creation of the service account user within the VM. Thank you again


Limit with trinomials in numerator and denominator

Question: Again, I have a problem in the indeterminate form and I can't find out how to reduce this to something that'll give me the limit.
$$\lim_{ x\to-2} \frac{x^2-x-6}{x^2+x-2}=\frac{(-2)^2+2-6}{(-2)^2-2-2}=\frac{4+2-6}{4-2-2}=\frac{0}{0}$$
Comment: Do you know how to factor (quadratic) polynomials? If it makes it easier to factor, $x+2$ must divide both the numerator and the denominator. (Do you see why?)
Comment: Since the polynomials evaluate to $0$ at $x=-2$, they can both be factored as $(x+2)(\text{something})$. Find the something for each, cancel and take the limit again.
Answer: HINT $\ $ Consider the limit of a rational function $\rm\:f(x)/g(x)\:$ as $\rm\:x\to c\:.\:$  If $\rm\:f(c)=0=g(c)\:$ then, by the Factor Theorem, both $\rm\:f,\: g\:$ are divisible by $\rm\:x-c\:.\:$ Keep cancelling $\rm\:x-c\:$ from both $\rm\:f,\: g\:$ till this is no longer possible. Then at least one of $\rm\:f,\: g\:$ will no longer be divisible by $\rm\:x-c\:,\:$ so they will no longer both have $\rm\:c\:$ as a root, hence the limit will no longer be of indeterminate form $\:0/0\:.\:$ Thus you can always remove such "apparent singularities" from rational functions by simply cancelling such common factors.
Applying this to your specific example we deduce that it has numerator and denominator both divisible by $\rm\:x + 2\:.\:$ What happens when you proceed as above and cancel this common factor?
Later you'll learn about l'Hôpital's rule - which provides a more general method to resolve such indeterminate forms. For the special case of rational functions, l'Hospital's rule is essentially equivalent to the cancellation method described above.


How to prove that $|d(x,y)-d(z,w)|\leq |d(x,z)+d(y,w)|$?

Question: How to use the triangle inequality to prove that $|d(x,y)-d(z,w)|\leq |d(x,z)+d(y,w)|$?
Comment: $$x\to z \to w \to y$$
Comment: Use it twice, write it twice, combine.
Comment: You can drop the absolute value on the right hand side since a metric is non-negative
Answer: Combine the following two applications of the triangle inequality.
\begin{align}
d(x,y)
&\le d(x,z) + d(z,w) + d(y,w)\\
d(z,w)
&\le d(z,x) + d(x,y) + d(y,w)
\end{align}

Explicitly, rearranging each inequality above gives
\begin{align}
d(x,y) - d(z,w) &\le d(x,z) + d(y,w)\\
d(z,w) - d(x,y) &\le d(x,z) + d(y,w).
\end{align}
This implies
$$|d(z,w) - d(x,y)| \le d(x,z) + d(y,w).$$
Finally, the right-hand side is always positive, so you can include absolute value signs if you want.
Comment: Does the answer become $|d(x,y)−d(z,w)|\leq |(d(x,z)+d(y,w)+d(z,w))-d(z,w)\leq|d(x,z)+d(y,w)|$?
Answer: Use
$$d(x,y)\leq d(x,z)+d(y,w)+d(z,w)\implies d(x,y)-d(z,w)\leq d(x,z)+d(y,w)$$
and
$$d(z,w)\leq d(z,x)+d(x,y)+d(y,w)\implies d(z,w)-d(x,y)\leq d(z,x)+d(y,w)$$
to obtain the desired result.
Comment: Does the answer become $|d(x,y)−d(z,w)|\leq |(d(x,z)+d(y,w)+d(z,w))-d(z,w)\leq|d(x,z)+d(y,w)|$?
Comment: See the edited answer.
Answer: It may help to draw a picture.
Let $d(x,y)=a, d(w,z)=b, d(x,z)=c,d(y,w)=d, d(x,w)=e,d(y,z)=f.$ We wish to show $|a-b|\leq c+d. $ We have $$(i).\quad b\leq c+e\leq c+(d+a)\implies b-a\leq c+d. $$ $$(ii).\quad a\leq c+f\leq c+(b+d)\implies a-b\leq c+d. $$
Answer: First prove this result, which is a very useful lemma in itself:
$$
\tag{*}\label{eq:ti}
|d(x, z) - d(y, z)| \leqslant d(x, y).
$$
Proof: this is equivalent to the conjunction of the inequalities
\begin{align*}
d(x, z) - d(y, z) & \leqslant d(x, y), \\
d(y, z) - d(x, z) & \leqslant d(x, y),
\end{align*}
each of which is a case of the Triangle Inequality (taking into account the symmetry of the function $d$).
Using \eqref{eq:ti} (twice), the symmetry of $d$ (again), and the Triangle Inequality for absolute values, we have
\begin{align*}
|d(x, y) - d(z, w)| & = |d(x, y) - d(z, y) + d(z, y) - d(z, w)| \\
& \leqslant |d(x, y) - d(z, y)| + |d(z, y) - d(z, w)| \\
& \leqslant d(x, z) + d(y, w).
\end{align*}


$\phi_i$ measurable $\implies$ pointwise $\lim\sup\phi_i$ measurable - explanation of proof?

Question: I saw this post but was still a little confused. My book shows that to prove this it is sufficient to make use of the following identity:
$$\limsup_{i\rightarrow\infty}\phi_i(x) = \inf_i\{\sup_{j\geq i}\phi_j(x)\}$$
And I'm having a bit of trouble understanding what this says. I've gotten a better understanding of $\limsup$ from my post yesterday, but still a bit confused.
So for a sequence of functions $\phi_i$, the $\limsup$ for a given point x is the limit of the supremum of $\phi_i(x)$. This could be increasing as i increases, or decreasing, or staying the same. And this is the same as taking the infimum over i of the set of supremums of $\phi_j$ for $j\geq i$? This last part is confusing me.
So for all i, we look at the $\{\sup_{j\geq i}\phi_j(x)\}$. And then we look at the greatest lower bound of this set? How is this the same as working with the limit?
And finally, how does this help us make it measurable?
Thanks for any help!
Comment: The sequence $\{\sup_{j\ge i} \varphi_j(x) \}_{i\in\mathbb{N}}$ is **decreasing** in $i$.  Basically, the larger a set is, the more chances you have of getting a large value.  As you remove things from the set, you potentially remove big things.  More formally, $$A\subseteq B \subseteq \mathbb{R} \implies \sup_{x\in A} x \le \sup_{x\in B} x. $$  This can be applied to the sequence of values above by setting $A_i = \{ \varphi_j(x)\}_{i\ge j}$, and noting that $A_i \supseteq A_{i'}$ whenever $i < i'$.
Comment: Could you explain why $\{\sup_{j\geq i}\phi_j(x)\}$ is a sequence? I guess I was confused in the first place about what makes it a set. So I thought $\sup_{j\geq i}\phi_j(x)$ is just the supremum of the sequence of functions, over all functions in the sequence past a certain point. But shouldn't there still only be one value that is a supremum of all those values?
Comment: $\sup_{j\ge i} \varphi_j(x)$ is just a number---the supremum of a set is a number.  For each $i$, you obtain a different such number.  Taken together, these numbers form a sequence, i.e. the sequence $\{ \sup_{j\ge i} \varphi_j(x) \}$, indexed by $i$.
Answer: Observe that when $k\geq j,$ $\sup_{i\geq k}\{\phi_{i}(x)\}\leq\sup_{i\geq j}\{\phi_{i}(x)\},$ since the set $\{\phi_{i}(x):i\geq j\}$ contains $\{\phi_{i}(x):i\geq k\}.$ Then the sequence $(\sup_{i\geq j}\{\phi_{i}(x)\})_{j\geq 1}$ is decreasing, so the limit of this sequence must equal its infimum, which proves the identity in your book.
Now from the answer you linked, $\overline{\phi}_{j}(x):=\sup_{i\geq j}\{\phi_{i}(x)\}$ is a measurable function for every $j\geq 1,$ and by our reasoning above, $(\overline{\phi}_{j})_{j\geq 1}$ is a decreasing sequence of functions. Therefore the pointwise limit is measurable, as we wanted to prove.


Must a complex power series *fail* to be convergent somewhere on its circle of convergence?

Question: My textbook asserts so, but I can't find its proof of the claim. On the other hand, a lecture slide I'm cross-referencing claims that a power series is allowed to be convergent at ALL points of its circle of convergence.
So who is correct?
Comment: It fails to converge where it hits a non-correctable singularity, where the function has no value.
Comment: I think the power series for $\sqrt {z+1}$ should converge on all of $S^1$, but it cannot extend beyond $z=-1$.
Answer: It seems to me that
$$\sum_n \frac{x^n}{n^2}$$
is convergent everywhere on the circle of convergence. This follows immediately from the triangle inequality and the convergence of $\sum_n \frac{1}{n^2}$.
Comment: See also http://en.wikipedia.org/wiki/Radius_of_convergence#Convergence_on_the_boundary. What happens "just outside" the compact disk of convergence?
Comment: I was thinking to the same one. This is absolutely convergent for |x|=1, hence it must be convergent.
Answer: As @N.S. demonstrated, the power series itself can converge at every point of the circle of convergence. The correct statement is that the function represented by the power series on the interior of the disk of convergence must have a singularity at (at least) one point of the circle. That is, it cannot be analytically continued to give an analytic function on a larger disk. For, if it could, the radius of convergence would have to be larger.


Syntax error near unexpected token `fi'

Question: I have been trying to write a while loop bash script that will check the size of two directories, and echo that out to a text file in the www folder. After my while loop stubbornly refused to work, I decided to simplify it to make sure I could get aa basic "if statement" to work, but for some reason after hours of frustration, even with the most basic of if statements, it will not work. Any idea what I am missing?
echo "hello World"
#
a=1
b=2
echo "A: $a"
echo "B: $b"
if [ "$a" -eq "$b" ];
then
echo "equal"
else
echo "Not equal"
fi

Every time I run that, I get this error:

test.sh: line 12: syntax error near unexpected token `fi'
test.sh: line 12: `fi'

If I put "exit" or "done" at the end of the file, it tells me unexpected end of file.
Thanks so much!
Comment: Works for me as entered above.
Answer: I copied pasted your script and ran it through bash and got the expected output:
$ bash test1.sh
hello World
A: 1
B: 2
Not equal
$ cat test1.sh
echo "hello World"
#
a=1
b=2
echo "A: $a"
echo "B: $b"
if [ "$a" -eq "$b" ];
then
echo "equal"
else
echo "Not equal"
fi

Perhaps you've got an extra linefeed after the fi or something?  Can you check with cat -v or simply create a new script and copy/paste the above yourself?
Comment: Thank so much guys! I just realized that my encoding in Notepad++ had somehow gotten changed to "Windows/Dos" instead of "UNIX". I went and used the menu to switch it back, and now it works like a charm!
Comment: Mind if I point out that it is good practice to always prefix shell scripts with the appropriate hashbang line, "#!/bin/sh" if it is really within the confines of POSIX shell syntax or "#!/bin/bash" if the script is assumed bash specific. Some environments DO get it wrong in unexpected ways if you don't. For example, some linux systems will use busybox, dash, ash... by default for a noninteractive script (eg cron driven), which usually ends up in a script that was written with bash specific features to almost work in the most confusing way.
Comment: @rackandboneman agreed, that's why I called this snippet using `bash test1.sh`.

Syntax error near unexpected token `fi'


Modify Bootstrap DataTable

Question: I used an example of bootstrap datatable to have a sortable table on my symfony project. I use datatables.net
With this javascript code :
$(document).ready(function() {
    $('#datatable').DataTable();
} );

And my table in HTML :

...

The only problem is that I don't want all my columns to be sortable (some columns only contain a checkbox or a button for example). Moreover with this table and this javascript code I automatically have a search bar as well as Previous Next buttons and other options that appear automatically. I would like to be able to modify this default template by removing all these options that I don't need. Do you have an idea how to adapt this datatable?
Thanks in advance
Comment: https://stackoverflow.com/questions/16335928/how-to-remove-sorting-option-from-datatables Might help you. Or directly into the DataTables doc: https://datatables.net/reference/option/columnDefs
Comment: Yes I found It thanks
Answer: You should read documentation for datatables. https://datatables.net/extensions/rowreorder/examples/initialisation/restrictedOrdering.html
Example:
// add class reorder on column, that you want to order
$(document).ready(function() {
    var table = $('#example').DataTable( {
        rowReorder: true,
        columnDefs: [
            { orderable: true, className: 'reorder', targets: 0 },
            { orderable: false, targets: '_all' }
        ]
    } );
} );

Comment: Thank you! Indeed I found the solution in this part of the documentation

Modify Bootstrap DataTable


Why are there four solutions to $x^2-2x-8=0$ in $\mathbb{R}$? Or am I wrong?

Question: It might be a very trivial question to ask but why do we get four different solutions for a quadratic equation using these two methods?
$x^2-2x-8=0$
We see that factors are $(x-4)$ and $(x+2)$ so we get $x=4$ or $- 2$.
Now when we factorise in the following way we get different answers:
$x^2-2x=8$
$x(x-2)=8$ [How can this be!?]
And we get $x=8$ or $x=10$ [How!?]
I am very confused.
Comment: Why are there downvotes? This is easy math for several people but effort has been shown. Downvotes are really unnecessary.
Comment: Now I did not dowbvote and it is not a big deal, but then that the title is "Flaw in Mathematics??" is, well, maybe a bit over the top.
Comment: The question is pretty clear too. It asks whether there's a flaw in the chain of reasoning. In fact there _is_ a flaw, so the answer is to show where the flaw is.
Comment: @crash Agreed, though it's come to be expected, no?
Comment: To reinforce what quid said, the title "Flaw in Mathematics" is bound to be highly provocative (and negatively received). So a more apropos title would be nice. Nonetheless, kudos for being daring enough to post your question and to ask for feedback. You've certainly gotten quite a bit of it!
Comment: @JonathanHebert Unfortunately, yes, that has come to be expected. A small handful of people here hellbent on ruining the site. :/
Comment: For your second factorization: When you plug in $x = 8$, you get $8(8 - 2) = 8$. Is that really solution to your equation?
Comment: How stupid of me... Lol....
Comment: FOR all the people who think answering this question was easy i request you to please please please answer this question in the easiest possible way...http://math.stackexchange.com/questions/1216246/determining-the-3rd-vertex-of-an-equilateral-and-right-angled-isoceles-triangle/1216348?noredirect=1#comment2475059_1216348
Comment: Typically, for example on an exam, when you get what you think is a solution to a quadratic equation, it is easy to check your result by substitution. Had you done that and had that realization, you might have phrased your question a bit more responsibly - such as "Flaw in my reasoning" rather than as you did.
Comment: See the topic... And read my last comment :)
Comment: Well, crash certainly improved things, yet you might further consider changing "we" to "I"
Comment: Guys forget about this question now please... Answer this one please..http://math.stackexchange.com/questions/1216246/determining-the-3rd-vertex-of-an-equilateral-and-right-angled-isoceles-triangle/1216348?noredirect=1#comment2475059_1216348
Comment: @Andrew The "royal we" strikes again!
Answer: Your mistake is to assume that for $a,b,c\in\mathbb{R}$,
$$ab=c\quad\implies\quad  a=c\quad\text{or}\quad b=c.\tag{1}$$
This is not true in general.
For example $2\times\frac{1}{2}=1$, but neither $2=1$ nor $\frac{1}{2}=1$.
Further explanations: However, $(1)$ is true when $c=0$, and this is why when finding the roots of a polynomial we can factor it and use the argument
$$(x-\alpha)(x-\beta)=0\quad\implies\quad x=\alpha\quad\text{or}\quad x=\beta.$$
To prove that if $ab=0$ then $a=0$ or $b=0$, you can argue by contradiction. If both $a\neq 0$ and $b\neq 0$, then surely $ab\neq 0$.
Comment: @user166748 You should consider placing a bounty on that question, when possible, if you really want the answer. This isn't the way of going about obtaining one.
Comment: Great answer.. @Spenser please answer this question also in the easiest possible way...http://math.stackexchange.com/questions/1216246/determining-the-3rd-vertex-of-an-equilateral-and-right-angled-isoceles-triangle/1216348?noredirect=1#comment2475059_1216348
Answer: Spenser's answer is great because it addresses the general issue. Perhaps to add some clarity, let's look at your particular quandary.
You realized that
$$
x^2-2x-8 = 0\Longleftrightarrow (x-4)(x+2)=0\Longleftrightarrow x=4,-2.
$$
Okay. That's great. Now we can rearrange, as you did, for the following:
$$
x^2-2x-8=0\Longleftrightarrow x^2-2x=8\Longleftrightarrow x(x-2)=8\Longleftrightarrow x=?
$$
Try plugging in $x=8$ or $x=10$. They do not work. However, plugging in $x=4$ and $x=-2$ works. It's simply less obvious that these $x$-values work when you express your quadratic in this way.
Answer: You should notice that's only when ab=0 then a=0 or b=0
but when you have a*b=8 you can't say a=8 or b=8
for instance a could be equal to 4 and b to 2
Comment: It's funny that as a counter-example, you picked the one instance (WLOG) that isn't actually a counter-example.
Comment: Much better ;-)
Answer: You should do a simple sanity check in a case like this. Plug your answer back into the original equation and make sure it works. Use a calculator and nothing more complicated:
Your tentative answers $x = 4, -2, 8, 10$ check out  as
$$(4)^2-2\times(4)-8\stackrel{?}{=}0\space \space \checkmark$$
$$(-2)^2-2\times(-2)-8\stackrel{?}{=}0\space \space \checkmark$$
$$(8)^2-2\times(8)-8\stackrel{?}{=}0\space \space  NO!$$
$$(10)^2-2\times(10)-8\stackrel{?}{=}0\space \space NO!$$


Como usar o comando between no codeigniter para gerar relatório

Question: Preciso de gerar o seguinte relatório:
O usuário seleciona o intervalo de tempo (data início e data fim) e o relatório de receitas é gerado exibindo o valor das receitas somados.
Segue a foto da view que chama a função na classe Model:

Código da View (foto acima):

Relatório de Receitas

    <form action="<?= base_url() ?>receita/pagas" method="post">

                Data do recebimento entre:

                    <input id="dtrecinicio" class="form-control" type="text" name="dtrecinicio" required>

                E

                    <input id="dtrecfim" class="form-control" type="text" name="dtrecfim" required>

                        <button class="btn btn-md btn-primary btn-block" type="submit">GERAR</button>

                        <a class="btn btn-md btn-default btn-block hidden-print" href="<?= base_url(); ?>">CANCELAR

    </form>

Função da Classe Model:
    function get_receitas_like() {
        $inicio = $this->input->post('dtrecinicio');
        $fim = $this->input->post('dtrecfim');
//        $this->db->select('*');
        $this->db->select_sum("valor_receita");

        $this->db->where('dtrecebimento' >= $inicio);
        $this->db->where('dtrecebimento' <= $fim);
//        $this->db->where('dtrecebimento' BETWEEN  AND $dtrecebimento2');
//        $this->db->where('dtrecebimento', $termo);
        return $this->db->get('receitas')->result();
    }

Código da View que exibe os dados somados:

RELATÓRIO - RECEITAS

                        Valor total recebido no período selecionado: R$ <?php echo number_format($receitas[0]->valor_receita,2,",","."); ?>

Foto da tabela Receitas:
Answer: Algumas modificações:
Se sua data está no formato correto a data que vem da tela não está, precisa ser convertida para Date e depois na SQL passa a formatação correspondente, exemplo:
$inicio = DateTime::createFromFormat('d-m-Y', $this->input->post('dtrecinicio'));
$fim = DateTime::createFromFormat('d-m-Y', $this->input->post('dtrecfim'));

e mude:
$this->db->where('dtrecebimento' >= $inicio));
$this->db->where('dtrecebimento' <= $fim);

para
$this->db->where('dtrecebimento >=', $inicio->format('Y-m-d'));
$this->db->where('dtrecebimento <=', $fim->format('Y-m-d'));

os sinais tem que ficar dentro do primeiro parâmetro.
Referencia: Custom Where
Comment: Obrigado pelo retorno @Virgilio Novic só que apesar da sintaxe correta do "where" a soma não está sendo feita de forma correta, está gerando um número muito alto, não condizente com os dados do banco.
Comment: @Ramiro é o seguinte a sintaxe é essa para `between` entre valores no caso data e a sua duvida era sobre isso, se a soma está vindo diferente é porque está faltando alguma coisa antes, você na sua pergunta poderia disponibilizar mais informações, tipo eu tenho esse conjunto de informações e o resultado da soma é !!! pra gente pode ajudar mais... essa parte da data foi solucionada, agora a SQL antes não está bem clara na pergunta se puder atualizar a pergunta podemos ajudar.
Comment: Ok @Virgilio Novic consegui resolver meu problema com o 'like' mesmo, pois com o between não sei porque a soma gerada estava vindo errada. Obrigado mais uma vez.


Statistical analysis for replication in very small experimental datasets

Question: A colleague does replication on a quite coslty experiments. There are four different conditions, each one duplicated. The  outcome with $4\times 2 = 8$ points is illustrated below:

The analysis is standardized: each pair of colored points  yields an average $\mu$ associated with its dispersion $d$, to assess the variability in the  given condition. Black diamonds apparently exhibit higher variation, for instance. The colleague directly computes the standard deviation on each pair  of points, and uses $d=2\sigma$. I know statistics on such small dataset can be touchy. I would like to provide advices to keep numbers as clean as possible on this condition.
My questions are the following (I am not a trained statistician):

The variance is computed directly, and not corrected by the $\frac{1}{n-1}$ factor (Bessel's correction), which I believe would be the standard with sample estimation (see here). In fact, this is the same here, since $n=2$, for each pair, so $n-1 = \sqrt{n-1} = 1$. Am I correct in insisting on mentioning this factor in an explicit manner, although it does not change the results? By "explicit manner", I mean writing in documents, or even in Excel macros (in case somebody use them in a different context), that $d=2\sqrt{\frac{(x_1-\mu)^2+(x_2-\mu)^2}{n-1}}$, instead of $d=2\sqrt{(x_1-\mu)^2+(x_2-\mu)^2}$, which the colleague uses, even if it does not make a difference. Some people are afraid of formulas in some fields.
Sometimes, the  colleague gathers the  red and black dots. Am I correct that in this case the correction on $\sigma$ is  mandatory, as now $n =4$ for one quadruplet, and $n=2$ for each of the two remainging couples?
I feel that if the experiments could be triplicated, one could expect a "huge" gain in estimating sample  dispersion, since one morally would expect to divide the estimate by $\sqrt{3-1}$ instead of $\sqrt{2-1}$. Does that sounds sound enough?

I know two points is not much, and we are close to the singularity limit of one-sample statistics. I have heard a colleague say: "I do not replicate, to avoid dispersion".
Comment: I'm hesitant to answer because I'm not a publishing academic, but I certainly think it's important to mention when nonstandard computations are used. By "directly," do you mean "dividing by $n$" instead of "dividing by $n-1$?"
Comment: @ssdecontrol Do not hesitate, I am pretty such datasets rarely appear in the academic statistical literature. I meant that I would like the colleague to write the correct formula, even if it gives the same results as a casual one
Answer:
Yes, you are correct. That calculation is standard for a reason.if your colleague feels the need to use a different calculation, it is their responsibility explain why, and to enumerate the different calculation they use. That formula is not "casual," it is wrong. If anything, what would be "casual" would be to use the standard calculation from any statistics software, which in most cases will use $n-1$ in the denominator.
Errors propagate. In this case, using the wrong calculation will yield the wrong results even on a relative scale, so your colleague's wrongness will just be compounded.
Keep in mind that you will be increasing the numerator as well: there's no way to tell (without some prior knowledge) whether the third point will increase dispersion or decrease it. However, adding a point will always improve the precision of the dispersion estimate: with $n=3$ the variance of the variance estimate would be $\frac{\mu^4}{3}$, whereas with $n=2$ is would be $\frac{\mu^4}{2} + \frac{\sigma^4}{2}$. The latter is always larger than the former.
Comment: Could you please provide me with a pointer for the  variance of the variance estimate?
Comment: @LaurentDuval it's in that link


Is there a safe way to reuse the same code for immutable and mutable variants of a function?

Question: I did a bit of searching, and found this Reddit post from four years ago, but other than unsafe code:
fn find_mut <'a> (&'a mut self, elem: &T) -> Option<&'a mut Node<T>> {
    unsafe{std::mem::transmute(self.find(elem))}
}

or a macro, I can't think of any ways to do this.
Alternative restatements of the problem:

Make a function generic over mutability.
Mutability modifier polymorphism.

Is there a way to do this now?

Motivation, just in case this is another XY problem: I'd like to write a function that processes a vector of references, reading them but not modifying them, and I would like to use those functions both in cases where I will have &Ts and &mut Ts.
Comment: Note that transmuting an immutable reference to a mutable reference is **always** unsafe, [don't do it ever](https://doc.rust-lang.org/nomicon/transmutes.html).
Answer: Based on the motivation you described at the end of your message, my understanding is that you'd like to inspect Vec<&T> and Vec<&mut T> with one function and without modifying the underlying Ts. Does this do what you want? Note that I used references to slices because it's more idiomatic as parameter than references to vectors as the latter coerce to the former.
Playground
use std::ops::Deref;

struct Foo;

fn do_stuff(_: &Foo) {}

fn process<T>(v: &[T])
where
    T: Deref<Target = Foo>,
{
    for x in v.iter().map(|x| x.deref()) {
        do_stuff(x);
    }
}

fn main() {
    process(&[&Foo]);
    process(&[&mut Foo]);
}

Using Borrow instead of Deref, you can also pass &[Foo].


How does postDelayed work?

Question: I generate a number of ImageViews in the same location. Now, I want to apply an animation to each of these images, but not in the same time.
Let's say I want the first image to start moving at sec 0, then image2 at sec 2 and so on.
I have the images in an array, imb_items[].
if i use this:
for(int j = 0; j < i; j++) {
    final int finalJ = j;
    new android.os.Handler().postDelayed(
        new Runnable() {
            public void run() {
                imb_items[finalJ].startAnimation(translateAnim);
                Log.i("tag", "This'll run 5000 milliseconds later");
            }
        },
        5000*finalJ
    );
}

When the second image start moving, the first one stops and disappears.
How could I solve this?  I would like all the images to continue their movement.
Comment: what's the duration of animation?
Comment: the animation has 8.5 seconds
Comment: And the images are all visible, and in the same location. I want them to start moving, sequentially, one after another, until they all move 8.5 seconds
Comment: what does `5000*finalJ` mean?
Answer: it's not working because you're trying to interact a for-loop with the delayed execution of something else. It doesn't work like this.
but why not just set the start offset of the animation?
https://developer.android.com/reference/android/view/animation/Animation.html#setStartOffset(long)
have an array of ImageView and an array of Animation (one animation for each ImageView)
for(int j=0; j<i;j++) {
   translateAnim[j].setStartOffset(5000*j);
   imb_items[j].startAnimation(translateAnim[j]);
}

How does postDelayed work?


Async Await with many LongRunning Parallel tasks (threads)

Question: I'm trying to launch 500 requests to a REST service simultaneously.  I wrote this extension method to help with that and it works sometimes but other times it hangs on the WhenAll with tasks that haven't been activated.  Can anyone help me understand this and/or show me a better way?
I've tried to do this without the LongRunning flag and it hangs (or takes longer than I'm willing to wait) every time and my understanding is that it was just overwhelming the thread pool.  When it does work it only takes about 4 seconds.
public static async Task<List<TFunc>> ForEachLongRunningAsync<TSource, TFunc>(this IEnumerable<TSource> source, Func<TSource, Task<TFunc>> func)
{
    var tasks = source.Select(x => Task.Factory.StartNew(() => func(x), TaskCreationOptions.LongRunning | TaskCreationOptions.AttachedToParent).Unwrap());
    var results = await Task.WhenAll(tasks);
    return results.ToList();
}

As a troubleshooting step, I modified the method to this.  This is how I determined that the tasks aren't being activated (see below).  Again, in this case it works sometimes but hangs in others.
    public static async Task<List<TFunc>> ForEachLongRunningAsync<TSource, TFunc>(this IEnumerable<TSource> source, Func<TSource, Task<TFunc>> func)
    {
        var tasks = source.Select(x => Task.Factory.StartNew(() => func(x), TaskCreationOptions.LongRunning | TaskCreationOptions.AttachedToParent).Unwrap()).ToList();
        var results = Task.WhenAll(tasks);
        while (true)
        {
            if (results.IsCompleted) break;
            await Task.Delay(100);
        }
        return results.Result.ToList();
    }

Comment: By declaring a task to be long-running, you're declaring that you don't want it to use the threadpool. Long-running tasks spawn up new threads. Why do you call Unwrap?
Comment: "I'm trying to launch 500 requests to a REST service simultaneously" and does your http client allow that many concurrent requests to the same server? Will those requests timeout before they have even started? And what about limits imposed by the server?
Comment: Yes, it allows that many concurrent requests. No, they will not timeout. The limits are well beyond what I'm trying to do.
Comment: Does this answer your question? [Throttling asynchronous tasks](https://stackoverflow.com/questions/22492383/throttling-asynchronous-tasks)
Answer: If your func is known to be asynchronous (i.e., just invoking a REST call), then there's no need to involve the thread pool at all:
public static async Task<List<TFunc>> ForEachLongRunningAsync<TSource, TFunc>(this IEnumerable<TSource> source, Func<TSource, Task<TFunc>> func)
{
  var tasks = source.Select(x => func(x));
  var results = await Task.WhenAll(tasks);
  return results.ToList();
}

If you do need the thread pool for some reason (i.e., if func may be synchronous and you want to ensure it runs in parallel in that case), then the proper tool to use is Parallel.ForEachAsync or Task.Run:
public static async Task<List<TFunc>> ForEachLongRunningAsync<TSource, TFunc>(this IEnumerable<TSource> source, Func<TSource, Task<TFunc>> func)
{
  var tasks = source.Select(x => Task.Run(() => func(x)));
  var results = await Task.WhenAll(tasks);
  return results.ToList();
}

Note that both LongRunning and AttachedToParent are wrong to use with asynchronous code, and StartNew is not recommended. LongRunning forces new threads to be created and then almost immediately discarded, making the code far less efficient. AttachedToParent attempts to attach to the currently-running task which is hopefully nothing - if it actually did do something it would almost certainly interfere with the generated async state machine.
Answer:
In spite of what you said, I suspect that you will run into timeouts if you try to launch 500 requests to a REST service simultaneously. Keep in mind that client timeouts, server-side rate limiting, server-side exhaustion, and server-side timeouts are all in play.
And I doubt that you have a business requirement to truly launch 500 requests to a REST service simultaneously.

Instead of running a pure Task.WhenAll(tasks) with 500 tasks, I suggest you throttle with, perhaps, 5 simultaneous tasks.
And I encourage you to benchmark this change. I think you will be pleasantly surprised at the throughput you'll achieve if you constrain your Task.WhenAll(tasks) with just 5 simultaneous tasks.

using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

namespace ConsoleApp2
{
    internal class Program
    {
        static void Main(string[] args)
        {
            var rnd = new Random();
            var throttler = new SemaphoreSlim( /*degreeOfParallelism:*/ 5);

            var tasks = Enumerable.Range(1, 20).Select(i => PrintNumber(i, throttler, rnd));
            Task.WhenAll(tasks).GetAwaiter();

            Console.ReadKey();
        }

        private static async Task PrintNumber(int number, SemaphoreSlim throttler, Random random)
        {
            using (await throttler.Throttle().ConfigureAwait(false))
            {
                await Task.Delay(TimeSpan.FromSeconds(random.Next(1, 5))).ConfigureAwait(false);
                Console.WriteLine(number);
            }
        }
    }

    internal static class ThrottlerExt // https://gaevoy.com/2019/01/27/throttling-async-task.html, 2023-12-21
    {
        internal static async Task<IDisposable> Throttle(this SemaphoreSlim throttler)
        {
            await throttler.WaitAsync().ConfigureAwait(false);
            return new Throttler(throttler);
        }

        private class Throttler : IDisposable
        {
            private readonly SemaphoreSlim _throttler;

            public Throttler(SemaphoreSlim throttler) => _throttler = throttler;

            public void Dispose() => _throttler.Release();
        }
    }
}


Conversion fails (SQL Server 2008)

Question: I have a column
StudentData (varchar)
Sample values:
Not regular
90
15.2
55
4
0
>15
regular
35

I have a condition to find out all the students which have StudentData value greater than 10
or have a NULL value.
When I try to convert it fails saying conversion fails. How can get those values? I am expecting a resultset something like
studentdata
90
15.2
55

35

Please advice.
Comment: You want to look at using the ISNUMERIC function
Comment: how can i handle the NULL values?
Answer: I think you're looking for this:
select * from t
where studentdata is null or
  (isnumeric(studentdata) = 1 and CAST(studentdata AS float) > 10)

Anyway, you should consider storing appropriate information in that column, instead of a mix of things.
Let me know if you have any issue with it.
Answer: You could use isnumeric() but it has some issues. IsNumeric() Broken? Only up to a point.
Something like should work for you.
select *
from YourTable
where
  case when isnumeric(studentdata+'E0') = 1
    then cast(studentdata as float)
    else 0
  end > 10

Conversion fails (SQL Server 2008)


Flex div with table not following the flex width

Question: I have a composition of flex itens in a column and row, as the follow image:

This images are built using the following code:

    THIS IS A TEST CODE WITH TEXT

      CONTENT OF LEFT SIDE

      CONTENT OF RIGHT SIDE

    THIS IS A TEST CODE WITH TABLE

            Column 1
            Column 2
            Column 3
            Column 4

            Data 1
            Data 2
            Data 3
            Data 4

      CONTENT OF RIGHT SIDE

CSS:
.container {
  display: flex;
  flex-direction: column;
  background-color: grey;
  width: 100%;
  height: 200px;
}

.header {
  flex: 1;
  text-align: center;
}

.content {
  flex: 1;
  display: flex;
  flex-direction: row;
}

.leftcontent {
  flex: 5;
  background-color: green;
}

.rightcontent {
  flex: 5;
  background-color: yellow;
}

The upper schema is a test made with text. The bottom schema is a try using a table on the left side.
As you can see, at the bottom schema, the table is larger than expected and is not following my flex rules. I know tables has a special rendering mechanism, but I need help to set the table width as 50% like the schema with text only. In that case I need the overflowed table to show up an horizontal scroll, just in the left side of the content (that related to the table so I can scroll it horizontally and see its contents).
Here is the JSFiddle for it
Comment: to keep table within a width set, you need to reset table-layout . but, that works also with a width set as reference https://jsfiddle.net/z2r6qxz1/2/ or  overflow on parent https://jsfiddle.net/z2r6qxz1/5/
Comment: Just add `overflow: auto` to `.leftcontent`. https://jsfiddle.net/z2r6qxz1/4/
Answer: min-width defaults to auto, which mean it won't shrink smaller than its contents size.
Add min-width: 0 to left-content and it will allow it to shrink.
Updated fiddle
Stack snippet

.container {
  display: flex;
  flex-direction: column;
  background-color: grey;
  width: 100%;
  height: 200px;
}

.header {
  flex: 1;
  text-align: center;
}

.content {
  flex: 1;
  display: flex;
  flex-direction: row;
}

.leftcontent {
  min-width: 0;
  flex: 5;
  background-color: green;
}

.rightcontent {
  flex: 5;
  background-color: yellow;
}

    THIS IS A TEST CODE WITH TEXT

      CONTENT OF LEFT SIDE

      CONTENT OF RIGHT SIDE

    THIS IS A TEST CODE WITH TABLE

            Column 1
            Column 2
            Column 3
            Column 4

            Data 1
            Data 2
            Data 3
            Data 4

      CONTENT OF RIGHT SIDE

If you want to be able to scroll the content, use overflow: auto or òverflow: scroll instead
Stack snippet

.container {
  display: flex;
  flex-direction: column;
  background-color: grey;
  width: 100%;
  height: 200px;
}

.header {
  flex: 1;
  text-align: center;
}

.content {
  flex: 1;
  display: flex;
  flex-direction: row;
}

.leftcontent {
  overflow: auto;
  flex: 5;
  background-color: green;
}

.rightcontent {
  flex: 5;
  background-color: yellow;
}

    THIS IS A TEST CODE WITH TEXT

      CONTENT OF LEFT SIDE

      CONTENT OF RIGHT SIDE

    THIS IS A TEST CODE WITH TABLE

            Column 1
            Column 2
            Column 3
            Column 4

            Data 1
            Data 2
            Data 3
            Data 4

      CONTENT OF RIGHT SIDE

Comment: Thanks @LGSon. I had to add `overflow-x: scroll` also to allow horizontal scroll of table.
Comment: @Mendes Yes, based on how you want to be able to access the content you would might need that.
Comment: @Mendes Updated my answer with a 2nd sample...and you don't need both properties


performing multiple pairwise comparisons against a single control with ggstatsplot

Question: In my dataset I have a single negative control and then numerous groups that are treated with a compound in increasing doses. I have figured out how to automate running ggbetweenstats on my large list of these treatments.
library(Rmpfr)
library(ggstatsplot)

 ggstatsplot::ggbetweenstats(
          data = data,
          x = treatment,
          y = value,
          messages = FALSE )

However, the ggbetweenstats method compares every single group against every other group and adjust accordingly for multiple hypothesis testing. However, in my case I only would like to compare each treatment against the untreated control (with and without multiple hypothesis testing).
Is there a way to use one group as the single baseline against which automatically all other treatments are measured? Or do I need to implement this in a separate loop/function? I would prefer to run this in ggstatsplot since it would generate a convenient set of plots that are easy to scroll through.
EDIT: It is unclear to me if 'Cross Validated' or 'Stack Overflow' is the better place to ask.
Comment: As this question is fundamentally about statistical practice rather than programming per se, this is the correct place to post. If you cross-posted this question on Stack Overflow, please remove from there.
Answer: A standard test for comparing multiple treatments against a single control is Dunnett's test. This page shows an example of how to implement that test in R with the DescTools package.
In your situation, however, that test will throw away the quantitative dose-response characteristics of your study. Your study might be better served by using an analysis designed for dose-response studies, like that provided by the R drc package.


Find an expression equalling 69 using the digits 1 9 9 2 in order

Question: A friend gave me a puzzle to find an expression for each of the integers $1$ to $100$ using the digits $1, 9, 9, 2$ such that they appear in that exact order. You're allowed any combination of the four basic operations, brackets, square roots, the factorial, exponentiation and also you can put digits together, e.g. we could have $23=19+(\sqrt{9})!-2$. I've managed every number relatively quickly except $69$ which I've been stuck on for ages and it's infuriating! Any ideas?
Edit: Apologies for my unclear explanation. You're only allowed to put the original digits together in the equation. The equation must make sense as a normal equation so you can't do things like combine $1$ with $\sqrt{9}$ to get $13$ because this can't be written down in a way which makes sense as a normal equation.
Also you aren't allowed to just use powers, unless you're using one of the four digits themselves, e.g. $2=1+(9/9)^2$ is okay because we've used the $2$ as a power but $69=(1-9)^2+\sqrt{9}+2$ isn't because there's now a superfluous $2$.
Comment: How did you get 46
Comment: $(-1+9).(\sqrt{9})!-2$
Answer: Since we can put digits together, one solution might be
$1+ ((\sqrt{9})!)((\sqrt{9})!) +2$
One expression which does not keep the exact order is
$9*9-12$ or $((\sqrt{9})!)((\sqrt{9}+1) +2$
Comment: No, this violates the edit about the concatenation about digits.
Answer: I came up with:
$$(1-9)^2+\sqrt{9}+2.$$
Comment: Unfortunately you're not allowed squares only square roots.
Answer: I'm not sure if this meets the "makes sense as a normal equation" condition, but if we can apply the square root operator an infinite number of times we have:
$$69 = 71-2 = \sqrt{5041} - 2 = \sqrt{1 + 7!} - 2 = \sqrt{1 +(3!+1)!}-2$$
$$= \sqrt{1 + \Big((\sqrt9)! + 1\Big)!}-2 = \sqrt{1 + \Bigg((\sqrt9)! + \sqrt{\sqrt{\sqrt{ \cdots9}}}\Bigg)!}-2$$
where the dots indicate an infinite sequence of nested square roots.  Note that:
$$\sqrt{\sqrt{\sqrt{ \cdots9}}} = \lim_{n\rightarrow\infty}\Big(9^{(1/2)^n}\Big) = 1$$
Comment: Such contests occasionally allow bracket=floor function. Meaning that in place of that limit you can use $\lfloor\sqrt{\sqrt{9}}\rfloor=1$. Not sure?
Comment: No, the "infinite number of times of a square root" does not follow, as it was explained byJyrki Lahtonen, gives it for the floor function.
Answer: I can't comment, therefore I am posting this as an answer and delete it later.
$(\sqrt{9})!9 = 69$, since $(\sqrt{9})!=6$, since you said, it is allowed to put digits together.
Getting rid of the 1 is not a problem $1\cdot n=n$.
But still I need to get rid of the 2.
edit:
$1 + \underbrace{(\sqrt{9})! (\sqrt{9})!}_{=66} + 2 = 69$
Comment: Apologies for not being clear, you're only allowed to put the original digits together, i.e. it has to make sense as an equation to someone viewing it unfamiliar with the puzzle. Although if we do allow that you could get $6$ by doing $1*(\sqrt{9})!$ and $9$ by $\sqrt{9^2}$ and put them together to get $69$.
Comment: It is standard practice, at the times when allowing concatenation, that it only mean the placement of immediate individual digits to form two-digit or longer numbers.  Or, if in addition that decimal points are used, decimal number can be created such as .12, 1.2, 12.3, 12.34, etc.
Answer: How about $\ \ \ \ $ $$-1+9!!!-92$$
or
$$(1+\sqrt{9})!+9\div.2$$
Comment: No multifactorials only the usual one sorry.
Comment: No decimal points sorry! Although very ingenious.
Comment: Isn't 9!!! huge, how is this supposed to be 69
Comment: $!!!$ can mean the triple factorial, i.e. multiply every third number so $9!!!=9.6.3$


Algorithm to make a face on a photo look fatter?

Question: What is the most general and simple approach to make look fatter a face on an image ?I am working with java but programming details are not important i am looking for the algorithm .
Comment: Why not simply stretch it horizontally ?
Comment: I am trying to get more professional result.Something like  iphone application called "fat booth".
Answer: You could try to get some Java image morph code, use a fat face picture as a target and morph only the vertexes, not the color. This is a very complex solution for two reasons: First it's hard to code (however you might find some good library) and second, you'll also have to do some face recognition to get the source morph points. But it will probably give the best (most realistic) results.