How to speed up calculations involving previous row and multi-row pandas?

Question: I'm trying to make a calculation on multiple rows for every row in a dataframe.
My current solution takes so much time when I run 2971000 rows. it almost takes more than 2hours.
So, I want know other solutions to speed up a function
my data looks like this for example.
                        sig1    sig2   sig3   sig4  sig_p   sig_t
20210114 05:52:02.00     0.0    0.0    0.0    0.0   11.5    -3.5
20210114 05:52:02.01     0.0    0.0    0.0    0.0   11.6    -3.5
20210114 05:52:02.02     0.0    0.0    0.0    0.0   11.5    -3.5
20210114 05:52:02.03     0.0    0.0    0.0    0.0   11.6    -3.5
20210114 05:52:02.04     0.0    0.0    0.0    0.0   11.7    -3.5
...                      ...    ...    ...    ...   ...     ...
20210114 22:38:59.85     0.0    0.0    0.0    0.0   0.0     -0.5
20210114 22:38:59.86     0.0    0.0    0.0    0.0   0.0     -0.5
20210114 22:38:59.87     0.0    0.0    0.0    0.0   0.0     -0.5
20210114 22:38:59.88     0.0    0.0    0.0    0.0   0.0     -0.5
20210114 22:38:59.89     0.0    0.0    0.0    0.0   0.0     -0.5

I have a function which loops through and calculates value for newcol based on sig1, sig_p, sig_t,previous newcol. the function runs repeat for sig1, sig2, sig3, sig4.
I'll show you the code I currently have, but it's too slow.
parameter.py

from typing import NamedTuple
class Param(NamedTuple):
    RATIO                     : float
    D                         : float
    T                         : float
    M                         : float
    S                         : float
    W                         : float
    DYNAMIC                   : float

    T_CONST                   : float
    P_CONST                   : float

    L_COEF                    : float
    O_COEF                    : float

    @property
    def A(self):
        return (self.D**2)*math.pi
    @property
    def FACTOR(self):
        return self.S / self.A

Param1 = Param(
    RATIO                     = 0.74,

    D                         = 172e-3,
    T                         = 23e-3,
    M                         = 6,

    S                         = 53.7e-4,#4232.5e-6,
    W                         = 0.805,
    DYNAMIC                   = 0.3150,

    T_CONST                   = 2, #4,
    P_CONST                   = 0.2,#3,

    L_COEF                    = 0.8,#4,
    O_COEF                    = 2.5
)

rear = Param(
    RATIO                     = 0.26,

    D                         = 204e-3,
    T                         = 10e-3,
    M                         = 4,

    S                         = 26.8e-4,
    W                         = 0.38,

    DYNAMIC                   = 0.3150,

    T_CONST                   = 1.8,
    P_CONST                   = 0.2,

    L_COEF                    = 0.2,
    O_COEF                    = 1.8
)

test.py

import pandas as pd
import numpy as np
from scipy.interpolate import interp1d

TIME_STAMP = 0.1
SPEC = 449
SPECIFIC = 935

EMISSIVITY                      = 0.7
ABSORBTIVITY                    = 0.3

DYNAMIC_SPEED = 12

COEFFICIENT = 0.9506173967164384

input_KV = [-75, -50, -25, -15, -10, -5, 0, 5, 10, 15, 20, 25, 30, 40, 50, 60,
                        80, 100, 125, 150, 175, 200, 225, 300, 412, 500, 600, 700, 800, 900, 1000, 1100]

viscosity_value = [7.4, 9.22, 11.18, 12.01, 12.43, 12.85, 13.28, 13.72, 14.16, 14.61, 15.06, 15.52, 15.98, 16.92, 17.88, 18.86, 20.88,
               22.97, 25.69, 28.51, 31.44, 34.47, 37.6, 47.54, 63.82, 77.72, 94.62, 112.6, 131.7, 151.7, 172.7, 194.6]

input_ka = [-190, -150, -100, -75, -50, -25, -15, -10, -5, 0, 5, 10, 15, 20, 25, 30, 40,
                     50, 60, 80, 100, 125, 150, 175, 200, 225, 300, 412, 500, 600, 700, 800, 900, 1000, 1100]

conductivity_value = [7.82, 11.69, 16.2, 18.34, 20.41, 22.41, 23.2, 23.59, 23.97, 24.36, 24.74, 25.12, 25.5, 25.87, 26.24, 26.62,
                    27.35, 28.08, 28.8, 30.23, 31.62, 33.33, 35, 36.64, 38.25, 39.83, 44.41, 50.92, 55.79, 61.14, 66.32, 71.35, 76.26, 81.08, 85.83]

def viscosity(input):
    fq = interp1d(input_KV,
                  viscosity_value, kind='quadratic')
    return (fq(input)*10e-6)

def conductivity(input):
    fq = interp1d(input_ka,
                  conductivity_value, kind='quadratic')
    return (fq(input)*10e-3)

def calculation(Param, sig, sig_p, sig_t):
    new_col1 = np.empty(len(sig_p))
    new_col1[0] = sig_t[0]

    my_goal = np.empty(len(sig_p))
    my_goal[0] = sig_t[0]

    calc1 = COEFFICIENT * Param.RATIO * sig_p * sig /2

    for n in range(1, len(sig_p)):
        calc2 = EMISSIVITY * Param.A * (new_col1[n-1]**4 - sig_t[n]**4)

        Ka = conductivity(sig_t[n])
        if sig[n] == 0:
            h = Param.O_COEF
        else :
            KV = viscosity(sig_t[n])

            if sig[n] < DYNAMIC_SPEED:
                h = (0.7*(sig[n]/KV)**0.4) * Ka + Param.O_COEF
            else :
                h = (0.04*(sig[n])/KV**0.8) * Ka + Param.L_COEF

        calc3 = h * Param.A * (new_col1[n-1] - sig_t[n])
        calc4 = Ka *Param.A * (new_col1[n-1] - sig_t[n]) / Param.T

        a1 = (calc1[n] - (calc2 + calc3 + calc4)) / (SPEC * Param.M)
        new_col1[n] = new_col1[n-1] + a1 * TIME_STAMP

        if sig_p[n] == 0 :
            val1 = ABSORBTIVITY * Param.FACTOR * calc2
        elif (sig_p[n] > 0) & (sig_p[n] <= 20):
            val1 = ABSORBTIVITY * Param.FACTOR * calc2* (20-sig_p[n])/20 + ((1-COEFFICIENT) * calc1[n] / (4)) * sig_p[n] / 20
        else:
            val1 = (1-COEFFICIENT) * calc1[n] / 4

        if sig[n] == 0:
            val2 = Param.T_CONST
        else:
            h_bar = Param.P_CONST * (sig[n] *Param.DYNAMIC)**0.8
            val2 = h_bar * Param.S * (my_goal[n-1] - sig_t[n])

        a2 = (val1 - (val2)) / (SPECIFIC * Param.W)
        my_goal[n] = my_goal[n-1] + a2 * TIME_STAMP
        if my_goal[n] < sig_t[0] : my_goal[n] = sig_t[0]

    return my_goal

df = pd.read_csv('data.csv', index_col=0)

df['newcol1'] = calculation(Param1, df['sig1'].values, df[sig_p].values, df['sig_t'].values)
df['newcol2'] = calculation(Param1, df['sig2'].values, df[sig_p].values, df['sig_t'].values)
df['newcol3'] = calculation(Param2, df['sig3'].values, df[sig_p].values, df['sig_t'].values)
df['newcol4'] = calculation(Param2, df['sig4'].values, df[sig_p].values, df['sig_t'].values)

I now need to apply this function to several million rows and it's impossibly slow so I'm trying to figure out the best way to speed it up. I've heard that Cython can increase the speed of functions but I have no experience with it (and I'm new to both pandas and python).
My question is if there is anyway to enhance or speed up this computation method.
I'm run this python code on AWS(sagemaker>notebook instance, jupyter) and my computer OS is window.
Comment: dataframe performs the best with vectorized operation. iteration is usually not recommended. also, what'd fun1, fun2... do? any sample param you can provide as well?
Comment: param is class NamedTuple.
Comment: Sara, can you in simple words explain what you want to get done. You have a very complex function written up. It would be best if we can understand what you want to get done so we can use alternate solutions. Like Simon said, it would be best to use vectorized operations.
Comment: The functions are base on physics formulas, so it's difficult to explain in simple words. What I want to get done is `newcol` values. `newcol` is an estimation value involving previous row and current rows. @JoeFerndz
Comment: I have uploaded all the code including param to get the `newcol` value. @Simon
Comment: @sara, glance over your long code. found lots of typo. The Param doesn't match to the one in fun(). And i can't see anything that can exit the while loop inside fun().
Comment: basically, to improve the performance. Especially for dataframe, you should avoid looping / iteration and make use of dataframe function as long as it's possible.
Comment: @Simon I corrected the typo in my code. I try to modified my code like this.
`def calculation(Param, df):`
`calc2 = EMISSIVITY * Param.A * (df['new_col1'].shift(1)**4 - df['sig_t']**4)`
Comment: First, I was only going to get the value of new_col1 like calc2. But it didn't work well.
Answer: Iteration is easy to code but slow for dataframe. Here is a hint to your solution. You need to vectorize the code inside the while loop while n < len(sig_p):.
For example, previously your code:
def fun(Param, sig_p, sig, sig_t):
    tempvalue =   np.empty(sig_p.shape)
    tempvalue[0] = sig_t[0]
    newcol = np.empty(sig_p.shape)
    newcol[0] = sig_t[0]

    n = 1
    while n < len(sig_p):
        # calc1 = fun1()
        calc1 = Param.COEF * (sig_p[n]) * Param.NO * Param.EFF # fun1()

        # calc2 = fun2()
        if sig[n] > Param.THRESHOLD:
            calc2 = 0
        else:
            calc2 = Param.EMISSIVITY * Param.CONSTANT * (tempvalue[n-1]**4 - sig_t[n]**4)

        # calc3
        # calc4
        # ......

df['newcol1'] = fun(param1, df['sig_p'].values, df['sig1'].values, df['sig_t'].values)

To eliminate the while loop, the fun1() and fun2 can be rewritten like this:
def fun(Param, df, sigTag):
    # df['calc1'] = vectorized fun1()
    df['calc1'] = Param.COEF * df['sig_p'] * Param.NO * Param.EFF

    # df['calc2'] = vectorized fun2()
    df['calc2'] = Param.EMISSIVITY * Param.CONSTANT * (df['sig_t'].shift(1)**4 - df['sig_t']**4)
    df.loc[df[sigTag] > Param.THRESHOLD, 'calc2'] = 0

    # df['calc3'] = vectorized fun3()
    # df['calc4'] = vectorized fun4()
    # ......

df['newcol1'] = fun(param1, df, 'sig1')

you might also want to input the dataframe to the fun() rather than in separated ndarray(s).
This approach'd greatly enhance the performance. You might want to do some research on how to vectorize the calculation.
Comment: my function is `df['calc2'] = Param.EMISSIVITY * Param.CONSTANT * (df['new_col'].shift(1)**4 - df['sig_t']**4)`
Comment: Is it possible to vectorize dataframe when i wnat to get recursive values?
Comment: i suggest you break the calculations into several columns, maybe try it out in an excel first. it will be easier to search for the answer or you might probably want to ask a new question about dataframe vertorization for some of your breakdown issue.
Comment: I searched about my issue. And I found this link. [link](http://https://stackoverflow.com/questions/4407984/is-it-possible-to-vectorize-recursive-calculation-of-a-numpy-array-where-each-el) According to the answer to this link, **It is possible to vectorize operations on vectors as long as the calculation is not recursive. Because a recursive operation depends on the previously calculated value it is not possible to parallel process the operation. This does therefore not work**.So, I can't understand your answer. If it is possible to the vectorization how can I possibly about my issue?
Comment: The OP said the recursive calculation was too slow, so i suggest you to vertorize your formula. This answer won't be helpful if you can't come up with a the formula or intended to use recursive calculation. Again, please try with excel 1st and ask a new question with excel result. it would be easier to breakdown the issue(maybe how to formulate a single column) and get the answer with limited lines of code.


Determining Vocal Melodies in Contemporary Music

Question: I've been learning to sing over a year or so. I'm also an adept guitarist (for context). One of my biggest challenges in learning a song is the vocal melody. I'm unable to find sheet music for some music and my ear is still improving.
What are some techniques you found to help you learn a vocal melody for a song?
Answer: I am a big fan of solfege, you know, "do, re, mi." ("do" is pronounced like "dough", not "doo") There is a lot of (ego-based) disagreement about whether to use the fixed-do or the movable-do system. If you want to develop both the ability to read a melody and, having heard a melody, the ability to write it down, I would practice with movable do. Fixed do is useful for more advanced uses. As far as I can tell, there is no real conflict between the two systems.
I learned solfege using the techniques developed by Zoltan Kodaly, the Hungarian composer, and I recommend them. But I'd advise adults not to worry too much about the hand-signs, they are not the important part, which is the syllables. There are other solfege methods around that are also good. It doesn't really matter which one you use, what you want to do is learn the intervals by ear, so they come to you automatically; and solfege is a great tool for that. Don't just sing scales, that won't get you anywhere. Start singing songs in solfege. For instance, Row, Row, Row Your Boat goes like this:
Do, Do, | Do, Re, Mi | Mi, Re, Mi, Fa | Sol | Do', Do', Do', | Sol, Sol, Sol | Mi, Mi, Mi | Re, Re, Re | Sol, Fa, Mi, Re | Do.
My Bonnie Lies over the ocean starts out with a major 6th, or Sol, Mi:
Sol, | Mi, Re, Do, | Re, Do, La, | Sol, Mi |
(There are supposed to be little marks you can put after each syllable to show what octave you are in, but I can't type them here.)
If you can find a copy of Kodaly's 333 Exersizes, that is a good place to start. They are incredibly boring, but if you do some every day, you will be surprised at how quickly writing and reading music start to become second nature.
Comment: The movable do is good. I sometimes teach in France, where the fixed do (C) is 'de rigueur'.So when in,say, E, there HAS to be 'sharp' applied to C,D,F and G. This necessitates singing two syllables instead of just the one. Flat keys get worse! 'Bemol' adds two syllables !
Comment: Sometimes you have to work out what a singer SHOULD have been singing, unless your aim is to mimic his performance 'warts and all'. :-)
Comment: @Jake Strling: I swear for this method, help us to promote it:

 LET'S START A Do-Re-Mi REVOLUTION!


symfony2 how to register bundles not installed throught composer

Question: I have a bundle which I had to clone from a private repo, and cloned it from the vendor directory, and registered the bundle in the AppKernel.php now, I am getting an error that says, PHP fatal error: Class 'foo\foobar\example not found in C:\... AppKernel.php
So, my question is how do I make sure, pacakges not installed through composer get registered in the autoloader
Comment: why not use composer to clone your private repo ? works for me -> [docs here](http://getcomposer.org/doc/05-repositories.md#using-private-repositories)
Answer: The best way would be not to pull manualy from your private repo, but use composer also in this case.
To accomplish this, you will need a private composer repository - like one with satis.
If that is not an option for any reason, you can configure the autoloader in your project /composer.json like this:
{
    "name": "...",
    "autoload": {
        "psr-0": {
            "MyVendor\\MyBundleBundle": "vendor/foo",
        }
    },
    ....
}

But I strongly recommend the satis way.
Comment: I know that is the best, way. but the package is not yet in pacakgist, or any other online repository.
Comment: satis can be a *private* repository - not online like packagist
Comment: Isn't there any other way to do it, just a simple approach
Comment: I posted a simple approach to modify the autoloader above. Even its not a good practise, it should work. Another way: http://getcomposer.org/doc/05-repositories.md#vcs


What is the topic of this maths question?

Question: I've come across this maths question that I've never seen before and I wanted to know what topic does it cover so that I can research how to solve this question. (Solving this question is optional for any question answerers)
Let $f:\mathbb{R^2} \rightarrow \mathbb{R}$ be defined by
$f((x,y)) := x^2 + 2y^2 - 3xy$, for every $(x,y) \in \mathbb{R^2}$
The value of $f_x((0,0)) + f_y((0,0)) + f_{xy}((0,0))$ is:
Comment: That is correct. It concerns so-called "partial differentiation".
Comment: Can you find e.g. the function $f_x$? Do you know what the notation means?
Comment: @drhab $f_x$ would equal $2x$ at $f_x(0,0)$ right? Also I believe that $f_y$ would equal to $4y$ at $f_y(0,0)$. I'm a bit stuck on $f_{xy}$
Comment: $f_x(x,y)=2x-3y$ (differentiating wrt $x$ where $y$ is considered to be constant) and consequently $f_x(0,0)=2\times0-3\times0=0$. For finding $f_{xy}$ you should differentiate $f_x$ wrt to $y$ and consider $x$ as a constant at that time. That results in $f_{xy}(x,y)=-3$
Comment: @drhab Yes precisely, so then after inputing (0,0) both $f_y(0,0) $ and $f_x(0,0) $ end up equalling to 0 right? what about the $f_{xy}$?
Comment: That is right. See former comment for $f_{xy}$. Also it is good to know that in cases like this we always have $f_{xy}=f_{yx}$ (order of differentiation does not count).
Comment: ok so I just reviewed a youtube video. So I believe that $f_{xy}$ at (0,0) ends up being -3? Is that correct?
Comment: Awesome, thanks for confirming @drhab, now I have a basic understanding of tackling similar questions.
Comment: You are welcome. Good luck with it.
Answer: This seems to be a fairly basic exercise in multivariable calculus. $f_x$, for example, denotes the partial derivative of $f$ with respect to $x$.
Answer: This is simple calculus. The subscripts indicate partial derivatives: $f_x$ means $\frac{\partial f}{\partial x}$, $f_y$ means $\frac{\partial f}{\partial y}$, and $f_{xy}$ means $\frac{\partial^2f}{\partial x\partial y}$.
You are being asked to evaluate each of these at $(0,0)$ and add them up.


Why is R returning NA coefficient-values on dummy variables in VAR-model?

Question: When estimating this VAR-model the returned coefficients on BNP.min4 and INT.min4 are NA, which I do not understand. My intuition is that there should not be any collinearity between these variables. The reason for the dummy's is to make the residuals normal, so by excluding the biggest outliers.
If anyone has any thought on why R is returning NA's on these dummy's and how to solve this that would be really helpful!
Below you will find the R code to recreate the results:
library(quantmod)
library(urca)
library(vars)
library(seasonal)
library(xtable)
library(tseries)

#Constant Price Gross Domestic Product in Sweden, Seasonally Adjusted
#Percent Change from Year Ago, Quarterly, Seasonally Adjusted
getSymbols("SWEGDPRQPSMEI", src = "FRED")
BNP <- ts(as.ts(SWEGDPRQPSMEI), start = c(1961, 1), frequency = 4)

#Unemployment Rate: Aged 15-64: All Persons for Sweden
#Percent, Quarterly, Seasonally Adjusted
getSymbols("LRUN64TTSEQ156S", src = "FRED")
U <- ts(as.ts(LRUN64TTSEQ156S), start = c(2001, 1), frequency = 4)

#Consumer Price Index: Total All Items for Sweden Growth Rate Previous Period,
#Not Seasonally Adjusted
getSymbols("CPALTT01SEQ657N", src = "FRED")
CPI <- ts(as.ts(CPALTT01SEQ657N), start = c(1960, 1), frequency = 4)

#Global price of Brent Crude
#U.S. Dollars per Barrel, Not Seasonally Adjusted
getSymbols("POILWTIUSDQ", src = "FRED")
OIL <- ts(as.ts(POILWTIUSDQ), start = c(1990, 1), frequency = 4)
OIL <- log(OIL)

#3-Month or 90-day Rates and Yields: Treasury Securities for Sweden
#Percent,Not Seasonally Adjusted
getSymbols("IR3TTS01SEQ156N", src = "FRED")
INT <- ts(as.ts(IR3TTS01SEQ156N), start = c(1982, 1), frequency = 4)

##SEASON AJDUSt##

# seasonal adjust time series using X11.
CPI1 <- final(seas(as.ts((CPI),freq=4)))
OIL1 <- final(seas(as.ts((OIL),freq=4)))

sweden.data <- ts.intersect(OIL, U, BNP, CPI, INT)
#OIL & CPI S.ADJ
sweden.data1 <- ts.intersect(OIL1, U, BNP, CPI1, INT)
plot(sweden.data)
plot(sweden.data1)
# save(okun, file = "okun.rda")
# Data nedladdade 2018-12-11.

var.data1 <- window(sweden.data1, start=c(2001, 1), end=c(2019, 1))

var.sa <- ts.intersect(diff(var.data1[, "OIL1"]), diff(var.data1[, "U"]), var.data1[,"BNP"],var.data1[,"CPI1"], var.data1[,"INT"])
colnames(var.sa) <-c("DOIL", "DU","BNP","CPI","INT")
var.sa #OIL, Unemployment diffade

model.sa1<-VAR(var.sa, type = c("const"), p=1 ,ic = c("AIC"))

res.sa1<-residuals(model.sa1)
DOIL.min4 <- res.sa1[, "DOIL"] == min(res.sa1[, "DOIL"])
DU.max4 <- res.sa1[,"DU"] == max(res.sa1[,"DU"])
BNP.min4 <- res.sa1[, "BNP"] == min(res.sa1[, "BNP"])
INT.min4 <- res.sa1[, "INT"] == min(res.sa1[, "INT"])
var.sa.extDD <- cbind(DOIL.min4, DU.max4, BNP.min4,INT.min4)
var.sa.DD <- window(var.sa, start=c(2001, 3))

model.sano1<-VAR(var.sa.DD, type= "const", p=1, exogen = var.sa.extDD)
model.sano1

Estimated coefficients for equation INT:
========================================
Call:
INT = DOIL.l1 + DU.l1 + BNP.l1 + CPI.l1 + INT.l1 + const + DOIL.min4 + DU.max4 + BNP.min4 + INT.min4

    DOIL.l1       DU.l1      BNP.l1      CPI.l1      INT.l1       const   DOIL.min4     DU.max4    BNP.min4
 0.79386673 -0.27122335  0.04046045  0.04209832  0.97891032 -0.12187917 -1.30325783 -0.12248275          NA
   INT.min4
         NA

Comment: Please add a [reproducible example](https://stackoverflow.com/q/5963269/)
for people to work with.
Comment: Could you show us how you know there is no collinearity?  The code shown to construct the *.min4 and *.max4 variables strongly suggests the possibility of perfect collinearity.
Comment: @gung-ReinstateMonica added the rest of the code, should work now
Comment: @whuber I don't know for sure, but my intuition is that there should be no collinearity since the dummy's are created from different data-series. I can on the otherhand be completly wrong about this.
Comment: Depending on the nature of the data--that is, assuming few data are tied at the maximum or minimum values--these variables appear to be almost all zeros.  That leads to high multicollinearity as well as the possibility of perfect collinearity.  You need to check.
Comment: Alright, so I should check if the same observation is tied to the same minimum value and correct for this if true? On an other note, when multicollinearity appears in dummy's usually only one dummy is affected, why is both in this case?
Comment: Thank you, added!
Answer: You have the line combining indicators
var.sa.extDD <- cbind(DOIL.min4, DU.max4, BNP.min4,INT.min4)

apparently to set up the exogenous variables for model.sano1
But three of these DOIL.min4, BNP.min4, INT.min4 are identical: all FALSE except for the 30th value which is TRUE
So VAR seems to have chosen DOIL.min4 to give a coefficient and to have decided the presence of BNP.min4 and INT.min4 are unnecessary or undeirable
Comment: thank you for taking your time to look in to it!


Using WSUS to install updates over a mult-site network

Question: I have searched through SF and could not really find what I was looking for.
My question is this:
I have a network containing of 50 remote locations that all connect over a MPLS network. I am needing to update all of the workstations. I am currently using WSUS SP 3 to deploy updates to the head office.
There is a total of about 1000 workstations. Can my current WSUS server handle updating this many locations or do I need to move to SCCM? All locations connect to us over a T1 circuit.
Answer: I'm handling nearly that may clients (between 900 and 1000) w/ a single WSUS server with no problems (on an underpowered old box, too). I don't think your problems are going to come from the WSUS server side.
Bandwidth to get updates deployed in an expedited fashion is going to be your biggest concern (especially if by "All locations connect to us over a T1 circuit" you mean that your head-office has only a T1 to the MPLS cloud).
If you have some locations with larger concentrations of client computers than others I'd consider deploying a replica WSUS server into those locations to cut down on duplicate update traffic crossing the wire. You're going to have to expend Windows Server licenses to deploy those replicas, but it may be worth the cost re: WAN bandwidth usage and expedited update deployment. It looks like WSUS can be hosted from a machine running the "Windows Web Server 2008 R2" edition of Windows, so you should be able to get by w/ cheap licensing (obviously, try it in a VM and see before you commit to spending money).
Comment: Thank you for your fast response, we have 3 bonded t1's to the mpls cloud, soon to be moved to a ds3. I should have made that more clear.
Answer: There should be no issues with your configuration.  The bandwidth may be a small issue but WSUS updates download from the parent server using BITS and you should not have an issue.  I am fairly certain you can configure the clients to get approvals from the parent server but still use the internet connection to get the updates from Microsoft.  This may be an easier solution to cut down the WAN traffic.  It will only work if you have a connection to the internet for all of your hosts.
SCCM will not give you much of an advantage if you are just trying to patch the Microsoft products.  The only significant advantage you will get from SCCM is that you can set a workstation OS up as the DP however you will still be limited with the 10 concurrent connections issue of the workstation OS.


Variável muda valor quando chama função

Question: Criei uma variável v, que recebe n números aleatórios. quando crio outra variável chamando uma função e passando v por parâmetro, o v muda.
n = int(input("Entre com o tamanho de vetor: "))
v = [0]*n
for i in range(0,n):
    v[i] = randint(0,100)
print(v)

bS=bubbleSort(v)
print(bS)
cS=countingSort(v)
print(cS)
iS=insertionSort(v)
print(iS)
mS=mergeSort(v)
print(mS)
qS=quickSort(v,0,n-1)
print(qS)
sS=selectionSort(v)
print(sS)

print(v)

Comment: Só muda porque uma (ou mais) dessas funções que você chamou está alterando a variável. Não deveria alterar o valor?
Comment: Todas as funções estão alterando o valor? Coloca o código de alguma destas que estão mudando o valor.
Answer: O tipo de dado que está usando é chamado de tipo por referência, por isso você não tem o objeto de forma direta, você tem uma indireção para o objeto. A variável possui uma referência para para o objeto e não tem esses valores que está colocando nele de forma direta (não vou entrar em detalhes específicos como Python lida com isto que não vem ao caso aqui).
Quando chama a função passando essa lista como argumento está passando essa referência e não está passando o objeto em si. Portanto tudo o que for feito dentro da função fará no objeto apontado pela referência, então uma mudança lá se reflete na variável que foi usada como argumento porque foi mudado um objeto único.
Para passar o objeto ele teria que ser copiado por completo e em geral não é o que deseja fazer. A não ser neste caso, pelo descrito na pergunta não quer que o objeto seja mudado, por isso precisa copiá-lo manualmente para obter o resultado desejado.
Uma forma de copiar a lista seria assim:
bS = v[:]
bS = bubbleSort(v)

Coloquei no GitHub para referência futura.
Pode fazer isso quantas vezes quiser, cada uma será uma cópia diferente e cada variável nova terá uma referência para um objeto diferente. A função não mudará o comportamento, ela vai mudar seu objeto, mas agora cada um deles é independente do outro, você não aponta mais para o mesmo objeto sempre.
Você nem precisa criar mais variáveis do que já criou, apenas tem que fazer isto antes, realizando a cópia.
Só tenha claro que a cópia é uma operação custosa se a lista for muito grande, o que nem é o caso mais comum do exemplo (será se colocar um n muito grande, mas se só faz isso não será um problema tão grande), mas é bom saber pra não abusar disto.
A única forma disto não acontecer é se as funções fizerem a cópia internamente, o que explicaria ter um retorno. Se foi você que fez essas funções o problema pode estar aí, não está fazendo a cópia lá dentro. Se fizer a cópia seria bom documentar isso porque ela será uma função bem "cara", fazer uma cópia e uma classificação, e nem sempre é óbvio que está fazendo isso, ser óbvio é importante. Aí não precisaria fazer a cópia fora. Se não fizer esta cópia então não tem porque retornar nada.

Variável muda valor quando chama função


how to set different authorization header for different endpoint in wso2 esb

Question: I have a scatter gather proxy implementation in which I need to call multiple endpoint systems which have different authorization header (username and password is different for different systems). Currently my esb calls only one endpoint and it discards others having different credentials.
Please help.
Answer: If your backends use basic authentication, it should work if you set the "Authorization" header before each call of the backends.
<property xmlns:ns="http://org.apache.synapse/xsd"
       name="Authorization"
       expression="fn:concat('Basic ', base64Encode('username:password'))"
       scope="transport"/>
  <send>  endpoint1

.....
<property xmlns:ns="http://org.apache.synapse/xsd"
           name="Authorization"
           expression="fn:concat('Basic ', base64Encode('username2:password2'))"
           scope="transport"/>
      <send>  endpoint2


Scoped service in DBContext with DBContextFactory can't resolve from root provider

Question: I want to register a scoped ICurrentUserService and then use it in the DbContext to set the CreatedBy property on every entity. I want it scoped because it uses the current HttpContext to get the user id from the claims and the HttpContext is scoped by definition.
services.AddTransient<ICurrentUserService, CurrentUserService>();

However when I try to use this with the DbContextFactory like this:
services.AddDbContextFactory<MyDbContext>(options =>
    options.UseSqlServer(config.ConnectionStrings.MSSQLConnection, x =>
        {
            x.MigrationsHistoryTable("...");
            x.MigrationsAssembly("...");
        }));

services.AddScoped<IMyDbContext>(provider => provider.GetRequiredService<IDbContextFactory<MyDbContext>>().CreateDbContext());
services.AddScoped<MyDbContext>(provider => provider.GetRequiredService<IDbContextFactory<MyDbContext>>().CreateDbContext());

I get the error

Cannot resolve scoped service 'SomeNamespace.ICurrentUserService' from root provider.

I can change it to Transient but that seems wrong and will probably be an issue later on for testing when I want to mock this and every class gets a new instance.
I'm actually not sure why Scoped does not work here. The DbContextFactory is registered Singleton but the Context is resolved as Scoped too.
Answer: Try it with ServiceLifetime.Scoped
services.AddDbContextFactory<ApplicationDbContext>(contextOptions =>
                    ... YOUR OPTIONS
), ServiceLifetime.Scoped);

Comment: maybe related
https://stackoverflow.com/questions/65022729/use-both-adddbcontextfactory-and-adddbcontext-extension-methods-in-the-same
Comment: Thanks, that's it. Do you know why this is necessary? I'm not entirely sure.
Comment: its singleton by default


Bootstrap 4 - col vertical alignment not working as expected

Question: Reading the Bootstrap docs you can get to the alignment examples section.
It tells you can use use flexbox alignment utilities to vertically and horizontally align columns as Bootstrap CSS classes align-items-start, align-items-center and align-items-end but when I'm trying it the content is not vertically aligned as expected with a full height container, see the following code:

html,
body {
  height: 100%;
}

.fill {
  min-height: 100%;
  height: auto !important;
  height: 100%;
  background-color: lightblue;
}
.col {
  border: 1px solid red;
}
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" integrity="sha384-rwoIResjU2yc3z8GV/NPeZWAv56rSmLldC3R/AZzGRnGxQQKnKkoFVhFQhNUwEyJ" crossorigin="anonymous">

      align-items-start

      align-items-center

      align-items-end

I expected to have align-items-center content on the vertical middle and align-items-end at the bottom of the container.
What I'm doing wrong or what I misunderstood?
Answer: In order to vertically align items in flex-container you need to give  height to flex-container.
.row{
  min-height:100px;
}

html,
body {
  height: 100%;
}

.fill {
  min-height: 100%;
  height: auto !important;
  height: 100%;
  background-color: lightblue;
}
.col {
  border: 1px solid red;
}

.row{
  min-height:100px;
}
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" integrity="sha384-rwoIResjU2yc3z8GV/NPeZWAv56rSmLldC3R/AZzGRnGxQQKnKkoFVhFQhNUwEyJ" crossorigin="anonymous">

      align-items-start

      align-items-center

      align-items-end


How do I check if TwigTweak is enabled before declaring a function in a template?

Question: I am cretaing a theme which uses the twigtweak module to display some views/blocks in some of the templates.
If a person enables the theme before enabling twigtweak module they get an error.
For example if I put the following in a template, the site gets a php error 'drupal_entity is an unknown function'
{{ drupal_entity('block_content', 1) }}

So I try
{% if drupal_entity ?? false %}
    {{ drupal_entity('block_content', 1) }}
{% else %}

You should enable Twig Tweak module for this functionality

{% endif %}

But I am still getting similar errors.
How can I check if Twig Tweak is enabled or if the Twig Tweak function is valid/exists?
Thanks
Answer: On Drupal 8.9.x and successive Drupal versions, a theme can define its dependencies from modules. When that theme is installed, Drupal core will require that module to be installed, before allowing the theme installation. This means that when a theme template file is used, the require to use it.
Themes can declare dependencies on modules gives more details, but essentially that is done as it is done with modules. In the .info.yml file for the theme, it is sufficient to add lines like the following ones. (Replace <module> with the module name, name-spaced, for example drupal:user for the User module that is part of Drupal core, or twig_tweak:twig_tweak for the Twig Tweak module.)
dependencies:
  - <module>

If instead you want the theme to optionally depend on that module, you need to implement a preprocess function that is used for that template file. Since the question does not make clear which template file is used, I will make an example that is valid for the user.html.twig template file, but that can be adapted to other template files. (Replace <theme_name> with the theme machine name.)
function <theme_name>_preprocess_user(&$variables) {
  $variables['twig_tweak'] = \Drupal::moduleHandler()->moduleExists('twig_tweak');
}

Then, the template file should check the value of that variable.
{% if twig_tweak %}
  {{ drupal_entity('block_content', 1) }}
{% else %}
  { # Show the content in a different way #}
{% endif %}

Since that module is an optional dependency, users should not see a message telling them to install a module when the module is not installed, which would probably be seen even from users who cannot install new modules on that site. Instead, the template file should shown alternative content, or show the same content in a different way.
If the same variable is required from more template files, instead of a preprocess function that is called for a single template file, I would implement <theme_name>_preprocess(), which is called for every template file.
Comment: Hi, thanks but I am trying to make it optional. So added functionality if the module is enabled but the theme still works if it isn't.
Answer: If your theme depends on a module to function you should add the module as a dependancy in your themes .info.yml file. e.g.:
dependencies:
  - drupal:twig_tweak

Then people won't be able to install your theme without it
Comment: How to define dependencies is explained in [How would we add Drupal Core modules as dependencies on custom modules post 8.6.2?](https://drupal.stackexchange.com/questions/272106/how-would-we-add-drupal-core-modules-as-dependencies-on-custom-modules-post-8-6) which also quotes documentation with an example for a module that is not part of Drupal core.
Comment: That module is not part of Drupal core.
Comment: I know but what's your point? That's how the module is named in Drupal's package list and it's how to add it as a dependency
Comment: With `drupal:twig_tweak`, Drupal will search the module in the Drupal core modules. That is not the namespaced name of that module, which must follow the `:` schema.
Comment: Feel free to try what I suggested. I have and it worked perfectly well with a contrib module.
Comment: It works when the module is already installed. If Drupal needs to give a link to the missing module, it needs the correct project name.


Solving differential equation $x'' + w^2x = \Omega^2x^{-3}$

Question: How to solve the differential equation ? [both $\Omega^2$ are constants and differentiation is w.r.t to $t$]
I found the complementary solution, but how to find the particular solution? I haven't solved D.E with negative power of $x$.
$$\ddot{x}+w^2x=\Omega^2x^{-3}$$
Comment: There is no "complementary solution": this is not a linear differential equation.
Answer: If you switch variables, you face
$$\frac {t''}{[t']^3}-w^2 x+\frac{\Omega ^2}{x^3}=0$$ First, reduction of order $p=t'$ leads to
$$p=t'=\pm\frac{x}{\sqrt{-w^2 x^4+c_1 x^2-\Omega ^2}}$$
$$t+c_2=\pm\int   \frac{x}{\sqrt{-w^2 x^4+c_1 x^2-\Omega ^2}}\,dx$$ Let $x=\sqrt y$ to face a simple integral (complete the square).
May be, the inversion could not be very easy using this simple way. But this part is easy to do by hand.
Otherwise, $x$ is given as a Jacobi elliptic function (which is not the most pleasant result)
$$x=\pm i \sqrt{\frac{-w^2+\sqrt{w^4-2 c_1 \Omega ^2}}{\Omega ^2}} \,\,\times $$ $$   \text{sn}\left(\frac{\sqrt{(t+c_2){}^2 \left(w^2+\sqrt{w^4-2 c_1
   \Omega ^2}\right)}}{\sqrt{2}}|\frac{w^4-w^2 \sqrt{w^4-2 c_1
   \Omega ^2}-c_1 \Omega ^2}{c_1 \Omega ^2}\right)$$
Answer: The command of Maple 2023
<code>dsolve((D@@2)(x)(t) + w^2*x(t) = Omega^2/x(t)^3)
</code>
results in $$x\! \left(t\right) {=} -\frac{\sqrt{-c_{1} \left(16 c_{1}^{2} c_{2}^{2} w^{2}-8 {\mathrm e}^{-2 \,\mathrm{I} w t} c_{1}^{2} c_{2} w-64 \Omega^{2} w^{2}+\left({\mathrm e}^{-2 \,\mathrm{I} w t}\right)^{2} c_{1}^{2}\right)}}{4 c_{1} {\mathrm e}^{\mathrm{-I} w t} w},
\\
x\! \left(t\right){=}\frac{\sqrt{-c_{1} \left(16 c_{1}^{2} c_{2}^{2} w^{2}-8 {\mathrm e}^{-2 \,\mathrm{I} w t} c_{1}^{2} c_{2} w-64 \Omega^{2} w^{2}+\left({\mathrm e}^{-2 \,\mathrm{I} w t}\right)^{2} c_{1}^{2}\right)}}{4 c_{1} {\mathrm e}^{\mathrm{-I} w t} w} $$
Comment: The command `odetest(x(t) = -sqrt(-c__1*(16*c__1^2*c__2^2*w^2 - 8*exp(-2*I*w*t)*c__1^2*c__2*w - 64*Omega^2*w^2 + exp(-2*I*w*t)^2*c__1^2))/(4*c__1*exp(-I*w*t)*w), (D@@2)(x)(t) + w^2*x(t) = Omega^2/x(t)^3)` results in $0$, confirming the first solution. The same with the second one.
Comment: So there is no way to solve this by hand ?
Comment: I tried solving by variation of parameters method, I didn't get the solution
Comment: @kumar: Did you look in the handbook by E. Kamke?
Comment: Could you tell me name of the book ?
Comment: @kumar:  See [that](https://www.goodreads.com/book/show/18237747-differentialgleichungen-losungsmethoden-und-losungen).
Comment: @kumar: Variation of parameters only works for *linear* ODEs. But your ODE has the form $x''=V(x)$ (which is a standard Newton-type equation from physics), so you can multiply by $x'$ and integrate, and then solve for $x'$ to get a separable equation $dx/dt = \pm \sqrt{f(x)}$.
Answer: After multiplying by $\dot x$ we have
$$
\frac 12(\dot x^2)'+\frac 12(w^2x^2)'+\frac 12\Omega^2(x^{-2})' = 0
$$
then
$$
\dot x = \sqrt{c - w^2x^2-\Omega^2x^{-2}}\Rightarrow dt = \frac{dx}{\sqrt{c - w^2x^2-\Omega^2x^{-2}}}
$$
etc.


$L^2(\Omega) \subset D'(\Omega)$ , can we logically give a sense to this?

Question: How can "$L_2(\Omega) \subset D'(\Omega)$" not be an aberration in term of logical symbolism ? $*_1$
I get that , the idea is to identify a L_2 function to it's regular associated distribution and to continusously inject the latter into D' , but i'm also very "shocked" that the logical symbol of "$\subset$" has been defined (in most books i've read) by :
$B \subset A \Leftrightarrow(\forall x)(x \in B \rightarrow x\in A)$
I'd see something like , maybe it exists a terminology for the symbol "$\subset$" such as , for two sets of differents objects , it is defined by "it exists a continuous injection from the first set to the second";
$*_1$: D'($\Omega$) is a set of function of $D(\Omega)\rightarrow R$ ,$D(\Omega)$ being itself a set of function ,$L_2$($\Omega$) is a set of function of $\Omega\rightarrow R$ , so it defines two different mathematical objects in my opinion.
Edit : $L_2$($\Omega$) is a set of Class of functions,
Comment: To use an identification means exactly that there is an embedding that they do not bother to write. Indeed, some people give specific notation say $T_f$ for the distribution induced by $f$. If it makes you feel better, you were already 'lied' to when you decided that $L^2$ Is a quotient space of functions, but we should still call them 'functions'
Comment: A more correct symbol would probably be something like $\hookrightarrow$ to show that it's not a pure set inclusion but an injection.
Comment: Absolutely ,thanks for your answer it's more clear now !
Answer: Do you think that $\mathbb{Q} \subseteq \mathbb{R}$? Technically, these are completely different beasts: $\mathbb{Q}$ consists of pairs of integers (the second component non-zero) subject to a certain equivalence relation. $\mathbb{R}$ consists (depending on your taste) Cauchy sequences subject to a certain equivalence relation or Dedekind cuts (additional possibilities exist). These are certainly different types of mathematical objects but I think most people would agree that having $\mathbb{Q} \subseteq \mathbb{R}$ is useful. For example, we can now take the square root of a (positive) rational number even though it might not be rational again.
So here's what we do: We find a copy of $\mathbb{Q}$ within $\mathbb{R}$ (i.e. an injection $\mathbb{Q} \hookrightarrow \mathbb{R}$) and then essentially redefine $\mathbb{Q}$ to mean the image of this injection. This is what we mean when we say that we "identify" the two sets. (We keep the original definition around because it often makes thinking about these objects easier.) One thing that is important here is that the inclusion is canonical, so there is no confusion which real number I mean when I talk about the rational number $\frac{23}{42}$.
The inclusion $L_2(\Omega) \subseteq D'(\Omega)$ works the same way. Here the canonical inclusion is given by the regular associated distribution you mentioned. But sometimes it is more convenient to just think of $L_2$-functions as distributions so you can apply stuff to it that generally only works for distributions (like the square root above).
This is a powerful tool but one also needs to be a bit careful with this trick. You shouldn't simply apply it to any injection you see. For example, if you have three sets $A$, $B$, $C$ and injections $i : A \hookrightarrow B$, $j : B \hookrightarrow C$ and $k : A \hookrightarrow C$, the composition $j \circ i$ need not agree with $k$. If you use all three to make identifications $A \subseteq B \subseteq C$ and $A \subseteq C$ you can now consider elements $a \in A$ as elements of $C$ in two different ways; if these don't agree, things get confusing.
Comment: Thanks a lot , exactly what kind of answer i needed.
Comment: A concrete example of the last cautionary tale is of course $H^{1} \subset L^2 \subset H^{-1}$, with $H^{-1}\setminus H^1 \neq \{\}$, and yet $H^{-1} = H^1$ as Hilbert spaces


Clique factorization

Question: I'm reading about Clique factorization in wikipedia:
http://en.wikipedia.org/wiki/Gibbs_random_field#Clique_factorization
but I'm unable to understand this:

What is $X_C$ here? Ok I understood that for a value $x$ we multiply the values of functions, where each function correspond to a clique. But what's the input to the function!
Comment: An identical question has also been posted on [stats.SE](http://stats.stackexchange.com/q/94233/6633)
Comment: @DilipSarwate I removed it
Answer: There is no $X_C$ here and, by definition, $$x_C=(x_v)_{v\in C}.
$$
(This notation is used everyuwhere on the page linked to, and well before the paragraph reproduced in the question.)
Comment: Actually, $x_C$ is a set of values indexed by the clique $C$. A single random variable would be $X_v$ with $v$ in $V$, and the set of random variables corresponding to a clique $C$ might be denoted $X_C=(X_v)_{v\in C}$ (although there is no $X_C$ in your question, only $x_C$).
Comment: so $x_c$ is a single random variable of a node in the clique or a vector of the random variables of the nodes of the clique?

Clique factorization


Deploy pre-trained Inception in TensorflowServing fails: SavedModel has no variables

Question: Description
I'm trying to deploy one of the pre-trained TensorFlow object detection models (faster_rcnn_inception_v2_coco_2018_01_28) in TF Serving. I follow these steps:

Clone https://github.com/tensorflow/models
Download pre-trained Inception model checkpoint
Export Inception as described here with this command. Note I'm changing input_type to encoded_image_string_tensor so that I won't need to attach an input tensor to de-serialize the input string within the API.
Convert Inception from the previous step to a servable with the code below
Run TF Serving, pointing model_base_path to the folder created by the previous step.

Issue
When querying the API predict endpoint it fails, it seems the model has not been initialized properly:
{ "error": "Attempting to use uninitialized value SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_3/Conv2d_0b_1x1/BatchNorm/moving_mean\\n\\t [[Node: SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_3/Conv2d_0b_1x1/BatchNorm/moving_mean/read = Identity[T=DT_FLOAT, _output_shapes=[[128]], _device=\\"/job:localhost/replica:0/task:0/device:CPU:0\\"](SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_3/Conv2d_0b_1x1/BatchNorm/moving_mean)]]" }

Indeed the TF Serving log warned The specified SavedModel has no variables; no checkpoints were restored.  (see Appendix for Step 5).
What am I missing in the model export phase? From older issues (this and this), I see this might be due to step (3) freezing variables as constants into the saved model. Is this the case?

Appendix for Step 3 - Re-export Inception
python object_detection/export_inference_graph.py \
  --input_type encoded_image_string_tensor \
  --pipeline_config_path ${MODEL_DIR}/pipeline.config   \
  --trained_checkpoint_prefix ${MODEL_DIR}/model.ckpt   \
  --inference_graph_path ${MODEL_DIR} \
  --export_as_saved_model=True \
  --write_inference_graph=True  \
  --output_directory ${OUTPUT_DIR}

This command produces this model
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: encoded_image_string_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['detection_boxes'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 100, 4)
        name: detection_boxes:0
    outputs['detection_classes'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 100)
        name: detection_classes:0
    outputs['detection_scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 100)
        name: detection_scores:0
    outputs['num_detections'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1)
        name: num_detections:0
  Method name is: tensorflow/serving/predict

Appendix for Step 4 - Saved model to servable
import os
import shutil
import tensorflow as tf

tf.app.flags.DEFINE_string('checkpoint_dir', '/tmp/inception_train',
                           """Directory where to read training checkpoints.""")
tf.app.flags.DEFINE_string('output_dir', '/tmp/faster_rcnn_inception_v2_coco_2018_01_28-export/',
                           """Directory where to export inference model.""")
tf.app.flags.DEFINE_integer('model_version', 1,
                            """Version number of the model.""")
tf.app.flags.DEFINE_string('summaries_dir', '/tmp/tensorboard_data',
                            """Directory where to store tensorboard data.""")

FLAGS = tf.app.flags.FLAGS

def main(_):
     with tf.Graph().as_default() as graph:
        saver = tf.train.import_meta_graph(meta_graph_or_file=os.path.join(FLAGS.checkpoint_dir, 'model.ckpt.meta'))

        with tf.Session(graph=graph) as sess:
            saver.restore(sess, tf.train.latest_checkpoint(FLAGS.checkpoint_dir))
            # (re-)create export directory
            export_path = os.path.join(
                tf.compat.as_bytes(FLAGS.output_dir),
                tf.compat.as_bytes(str(FLAGS.model_version)))
            if os.path.exists(export_path):
                shutil.rmtree(export_path)

            tf.global_variables_initializer().run()
            tf.local_variables_initializer().run()
            print("tf.global_variables()")
            print(sess.run(tf.global_variables()))
            print("tf.local_variables()")
            print(sess.run(tf.local_variables()))

            # create model builder
            builder = tf.saved_model.builder.SavedModelBuilder(export_path)

            # Build the signature_def_map.
            predict_inputs_tensor_info = tf.saved_model.utils.build_tensor_info(graph.get_tensor_by_name('encoded_image_string_tensor:0'))
            boxes_output_tensor_info = tf.saved_model.utils.build_tensor_info(graph.get_tensor_by_name('detection_boxes:0'))

            prediction_signature = (
                tf.saved_model.signature_def_utils.build_signature_def(
                    inputs={
                        'images': predict_inputs_tensor_info
                    },
                    outputs={
                        'classes': boxes_output_tensor_info
                    },
                    method_name=tf.saved_model.signature_constants.
                        PREDICT_METHOD_NAME
                )
            )

            builder.add_meta_graph_and_variables(sess,
                [tf.saved_model.tag_constants.SERVING],
                signature_def_map={
                    tf.saved_model.signature_constants.
                    DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                        prediction_signature,
                },
                legacy_init_op=None)

            builder.save(as_text=False)

            merged = tf.summary.merge_all()
            train_writer = tf.summary.FileWriter(FLAGS.summaries_dir + '/' + str(FLAGS.model_version),
                                      sess.graph)

        print("Successfully exported Faster RCNN Inception model version '{}' into '{}'".format(
        FLAGS.model_version, FLAGS.output_dir))

if __name__ == '__main__':
    tf.app.run()

This produces the following servable:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['images'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: encoded_image_string_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['classes'] tensor_info:
        dtype: DT_FLOAT
        shape: unknown_rank
        name: detection_boxes:0
  Method name is: tensorflow/serving/predict

Appendix for Step 5 - docker-compose log for deploying the servable
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.191941: I tensorflow_serving/model_servers/main.cc:153] Building single TensorFlow model file config:  model_name: faster_rcnn_inception model_base_path: /tmp/faster_rcnn_inception_v2_coco_2018_01_28_string_input_version-export/
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.192341: I tensorflow_serving/model_servers/server_core.cc:459] Adding/updating models.
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.192465: I tensorflow_serving/model_servers/server_core.cc:514]  (Re-)adding model: faster_rcnn_inception
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.195056: I tensorflow_serving/core/basic_manager.cc:716] Successfully reserved resources to load servable {name: faster_rcnn_inception version: 1}
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.195241: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: faster_rcnn_inception version: 1}
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.195404: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: faster_rcnn_inception version: 1}
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.195652: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:360] Attempting to load native SavedModelBundle in bundle-shim from: /tmp/faster_rcnn_inception_v2_coco_2018_01_28_string_input_version-export/1
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.195829: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:242] Loading SavedModel with tags: { serve }; from: /tmp/faster_rcnn_inception_v2_coco_2018_01_28_string_input_version-export/1
tf-serving-base exited with code 0
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.313633: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.492904: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:161] Restoring SavedModel bundle.
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.493224: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:171] The specified SavedModel has no variables; no checkpoints were restored.
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.493329: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:196] Running LegacyInitOp on SavedModel bundle.
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.512745: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:291] SavedModel load for tags { serve }; Status: success. Took 316614 microseconds.
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.513085: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /tmp/faster_rcnn_inception_v2_coco_2018_01_28_string_input_version-export/1/assets.extra/tf_serving_warmup_requests
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.513927: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: faster_rcnn_inception version: 1}
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.516875: I tensorflow_serving/model_servers/main.cc:323] Running ModelServer at 0.0.0.0:8500 ...
tf-serving-faster_rcnn_inception | [warn] getaddrinfo: address family for nodename not supported
tf-serving-faster_rcnn_inception | 2018-08-16 16:05:28.518018: I tensorflow_serving/model_servers/main.cc:333] Exporting HTTP/REST API at:localhost:8501 ...
tf-serving-faster_rcnn_inception | [evhttp_server.cc : 235] RAW: Entering the event loop ...

Notes:

python 2.7
tensorflow 1.10.0 in both steps (3), (4) and (5). Installed via pip.
Answer: Copying this from my answer here
I was also struggling with this, and was able to export a model (include the variables file) by updating @lionel92's suggestion for the current OD API code version (as of July 2, 2019). It mostly involves changing the write_saved_model function in models/research/object_detection/exporter.py
Update write_saved_model in exporter.py
def write_saved_model(saved_model_path,
                      trained_checkpoint_prefix,
                      inputs,
                      outputs):

  saver = tf.train.Saver()
  with tf.Session() as sess:
    saver.restore(sess, trained_checkpoint_prefix)

    builder = tf.saved_model.builder.SavedModelBuilder(saved_model_path)

    tensor_info_inputs = {
        'inputs': tf.saved_model.utils.build_tensor_info(inputs)}
    tensor_info_outputs = {}
    for k, v in outputs.items():
      tensor_info_outputs[k] = tf.saved_model.utils.build_tensor_info(v)

    detection_signature = (
        tf.saved_model.signature_def_utils.build_signature_def(
            inputs=tensor_info_inputs,
            outputs=tensor_info_outputs,
            method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
        ))

    builder.add_meta_graph_and_variables(
        sess,
        [tf.saved_model.tag_constants.SERVING],
        signature_def_map={
            tf.saved_model.signature_constants
            .DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                detection_signature,
        },
    )
    builder.save()

Update _export_inference_graph in exporter.py
Then, within the _export_inference_graph function, update the final line to pass the checkpoint prefix like:
  write_saved_model(saved_model_path, trained_checkpoint_prefix,
                    placeholder_tensor, outputs)

Call export script
Call models/research/object_detection/export_inference_graph.py normally. For me, that looked similar to this:
INPUT_TYPE=encoded_image_string_tensor
PIPELINE_CONFIG_PATH=/path/to/model.config
TRAINED_CKPT_PREFIX=/path/to/model.ckpt-50000
EXPORT_DIR=/path/to/export/dir/001/

python $BUILDS_DIR/models/research/object_detection/export_inference_graph.py \
    --input_type=${INPUT_TYPE} \
    --pipeline_config_path=${PIPELINE_CONFIG_PATH} \
    --trained_checkpoint_prefix=${TRAINED_CKPT_PREFIX} \
    --output_directory=${EXPORT_DIR}

If it works, you should see a directory structure like this. This is ready to be dropped into a TF Serving Docker image for scaled inference.


Edit user profile form

Question: I have found many articles about showing sections of the user profile edit page based on $user->roles but none that deal with other custom fields.
For example,
global $user;
if (in_array('administrator', $user->roles)) {

}
else {
  drupal_add_css(drupal_get_path('theme', 'my_theme') . '/css/normal-user.css');
  //Disable admin-only fields
  $fields = array('field_employee_types');
  foreach ($fields as $field) {
    $form[$field]['#disabled'] = TRUE;
  }
}

BUT what I am needing to do is check a custom user profile field (which happens to be a taxonomy term array) and if the user has a certain term,  then do something such as show fields.  Can someone lead me in the right direction?  I am doing all of this in a custom module to alter profile form.
thanks!

So I somewhat got the info to print on the user profile page
[field_affiliation] => Array
    (
        [und] => Array
            (
                [0] => Array
                    (
                        [tid] => 28
                        [taxonomy_term] => stdClass Object
                            (
                                ...
                            )
                    )
            )
    )

Answer: (upon writing back and forth, the solution to this problem is listed underneath Edit 2)
not sure where exactly you are stuck. are you asking how to check which taxonomy term they have assigned in a field, or how to show/hide fields?
if you already have access to the taxonomy term id and you need to further check the content of the taxonomy term, you can assign that id to a variable like$tid, and then do something like this:
$myfield_term = entity_load('taxonomy_term',array($tid))[$tid];

I find myself doing this a bunch of times, because of the way I use taxonomy terms.
this gives you an object with all the content of the taxonomy term.
if you wanna fetch information out of it, keep in mind that php uses a different approach for objects than for arrays.
so if you need the title value, instead of doing $myfield_term['title'], you need to do $myfield_term->title

EDIT 1:
can you please show us what the structure of the array looks like?
this can be done in various ways. drupal has its own technique of printing arrays (I believe it's caled devel or something), but I still prefer the good old native php way via print_r.
how I'd do it:

assign the array to a value, e.g. $user_apartments
then in the next line add print_r($user_apartments);
switch back to your browser, right click > view page source (may vary depending on your browser, this would be Chrome)

somewhere at the beginning or further down the code (depending on the file that executed the print_r command) should be a printed list of an array, that may look something like this:
Array
(
[0] => 4
[2] => 11
[3] => 28
[4] => ...
)

but it may as well be more complicated than this, if for example it not only lists the term IDs but all the content of the terms as well.

now just grab it and past a part of it here please (if it is a huge list, no need to post the repetitive part)

the way the print_r command works is it takes the object that you feed into it and displays the content in a more human-readable manner, giving you an insight of its structure, whether something is an object, an array, what else it contains etc.
so depending on what this array looks like, we will need to use a different approach on checking the array for your specific ID, this is why I am asking for an insight of the array we want to check.

EDIT 2:
so upon looking at the structure of the array, this is how you check for your term ID:
the array that we need to look at is ...['field_affiliation']['und']. you can (optionally) assign it to a variable, e.g. $field_affiliation, and then do something like this:
$hasDepartment28 = false;
foreach( $field_affiliation as $thisTerm ) {
    if( $thisTerm['tid'] == 28 ) {
        $hasDepartment28 = true;
        break;
    }
}

then, where- and whenever needed, you can simply check by doing
if( $hasDepartment28 ) {
    // do..
}

and here is how it works:

$hasDepartment28: we declare a boolean, and we say by default the department with the term id 28 is not listed
using foreach, we loop through every entry that is listed in the taxonomy term array we named $field_affiliation and every time we assign current entry to the variable name $term
now, as we could see in the structure of the array, what we want to look for is the 'tid' key of every entry
we compare it with the ID you mentioned in the comment below, and if we have a match, we set $hasDepartment28 to TRUE
and via break; we stop the current foreach. since we found what we were looking for, there is no need to loop through any other entry in the array that may come after it
if we found no entry at all where the tid equals 28, the boolean we set earlier will still equal FALSE

let me know if it worked!
keep in mind all the variable names are arbitrary and you should use whatever describes the variable best.
one last thing I'd add is if you were delete and re-create that term entry, you'd have to update this code and check for a different Term ID. perhaps a more readable and future-proof approach would be to check inside the foreach loop for something like:
if( $thisTerm['taxonomy_term']->name == 'Apartment A' ) { ... }

but this in return only works if the [taxonomy_term] gets loaded into your array for every entry. chances are only the first entry has all the term content, but based on the array you submitted I cannot tell you if this is the case. if you wanna stick to this approach though, this is exactly where drupal's entity_load function becomes super useful!
Comment: Well,  the custom profile field can have multiple selections (values),  so I need to check if a certain term is in the custom field,  and if so, disable certain fields.   So,  if value is in the custom field, then do something.. else.. do something else.
Comment: @Stephen_lex
ah, so it's a multi-values taxonomy term reference.
do you want to check for a specific term id?
if so, in PHP there is a function called `in_array`. so for example if the term id you are checking for is `34`, you can do
`if (in_array(34, $myfield_terms)) { #do.. }`, whereas here `$myfields_terms` must be an array that lists all term IDs as values (not as keys!).
can you show us how your taxonomy term array is structured?
Comment: Yeah,  you are right,  that is what I am trying to do.  The term id in my case is 28.  The structure is simply a list of Departments,  and users can be associated with 0 to many departments. I am a .net programmer by trade,  so I am in the very early stages of learning php and drupal for a small project,  so I am kind of at the beginning of the learning phase - I appreciate your patience.  I was experimenting with the in_array earlier, but couldn't ever get it to find the value so I am obviously declaring something wrong.  I had no problem doing the $user->role function
Comment: in that case, welcome to drupal (: I too am currently working on my first big PHP and drupal project! –
so yeah as I mentioned, in order for the `in_array` function to achieve the desired goal, the array that we check has to be of a certain structure. depending on how and where you grab the list of term ids, the array structure of the array can vary.
I've updated my post above with some instructions on how to get an insight of the array, so we can decide what method of checking for a value is appropriate.
Comment: I am doing this inside a module to modify the edit user form that I don't think will do a print_r.  It is simply altering the form.  Is there a way to do this inside my custom user profile display page template?
Comment: It wouldn't let me paste into the comment that much info,  but I was somewhat able to get the info to print on the user profile page, I added it above.
Comment: thanks, that's just what we needed! I moved the array to your original post though and am going to update mine with instructions on how to check for your term ID (:
Comment: updated the post with a solution and also suggest a slightly different approach than checking for the term ID
Comment: I kept getting an error about the field_affiliation not being found,  so after digging I found that I needed to load the entire user array using something like: $user_full = user_load($user->uid); .  I did this,  but now using the code you have I get : Notice: Undefined index: tid in artsci_form_edits_form_user_profile_form_alter() (line 56).. Is there a special way for the variable to load the entire array?
Comment: did the structure of the array change or does the array now contain additional entries other than `[0]`, `[1]` etc? again you can check this by doing a `print_r` and then looking at the source code. something that would help: save the result of the `print_r` in a .TXT text file and upload that file here. (if it contains some private data, just erase or replace it.)
Comment: *facepalms self.* – just noticed there is no upload function here, sorry about that. still, if you could send me the entire array somehow, I can tell you with more certainty how to avoid that error.
Comment: The array I sent was as it was rendered on the user profile display template, and I am doing this inside of a module to alter the form... so the variable $field_affiliation should automatically be an array or do I have to do some sort of case or something?
Comment: again, without having a look at the entire array I can only assume what could be the issue here, such as if there are additional entries in the array.
try something like this: instead of just `if( $thisTerm['tid'] == 28 ) {` try this: `if( is_array($thisTerm) && $thisTerm['tid'] == 28 ) {`
Comment: Thank YOU!!  I got it working - of course it was a mistake in the way I was declaring the variable.  Once I made the last batch of changes,  I had left off the ['und]  so it should have been $user_full = user_load($user->uid);
$user_departments = $user_full->field_affiliation['und'];
Comment: oh dear, it's often these little things like this little extra key. been through that so many times when I started with my current project. – glad you finally got it to work! (:

Edit user profile form


Check whether it's a submanifold of $\Bbb R^2$

Question: I was asked to prove the following:

Consider $A:=\left\{\frac1n: n=1,2,3,...\right\}$ and $B:=A\cup\{0\}$.
Then, $B\times \Bbb R$ is not a smooth sub-manifold of $\Bbb R^2$, but
$A\times \Bbb R$ is a smooth submanifold of $\Bbb R^2$.

I have the following reasonings:
$(1)$ There doesn't exist an open subset $U$ of $\Bbb R^2$ containing $(0,0)$ such that $U\cap B$ homeomorphic to an open subset of $\Bbb R$: The set $U\cap B$ consists of a sequence of vertical open intervals converging to another vertical interval. Also, any open subset of $\Bbb R$ is a collection of pairwise disjoint open intervals. So, $B$ is not a sub-manifold of $\Bbb R^2$.
$(2)$ Note that each vertical slice of $A\times \Bbb R$ is isolated from any other vertical slices, maintaining a fixed distance(though it may be small) at least from other slices.

Are these reasonings correct?

Thanks in advance.
Answer: Your argument is convincing, though not rigorous enough.
For (1) maybe the cleanest way is to observe that $(0,0)$ doesn't have a connected neighborhood in $B\times\Bbb R$, whereas in $\Bbb R^n$ every point has.
For (2) every point has a vertical open segment as neighborhood.


Alterar favicon em um projeto Wamp

Question: Usei o seguinte código dentro do  para alterar o favicon do meu projeto
<link rel="shortcut icon" href="img/favicon.ico"/>

Deu certo no index, mas ao inserir o mesmo codigo nas outras páginas o favicon continua sendo o padrão do Wamp.
Ps. o diretório das paginas são os mesmos, então o link está direcionando para a mesma imagem, dentro da mesma pasta (img).
Alguém consegue enxergar uma luz para o meu problema?
Meu codigo está assim:

Não sei por qual motivo o código inserido que altera o favicon só esta sendo reconehcido na página index. A imagem a seguir foi postada apenas para mostrar que o navegador não reconhece o codigo em questão:
Nas demais paginas ele não reconhece a linha de codigo:
Comment: Cara o Favicon costuma ficar no Cash, tenta limpar o cash para ver se resolve
Comment: ja limpei. Apaguei o histórico do navegador selecionando todas as opções
Comment: Já que vc está declarando o arquivo como .ico tente declara o Type dele na tag. Assim:  as vezes resolve...
Comment: Amigo *sem o link do favicon fica difícil*.... Pq vc não posta a imagem da `` já com o link do favicon certinho?
Comment: Foi postado para a conferencia.
Comment: tem algo errado na segunda imagem nao com diz com a primeira, a header tem somente 2 css ja no seu html tem 3 css, 2 script, icon...
Comment: A segunda imagem é um print do CTRL + U no navegador. Por isso é normal o javascript não parecer. Mas deveria aparecer o icon. Se não está aparecendo é porque o navegador não está reconhecendo o codigo. E é este o meu problema
Answer: Isso ocorre porque o próprio navegador tenta sozinho procurar a imagem favicon.ico na pasta root do seu site ou localhost. Isto ocorre geralmente ocorre no Safari e no Chrome (nos demais navegadores não notei este comportamento), então quando não define o:
<link rel="shortcut icon" href="...."/>

O navegador sozinho vai tentar procurar o http://localhost/favicon.ico ou http://site.com/favicon.ico
O que pode fazer é colocar na pasta root do Wamp o seu favicon e limpar o cache do navegador, ou simplesmente pode criar um include via PHP e adicionar em todas páginas:
header.php:
<link rel="shortcut icon" href="...caminho do favicon..."/>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

E em todas paginas adicionar:

  <title>Meu titulo</title>
  <?php include 'header.php'; ?>

Caminhos relativos vs absolutos
O caminho relativo é aquele que carrega em relação ao caminho da página atual, então se estiver na página
 http://localhost/pagina.php

Ele vai procurar o favicon em:
 http://localhost/img/favicon.ico

se estiver na página
 http://localhost/foo/bar/baz/pagina.php

Ele vai procurar o favicon em:
 http://localhost/foo/bar/baz/img/favicon.ico

Ou seja é o caminho errado, então para evitar isto adicione o caminho completo (absoluto) do favicon, por exemplo:
header.php:
<link rel="shortcut icon" href="http://localhost/img/favicon.ico"/>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

No entanto se você for enviar para um servidor, isto não irá funcionar então terá que trocar o endereço, no caso você pode usar somente o caminho absoluto, sem o HTTP, colocando uma barra na frente assim /img/favicon.ico, desta forma ele vai procurar a partir do root, exemplo:
<link rel="shortcut icon" href="/img/favicon.ico"/>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

Comment: Na verdade a unica coisa que alterou da sua sugestão no meu código foi retirar a linha em cada pagina e inserir dentro do header.php. Mesmo assim eu fiz as alterações, mas não resolveu. Continua a mesma coisa. E sim, eu limpei o cache do navegador
Comment: @Amauri abra em uma aba anonima e veja se ainda ocorre. Provavelmente o caminho do favicon no teu HTML esta relativo e não absoluto
Comment: modo anonimo continua a mesma coisa
Comment: @Amauri veja a edição da resposta
Comment: Eu coloquei o caminho absoluto, e continua a mesma coisa. O problema não é na referencia a imagem, pois todas as paginas se encontram no mesmo diretório.
Comment: @Amauri abra o console do navegador e veja se ocorre algum erro, confira se todas paginas realmente foram salvas, confira se o PHP esta usando XCache ou Opcache (que não tem nada haver com cache do navegador) ... Confira o codigo fonte gerado no navegador das paginas com problema apertando Ctrl+U. Com certeza vc fez alguma confusão.
Comment: Conforme postei acima, o navegador não está reconhecendo a linha de código que altera o favicon. O estranho é que no index ele reconhece.
Comment: @Amauri olha a sua imagem https://i.stack.imgur.com/Uq7i9.png, dentro do head não tem nenhuma tag LINK. Como eu disse, você deve estar fazendo alguma confusão na hora de salvar.
Comment: Não amigo, essa imagem é do f12 no navegador. No código foi inserido, e foi salvo. Não sei por qual motivo o navegador é que não está reconhecendo. Vou postar a imagem do codigo completo, e salvo. –
Comment: @Amauri eu sei que é do navegador, eu trabalho com isso, eu sei a diferença de um console de navegador pra um editor de texto amigo. O problema é que se no console do navegador não aparece a TAG LINK é porque não esta lá mesmo, por isso que disse que vc se confundiu em algo, se confundir é quando vc pensa que fez algo certo, mas se enganou, você pode estar convicto de que fez tudo ok, mas o resultado não aparece é porque tem algo errado.
Comment: @Amauri Esse 192.168.1.220 é o seu computador ou é outro computador na sua rede?
Comment: Eu entendi. Só discordei quando você disse que ''não está la''. Veja a foto que postei do codigo. Está la, e está salvo. O problema é que o navegador não está ''puxando'' essa informação do código fonte.
Comment: O teste está sendo feito tanto no 192.168.1.220, quanto por outro computador que está conectado em rede. Os dois apresentam o mesmo problema no chrome
Comment: Não esta mesmo lá, olha as duas imagens, pelas duas que vc postou os HTMLs estão totalmente diferentes, o problema não tem nada haver com favicon, ou seja você deve estar salvando no seu computador, mas o endereço 192.168.1.220 é outro computador, ou seja vc salvou só na sua maquina local, entendeu?
Comment: Não amigo, o código fonte está apenas no 192.168.1.220. Todas as alterações estão sendo feitas neste computador (192.168.1.220). O codigo fonte está realmente diferente, o chrome não está reconhecendo o link para o favicon. este é o problema
Comment: Vamos [continuar esta discussão no chat](http://chat.stackexchange.com/rooms/71237/discussion-between-amauri-and-guilherme-nascimento-p).
Comment: @Amauri esquece o favicon, olha os códigos HTML das duas imagens, estão diferentes, esta claro que não é o mesmo arquivo, entenda você deve estar salvando o home.php na sua máquina, mas precisa salvar na maquina 192.168.1.220 também, que presumo que é uma máquina diferente
Comment: tentando abrir o chat aqui, so um minuto

Alterar favicon em um projeto Wamp


D7, AmazonS3, composer_manager and awssdk libraries understanding

Question: I have been struggling with the installation of amazonS3. Although I was finally able to have it working pretty well, I would like to confirm with experts my understanding of the problem, and check that I got it right.
This is what I did :

installed composer_manager
installed with drush awssdk and awssdk_ui.
installed amazons3 module
while composer_manager states that all required libraries were installed, drupal status report still says that awssdk fails to load.
S3 is working fine and stores files attached to  a node.

I think I understand my mistake: with Composer Manager installed and all requirements met, I don't need awssdk and awssdk_ui modules.
These modules check that the libraries/awssdk/sdk.class.php is present, but composer_manager install the required library in vendor directory.
With this understanding, I disabled AWS SDK for PHP and awssdk_ui, and it seems to work well.
I believe that i have read to much docs on the Net, and mix two incompatible ways to install amazonS3.
Do I have the right understanding?
Answer: In the last days, I have repeated the painful procedure many times on different AWS EC2 instances to be sure to have a precise and repeatable procedure.
I am thus very confident about my understanding of it today.
I have written a step-by-step procedure to deploy amazonS3, with the main encountered problems and how I dealt with them. I believe that such a document would have helped me a lot some days ago (and back then, I didn't found any), so here is my proposal. I would appreciate some real experts validation there !
1) install Drush or update to get 5+ version
2) apply all valid patches from the amazonS3 README (https://www.drupal.org/project/amazons3), mainly to allow S3 generated URI s3:///filename.ext" to be supported by media and fileentity related modules
3) Install composer (goes without saying, but I missed it at first!) with :
curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer
alias composer='/usr/local/bin/composer.phar'

composer command should return usage tips
4) Install PHP's cURL extention. On Ubuntu, sudo apt-get install php5-curl is sometimes not enough. (See "cURL library required to enable drupal ...")
sudo apt-get update
sudo apt-get install curl libcurl3 libcurl3-dev php5-curl
sudo /etc/init.d/apache2 restart

5) From the drupal_root folder, launch amazons3 download and install
drush  dl composer_manager
drush  en composer_manager

6) Configure the directory where to put vendor and composer.json files. The default directory are outside of Drupal's directory (vendor : "../vendor", composer: "../")
Once the composer_manager is enabled, I went admin/config/system/composer-manager/settings  and set the fields as follows (again, refer to best practices for RW permissions of Drupal folders).
7) Be sure to give write permissions to www-data to the folder. Especially with the folder being outside of the drupal_root folder.
8) Download and install amazonS3 using Drush and composer, since AWSSDK libraries will be automatically installed with all dependencies.
drush dl amazons3
drush en amazons3

These commands should automatically generate composer.json in the previously indicated folder. You can check that all the dependencies were correctly gathered from amazonS3 and other modules requirements.
In my case, composer.json contains :
{
"require": {
    "aws/aws-sdk-php": "2.8.5-p1",
    "capgemini/drupal_doctrine_cache": "0.0.1-p1",
    "doctrine/cache": "~1.4",
    "php": ">=5.3.0"
},
"config": {
    "autoloader-suffix": "ComposerManager"
},
"require-dev": {
    "phpunit/phpunit": "~4.6",
    "satooshi/php-coveralls": "0.6.*"
},
"repositories": [
    {
        "type": "vcs",
        "url": "https://github.com/deviantintegral/drupal-doctrine-cache.git"
    },
    {
        "type": "vcs",
        "url": "https://github.com/deviantintegral/aws-sdk-php.git"
    }
],
"autoload": {
    "psr-4": {
        "Drupal\\amazons3\\": [
            "../../modules/contrib/amazons3/src/"
        ]
    }
}

}
9) I can't explain why, but the first install generated an incomplete composer.json . You can regenerate the file using by accessing /admin/config/system/composer-manager and click regenerate, or running from drupal_root folder the following drush command :
composer-json-rebuild

9bis) You can also, directly from the composer folder (which location was set previously) and run :
composer install. The command will download all required PHP libraries in composer vendor's folder.
I personally found it very confusing all the locations on my server where composer files where located :

Some docs advise to put composer folder in a ~/.composer/vendor folder in user's home directory. - - Drupal created a .composer in a .drush/composer (to indicate that Drush is required for that user)
I found a ~/.composer in my root folder. It contains users's credentials to access github repository to load the required packages
Most of the docs recommend to store a composer folder just above the drupal_root
I choose to follow these last recommendations to put it in the sites/all/libraries/composer in order to commit the code. But I can't say whether this is the right way to proceed.

10) Finally, retrieve you AWS API KEY and AWS SECRET KEY from console.aws.amazon.com and  configure amazonS3 in /admin/config/media/amazons3, by setting your Security Credentials: Access Key ID and Secret Access Key, and S3 Bucketname. You can also put the values in the settings configuration file, by adding the following lines to your settings.php :
$conf['amazons3_key'] = "<AWS API KEY>";
$conf['amazons3_secret']="<AWS SECRET KEY>";
$conf['amazons3_bucket']="<bucketname>";

11) AmazonS3 should be working : create a file field on a content type, select Amazon S3 as the location to save to. Create a node, attach a file,Save and view the node. The attached file should be displayed as a link to you S3 bucket.
During  my last install, I was able to avoid most of the problems encountered in my previous tries : cURL, read/write permissions, composer unable to parse version requirements, etc.
The last error I was confronted to, was a composer out of memory problem, which I fixed with the following :
sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
sudo /sbin/mkswap /var/swap.1
sudo /sbin/swapon /var/swap.1

I hope this note will help someone, since S3 really worth a few headaches with the installation. Besides, the last release of the module is only two months old.
Comment: Sorry, the folder is: sites/default/files/composer
Comment: After rebuilding json, from command line, I had to go to:
/sites/all/modules/amazons3
and run:
composer install


Animation not working in Windows Phone 8.1

Question: I am just trying to create a simple animation in my Windows Phone 8.1 Application
I am thinking to create an animation effect by changing 6 images at some 300 milliseconds period, which ends up leaving a custom animation using 6-7 images.
but the problem with the below code is, it isn't creating a flicker effect, it just displays 1st image and then last one.
I tried increasing the time period, but still the same.
<code>animationBg = new ObjectAnimationUsingKeyFrames();
storyboardBG = new Storyboard();
Storyboard.SetTarget(animationBg, imgMinuteBackground);
Storyboard.SetTargetProperty(animationBg, "Source");
storyboardBG.Children.Add(animationBg);
AddFrame(animationBg, "Images/clockbg2.png", 50);
AddFrame(animationBg, "Images/clockbg3.png", 100);
AddFrame(animationBg, "Images/clockbg4.png", 150);
AddFrame(animationBg, "Images/clockbg5.png", 200);
AddFrame(animationBg, "Images/clockbg6.png", 250);
AddFrame(animationBg, "Images/clockbg1.png", 300);
storyboardBG.Begin();

//AddFrame Function:
public void AddFrame( ObjectAnimationUsingKeyFrames animation, string strImagePath, int interval)
{
    var keyframe = new DiscreteObjectKeyFrame
    {
         KeyTime = KeyTime.FromTimeSpan(TimeSpan.FromMilliseconds(interval)),
         Value = strImagePath
    };
    animation.KeyFrames.Add(keyframe);
}
</code>
I changed the "Source" to new PropertyPath("Source") and it is working for Windows Phone 7 Applications.
Comment: So are you saying that `Storyboard.SetTargetProperty(animationBg, new PropertyPath("Source"));` works for Windows 7, but not Windows 8?  That's strange.  I would also try using this:  `Storyboard.SetTargetProperty(animationBg, new PropertyPath(Image.SourceProperty));`
Comment: it doesn't works in 8.1
Answer: As stated on MSDN tutorial, animation only available for Windows Phone 8.1 Silverlight app.
http://msdn.microsoft.com/en-us/library/windows/apps/jj206955(v=vs.105).aspx#BKMK_Startingstoppingpausingandresuming

Applies to: Windows Phone 8 and Windows Phone Silverlight 8.1 | Windows Phone OS 7.1

It will not work for Windows phone 8.1 app


Why Jackson is so slow?

Question: I use simple Jackson code to convert from my object to json String,
but it really slow. It takes 70 milisecond to convert only one object as below
in my machine. Did I do something wong?
<code>ObjectMapper myObjectMapper = new ObjectMapper();
void testJson()
{

    MyClass state = new MyClass();
    try
    {
        String result = myObjectMapper.writeValueAsString(state);
    } catch (Exception ex)
    {

    }

}
</code>
MyClass with only 4 members
<code> MyClass
    {
    public int a;
        public  int b;
        public int c;
        public String d;
    }
</code>
Comment: You cannot just measure performance from _one_ creation... Try and run it 1000 times and you will see that it takes far less than 70 ms * 1000.
Comment: Builtin serialization and Jackson don't have the same purpose at all! You are comparing apples and oranges here. If you want to use Jackson, it means you have a use for JSON; builtin serialization cannot do that for you.
Comment: I agree, but comparing to built-in serialization of java.
I've tested 10000 times and serialization has won.(I've changed Jackson to writeValueAsBytes). My main purpose is to convert from object to byte[]. Thanks
Comment: Yeah, do you know some other ways to convert from object to byte array?. That's what i'm doing.
Comment: Your question is vague; what do you want to do with this byte array? Send it to a peer? Use it locally only?
Comment: How about not treating the byte array as an object but just as it is - an array of bytes? Basic IO allows you to read and write them.
Comment: I convert object to byte array to send it over TCP network. And the I need both high speed and low size
Comment: @Gimby I don't understand your aprroach. I need to convert MyClass's instance into byte array, then send it over network
Comment: Oh sorry, I was being confused by you saying "do you know some other ways to convert from object to byte array?". You made it sound like you were trying to serialize a byte array, not an object of your own custom class.
Comment: Now I found out three solutions for sending MyClass's instance over network:
1: use Jackson to convert into byte[] and send.
2: use built-in serialization to convert into byte[] and send.
3: convert MyClass's members into String, then into byte[] and send.
Is there any other better solution ?. Thank to all you
Comment: are you reusing the ObjectMapper instance? Its my understanding it does some sort of internal caching. You could try reusing the ObjectMapper and see if you see any gains.
Answer:
I use simple Jackson code to convert from my object to json String, but it really slow. It takes 70 milisecond to convert only one object as below in my machine. Did I do something wrong?

70 milliseconds to encode and send a little JSON message is implausible.  There is little doubt in my mind that the real explanation is that the 70 millisecond measurement an artifact of the way that you benchmarked your code.  Probably, you didn't allow for JVM warm-up effects.
So, yes, you did your benchmarking incorrectly.  Probably.  For more information on what you may have done wrong, see:

How do I write a correct micro-benchmark in Java?, and
the final section of this answer.

Now I found out three solutions for sending MyClass's instance over network: 1: use Jackson to convert into byte[] and send. 2: use built-in serialization to convert into byte[] and send. 3: convert MyClass's members into String, then into byte[] and send. Is there any other better solution ?

In theory (i.e. if you have enough skill, time and patience) the best performance can be achieved by encoding the data by hand into a byte buffer and sending that.  But that is only theoretical.
If you are looking for practical solutions that are potentially faster than the alternatives you have tried, take a look at Google Protocol Buffers.  They are reputed to be fast ...
But I suspect that you may have drawn the wrong conclusion from your (probably) poorly designed benchmark; see above.  I strongly recommend you redo that before you look for faster alternatives.

70 milliseconds for one object, but when I try 1000 objects, it only takes 114 milliseconds, isn't it strange?

Actually, it is not at all strange when you take account of the way that the JVM works.
When the JVM starts, it loads your code and starts running it using the bytecode interpreter.  After the code has been interpreted for a bit, the JVM runs the JIT compiler to produce optimized native code for the methods that are being called frequently.  This compilation process takes a significant amount of time.
So when you are measuring the time taken to send one object, you are probably really measuring the time to send one object AND do a bunch of JIT compilation.  But that JIT compilation work won't need to repeated.  Net result - processing one object appears to takes a surprising long time, compared to 1000.
JIT compilation is one of the common JVM warmup effects that can distort a poorly written Java benchmark.
Comment: 70 miliseconds for one object, but when I try 1000 objects, it only takes 114 miliseconds, isn't it strange?.
I willtake a look at Google Protocol... later.
MyClass is quite simple, it just contains some members that stores data to transfer. So I just build a String such as: "1,100,abc", then encode it into bytes, and implement method to decode.
Comment: *"Isn't it strange?"* - Nope.  It is just consistent with a poorly designed benchmark.

Why Jackson is so slow?


Alternative terminal emulation for Google Cloud browser SSH login?

Question: Can the Google Cloud SSH browser support any other character sets besides ASCII?
I go into my Dashboard, then click on SSH.

After that, the CLI "gui" uses extended ASCII. Instead of extended ASCII lines being used, it seems like only US7ASCII is being used. Is there a way to change the character set or terminal emulation used by default?
Answer: You can use alternative methods to SSH your instance, like PuTTY

https://cloud.google.com/compute/docs/instances/connecting-advanced#thirdpartytools

And you can use the Cloud Shell for SSH running this command
gcloud compute ssh --project [PROJECT_ID] --zone [ZONE] [INSTANCE_NAME]

Which character encoding are your OS using? You can check it running the command locale if this looks fine try commenting out SendEnv LANG LC_* in the local /etc/ssh/ssh_config file.
https://unix.stackexchange.com/questions/3218/ssh-and-character-encoding


Does the $\lim_{(x,y) \to (\infty,\infty)} ({x+y})/({x^2 - xy + y^2})$ exists?

Question: Can you help me with counting this limense. Any hints, ... etc, If It exists.
Thanks in advance.
Answer: Hint: $\left|\dfrac{x+y}{x^2-xy+y^2}\right| \le \dfrac{2|x+y|}{x^2+y^2}\le \dfrac{4}{|x+y|}$ By Cauchy-Schwarz inequality for the second one and $x^2 -xy + y^2 \ge \dfrac{x^2+y^2}{2}$ for the first one.
Answer: What about evaluating the limits by using substitution $x=1/X, y=1/Y$ so that
$lim_{(X,Y)\rightarrow (0,0)}\frac{XY(X+Y)}{X^2-XY+Y^2}$
Can you proceed from here?