append or concatenate audio files in swift

Question: Hi I want to append voice files.
I'm recording voice with AVAudioRecorder, but to play the recording I need to call "stop", but after playing it I want to continue record. Like the native iOS Voice memo app.
Should I use AVMutableCompositionTrack and how do I do that in swift? Thanks!
Answer: If you are looking to simply pause your recording and continue it later you can use AVAudioRecorder's pause() function rather than stop() and it will continue the recording when you use play() again.
However, if you are looking to actually concatenate audio files, you can do it like this:
<code>func concatenateFiles(audioFiles: [NSURL], completion: (concatenatedFile: NSURL?) -> ()) {
    guard audioFiles.count > 0 else {
        completion(concatenatedFile: nil)
        return
    }

    if audioFiles.count == 1 {
        completion(concatenatedFile: audioFiles.first)
        return
    }

    // Concatenate audio files into one file
    var nextClipStartTime = kCMTimeZero
    let composition = AVMutableComposition()
    let track = composition.addMutableTrackWithMediaType(AVMediaTypeAudio, preferredTrackID: kCMPersistentTrackID_Invalid)

    // Add each track
    for recording in audioFiles {
        let asset = AVURLAsset(URL: NSURL(fileURLWithPath: recording.path!), options: nil)
        if let assetTrack = asset.tracksWithMediaType(AVMediaTypeAudio).first {
            let timeRange = CMTimeRange(start: kCMTimeZero, duration: asset.duration)
            do {
                try track.insertTimeRange(timeRange, ofTrack: assetTrack, atTime: nextClipStartTime)
                nextClipStartTime = CMTimeAdd(nextClipStartTime, timeRange.duration)
            } catch {
                print("Error concatenating file - \(error)")
                completion(concatenatedFile: nil)
                return
            }
        }
    }

    // Export the new file
    if let exportSession = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetPassthrough) {
        let paths = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true)
        let documents = NSURL(string: paths.first!)

        if let fileURL = documents?.URLByAppendingPathComponent("file_name.caf") {
            // Remove existing file
            do {
                try NSFileManager.defaultManager().removeItemAtPath(fileURL.path!)
                print("Removed \(fileURL)")
            } catch {
                print("Could not remove file - \(error)")
            }

            // Configure export session output
            exportSession.outputURL = NSURL.fileURLWithPath(fileURL.path!)
            exportSession.outputFileType = AVFileTypeCoreAudioFormat

            // Perform the export
            exportSession.exportAsynchronouslyWithCompletionHandler() { handler -> Void in
                if exportSession.status == .Completed {
                    print("Export complete")
                    dispatch_async(dispatch_get_main_queue(), {
                        completion(file: fileURL)
                    })
                    return
                } else if exportSession.status == .Failed {
                    print("Export failed - \(exportSession.error)")
                }

                completion(concatenatedFile: nil)
                return
            }
        }
    }
}
</code>
Comment: I get following error when I try to concatenate multiple audios to the same file like record -> stop -> Play -> Record -> stop -> Play -> repeat.             FigFormatReaderCreateForStreamReturningResults signalled err=-12848 (instantiation.firstFailedAttempt.error) (Error creating FormatReader) at /Library/Caches/com.apple.xbs/Sources/EmbeddedCoreMedia_Sim/EmbeddedCoreMedia-2765.6/Prototypes/FormatHandlers/FigFormatReader_Common.c:2870


Converting word problem into algebra

Question: A microprocessor plant that manufactures computer chips costs $\$400,000$ per day to operate. It costs $\$30$ to produce one chip which sell for $\$190$ dollars. How many chips does the plant have to make to break even on one day operation?
Let $X$ be the number of chips I need to make.
$400,000 = 190X-30X$ (but I'm not sure this is even right). I'll continue anyway, I'll check it later...
$400000 = 160X$
$X=2500\quad$ ?
$2500 * 190 = 475000$ (its wrong)
Can someone help me? I think that formula I have is very wrong and I can't figure out another one...
Comment: 2500 is the answer. It is not wrong. Please explain clearly why you think it is wrong. (See Gadi A's comments below.)
Comment: You should calculate number of chips*(selling price per chip - manufacturing cost per chip) to get 400 000. In this case you have just calculated number of chips*selling price per chip. So, what is $2500 \times (190-30)$?
Answer: You seem to have the correct answer (definitely the correct method to solve this). Why do you think "2500 * 190 = 475000" means you are wrong? Remember that the number of chips also takes into account their manufacture cost.
Comment: I still don't understand why you consider the solution wrong. There is no sense in expecting 400*190 to come out 400000, since what you are trying to "break even" here is the total costs against the total revenues, and the total cost is larger than 400000.
Comment: When you check the problem it has to be 400000 not 75000 more, that's where I'm lost.
Comment: Why? Try to elaborate.
Comment: I made this problem up to help me solve similar problems, the original problem has multiple choice answers where the solution works out to exactly break even on one day of operation.
Answer: Your method of checking the answer is incorrect.  $2500\cdot 190=475000$ is the total gross income from producing 2500 chips, but the cost of producing those 2500 chips is not just 400000, but $400000+2500\cdot 30=475000$, since each of the 2500 chips costs \$30 to produce.

Converting word problem into algebra


Select DISTINCT with multiple OR in sql query

Question: I am trying to write a query with multiple OR and using DISTINICT for multiple column.But the result is NULL what i am missing in my query. Anyone can help me here please.
SELECT DISTINCT
   P.user_p_id,
   P.surl,
   M.user_id,
   M.user_p_id
   FROM users P, page M
   WHERE (P.user_p_id = '$urlid' OR P.surl = '$urlid' OR M.user_id = '$urlid' OR M.user_p_id = '$urlid')

Comment: do you get results without the where?
Comment: you should also show the demo of the table
Comment: @SayedMohdAli Sorry question edited
Answer: You should do a proper join with an ON clause:
SELECT DISTINCT
   P.user_p_id,
   P.surl,
   M.user_id,
   M.user_p_id
   FROM users P INNER JOIN page M
   ON M.user_id = P.user_p_id
   WHERE '$urlid' IN (P.user_p_id, P.surl, M.user_id, M.user_p_id)

I use ON M.user_id = P.user_p_id although it's not clear if you want these columns to be matched.
You may change the type of the JOIN to LEFT if this is what you need.
Comment: Thanks for your answer, it worked. But i don't understand why someone gives you - vote.


Where can I find useful stuff to learn music production?

Question: I'm trying to learn how to produce music. More specifically, I use a lot of MIDI for orchestral/chamber works. And as a classically trained musician, I don't find it useful when they start trying to explain things outside of the production subject, like harmony, which it's actually a topic within music composition/analysis.
I want to learn what's a compressor, a de-compressor; how to use reverb; what buses are and why the are millions of them when I open a DAW; what are the parameters that control MIDI data and what they do; what is side-chain, a limiter, an expander; how to control MIDI from other programs; how to use VSTi's/AU's; what's CC11, CC4, CC<code>(some random number)</code>; how can I use these things whenever I need to... And I think the list goes on.
And whenever I go, I get confused because there's information I don't know yet.
(I don't know if you could tell by my writing, but I'm quite stressed writing this x'D I don't know where to start)
Comment: Tip: there are plenty of producers who want to become better at the things you are proficient in (arrangement, harmony...). Look for somebody you can personally exchange knowledge with, it will be more worth that a hundred youtube videos!
Comment: Glen Fricker has a great youtube channel. If you don't mind the large amount of cynicism and swearing, but I'm sure his paid for content is civil. https://www.youtube.com/user/SpectreSoundStudios
Comment: If all you're writing are orchestral and chamber works, you don't need to learn what a compressor, a de-compressor, and side-chain are because you won't be using them.
Answer: The most important thing is to start producing if you aren't already. I'm guessing you are. But if you aren't or you are getting hung up on things that you don't know instead of finishing something, just finish it. Do the best you can on what you're working on now and then ask "how could this be better" and use that as a guide for what to learn next or practice more of. But the point, again, is to just start and learn as you go. It's a much harder task to try to learn everything and then get to work.
Make a list of those things that you either don't know or want to improve upon—it looks like you've started already—and tackle them one by one. Open a new browser window, google "audio compression" (or whatever), open a bunch of the results and read them all. You'll probably have more questions but that's okay. Take notes and write down the things that you learn and the things that you're still confused about. When you have a specific question that google doesn't seem to answer, try asking that as it's own question here or at an audio production forum.
Also aim for learning practical things incrementally. For instance, do you really have an immediate need to know what MIDI CC11 is? If so, then definitely learn it. But don't waste time learning something just because you think you're expected to know it. Learn what is practical to your work right now and it you'll eventually get around to most of it. For example because you're working with MIDI a lot it might be good to read up a bit about how MIDI works generally and what a CC (continuous control) is vs a note. But it's not like you need to memorize what each CC number is. When you have a need to map something to a CC then look up a CC number list and find the most appropriate number. Learn it when you need it.
And, on the note of learning incrementally, it's worth noting that no matter how hard you try, you won't learn everything there is to know about, say, compression in your first go. Learn the basics and try using it in practice. Using compression well takes a while because you need experience using it to hear how it sounds and how using too much or too little affects the mix. Even if you did the deepest dive of books and video tutorials, you still wouldn't really understand it until you've used it for a while. And even years later you'll probably learn new things.
I say this not to be discouraging or make it sound harder than it is but rather to underscore my first paragraph. Make sure you are constantly making stuff and mixing it and then critiquing what you've made. There are a lot of things to learn but that process of making something and then critiquing it will guide your learning path more than anybody here could.
Comment: You forgot to mention the workgroup, peers or friends and the web forums for questions, help and exchange.
Comment: You got a point on that. Since what I study is more focused on music notation rather than audio and MIDI, getting to learn very little things about MIDI and Production has been really a challenge. And putting those things in practice made my questions grow too much (and all because I wanted to hear what I write with better sounds). Getting to understand how MIDI works is just my starting point to Production.
Answer: If you are going to be a sound engineer then MAYBE, harmony and arrangement skills are not necessary, but that is a big IF. If you want to produce actual music with a holistic approach then you need all of it.
You don't need to be a savant in an instrument, but decent piano and guitar chops are very useful. If you can get your head around basic vocal technique then working with singers will be much improved.
The thing is pure engineering work is not really all that profitable. There are too many people who know how to work a sound-board. Now what is really rare is a Quincy Jones / Eddie Kramer type who can do the technical stuff, but can help get the best out of musicians and make some of the most iconic records ever. That is what you win Grammy's for and that is what makes you a millionaire.
The main problem I have with your post is that you don't know what work you are going to fall into eventually. Denying yourself an education is never a good thing. Have you considered that if your production school teaches harmony it may be that the skills are necessary? Do you think schools teach subjects just for the heck of it? It may very easily be that your schools knows something about this subject that you do not.
Believe you me, you will not be the first, or the last, student who was taught something he / she did not like at college, but was glad later-in-life he learned.
Comment: Well, I think I didn't express myself correctly. I say that I don't find it useful because I'm being taught those subjects already at university. Moreover, my teachers are great at what they do, and some are composers themselves; so, any question I had regarding Music Theory could be answered by them. And I'm already used to those topics, so any technical information about music theory is no big deal (not that it couldn't be difficult, though. And I'm crazy about music theory). Production and Sound Engineering for me is "that thing that's not taught at school but I find useful to learn".
Answer: A contemporary music school or conservatory offers courses in music and computer.
If this doesn't exist it might be up to the students to ask for workshops or organize them.
What we did as teachers: we organized courses together with music centers, producers like steinberg, klemm and others, regional and national workshops.
But this was 30 years ago before internet and youtube tutorials. At each corner you find a crack in music, wavelab, technology. Look at your school for partners with same interests and ask your teacher for a workshop.
But don't think that you will handle the technology without learning basic music theory, like melody, rhythm, chords, progression and ear training - and practicing an instrument, at least a keyboard.
Comment: If you're in the US I'd also suggest checking your local community college.  They may have introductory courses if not on music production then perhaps one of the software packages.
Comment: I'm not in the US, though. But I'll see what I can get in the place I live in. Thank you!


MassTransing SQS/SNS - how to enable encryption at rest on all topics

Question: I know that to enable encryption at rest for SNS Topics I need to add topic attribute like this one
cfg.Publish<SomeEvent>(p =>
{
    p.TopicAttributes.Add(QueueAttributeName.KmsMasterKeyId, "<key arn>");
});

But the problem I have is that we are using very simple configuration for aspnetcore with UsingAmazonSQS and cfg.ConfigureEndpoints(context) doing all those configurations automatically so there is no call to cfg.Publish. This is very important as we have many messages to configure. I believe the same thing for queue attributes can be achieved by registering custom implementation of IConfigureReceiveEndpoint but I can't find equivalent for topics.
How can I keep using the automatic method and add this encryption topic attribute for all topics?
Comment: Added, it should be in the latest prerelease package.
Answer: There isn't currently a way to apply topic attributes to all topics, but it isn't an unreasonable request for an addition. Surely someone can build it as a pull request adding to the general publish topology for Amazon SQS.
Update
I added the ability to specify attributes on the root PublishTopology which should be applied to all topics.
Used in code (from the comment below):
configure.UsingAmazonSqs((context, cfg) =>
{
   cfg.PublishTopology.TopicAttributes
       .Add(QueueAttributeName.KmsMasterKeyId, encryptionKeyArn);

   cfg.ConfigureEndpoints(context);
});

Comment: As a thank you here are few more details if any will be trying to use this new functionality (available since v7.2.2) - making all topics encrypted is now as simple as

`configure.UsingAmazonSqs((context, cfg) => {
 cfg.PublishTopology.TopicAttributes.Add(QueueAttributeName.KmsMasterKeyId, encryptionKeyArn); cfg.ConfigureEndpoints(context); })`


For the given endomorphism calculate $\varphi^{106}(1,1,4)$

Question: I have an endomorphism $\varphi: \mathbb{R}^3\rightarrow\mathbb{R}^3$ and $\varphi(0,1,1)=(0,1,1), \varphi(2,2,0)=(0,0,0), \varphi(1,0,0)=(-1,0,0)$. I want to calculate $\varphi^{106}(1,1,4)$.
I start by making up a matrix A:
$$A=\begin{bmatrix} -1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}$$
Then I calculate eigenvalues using $(A-I\lambda) = 0$ which is true for $\lambda=\{-1, 0, 1\}$. I finish up with eigenvectors:
$$C=\{[1, 0, 0], [0, 1, 0], [0, 1, -1]\}$$
What do I do next and is this the correct start to this solution?
Comment: Whith what you have you can find easily the diagonalization $A=PDP^{-1}$. Now apply the $106th$ power to get $A^{106}=PD^{106}P^{-1}$ and finaly compute $A^{106}(1,1,4)^t$ which is the value you want.
Comment: Note that $(1,1,4)=4\times (0,1,1)-\frac 32\times (2,2,0)-2\times (1,0,0)$.
Comment: I think you've got the wrong matrix.
Answer: Let $v_1=(0,1,1)$, $v_2=(2,2,0)$, and $v_3=(1,0,0)$. Then,

since $\varphi(v_1)=v_1$, $\varphi^{106}(v_1)=v_1$;
since $\varphi(v_2)=0$, $\varphi^{106}(v_2)=0$;
since $\varphi(v_3)=-v_3$, $\varphi^{106}(v_3)=v_3$.

And now, since$$(1,1,4)=4v_1-\frac32v_2+4v_3,$$you have\begin{align}\varphi^{106}(1,1,4)&=\varphi^{106}\left(4v_1-\frac32v_2+4v_3\right)\\&=4v_1+4v_3\\&=(4,4,4).\end{align}


Qual a diferença entre RMI e JRMP?

Question: Eu só conheço o RMI...
JRMP  é só uma forma diferente de chamar o RMI ou é um outro protocolo?
No caso de JRMP ser um outro protocolo, qual a sua diferença em relação ao RMI?
Answer: RMI
O RMI (Remote Method Invocation) é uma interface de programação que permite a execução de chamadas remotas no estilo RPC em aplicações desenvolvidas em Java. É uma das abordagens da plataforma Java para prover as funcionalidades de uma plataforma de objetos distribuídos. Esse sistema de objetos distribuídos faz parte do núcleo básico de Java desde a versão JDK 1.1, com sua API sendo especificada através do pacote java.rmi e seus subpacotes.
Através da utilização da arquitetura RMI, é possível que um objeto ativo em uma máquina virtual Java possa interagir com objetos de outras máquinas virtuais Java, independentemente da localização dessas máquinas virtuais.
JRMP
JRMP (Java Remote Method Protocol) é o protocolo específico da tecnologia Java para olhar para cima e fazer referência a objetos remotos. É um protocolo de nível fio funcionando em um nível abaixo de RMI (RMI) e através de TCP/IP.
Camadas de Software do Sistema RMI
A implementação do RMI é essencialmente feita de três camadas de abstração. A camada Stub e Skeleton está abaixo dos olhos do desenvolvedor. Esta camada intercepta as chamadas de métodos feitas pelo cliente para que a variável de referência da interface redirecione essas chamadas para o serviço RMI remoto.
A próxima camada é a Remote Reference Layer. Esta camada sabe como interpretar e gerenciar referências feitas dos clientes para os objetos do serviço remoto. A conexão do cliente ao servidor é Unicast (uma-para-um).
A camada de transporte é baseada nas conexões TCP/IP entre as máquinas em uma rede. Usando essa arquitetura de camadas, cada uma das camadas poderia ser facilmente melhorada ou substituída sem afetar o resto do sistema. Por exemplo, a camada de transporte poderia ser substituída por uma camada que implemente conexões UDP/IP, sem afetar as camadas superiores.
Esta camada lida diretamente com a comunicação entre as várias JVM's, usando TCP/IP. É importante referir que mesmo que as JVM's sejam executadas no mesmo computador, o RMI recorre sempre à comunicação TCP/IP. Isto significa que é sempre é necessário possuir uma interface de rede funcional para se poder utilizar RMI (mesmo tendo a aplicação cliente e a aplicação servidora a executar no mesmo computador). Sobre a pilha TCP/IP o RMI possui um protocolo denominado JRMP, e que permite ultrapassar alguns obstáculos que podem surgir na comunicação de rede via TCP/IP. Por exemplo, o JRMP permite multiplexar várias ligações TCP/IP numa única ligação TCP/IP ultrapassando imposições de utilização de apenas uma ligação em alguns ambientes (ex.: certos browsers a correr Applets RMI)

Java RMI não é exatamente um protocolo de transporte?
Não. Java RMI é um conjunto de APIs e um modelo para objetos remotos que permite aos desenvolvedores construir facilmente aplicações distribuídas em Java. Por exemplo, usa interfaces Java normais para definir objetos remotos em uma linguagem separada como IDL. Normalmente Java RMI usa uma combinação de serialização Java e o Java Remote Method Protocol (JRMP) para converter a aparência normal de um método em uma invocação remota de métodos.
Com Java RMI, JRMP continuará suportando e melhorando o protocolo nativo para Java RMI.
Que é um protocolo de transporte?
Um protocolo de transporte define um conjunto de formatos de mensagem que permitem passar dados através de uma rede de um computador a outro. Java RMI suporta seu próprio protocolo de transporte (JRMP) e outros protocolos regulares da indústria incluindo IIOP.
Veja mais em: Java: Remote Method Invocation
Comment: Obrigdado pela Atenção , poderia esclarecer mais essa parte :  " É um protocolo de nível fio funcionando ..."
Comment: @PenaPintada, é tradução ao pé da letra de "wire level protocol". Em uma rede, um protocolo de fio é o mecanismo de transmissão de dados do ponto A ao ponto b. O termo é um pouco confuso, porque soa como camada 1 da rede, o que coloca fisicamente os bits "no fio". Em alguns casos, pode referir-se a camada 1; No entanto, ele geralmente se refere a camadas mais altas, incluindo Ethernet e ATM (camada 2) e ainda mais elevados camada distribuída protocolos de objetos, tais como SOAP, CORBA ou RMI.

Qual a diferença entre RMI e JRMP?


MVC 5 application with AJAX and SignalR hangs in Chrome

Question: I have a MVC 5 application that uses AJAX calls and SignalR. The application hangs in Chrome when the following order of actions get executed:

User visits Page A and after the page has loaded the console looks like this
SignalR: serverSentEvents transport starting.
SignalR: Attempting to connect to SSE endpoint
SignalR: EventSource connected.
SignalR: serverSentEvents transport connected. Initiating start request.
SignalR: The start request succeeded. Transitioning to the connected state.
SignalR: Now monitoring keep alive with a warning timeout of 13333.333333333332, keep alive timeout of 20000 and disconnecting timeout of 30000

User clicks on a link of Page A that invokes an AJAX call
The AJAX call from 2. returns successfully and the UI is updated via jQuery
User initiates a page refresh via F5 and after the page has loaded the console looks like this
SignalR: serverSentEvents transport starting.
SignalR: Attempting to connect to SSE endpoint
SignalR: EventSource connected.
SignalR: serverSentEvents transport connected. Initiating start request.

User tries to navigate by clicking a simple link (a tag with href attribute) and the site hangs for roughly 2 minutes until the new page is served

I noticed that after 4. the console does not show the
SignalR: The start request succeeded. Transitioning to the connected state.
SignalR: Now monitoring keep alive with a warning timeout of ...

lines, so it has something do to with SignalR i guess.
Has anyone encountered such a huge delay in the past and knows how to avoid it?
I have tried

changing transport protocol of SignalR
the window.onbeforeunload workaround
different jQuery versions
1000 other things i can't remember already

Any help is greatly appreciated as i'm running out of ideas.
Comment: @Dimitry Nope, couldn't find a way thus sadly needed to remove SignalR functionality
Comment: Check this link, it may be helpful http://stackoverflow.com/questions/24668835/debugging-signalr-hang
Comment: This might also be an issue here: https://github.com/OrchardCMS/Orchard/issues/5975
Answer: This question was posted while using Orchard. After updating Orchard to 1.10.1 the timeouts are gone and SignalR works perfectly in Chrome now.
I inspected each code file while updating to 1.10.1 and there were some changes in the Owin and session stuff, so i suppose it was an issue with the Owin or session implementation / handling.


Java SWT: How to set the window to be resizable \ not resizable on the fly?

Question: After a lot of search and research, I couldn't find any answer.
I will very appreciate the person that could help me.
In SWT java,
I want my window to be not resizable, so I define it like this:
shell = new Shell(SWT.CLOSE | SWT.TITLE | SWT.MIN);

But then I want to change it to be resizable (for example after I click a button).
How can I change the style of the shell on the fly?
Comment: AFAIK, you can't change the style of a `Shell` after creation. Unfortunately, I'm not able to provide an alternative. Looking forward to answers to this.
Answer: You can't change the style of a Shell once it has been created so you can't do this directly.
What you can do is create a new Shell (and all its contents) with the desired style and set its bounds to be the same as the old Shell and then close the old shell. Eclipse does this when it wants to convert a tool tip window in to a normal window.
Comment: OK, thanks, I tried it and managed to get something close to what I wanted.
But I don't like the effect of it - it seems like a window is closed and another one is opened.
Moreover, I need to assign all the widgets again to the new shell.
Good to know this option exists, but I rather prefer to limit the resizability manually (by capturing events of window control and resize the window to the default dimensions).
Anyway, thanks a lot.
Comment: Specifying `SWT.ON_TOP` for the new shell helps make the change more seamless, but yes it is messy if there are a lot of controls.
Answer: If you develop exclusively for windows, you can achieve this with non pure SWT code (non portable). There are no visible glitches, no problems with re-parenting controls (see this bug, for example). Only the resizable property is changed. Here is the snippet:
public static void main(String[] args) {
    final Display display = new Display();
    Shell shell = new Shell(display);

    final int style = OS.GetWindowLong(shell.handle, OS.GWL_STYLE);
    OS.SetWindowLong(shell.handle, OS.GWL_STYLE, style & ~0x00040000);

    shell.open();
    while (!shell.isDisposed()) {
        if (!display.readAndDispatch()) {
            display.sleep();
        }
    }
    display.dispose();
}

WS_SIZEBOX = 0x00040000L
Edit:
simple functions to use:
private static final int WS_SIZEBOX = 0x00040000;

public static void setResizable(Shell shell, boolean resizable) {
    final int style = OS.GetWindowLong(shell.handle, OS.GWL_STYLE);
    final int newStyle = resizable ? style | WS_SIZEBOX : style & ~WS_SIZEBOX;
    OS.SetWindowLong(shell.handle, OS.GWL_STYLE, newStyle);
}

public static boolean isResizable(Shell shell) {
    final int style = OS.GetWindowLong(shell.handle, OS.GWL_STYLE);
    return (style & WS_SIZEBOX) != 0;
}


How can i export all events from Icalendar to CSV

Question: One of my clients has allowed me to view their events on ical via icloud.com (I do not have access to a MAC)
What i need to do is export all the events into either a XML or CSV file.
Currently i cannot see anyway to do this. Can anyone suggest a way to get the data out of ical and into either of the above filetypes.
Many Thanks,
Chris
Answer: xCal is a format that encodes iCalendar objects in XML.  You can read more about the xCal in RFC 6321


Primes congruent to 1 modulo n

Question: I was wondering whether the following statement is true, and if so, how to prove it:

for all $n\in\mathbb{N}$, there exist prime numbers $p$ and $q$ such that $p\equiv1$ (mod $n$) and $q\equiv-1$ (mod $n$)

Feels like it should be true and that it should be fairly easy to prove, but I don't have any idea how. Could someone please shine some light on this for me?
Comment: [Dirichlet's theorem on primes in arithmetic progressions](https://en.wikipedia.org/wiki/Dirichlet%27s_theorem_on_arithmetic_progressions) tells us that there will in fact be infinitely many such primes.
Comment: There a relatively simple way to prove there are infinitely many $p$ using the cyclotomic polynomials, roughly like Euclid's proof of infinitely name primes. I don't know of any way for proving even one $q$ exists.
Comment: This answer shows the case of $p$: https://math.stackexchange.com/a/1160778/7933
Comment: Dirichlet proved that $an+b$ produces infinite many primes if and only if $a$ and $b$ are coprime, but this proof is not easy. This proof implies your statement.
Comment: Thanks. Is there a simple proof for the existence of at least 1?
Answer: Comment: You can construct such primes; tank an arbitrary number like n= 7 we can have:
$p=2\times7-1=13$ ⇒ $p=13\equiv -1 \mod (2)$
$q=6\times 7+1=43$ ⇒ $q=43 \equiv 1 \mod 7$
In fact we must have:
$p=k\cdot n+1$
$q=t\cdot n-1$
From second relation we have:
$n=\frac{q+1}t$
Plugging this in first relation we get:
$p=\frac k t (q+1)+1=\frac kt q+\frac kt +1$
Let $\frac kt=a$ then:
$p=a q + a+1$
This equation has infinite solutions for p, q for certain value of a, among them primes for p and q. The condition is  $k>t ; t|k$.

Primes congruent to 1 modulo n


Entity Translation and Taxonomy

Question: I'm having hard time integrating Entity Translation module with tags (Taxonomy and TagClouds modules). I want the following:

Cloud of tags to be showed on the site
Tags must be translatable, i.e. each tag must be showed in English or Russian, depending on the current language.

There are at least two problems. The first is that ET module breaks TagClouds. More specifically, whenever I enable these two modules (in any order), showed cloud of tags disappears. What is more suprisingly, it even doesn't matter was translation of tags enabled in ET module or not ( admin/config/regional/entity_translation -> translatable entity types -> taxonomy term) -- cloud of tags always ends up disappeared. Dblog doesn't show any errors. Is there any other place I can check for them?
The second problem is that if I make taxonomy term translatable in ET, I get strange errors when I'm trying to translate terms. For example:

If I get to the page of some tag created before enabling ET, for exampleen/taxonomy/term/8, than click Translate tab, I will see that current name is not English nor Russian. If click add button in either language, I will get not found error -- see screenshots.

If I click to edit button in list of tags (admin/structure/taxonomy/tags), I will get even more interesting error -- Access denied, You are not authorized to access this page.
Finally, If I click add items on some vocabulary and save it, I will get the following exception: Exception: Invalid translation language in EntityTranslationDefaultHandler->setTranslation() (line 635 of /var/www/drupal/sites/all/modules/entity_translation/includes/translation.handler.inc).

Name and Description fields of tags are marked as translatable (admin/structure/taxonomy/tags/fields/name_field -> Field translation
Users may translate all occurrences of this field: Name in Теги)
What I am doing wrong?
Drupal version is 7, all modules are up to date. I also use i18n modules for translating the interface, by the way.
Answer: The exception can be fixed by visiting the following page: 'admin/config/regional/entity_translation'.
Then make sure that under the vertical tab 'taxonomy term' and the fieldset of your vocabulary the following is unchecked: 'Hide language selector'.
You may need to check 'Exclude Language neutral from the available languages' if that doesn't work.

Entity Translation and Taxonomy


$X_n\rightarrow X$ in probability, but $\mathbb{E}(X_n)$ does not converge to $\mathbb{E}(X)$

Question: What is an example of a sequence $X_1,X_2,...$ such that $X_n\rightarrow X$ in probability, but $\mathbb{E}(X_n)$ does not converge to $\mathbb{E}(X)$?
Answer: Consider for instance:
$$
X_n = \begin{cases}
n \text { with probability }\frac 1n\\
0 \text { with probability }1 - \frac 1n\\
\end{cases}
$$

$P(|X_n| > \epsilon) = \frac 1n\to 0$ hence $X_n\to X = 0$ in probability.
$EX_n = \frac 1n\times n + \left(1-\frac 1n\right)\times 0 = 1
\to 1 \neq EX$
Comment: @TonyStrong I edited with more details.
Comment: But how do i prove that this converges in probability but the expectation doesn't?
Comment: the exprectation isn't a problem... for the convergence, go back to the definition?
Comment: Why isnt the expectation a problem?


Magento Apache Config & Memory Issues

Question: I have a Magento installation on a VPS that is giving me a headache.
This particular VPS has a reasonable spec - 2gb Memory and 50gb storage. It runs a single domain, with a single Magento install - and nothing else.
About 5 months ago we started having issues. Every so often (about once every 2 or 3 weeks) the VPS would crash - all processes stopped and the only way to restart the container is via Virtuozzo.
Now, however its 2 or 3 times a week. My VPS hosts confirm I am breaching the 2gb memory limit, at which point all VPS processes are killed to stop it bringing the entire node down.
I have not made any config changes to it at all - I was running New Relic on it for a short while, but have removed that in case it was contributing to the issues. I can see nothing in the logs which indicates an issue and we have no CRON jobs running at the time the crashes happen.
The site generates steady, but not huge amounts of traffic (averaging usually less than 100 visits per day)
Is there anything in particular I should have done to the Apache or PHP configs to help? Im not a massivley experienced Apache admin, but know more than enough to solve most problems...
Failing that, any other ideas that might help? Can't afford for this site to be down this much.
Comment: Consider _not_ hosting a business critical website within a public Virtuozzo (or OpenVZ) VPS.
Comment: it all depends on the configuration and load on the server. MAgento does not need lot of visitors to hit the 2GB limit , see your memory configuration that is allowed for each request to use and do the math
Comment: and how big your product catalog is?
Answer: One other useful thing to look at for setting the MAXCLIENTS is ApacheBuddy.
See https://github.com/gusmaskowitz/apachebuddy.pl
This basically looks through the stats and your configuration files and gives you a guideline as to what your MAXCLIENTS setting should be. To be honest I can never afford to go as low as they recommend but it is probably the "correct" answer.
To use/install:
wget https://raw.github.com/gusmaskowitz/apachebuddy.pl/master/apachebuddy.pl
perl apachebuddy.pl
Useful options include -p (for port if you are running varnish for example) and -P (that is a capital "P") to also take into account the PHP memory settings.
Comment: Way to go for including the port flag.
Answer: Few things I can think of right now:
1) Check that you are running prefork MPM and not worker MPM
2) Edit apache2.conf (on Debian is's located under /etc/apache2/apache2.conf:
Make sure the following config is there (appears to be best for us:)
<code>Timeout 30
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 2

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
    StartServers          10
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   2000
</IfModule>
</code>
3) Look at using Varnish as a caching mechanism which will reduce the Apache load
4) Install APC/Memcache
Comment: Yes Varnish will run alongside APC. Make sure to use the module they have too: http://www.magentocommerce.com/magento-connect/pagecache-powered-by-varnish.html
Comment: Many thanks - ok, am definitely running mpm-prefork and already have APC installed. Will check out Varnish as well - will this run alongside APC ok? Have tried to implement your apache conf in my vhost.conf (as I am running under plesk). When restarting apache I get an error about 'startserver not allowed in section'...
Answer: You may wish to try apache-mpm-worker with php5-fpm instead. I noticed a drastic difference when I switched from the standard prefork/mod_php combo.
A few other suggestions:

Check you mysql configuration by running the tuning-primer.sh script. This will provide a good baseline to ensure your mysql is configured properly and you aren't wasting memory on it.
Are you taking advantage Magento's compilation and caching components?
As you only have 2GB of RAM available, ensure that your Apache MaxClients is at a reasonable level. Start with a low number (e.g. 25) and monitor your RAM usage to see if you can increase that some.
2GB isn't a lot of memory for a Magento server with a decent amount of traffic. Have you looked into increasing the RAM or perhaps getting a second server to use for dedicated mysql to free up that RAM for Apache?
Have you verified using a log analyzer (e.g. webalizer, awstats, etc) or possibly even just Apache's extended status module that Apache is the problem at the time of the excessive memory usage?
Answer: It's generally simple math on what to set in your apache config file. You have 2 gigs of ram and probably 1.5 of that can be used by apache.
so that's ~1,500 megs. Make sure your .htaccess file for magento is sane.
php_value memory_limit 128M
php_value max_execution_time 120
(To be honest we have ours set to 512m because many users were getting blank pages. Go read var/log/exception.log to see if that's the case)
In this scenario you can only support a MaxClients of 12! And to be honest I think magento default memory limit is now 256. Which would imply maxclients of 6! I would also suggest using less MaxRequestsPerChild. Perhaps as low as 100. What I would see is a few processes getting bigger and bigger. Apache children do not free up memory until they are destroyed in my experience. That is what this setting will do. Respawn the child process after 100 requests.
I have had lots of problems with apache. We have servers with 7 gigs that fall over like you have described. Our traffic is much greater than yours though.
I just saw you ran mysql on this box too. Not good. You may want to look into running nginx with php-fastcgi. I got it working pretty well and it uses much less memory. Our only deal breaker was we had an extension that uses ioncube that we needed and I could not get to cooperate with it. (I also used xcache instead of apc as our opcode cache because I heard apc will not share memory between the cgi processes)
I have also used nginx as a reverse proxy to serve all static content. That may also work for you. This was much better and we used it for a long time until we decided to implement varnish.
At the very least you need to implement some kind of caching solution for magento.

Magento Apache Config & Memory Issues


Como concatenar um valor com uma string

Question: Olá, eu preciso criar uma função em postgresql que recebe duas idades e a função retorna as datas de nascimento. Eu estou com dificuldade para fazer o calculo da data de nascimento, já tentei usar interval e outros comandos do tipo data para solucionar esse problema.
Answer: Para diminuir 48 anos de uma data:
<code>select now() - 48 * interval '1 year';
           ?column?
-------------------------------
 1968-10-07 20:46:38.699564+00
</code>
Veja que não é preciso fazer concatenação. Basta multiplicar.
Comment: Muito Obrigado, deu certo!

Como concatenar um valor com uma string


SyntaxError: invalid syntax, o que aconteceu?

Question: Estou usando Python 3.7.4, tentei escrever de várias maneiras diferentes e nada funcionou.
character_name = "Johnny"
character_age = "64"
print(+character_name "is cool.")
print("And he is " character_age " years old.")

Erro:

    print(+character_name "is cool.")
                                   ^
SyntaxError: invalid syntax

Process finished with exit code 1

Answer: Este código é quase aleatório. Existem regras a serem seguidas para escrever um código, não pode escrever do jeito que bem entende. Assim funciona:
character_name = "Johnny"
character_age = "64"
print(character_name + " is cool.")
print("And he is " + character_age + " years old.")

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.
Alguns problemas aí:

Tinha um operador de adição, que neste casos seria considerado concatenação logo no começo da expressão dentro dos parenteses da função, e isto não faz sentido, este operador é binário e espera um operando na esquerda e outro na direita, não pode ter nada na esquerda (neste caso ele até está trabalhando como operador unário que é só para confirmar que algo é positivo e só pode ser usado com números, não com textos).
Depois você parece querer concatenar uma variável com um literal string, e não tem o operador, provavelmente porque usou ele no lugar errado.
Aí em seguida nem usou o operador para concatenar a variável com as strings, tem que ter operadores.
O ideal é usar uma forma de interpolação do que concatenar na maioria das situações.

Pode ler mais em Como fazer interpolação de string em Python?.
Seria bom dar uma estuda em como funciona a sintaxe e entender o que cada parte faz, não tente juntar trechos aleatórios no código, isto não é programar.
Comment: Vale comentar que existe o operador `+` unário também, só não se aplica nativamente em *strings*, como é o caso.
Comment: @AndersonCarlosWoss o operador mais sem sentido já criado antes que uma linguagem teve a péssima ideia de fazer e as outras copiaram


What is a selector engine?

Question: I've seen news of John Resig's fast new selector engine named Sizzle pop up in quite a few places, but I don't know what a selector engine is, nor have any of the articles given an explanation of what it is.  I know Resig is the creator of jQuery, and that Sizzle is something in Javascript, but beyond that I don't know what it is. So, what is a selector engine?
Thanks!
Answer: A selector engine is used to query a page's DOM for particular elements, based on some sort of query (usually CSS syntax or similar).
For example, this jQuery:
$('div')

Would search for and return all of the
 elements on the page.  It uses jQuery's selector engine to do that.
Optimizing the selector engine is a big deal because almost every operation you perform with these frameworks is based on some sort of DOM query.
Answer: A selector engine is a JavaScript library that lets you select elements in the DOM tree using some kind of string for identifying them (think regular expressions for DOM elements). Most selector engines use some variation of the CSS3 selectors syntax so, for example, you can write something like:
var paragraphs = selectorengine.select('p.firstParagraph')

to select all P elements in the document with class firstParagraph.
Some selector engines also support a partial implementation of XPath, and even some custom syntaxes. For example, jQuery lets you write:
var checkedBoxes = jQuery('form#login input:checked')

To select all checked check boxes in the login form in the document.
Comment: I guess, you just need to include the "js" file in your page. Do you need JQuery or some other library to use it?
Answer: A selector engine is a way to traverse the DOM looking for a specific element.
An example of a built in selector engine:
var foo = document.getElementById('foo');

Comment: thanks for nice example `built-in selector engine` I read some answers and still ask myself, `getElementById` is a selector engine or not (it looks like, but too obvious so no one mentions :D)
Answer: Also, Sizzle is the engine John Resig is working on currently to replace jQuery's already fantastic selector engine.
Answer: A selector engine is used to find elements in a document, in the same way as CSS stylesheets does. Currently only Safari has the built-in querySelectorAll function which does just that. With other browser you have to use external JavaScript implementations as LlamaLab Selector or Sizzle instead.

What is a selector engine?


Enviar formulario ao controller com ajax

Question: Preciso enviar um formulario com ajax, para que não haja recarregamento de página.
Caso não tenha como, preciso que a tela volte no ponto onde ela estava.

No website tenho estas tabs, a tab onde haverá o submit é AGENDA DE REUNIÕES caso tenha jeito de ir e o controller retorna automaticamente para ela.
Meu código esta assim:

Informações básicas

Membros

Agenda de reuniões

Relatórios

 Informações básicas

                                    {!! link_to_route('igrejas.edit', 'Editar', $igreja->COD_IDENT_IGREJ, []) !!}

Nome

{{ $igreja->TXT_NOMEX_IGREJ }}

Nº de membros

{{ count($igreja->pessoas) }}

Nº de {{ (($igreja->FLG_MODEL_IGREJ == 'C')? 'celulas' : (($igreja->FLG_MODEL_IGREJ == 'P')? 'pequenos grupos' : 'salas dominicais' )) }}

{{ count($igreja->celulas) }}

Cliente desde

{{ date('d/m/Y', strtotime($igreja->DAT_INICI_CLIEN)) }}

Final do contrato

{{ date('d/m/Y', strtotime($igreja->DAT_FIMXX_CONTR)) }}

 Listagem de membros

                            @foreach($celula->membros as $membro)
                            <?php
                            $iniciais = strstr($membro->TXT_NOMEX_USUAR, ' ', true)[0] . trim(strstr($membro->TXT_NOMEX_USUAR, ' ')[1]); ?>

                                        {{ $iniciais }}

                                        {{ $membro->TXT_NOMEX_USUAR }}
                                        {{ (($membro->pivot->FLG_IDENT_USUAR == 'L')? 'Líder' : (($membro->FLG_IDENT_USUAR == 'M')? 'Membro' : 'Visitante') ) }}

                                    COD_IDENT_USUAR}}">

                                        <button class="waves-effect"> Ver
                                        </button>

                            @endforeach

 Agenda de reunições

 Rélatorio

                        aaaaaa

Adicionar reunião

{!! (($igreja->FLG_MODEL_IGREJ == 'C')? 'A célula' : (($igreja->FLG_MODEL_IGREJ == 'P')? 'O pequeno grupo' : 'A sala dominical' )) !!} tem o endereço {{ (($celula->FLG_LOCAL_CELUL == 'I')? 'intinerante.' : 'fixo por este motivo será pre carregado o endereço para esta reunião')}}

            {!! Form::open(['route' => 'reunioes.store', 'class' => 'addEvent', 'role' => 'form']) !!}

                    Data
                    <input type="text" class="input-sm form-control" name="DAT_DATAX_REUNI" id="getStartDate" disabled="">

                    Hora

                        <input name="TMP_HORAX_REUNI" type='text' class="form-control time-picker">

                    Tema

                        <input type="text" class="input-sm form-control" id="eventName" name="TXT_TEMAX_REUNI">

                    Descrição

                        <input type="text" class="input-sm form-control" id="eventName" name="TXT_DESCR_REUNI">

                        <input type="radio" name="FLG_IDENT_ENDER" value="M">

                        Endereço de membros

                        <input type="radio" name="FLG_IDENT_ENDER" value="O">

                        Outro endereço

                                <select name="COD_USUAR_ENDER" class="form-control" placeholder="Escolha um membro">
                                    @foreach($celula->membros as $membro)
                                    <option value="{{ $membro->COD_IDENT_USUAR}}">{{ $membro->TXT_NOMEX_USUAR}}</option>
                                    @endforeach
                                </select>

                            Rua

                                <input type="text" class="input-sm form-control" id="eventName" name="TXT_RUAXX_REUNI" value="{{$celula->TXT_RUAXX_CELUL}}">

                            Bairro

                                <input type="text" class="input-sm form-control" id="eventName" name="TXT_BAIRR_REUNI" value="{{$celula->TXT_BAIRR_CELUL}}">

                            Cidade

                                <input type="text" class="input-sm form-control" id="eventName" name="TXT_CIDAD_REUNI" value="{{$celula->TXT_CIDAD_CELUL}}">

                            Estado

                                <input type="text" class="input-sm form-control" id="eventName" name="TXT_ESTAD_REUNI" value="{{$celula->TXT_ESTAD_CELUL}}">

                            Numero

                                <input type="text" class="input-sm form-control" id="eventName" name="TXT_NUMER_REUNI" value="{{$celula->TXT_NUMER_CELUL}}">

                            Complemento

                                <input type="text" class="input-sm form-control" id="eventName" name="TXT_COMPL_REUNI" value="{{$celula->TXT_COMPL_CELUL}}">

                <input type="hidden" id="getStart"/>
                <input type="hidden" id="getEnd"/>
                {!! Form::close()  !!}

                <button type="submit" class="btn btn-link" id="addEvent">Adicionar</button>
                <button type="submit" class="btn btn-primary btn-sm" id="editEvent">Editar</button>
                <button type="button" class="btn btn-link" data-dismiss="modal">Fechar</button>

O meu submit virá dentro do código:
     $("body").on("click", "#addEvent", function() {
        var eventName = $("#eventName").val();
        $("#calendar").fullCalendar("renderEvent", {
          title: eventName,
          start: $("#getStart").val(),
          end: $("#getEnd").val()
      }, true);

        $("#addNew-event form").submit();

        $("#addNew-event form")[0].reset();
        $("#addNew-event").modal("hide");
    });

Estou utilizando o fullcalendar jquery e o que preciso salvar é um evento, então faço duas formas, a primeira eu ploto no calendario meu evento e depois faço o submit dele para salvar no banco de dados (esta é a parte que falta), efetivar o save no controller é tranquilo, o problema é enviar isto e receber de volta a resposta.

Teria como fazer umas das duas opções ?

Por AJAX sem recarregar a página
Normal, voltando a tela para a tab AGENDA DE REUNIÕES
Comment: Pode colocar o codigo que tens na view sff, a parte relevante (form, mecanica das tabs...)
Comment: Ok irei editar tudo
Comment: @Miguel Atualizada a pergunta.
Comment: Ok obragado, já estou a pensar numa solução
Comment: Renan diz-me uma coisa, só vejo um form, queres que ao submeter o form volte a tab "AGENDA DE REUNIÕES" certo?
Comment: Sim, este form, é o que captura os dados.
Comment: Mas é só mesmo nesse caso é que queres que volte para tab "agenda de reuniões certo? Em mais nenhum caso é preciso voltar para uma tab em especifíco certo?
Comment: Ou ele volte para tab, ou ele nao recarregue a página, apesar que recarregar é melhor que pega os eventos atualizados.
Comment: Sim, so preciso voltar quando ele for submetido, os outros form eventuais que tiver na página será para sair dela de forma normal mesmo.
Answer: A minha sugestão é que no controller, depois de processados os dados enviados no controller, redireciones para a mesma página com uma flag:
$urlBack = url()->previous(); // buscar o url anterior, referer
return redirect($urlBack. '#abaAgenda'); // redirecionar para url e concatenar id da tab

Do lado cliente, via javascript fazes, depois da parte onde delegas o evento click nesta tab:
...
if(window.location.hash) {
    var tab = location.hash; // #abaAgenda
    $(tab).trigger('click');
}
...

Apoiado numa resposta minha há algum tempo


Microsoft Web Farm 2.0: how to get it to work with ASP MVC 3 which also has a redirector?

Question: The error I get is:
<code>The virtual path 'null' maps to another application, which is not allowed.
</code>
I created an MVC 3 project and posted it as-is.  Turn off the "URL Rewrite" and everything works fine.  Turn it back on and you get the error.
Any ideas?
Comment: On the same server?
Comment: I have two servers, one being a controller and a Primary, and the other being a Secondary.  So, it will be the same server 1/2 the time.  Scott Guthrie said you should be able to do this "DemoController doesn't technically need to be a separate machine – but makes it easier to understand the various roles in the walkthrough." http://weblogs.asp.net/scottgu/archive/2010/09/08/introducing-the-microsoft-web-farm-framework.aspx.
Answer: The key to making this work is have MVC on port 80 and have the inbound websites on a different port, so the Controller can rewrite them to port 80.  I sent port 80 web traffic on my Cisco to get to the web farm on port 591, then had the controller rewrite it to port 80 on the web farm.  Why?  This allows you to have a web server on the Controller as well.  Ultimately probably a bad idea.  LOL.


DHCP vs. Static IPs in an Static IP network with 40 clients and ~10 servers

Question: I am arguing with a colleague about DHCP. We are using static IPs at the moment and an inventory tool like Lansweeper to have the overview of our network. Besides we maintain a list of our clients and servers.
If we would setup a new network I would definitely say, that you should use DHCP, because of the configuration work. But in our case every client has it's own IP and there is no adress conflict.
So what would be the advantage to switch from Static IPs to DHCP?
We are using Windows Server 2016 and the OS supports DHCP-Failover so I think it is ok to rely on that. I also mean that there is a small benefit for changing the dns very fast, but how often will you do that?
We will independently from the DHCP topic have to go to any of the clients and configure another thing (sneaker administration FTW!).
Answer: To make my work easy in maintaining network inventory, I will bind MAC address to an IP address. This always gives same IP address when a particular server/Desktop/Laptop comes online after some time or maintenance period. Here we can always keep Interface in DHCP mode to receive Static IP address.
This way we can maintain Static ranges of IP addresses which is again maintained by DHCP.
Hope this solution helps..!!


Pandas Create new Column Based on Column's Value

Question: hope you can help me
i have a dataframe like this :

df = pd.DataFrame   (   {   'voucher_id': ['ugp_0008', 'ugp_0008', 'ugp_0008', 'ugp_0001', 'ugp_0009', 'ugp_0002', 'ugp_0003', 'ugp_0004', 'ugp_0005', 'ugp_0006', 'ugp_0007', 'ugp_0007'],
                        'status': ['REDEEMED', 'REDEEMED', 'REDEEMED', 'REDEEMED', 'AVAILABLE', 'AVAILABLE', 'REDEEMED', 'EXPIRED', 'EXPIRED', 'REDEEMED', 'REDEEMED', 'REDEEMED'],
                        'redeem_type': ['OTHERS', 'VOUCHER', 'VOUCHER', 'BILLER', 'exp_or_avail', 'exp_or_avail', 'VOUCHER', 'exp_or_avail', 'exp_or_avail', 'VOUCHER', 'OTHERS', 'DIRECT-MERCHANT'],
                        'nominal': [200000, 200000, 200000, 100000, 100000, 100000, 500000, 100000, 100000, 100000, 100000, 100000],
                        'value_used': [10000, 30000, 150000, 20000, 0, 0, 500000, 0, 0, 20000, 20000, 80000],
                    }
                )

and the expected result is :

df_expect = pd.DataFrame   (   {   'voucher_id': ['ugp_0008', 'ugp_0001', 'ugp_0009', 'ugp_0002', 'ugp_0003', 'ugp_0004', 'ugp_0005', 'ugp_0006', 'ugp_0007'],
                        'status': ['REDEEMED', 'REDEEMED', 'AVAILABLE', 'AVAILABLE', 'REDEEMED', 'EXPIRED', 'EXPIRED', 'REDEEMED', 'REDEEMED'],
                        'nominal': [200000, 100000, 100000, 100000, 500000, 100000, 100000, 100000, 100000],
                        'VOUCHER' : [180000, 0,0,0, 5000000,0,0,20000,0],
                        'BILLER' : [0,20000,0,0,0,0,0,0,0],
                        'OTHERS' : [10000,0,0,0,0,0,0,0,20000],
                        'DIRECT-MERCHANT' : [0,0,0,0,0,0,0,0,80000]
                        ,'remaining_nominal' : [10000, 80000, 100000, 100000, 0, 100000, 100000, 80000, 0]
                        }
                )

im expect voucher_id grouped, and redeem_type value become new columns based on condition
i have try using np.where
df2 = df.copy()
df2['BILLER'] = np.where(df['redeem_type'] ==
'BILLER', df['value_used'], 0)

df2['VOUCHER'] = np.where(df['redeem_type']
== 'VOUCHER', df['value_used'], 0)

df2['OTHERS'] = np.where(df['redeem_type'] ==
'OTHERS', df['value_used'], 0)

df2['DIRECT-MERCHANT'] =
np.where(df['redeem_type'] == 'DIRECT-
MERCHANT', df['value_used'], 0)

df2['exp_or_avail'] =
np.where(df['redeem_type'] == 'exp_or_avail',
df['value_used'], 0)

and i got like this

And then i using pivot :
ss=df.groupby(['voucher_id','redeem_type']).sum().reset_index()
ss.pivot(index='voucher_id',columns='redeem_type',values='value_used').fillna(0)

and i got same result (voucher_id not grouped)

hope you guys can help me, thank you! have a great day for everyone
Answer: Aggregate the dataframe to calculate sum of nominal values per voucher and status
keys = ['voucher_id', 'status']
nominal = df.groupby(keys)['nominal'].sum()

# voucher_id  status
# ugp_0001    REDEEMED     100000
# ugp_0002    AVAILABLE    100000
# ugp_0003    REDEEMED     500000
# ugp_0004    EXPIRED      100000
# ugp_0005    EXPIRED      100000
# ugp_0006    REDEEMED     100000
# ugp_0007    REDEEMED     200000
# ugp_0008    REDEEMED     600000
# ugp_0009    AVAILABLE    100000
# Name: nominal, dtype: int64

Pivot the dataframe to calculate sum of value_used per voucher, status and redeem_type
df1 = df.pivot_table(index=keys, columns='redeem_type', values='value_used', aggfunc='sum', fill_value=0)

# redeem_type           BILLER  DIRECT-MERCHANT  OTHERS  VOUCHER  exp_or_avail
# voucher_id status
# ugp_0001   REDEEMED    20000                0       0        0             0
# ugp_0002   AVAILABLE       0                0       0        0             0
# ugp_0003   REDEEMED        0                0       0   500000             0
# ugp_0004   EXPIRED         0                0       0        0             0
# ugp_0005   EXPIRED         0                0       0        0             0
# ugp_0006   REDEEMED        0                0       0    20000             0
# ugp_0007   REDEEMED        0            80000   20000        0             0
# ugp_0008   REDEEMED        0                0   10000   180000             0
# ugp_0009   AVAILABLE       0                0       0        0             0

Subtract sum of all value_used along each row from nominal to calculate remaining
df1 = df1.assign(nominal=nominal, remaining=nominal - df1.sum(axis=1))

# redeem_type           BILLER  DIRECT-MERCHANT  OTHERS  VOUCHER  exp_or_avail  nominal  remaining
# voucher_id status
# ugp_0001   REDEEMED    20000                0       0        0             0   100000      80000
# ugp_0002   AVAILABLE       0                0       0        0             0   100000     100000
# ugp_0003   REDEEMED        0                0       0   500000             0   500000          0
# ugp_0004   EXPIRED         0                0       0        0             0   100000     100000
# ugp_0005   EXPIRED         0                0       0        0             0   100000     100000
# ugp_0006   REDEEMED        0                0       0    20000             0   100000      80000
# ugp_0007   REDEEMED        0            80000   20000        0             0   200000     100000
# ugp_0008   REDEEMED        0                0   10000   180000             0   600000     410000
# ugp_0009   AVAILABLE       0                0       0        0             0   100000     100000

Comment: can you help me make the voucher_id not as index ? so i can search particular voucher_id
Answer: Code
problem that can be solved with pivot_table.
cols = ['voucher_id', 'status', 'nominal']
df.pivot_table('value_used', index=cols, columns='redeem_type', fill_value=0)\
  .reset_index()\
  .assign(remain_nominal=lambda x: x['nominal'].sub(x[df['redeem_type'].unique()].sum(axis=1)))

output:


Intent.action.VIEW does not work on some devices

Question: I need to open my application when clicked the URL Link in E-mail, Message or Browser applications. So, see the my code for this feature:
<activity
    android:name=".ui.main.MainActivity"
    android:configChanges="orientation"
    android:screenOrientation="portrait">
    <intent-filter>
        <action android:name="android.intent.action.MAIN" />
        <category android:name="android.intent.category.LAUNCHER" />
    </intent-filter>

    <intent-filter>
        <data android:scheme="https" />
        <data android:scheme="https" android:host="www.mysite.org"/>
        <action android:name="android.intent.action.VIEW"/>
        <category android:name="android.intent.category.BROWSABLE"/>
        <category android:name="android.intent.category.APP_EMAIL"/>
        <category android:name="android.intent.category.APP_MESSAGING"/>
        <category android:name="android.intent.category.APP_BROWSER"/>
        <category android:name="android.intent.category.DEFAULT"/>
    </intent-filter>
</activity>

This code is sufficient and works on some mobile devices, especially devices with android versions 5.0, 6.0 and 7.0 but does not work on versions 7.1 and 8.0. Why does this happen? Is it a permission problem?
Comment: How does your `Intent` look like?
Comment: @MuratK. I need to open my app when the user clicked a link (mysite.org) from gmail.
Answer: Ex: Your url will be something like https://example.com and you have the intent filter in Android Manifest as below:
<activity
    android:name="com.droid.MainActivity"
    android:label="@string/app_name" >

    <intent-filter>
        <action android:name="android.intent.action.VIEW" />
        <category android:name="android.intent.category.DEFAULT" />
        <category android:name="android.intent.category.BROWSABLE" />
        <data
            android:host="example.com"
            android:scheme="https" />
    </intent-filter>
</activity>

Use the URL Mapping editor to easily add URL intent filters to your Activities with Android App Links (Android Studio > Tools > Android App Links).
Answer: After reading changes in Android O I came to the conclusion that the problem is described as follows

URIs cannot contain empty labels. Previously, the platform supported a
  workaround to accept empty labels in host names, which is an illegal
  use of URIs. This workaround was for compatibility with older libcore
  releases. Developers using the API incorrectly would see an ADB
  message: "URI example..com has empty labels in the hostname. This is
  malformed and will not be accepted in future Android releases."
  Android 8.0 removes this workaround; the system returns null for
  malformed URIs.

So try adding android:label="string resource" to your intent-filter  syntax so  the user will see it in the alternatives menu.
Comment: So thr case WAS actually the missing label! Just for people to know if they look for it in future


SQL Count Error

Question: I am trying to use a count function to show me how many employees have the title "Sales"
if( !$connect)
{
die("ERROR: Cannot connect to database $db on server $server
using user name $user (".mysqli_connect_errno().
", ".mysqli_connect_error().")");
}
else
{
$userQuery = "COUNT(empID) FROM personnel WHERE jobTitle='Sales'";

$result = mysqli_query($connect, $userQuery);

if (!$result)
{
    die("Could not successfully run query ($userQuery) from $db: " .
mysqli_error($connect) );
}

if (mysqli_num_rows($result) == 0)
{
    print("No records found with query $userQuery");
}
else
{
     print("
SALES STAFF REPORT
");
     while ($row = mysqli_fetch_assoc($result))
    {
        print("
There are
        ".$row['COUNT(empID)']."
");
    }

}
 mysqli_close($connect);   // close the connection
}
?>

I am getting this error:

You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'COUNT(empID) FROM personnel WHERE jobTitle='Sales'' at line 1

on this line:
$userQuery = "COUNT(empID) FROM personnel WHERE jobTitle='Sales'";

I am unsure of what I am doing incorrectly, but any help would be appreciated.
End result should read "There are 4 sales staff".
My professor did include this in the assignment, not sure if that will help.
"There are two different MySQL functions that you can use here: you can
modify the SELECT statement to use a MySQL aggregation function,
or you can use the MySQL function that returns the number of records in the
result set. "

Comment: You are missing the `SELECT` statement, it should be `SELECT COUNT(empID) FROM personnel WHERE jobTitle='Sales'`.
Comment: I don't see a SELECT in your SELECT statement
Comment: you should $userQuery = "SELECT COUNT(empID) FROM personnel WHERE jobTitle='Sales'";
Comment: And use PDO, prepared statements
Comment: Thank you for the help! On the guide my professor provided going over the aggregate operations, she did not include select in front of it so I did not think it was needed.
Answer: You are missing the SELECT keyword when making the statement.
You have:
$userQuery = "COUNT(empID) FROM personnel WHERE jobTitle='Sales'";
It should be:
$userQuery = "SELECT COUNT(empID) FROM personnel WHERE jobTitle='Sales'";

SQL Count Error


Best practice for Post-Double Selection LASSO (pdslasso)

Question: I'd like to have a clearer idea of the optimal approach to the post-double selection LASSO (paper, webpage). Take data on an RCT with 2 treatment arm dummies $D_1, D_2$ and a potential driver of heterogeneous treatment effects $Z$.
One possibility is to run the PDS lasso on our outcome variable $Y$ and the pooled treatment dummy $D$ and subsequently use the chosen variables in all other regressions, also the ones with potentially different specifications, such as heterogeneous effects wrt $Z$.
On the other hand, if we run PDS lasso for each different specification:

Imagine I want to test the coefficient of the pooled treatment $D$ against the coefficient on treatment 2 $D_2$. Should I run the pds lasso using $D$ as exogenous first to get the coefficient on $D$, and then another pds lasso using $D_1, D_2$ as exogenous to get the coefficient on $D_2$, and then run the test? I feel this is a bit strange, since we're using potentially different controls in each of the regressions instead of testing the difference based on the exact same specification.

Imagine I want to run a regression with heterogeneous treatment effects, such as

$$ Y_i =  \beta^\prime X_i +  \beta_Z Z_i +  \rho_{D} D_i +  \rho_{D\cdot Z} D_i \cdot Z_i + u_i$$
Should I also have $D_i \cdot Z_i$ as an exogenous variable to be used in the first step of the variable selection? Also, if I don't use it as an exogenous variable, the standard errors of this coefficient will not be valid in the PDS lasso output. Would I then need to reestimate it in an OLS with the selected variables?

It's preferable to add a small sample correction to the standard errors obtained in pds lasso

I feel that the first option, with one single pds lasso selection of controls for each outcome variable, which then also selects controls for any additional specification we might want to try, seems to make more sense, creating a comparable framework throughout the analysis. Am I missing something?
Answer: Let me first briefly summarize the setting: We have a scalar treatment variable $D_i$, a grouping variable $Z_i$ (driver of heterogeneity) and high-dimensional controls $X_i$. $X_i$ can be high-dimensional (i.e. many controls relative to the sample size).
If we ignore treatment effect heterogeneity, our model is simply:
$ Y_i = \alpha D_i + X_i'\beta + \epsilon_i $
The model has two parts: a low-dimensional part ($D_i$) and a high-dimensional part comprising all the controls. The aim of the analysis is to estimate the treatment effect $\alpha$ -- we don't really care about the $\beta$ parameter. On the other hand, ignoring $X$ would lead to ommitted variable bias.
The Post Double Selection Lasso approach involves two auxiliary Lasso regressions: $Y$ against $X$, and $D$ against $X$. The union of selected controls gives us our full set of controls, which we will use in the final OLS regression. You can obtain asymptotically valid standard errors for the treatment effect. (This is not so easy for the high-dimensional parameters.)
To your question, which I summarize as How can we accommodate a grouping variable $Z$?
For simplicity, say we have only two groups (male/female) and $Z_i$ is dummy for female. Our model becomes: $ Y_i = \alpha D_i + \alpha_F (D_i Z_i) + X_i'\beta + \epsilon_i $.
Our low-dimensional part now includes two variables. That's perfectly fine, as long as our low-dimensional part doesn't get "too" large relative to the sample size. The PDS algorithm now has three auxiliary Lasso regressions: $Y\rightarrow X$, $D\rightarrow X$, $(DZ)\rightarrow X$. Again, our final OLS regression includes the union of controls. The pdslasso package in Stata allows for multiple treatment/low dimensional variables. So not much to worry about.
Additional comments:

As you say, an alternative, valid approach would be to estimate your model on sub-samples of your data (one estimation for female, one for male). That's more flexible, but also more costly.
One rationale for using Lasso approaches is to allow for non-linear effects. So, depending on the dimension of $X$, I would highly recommend to interact your controls to capture interactions. Also consider higher-level polynomials, splines etc.
Related to the two previous points: If you go for the full sample approach, you should also consider interacting your controls with $Z$. You assume that the treatment affect varies with $Z$. Hence, it also seems plausible that the role of $X$ varies with $Z$.
An alternative valid approach to PDS-Lasso relies on orthogonalization. You would run the same auxiliary Lasso regression, but use the residuals in the final OLS regression. (This is also implemented in pdslasso and referred to as "CHS" (due to Chernozhukov, Hansen, Spindler 2015).) Check the pdslasso help file for more information.
You seem to conflate "exogeneity" and "low vs high-dimensionality". This is not the same.
Addendum: If you have two treatments ($D_1$ and $D_2$) nothing changes. Again, the main constraint is that the low-dimensional part has to be finite and small relative to the sample size.

References:

Chernozhukov, V. Hansen, C., and Spindler, M. 2015.  Post-selection
and post-regularization inference in linear models with many controls
and instruments.  American Economic Review: Papers & Proceedings
105(5):486-490. http://www.aeaweb.org/articles.php?doi=10.1257/aer.p20151022
Belloni, A., Chernozhukov, V. and Hansen, C. 2014.  Inference on
treatment effects after selection among high-dimensional controls.
Review of Economic Studies 81:608-650.
https://doi.org/10.1093/restud/rdt044
Belloni, A., Chernozhukov, V. and Hansen, C. 2015.
High-dimensional methods and inference on structural and treatment
effects.  Journal of Economic Perspectives 28(2):29-50.
http://www.aeaweb.org/articles.php?doi=10.1257/jep.28.2.29
Belloni, A., Chernozhukov, V., Hansen, C. and Kozbur, D. 2016.
Inference in High Dimensional Panel Models with an Application to Gun
Control.  Journal of Business and Economic Statistics 34(4):590-605.
http://amstat.tandfonline.com/doi/full/10.1080/07350015.2015.1102733
Comment: Thanks a lot! An additional point I just want to be sure about: in your example model, we have $Y_i = \alpha D_i + \alpha_F (D_i Z_i) + X^\prime_i \beta + \epsilon_i$. Given the heterogeneity term, we'd certainly want to add the control on $Z_i$ in this specification. Thus, $Z_i$ (without the interaction with $D_i$) would be added as a non-penalized regressor, such as pnotpen(Z_i), right?
Comment: Thank you very much for your answer, @aahr1! If possible, I'd like to ask further on 2 points:

1. How would you say the PDSLASSO approach is connected with [bad controls](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3689437)? If we allow the algorithm to select the controls unilaterally, we're opening paths for bad controls, right? Since different variables will have different potential bad controls, would you say that one way to go would be to pre-select a (dependent) variable-specific list of potential controls for the PDSLASSO? Or is there some other best practice regarding this?
Comment: 2. One focus of my question is the comparability across specs. If we have two grouping variables $Z^a_,Z^b$ with some connection (ie, they are 2 indexes of the same characteristic), and want to show the results of the spec above, we would have the PDSLASSO procedure on both, potentially choosing different controls for each. Wouldn't it be desirable to have the same controls in both regressions, making their coefficients comparable? Maybe if they indeed represent the same underlying characteristic, the PDSLASSO would still select roughly the same variables anyway, so this is a non-issue?
Comment: Regarding #1: Yes, bad controls are a risk. You shouldn't include them in your high-dimensional list of controls.
Comment: Regarding #2: I don't understand why this would be desirable. $D_iZ^a_i$ and $D_iZ^b_i$ might be affected by different confounders. -- Keep in mind that you only use the lasso to approximate an unknown nuisance function. There is no guarantee that the lasso identifies the "true" model. That's also why you shouldn't interpret (too closely) what the lasso selects and that's also why you don't need that comparability.


Running script with ASP.NET MVC Project

Question: I am creating a project in MVC that involves using AJAX to grab XML from an external url. The issue is that I can't do something like this:
$.ajax({
  url: url,
  data: data,
  success: success,
  dataType: dataType
});

because the site I'm accessing for the xml is public and I get a XMLHttpRequest same domain policy error. My solution is to run a server side script (like PHP?) to grab the XML info and then forward it to be used by my program (a Google Maps project with marker data that updates every 15 seconds).
My question is, how would I incorporate the script to get the XML to my project? Can I use PHP? ASPX? I've never done anything quiet like this, so I just don't have a clue on how it would all fit together.
For clarification this is what I was doing:
public class HomeModel
{
    public string XmlData { get; set; }

    public void GetXml()
    {
        XmlDocument doc1 = new XmlDocument();
        doc1.Load("http://www.nfl.com/liveupdate/scorestrip/ss.xml");
        XmlData = doc1.InnerXml;
    }
}

But this won't work because I want to use AJAX to grab the XML data so that I don't have to do a full page refresh.
Comment: If you're already using ASP.Net in your project, why not just use that to create the proxy?
Comment: I updated the OP to include what I am currently doing, but it isn't what I want because I want to update the data "in the background" (via AJAX) instead of a full page refresh.
Answer: Well I guess the issue here is more of an ethical one. The owner of the site you are trying access the data from doesn't want you to directly access that data and re-render to another client. Hence the same domain policy error.
You could still download the file on your server side. I.E. in C# you could user HttpClient.
HttpClient client = new HttpClient();
string file = await client.GetStringAsync('http://www.example.com/myfile.xml');

However if you actually controller the server where the xml file is being served from you should just enable cors (Cross-origin resource sharing) of the xml file on your server.
EDIT
Based on your edit you want to create an MVC Action that calls your code. You could then make your ajax call on your own server side code, thus negating the cors issue.
The url www.mysite.com/MyXML would now return the file from http://www.nfl.com/liveupdate/scorestrip/ss.xml.
public class MyXMLController : Controller
{
    public ActionResult Index()
    {
        XmlDocument doc1 = new XmlDocument();
        doc1.Load("http://www.nfl.com/liveupdate/scorestrip/ss.xml");
        XmlData = doc1.InnerXml;
        return Content(doc1.InnerXml, "text/xml", System.Text.Encoding.UTF8);
    }
}

NOTE: Ethically I do not agree with this approach.
Comment: That will allow me to update marker info every X seconds without having to refresh the entire page? I guess I'm confused on how AJAX would fit in with this.
Comment: You would just do the AJAX request that you would have made to `http://www.nfl.com/liveupdate/scorestrip/ss.xml` but instead you would make the request to `www.mysite.com/MyXML`. Your server side code then requests the file and returns it to your client.
Answer: Assuming I'm getting this right. You want to do a 15 mins client side poll of a service that will get you an XML? and the client you are requesting data from is complaining about cross site scripting?
You could potentially do an ajax call to one of your own endpoint on the same application and in there you do a http call:
var myHttpClient = new HttpClient();
var response = await myHttpClient.PostAsync(uri, Content)

You do need to build your content. There are multiple ways of doing it depending on what it is (form, string, binary, json).
Front End ===(ajax)===> MVC endpoint ===(httpclient/webclient)===> Third party
Front End <===(response)=== MVC endpoint <===(response)=== Third party
Comment: I want to grab XML from a site every 15 seconds and so that I can use it for my Google Maps project. The entire point is to update map marker data without having to do a full page refresh, therefore AJAX seems to be the way to go. A coworker explained that the only way to achieve this is to run a server side script to get the XML, then "forward" it to my application. I'm not sure if your solution would allow for that?
Comment: Yes, your colleague is right. But if you can't do an ajax call to a different site because the other site is refusing to, then you can do an ajax call to an endpoint on your site (same domain). What that enpoint (on your site) does is pretty much a call to the url of the third party and return is as a response.

Running script with ASP.NET MVC Project


10GE network: Is it still deadly expensive? Any options?

Question: I am building home cluster where I going to have about 16 nodes which can live with 1G ports, but I want to have 10GE on file server & central node. It's all local, so no need for cables longer than 3-5m. And of course I want to spend as little money as possible (not going to spend more than whole cluster costs) :-)
What are my options?

Legacy solution is to take some 24-48 port 1GE switch, and connect to file/central nodes via 4-8 aggregated links. This will work I guess, cost is very acceptable, but I am not sure if it's ok to use that much aggregated links. And of course it would be hard to double bandwidth when needed... :-D
Switch with several 10GE uplink 'ports'. As far as I see, they all require modules which costs about 1000$, so I will need 4 10G modules, and 2 10GE cards... Smells like way more than 5000$+...
Connect file & central node via 2 10G cards directly, and put 4 quad-port 1GE NICs on fileserver. I am saving on 2 10G modules and a switch, fileserver will have to do packet routing, but it's still gonna have a lot of CPU's left :-)
Any other options? Infiniband?
Are MyriNet adaptors works fine? I guess there are no cheaper options?
Hmm... Scrap fileserver, put it all on central node and provide dedicated 1GE port for each of the nodes... This is sad...
Answer: I don't think you're not seeing the entire picture here.
You are wanting to connect a file server at 10Gbps speed, which may sound like a sexy idea. The thing you are not seeing is the ability of that server to generate that amount of traffic reading from disks. Getting 1GBps from a file server is, today, a very good achievement. 10Gbps will not only be expensive as you have realized yourself but at minimum 90% useless.
Your best option is to start putting in some blazing fast disks in your file servers if it needs to provide such great amounts of IOs. I strongly believe the "affordable" (notice the quotes) path to this is SSD drives in fast RAID configurations (that is RAID10).
As for networking, a 4x1Gbps agregate will do the trick fine and you can even add more later. Watch out for the fact that internal buses (read PCI*) are not always capable of handling multi-gigabit speeds. This is especially true if you are not using server-grade motherboards.
I believe this is your only "affordable" option here. Infiniband cards are not horribly expensive. I believe you can find some for ~150$ but the switch will be very expensive.
Comment: Have to agree with this. We have a 10Gb infrastructure at work, based on copper SFP+ connections. This was expensive enough, but the real expense was the server infrastructure that would actually use it.
Comment: 10Gbps will most likely mean less latency but I don't believe the improvement will really be meaningful. You're transmitting data at higher rates so it should get there faster. What file sharing protocol are you using to obtain these numbers or is it just raw local performance ?
Comment: this is for user "BarsMonster": could you pl explain your home fileserver? hardware? OS? filesystem etc
Comment: Well, even my home fileserver built from junk can read 400Mb/s sequentially. I can easily make large seqientical storage of HDD able to do 1Gb/s and tiny random storage of SSD able to do 1Gb/s (600Mb/s from onboard controller+few PCI-E raid cards). I am more interested in bandwidth between central node & 'small' nodes, as traffic will be between pieces of software - no limited by disks. So both bandwidth and latency matters. Does 10GE also means less latency?
Comment: Yes, surely I mean raw local performance. I don't have anything faster than 1GE yet.
Comment: You lose a whole lot through network file sharing protocols.
Answer: you may want to consider ATA over Ethernet, if you want to save on some expensive layer-3 switches. this is the protocol of chioce for low cost but high perf solution than any other filesystem i know of today.
But there are no 10Gb vanilla switches (without L3 switching).
consider as a POC : Ubuntu server 10.x and AOE tools project
http://sourceforge.net/projects/aoetools/files
https://help.ubuntu.com/community/ATAOverEthernet
Comment: a switch without L2 switching is not a switch.
Comment: okay, got that, my answer is corrected now. thanks


How to show upper/lower-hemicontinuity of a correspondence - with non-hand-wavy arguments?

Question: I understand (at least I think I understand) the notion behind these topics.
In my class we use the following criterion:
A correspondence $\Gamma: X \rightrightarrows Y$ is LHC at $x$ if whenever $x$ is in the lower inverse of an open set $E \subset Y$, there is an open set $F$ containing $x$ that satisfies $F \subset \Gamma^l[E]$
and
A correspondence $\Gamma: X \rightrightarrows Y$ is UHC at $x$ if whenever $x$ is in the upper-inverse of an open set $E \subset Y$, then there is an open set $F$ containing $X$ such that $F \subset \Gamma^U[E]$
Where the upper inverse and lower inverse are defined as:
$\Gamma^U[E]$ is the $\textbf{Upper inverse}$ of $E$, defined as:
$$\Gamma^U[E] = \{ x \in X | \Gamma(x) \subset E\}$$
and
$\textbf{The lower inverse}$ of $E \subset Y$, denoted by $\Gamma^l[E]$ is the set:
$$\Gamma^l[E] = \{ x \in X | \Gamma(x) \cap E \neq 0\} $$
But I can never score full points on a question, and can't seem to find any rigorous proofs, only pictures and one sentence arguments.
For instance, consider the two correspondences:
\begin{equation*}
\Gamma(x) = \begin{cases}
             [0, \frac{1}{x}]  & \text{if } x > 0 \\
             \{0\}  & \text{if } x = 0
       \end{cases}
\end{equation*}
Or: $\phi: \mathbb{R}_+ \rightrightarrows \mathbb{R}$ defined as:
$$\phi(x) = \{x^{0.5}, -x^{0.5}\} $$
Yes, looking at the graph I can immediately tell that $\Gamma(x)$ is not UHC at $x=0$, and is LHC at $x=0$, and $\phi(x)$ is UHC LHC everywhere, but I just don't know how to rigorously prove it. How would you go about the proof, using the above definitions, to a blind person who had a text-to-voice translator (but obviously couldn't see the graph)?
Comment: Are you comfortable writing $\epsilon$-$\delta$ proofs for ordinary continuity?  The process here is very similar.
Comment: Thanks - I have fixed it. Yes, I can do $\delta \epsilon$ proofs for continuity, but I feel I have never seen such a challenge-response proof for UHC/LHC, so I just don't know how to do it. I guess this is because the 'closeness' or distance between points and their correspondences has always been defined in terms of open sets and pre-images for the proofs we are given.
Answer: Well, you just go through the definitions step by step.  Let's prove $\Gamma$ is not UHC at $x=0$.  Negating the definition of UHC, this means we must find an open set $E$ such that $0\in \Gamma^U(E)$, but there is no open set $F$ such that $0\in F$ and $F\subseteq\Gamma^U(E)$.  In order to get $0\in\Gamma^U(E)$, we need $\Gamma(0)=\{0\}$ to be a subset of $E$, so let's just pick $E=(-1,1)$, say.
Now we need to prove this choice of $E$ works, so let $F\subseteq\mathbb{R}_+$ be any open set containing $0$.  We wish to show $F\not\subseteq\Gamma^U(E)$, so we want to find an element $x\in F$ that is not in $\Gamma^U(E)$, in other words such that $\Gamma(x)\not\subseteq (-1,1)$.  Looking back at our definition of $\Gamma$, this just means we need $x\neq 0$ and $1/x\geq 1$, or $x\leq 1$.  But since $F$ is open and contains $0$, it contains an interval $[0,\epsilon]$ for some $\epsilon>0$, and then $x=\min(1,\epsilon)$ is an element of $F$ that works.
Let's now show $\Gamma$ is LHC at $0$.  By the definition, this means we start with an arbitrary open set $E$ such that $0\in \Gamma^l(E)$, and wish to find an open set $F$ containing $0$ such that $F\subseteq\Gamma^l(E)$.  What does $0\in\Gamma^l(E)$ tell us?  It means that $\Gamma(0)\cap E\neq \emptyset$, or just that $0\in E$ since $\Gamma(0)=\{0\}$.
What does the desired condition $F\subseteq\Gamma^l(E)$ mean?  It means for each $x\in F$, $\Gamma(x)\cap E\neq\emptyset$.  Aha, but we know that $0\in E$, and looking back at our definition of $\Gamma$, we see that actually $0\in \Gamma(x)$ for all $x$!  So $\Gamma(x)\cap E\neq \emptyset$ is true for all $x$, and so we can take $F$ to be any open set containing $0$ at all.
Comment: Thank you, this helps a lot!


Git - problema de quebra de linha entre diferentes Sistemas Operacionais

Question: Estou tentando corrigir um problema relativo ao Git, é um problema de quebra de linha em arquivos que foram enviados de um sistema operacional baseado em DOS e posteriormente alterado em um SO baseado em UNIX.
O que acontece: quando eu altero 'n' linhas em meu SO Linux, por exemplo, o arquivo: "models.py", e executo o comando git diff, aparece apenas uma linha alterada.
Esse problema é devido a quebra de linha dos SOs serem diferentes...
Alguém já passou por esse problema e conseguiu resolver? Alguma dica?
Comment: Veja se te ajuda: http://pt.stackoverflow.com/q/44373/101
Answer: Os sistemas reconhecem a quebra de linha de formas diferentes. Você pode definir um padrão no próprio arquivo nas configurações, a quebra de linha dos sistemas podem estar distintos.


Get thumbail / preview image from a server video URL in Swift 3.0

Question: I want a thumbnail image from a video underlying on the server. The video file is not on local. It's on my server. The video file has extension .m3u8.
Answer: You can do it.
First step: You need to import AVFoundation:
    import AVFoundation

Then add the code below to your controller:
func getThumbnailImage(forUrl url: URL) -> UIImage? {
    let asset: AVAsset = AVAsset(url: url)
    let imageGenerator = AVAssetImageGenerator(asset: asset)

    do {
        let thumbnailImage = try imageGenerator.copyCGImage(at: CMTimeMake(value: 1, timescale: 60), actualTime: nil)
        return UIImage(cgImage: thumbnailImage)
    } catch let error {
        print(error)
    }

    return nil
}

Usage:
    let imageView = UIImageView()
    let url = URL(string: "your_video_url")

    if let thumbnailImage = getThumbnailImage(forUrl: url) {
        imageView.image = thumbnailImage
    }

Change url to your video link.
Comment: its not working for me..
this is the error i'm getting
Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSUnderlyingError=0x608000240870 {Error Domain=NSOSStatusErrorDomain Code=-12782 "(null)"}, NSLocalizedFailureReason=An unknown error occurred (-12782), NSLocalizedDescription=The operation could not be completed}
Comment: Note: if your image turns out rotated, try `imageGenerator.appliesPreferredTrackTransform = true`.
Comment: @MANCHIKANTIKRISHNAKISHORE im getting that error as well
Comment: This does work for firebase url. But it works for other URL
Answer: Step 1
import AVFoundation

Step 2
Add the following function in your ViewController:
func getThumbnailImageFromVideoUrl(url: URL, completion: @escaping ((_ image: UIImage?)->Void)) {
    DispatchQueue.global().async { //1
        let asset = AVAsset(url: url) //2
        let avAssetImageGenerator = AVAssetImageGenerator(asset: asset) //3
        avAssetImageGenerator.appliesPreferredTrackTransform = true //4
        let thumnailTime = CMTimeMake(value: 2, timescale: 1) //5
        do {
            let cgThumbImage = try avAssetImageGenerator.copyCGImage(at: thumnailTime, actualTime: nil) //6
            let thumbNailImage = UIImage(cgImage: cgThumbImage) //7
            DispatchQueue.main.async { //8
                completion(thumbNailImage) //9
            }
        } catch {
            print(error.localizedDescription) //10
            DispatchQueue.main.async {
                completion(nil) //11
            }
        }
    }
}

Step 3
self.getThumbnailImageFromVideoUrl(url: videoUrl) { (thumbNailImage) in
    self.yourImageView.image = thumbNailImage
}

If you need a complete explanation, refer to Get a thumbnail from a video URL in the background in Swift.
Author: https://www.swiftdevcenter.com
Answer: In Swift 5.1 (older versions too), you can do it like this:
private func createVideoThumbnail(from url: URL) -> UIImage? {

    let asset = AVAsset(url: url)
    let assetImgGenerate = AVAssetImageGenerator(asset: asset)
    assetImgGenerate.appliesPreferredTrackTransform = true
    assetImgGenerate.maximumSize = CGSize(width: frame.width, height: frame.height)

    let time = CMTimeMakeWithSeconds(0.0, preferredTimescale: 600)
    do {
        let img = try assetImgGenerate.copyCGImage(at: time, actualTime: nil)
        let thumbnail = UIImage(cgImage: img)
        return thumbnail
    }
    catch {
      print(error.localizedDescription)
      return nil
    }

}

Note that AVKit needs to be imported.
Answer: You can use Kingfisher to generate the thumbnail.
Note:- It will handle all the cache work for you.
Usage:-
Step 1
import Kingfisher

Step 2
guard let url = URL(string: videoUrlString) else { return }
self.imageView.kf.setImage(with: AVAssetImageDataProvider(assetURL: url, seconds: 1))

Here is the link of Kingfisher SDK

https://github.com/onevcat/Kingfisher

Note:- It will work fine with latest version of kingfisher/ higher or equals to version 7.2.3
Answer: In this solution, I have created a cache as well for the image so that we don't need to fetch the same images again from the internet.
 func createVideoThumbnail( url: String?,  completion: @escaping ((_ image: UIImage?)->Void)) {

    guard let url = URL(string: url ?? "") else { return }
    DispatchQueue.global().async {

        let url = url as URL
        let request = URLRequest(url: url)
        let cache = URLCache.shared

        if
            let cachedResponse = cache.cachedResponse(for: request),
            let image = UIImage(data: cachedResponse.data)
        {
            DispatchQueue.main.async {
                completion(image)
            }
        }

        let asset = AVAsset(url: url)
        let imageGenerator = AVAssetImageGenerator(asset: asset)
        imageGenerator.appliesPreferredTrackTransform = true

        var time = asset.duration
        time.value = min(time.value, 2)

        var image: UIImage?

        do {
            let cgImage = try imageGenerator.copyCGImage(at: time, actualTime: nil)
            image = UIImage(cgImage: cgImage)
        } catch { DispatchQueue.main.async {
            completion(nil)
        } }

        if
            let image = image,
            let data = image.pngData(),
            let response = HTTPURLResponse(url: url, statusCode: 200, httpVersion: nil, headerFields: nil)
        {
            let cachedResponse = CachedURLResponse(response: response, data: data)

            cache.storeCachedResponse(cachedResponse, for: request)
        }

        DispatchQueue.main.async {
            completion(image)
        }

    }

}

Usage:
      createVideoThumbnail(url: data.url ?? "") { [weak self] (img) in
            guard let strongSelf = self else { return }
            if let image = img {
                strongSelf.mediaImg.image = image
            }
        }


Atom PHP lint not working Windows 10

Question: I have fresh installation of Atom. Installed linter package and then linter-php.
In config.cson file i have following
"*":
  "exception-reporting":
    userId: "c545e431-a953-b271-c123-c021c950953b"
  welcome:
    showOnStartup: false
  core: {}
  editor:
    invisibles: {}
    tabLength: 4
  linter: {}
  "linter-php":
    executablePath: "C:/php/"

I downloaded PHP zip from http://windows.php.net/downloads/releases/php-5.6.16-Win32-VC11-x86.zip and extracted inside C:/php
Can any one help me out in fixing this. I don't find any thing i am missing.
Update
This is the error i am getting
Error: 'C:/php/' is not recognized as an internal or external command,
operable program or batch file.
    at parameters.exit (C:\Users\Raheel\.atom\packages\linter-php\node_modules\atom-linter\lib\helpers.js:70:20)
    at triggerExitCallback (C:\Users\Raheel\AppData\Local\atom\app-1.2.4\resources\app.asar\src\buffered-process.js:213:47)
    at ChildProcess.<anonymous> (C:\Users\Raheel\AppData\Local\atom\app-1.2.4\resources\app.asar\src\buffered-process.js:235:18)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

Comment: Reading the error message, what about if you put `executablePath: "C:/php/php.exe"`, or `executablePath: "C:\\php\\php.exe"` ? Sorry if i'm wrong, I I'm not usual with the tools you're talking about, but the error message is quite self-explanatory
Comment: you need the Visual C++ Redistributable for Visual Studio, see http://windows.php.net/
Comment: Are you sure that php.exe is in c:\php and not in c:\php\php-5.6.16-Win32-VC11-x86?
Comment: yes i am sure. All the php files are directly inside C:/php
Comment: I updated question and placed screenshot of file file strucutre
Comment: Is your c:\php directory in the PATH system variable? what happens when you run php -l from the command line?
Comment: I entered C:\php in my System Environment variables PATH already. However when i do php -l from command line i get error box

`"The program can't start because MSVCR110.dll is missing from your computer. Try reinstalling the program to fix this problem".`
Comment: @yolenoyer i treid both paths you mentioned. And upon saving config file i get this error `Unable to determine the version of "C:/php/php.exe", please verify that this is the right path to PHP`
Comment: @dev0 I installed this component from `http://www.microsoft.com/en-us/download/details.aspx?id=30679` and still on command line `php -l` i get the same error
Comment: @dev I installed 86 version and now php -l running fine
Comment: ok so the path is "C:\\php\php.exe" Worked
Answer: I just went through the same thing. Here is my config.cson file. I'm using wamp for local PHP testing so my path will look different to the php.exe but the concept is the same.
"*":
  core:
  ...
  welcome:
    showOnStartup: false
  linter: {}
  "linter-php":
    executablePath: "C:/wamp/bin/php/php6.5.12/php.exe"

Hope this helps! (BTW the ... is just extra stuff in my config.cson file that wouldn't have helped)
Comment: It may be the 64 bit version too, but this worked for me too,
    executablePath: "C:/wamp64/bin/php/php6.5.12/php.exe"

Atom PHP lint not working Windows 10