What is an application binary interface (ABI)?

Question: I never clearly understood what an ABI is. Please don't point me to a Wikipedia article. If I could understand it, I wouldn't be here posting such a lengthy post.
This is my mindset about different interfaces:
A TV remote is an interface between the user and the TV. It is an existing entity, but useless (doesn't provide any functionality) by itself. All the functionality for each of those buttons on the remote is implemented in the television set.

Interface: It is an "existing entity" layer between the
  functionality and consumer of that functionality. An interface by itself
  doesn't do anything. It just invokes the functionality lying behind.
Now depending on who the user is there are different type of interfaces.
Command Line Interface (CLI) commands are the existing entities,
  the consumer is the user and functionality lies behind.
functionality: my software functionality which solves some
  purpose to which we are describing this interface.
existing entities: commands
consumer: user
Graphical User Interface(GUI) window, buttons, etc. are the existing
  entities, and again the consumer is the user and functionality lies behind.
functionality: my software functionality which solves some problem to which we are describing this interface.
existing entities: window, buttons etc..
consumer: user
Application Programming Interface(API) functions (or to be
  more correct) interfaces (in interfaced based programming) are the
  existing entities, consumer here is another program not a user, and again
  functionality lies behind this layer.
functionality: my software functionality which solves some
  problem to which we are describing this interface.
existing entities: functions, Interfaces (array of functions).
consumer: another program/application.
Application Binary Interface (ABI) Here is where my problem starts.
functionality: ???
existing entities: ???
consumer: ???

I've written software in different languages and provided different kinds of interfaces (CLI, GUI, and API), but I'm not sure if I have ever provided any ABI.

Wikipedia says:

ABIs cover details such as

data type, size, and alignment;
the calling convention, which controls how functions' arguments are
  passed and return values retrieved;
the system call numbers and how an application should make system calls
  to the operating system;

Other ABIs standardize details such as

the C++ name mangling,
exception propagation, and
calling convention between compilers on the same platform, but do
  not require cross-platform compatibility.

Who needs these details? Please don't say the OS. I know assembly programming. I know how linking & loading works. I know exactly what happens inside.
Why did C++ name mangling come in? I thought we are talking at the binary level. Why do languages come in?

Anyway, I've downloaded the [PDF] System V Application Binary Interface Edition 4.1 (1997-03-18) to see what exactly it contains. Well, most of it didn't make any sense.

Why does it contain two chapters (4th & 5th) to describe the ELF file format? In fact, these are the only two significant chapters of that specification. The rest of the chapters are "processor specific". Anyway, I though that it is a completely different topic. Please don't say that ELF file format specifications are the ABI. It doesn't qualify to be an interface according to the definition.
I know, since we are talking at such a low level it must be very specific. But I'm not sure how is it "instruction set architecture (ISA)" specific?
Where can I find Microsoft Windows' ABI?

So, these are the major queries that are bugging me.
Comment: I think the question is so clear; exactly describing what is the answer format expected and yet not a single satisfactory answer that which can be accepted.
Comment: "Please don't say, OS" Compilers need to know the ABI. Linkers need to know the ABI. The kernel needs to know the ABI in order to set up the program in RAM for it to run properly. As for C++ see below, it intentionally turns labels into gibberish because of overloading and private methods, and the linker and any other compiler need to have compatible name mangling to work with it, in other words the same ABI.
Comment: @legends2k My take on the issue is that OP does indeed know what an ABI is, but doesn't realize that. The vast majority of programmers will never ever design or provide an ABI, because that is the job of OS/platform designers.
Comment: @JesperE: I do agree to your point. But probably the OP wants to know it clearly, in the format s/he sees fit, even though s/he mightn't need to provide an ABI.
Comment: I was ignorant. Recently while working with all these things. I realized what ABI is actually is. Yeah, I do agree that my template is faulty. Its not appropriate to fit  ABI into my template. Thanks @ JasperE. It just took work experience to realize your answer.
Comment: I'm not at all convinced that my template is faulty. Because every where this template for interface is holding true. So, yes I want I expect ABI also to fit into this template but thats not it. IMPORTANT thing is I still don't understand. I don't know if I'm so dumb or something else but it just not getting into my head. I'm unable to realize the answers and the wiki article.
Comment: "how is it "Instruction Set Architecture(ISA)" specific?" you cannot run an x86 executable on an ARM processor, period. So the ABI used on an x86 system cannot be the same one used on ARM.
Comment: You are using a very strange definition of "interface". Consider merely an interface as the point at which, or mechanism through which, two or more entities interact.

An ABI is merely a description of the low-level (i.e. bit twiddling) interaction of components in a software system---allowing, for instance, code written in C to call code written in Fortran by appropriate translation of data types and selection of calling conventions.

No need to get philosphical. A CLI and an API are not meaningfully in the same class of entity, and attempting to find such a framework is fruitless.
Comment: I think the answer should be something like `Consumer: Compiler (Writer)`, `Provider: OS (Maker)`, `Functionality: Defining protocol on how to pass data`... (am just giving examples here).
Comment: @JesperE: As a matter of fact, I didn't :)
Comment: @claws, This explanation may help: http://stackoverflow.com/questions/3784389/difference-between-api-and-abi/3784724#comment39838598_3784724
Comment: The easiest way to think of an interface, in general, is as a contract between a framework, library, platform, etc. If you want to use that system, you are expected to adhere to the rules laid out in the interface. It is a "Rules of engagement" notice.
Answer: One easy way to understand "ABI" is to compare it to "API".
You are already familiar with the concept of an API.  If you want to use the features of, say, some library or your OS, you will program against an API.  The API consists of data types/structures, constants, functions, etc that you can use in your code to access the functionality of that external component.
An ABI is very similar.  Think of it as the compiled version of an API (or as an API on the machine-language level).  When you write source code, you access the library through an API.  Once the code is compiled, your application accesses the binary data in the library through the ABI.  The ABI defines the structures and methods that your compiled application will use to access the external library (just like the API did), only on a lower level.  Your API defines the order in which you pass arguments to a function.  Your ABI defines the mechanics of how these arguments are passed (registers, stack, etc.).  Your API defines which functions are part of your library.  Your ABI defines how your code is stored inside the library file, so that any program using your library can locate the desired function and execute it.
ABIs are important when it comes to applications that use external libraries.  Libraries are full of code and other resources, but your program has to know how to locate what it needs inside the library file.  Your ABI defines how the contents of a library are stored inside the file, and your program uses the ABI to search through the file and find what it needs.  If everything in your system conforms to the same ABI, then any program is able to work with any library file, no matter who created them.  Linux and Windows use different ABIs, so a Windows program won't know how to access a library compiled for Linux.
Sometimes, ABI changes are unavoidable.  When this happens, any programs that use that library will not work unless they are re-compiled to use the new version of the library.  If the ABI changes but the API does not, then the old and new library versions are sometimes called "source compatible".  This implies that while a program compiled for one library version will not work with the other, source code written for one will work for the other if re-compiled.
For this reason, developers tend to try to keep their ABI stable (to minimize disruption).  Keeping an ABI stable means not changing function interfaces (return type and number, types, and order of arguments), definitions of data types or data structures, defined constants, etc.  New functions and data types can be added, but existing ones must stay the same.  If, for instance, your library uses 32-bit integers to indicate the offset of a function and you switch to 64-bit integers, then already-compiled code that uses that library will not be accessing that field (or any following it) correctly.  Accessing data structure members gets converted into memory addresses and offsets during compilation and if the data structure changes, then these offsets will not point to what the code is expecting them to point to and the results are unpredictable at best.
An ABI isn't necessarily something you will explicitly provide unless you are doing very low-level systems design work.  It isn't language-specific either, since (for example) a C application and a Pascal application can use the same ABI after they are compiled.
Edit: Regarding your question about the chapters regarding the ELF file format in the SysV ABI docs:  The reason this information is included is because the ELF format defines the interface between operating system and application.  When you tell the OS to run a program, it expects the program to be formatted in a certain way and (for example) expects the first section of the binary to be an ELF header containing certain information at specific memory offsets.  This is how the application communicates important information about itself to the operating system.  If you build a program in a non-ELF binary format (such as a.out or PE), then an OS that expects ELF-formatted applications will not be able to interpret the binary file or run the application.  This is one big reason why Windows apps cannot be run directly on a Linux machine (or vice versa) without being either re-compiled or run inside some type of emulation layer that can translate from one binary format to another.
IIRC, Windows currently uses the Portable Executable (or, PE) format.  There are links in the "external links" section of that Wikipedia page with more information about the PE format.
Also, regarding your note about C++ name mangling:  When locating a function in a library file, the function is typically looked up by name.  C++ allows you to overload function names, so name alone is not sufficient to identify a function.  C++ compilers have their own ways of dealing with this internally, called name mangling.  An ABI can define a standard way of encoding the name of a function so that programs built with a different language or compiler can locate what they need.  When you use extern "c" in a C++ program, you're instructing the compiler to use a standardized way of recording names that's understandable by other software.
Comment: Nice answer. Except this isn't what an ABI is. An ABI is a set of rules that determines calling convention, and rules for laying out structures. Pascal passes arguments on the stack in the reverse order from C applications, so pascal and C compilers do NOT compile to the same ABI. The respective standards for C and Pascal compilers implicitly ensure that this will be the case. C++ compilers cannot define a "standard" way to mangle names, since there is no standard way. C++ name mangling conventions were not compatible between C++ compilers when there were competing C++ compilers on Windows.
Comment: @bta, Thanks for the great answer. Does calling convention is a kind of ABI?  Thanks
Comment: Definitely also see https://autotools.io/libtool/version.html and http://fedoramagazine.org/gcc-5-in-fedora-whats-an-abi-and-what-happens-when-we-change-it/
Comment: @RobinDavies: On platforms where Pascal compilers would have called functions pop arguments given by their callers, C compilers would generally define means by which a programmer could indicate that particular functions should use, or should be expected to use, the same calling conventions as the Pascal compilers even though C compilers would generally by default use a convention where called functions leave on the stack anything placed there by their callers.
Comment: @RobinDavies  _determines calling convention_  Dealing with stdcall, pascal, cdecl over the years, your comment made instant sense.
Answer: If you know assembly and how things work at the OS-level, you are conforming to a certain ABI. The ABI govern things like how parameters are passed, where return values are placed. For many platforms there is only one ABI to choose from, and in those cases the ABI is just "how things work".
However, the ABI also govern things like how classes/objects are laid out in C++. This is necessary if you want to be able to pass object references across module boundaries or if you want to mix code compiled with different compilers.
Also, if you have an 64-bit OS which can execute 32-bit binaries, you will have different ABIs for 32- and 64-bit code.
In general, any code you link into the same executable must conform to the same ABI. If you want to communicate between code using different ABIs, you must use some form of RPC or serialization protocols.
I think you are trying too hard to squeeze in different types of interfaces into a fixed set of characteristics. For example, an interface doesn't necessarily have to be split into consumers and producers. An interface is just a convention by which two entities interact.
ABIs can be (partially) ISA-agnostic. Some aspects (such as calling conventions) depend on the ISA, while other aspects (such as C++ class layout) do not.
A well defined ABI is very important for people writing compilers. Without a well defined ABI, it would be impossible to generate interoperable code.
EDIT: Some notes to clarify:

"Binary" in ABI does not exclude the use of strings or text. If you want to link a DLL exporting a C++ class, somewhere in it the methods and type signatures must be encoded. That's where C++ name-mangling comes in.
The reason why you never provided an ABI is that the vast majority of programmers will never do it. ABIs are provided by the same people designing the platform (i.e. operating system), and very few programmers will ever have the privilege to design a widely-used ABI.
Comment: this is *the* correct and *precise* answer without the verbosity (rather *noise*)!
Comment: Yes. The proper name is "calling convention", which is a part of the ABI. http://en.wikipedia.org/wiki/X86_calling_conventions
Comment: @jesperE,  "The ABI govern things like how parameters are passed, where return values are placed. " is refer to "cdecl,stdcall,fastcall,pascal" right?
Comment: I'm not at all convinced that my template is faulty. Because every where this template for interface is holding true. So, yes I want I expect ABI also to fit into this template but thats not it. IMPORTANT thing is I still don't understand. I don't know if I'm so dumb or something else but it just not getting into my head. I'm unable to realize the answers and the wiki article.
Comment: I recommend writing a bit of assembly. This will help people understand ABI in a more tangible way.
Answer: You actually don't need an ABI at all if--

Your program doesn't have functions, and--
Your program is a single executable that is running alone (i.e. an embedded system) where it's literally the only thing running and it doesn't need to talk to anything else.

An oversimplified summary:

API: "Here are all the functions you may call."
ABI: "This is how to call a function."

The ABI is set of rules that compilers and linkers adhere to in order to compile your program so that will work properly.  ABIs cover multiple topics:

Arguably the biggest and most important part of an ABI is the procedure call standard sometimes known as the "calling convention".  Calling conventions standardize how "functions" are translated to assembly code.
ABIs also dictate the how the names of exposed functions in libraries should be represented so that other code can call those libraries and know what arguments should be passed.  This is called "name mangling".
ABIs also dictate what type of data types can be used, how they must be aligned, and other low-level details.

Taking a deeper look at calling convention, which I consider to be the core of an ABI:
The machine itself has no concept of "functions".  When you write a function in a high-level language like c, the compiler generates a line of assembly code like _MyFunction1:.  This is a label, which will eventually get resolved into an address by the assembler.  This label marks the "start" of your "function" in the assembly code.  In high-level code, when you "call" that function, what you're really doing is causing the CPU to jump to the address of that label and continue executing there.
In preparation for the jump, the compiler must do a bunch of important stuff.  The calling convention is like a checklist that the compiler follows to do all this stuff:

First, the compiler inserts a little bit of assembly code to save the current address, so that when your "function" is done, the CPU can jump back to the right place and continue executing.
Next, the compiler generates assembly code to pass the arguments.

Some calling conventions dictate that arguments should be put on the stack (in a particular order of course).
Other conventions dictate that the arguments should be put in particular registers (depending on their data types of course).
Still other conventions dictate that a specific combination of stack and registers should be used.

Of course, if there was anything important in those registers before, those values are now overwritten and lost forever, so some calling conventions may dictate that the compiler should save some of those registers prior to putting the arguments in them.
Now the compiler inserts a jump instruction telling the CPU to go to that label it made previously (_MyFunction1:).  At this point, you can consider the CPU to be "in" your "function".
At the end of the function, the compiler puts some assembly code that will make the CPU write the return value in the correct place.  The calling convention will dictate whether the return value should be put into a particular register (depending on its type), or on the stack.
Now it's time for clean-up.  The calling convention will dictate where the compiler places the cleanup assembly code.

Some conventions say that the caller must clean up the stack.  This means that after the "function" is done and the CPU jumps back to where it was before, the very next code to be executed should be some very specific cleanup code.
Other conventions say that the some particular parts of the cleanup code should be at the end of the "function" before the jump back.

There are many different ABIs / calling conventions.  Some main ones are:

For the x86 or x86-64 CPU (32-bit environment):

CDECL
STDCALL
FASTCALL
VECTORCALL
THISCALL

For the x86-64 CPU (64-bit environment):

SYSTEMV
MSNATIVE
VECTORCALL

For the ARM CPU (32-bit)

AAPCS

For the ARM CPU (64-bit)

AAPCS64

Here is a great page that actually shows the differences in the assembly generated when compiling for different ABIs.
Another thing to mention is that an ABI isn't only relevant inside your program's executable module.  It's also used by the linker to make sure your program calls library functions correctly.  You have multiple shared libraries running on your computer, and as long as your compiler knows what ABI they each use, it can call functions from them properly without blowing up the stack.
Your compiler understanding how to call library functions is extremely important.  On a hosted platform (that is, one where an OS loads programs), your program can't even blink without making a kernel call.
Answer: Linux shared library minimal runnable ABI example
In the context of shared libraries, the most important implication of "having a stable ABI" is that you don't need to recompile your programs after the library changes.
So for example:

if you are selling a shared library, you save your users the annoyance of recompiling everything that depends on your library for every new release
if you are selling closed source program that depends on a shared library present in the user's distribution, you could release and test less prebuilts if you are certain that ABI is stable across certain versions of the target OS.
This is specially important in the case of the C standard library, which many many programs in your system link to.

Now I want to provide a minimal concrete runnable example of this.
main.c
#include <assert.h>
#include <stdlib.h>

#include "mylib.h"

int main(void) {
    mylib_mystruct *myobject = mylib_init(1);
    assert(myobject->old_field == 1);
    free(myobject);
    return EXIT_SUCCESS;
}

mylib.c
#include <stdlib.h>

#include "mylib.h"

mylib_mystruct* mylib_init(int old_field) {
    mylib_mystruct *myobject;
    myobject = malloc(sizeof(mylib_mystruct));
    myobject->old_field = old_field;
    return myobject;
}

mylib.h
#ifndef MYLIB_H
#define MYLIB_H

typedef struct {
    int old_field;
} mylib_mystruct;

mylib_mystruct* mylib_init(int old_field);

#endif

Compiles and runs fine with:
cc='gcc -pedantic-errors -std=c89 -Wall -Wextra'
$cc -fPIC -c -o mylib.o mylib.c
$cc -L . -shared -o libmylib.so mylib.o
$cc -L . -o main.out main.c -lmylib
LD_LIBRARY_PATH=. ./main.out

Now, suppose that for v2 of the library, we want to add a new field to mylib_mystruct called new_field.
If we added the field before old_field as in:
typedef struct {
    int new_field;
    int old_field;
} mylib_mystruct;

and rebuilt the library but not main.out, then the assert fails!
This is because the line:
myobject->old_field == 1

had generated assembly that is trying to access the very first int of the struct, which is now new_field instead of the expected old_field.
Therefore this change broke the ABI.
If, however, we add new_field after old_field:
typedef struct {
    int old_field;
    int new_field;
} mylib_mystruct;

then the old generated assembly still accesses the first int of the struct, and the program still works, because we kept the ABI stable.
Here is a fully automated version of this example on GitHub.
Another way to keep this ABI stable would have been to treat mylib_mystruct as an opaque struct, and only access its fields through method helpers. This makes it easier to keep the ABI stable, but would incur a performance overhead as we'd do more function calls.
API vs ABI
In the previous example, it is interesting to note that adding the new_field before old_field, only broke the ABI, but not the API.
What this means, is that if we had recompiled our main.c program against the library, it would have worked regardless.
We would also have broken the API however if we had changed for example the function signature:
mylib_mystruct* mylib_init(int old_field, int new_field);

since in that case, main.c would stop compiling altogether.
Semantic API vs Programming API
We can also classify API changes in a third type: semantic changes.
The semantic API, is usually a natural language description of what the API is supposed to do, usually included in the API documentation.
It is therefore possible to break the semantic API without breaking the program build itself.
For example, if we had modified
myobject->old_field = old_field;

to:
myobject->old_field = old_field + 1;

then this would have broken neither programming API, nor ABI, but main.c the semantic API would break.
There are two ways to programmatically check the contract API:

test a bunch of corner cases. Easy to do, but you might always miss one.
formal verification. Harder to do, but produces mathematical proof of correctness, essentially unifying documentation and tests into a "human" / machine verifiable manner! As long as there isn't a bug in your formal description of course ;-)
This concept is closely related to the formalization of Mathematics itself: https://math.stackexchange.com/questions/53969/what-does-formal-mean/3297537#3297537

List of everything that breaks C / C++ shared library ABIs
TODO: find / create the ultimate list:

https://github.com/lvc/abi-compliance-checker automated tool to check it
https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B KDE C++ ABI guidelines
https://plan99.net/~mike/writing-shared-libraries.html

Java minimal runnable example
What is binary compatibility in Java?
Tested in Ubuntu 18.10, GCC 8.2.0.
Comment: @starriet it defines the bash shell variable `cc` to `gcc ...`. Then I use gcc with those flags multiple times. It is just to not repeat the flags.
Comment: what does `cc='gcc...'` mean? `cc` should be the same as `gcc` in most Linux systems, but I don't get what `='gcc some_flags...'` part means.
Comment: adding a new field will break ABI, you don't see it in your example because in your code the main is too small using only 1 library, had used a few others would have shown the violation
Comment: @codeswimmer can you provide an example?
Comment: @CiroSantilliOurBigBook.com just reading a book https://books.google.com.au/books?id=kfRWvKSePmAC&pg=PA1&source=gbs_toc_r&redir_esc=y#v=onepage&q&f=false page 10
Answer: An application binary interface (ABI) is similar to an API, but the function is not accessible to the caller at source code level. Only a binary representation is accessible/available.
ABIs may be defined at the processor-architecture level or at the OS level.
The ABIs are standards to be followed by the code-generator phase of the compiler. The standard is fixed either by the OS or by the processor.
Functionality: Define the mechanism/standard to make function calls independent of the implementation language or a specific compiler/linker/toolchain. Provide the mechanism which allows JNI, or a Python-C interface, etc.
Existing entities: Functions in machine code form.
Consumer: Another function (including one in another language, compiled by another compiler, or linked by another linker).
Comment: Why would the ABI be defined by the architecture? Why would different OSes on the same architecture not be able to define different ABIs?
Answer: Let me at least answer a part of your question. With an example of how the Linux ABI affects the systemcalls, and why that is usefull.
A systemcall is a way for a userspace program to ask the kernelspace for something. It works by putting the numeric code for the call and the argument in a certain register and triggering an interrupt. Than a switch occurs to kernelspace and the kernel looks up the numeric code and the argument, handles the request, puts the result back into a register and triggers a switch back to userspace. This is needed for example when the application wants to allocate memory or open a file (syscalls "brk" and "open").
Now the syscalls have short names "brk", etc. and corresponding opcodes, these are defined in a system specific header file. As long as these opcodes stay the same you can run the same compiled userland programs with different updated kernels without having to recompile. So you have an interface used by precompiled binarys, hence ABI.
Answer: Functionality: A set of contracts which affect the compiler, assembly writers, the linker, and the operating system. The contracts specify how functions are laid out, where parameters are passed, how parameters are passed, how function returns work. These are generally specific to a (processor architecture, operating system) tuple.
Existing entities: parameter layout, function semantics, register allocation. For instance, the ARM architectures has numerous ABIs (APCS, EABI, GNU-EABI, never mind a bunch of historical cases) - using the a mixed ABI will result in your code simply not working when calling across boundaries.
Consumer: The compiler, assembly writers, operating system, CPU specific architecture.
Who needs these details? The compiler, assembly writers, linkers which do code generation (or alignment requirements), operating system (interrupt handling, syscall interface). If you did assembly programming, you were conforming to an ABI!
C++ name mangling is a special case - its a linker and dynamic linker centered issue - if name mangling is not standardized, then dynamic linking will not work. Henceforth, the C++ ABI is called just that, the C++ ABI. It is not a linker level issue, but instead a code generation issue. Once you have a C++ binary, it is not possible to make it compatible with another C++ ABI (name mangling, exception handling) without recompiling from source.
ELF is a file format for the use of a loader and dynamic linker. ELF is a container format for binary code and data, and as such specifies the ABI of a piece of code. I would not consider ELF to be an ABI in the strict sense, as PE executables are not an ABI.
All ABIs are instruction set specific. An ARM ABI will not make sense on an MSP430 or x86_64 processor.
Windows has several ABIs - for instance, fastcall and stdcall are two common use ABIs. The syscall ABI is different again.
Answer: Summary
There are various interpretation and strong opinions of the exact layer that define an ABI (application binary interface).
In my view an ABI is a subjective convention of what is considered a given/platform for a specific API. The ABI is the "rest" of conventions that "will not change" for a specific API or that will be addressed by the runtime environment: executors, tools, linkers, compilers, jvm, and OS.
Defining an Interface: ABI, API
If you want to use a library like joda-time you must declare a dependency on joda-time-<major>.<minor>.<patch>.jar. The library follows best practices and use Semantic Versioning. This defines the API compatibility at three levels:

Patch - You don't need to change at all your code. The library just fixes some bugs.
Minor - You don't need to change your code since things where addded (open closed principle was respected)
Major - The interface (API) is changed and you might need to change your code.

In order for you to use a new major release of the same library a lot of other conventions are still to be respected:

The binary language used for the libraries (in Java cases the JVM target version that defines the Java bytecode)
Calling conventions
JVM conventions
Linking conventions
Runtime conventions
All these are defined and managed by the tools we use.

Examples
Java case study
For example, Java standardized all these conventions, not in a tool, but in a formal JVM specification. The specification allowed other vendors to provide a different set of tools that can output compatible libraries.
Java provides two other interesting case studies for ABI: Scala versions and Dalvik virtual machine.
Dalvik virtual machine broke the ABI
The Dalvik VM needs a different type of bytecode than the Java bytecode. The Dalvik libraries are obtained by converting the Java bytecode (with same API) for Dalvik. In this way you can get two versions of the same API: defined by the original joda-time-1.7.2.jar. We could call it joda-time-1.7.2.jar and joda-time-1.7.2-dalvik.jar. They use a different ABI one is for the stack-oriented standard Java vms: Oracle's one, IBM's one, open Java or any other; and the second ABI is the one around Dalvik.
Scala successive releases are incompatible
Scala doesn't have binary compatibility between minor Scala versions: 2.X . For this reason the same API "io.reactivex" %% "rxscala" % "0.26.5" has three versions (in the future more): for Scala 2.10, 2.11 and 2.12. What is changed? I don't know for now, but the binaries are not compatible. Probably the latest versions adds things that make the libraries unusable on the old virtual machines, probably things related to linking/naming/parameter conventions.
Java successive releases are incompatible
Java has problems with the major releases of the JVM too: 4,5,6,7,8,9. They offer only backward compatibility. Jvm9 knows how to run code compiled/targeted (javac's -target option) for all other versions, while JVM 4 doesn't know how to run code targeted for JVM 5. All these while you have one joda-library. This incompatibility flies bellow the radar thanks to different solutions:

Semantic versioning: when libraries target higher JVM they usually change the major version.
Use JVM 4 as the ABI, and you're safe.
Java 9 adds a specification on how you can include bytecode for specific targeted JVM in the same library.

Why did I start with the API definition?
API and ABI are just conventions on how you define compatibility. The lower layers are generic in respect of a plethora of high level semantics. That's why it's easy to make some conventions. The first kind of conventions are about memory alignment, byte encoding, calling conventions, big and little endian encodings, etc. On top of them you get the executable conventions like others described, linking conventions, intermediate byte code like the one used by Java or LLVM IR used by GCC. Third you get conventions on how to find libraries, how to load them (see Java classloaders). As you go higher and higher in concepts you have new conventions that you consider as a given. That's why they didn't made it to the semantic versioning. They are implicit or collapsed in the major version. We could amend semantic versioning with <major>-<minor>-<patch>-<platform/ABI>. This is what is actually happening already: platform is already a rpm, dll, jar (JVM bytecode), war(jvm+web server), apk, 2.11 (specific Scala version) and so on. When you say APK you already talk about a specific ABI part of your API.
API can be ported to different ABI
The top level of an abstraction (the sources written against the highest API can be recompiled/ported to any other lower level abstraction.
Let's say I have some sources for rxscala. If the Scala tools are changed I can recompile them to that. If the JVM changes I could have automatic conversions from the old machine to the new one without bothering with the high level concepts. While porting might be difficult will help any other client. If a new operating system is created using a totally different assembler code a translator can be created.
APIs ported across languages
There are APIs that are ported in multiple languages like reactive streams. In general they define mappings to specific languages/platforms. I would argue that the API is the master specification formally defined in human language or even a specific programming language. All the other "mappings" are ABI in a sense, else more API than the usual ABI. The same is happening with the REST interfaces.
Answer: In order to call code in shared libraries, or call code between compilation units, the object file needs to contain labels for the calls. C++ mangles the names of method labels in order to enforce data hiding and allow for overloaded methods. That is why you cannot mix files from different C++ compilers unless they explicitly support the same ABI.
Answer: The best way to differentiate between ABI and API is to know why and what is it used for:
For x86-64 there is generally one ABI (and for x86 32-bit there is another set):
http://www.x86-64.org/documentation/abi.pdf
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/LowLevelABI/140-x86-64_Function_Calling_Conventions/x86_64.html
http://people.freebsd.org/~obrien/amd64-elf-abi.pdf
Linux + FreeBSD + MacOSX follow it with some slight variations.   And Windows x64 have its own ABI:
http://eli.thegreenplace.net/2011/09/06/stack-frame-layout-on-x86-64/
Knowing the ABI and assuming other compiler follows it as well, then the binaries theoretically know how to call each other (libraries API in particular) and pass parameters over the stack or by registers etc.   Or what registers will be changed upon calling the functions etc.   Essentially these knowledge will help software to integrate with one another.  Knowing the order of the registers / stack layout I can easily piece together different software written in assemblies together without much problem.
But API are different:
It is a high level functions names, with argument defined, such that if different software pieces build using these API, MAY be able to call into one another.   But an additional requirement of SAME ABI must be adhered to.
For example, Windows used to be POSIX API compliant:
https://en.wikipedia.org/wiki/Windows_Services_for_UNIX
https://en.wikipedia.org/wiki/POSIX
And Linux is POSIX compliant as well.   But the binaries cannot be just moved over and run immediately.   But because they used the same NAMES in the POSIX compliant API, you can take the same software in C, recompile it in the different OS, and immediately get it running.
API are meant to ease integration of software - pre-compilation stage.   So after compilation the software can look totally different - if the ABI are different.
ABI are meant to define exact integration of software at the binary / assembly level.
Comment: The Windows x86-64 calling convention doesn't use the SysV calling convention that all other x86-64 OSes use.  Linux/OS X/FreeBSD all share the same calling convention, but they *don't* share the full ABI.  An OS's ABI includes system-call numbers.  e.g. https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/x86-portable-code.html says that `SYS_execve` is 11 on 32bit linux, but 59 on FreeBSD.
Comment: thanks for your comment, i have modified my comment to better answer the difference between ABI and API.
Comment: You're still missing the difference between a calling convention and a full ABI (system calls and everything).  You can run some FreeBSD binaries on Linux, because Linux (the kernel) provides a FreeBSD compatibility layer.  Even then, this is limited to binaries that don't try to use any part of the FreeBSD ABI that Linux doesn't provide.  (e.g. any FreeBSD-only system call).  ABI-compatible means you can run the same binary on both systems, not just that they'd compile similarly.
Comment: "FreeBSD compatibility layer", never heard of that.   Can you point to the relevant linux kernel source code?   But the reverse do exists:   https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/linuxemu.html.
Comment: It's not something I use.  I *thought* something like that existed, but maybe it doesn't anymore.  http://www.tldp.org/HOWTO/Linux+FreeBSD-6.html says it's unmaintained, and that howto is from 2000. xD.  http://unix.stackexchange.com/questions/172038/what-allows-bsd-to-run-linux-binaries-but-not-vice-versa confirms that it was abandoned and never re-done (since nobody wanted it badly enough to get it done).  `personality(2)` can set `PER_BSD`.  I think I remember seeing `personality(PER_LINUX)` in `strace` output all the time, but modern 64bit Linux binaries don't do that anymore.
Comment: Also, I removed my downvote since sometimes *ABI* does just mean calling convention, name-mangling, and stuff like that, but not the whole system call interface to the OS.  I still think this answer is bit messy and scattered, so I'm not upvoting it.  IIRC, the other answers do a good job, and this one doesn't cover much if anything that isn't already covered by higher-voted answers.
Answer: The term ABI is used to refer to two distinct but related concepts.
When talking about compilers it refers to the rules used to translate from source-level constructs to binary constructs. How big are the data types? how does the stack work? how do I pass parameters to functions? which registers should be saved by the caller vs the callee?
When talking about libraries it refers to the binary interface presented by a compiled library. This interface is the result of a number of factors including the source code of the library, the rules used by the compiler and in some cases definitions picked up from other libraries.
Changes to a library can break the ABI without breaking the API. Consider for example a library with an interface like.
void initfoo(FOO * foo)
int usefoo(FOO * foo, int bar)
void cleanupfoo(FOO * foo)

and the application programmer writes code like
int dostuffwithfoo(int bar) {
  FOO foo;
  initfoo(&foo);
  int result = usefoo(&foo,bar)
  cleanupfoo(&foo);
  return result;
}

The application programmer doesn't care about the size or layout of FOO, but the application binary ends up with a hardcoded size of foo. If the library programmer adds an extra field to foo and someone uses the new library binary with the old application binary then the library may make out of bounds memory accesses.
OTOH if the library author had designed their API like.
FOO * newfoo(void)
int usefoo(FOO * foo, int bar)
void deletefoo((FOO * foo, int bar))

and the application programmer writes code like
int dostuffwithfoo(int bar) {
  FOO * foo;
  foo = newfoo();
  int result = usefoo(foo,bar)
  deletefoo(foo);
  return result;
}

Then the application binary does not need to know anything about the structure of FOO, that can all be hidden inside the library. The price you pay for that though is that heap operations are involved.
Answer: The ABI needs to be consistent between caller and callee to be certain that the call succeeds. Stack use, register use, end-of-routine stack pop. All these are the most important parts of the ABI.
Answer: Application Binary Interface(ABI)
ABI - Application Binary Interface is about a machine code communication in runtime between two binary parts like - application, library, OS... ABI describes how objects are saved in memory, how functions are called(calling convention), mangling...
A good example of API and ABI is iOS ecosystem with Swift language from v5.

Application layer - When you create an application using different languages. For example you can create application using Swift and Objective-C[Mixing Swift and Objective-C]

Application - OS layer - runtime - Swift Standard Library and Swift Run Time Library[About] are parts of OS and they should not be included into each bundle(e.g. app, framework). It is the same as like Objective-C uses. Available from iOS v12.2

Library layer - Module Stability case - compile time - you will be able to import a framework which was built with another version of Swift's compiler. It means that it is safety to create a closed-source(pre-build) binary which will be consumed by a different version of compiler( .swiftinterface is used with .swiftmodule[About]) and you will not get
Module compiled with _ cannot be imported by the _ compiler
//or
Compiled module was created by a newer version of the compiler

Library layer - Library Evolution case

Compile time - if a dependency was changed, a client has not to be
recompiled.
Runtime - a system library or a dynamic framework can
be hot-swapped by a new one.

[API vs ABI]
[Swift Module Stability and Library Stability]
Answer: Application binary interface (ABI)
Functionality:

Translation from the programmer's model to the underlying system's domain data
type, size, alignment, the calling convention, which controls how
functions' arguments are passed and return values retrieved; the
system call numbers and how an application should make system calls
to the operating system; the high-level language compilers' name
mangling scheme, exception propagation, and calling convention
between compilers on the same platform, but do not require
cross-platform compatibility...

Existing entities:

Logical blocks that directly participate in program's execution: ALU,
general purpose registers, registers for memory/ I/O mapping of I/O, etc...

consumer:

Language processors linker, assembler...

These are needed by whoever has to ensure that build tool-chains work as a whole. If you write one module in assembly language, another in Python, and instead of your own boot-loader want to use an operating system, then your "application" modules are working across "binary" boundaries and require agreement of such "interface".
C++ name mangling because object files from different high-level languages might be required to be linked in your application. Consider using GCC standard library making system calls to Windows built with Visual C++.
ELF is one possible expectation of the linker from an object file for interpretation, though JVM might have some other idea.
For a Windows RT Store app, try searching for ARM ABI if you really wish to make some build tool-chain work together.
Answer: I was also trying to understand ABI and JesperE's answer was very helpful.
From a very simple perspective, we may try to understand ABI by considering binary compatibility.
KDE wiki defines a library as binary compatible "if a program linked dynamically to a former version of the library continues running with newer versions of the library without the need to recompile." For more on dynamic linking, refer Static linking vs dynamic linking
Now, let's try to look at just the most basic aspects needed for a library to be binary compatibility (assuming there are no source code changes to the library):

Same/backward compatible instruction set architecture (processor instructions, register file structure, stack organization, memory access types, along with sizes, layout, and alignment of basic data types the processor can directly access)
Same calling conventions
Same name mangling convention (this might be needed if say a Fortran program needs to call some C++ library function).

Sure, there are many other details but this is mostly what the ABI also covers.
More specifically to answer your question, from the above, we can deduce:

ABI functionality: binary compatibility
existing entities: existing program/libraries/OS
consumer: libraries, OS

Hope this helps!
Answer: A. Plainly speaking, one common thing an ABI has with an API is that it is an interface. A reusable program exposes a stable interface (API) that can be used to reuse the program in another program.
B. However, an ABI is an interface issued for some specific processor-platform for some specific language. All compiler-vendors desiring to  target that platform for that same language will have to ensure that not only compiled code in form of relocatable object codes comply with the interface to be able to link and cross-link with each other but also executables comply with it to be able to run on the platform at all. So, ABI is much broader set of specifications/standard than a typical function API. It may include some API objects to be enforced upon the language-users by the compiler. The compiler-vendor will have to include support for the same in their distributions. Needless to say, the platform vendor is the rightful authority to issue ABIs for its platform. Both compiler vendors and ABIs need to comply with the corresponding language-standard (e.g. ISO standard  for C++).
C. A definition of an ABI by a platform vendor is:
"1. The specifications to which an executable must conform in order to execute in a specific execution environment. For example, the Linux ABI for the Arm Architecture.

A particular aspect of the specifications to which independently produced relocatable files must conform in order to be statically linkable and executable. For example, the C++ ABI for the Arm Architecture, the Run-time ABI for the Arm Architecture, the C Library ABI for the Arm Architecture."

D. For example. A generic ABI for C++ based on Itanium architecture has also been issued by a consortium. The extent to which platform-vendors's own ABIs for C++ comply with it is entirely up to the platform vendors.
E. As another example. The C++ ABI for Arm Architecture is here.
F. Having said that, under the hood, it is the ABI of a processor-architecture that will ensure that the API between one reusable program and another program that reuses it works for that processor-architecture.
G. That brings us to service-oriented components (e.g. SOAP-based web services). They too require an API to exist between a SOAP-based web service and client program (could be an app, front-end or another web service) for the client program to reuse the web service.The API is described in terms of standardized protocols like WSDL  (interface description) and SOAP(message format) and is language-neutral and platform-neutral. It is not targeted to any specific processor-platform and thus it is not "binary" like ABI. A client-program on any one platform type and written in any language can remotely reuse a web service written in any other language and hosted on an entirely different processor-platform. This is made possible by the fact that both WSDL and SOAP are text-based (XML) protocols. In case of RESTful web services, the  transport protocol http--also a text-based protocol-- itself acts as the API (CRUD methods).
Answer: In short and in philosophy, only things of a kind can get along well, and the ABI could be seen as the kind of which software stuff work together.


Email notifications to AD group members - no access to Exchange

Question: Using SharePoint Designer 2010, I created a group and had AD group as a member. I tried to send email notification to the individuals in that AD group but the email only received by the group email address. I'm trying to make it distributed to the individuals email addresses within the group with no luck. I looked up and found several solutions require changes in Exchange which I don't have access to. Is there any other solution?
Comment: did you created the SharePoint Group and then add AD group in it?
Comment: Yes, I did. Same result.
Answer: You can create the group in SharePoint and send de email to the SharePoint Group.
Let me explain. Understanding that you can not configure Exchange, I think there is no solution using AD groups. The alternative I propose is: create native sharepoint groups and associate users directly to SharePoint groups.
I know that probably you prefer to manage AD groups but not think you can solve it if you do not have access to Exchange.
Comment: Shadin, already mentioned he did and no result, also this is not right answer.could you please edit it and provide him relevant answer.
Comment: Edited. Mi idea is use sharepoint groups with users directly inside, without ad groups.
Comment: Thank you both! Now I can stop looking and move on thinking of another solution.
Answer: There is a known limitation related to send SPD workflow e-mails to AD groups.
Basically AD groups are seen by SPD as a regular user... let's remember that AD groups have properties like e-mail, name, login and everything else.
So, the SPD can't understand that you are trying to send e-mails to everyone inside that group. What you can do is create something that iterates through every user inside that AD group... but this solution goes to the custom .NET way of living.
By the other hand, SPD knows that a SharePoint group is A GROUP. And when you send e-mail to the group, the SPD will send it to everyone inside it.
Comment: I see, thanks for your explanation. I have multiple sub-site which make creating separate groups for each one a painful task, moreover, the process of updating these groups manually whenever needed is not ideal. They better be associated and synchronized with AD. It's strange that SharePoint has such limitation.


What makes a piece of software an Engine?

Question: The descriptor 'Engine' gets thrown around a lot: graphics engine, RegEx engine, AI engine, etc. but what actually makes a piece of software an Engine? Design, Input/Output, Purpose, Size?
Comment: The marketing department.
Comment: @Oded: term 'engine' is commonly used by programmers themselves. Try to find alternatives and you'll see why.
Comment: an engine is what makes the wheels I don't want to be reinventing turn
Comment: @Flexo, so - a library?
Answer: An engine would be something that is "under the hood", so to speak. It is not, or at least very rarely is, visible to the end user. A graphics engine, for instance, drives all the rendering calculations but passes those changes on to the actual environment to be modeled. Input: math. Output: pretty colors. An engine might also have very different working variables than a more high level interface. For instance, in the previous example, it is using raw numerical data to manipulate graphics without worrying about whether something is a shadow or a texture, all of that is abstracted into the equations and matrix operations to be performed by that engine. Think of the Engine as the "Kernel" of a given system while the rest would be more like the "Shell".
To use a real world, CS101 analogy, an engine is just like a car engine. It takes two inputs, air and gas. It then passes them into a chamber, whereupon electricity is used to generate the world's smallest use of arc welding. Stuff then explodes. This produces two outputs, exhaust and a pressure wave which drives a piston. The rest is transferred into wheel motion by the various drive shafts and such. So the Engine is the engine and the car itself is the shell. You could use a car engine for a different purpose, say driving a generator for electricity or a mill to grind grain. You could use different inputs if the Engine has the coatings and such to handle things like ethanol or biodiesel.
To sum it up, an Engine is a piece of software that is usually not found in isolation. It acts as motive force for that piece of software but typically interacts very little if at all with the outside world. Several engines may work together to produce complimentary outputs or may be pipelined together as needed. An engine does not do things related to user experience in an aesthetic sense directly but drives those experiences none the less by motivating dataflow and being responsive enough to allow for good application performance.
Comment: So a driver is the tool that controls the engine & crashes the shell?
Comment: i really liked and benifited from your answer but i think the question was more about how to now this piece of code/software should be called a library or API or engine ? if you can add explaination for this as well it will be perfect answer
Comment: Libraries and APIs operate at a different level than an Engine does. Better to keep it focused on the term at hand than diverge into a potentially endless sea of variations and names.
Answer: The definition of engine is
noun /ˈenjən/
engines, plural

A machine with moving parts that converts power into motion
A thing that is the agent or instrument of a particular process

#1 is the semantic equivelant to engine in the software sense that it causes something to happen. 3D Graphics Engine, takes inputs and converts that to motion on the screen. A regex engine takes inputs and converts them to a different output.
#2 is like using the sentence an engine of change. This semantic can be applied to software as well.
Software engines are more than frameworks and libraries that passively do things. Software engines act on their own based on inputs, they are not passive, and usually have their own run loop to do processing on their own volition.
Answer: The closest answer so far, in my opinion, was the one that said it's a marketing thing.
Let's try to think where the use of the word in this context came from.
Certainly car or jet engines aren't called that way because they take input and produce some output. If that were it, then not just functions, but many things in life would be engines.
It's not by coincidence, in my opinion, that a cow is most definitely not an engine, not technically speaking, and not in any sane way of speaking.
Engines use various types of energy and turn it into motion (i.e. a specialized type of energy).
Why do we use it in software? I'd say the most realistic guess is because it simply sounds cool. Why do we call some programmers 'Architects'? Same reason, if you ask me.
Another reason could be that programmers generally like to use metaphors to describe parts of their software, so they could be easier to grasp (because naturally the're often not).
My hunch is that it's bad practice to abuse metaphors in such a way that your application domain is full of them. I think it could be useful when talking to clients, or people who don't know or care about programming. Other than that, simple flow charts and diagrams do the job for me.
Answer: Engine is anything that takes some input, process that input, and delivers something as output. By this definition, a CMS engine is the part which generates the HTTP response on the fly based on the input. A database engine is what takes the query, draws an execution plan, and the executes it, and returns result.
Technically speaking, even a cow is a kind of engine. It takes grass, processes it, and delivers milk.
Don't forget that the term engine is more used in Mechanical engineering, and it does exactly the same.
Another aspect of an engine, is being the core of a system. For example, database engine is the core of any RDMS. However, IDEs to connect to that engine, while taking input and returning result, is not considered an engine.
Comment: +1 for the "milk engine", I was eating yoghurt and nearly choked on it when I read that. ;)
Comment: Your definition would make every function an engine.
Comment: A programmer is an engine too. It takes coffee on input and produces code on output.
Comment: Not every function is a function of its inputs.
Comment: I have to agree. Your explanation sounds a bit blurry and arbitrary.
Comment: Yeah, you're right. Maybe I should emphasize the **most important part of a system, or subsystem** to my definition.
Comment: This isn't the same meaning of "engine" to which the question refers.
Answer: IMHO, an engine is just another word for subsystem, module, or library.  I.E. - one or more components cohesively designed for a specific purpose.
Comment: While engine is implemented typically as a subsystem, the term is commonly used only for subsystems "under the hood". They are not used as synonyms.
Comment: downvoters: please elaborate. That an engine (e.g, the Quake Engine) is a subsystem, a module, or a library is not obviously false.
Comment: The premise of this question is that an engine isn't *just* a subsystem, it's a particular type of subsystem.
Answer: An engine is basically software that can be extended to create something.
For example, in gaming, you'll hear a lot about 'something' game engine. This means a particular software was modified and extended to create a totally unique game.
Answer: At my job the term seems to mean "A single code file in the format XEngine.cs with multiple classes inside that code file, all of which contain nothing but static methods".  I throw up a little inside every time I look at it; might as well be a VB6 module.
I personally wouldn't use the term, it's ambiguous and meaningless (unless, I suppose, you're writing software that simulates an automobile or perhaps a racing game) in most cases; it's like the old "Manager" classes of yore - the name tells you nothing other than "This does a lot of stuff" and usually that means it's a gross violation of SOLID.


markov-switching model and stationarity

Question: To test the structural breaks and to perform markov-switching model  in time series data,  should i have stationary data.
Thank you in advance.
Answer: In this lecture it is stated at chapter 2.3 that "The Markov switching model and its variants discussed in the preceding sections are
only suitable for stationary data". Perhaps you'd like to read into that more thoroughly.
Possible solution:
Since differentiation is allowed, you should also be able to produce an Error-Correcting Model -and its variants- through cointegration if stationarity is not achieved.

markov-switching model and stationarity


What is the magnetic-ordering wave vector?

Question: Like ferromagnetic, antiferromagnetic, the magnetic-ordering are (0,0),(π,0), what is the definition of it? Is there a formula about it?
Answer: The magnetic ordering momentums (wave vectors) are the momentums at which the spin structural factor diverges (in the thermodynamic limit).
Let $\mathbf{S}(\mathbf{r})$ be the spin operator at position $\mathbf{r}$ in the material, the spin structural factor is defined as the Fourier transformation of the spin-spin correlation to the momentum space:
$$\chi(\mathbf{q})=\int\mathrm{d}^d\mathbf{r}_1\mathrm{d}^d\mathbf{r}_2\langle \mathbf{S}(\mathbf{r}_1)\cdot \mathbf{S}(\mathbf{r}_2)\rangle \mathrm{e}^{-\mathrm{i}\mathbf{q}\cdot(\mathbf{r}_1-\mathbf{r}_2)}.$$
Long-range magnetic ordering is signified (and defined) by the divergence of $\chi(\mathbf{q})$ at specific momentums $\mathbf{Q}$, s.t.
$$\chi(\mathbf{q}\to\mathbf{Q})\to\infty.$$
These momentums $\mathbf{Q}$ are called magnetic ordering momentum, and the peak of $\chi(\mathbf{q})$ around $\mathbf{q}\to\mathbf{Q}$ is called the magnetic Bragg peak. The physical significance of the magnetic ordering momentums $\mathbf{Q}$ is that they label the (periodic) patterns of magnetization in the magnet. Elastic neutron scattering (ENS) is a commonly used experimental approach to detect the magnetic ordering momentums.
Comment: Thanks. Is there any trick to know the magnetic ordering wave vector from magnetic order( the spins' array in real space) without calculating the divergent momentum point ? Or how to quickly find the divergent momentum point?
Comment: @ZJX The trick is to imagine the pattern of spin as a wave. Up spin = + and down spin  = -. Then the wave vector can be read off directly by looking at the wave. Or you can measure the wavelength $\lambda$ as the up-spin-to-up-spin distance and calculate the wave number as $2\pi/\lambda$, and the direction of the wave vector is perpendicular to the equal-spin plane (or line).


Raspberry Pi UPS circuit review request + improvements

Question: Based on this question I created the following circuit:

Since I made some (component) changes, first I would like to have a validation of the circuit for long-term usage (and some improvements).
Used components (different from linked circuit):

1N5822 diode
AO3401 P-channel mosfet
3.7 V 2600 mAh 9.62 Wh (protected) Li-Po battery (from another project)
Added a ceramic capacitor and a toggle switch (I hope to the right position)

I made this circuit as a battery backup solution, so 99.9% of the time it will be powered externally, but I don't want the Pi to shut down when there is a power outage.
Here comes my question #1: do Li-Po or Li-Ion batteries last a long time if I connect them to a TP4056 charger constantly?
Currently I'm testing with a Pi 3b+, but I want to upgrade to a Pi 4 later. The circuit works fine when there's external power, but I have issues when I switch to battery powered mode; during boot, the Pi shuts down. If I'm right, the battery protection in the charger module kicks in and saves the battery from over-discharge. When this happens, I get no voltage from the out pins, so I made this modification:

So I connected the battery directly to the MOSFET, because the battery itself has a protection circuit. With this modification I almost get to the desktop, but just before that I get a reboot. If I turn off the external power when the Pi is in idle state, the circuit works until I open an app and then it reboots.
I don't know the C rating of my battery unfortunately; I got it from China. But I have a linear bench power supply.
Question #2: Is it safe to connect it to the TP4056 as a battery replacement to test how much current the boost converter needs in battery mode?
Question #3: I'm thinking about buying a reliable 18650 Li-Ion battery with a high current rate. Do you think it will solve my problem?

As an alternative solution I just connected another charger module in parallel like this (with a second battery), but I saw weird things happen, like they randomly showed the 'charging-done' state when they weren't full. After reconnecting the external power they started charging again. So I would like to stick to one charger/battery if I can.

My question #4: is there a DC/DC boost converter that has a higher efficiency than the MT3608 when using batteries? Also, I don't really need the adjustable output, and actually, that potentiometer looks like a cheap component which is not reliable at all. I just need a stable 5 V output.
Thank you for your help and answers!
Comment: MT3608 seems a little underpowered for that application https://electronics.stackexchange.com/questions/445948/mt3608-gets-very-hot-only-handle-1a
Comment: Question 4 is off-topic because it asks about component recommendations. Consider asking separate, narrowly focused questions to get better results.
Answer: This situation depends on the functionality of the "OUT" pin of the charging module. The following is referencing your original schematic.  Your second schematic is invalid, as the charging circuit is expecting the storage cell to be isolated:
If the "OUT" is always active, then you don't need the MOSFET or the diode.  If it only switches on whenever the input power is no longer present, then you just eliminate the MOSFET(unless the "OUT" pin doesn't have some sort of PN junction on it's output already, which it likely does) or replace it with another diode.
In either case, the charging circuit you are currently using is only specified to output 1 Amp of current, which is WAY too little to power a RaspberryPi.


C++ searching a .txt file for all occurrences of a string and listing all results

Question: I am trying to write a console utility for maintaining a court date list in .txt format so I can keep better and easier track of them. To that effect I started with a piece of code I liked, as a template, and worked my way upwards.
The problem I am encountering is with implementing a search function that goes through the whole file and lists all found occurrences of a string (namely a calendar type date stored in the format DDMMYY).
Sorry if my coding is far from being acceptable, but keep in mind that I'm just a computer passionate lawyer! :)
Without further jib-jabbing here is the code in question:
#include "stdafx.h"

using namespace std;

void name();
void ID();

int main()
{
    system("cls");
    system("color 0F");
    int ch1;
    cout << "1 pentru a introduce un termen nou" << endl;
    cout << "2 pentru a cauta o data" << endl;
    cin >> ch1;
    if (ch1 == 1)
    {
        name();
    }
    else if (ch1 == 2)
    {
        ID();
    }
}

void name()
{
    system("cls");
    string name, salary;
    ofstream worker("termene.txt", ios::app);
    int ID;
    cout << "Termen la instanta : ";
    cin >> name;
    cout << "Data termenului format ZZLLAA : ";
    cin >> ID;
    cout << "Client : ";
    cin >> salary;

    worker << name << " " << ID << " " << salary << endl;
    worker.close();
    cin.get();
    main();
}

void ID()
{
    ifstream worker("termene.txt");
    string name, salary, ID;
    unsigned int curLine = 0;
    string search;
    cout << "Introdu data format ZZLLAA : ";
    cin >> search;
    string line, line2, line3;
    if (worker.is_open())
    {
        while ((getline(worker, line)))
        {

            if (line.find(search, 0) != string::npos)
            {
                cout << "Termen: " << endl;
                cout << line << endl;
                if (line.find(search, 1) != string::npos)
                {
                    getline(worker, line2);
                    cout << "Termen: " << endl;
                    cout << line2 << endl;
                    if (line.find(search, 2) != string::npos)
                    {
                        getline(worker, line3);
                        cout << "Termen :" << endl;
                        cout << line3 << endl;
                    }
                }
                else
                {
                    system("cls");
                    cout << "Bad command or court date." << endl;

                }
            }
            worker.close();
        }
        system("pause");
        main();
    }
}

Comment: `main()` is called in `name()`?
Comment: And what problem exactly is?
Comment: There is no good in storing a string without parsing (escaping) in a file having lines with different fields!
Comment: Could you clarify your question. Please read Stack Overflow standards to keep this powerfull knowledge database efficient.
Answer: Call main() does not look like a good idea. It can potentially create a endless loop which will consume the function call stack. Here the growing is limited by escaping standard input, but it still overloads and may overflow the call stack. Bad practice, avoid it. Create a while loop within main().
The following code will prevent call stack overflow:
int main() {
   while(true) {
      // Branch your code
   }
   return 0;
}

Then, each time a function is called within the while loop, the call stack is released when the function exits. Therefore the stack does not grow forever and you avoid the stack overflow. By the way it is better to provide an exit condition  instead of creating an endless loop.
Answer: Why don't you use fgrep? of course this will only work on Unix.
fgrep COMMAND:

fgrep command is used to search one or more files for lines that match
  the given string or word. fgrep is faster than grep search, but less
  flexible: it can only find fixed text, not regular expressions.

I think you can use UnixUtils to get a few commands on Windows as well.


MongoDB SocketIO pub sub performace over rooms

Question: I am recently trying an application with NodeJS, MongoDB and Socekt IO.
I just heard of SocketIO stores that can be used to cache the messages over the interaction between clients, I am having some doubts on it.

Which is better in terms of performance (SocketIO Rooms vs store pub sub) ?
Will mong.socket.io suffice my requirement?
Is there any other advantages in selection pub-sub or Rooms?
Or am i Completely mistaken ?
Answer: I think that redis would be the better choice here.
(https)://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO
Redis has the great benefit, that its implementation already supports pub/sub functionality native.
(http)://redis.io/topics/pubsub
By the way redis is a good choice for storing session data, because it supports expiring ((http)://redis.io/commands/expire) for keys, which means the database cleans up itself. A great advantage is, that redis supports persistence, which sound ideal for restoring for example messages or session data after a crash.
(http)://redis.io/topics/persistence
And for high workload there is also a way to replicate: (http)://redis.io/topics/replication


How to avoid "bad" requirements

Question: I frequently hear "X% of software project fail due to bad requirements".  The X in that statement has ranged from about 70 to 95.  However, I seldom hear how requirements go bad.  In fact, the statement itself suggests there were actually requirements.
What makes a "bad" requirement?  How can one be avoided?
Comment: When you hear "X% of software project fail" do you see a definition of "fail"?  I'll bet that a project that has had a change in focus could be called "failed".  I think -- without a definition of "fail" -- that this question is unanswerable.
Comment: This is a good question but is a little off base.  Requirements don't go 'bad'.  They are missing, ambiguous, or change unexpectedly (and sometimes uncontrollably depending on the management).
Comment: I would suggest this question to be How to work with Bad Requirements as oppose to How to avoid Bad Requirements coz later would be never the case.
Answer: For successful requirement elicitation you need to

have your customer on site, discuss the requirements, let him explain them to you
the requirements have to be testable, verifiable. Having a list of them, at the end you should be able to go over the list and directly verify their correct implementation on the end-product.
they should have an appropriate level of detail. There exist different type of requirements (goal-level, domain-level, product-level, design-level). Requirements should be classified appropriately.

Usually the problem lies in a lack of communication and understandability between the customer and the developer. Moreover keep in mind that sometimes even the customer itself doesn't exactly have a good picture of what he wants. Therefore discussion, paper prototypes etc. are really important.
This pic is my favourite :)
Comment: @User1: you can create one yourself at http://www.projectcartoon.com/. Have fun ;)
Comment: +1 I've seen this graphic before. I wonder who originally made it.
Comment: The most interesting thing started to happen when my 4-year daughter asked to explain what these pictures mean ;-))).
Comment: Link/image is missing.
Answer: A big portion of agile development methodologies is to accept the fact that requirements WILL change.
Therefore, you should not try to fight that and instead create a process which embraces that.
Answer: Whenever I see those statistics, I am reminded of expensive, top-heavy, waterfall projects where the first release was completed and presented to the customer, who quickly told the project, "This isn't really what I wanted."
That's why most successful projects nowadays are done using an "iterative" model, where the customer is constantly involved in the design process.
In this context, the "requirements" are more loosely defined, and they evolve somewhat as the project progresses.
Answer: In agile, we use the acronym INVEST.  Stories (which stand in for requirements) should be:

I - Independent
N - Negotiable
V - Valuable
E - Estimable
S - Small
T - Testable

Requirements are not an artifact to be handed to you from a mountaintop.  They are a living byproduct of a process of discovery and conversation between you and your customers (or their proxies).
Answer: First off, for a requirement to be valid it needs to be testable. If not, there is no possibility of tracking it, measuring it, reporting on it...  this is a root cause of evil.
How can this situation be avoided?  Make sure that a requirement:

is bounded in both time & resources (e.g. $)
testable

Or else, you are working on an "open loop" and I am sure you can appreciate the consequences.
Note that sometimes requirement come in rather with a "qualitative" nature: it is up to the product manager/team to define a "quantitative" definition for it.
Answer: I think you will find that if you interpret it as follows it will make more sense:
"X% of software projects fail due to bad definition of requirements"
There are a lot of things you can do

Make sure you can actually test the requirement
Make sure that the analyst actually understands what the user really means. Often what the user asks for is not what they actually want.
Make sure the developer understands the requirement. If the developer gets a bad spec and has to make an assumption that turns out wrong, then time is going to be wasted when the programmer has to correct that assumption, on top of the usual bugs.
Make sure the user actually tests that their requirement(s) have been met. Better (discovered) late than never.
Answer: In addition to impossible/impractical or unverifiable requirements, the "bad" likely refers to incorrectly gathered - they requirements you have don't match what is actually needed for the application.  One source of this is that users frequently don't actually know what they need or want.
Answer: Probably they mean "miscommunicated" requirements.
If you think about it, there are many ways you can mis-state requirements, either intentionally or otherwise.  Some ways to deal with the problem:

Realize that the requirements of a system can change continuously.  Otherwise the client will say "yeah, that changed, nobody told you?"
Ask the requirements of several key people - It;s not enough to ask the CEO, and likewise, it's not enough to ask the lower ranks that will actually be using your system.
Make sure there are a handful of people accountable for communicating the requirements to you - These people (no more than 5 in a mid-size project) should have a BIG incentive to give you all the information for a successful implementation.  If you do not have these people, you  are likely to fail as everyone will be too busy to explain things to you, and they will have an incentive not to talk to you, since you will be able to claim X person told you to implement the system they way you did.  You need management's support in creating this group of people.
You need to verify assumptions with other people.  Sometimes you need to ask the same question five different ways.
Be afraid of absolutes... "The sale price cannot be changed" sometimes means "I'd like an supervisor override to be implemented in case a price needs to be changed for the current customer."
Understand the business process as much as possible.  If you are writing a banking application ask to spend a day at the bank to see how people would use the system.  If you deliver a phase of the project do the same thing: Watch the system being used, and be proactive in looking for holes.
Recognize when something is not specified in enough detail and insist on getting it right.  Do mockups, hand drawings, flowcharts, whatever it takes to make sure the source of the requirements and you are on the same page.

These are all just from experience...  I think "bad requirements" really means "bad communications between client and implementor."
Answer: My experience show next possible sources of bad requirements:

Users / clients often don't know what they want.
The possible way to handle this either means have good business analysts who could
perform required analyze or have good ready product which could suit this user (or not).
Analysts can't provide appropriate quality of requirements.
Yes, it happens. Hire better analysts / technology experts but before failure, not
after. Test requirements, analyze usage cases, draw state an sequence diagrams
as early as possible to understand user cases coverage and
so on. In other word this is related to general modelling.
Well, there is possibility of bad translation from marketing requirements / model into
technical specifications.
Design quality problem (implementation can't meet requirements).

What should be done to overcome these problems? Let's allow engineers to provide feedbacks, let's not close requirements and make them flexible as possible. Often even with generally good consistent requirements we face some low level hardware limitation on implementation stage and need to track changes back. From other side let's understand customers, not only technologies. I saw number of projects with large parts of work thrown away just because they look good for developers but not for customers. The better communications with customer you have the lower is possibility of such cases.
My understanding is process should allow flexible requirements change during all the stages  but from other side should make all of this work trackable and limit scope to minimum that required. The problem is to balance between all of this. At least my suggestion is we should move to shortest development cycles to lower all the risks.
Answer: One of the most valuable things that a development organization can do (but is rarely done) is to validate the requirements. Mock up a design, as quickly and inexpensively as possible, and review it with the customers. If at all possible, do it in a way that the review can be structured as a task walkthrough, so developers and users together can walk through use cases and decide whether the proposed design solves the problem. Then, if necessary, do it again.
There's an excellent book on gathering and understanding requirements called User and Task Analysis for Interface Design by JoAnn Hackos and Janice Redish. It's a big book, but it's very readable and filled with practical tips and tools.
Answer: What makes a bad requirement? One that's not there
I see a lot of good answers here about the a bad requirement being one that is miscommunicated or half-baked. And they're probably correct.
But for me one of the worst types of "bad requirement" is the one that's simply missing. I see this time and again in systems. A day after going live, the users say, "Oh, what about XYZ? We really need that." To which the project team responds, "XY what? We've been working on this project for a year and NOW you tell us?"
Why is it bad?
This is a killer because now everyone has to scramble and rush out a solution, not that the average developer needs any help promoting half-assed things to production, but you just know it's going to spell lots of production support for all the poor people this 'solution' is handed over to for maintenance...you know, the ones that didn't get project bonuses.
Again, this is not a bad requirement but one that was never a requirement to begin with. That doesn't mean it's invalid; it most certainly could be critical. But between the rush to get things done and an aggressive project pace, and the fact that we're all humans and we make mistakes, this was overlooked.
How do you avoid it?
You could spend more time up front and hope a sharp subject matter expert picks up the missing gap. A more effective and more costly method is taking the time to engage what some call a "model office" phase. This is like a system test, but designed to simulate real life conditions. The testers aren't just verifying that the system delivers correct output for 1 + 1, but that all its parts work within the context of the business process.
This is a hard sell of course. Many projects will give business analysis and testing the short shrift in order to uphold the almighty metrics of "on time and on budget". But if you want to shake these missing requirements out, you have to let the user run with it. It's then that they'll recognize things they took for granted in a verbal requirements definition session. Agilists would add that this test needs to be done as early and as often as possible to uncover these risks and give the project team time to identify their priorities and make adjustments where warranted.

How to avoid "bad" requirements


Lorentz invariance of Action or Lagrangian?

Question: For what I know, the Action should be Lorentz-invariant in accordance to the axioms of SR. For example for a free particle the Action would be the integral of the proper time, which of course is invariant. Now my doubt is the following: does that mean that the Lagrangian should be Lorentz-invariant as well? In some texts I've read it is often stated that the Lagrangian should be a Lorentz scalar, but how does this assures the Action to be Lorentz-invariant too? For example, once one gets the Lorentz-invariant Lagrangian for the EM field $\mathcal{L}=-\frac{1}{4}F^{\mu\nu}F_{\mu\nu}$, why should its action:$$\int -\frac{1}{4}F^{\mu\nu}F_{\mu\nu} d^4x$$ be Lorentz invariant too? Aren't we integrating a scalar over a specific set of  coordinates which are not Lorentz-invariant?
Comment: Does this answer your question? [Lorentz invariance of the integration measure](https://physics.stackexchange.com/q/47556/)
Comment: The 4-dimensional volume-element $d^4x$ is invariant on Lorentz-transformations
for a similar reason,
why the 3-dimensional volume-element $d^3x$ is invariant on space rotations.
Answer: It's true that the specific set of coordinates that we choose is not Lorentz invariant, but when you integrate over all space, it doesn't matter. One however has to check that the measure is invariant.
It's a nice excercise to show that, under a Lorentz transformation $\Lambda$ the integral transforms are
$$
\int \mathrm{d}^4x\, f(x) \mapsto \int \mathrm{d}^4x' \,|\det\Lambda|\, f(\Lambda x')\,.
$$
Since, by definition, Lorentz transformations have determinant $1$ or $-1$, the integral is invariant.
Answer: Action is constructed as Lorentz invariant dencity integrated over all Minkowski space. $\mathcal{L}=-\frac{1}{4}F^{\mu\nu}F_{\mu\nu}$ is Lorentz invariant dencity.
$d^4x$ is Lorentz invariant measure of integration in rectangular coordinate system.
If you wanna to choose some another coordinate system (curvilinear coordinate system), in general you need consider another measure of integration $d^4y \sqrt{-g}$, $g_{\mu\nu}$ is Minkowski metric in specific coordinates, that one can choose.
$$
g_{\mu\nu}(y) = \eta_{\rho\sigma}\frac{\partial x^{\rho}}{\partial y^{\mu}}\frac{\partial x^{\sigma}}{\partial y^{\nu}}
$$


Why do tail dragging seaplanes not exist?

Question: I am in the process of starting to build an ultralight that will be a tail dragger with floats, and in my conversations with the fellows I'm building it with, we've realized we've never seen a plane like it before. Does this mean we are doing something dumb? Thanks in advance.
Comment: Welcome to aviation.SE! Where the wheels are isn't relevant to an aircraft on floats so I'm not sure what you're asking. There are lots of J-3s and other 'tailwheels' on floats. There's even a DC-3 out there.
Comment: Maybe for floats to be sufficiently buoyant they have to be so big that they can also be statically stable on their own (like having four main wheels and no tailwheel) so there is no need to tail drag, whereas on land you can save weight and just use two main wheels (or skis, but those are still much smaller than floats) but then you need a tail wheel to balance.
Comment: @Pondlife I think the OP is not referring to tail draggers that have been converted to have floats, but planes that literally drag something at the tail through the water.
Comment: @DKNguyen Thanks, that does make sense and John's answer describes exactly that
Comment: Taildraggers are susceptible to ground loops. That's bad enough on land, but much worse on water.
Answer: It's been tried before (there isn't all that much in aviation that hadn't already been tried by WWII).  The tail float means the aircraft CG is aft of the hydrodynamic center of the main floats, and like a tailwheel airplane, the machine will be dynamically unstable, steering wise, when on the step, same as a wheeled taildragger and you will have to prevent it from switching ends with your feet the entire time it's on the water moving.
It could be a real handful to take off and land, especially when there are waves and gusty winds (and seaplanes do have to cope with crosswinds from time to time).  You have to make the rear float steerable, and you have to give it enough lateral bite in the  water to have good leverage in steering the thing when below the speed that the air rudder is working.  I don't think there is any drag advantage to 3 smaller floats vs 2 larger ones.
Traildragger advantages, like prop clearance, aren't that much of a benefit here, and since the conventional float configuration is so much easier to control, why bother with a taildragger float setup?
Ideas come along for new configurations all the time, but if you dig enough you find that somewhere in the distant past it's already been tried, and while it may have been functional, the "old" conventional way that everybody uses has been around so long because it's simply what works best.
Comment: When you start the takeoff run on a normal floatplane, you drive the nose up high with full aft elevator at the start until you start to come up on the step.  So there isn't really the need to have the nose pointing up all the time.  Also the level attitude when displacement taxiing makes you much less susceptible to winds.  Tail low slow taxiing on the water adds a whole new flavour to the hazards.  Water impingement on the  prop on seaplanes (which does a lot of damage over time) is mostly a problem of taking off in significant chop when you are just getting started.
Comment: @DKNuygen The floats have to be big anyway, and because they're long the CoG nearly always lies within their footprint.  You could, if you wanted, have a landplane with four wheels (roughly where the ends of the floats would be) on long legs which would sit almost horizontal at rest.  But why bother? It'd be heavier than it needs to be.
Comment: So it kind of begs the question...why not take the same approach on land? Why is prop clearance not a problem on float planes anyways?
Comment: On the airplane pictured, it looks like a wave could rip the rudder right off. Eeeek!


Are there valid reasons for using a hardware encryption device (e.g. USB encryption dongle/keystore) over using a software library?

Question: If your application needs to encrypt/decrypt data (for various reasons), are there any reasons why you would use a hardware device (e.g. a USB encryption device - like a Marx CryptoBox) instead of using a software encryption library (like .net Cryptography or writing your own) and keep your keys in a safe key store?
I am looking for some objective views on this matter.

To narrow down the question posed:  what would your opinion be if the system that used the usb encrpytion dongle was housed in a physically secure server vault and there was only one system in existence (i.e. its not a software product that is distributed and run on many desktops) ?  In very simplistic terms, the purpose of the above system is to validate (decrypt and compare) a piece of incoming encrypted data.

Thanks so far for your great answers!
Answer: It's not about what's more secure because nothing is 100% bullet proof. It's a question on "how to make it as difficult as possible"
You could see it from this point: If you store keys on the computer, they're there 24/7. If my pair of keys is on an external device, the keys are only accessible while attached to the device. ==> You reduce the timeframe in which somebody else can copy your keys. It's much easier to access something if physical access is not needed.
Think of online banking: Many banks have added "external" ways of authentifcation such as Tan / Tac /tanSMS/ tokengenerators etc. etc. Neither of those is secure for itself: I can steal your login password, I can steal your mobile phone, I can steal your Tac/Tan list and so on. But chaces are very low that I can steal all necessary elements at once => All pieces of the puzzle together create a quite secure solution.
Also think of these factors:

money: Do you really need a 70$ token-based-protection for your 100$ app?
time: I would say that software based systems are faster
relevance: Does it make sense to provide my apps with such a complex protection system
Comment: The $100 app may very well be handling $100.000.000 transactions. If it's Excel, for instance.
Comment: "more secure" does in no way imply that it is bullet proof. More secure by definition implies that it's more difficult than the other one. Just saying :)
Comment: What would your opinion be if the software resided in a secure environment (server vault) and used the device purely to store its keys as well as the device's hardware encryption capabilities?  The purpose of the system is to validate (decrypt and compare) an incoming encrypted piece of data.
Answer: Yes, there are.
For one thing, this allows you to transmit the private keys physically through secure channels, rather than trusting a network.
For another, if you have to move around between a lot of systems that aren't nessecarily all networked, the USB key is just way more convienent. That's why the military uses a system very much like that (EKMS). They don't use USB, but they use little dongles that look like big plastic keys. The idea is the same, but USB didn't exist back in the early 90's when they were developing this.

(note: Its kinda scary how complete that Wikipedia article is. When I was working on the KP I was told we were allowed to put acronyms like FIREFLY on our resumes, but we weren't allowed to tell anybody what they meant.)
Answer: A hardware key allows use to constrain usage of the software to a machine where the key is plugged in.
With software encryption it would be easier to copy the software to many machines and run it many times in parallel.
Answer: I think it depends on the purpose/usage of your application.  If you're developing a product for a high security need, such as banking or government, a hardware addition to your solution could be entirely appropriate.  You have to consider the additional costs that a hardware solution would add to the project.  Not just in the initial development, but the ongoing hardware maintenance costs that would be required after development is complete.  From my perspective, having worked banking in the past, there were very, very few solutions where we felt that a hardware solution was worth the additional costs it would entail.


Display Out of Stock Configuration not working in magento1.9

Question: I have a configurable product with 3 simple products associated to it that were added via REST.
I have set the "Display Out of Stock Products" option in System->Configuration->Inventory to "Yes"
Configurable Product

Manage Stock => Yes; Use Config Settings
Stock Availability => In Stock

Simple Product 1

Manage Stock => Yes; Use Config Settings
Stock Availability => In Stock
Quantity => 5

Simple Product 2

Manage Stock => Yes; Use Config Settings
Stock Availability => In Stock
Quantity => 1

Simple Product 3

Manage Stock => Yes; Use Config Settings
Stock Availability => Out of Stock
Quantity => 0

Why won't Product 3 show on my website (front-end)? It displays when I set the product to "In Stock".
EDIT: So I got it working in a roundabout way. I sent the URL twice and it now displays as "backordered". Full disclosure - I am using someone else's code and trying to make this work. Here is the important part of the URL (minus my website info)

https://****/index.php/****/****/push?&prodcode=29543&sku=E29543&name=Ethyl%206-(trifluoromethyl)pyridazine-3-carboxylate&name1=Ethyl%206-(trifluoromethyl)pyridazine-3-carboxylate&name2=&inchi=&inchi_key=&formula=C8H7F3N2O2&mass=220.148&cas=1192155-06-8&purity=&mdl=&msds=&total_stock=5.1g&images[0]=29543.png&categories[0]=&categories[1]=&categories[2]=&categories[4]=&categories[5]=&categories[6]=&options[1][price]=940&options[1][code]=E29543G1&options[1][unit]=G1&options[1][size]=1g&options[1][instock]=yes&options[1][stock]=5&options[2][price]=2810&options[2][code]=E29543G5&options[2][unit]=G5&options[2][size]=5g&options[2][instock]=yes&options[2][stock]=1&options[3][price]=4610&options[3][code]=E29543G10&options[3][unit]=G10&options[3][size]=10g&options[3][instock]=yes&options[3][stock]=0&options[4][price]=&options[4][code]=&options[4][unit]=&options[4][size]=&options[4][instock]=yes&options[4][stock]=&options[5][price]=&options[5][code]=&options[5][unit]=&options[5][size]=&options[5][instock]=yes&options[5][stock]=&options[6][price]=&options[6][code]=&options[6][unit]=&options[6][size]=&options[6][instock]=yes&options[6][stock]=%3F

Why would this work the 2nd time but not the first??? Not sure if it's relevant, but it would seem the previous developer created a custom data controller.
Comment: > https://****/index.php/****/****/push?&prodcode=29543&sku=E29543&name

Unfortunately, this is custom solution using a URL and not the Rest API as mentioned prior. (http://devdocs.magento.com/guides/m1x/api/rest/introduction.html#RESTAPIIntroduction-CreateasimpleproductasanAdminuserwithOAuthauthentication)
Answer:
I have set the "Display Out of Stock Products" option in
  System->Configuration->Inventory to "Yes"

Please refresh the indexes/cache in the admin.
Then, when putting up the product via REST, try the following:
"stock_data" : {
      "qty" : 0,
      "is_in_stock" : 1,
      "use_config_manage_stock" : 1,
}
Comment: Josh - The way I have it configured, shouldn't it display the product even if "is_in_stock": 0 ?