Odds ratio with zero cell counts

Question: I am trying to run a binomial GLM but have some zero cell counts resulting in estimates of +/- infinity. In the text book  "An Introduction to Categorical Data Analysis" Agresti A. (1996) it suggests to add a small constant to the zero cells - which I also came across in the more accessible paper here.
However, when I try this in R I get a warning message of:

non-integer counts in a binomial glm!.

So:

I'm not sure if this is the right thing to do?
Is there a better way to deal with zero cell counts?

Data:
<code>A<-c(10,10,10,10,10,10,19,19,19,19,19,19)
B<-c("0","1","2","0","1","2","0","1","2","0","1","2")
C<-c("-ve","-ve","-ve","+ve","+ve","+ve","-ve","-ve","-ve","+ve","+ve","+ve")
Dead<-c(1,1,27,0,6,18,2,10,23,0,14,21)
Alive<-c(29,32,2,22,19,4,28,22,3,20,11,0)
mod2<-glm(cbind(Dead,Alive)~A*B*C, family=binomial)
</code>
Comment: This doesn't help with the zero cell-count problem but I noticed that you create the object `gaf` but never use it anywhere in the code; the `glm` call only looks at the vectors `Dead` and `Alive` etc, rather than the `data.frame` where you collected them together.
Comment: Thanks - I used it later but forgot to remove it for this question
Answer: I think the confusion is stemming from the fact that in a binomial glm formula the mean proportion of deaths cannot be zero.  It's ok for some of your observed proportions of deaths to be zero.  The fitting procedure isn't simply taking the log of Dead/Alive, it's finding the maximum likelihood estimates for the following model:
\begin{eqnarray}
Y_i &\sim& Bin\left( m_i, \mu_i \right), ~~~~\text{for}~~ i=1,...,n, \\
\log\frac{\mu_i}{1-\mu_i} &=& \beta_0 + \beta_1 A_i + \beta_2 B_i + \beta_3 C_i.
\end{eqnarray}
I know your A, B and C covariates are actually factors but the story is the same.  If your observed response $y_i$ is zero, your modelling assumption is that the probability of death $\mu_i$ in this case is probably very low, so that in a random sample of $m_i$ observations you may well see no deaths (which is in fact what you've observed here).
Hope that helps.
Comment: I'm not sure I fully understand - because it seems to contradict what I have read in the previous references which indicate the observed proportions of deaths cannot be zero. I first notice the problem when I tried to calculate the CIs for the proportions and the CIs for the 0 deaths groups spanned from 0-100%.
Comment: I'm not sure what references you've been looking at.  How about this from Prof. Brian Ripley at Oxford: http://r.789695.n4.nabble.com/glm-binomial-with-zero-proportions-td794849.html
Comment: He references the text book 'Modern Applied Statistics with S' 4th edition.
Comment: Do you mean intervals for each $\mu_i$?  How are you calculating them?
Comment: Thanks, I'll try and get hold of it. The references I was referring to are in my original post - including a text book example. Also the following link explains the problem http://rfd.uoregon.edu/files/rfd/StatisticalResources/lgst_zero.txt  - Sorry I don't wan't this to sound rude, it is just difficult to know what to do when there is seemingly conflicting information (it could be me misunderstanding of course)
Comment: I'm using the predict function in R - p<-predict(mod2,newdata=gaf,se.fit=TRUE)
up<-with(p,fit+se.fit)
low<-with(p,fit-se.fit)
invLink<-family(mod2)$linkinv
av2<-with(p,invLink(fit))
upr<-invLink(up)
lwr<-invLink(low)
Comment: The odds ratios calculated at the top of the link refer to observed proportions, so you will get a different answer by fitting a GLM (you are making different modelling assumptions).  I think the problem is more to do with the model you've fitted - some of the confidence intervals for your parameters are enormous, and this can translate to large prediction intervals for your proportions.  If you fit a simpler model instead for example mod3 = glm(cbind(Dead,Alive)~A+B+C, family=binomial), then you get much tighter prediction intervals (including for the 0 death groups).
Comment: Thanks again...but, if I simplify the model the predicted values  differ from the observed - which 1. suggest a bad model (due to zero counts?) and 2. means I can't calculate the CIs for the observed data
Comment: I'm not suggesting you use the exact model I suggested, just that the original model with all the interactions results in many statistically insignificant parameters, therefore very large confidence intervals and therefore you can get large prediction intervals.  If you have zero counts then in a binomial glm there is no way your predicted proportion can be the same as observed (see my answer above).  It's mathematically impossible.  Zero counts may well make your model harder to fit, that is just a consequence of not having a large sample so small prob. death results in no observed deaths.
Comment: You can see that the prediction interval for the case where you have 100% deaths also spans 0-1.  So it's not zero counts, it's cases where the prob. death is either very high or very low, so that in a small sample you may see all deaths or no deaths, which means you have to (to some extent) rely on the other samples to fit your model (since it's hard to know the exact prob. death in these cases - how high is very high?).  I hope that makes things a little clearer.
Comment: OK, both 0% or 100% cause a problem and make it difficult to analyse a full model - which is why it has been suggested by the refs in my original post to add a small constant (presumably take something away from the 100%). But it seems this is not a very recognised thing to do. Thanks again.

Odds ratio with zero cell counts


Como limitar o numero de casas decimais de uma propriedade tipo FLOAT com Data Annotation na Model da Tabela utilizando EF do C# .NET?

Question: <code>    public class Alunos
    {
        public int Id { get; set; }
        public string Nome { get; set; }

        ["format(0.00F)"]  <-  QUAL A SINTAXE PARA FAZER ESSA FORMATAÇÃO NA MODEL DO EF
        public float Media { get; set; }
    }
</code>
Preciso truncar as casas decimais para evitar que o Entity Framework salve no Banco com o número máximo de de casas decimais permitida por um campo float.
Exemplo se eu envio o valor 1.12
Ele salva no banco o valor 1.1199999255845669
Obs. se eu fizer o insert direto no Banco ele inseri apenas duas casa decimais 1.12, esse problema só acontece quando o EF persiste os dados.
Comment: Qual versão do EF?
Comment: https://stackoverflow.com/questions/9558549/validate-decimal-value-to-2-decimal-places-with-data-annotations
Comment: Por que não usar `decimal` ao invés de `float` nesse caso?
Comment: @BrunoWarmling Entity Framework 6
Comment: @NatanFernandes a resposta que eu obtive foi para economia de memória ja que o Decimal utiliza 4x mais memoria que o float

https://docs.microsoft.com/pt-br/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types
Comment: Infelizmente se você usar o float vai acabar perdendo essas casas decimais
Answer: A solução encontrada foi convencer a equipe a Utilizar o Decimal, como questionou o Natan Fernandes
<code>Decimal(6.5) //para formato 1.12345
Decimal(10.2) //para formato 12345678.12
</code>


GARCH forecast of series (in R) seems too high

Question: I am wondering why the mean of my model is so high leading to a high forecast of the time series data. I included a linear regression in the external regressors as there is a clear downward trend.
I think I may have specified the linear regression incorrectly in the external regressor. Should I use instead seq(112,1,by=-1) as there is a negative correlation? This gives me a mean in my model of 3 which seems more reasonable based on previous values.
Model specification using rugarch:
set.seed(1)
garchspec_ged = ugarchspec(variance.model=list(model="sGARCH", garchOrder=c(1,1)),
        mean.model=list(armaOrder=c(0,0), external.regressors = matrix(seq(1,112,1)),
        distribution.model="ged")
garchfit_ged <- ugarchfit(data = yts, spec = garchspec_ged)

summary output:
*---------------------------------*
*          GARCH Model Fit        *
*---------------------------------*

Conditional Variance Dynamics
-----------------------------------
GARCH Model : sGARCH(1,1)
Mean Model  : ARFIMA(0,0,0)
Distribution    : ged

Optimal Parameters
------------------------------------
        Estimate  Std. Error  t value Pr(>|t|)
mu      17.58611    1.149166 15.30337 0.000000
mxreg1  -0.12230    0.014254 -8.57981 0.000000
omega    0.67124    0.760973  0.88208 0.377736
alpha1   0.08434    0.053123  1.58763 0.112369
beta1    0.87592    0.067514 12.97392 0.000000
shape    2.19405    0.493805  4.44316 0.000009

Robust Standard Errors:
        Estimate  Std. Error  t value Pr(>|t|)
mu      17.58611    1.713427  10.2637 0.000000
mxreg1  -0.12230    0.020444  -5.9822 0.000000
omega    0.67124    0.548106   1.2246 0.220709
alpha1   0.08434    0.047150   1.7888 0.073651
beta1    0.87592    0.052031  16.8344 0.000000
shape    2.19405    0.529629   4.1426 0.000034

LogLikelihood : -337.3251

Information Criteria
------------------------------------

Akaike       6.1308
Bayes        6.2764
Shibata      6.1254
Hannan-Quinn 6.1899

Weighted Ljung-Box Test on Standardized Residuals
------------------------------------
                        statistic p-value
Lag[1]                     0.4786  0.4890
Lag[2*(p+q)+(p+q)-1][2]    0.5086  0.6896
Lag[4*(p+q)+(p+q)-1][5]    0.7354  0.9160
d.o.f=0
H0 : No serial correlation

Weighted Ljung-Box Test on Standardized Squared Residuals
------------------------------------
                        statistic p-value
Lag[1]                    0.01878  0.8910
Lag[2*(p+q)+(p+q)-1][5]   0.85125  0.8923
Lag[4*(p+q)+(p+q)-1][9]   3.64610  0.6493
d.o.f=2

Weighted ARCH LM Tests
------------------------------------
            Statistic Shape Scale P-Value
ARCH Lag[3]   0.08649 0.500 2.000  0.7687
ARCH Lag[5]   1.61721 1.440 1.667  0.5620
ARCH Lag[7]   3.16708 2.315 1.543  0.4826

Nyblom stability test
------------------------------------
Joint Statistic:  1.2853
Individual Statistics:
mu     0.06942
mxreg1 0.09061
omega  0.04553
alpha1 0.19344
beta1  0.12067
shape  0.11941

Asymptotic Critical Values (10% 5% 1%)
Joint Statistic:         1.49 1.68 2.12
Individual Statistic:    0.35 0.47 0.75

Sign Bias Test
------------------------------------

Adjusted Pearson Goodness-of-Fit Test:
------------------------------------
  group statistic p-value(g-1)
1    20     27.64      0.09055
2    30     44.43      0.03341
3    40     50.86      0.09675
4    50     53.18      0.31646

Forecast of series:
garchfit_ged.fcst = ugarchforecast(garchfit_ged, n.ahead=12)
plot(garchfit_ged.fcst, which = 1)

Forecast with model including seq(112,1,by=-1) in the regressor instead:
Answer: Given your model, your forecast is reasonable whichever of the two ways you specify the external regressor (I think the two ways are equivalent; note that the true value of $\mu$ is defined relative to how you specify the external regressor). You just extrapolate the negative trend, having adjusted for the mean. The mean is high relative to the few preceding observations, so it looks like the forecast is high.
The question is, do you think the model for the conditional mean (that says the next observation will follow the negative trend, adjusted for the overall mean) is fine? If yes, then your forecast is also fine as it follows from the model. If no, consider some other model for the conditional mean, e.g. that the next observation follows the trend, adjusted for the current observation rather than the mean: mean.model=list(armaOrder=c(1,0), external.regressors = matrix(seq(1,112,1)). (In many cases, the simple model that you have may be hard to beat. But only you know the specifics of the phenomenon you are modelling, so maybe there is a better model.)
Comment: Thanks for the reply. The last point is actually 1, I think that line is to link the actual last point to the prediction as the mean is so high. I have uploaded the forecast when I use the negative sequence in the external regressor.  Why are these forecasts be so different? One reason for creating the model is to show the value is decreasing over time after accounting for heteroscedasticity, if I change the regressor to the negative sequence then the coefficient for the linear trend is positive which doesn't make sense to me.
Comment: @user553480, I was too hasty with interpreting the mean model. See my updated answer. The fact that your external regressor changes from $x$ to $-x$ should not change the forecast. I wonder what else is going on...
Comment: Based on my ACF/PACF plot an ARMA model doesn't make sense. When I use the negative sequence in the external regressor my coefficient for  mxreg1 is 0.122298, is that logical for data with a decreasing linear trend? It also gives a mean of 3.8 which I suppose does not seem right with the whole data. I would be happy to provide the vector of data if you wanted to try it out yourself using the r code :)
Comment: @user553480, $\mu$ has to be understood relative to the mean of the external regressor. It does not have to make much sense on its own. What has to make sense as the level of the series is $\mu+mxreg1\times t$ where $t$ is the trend. $3.8+0.122298\times (122-t+1)$ has to make sense as the level for time period $t$.
Comment: that would leave me with a decreasing sequence from month 1 to 112 which makes sense based on the data, so I think that is okay to use as my model?
Comment: I cannot guarantee this for you :) But if (1) you find the forecast suprising and given that (2) the forecast is a consequence/implication of the model, perhaps you should consider another model.
Comment: Okay noted, thank you for your help Richard


Error detected from emxArray_uint8_T: .text+0xc1): undefined reference to `emxInit_uint8_T1(emxArray_uint8_T**, int)'?

Question: I write a matlab function that calculates entropy of gray scale image: "score=score_entropy(image)" then use matlab coder to convert it to c++
c++ code:

<code>#include "mex.h"
#include "math.h"
#include "score_entropy.cpp"
#include"score_entropy_emxAPI.cpp"

class image
{
public:
unsigned char *im;
int h1,w1;
voide scoreimage();
}
void image::scoreimage( )
{ emxArray_uint8_T* I=emxCreateWrapper_uint8_T(im,h1,w1);
double ent=score_entropy(I);
 emxDestroyArray_uint8_T(I);
}
void mexFunction( int nl, mxArray *pl[], int nr, const mxArray *pr[] )
{
image.im = ( unsigned char*) mxGetData(pr[0]);
image. h1 = (int) imgSize[0];
 image. w1 = (int) imgSize[1];
image.scoreimage();
}
</code>
The definition of score_entropy.cpp :
double score_entropy( const emxArray_uint8_T *im)
I found a list of errors :
<code>(.text+0xc1): undefined reference to `emxInit_uint8_T1(emxArray_uint8_T**, int)'
(.text+0xfb): undefined reference to `emxEnsureCapacity_uint8_T(emxArray_uint8_T*, int)'
(.text+0x16c):undefined reference to `emxInit_uint8_T(emxArray_uint8_T**, int)'
..
..
(.rdata$.refptr.rtMinusInf[.refptr.rtMinusInf]+0x0):
undefined reference to `rtMinusInf'
collect2.exe: error: ld returned 1 exit status
</code>
can any one help to correct this code?
Answer: Compile and link all of the generated CPP files together and that will fix the linker issue. It looks like you're just linking a few of them.
Also make sure you call the generated <code>_initialize</code> function before you call your generated entry point.
Comment: I call score_entropy_initialize() before call entry point but still have same errors plus (.text+0x107c):
undefined reference to `score_entropy_initialize()'. what can I do else and how link all files.


Trazer combo selecionada preenchida manualmente

Question: Estou com dificuldade de criar uma combo de "Habilitado" e "Desabilitado" passando o valor do banco para trazê-la preenchida. No meu controller, criei o seguinte código:

Na minha View, seguinte código:

O resultado é que, com o código rodando, sempre vai aparecer "Habilitado" lá na combo.
Comment: O `gestor.Status` é um bool ou um int?
Comment: É um bool @FilipeOliveira
Answer: Como o gestor.Status é um bool, ou você muda o Value para bool:
lista.Add(new SelectListItem { Text = "Habilitado", Value = "True" });
lista.Add(new SelectListItem { Text = "Desabilitado", Value = "False" });

Ou converte o seu gestor.Status para inteiro:
ViewBag.Status = new SelectList(lista, "Value", "Text", Convert.ToInt32(gestor.Status));

Comment: Mudei o Value para "True" e "False". Na hora de salvar, funciona perfeitamente, passando o valor correto, conforme o selecionado. Porém, na hora de editar, ele não traz o selecionado, conforme o banco... Tem alguma ideia do que pode ser? Atualmente estou passando a ViewBag assim:  `ViewBag.Status = new SelectList(lista.AsEnumerable(), "Value", "Text", gestor.Status);`
Comment: @IgorMacedo, o `gestor.Status` está vindo False mesmo? Por alto, não consigo imaginar qual seja o problema, se o boleano estiver correto, ele irá exibir correto também
Comment: @FilipeOliveria, debuguei e passa pelo controller com o valor certo. Porém, quando vou editar, passa pelo controller com valor certo, porém na hora de visualizar, ele não marca a que está selecionada, como se estivesse sem o 4º parâmetro: `gestor.Status`
Comment: @IgorMacedo, a lista está sendo preenchida da mesma forma? O value está correto?
Comment: @FelipeOliveira, está sim. Na hora de inserir um novo registro, ele insere conforme selecionei no combo. Só na hora de editar que ele não traz o selecionado :/
Comment: @IgorMacedo, o ideal seria ver como está o código. Poderia editar sua pergunta mostrando um exemplo de como está as actions de inserir e editar e suas respectivas views? E coloca em texto mesmo, não em imagem.


How to export embeded file which from excel using POI?

Question: I have made a java basic program written below, which is making 3 kind of files (ppt,doc,txt) embedded in Excel sheet using Apache POI. Now this file I want to Export with its original format. How to do this?
Reference link is Embed files into Excel using Apache POI.
I have made program from this link.
In short I want export functionality on Embedded file.
I have tried above problem using give below code but it not working for exporting embedded file in excel sheet:
Here this is the code which is tried  to solve:
public static void main(String[] args) throws IOException {
    String fileName = "ole_ppt_in_xls.xls";
    ReadExcel(fileName);
}

 public static void ReadExcel(String fileName) throws IOException {
    FileInputStream inputFileStream = new FileInputStream(fileName);

    POIFSFileSystem fs = new POIFSFileSystem(inputFileStream);
    HSSFWorkbook workbook = new HSSFWorkbook(fs);

    for (HSSFObjectData obj : workbook.getAllEmbeddedObjects()) {
        // the OLE2 Class Name of the object
        String oleName = obj.getOLE2ClassName();
        System.out.println(oleName);
        if (oleName.equals("Worksheet")) {
            System.out.println("Worksheet");
            DirectoryNode dn = (DirectoryNode) obj.getDirectory();
            HSSFWorkbook embeddedWorkbook = new HSSFWorkbook(dn, fs, false);

        } else if (oleName.equals("Document")) {
            System.out.println("Document");
            DirectoryNode dn = (DirectoryNode) obj.getDirectory();
            HWPFDocument embeddedWordDocument = new HWPFDocument(dn, fs);
        } else if (oleName.equals("Presentation")) {
            System.out.println("Presentation");
            DirectoryNode dn = (DirectoryNode) obj.getDirectory();
            SlideShow embeddedPowerPointDocument = new SlideShow(
                    new HSLFSlideShow(dn, fs));
        } else if (oleName.equals("Presentation")) {
            System.out.println("Presentation");
            DirectoryNode dn = (DirectoryNode) obj.getDirectory();
            SlideShow embeddedPowerPointDocument = new SlideShow(
                    new HSLFSlideShow(dn, fs));
        }else {
            System.out.println("Else part ");
            if (obj.hasDirectoryEntry()) {
                System.out.println("obj.hasDirectoryEntry()"+obj.hasDirectoryEntry());
                // The DirectoryEntry is a DocumentNode. Examine its entries

                DirectoryNode dn = (DirectoryNode) obj.getDirectory();
                for (Iterator entries = dn.getEntries(); entries.hasNext();) {
                    Entry entry = (Entry) entries.next();
                    System.out.println(oleName + "." + entry.getName());
                }
            } else {
                System.out.println("Else part 22");
                byte[] objectData = obj.getObjectData();
            }
        }
    }

}

Output screen of above program:
So ,how to exporting functionality implement?
Comment: I will repeat myself: **What have you tried so far? (regarding the exporting functionality)**. Did you tried something and it doesn't work? Or do you want us to do it for you? Look at the voted questions (green marked on right of this page), see that the users there tried something, and asked what is wrong with the code below, and good people answered.
Comment: Not sure if [this](http://pastebin.com/8MdM7zh5) helps you. I wrote it as an example of how to extract various embeddings a while ago
Comment: @besciualex: here the link http://stackoverflow.com/questions/16910503/embed-files-into-excel-using-apache-poi. I used this code for making a embeded excel file, Now exporting embedded file how to do?
Comment: @ besciualex: Actually I am confusing how to do it?
Comment: @kiwiwings: Thank you, I will check and try apply it.
Comment: @kiwiwings: I got error at { return ClassID.OLE10_PACKAGE.equals(clsId);}  OLE10_PACKAGE cannot be resolved or is not a field. how to solve it. i google it but does not solve.
Comment: @besciualex: I added code from I tried export functionality but not working.
Comment: @kiwiwings:please help me.for this link  http://stackoverflow.com/questions/16910503/embed-files-into-excel-using-apache-poi embedded file attach via image, double click on image  then open embedded file on excel.so, how to export this kind of file. can I apply export functionality on image or directly apply for file object.
Comment: @SIT consider in proper way
Answer: This is partly a duplicate of How to get pictures with names from an xls file using Apache POI, for which I've written the original paste.
As per request, I've added also an example of how to add and embedding with the help of a OLE 1.0 packager - in the meantime I've added the code to POI, so this easier now. For the OOXML based files have a look into this answer.
So the code iterates through all shapes of the DrawingPatriarch and extracts the pictures and embedded files.
I've added the full code - instead of a snippet - to this answer, as I expect the next "why can't I export this kind of embedding" to come up soon ...
package poijartest;

import java.awt.Color;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.Closeable;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Method;
import java.net.URL;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.imageio.ImageIO;

import org.apache.poi.ddf.EscherComplexProperty;
import org.apache.poi.ddf.EscherOptRecord;
import org.apache.poi.ddf.EscherProperty;
import org.apache.poi.hpsf.ClassID;
import org.apache.poi.hslf.usermodel.HSLFSlideShow;
import org.apache.poi.hssf.usermodel.HSSFClientAnchor;
import org.apache.poi.hssf.usermodel.HSSFObjectData;
import org.apache.poi.hssf.usermodel.HSSFPatriarch;
import org.apache.poi.hssf.usermodel.HSSFPicture;
import org.apache.poi.hssf.usermodel.HSSFPictureData;
import org.apache.poi.hssf.usermodel.HSSFShape;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFSimpleShape;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.openxml4j.opc.PackagePart;
import org.apache.poi.poifs.filesystem.DirectoryNode;
import org.apache.poi.poifs.filesystem.Entry;
import org.apache.poi.poifs.filesystem.Ole10Native;
import org.apache.poi.poifs.filesystem.Ole10NativeException;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.apache.poi.sl.usermodel.AutoShape;
import org.apache.poi.sl.usermodel.ShapeType;
import org.apache.poi.sl.usermodel.Slide;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.usermodel.WorkbookFactory;
import org.apache.poi.util.IOUtils;
import org.apache.poi.xssf.usermodel.XSSFDrawing;
import org.apache.poi.xssf.usermodel.XSSFPicture;
import org.apache.poi.xssf.usermodel.XSSFPictureData;
import org.apache.poi.xssf.usermodel.XSSFShape;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.openxmlformats.schemas.drawingml.x2006.spreadsheetDrawing.CTPicture;

/**
 * Tested with POI 3.16-beta1
 *
 * 17.12.2014: original version for
 *    http://apache-poi.1045710.n5.nabble.com/How-to-get-the-full-file-name-of-a-picture-in-xls-file-td5717205.html
 *
 * 17.12.2016: added sample/dummy data for
 *    https://stackoverflow.com/questions/41101012/how-to-export-embeded-file-which-from-excel-using-poi
 */
public class EmbeddedReader {

    private File excel_file;
    private ImageReader image_reader;

    public static void main(String[] args) throws Exception {
        File sample = new File("bla.xls");
        getSampleEmbedded(sample);
        ImageReader ir = new ImageReader(sample);

        for (EmbeddedData ed : ir.embeddings) {
            System.out.println(ed.filename);
            FileOutputStream fos = new FileOutputStream(ed.filename);
            IOUtils.copy(ed.is, fos);
            fos.close();
        }

        ir.close();
    }

    static void getSampleEmbedded(File sample) throws IOException {
        HSSFWorkbook wb = new HSSFWorkbook();
        int storageId = wb.addOlePackage(getSamplePPT(), "dummy.ppt", "dummy.ppt", "dummy.ppt");
        int picId = wb.addPicture(getSamplePng(), HSSFPicture.PICTURE_TYPE_PNG);
        HSSFSheet sheet = wb.createSheet();
        HSSFPatriarch pat = sheet.createDrawingPatriarch();
        HSSFClientAnchor anc = pat.createAnchor(0, 0, 0, 0, 1, 1, 3, 6);
        HSSFObjectData od = pat.createObjectData(anc, storageId, picId);
        od.setNoFill(true);
        wb.write(sample);
        wb.close();
    }

    static byte[] getSamplePng() throws IOException {
        ClassLoader cl = Thread.currentThread().getContextClassLoader();
        URL imgUrl = cl.getResource("javax/swing/plaf/metal/icons/ocean/directory.gif");
        BufferedImage img = ImageIO.read(imgUrl);
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ImageIO.write(img, "PNG", bos);
        return bos.toByteArray();
    }

    static byte[] getSamplePPT() throws IOException {
        HSLFSlideShow ppt = new HSLFSlideShow();
        Slide<?,?> slide = ppt.createSlide();

        AutoShape<?,?> sh1 = slide.createAutoShape();
        sh1.setShapeType(ShapeType.STAR_32);
        sh1.setAnchor(new java.awt.Rectangle(50, 50, 100, 200));
        sh1.setFillColor(Color.red);

        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ppt.write(bos);
        ppt.close();

        POIFSFileSystem poifs = new POIFSFileSystem(new ByteArrayInputStream(bos.toByteArray()));
        poifs.getRoot().setStorageClsid(ClassID.PPT_SHOW);

        bos.reset();
        poifs.writeFilesystem(bos);
        poifs.close();

        return bos.toByteArray();
    }

    public EmbeddedReader(String excel_path) throws IOException {
        excel_file = new File(excel_path);
        image_reader = new ImageReader(excel_file);
    }

    public String[] get_file_names() {
        ArrayList<String> file_names = new ArrayList<String>();
        for (EmbeddedData ed : image_reader.embeddings) {
            file_names.add(ed.filename);
        }
        return file_names.toArray(new String[file_names.size()]);
    }

    public InputStream get_stream(String file_name) {
        InputStream input_stream = null;
        for (EmbeddedData ed : image_reader.embeddings) {
            if(file_name.equals(ed.filename)) {
                input_stream = ed.is;
                break;
            }
        }
        return input_stream;
    }

    static class ImageReader implements Closeable {
        EmbeddedExtractor extractors[] = {
            new Ole10Extractor(), new PdfExtractor(), new WordExtractor(), new ExcelExtractor(), new FsExtractor()
        };

        List<EmbeddedData> embeddings = new ArrayList<EmbeddedData>();
        Workbook wb;

        public ImageReader(File excelfile) throws IOException {
            try {
                wb = WorkbookFactory.create(excelfile);
                Sheet receiptImages = wb.getSheet("Receipt images");
                if (wb instanceof XSSFWorkbook) {
                    addSheetPicsAndEmbedds((XSSFSheet)receiptImages);
                } else {
                    addAllEmbedds((HSSFWorkbook)wb);
                    addSheetPics((HSSFSheet)receiptImages);
                }
            } catch (Exception e) {
                // todo: error handling
            }
        }

        protected void addSheetPicsAndEmbedds(XSSFSheet sheet) throws IOException {
            if (sheet == null) return;
            XSSFDrawing draw = sheet.createDrawingPatriarch();
            for (XSSFShape shape : draw.getShapes()) {
                if (!(shape instanceof XSSFPicture)) continue;
                XSSFPicture picture = (XSSFPicture)shape;
                XSSFPictureData pd = picture.getPictureData();
                PackagePart pp = pd.getPackagePart();
                CTPicture ctPic = picture.getCTPicture();
                String filename = null;
                try {
                    filename = ctPic.getNvPicPr().getCNvPr().getName();
                } catch (Exception e) {}
                if (filename == null || "".equals(filename)) {
                    filename = new File(pp.getPartName().toString()).getName();
                }
                EmbeddedData ed = new EmbeddedData();
                ed.filename = fileNameWithoutPath(filename);
                ed.is = pp.getInputStream();
                embeddings.add(ed);
            }
        }

        protected void addAllEmbedds(HSSFWorkbook hwb) throws IOException {
            for (HSSFObjectData od : hwb.getAllEmbeddedObjects()) {
                String alternativeName = getAlternativeName(od);
                if (od.hasDirectoryEntry()) {
                    DirectoryNode src = (DirectoryNode)od.getDirectory();
                    for (EmbeddedExtractor ee : extractors) {
                        if (ee.canExtract(src)) {
                            EmbeddedData ed = ee.extract(src);
                            if (ed.filename == null || ed.filename.startsWith("MBD") || alternativeName != null) {
                                ed.filename = alternativeName;
                            }
                            ed.filename = fileNameWithoutPath(ed.filename);
                            ed.source = "object";
                            embeddings.add(ed);
                            break;
                        }
                    }
                }
            }
        }

        protected String getAlternativeName(HSSFShape shape) {
            EscherOptRecord eor = reflectEscherOptRecord(shape);
            if (eor == null) return null;
            for (EscherProperty ep : eor.getEscherProperties()) {
                if ("groupshape.shapename".equals(ep.getName()) && ep.isComplex()) {
                    return new String(((EscherComplexProperty)ep).getComplexData(),
                            Charset.forName("UTF-16LE"));
                }
            }
            return null;
        }

        protected void addSheetPics(HSSFSheet sheet) {
            if (sheet == null) return;
            int picIdx=0;
            int emfIdx = 0;
            HSSFPatriarch patriarch = sheet.getDrawingPatriarch();
            if (patriarch == null) return;
            // Loop through the objects
            for (HSSFShape shape : patriarch.getChildren()) {
                if (!(shape instanceof HSSFPicture)) {
                    continue;
                }
                HSSFPicture picture = (HSSFPicture) shape;
                if (picture.getShapeType() != HSSFSimpleShape.OBJECT_TYPE_PICTURE) continue;
                HSSFPictureData pd = picture.getPictureData();
                byte pictureBytes[] = pd.getData();
                int pictureBytesOffset = 0;
                int pictureBytesLen = pictureBytes.length;
                String filename = picture.getFileName();
                // try to find an alternative name
                if (filename == null || "".equals(filename)) {
                    filename = getAlternativeName(picture);
                }
                // default to dummy name
                if (filename == null || "".equals(filename)) {
                    filename = "picture"+(picIdx++);
                }
                filename = filename.trim();

                // check for emf+ embedded pdf (poor mans style :( )
                // Mac Excel 2011 embeds pdf files with this method.
                boolean validFile = true;
                if (pd.getFormat() == Workbook.PICTURE_TYPE_EMF) {
                    validFile = false;
                    int idxStart = indexOf(pictureBytes, 0, "%PDF-".getBytes());
                    if (idxStart != -1) {
                        int idxEnd = indexOf(pictureBytes, idxStart, "%%EOF".getBytes());
                        if (idxEnd != -1) {
                            pictureBytesOffset = idxStart;
                            pictureBytesLen = idxEnd-idxStart+6;
                            validFile = true;
                        }
                    } else {
                        // This shape was not a Mac Excel 2011 embedded pdf file.
                        // So this is a shape related to a regular embedded object
                        // Lets update the object filename with the shapes filename
                        // if the object filename is of format ARGF1234.pdf
                        EmbeddedData ed_obj = embeddings.get(emfIdx);
                        Pattern pattern = Pattern.compile("^[A-Z0-9]{8}\\.[pdfPDF]{3}$");
                        Matcher matcher = pattern.matcher(ed_obj.filename);
                        if(matcher.matches()) {
                            ed_obj.filename = filename;
                        }
                        emfIdx += 1;
                    }
                }

                EmbeddedData ed = new EmbeddedData();
                ed.filename = fileNameWithoutPath(filename);
                ed.is = new ByteArrayInputStream(pictureBytes, pictureBytesOffset, pictureBytesLen);
                if(fileNotInEmbeddings(ed.filename) && validFile) {
                    embeddings.add(ed);
                }
            }
        }

        private static EscherOptRecord reflectEscherOptRecord(HSSFShape shape) {
            try {
                Method m = HSSFShape.class.getDeclaredMethod("getOptRecord");
                m.setAccessible(true);
                return (EscherOptRecord)m.invoke(shape);
            } catch (Exception e) {
                // todo: log ... well actually "should not happen" ;)
                return null;
            }
        }

        private String fileNameWithoutPath(String filename) {
            int last_index = filename.lastIndexOf("\\");
            return filename.substring(last_index + 1);
        }

        private boolean fileNotInEmbeddings(String filename) {
            boolean exists = true;
            for(EmbeddedData ed : embeddings) {
                if(ed.filename.equals(filename)) {
                    exists = false;
                }
            }
            return exists;
        }

        public void close() throws IOException {
            Iterator<EmbeddedData> ed = embeddings.iterator();
            while (ed.hasNext()) {
                ed.next().is.close();
            }
            wb.close();
        }
    }

    static class EmbeddedData {
        String filename;
        InputStream is;
        String source;
    }

    static abstract class EmbeddedExtractor {
        abstract boolean canExtract(DirectoryNode dn);
        abstract EmbeddedData extract(DirectoryNode dn) throws IOException;
        protected EmbeddedData extractFS(DirectoryNode dn, String filename) throws IOException {
            assert(canExtract(dn));
            POIFSFileSystem dest = new POIFSFileSystem();
            copyNodes(dn, dest.getRoot());
            EmbeddedData ed = new EmbeddedData();
            ed.filename = filename;
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            dest.writeFilesystem(bos);
            dest.close();
            ed.is = new ByteArrayInputStream(bos.toByteArray());
            return ed;
        }
    }

    static class Ole10Extractor extends EmbeddedExtractor {
        public boolean canExtract(DirectoryNode dn) {
            ClassID clsId = dn.getStorageClsid();
            return ClassID.OLE10_PACKAGE.equals(clsId);
        }
        public EmbeddedData extract(DirectoryNode dn) throws IOException {
            try {
                Ole10Native ole10 = Ole10Native.createFromEmbeddedOleObject(dn);
                EmbeddedData ed = new EmbeddedData();
                ed.filename = new File(ole10.getFileName()).getName();
                ed.is = new ByteArrayInputStream(ole10.getDataBuffer());
                return ed;
            } catch (Ole10NativeException e) {
                throw new IOException(e);
            }
        }
    }

    static class PdfExtractor extends EmbeddedExtractor {
        static ClassID PdfClassID = new ClassID("{B801CA65-A1FC-11D0-85AD-444553540000}");
        public boolean canExtract(DirectoryNode dn) {
            ClassID clsId = dn.getStorageClsid();
            return (PdfClassID.equals(clsId)
            || dn.hasEntry("CONTENTS"));
        }
        public EmbeddedData extract(DirectoryNode dn) throws IOException {
            EmbeddedData ed = new EmbeddedData();
            ed.is = dn.createDocumentInputStream("CONTENTS");
            ed.filename = dn.getName()+".pdf";
            return ed;
        }
    }

    static class WordExtractor extends EmbeddedExtractor {
        public boolean canExtract(DirectoryNode dn) {
            ClassID clsId = dn.getStorageClsid();
            return (ClassID.WORD95.equals(clsId)
            || ClassID.WORD97.equals(clsId)
            || dn.hasEntry("WordDocument"));
        }
        public EmbeddedData extract(DirectoryNode dn) throws IOException {
            return extractFS(dn, dn.getName()+".doc");
        }
    }

    static class ExcelExtractor extends EmbeddedExtractor {
        public boolean canExtract(DirectoryNode dn) {
            ClassID clsId = dn.getStorageClsid();
            return (ClassID.EXCEL95.equals(clsId)
                    || ClassID.EXCEL97.equals(clsId)
                    || dn.hasEntry("Workbook") /*...*/);
        }
        public EmbeddedData extract(DirectoryNode dn) throws IOException {
            return extractFS(dn, dn.getName()+".xls");
        }
    }

    static class FsExtractor extends EmbeddedExtractor {
        public boolean canExtract(DirectoryNode dn) {
            return true;
        }
        public EmbeddedData extract(DirectoryNode dn) throws IOException {
            return extractFS(dn, dn.getName()+".dat");
        }
    }

    private static void copyNodes(DirectoryNode src, DirectoryNode dest) throws IOException {
        for (Entry e : src) {
            if (e instanceof DirectoryNode) {
                DirectoryNode srcDir = (DirectoryNode)e;
                DirectoryNode destDir = (DirectoryNode)dest.createDirectory(srcDir.getName());
                destDir.setStorageClsid(srcDir.getStorageClsid());
                copyNodes(srcDir, destDir);
            } else {
                InputStream is = src.createDocumentInputStream(e);
                dest.createDocument(e.getName(), is);
                is.close();
            }
        }
    }

    /**
     * Knuth-Morris-Pratt Algorithm for Pattern Matching
     * Finds the first occurrence of the pattern in the text.
     */
    private static int indexOf(byte[] data, int offset, byte[] pattern) {
        int[] failure = computeFailure(pattern);

        int j = 0;
        if (data.length == 0) return -1;

        for (int i = offset; i < data.length; i++) {
            while (j > 0 && pattern[j] != data[i]) {
                j = failure[j - 1];
            }
            if (pattern[j] == data[i]) { j++; }
            if (j == pattern.length) {
                return i - pattern.length + 1;
            }
        }
        return -1;
    }

    /**
     * Computes the failure function using a boot-strapping process,
     * where the pattern is matched against itself.
     */
    private static int[] computeFailure(byte[] pattern) {
        int[] failure = new int[pattern.length];

        int j = 0;
        for (int i = 1; i < pattern.length; i++) {
            while (j > 0 && pattern[j] != pattern[i]) {
                j = failure[j - 1];
            }
            if (pattern[j] == pattern[i]) {
                j++;
            }
            failure[i] = j;
        }

        return failure;
    }
}

Comment: @ kiwiwings: Hi, thank you.once again  thank you so very much.
Comment: @kiwiwiings: I got error in getSamplePPT()  static method The constructor HSLFSlideShow() is undefined.
Comment: alereday i attached poi-scratchpad-3.7.jar and poi-scratchpad-3.9.jar.
Comment: @ kiwiwings: I want PDF,xls,xlsx file to export.
Comment: getSampleEmbedded() File, I got error in wb.addOlePackage(getSamplePPT "dummy.ppt","dummy.ppt", "dummy.ppt"() ), what is addOlePackage???
Comment: I am sorry. I do not noticed there. I apologize for above comment.
Comment: @ kiwiwing : I am sorry.  I do not noticed there. I apologize for above comment. can you forgive me?
Comment: @ kiwiwing : In public void close() method its required to close wb.close() beacuse I got error "The method close() is undefined for the type Workbook"after i attached a3.16-beta1 jar.
Comment: @ kiwiwing: another confusion is How to save those embedded file into desktop? what are the chnges of code that want to do in the above code?
Comment: @ kiwiwing : Its working perfectly. Thank you so very much. Its great work done by you.
Comment: @ kiwiwing : How to get extracted file in its original format at the time of embedded in Excel. Is it possible ? What are the changes of code?
Comment: @ kiwiwings : As above code extracted file save as a .dat format. so I want this formate as a .png & .xls formate. so change in the public EmbeddedData extract(DirectoryNode dn) as a .xls and. png format but the file was not open. File generated successfully why? This file I want to open. what are the changes required in the code  to open this Generated file? please reply me fast as soon as possible. I must required to open this exported file.
Comment: Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/131122/discussion-between-kiwiwings-and-sit).
Comment: Does not open chat at my system.
Comment: @ kiwiwings : I got error java.io.FileNotFoundException: no such entry: "MBD00000000" i can not able find out what is problem?
Answer: import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.ArrayList;

/**
 * Demonstrates how you can extract embedded data from a .xlsx file
 */
public class GetEmbedded {

    public static void main(String[] args) throws Exception {
        String path = "SomeExcelFile.xlsx"
        XSSFWorkbook workbook = new XSSFWorkbook(new FileInputStream(new File(path)));

             for (PackagePart pPart : workbook.getAllEmbedds()) {
                            String contentType = pPart.getContentType();

                            if (contentType.equals("application/vnd.ms-excel")) { //This is to read xls workbook embedded to xlsx file
                                HSSFWorkbook embeddedWorkbook = new HSSFWorkbook(pPart.getInputStream());
                                int countOfSheetXls=embeddedWorkbook.getNumberOfSheets();

                 }
                            else if (contentType.equals("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")) { //This is to read xlsx workbook embedded to xlsx file
                                 if(pPart.getPartName().getName().equals("/xl/embeddings/Microsoft_Excel_Worksheet12.xlsx")){
                                 //"/xl/embeddings/Microsoft_Excel_Worksheet12.xlsx" - Can read an Excel from a particular sheet
                                // This is the worksheet from the Parent Excel-sheet-12

                                     XSSFWorkbook embeddedWorkbook = new XSSFWorkbook(pPart.getInputStream());
                                     int countOfSheetXlsx=embeddedWorkbook.getNumberOfSheets();
                                     ArrayList<String> sheetNames= new ArrayList<String>();
                                        for(int i=0;i<countOfSheetXlsx;i++){
                                        String name=workbook.getSheetName(i);
                                        sheetNames.add(name);
                                        }
                                }
                            }
                }
     }
}

Comment: The above code is to extract the embedded xlsx from another parent xlsx file.
Answer: Required jar File List:

commons-codec-1.10.jar
dom4j.jar
poi-3.16-beta1.jar
poi-ooxml-3.8.jar
poi-ooxml-schemas-3.9.jar
poi-scratchpad-3.9.jar
xmlbeans-2.3.0.jar

This is my Whole code implementation:
 import java.awt.Color;
 import java.awt.image.BufferedImage;
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.Closeable;
 import java.io.File;
 import java.io.FileOutputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.lang.reflect.Method;
 import java.net.URL;
 import java.nio.charset.Charset;
 import java.util.ArrayList;
 import java.util.Iterator;
 import java.util.List;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 import javax.imageio.ImageIO;
 import org.apache.poi.ddf.EscherComplexProperty;
 import org.apache.poi.ddf.EscherOptRecord;
 import org.apache.poi.ddf.EscherProperty;
 import org.apache.poi.hpsf.ClassID;
 import org.apache.poi.hslf.HSLFSlideShow;
 import org.apache.poi.hssf.usermodel.HSSFClientAnchor;
 import org.apache.poi.hssf.usermodel.HSSFObjectData;
 import org.apache.poi.hssf.usermodel.HSSFPatriarch;
 import org.apache.poi.hssf.usermodel.HSSFPicture;
 import org.apache.poi.hssf.usermodel.HSSFPictureData;
 import org.apache.poi.hssf.usermodel.HSSFShape;
 import org.apache.poi.hssf.usermodel.HSSFSheet;
 import org.apache.poi.hssf.usermodel.HSSFSimpleShape;
 import org.apache.poi.hssf.usermodel.HSSFWorkbook;
 import org.apache.poi.openxml4j.opc.PackagePart;
 import org.apache.poi.poifs.filesystem.DirectoryNode;
 import org.apache.poi.poifs.filesystem.Entry;
 import org.apache.poi.poifs.filesystem.Ole10Native;
 import org.apache.poi.poifs.filesystem.Ole10NativeException;
 import org.apache.poi.poifs.filesystem.POIFSFileSystem;
 import org.apache.poi.sl.usermodel.AutoShape;
 import org.apache.poi.sl.usermodel.Slide;
 import org.apache.poi.ss.usermodel.Sheet;
 import org.apache.poi.ss.usermodel.Workbook;
 import org.apache.poi.ss.usermodel.WorkbookFactory;
 import org.apache.poi.util.IOUtils;
 import org.apache.poi.xssf.usermodel.XSSFDrawing;
 import org.apache.poi.xssf.usermodel.XSSFPicture;
 import org.apache.poi.xssf.usermodel.XSSFPictureData;
 import org.apache.poi.xssf.usermodel.XSSFShape;
 import org.apache.poi.xssf.usermodel.XSSFSheet;
 import org.apache.poi.xssf.usermodel.XSSFWorkbook;
 import org.openxmlformats.schemas.drawingml.x2006.spreadsheetDrawing.CTPicture;

 public class EmbeddedReader {

public static final OleType OLE10_PACKAGE = new OleType("{0003000C-0000-0000-C000-000000000046}");
public static final OleType PPT_SHOW = new OleType("{64818D10-4F9B-11CF-86EA-00AA00B929E8}");
public static final OleType XLS_WORKBOOK = new OleType("{00020841-0000-0000-C000-000000000046}");
public static final OleType TXT_ONLY = new OleType("{5e941d80-bf96-11cd-b579-08002b30bfeb}");
public static final OleType EXCEL97 = new OleType("{00020820-0000-0000-C000-000000000046}");
public static final OleType EXCEL95 = new OleType("{00020810-0000-0000-C000-000000000046}");
public static final OleType WORD97 = new OleType("{00020906-0000-0000-C000-000000000046}");
public static final OleType WORD95 = new OleType("{00020900-0000-0000-C000-000000000046}");
public static final OleType POWERPOINT97 = new OleType("{64818D10-4F9B-11CF-86EA-00AA00B929E8}");
public static final OleType POWERPOINT95 = new OleType("{EA7BAE70-FB3B-11CD-A903-00AA00510EA3}");
public static final OleType EQUATION30 = new OleType("{0002CE02-0000-0000-C000-000000000046}");
public static final OleType PdfClassID = new OleType("{B801CA65-A1FC-11D0-85AD-444553540000}");

private File excel_file;
private ImageReader image_reader;

static class OleType {
    final String classId;

    OleType(String classId) {
        this.classId = classId;
    }

    ClassID getClassID() {
        ClassID cls = new ClassID();
        byte clsBytes[] = cls.getBytes();
        String clsStr = classId.replaceAll("[{}-]", "");
        for (int i = 0; i < clsStr.length(); i += 2) {
            clsBytes[i / 2] = (byte) Integer.parseInt(
                    clsStr.substring(i, i + 2), 16);
        }
        return cls;
    }
}

public static void main(String[] args) throws Exception {
    File sample = new File("D:\\ole_ppt_in_xls.xls");
    ImageReader ir = new ImageReader(sample);
    for (EmbeddedData ed : ir.embeddings) {
    FileOutputStream fos = new FileOutputStream(System.getProperty("user.home") + "/Desktop" + "/sumit/"+ ed.filename);
        IOUtils.copy(ed.is, fos);
        fos.close();
    }
    ir.close();
}

static byte[] getSamplePng() throws IOException {
    ClassLoader cl = Thread.currentThread().getContextClassLoader();
    URL imgUrl = cl.getResource("javax/swing/plaf/metal/icons/ocean/directory.gif");
    BufferedImage img = ImageIO.read(imgUrl);
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    ImageIO.write(img, "PNG", bos);
    return bos.toByteArray();
}

public EmbeddedReader(String excel_path) throws IOException {
    excel_file = new File(excel_path);
    image_reader = new ImageReader(excel_file);
}

public String[] get_file_names() {
    ArrayList<String> file_names = new ArrayList<String>();
    for (EmbeddedData ed : image_reader.embeddings) {
        file_names.add(ed.filename);
    }
    return file_names.toArray(new String[file_names.size()]);
}

public InputStream get_stream(String file_name) {
    InputStream input_stream = null;
    for (EmbeddedData ed : image_reader.embeddings) {
        if (file_name.equals(ed.filename)) {
            input_stream = ed.is;
            break;
        }
    }
    return input_stream;
}

static class ImageReader implements Closeable {
    EmbeddedExtractor extractors[] = { new Ole10Extractor(),new PdfExtractor(), new WordExtractor(), new ExcelExtractor(),new FsExtractor() };

    List<EmbeddedData> embeddings = new ArrayList<EmbeddedData>();
    Workbook wb;

    public ImageReader(File excelfile) throws IOException {
        try {
            wb = WorkbookFactory.create(excelfile);
            Sheet receiptImages = wb.getSheet("Receipt images");
            if (wb instanceof XSSFWorkbook) {
                addSheetPicsAndEmbedds((XSSFSheet) receiptImages);
            } else {
                addAllEmbedds((HSSFWorkbook) wb);
                addSheetPics((HSSFSheet) receiptImages);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    protected void addSheetPicsAndEmbedds(XSSFSheet sheet)throws IOException {
        if (sheet == null)
            return;
        XSSFDrawing draw = sheet.createDrawingPatriarch();
        for (XSSFShape shape : draw.getShapes()) {
            if (!(shape instanceof XSSFPicture))
                continue;
            XSSFPicture picture = (XSSFPicture) shape;
            XSSFPictureData pd = picture.getPictureData();
            PackagePart pp = pd.getPackagePart();
            CTPicture ctPic = picture.getCTPicture();
            String filename = null;
            try {
                filename = ctPic.getNvPicPr().getCNvPr().getName();
            } catch (Exception e) {
            }
            if (filename == null || "".equals(filename)) {
                filename = new File(pp.getPartName().toString()).getName();
            }
            EmbeddedData ed = new EmbeddedData();
            ed.filename = fileNameWithoutPath(filename);
            ed.is = pp.getInputStream();
            embeddings.add(ed);
        }
    }

    protected void addAllEmbedds(HSSFWorkbook hwb) throws IOException {
        for (HSSFObjectData od : hwb.getAllEmbeddedObjects()) {
            String alternativeName = getAlternativeName(od);
            if (od.hasDirectoryEntry()) {
                DirectoryNode src = (DirectoryNode) od.getDirectory();
                for (EmbeddedExtractor ee : extractors) {
                    if (ee.canExtract(src)) {
                        EmbeddedData ed = ee.extract(src);
                        if (ed.filename == null || ed.filename.startsWith("MBD")|| alternativeName != null) {
                            if (alternativeName != null) {
                                ed.filename = alternativeName;
                            }
                        }
                        ed.filename = fileNameWithoutPath(ed.filename);
                        ed.source = "object";
                        embeddings.add(ed);
                        break;
                    }
                }
            }
        }
    }

    protected String getAlternativeName(HSSFShape shape) {
    EscherOptRecord eor = reflectEscherOptRecord(shape);
        if (eor == null) {
            return null;
        }
        for (EscherProperty ep : eor.getEscherProperties()) {
            if ("groupshape.shapename".equals(ep.getName())
                    && ep.isComplex()) {
                return new String(
                        ((EscherComplexProperty) ep).getComplexData(),
                        Charset.forName("UTF-16LE"));
            }
        }
        return null;
    }

    protected void addSheetPics(HSSFSheet sheet) {
        if (sheet == null)
            return;
        int picIdx = 0;
        int emfIdx = 0;
        HSSFPatriarch patriarch = sheet.getDrawingPatriarch();
        if (patriarch == null)
            return;
        // Loop through the objects
        for (HSSFShape shape : patriarch.getChildren()) {
            if (!(shape instanceof HSSFPicture)) {
                continue;
            }
            HSSFPicture picture = (HSSFPicture) shape;
            if (picture.getShapeType() != HSSFSimpleShape.OBJECT_TYPE_PICTURE)
                continue;
            HSSFPictureData pd = picture.getPictureData();
            byte pictureBytes[] = pd.getData();
            int pictureBytesOffset = 0;
            int pictureBytesLen = pictureBytes.length;
            String filename = picture.getFileName();
            // try to find an alternative name
            if (filename == null || "".equals(filename)) {
                filename = getAlternativeName(picture);
            }
            // default to dummy name
            if (filename == null || "".equals(filename)) {
                filename = "picture" + (picIdx++);
            }
            filename = filename.trim();

            // check for emf+ embedded pdf (poor mans style :( )
            // Mac Excel 2011 embeds pdf files with this method.
            boolean validFile = true;
            if (pd.getFormat() == Workbook.PICTURE_TYPE_EMF) {
                validFile = false;
                int idxStart = indexOf(pictureBytes, 0, "%PDF-".getBytes());
                if (idxStart != -1) {
                    int idxEnd = indexOf(pictureBytes, idxStart,"%%EOF".getBytes());
                    if (idxEnd != -1) {
                        pictureBytesOffset = idxStart;
                        pictureBytesLen = idxEnd - idxStart + 6;
                        validFile = true;
                    }
                } else {
                    // This shape was not a Mac Excel 2011 embedded pdf  file.
                    // So this is a shape related to a regular embedded object
                    // Lets update the object filename with the shapes filename
                    // if the object filename is of format ARGF1234.pdf
                    EmbeddedData ed_obj = embeddings.get(emfIdx);
                    Pattern pattern = Pattern
                            .compile("^[A-Z0-9]{8}\\.[pdfPDF]{3}$");
                    Matcher matcher = pattern.matcher(ed_obj.filename);
                    if (matcher.matches()) {
                        ed_obj.filename = filename;
                    }
                    emfIdx += 1;
                }
            }

            EmbeddedData ed = new EmbeddedData();
            ed.filename = fileNameWithoutPath(filename);
            ed.is = new ByteArrayInputStream(pictureBytes,
                    pictureBytesOffset, pictureBytesLen);
            if (fileNotInEmbeddings(ed.filename) && validFile) {
                embeddings.add(ed);
            }
        }
    }

    private static EscherOptRecord reflectEscherOptRecord(HSSFShape shape) {
        try {
            Method m = HSSFShape.class.getDeclaredMethod("getOptRecord");
            m.setAccessible(true);
            return (EscherOptRecord) m.invoke(shape);
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }

    private String fileNameWithoutPath(String filename) {
        int last_index = filename.lastIndexOf("\\");
        return filename.substring(last_index + 1);
    }

    private boolean fileNotInEmbeddings(String filename) {
        boolean exists = true;
        for (EmbeddedData ed : embeddings) {
            if (ed.filename.equals(filename)) {
                exists = false;
            }
        }
        return exists;
    }

    public void close() throws IOException {
        Iterator<EmbeddedData> ed = embeddings.iterator();
        while (ed.hasNext()) {
            ed.next().is.close();
        }
        wb.close();
    }
}

static class EmbeddedData {
    String filename;
    InputStream is;
    String source;
}

static abstract class EmbeddedExtractor {
    abstract boolean canExtract(DirectoryNode dn);
    abstract EmbeddedData extract(DirectoryNode dn) throws IOException;
    protected EmbeddedData extractFS(DirectoryNode dn, String filename)
            throws IOException {
        assert (canExtract(dn));
        POIFSFileSystem dest = new POIFSFileSystem();
        copyNodes(dn, dest.getRoot());
        EmbeddedData ed = new EmbeddedData();
        ed.filename = filename;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        dest.writeFilesystem(bos);
         bos.close();
        ed.is = new ByteArrayInputStream(bos.toByteArray());
        return ed;
    }
}

static class Ole10Extractor extends EmbeddedExtractor {
    public boolean canExtract(DirectoryNode dn) {
        ClassID clsId = dn.getStorageClsid();
        return OLE10_PACKAGE.equals(clsId);
    }

    public EmbeddedData extract(DirectoryNode dn) throws IOException {
        try {
            Ole10Native ole10 = Ole10Native.createFromEmbeddedOleObject(dn);
            EmbeddedData ed = new EmbeddedData();
            ed.filename = new File(ole10.getFileName()).getName();
            ed.is = new ByteArrayInputStream(ole10.getDataBuffer());
            return ed;
        } catch (Ole10NativeException e) {
            e.printStackTrace();
            throw new IOException(e);
        }
    }
}

static class PdfExtractor extends EmbeddedExtractor {
    public boolean canExtract(DirectoryNode dn) {
        ClassID clsId = dn.getStorageClsid();
        return (PdfClassID.equals(clsId) || dn.hasEntry("CONTENTS"));
    }

    public EmbeddedData extract(DirectoryNode dn) throws IOException {
        EmbeddedData ed = new EmbeddedData();
        ed.is = dn.createDocumentInputStream("CONTENTS");
        ed.filename = dn.getName() + ".pdf";
        return ed;
    }
}

static class WordExtractor extends EmbeddedExtractor {
    public boolean canExtract(DirectoryNode dn) {
        ClassID clsId = dn.getStorageClsid();
        return (WORD95.equals(clsId) || WORD97.equals(clsId) || dn.hasEntry("WordDocument"));
    }

    public EmbeddedData extract(DirectoryNode dn) throws IOException {
        return extractFS(dn, dn.getName() + ".doc");
    }
}

static class ExcelExtractor extends EmbeddedExtractor {
    public boolean canExtract(DirectoryNode dn) {
        ClassID clsId = dn.getStorageClsid();
        return (EXCEL95.equals(clsId) || EXCEL97.equals(clsId) || dn
                .hasEntry("Workbook") /* ... */);
    }

    public EmbeddedData extract(DirectoryNode dn) throws IOException {
        return extractFS(dn, dn.getName() + ".xls");
    }
}

static class FsExtractor extends EmbeddedExtractor {

    public boolean canExtract(DirectoryNode dn) {
        return true;
    }

    public EmbeddedData extract(DirectoryNode dn) throws IOException {
        return extractFS(dn, dn.getName() + ".dat");
    }
}

private static void copyNodes(DirectoryNode src, DirectoryNode dest)
        throws IOException {
    for (Entry e : src) {
        if (e instanceof DirectoryNode) {
            DirectoryNode srcDir = (DirectoryNode) e;
            DirectoryNode destDir = (DirectoryNode) dest
                    .createDirectory(srcDir.getName());
            destDir.setStorageClsid(srcDir.getStorageClsid());
            copyNodes(srcDir, destDir);
        } else {
            InputStream is = src.createDocumentInputStream(e);
            dest.createDocument(e.getName(), is);
            is.close();
        }
    }
}

/**
 * Knuth-Morris-Pratt Algorithm for Pattern Matching Finds the first
 * occurrence of the pattern in the text.
 */
private static int indexOf(byte[] data, int offset, byte[] pattern) {
    int[] failure = computeFailure(pattern);

    int j = 0;
    if (data.length == 0)
        return -1;

    for (int i = offset; i < data.length; i++) {
        while (j > 0 && pattern[j] != data[i]) {
            j = failure[j - 1];
        }
        if (pattern[j] == data[i]) {
            j++;
        }
        if (j == pattern.length) {
            return i - pattern.length + 1;
        }
    }
    return -1;
}

/**
 * Computes the failure function using a boot-strapping process, where the
 * pattern is matched against itself.
 */
private static int[] computeFailure(byte[] pattern) {
    int[] failure = new int[pattern.length];

    int j = 0;
    for (int i = 1; i < pattern.length; i++) {
        while (j > 0 && pattern[j] != pattern[i]) {
            j = failure[j - 1];
        }
        if (pattern[j] == pattern[i]) {
            j++;
        }
        failure[i] = j;
    }

    return failure;
}

}
Answer: To simplify the processing of embedded data, I've added an extractor class to POI, which will be available in POI 3.16-beta2 or a nightly until then.
The following will extract the objects of .xls/x files - all which is left, is to write the embedded bytes somewhere. It's possible to extend the extractor classes by simply extending EmbeddedExtractor and provide your own iterator() method.
import java.io.FileInputStream;
import java.io.InputStream;

import org.apache.poi.ss.extractor.EmbeddedData;
import org.apache.poi.ss.extractor.EmbeddedExtractor;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.usermodel.WorkbookFactory;

public class BlaExtract {
    public static void main(String[] args) throws Exception {
        InputStream fis = new FileInputStream("bla.xlsx");
        Workbook wb = WorkbookFactory.create(fis);
        fis.close();

        EmbeddedExtractor ee = new EmbeddedExtractor();
        for (Sheet s : wb) {
            for (EmbeddedData ed : ee.extractAll(s)) {
                System.out.println(ed.getFilename()+" ("+ed.getContentType()+") - "+ed.getEmbeddedData().length+" bytes");
            }
        }
        wb.close();
    }
}

Comment: "java.lang.SecurityException: class "org.apache.poi.ss.usermodel.Workbook"'s signer information does not match signer information of other classes in the same package" error come I googled it i applied that solution but error is not gone still coming.
Comment: Although we are not signing the jars itself, i.e. within the jar - only the maven jars outside of the jar - my guess is, that you are still using mixed versions of the POI jars
Comment: @ kiwiwings:   i used "IOUtils.copy(ed.is, fos);" in above code for the extration but file still file does not open.
Comment: @ kiwiwings : can i sent one email to you? regrading my implementation, original file  and extracted file?
Comment: @SIT: of course you can send me files - I assume original file = Excel file, if not, please attach it too
Comment: @ kiwiwings : when i tried to extract embedded excel file its generated ".ole" Extension file generate how to open this file via application software or how to convert ".ole" file into xlsx.


How can i make my app run even when app is killed? killed in the sense when it is removed from recent apps

Question: I am developing an app which can count number of unlocks in whole day.
I have seen many similar questions but none of them worked!! Anyone that can help me please provide code snippet for the above question.
Even make sure the solutoin you provide works for Android O and above
MainActivity.java
public class MainActivity extends AppCompatActivity {
MyReceiver myReceiver;
IntentFilter filter;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    myReceiver = new MyReceiver();
    filter = new IntentFilter(Intent.ACTION_USER_PRESENT);
    registerReceiver(myReceiver, filter);
}}

MyReceiver.java
public class MyReceiver extends BroadcastReceiver {
   private static final String TAG = "BroadCast Receiver";
   @Override
   public void onReceive(Context context, Intent intent) {
       if(intent.getAction().equals(Intent.ACTION_USER_PRESENT)){
          Toast.makeText(context, "DEVICE_IS_UNLOCKED",Toast.LENGTH_LONG).show();
          Log.i(TAG, "Unlocked");
       }
   }
}

Answer: Start services in foreground
https://androidwave.com/foreground-service-android-example/?
Comment: A link to a solution is welcome, but please ensure your answer is useful without it: [add context around the link](//meta.stackexchange.com/a/8259) so your fellow users will have some idea what it is and why it's there, then quote the most relevant part of the page you're linking to in case the target page is unavailable. [Answers that are little more than a link may be deleted.](//$SITEURL$/help/deleted-answers)
Answer: You need work on Background Service.
Try below code.
public class BackgroundUpdateService extends Service {

    /**
     * Author:Hardik Talaviya
     * Date:  2019.08.3 2:30 PM
     * Describe:
     */

    private static final String TAG = "BackgroundLocation";
    private Context context;
    private boolean stopService = false;
    private Handler handler;
    private Runnable runnable;
    private NotificationCompat.Builder builder = null;
    private NotificationManager notificationManager;

    @Override
    public void onCreate() {
        Log.e(TAG, "Background Service onCreate :: ");
        super.onCreate();
        context = this;

        handler = new Handler();
        runnable = new Runnable() {

            @Override
            public void run() {
                try {
                    //Add Here your code if you want start in background
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    handler.postDelayed(this, TimeUnit.SECONDS.toMillis(2));
                }
            }
        };
        if (!stopService) {
            handler.postDelayed(runnable, TimeUnit.SECONDS.toMillis(2));
        }
    }

    @Override
    public IBinder onBind(Intent intent) {
        return null;
    }

    @Override
    public void onTaskRemoved(Intent rootIntent) {
        super.onTaskRemoved(rootIntent);
        Log.e(TAG, "onTaskRemoved :: ");
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        Log.e(TAG, "onStartCommand :: ");
        StartForeground();
        return START_STICKY;
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
        Log.e(TAG, "BackgroundService onDestroy :: ");
        stopService = true;
        if (handler != null) {
            handler.removeCallbacks(runnable);
        }
    }

    /*-------- For notification ----------*/
    private void StartForeground() {
        Intent intent = new Intent(context, MainActivity.class);
        intent.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP);
        PendingIntent pendingIntent = PendingIntent.getActivity(this, 0 /* Request code */, intent, PendingIntent.FLAG_ONE_SHOT);

        String CHANNEL_ID = "channel_location";
        String CHANNEL_NAME = "channel_location";

        notificationManager = (NotificationManager) getApplicationContext().getSystemService(Context.NOTIFICATION_SERVICE);
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
            NotificationChannel channel = new NotificationChannel(CHANNEL_ID, CHANNEL_NAME, NotificationManager.IMPORTANCE_DEFAULT);
            channel.setLockscreenVisibility(Notification.VISIBILITY_PRIVATE);
            notificationManager.createNotificationChannel(channel);
            builder = new NotificationCompat.Builder(getApplicationContext(), CHANNEL_ID);
            builder.setColorized(false);
            builder.setChannelId(CHANNEL_ID);
            builder.setColor(ContextCompat.getColor(this, R.color.colorPrimaryDark));
            builder.setBadgeIconType(NotificationCompat.BADGE_ICON_NONE);
        } else {
            builder = new NotificationCompat.Builder(getApplicationContext(), CHANNEL_ID);
        }
        builder.setOnlyAlertOnce(true);
        builder.setContentTitle(context.getResources().getString(R.string.app_name));
        builder.setContentText("Your Text");
        Uri notificationSound = RingtoneManager.getActualDefaultRingtoneUri(this, RingtoneManager.TYPE_NOTIFICATION);
        builder.setSound(notificationSound);
        builder.setAutoCancel(true);
        builder.setSmallIcon(R.mipmap.ic_notification_app_icon);
        builder.setContentIntent(pendingIntent);
        startForeground(101, builder.build());
    }
}

Add this line in your Manifest
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />

<service
    android:name=".BackgroundUpdateService"
    android:enabled="true" />

And Start & Stop service from your activity using below code
Intent someIntent = new Intent(MainActivity.this, BackgroundUpdateService.class);
startService(someIntent);//For start service
stopService(someIntent); //For stop service

I hope this can help You!
Thank You.


asp.net servindo arquivos estaticos

Question: Tenho uma aplicação em mvc3 (asp.net 4.0, dotnet 4.0),
rodando em iis8 (mas também está rodando no iis 7 e iis 7.5).
Dentro da aplicação tenho uma pasta chamada /dados,
exemplo localhost/minhaapp/dados.
Meus clientes salvam arquivos HTML dentro dessa pasta,
e esses arquivos HTML são acessados pelo sistema feito em mvc3.
Mas os usuários estão acessando os arquivos diretamente pela URL do browser,
exemplo localhost/minhaapp/dados/relatorio1.html, ao invés
de acessar por uma opção interna do sistema feito em mvc3.

Tem como bloquear o acesso via método GET dos arquivos HTML da pasta /dados?

Mas quero que o sistema feito em mvc3 consiga acessar esses arquivos via
método POST.
Esse não é o método mais seguro, deve existir muitas outras possibilidade,
mas se conseguir isso já estaria ótimo.
Tentei várias formas, rewrite, módulos, urlmapping, mvc controler, etc,
mas nenhuma dessas formas captura o requisição do browser para arquivos estáticos.
Sei que arquivo estáticos são servidos diretamente pelo iis.

Mas tem alguma forma do asp.net intercetar requisições de arquivos estático?
Se não for possível via asp.net, tem alguma configuração que funcione nos
iss 8, 7.5 r 7?

Não posso alterar para outra pasta, nem salvar os arquivos de outra forma, ou seja,
por questões internas (políticas internas, fato de já ser assim e muito difundido e
usados por vários clientes), a forma e o local como os arquivos são salvos,
NÃO pode ser alterado.
Comment: Uma dica que talvez possa ajudar: Salvar o arquivo com o nome criptografado (Ex: de "arquivo.html" para "123asd.html"). Sendo assim quando o usuário tentasse forçar o acesso o mesmo não iria conseguir visto que não ia encontrar arquivo utilizando o nome real.
Comment: Também pensei nisso, mas o arquivo é salvo via Excel pelo usuário, então teria que criar um serviço para monitorar o diretório, e ficar renomeando. Espero que exista outra forma, senão terei que fazer isso.
Answer: Você informa ao ISS que toda requisição deve passar por ele. O IIS não aplica permissão sobre arquivos estáticos.
Em seu Web.config coloque o seguinte código:
<configuration>
    ...
    <system.webServer>
        <modules runAllManagedModulesForAllRequests="true" />
    </system.webServer>
    ...
</configuration>

Comment: Sim já tentei isso, mas mesmo assim, as requisições de arquivos estáticos não chega na app. Tentei com global.asax e criando um módulo, em nenhum deles, a requisição chegou no BeginRequest. OBS: O IIS obriga a usar um POOL em modo Classic. Tem alguma outra forma de fazer isso?
Answer: Consegui fazer funcionar, coloquei runAllManagedModulesForAllRequests="true", alterei o pool da aplicação para o modo integrado, e coloquei o método BegingRequest no global.asax, reiniciei a aplicação, e agora está captura todas as requisições.
Answer: Conforme já respondi aqui, esta é a maneira incorreta de disponibilizar arquivos estáticos na sua aplicação.
Tem como bloquear o acesso via método GET dos arquivos HTML da pasta /dados?
Sim, criando no seu Controller um método Index para prevenir o acesso à raiz, ou então um arquivo index.html vazio.
Mas tem alguma forma do asp.net intercetar (sic) requisições de arquivos estático (sic)?
Tem. Você só precisa fazer um método no seu Controller que devolva um FileResult.
Adicionalmente, é bom definir uma rota que sempre redirecione para Index, no caso de seu usuário tentar acessar o arquivo diretamente:
App_Start/RouteConfig.cs
routes.MapRoute(
            name: "Dados",
            url: "dados/{id}",
            defaults: new { controller = "Dados", action = "Index", id = UrlParameter.Optional }
        );

Comment: @Juniorc-Sharp-Asp.Net Resolve sim. `FileResult` devolve qualquer arquivo em qualquer diretório passível de ser mapeado pelo servidor. Qualquer coisa diferente disso é expôr indevidamente a topologia de diretórios da sua aplicação.
Comment: @Juniorc-Sharp-Asp.Net Neste caso ainda é preciso configurar uma rota especial para redirecionar a requisição sempre para o método `Index` do seu *Controller*. Vou melhorar a resposta.
Comment: Sim entendo que não é a maneira correta, mas isso não está no meu controle infelizmente, e o que você sugeriu não revolve o problema na situação atual, infelizmente. Não existe nenhuma forma do asp.net/mvc3 interceptar requisições de arquivos estáticos?
Comment: Hum, acho que entendi o que você respondeu. Fiz conforme suas sugestões, mas continua permitindo acessar os arquivos no localhost/Dados/Relatorio1.html. Não posso alterar a situação atual, onde certo usuários salvam arquivos estátivos (html) na pasta dados do sistema, que "deveriam" ser acessados internamente pelo sistema, e não manualmente pelo usuário.
Comment: Isso tudo funciona conforme você comentou, criei uma aplicação mvc3 nova para fazer os testes. Mas deixa de funcionar quando é criado o diretório "Dados". Se tem apenas o controller dados, ok, mas depois de criar o diretório volta a permitir acessar o arquivo, além de não funcionar mais a rota especial criada. Você já chegou a testa nessas situações?
Comment: @Juniorc-Sharp-Asp.Net Sim, de fato tinha um erro. Veja agora.
Comment: Testei sua correção mas o comportamento continua o mesmo. Alguma outra sugestão? Estava lendo o pipeline/cycle do asp.net, e não consegui encontrar uma forma de fazer.

asp.net servindo arquivos estaticos


Yamaha P115 Aux out issue

Question: So I have a Yamaha P115 and I want to be able to mix the output with a YouTube video or some other reference. I've got myself a simple 4-channel audio mixer (r/http://amzn.eu/d/aP0TXbI) to try and do this. I've connected it up to the aux out of the piano and plugged some headphones into the output but sadly I'm not getting anything through the headphones, the sound still plays through the piano speakers.
I had a look at the manual for the P115 (https://jp.yamaha.com/files/download/other_assets/8/333278/p115_en_om_b0.pdf) which states on page 23 that "When the instrument's sound is output to an external device, first turn on the power to the instrument, then to the external device. Reverse this order when you turn the power off." but this has not helped.
I'm a bit of a noob with this stuff so apologies if I'm being stupid. Any help would be greatly appreciated.
EDIT: Just realised that if I whack the volume up on the input of the mixer I can just about hear the piano through the headphones. Do I have the wrong leads? I am using these: https://www.amazon.co.uk/gp/product/B003L6ZPQ4/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1
Answer: Page 23 of the Piano manual shows the correct way to connect an external device; but from your description, it sounds like you're doing that OK.
I believe the problem is that Behringer mixer. It's not the correct tool for the job you want to do. The output from that mixer is intended to be fed into a power amp, not a set of headphones. That explains why you can hear very little sound through it; the output is just far too low. Just because your headphones may fit into the output jack, doesn't mean they'll work.
To stay in the same price range, look at a small five channel mixer which has a dedicated headphones output - that'll do exactly what you need to do (and most also have RCA tape inputs for music from other devices).
Comment: Andy, don't waste money on the headphone amp. Those are designed to take one input, and give you five headphone outputs. It's useful if you play with friends, but you'll gain nothing but a mess of cables if you put it into your existing solution. Buy the 5-channel mixer you're looking at. It's still a basic model, but it's more expandable and useful.
Comment: I bought a Yamaha MG06 6 channel mixer which has independent amplifier and headphone outputs.  Something like that should work well for you.
Comment: Thanks guys. I'm returning the MX400 and have ordered the Behringer Xenyx 502 which should arrive tomorrow. I expect that will solve the problem. Will accept the answer once all confirmed and working :)
Comment: Thanks Pete. I think I just about came to the same conclusion whilst doing some research. I think I have indeed bought the wrong sort of mixer. Would something like this be a better choice? http://amzn.eu/d/6rdBAAZ. It has a dedicated phones output so I'm hoping so! I also read somewhere that I could keep what I have bit also get a headphone amp, something like this - http://amzn.eu/d/8lmAiqK. Is that also a viable option?

Yamaha P115 Aux out issue


Discrete surface interpolation

Question: I'm working on implementation of a Fast surace interpolation using hierarchial basis functions (Szeliski et al) algorithm.
The idea is: given a discrete function measurements of its values (depths)
$\{p_i\} = \{(u_i, v_i, d_i)\} = \{(u_i, v_i, \hat{f}(u_i, v_i))\}$
we are looking for such a function $f(u, v)$ so that it is very close to the original function, $\hat{f}(u, v)$ on its whole domain.
Note: we are dealing with discrete values. Thus, for convenience, let's denote a set of interpolated values as $\mathrm{x}$ and a set of measurement values as $\mathrm{d}$.
To do that, we try to minimize the energy function, $E(\mathrm{x}) = E_s(\mathrm{x}) + E_d(\mathrm{x}, \mathrm{d})$.
The data cost term, $E_d(\mathrm{x}, \mathrm{d})$, determines how close the interpolated surface (or function) is in reference to the original function.
The smoothness term, $E_s(\mathrm{x})$ encodes the curvature of the surface interpolated.
Now, the source problem is discretized using finite element method, which uses the stiffness matrix, $A$. To minimize the energy function, described above, the stiffness matrix is used too. It is said, to find matrix $A$ the authors solve this equation: $A\mathrm{x}=\mathrm{b}$ by setting:
$a_{(i, j), (k, l)} = \frac{\partial^2E(\mathrm{x}) }{\partial x_{i, j}\partial x_{k, l}} \Bigg|_{\mathrm{x}=0}$
and
$b_{i, j} = - \frac{\partial E(\mathrm{x}) }{\partial x_{i, j}} \Bigg|_{\mathrm{x}=0}$
My question is: why does the element of $A$ have those strange pair indices $a_{(i, j), (k, l)}$? Or maybe what is the stiffness matrix - is it 4-dimensional? Or what it looks like?
Would appreciate any help!
Answer: For linear algebra purposes, you want to view the solution, forcing, etc. as vectors. For geometric purposes, you want to view them as matrices (because they are 2D). The usual compromise is to actually view it as a vector in code where you convert the 2D indices into 1D indices. There are two ways to do this (called column major and row major) but neither one is really preferred over the other. This conflict is reflected in our mathematical notation as well: we can keep indices as tuples of integers or we can write them as vectors of integers. Your source has chosen to do the former.
Comment: @shybovycha That's right (though $\| \|$ is weird notation for that notion of "length").
Comment: So the matrix $A$ would have dimensions $(n, n)$ given $n = ||\mathrm{x}||$ (the length of $\mathrm{x}$ vector)?
Comment: true, my mistake - should be one pipe: $|\mathrm{x}|$

Discrete surface interpolation


Add header and expier mCache-Control time mdeflate on JS CSS and Images using AWS S3 php client

Question: AWS S3 php client  failed to add Cache-Control and Compressing resources with gzip or deflate  to Object.
Info1: EC2 as hosting and select Ubuntu 14 as OS and install Apache PHP and i have check that it php and http works.
Info2: Rds as mysql database instance. it is working
Info3: I am  using s3 clodefont  for image css js front cdn
Issue:try to upload images css js and fonts file from below using php aws  s3 client.
I have using this code but is not working:
            $client = S3Client::factory(array(
             'key'    => 'xxxxxxxx',
            'secret' => 'xxxxx',
            ));

             $bucket='mybucket';

             $result = $client->putObject(array(
                'Bucket'     => $bucket,
                'Key'        => $destinationpath,
                'Cache-Control' => 'max-age=86400',
                'Expires' => date('D, d M Y H:i:s T', strtotime('+24 hours')),
                'ACL'        => 'public-read',
                'Content-Encoding' => 'gzip',
                'SourceFile' => $sourcenpath,
              ));

The above code is not set  compressing   and Content-Encoding and Expires date
I need this thing for speedup
Want: Gzip compression or deflate,   header Cache-Controla and Expires to set to new upload object of S3
Can any one help me?
Answer: You need to use CacheControl & ContentEncoding (without dashes).
See the AWS PHP SDK putObject documentation for more details.


Graph coloring problem (possibly related to partitions)

Question: Given an undirected graph I'd like to color each node either black or red such that at most half of every node's neighbors have the same color as the node itself. As a first step, I'd like to show that this is always possible.
I believe this problem is the essence of the math quiz #2 in Communications of the ACM 02/2011 where I found it, so you might consider this a homework-like question. The quiz deals with partitions but I found it more natural to formulate the problem as a graph-coloring problem.
Coming from computer science with some math interest I'm not sure how to approach this and would be glad about some hints. One observation is that any node of degree 1 forces its neighbor to be of the opposite color. This could lead to a constructive proof (or a greedy algorithm) that provides a coloring. However, an existence proof would be also interesting.
Answer: Hint: Start with a random colouring and try to increase the number of edges which have differently coloured endpoints.
Spoiler:

 Pick a node which has more than half of it's neighbour of the same colour as itself and flip it's colour. Now show that, as a result, the number of edges with different coloured endpoints increases by at least 1. Repeat.
Answer: Perhaps related is the following well-known riddle: At each step, all the nodes (simultaneously) choose their color as the majority color of their neighbors (or black in case of a tie). Show that this process converges to either a fixed point or a $2$-cycle.
This charming riddle is solved by comparing the state at time $t$ to the state at time $t+2$, and using a potential function. This solution was published, and generalized in a sequence of papers.
In your case, you want to take the anti-majority. Perhaps the same techniques work, though I'm not sure how a $2$-cycle will help you.


Homology other than singular?

Question: Usually, one defines $n$-th homology functor on topological spaces as the composite functor
$$ \mathbf{Top} \to [\Delta^\mathrm{op},\mathbf{Set}] \to [\Delta^\mathrm{op},R\!-\!\mathbf{Mod}] \overset C \to \mathfrak{Ch}(R\!-\!\mathbf{Mod}) \to R\!-\!\mathbf{Mod} $$
where the first arrow is $X \mapsto \hom (\Delta^\mathrm{top}_\bullet, X)$ mapping a topological space to its simplicial set of singular simplexes,  the second arrow is induced by the free $R$-module functor, and the last one is the homology functor of chain complexes. Finally, $C$ maps $M_\bullet$ to
$$ \dots \to C_n \overset {d_n} \to C_{n-1} \to \dots $$
with $d_n = \sum_{i=0}^n (-1)^i \partial^n_i$ where the $\partial^n_i$ are the face operators.
So, defining the functor $H_n \colon \mathbf{Top} \to R\!-\!\mathbf{Mod}$, every step is pretty natural and kind of canonical except for the choice of the $d_n$ : we have to choose the functor $C$. I understand the motivation behind the chosen derivation, but I wonder if there are other functors $C$ which give interesting homology spaces. I have been told more or less that there is not a lot of choice for such a $C$. Why is that ? Is there theoretically very few choices (and how can I see it) ? Or is it that for a general functor $C$, we can't tell anything (in fact, is it even an homology theory according to Eilenberg-Steenrod axiomatic) ?
(I am aware of the normalization functor (Dold-Kan correspondence) that is a good choice of $C$, but then the homology is the usual one, so...)
Comment: I think your last comment about E-S axioms is right on the money. Since the homology theory is determined by coefficients there is really not much choice and the Dold-Kan correspondence is as a good a choice as any.
Answer: The book "Nonabelian algebraic topology:filtered spaces, crossed complexes, cubical homotopy groupoids" (published by the EMS, 2011), takes a nontraditional approach to the border between homology and homotopy.
The usual way, as you say,  is to replace a space $X$ by its simplicial singular complex $SX$ and then make constructions on that. A key tool is simplicial approximation. After some work, one is able to define the cellular chains of a $CW$-complex, and by applying this to a universal cover of a reduced $CW$-complex $X$ to obtain a chain complex with the fundamental group of $X$ as a group of operators.
Now the geometric realisation $Y=|SX|$ is a CW-complex and so is filtered by its skeleta, i.e. is a filtered space, i.e. has an increasing sequence  $Y_n$ of subspaces.
The approach in the book is to define a (actually somewhat classical) functor $\Pi: FTop \to Crs$ where $FTop$ is the category of filtered spaces, and $Crs$ is the category of what are called crossed complexes . This functor is defined homotopically using the fundamental groupoid $\pi_1(Y_1,Y_0)$ and the relative homotopy groups $\pi_n(Y_n,Y_{n-1},y), y \in Y_0$. This is a stronger structure than a chain complex with a group(oid) of operators. It has nonabelian features in dimensions $1,2$ and in higher dimensions gives $\pi_1$-modules. No use is made of simplicial approximation, but a key use is made of cubical methods: they are convenient because they allow  "algebraic inverses to subdivision", leading to higher order Seifert-van Kampen Theorems, and because of the rule $I^m \times I^n \cong I^{m+n}$. This leads to a monoidalk closed structure for crossed complexes.
There is also a functor $\nabla$ from crossed complexes to chain complexes with a groupoid of operators; this functor has a right adjoint, and for a $CW$-complex $X$ with its skeletal filtration, $\nabla \Pi (X_*)$ is indeed the chain complex of universal covers of $X$  at various base points, with the fundamental groupoid $\pi_1(X,X_0)$ as groupoid of operators. However $\Pi X_*$ has better realisation properties that the corresponding chain complex.
A number of these ideas go back to J.H.C. Whitehead in his 1949 paper "Combinatorial homotopy II", but the use of cubical methods in this way, of  colimit methods for calculation, and of monoidal closed categories, all  enable a better setting for Whitehead's ideas.
There are many issues, for example the use of $R$-modules for a ring $R$, and the book concludes with a list of 32 problems and problem areas.
Also included is history, motivation, and many calculations, for example of homotopy $2$-types via crossed modules, not available to traditional methods.
The work for the book started in the 1970s, and involved many colleagues  and research students.
The aim of the publication was
to put all this work in one place in a consistent style so that it would be easier to evaluate.

Homology other than singular?


IllegalLocationConstraintException whilst creating an S3 bucket in CFN resource provider test

Question: I'm creating a Cloudformation Resource Provider using the cloudformation cli.  When I try to test my resource locally using cfn test I get the following error:
An error occurred (IllegalLocationConstraintException) when calling the CreateBucket operation: The us-east-2 location constraint is incompatible for the region specific endpoint this request was sent to.

My code looks as follows:
s3 = session.client("s3")
s3.create_bucket(Bucket='mybucket123',CreateBucketConfiguration={'LocationConstraint': 'us-east-2'})

I've also tried other regions and have tried without the LocationConstraint and I get the same error with the word undefined where the region code is:
s3 = session.client("s3")
s3.create_bucket(Bucket='mybucket123')

cfn test actually runs the code within sam local using sam local start-lambda so I'm a bit perplexed as to why I would even need a region here?
Any ideas?
Answer: Your client must also be set to the region you want:
s3 = session.client("s3", region_name='us-east-2')
s3.create_bucket(Bucket='mybucket123',CreateBucketConfiguration={'LocationConstraint': 'us-east-2'})

Comment: bingo!  that worked.  Thanks
Answer: You can simply add and override location constraints.
Copying s3 bucket files to local example
By using AWS CLI: (configuration required)
aws s3 sync --region <request-bucket-region> --source-region <default-
region> s3://<bucket-name>/<path-if-specific> ~/<local-machine-directory>


Making a img appear/disappear in html5

Question: I've used a button at this link:
Positioning Images in HTML (Using a button creating images)
...But I want to know how to make the image disappear if it's already been placed by using a button. Hopefully this can use the same button. Any ideas?
Comment: Can you please update your question to show the relevant code (HTML, CSS and JavaScript)? Ideally *also* provide a link to a [JS Fiddle demo](http://jsfiddle.net/) showing your code in operation.
Comment: Working w/ JS fiddle.
Answer: give the image and id="myImage"
and then
if(myImage.style["visibility"]=="visible"){
myImage.style["visibility"]="hidden";
else
{
myImage.style["visibility"]="visible";
}

Comment: Well if reloading works for you then cool, but it will make you whole page blink and involves more data, if u dont mind then ok.  And yea, the function where you create the image set an id for it and u can simply use that id to remove it.
Comment: Wait, I'm making the image appear after you open the web page, so how do I attach the id to the image? Should I add another section into the function? Thanks for the answer, btw
Comment: I found that I could just use the location.reload() function to reload the website. (This wouldn't selectively remove images I placed though, and they would have to be originally not there/hidden)
Comment: Reloading the website is not effective.  You wrote that you placed the image by clicking a button,  so in that function that places the image make it give the image an id or class and use those
Comment: It works for me... Whatever, I'll try. Should I img.id in the function I use to place the image?
Comment: The button only worked once though, the rest of the times it wouldn't remove any more of the pictures unless I reload it.
Comment: Make sure the images have uniqye id each. If you want to let ur images have something in common, assign a class to them
Answer: You can hide the image by setting its 'display' attribute to 'none'(hidden) or 'initial'(visible). Let your button trigger a function that swaps the state of the image, as follows (Javascript):
var img = document.getElementById('yourImageID');

function toggleImg(){
if ( img.style.display == 'none' ){ img.style.display = 'initial'; }
else { img.style.display = 'none'; }
}

The attribute 'visibility' will keep the space of the image free and just blank it, where 'display' will not keep the space.

Making a img appear/disappear in html5


spring-data-neo4j in "Advanced Mapping" mode - How can I read an entity-value without starting a transaction manually?

Question: In my JavaEE7-project, I am using spring-data-neo4j standalone in "Advanced Mapping" mode (using spring-aspects). Everything works fine so far: CRUD on entities within a transaction, where the transaction is started manually or via @Transactional-annotation.
In my usecase, my view accesses an entity "directly":
<code>// User
@NodeEntity
public class User {

    private String firstName;

    // getter, setter, ...
}

// SessionBean
@SessionScoped
@Named
public class SessionBean {

    @Transactional
    public User getUser() {
         User user = ...;
         System.out.println(user.getFirstName()); // (1) gives firstName-value.
         return user;
    }
}

// sometpl.xhtml
${sessionBean.user.firstName} // (2) gives "null".
</code>
Somehow, this behavior (difference between (1) and (2)) is wanted, as spring-data-neo4j supposes read-access only within a transaction.
But I want to have my usecase(2) working (returning the user's firstName, not "null"). Is there any way to achieve this? So let's say, starting transaction automatically in read-access-case? Implicit read-transactions-support?
Comment: Neo4j 2.0 added mandatory read transactions. But I'm not sure why it returns null. What happens if you test your getUser() method w/o transactional? The user entity is not detached automatically
Comment: So it makes more sense to create a vie object within your tx method that contains all data from your users and their relationships
Comment: Removing Transactional at getUser() -> user.getFirstName() returns null.

Btw, adding Transactional to User::getFirstName() solves my problem, too.

Using a view object could be a workaround; anyhow, IMO it would be best to use the entites in the view directly.
Answer: My workaround:
Use a RequestScoped bean to start a transaction in "preRenderView" and to close this tx when the bean is destroyed.
This does not work on ajax-calls!
<code>@Named
@RequestScoped
public class SpringDataNeo4jHelperBean {

  @Inject
  @Named
  private Neo4jTemplate neoTemplate;

  private Transaction tx;

  @PreDestroy
  public void finishTransaction() {
    if (this.tx != null) {
        this.tx.success();
        this.tx.finish();
    }
  }

  public void startReadOnlyTransaction() {
    if (!this.neoTemplate.getGraphDatabase().transactionIsRunning()) {
        this.tx = this.neoTemplate.getGraphDatabaseService().beginTx();
    }
  }
}
</code>
In some template, for example s.th. like a central layout.xhtml:
<code><f:metadata>
    <f:event type="preRenderView" listener="#{springDataNeo4jHelperBean.startReadOnlyTransaction()}" />
</f:metadata>
</code>