ORA-22927: invalid LOB locator specified

Question: I have the plsql code to update a file content in blob into a table as below,
CREATE OR REPLACE PACKAGE BODY XXTXB_UPLOAD_FILE_PKG AS
PROCEDURE FILE_UPLOAD(P_FILE_NAME IN VARCHAR2,P_FILE_CONTENT IN BLOB) AS
GN_USER_ID       NUMBER:= APPS.FND_GLOBAL.USER_ID;
GN_LOGIN_ID      NUMBER:= APPS.FND_GLOBAL.LOGIN_ID;
LV_SEQ_NUM NUMBER:=XXTXB_UPLOAD_STG_S.NEXTVAL;
LV_FILE_NAME VARCHAR(500);
LV_FILE_PATH VARCHAR2(500):='/u01/Jdevloper/xxtxb';
LV_FILE      UTL_FILE.FILE_TYPE;
LV_BUFFER    RAW(32767);
LV_AMOUNT    BINARY_INTEGER := 32767;
LV_POS       INTEGER := 1;
LV_BLOB_LEN  INTEGER;
LV_USER_NAME VARCHAR2(100);
BEGIN
--Getting user name
SELECT USER_NAME INTO LV_USER_NAME FROM FND_USER WHERE USER_ID=GN_USER_ID;
--Server File Name
SELECT
TO_CHAR(SYSDATE,'yyyymmddhh24miss')||'_'||LV_USER_NAME||'_'||P_FILE_NAME
INTO LV_FILE_NAME FROM DUAL;
--Inserting into the staging table about the details
INSERT INTO XXTXB_UPLOAD_STG
VALUES (LV_SEQ_NUM,GN_USER_ID,
        P_FILE_NAME,
        LV_FILE_NAME,
        P_FILE_CONTENT,
        LV_FILE_PATH,
        SYSDATE,
        GN_USER_ID,
        SYSDATE,
        GN_USER_ID,
        GN_LOGIN_ID);
COMMIT;
DBMS_OUTPUT.PUT_LINE('TABLE UPDATED SUCESSFULLY');
--Length of the file
LV_BLOB_LEN := DBMS_LOB.GETLENGTH(P_FILE_CONTENT);
--Opeing the file
LV_FILE := UTL_FILE.FOPEN(LV_FILE_PATH,LV_FILE_NAME,'wb', 32767);
--Writing into the file
WHILE LV_POS < LV_BLOB_LEN LOOP
DBMS_LOB.READ(P_FILE_CONTENT, LV_AMOUNT, LV_POS, LV_BUFFER);
UTL_FILE.PUT_RAW(LV_FILE, LV_BUFFER, TRUE);
LV_POS := LV_POS + LV_AMOUNT;
END LOOP;
--Closing the file
UTL_FILE.FCLOSE(LV_FILE);
EXCEPTION
WHEN OTHERS THEN
IF UTL_FILE.IS_OPEN(LV_FILE) THEN
UTL_FILE.FCLOSE(LV_FILE);
END IF;
RAISE;
END FILE_UPLOAD;
END XXTXB_UPLOAD_FILE_PKG;
/

I'm calling this procedure in OAF in Impl code as follows:
public void WriteToServer(String P_FILE_NAME,String s) throws
SerialException,SQLException
{
byte[] buff = s.getBytes();
BLOB P_FILE_CONTENT=new BLOBChild();
P_FILE_CONTENT.setBytes(buff);
OADBTransaction oadbtransaction = (OADBTransaction)getTransaction();
OADBTransactionImpl oadbtransactionimpl =
(OADBTransactionImpl)getTransaction();
  StringBuffer str = new StringBuffer();
  str.append( " BEGIN ");
  str.append( " XXTXB_UPLOAD_FILE_PKG.FILE_UPLOAD( ");
  str.append( " P_FILE_NAME=>:1,");
  str.append( " P_FILE_CONTENT=>:2");
  str.append( " ); ");
  str.append( " END; ");
OracleCallableStatement oraclecallablestatement =(OracleCallableStatement)
oadbtransaction.createCallableStatement(str.toString(),1);
 try
 {
   oraclecallablestatement.setString(1, P_FILE_NAME);
   oraclecallablestatement.setBLOB(2, P_FILE_CONTENT);
   oraclecallablestatement.execute();
 }
 catch(Exception e)
 {
  throw OAException.wrapperException(e);
 }
 }

When I try to update the file to the server using OAF I'm facing the following error,
oracle.apps.fnd.framework.OAException: java.sql.SQLException: ORA-22927:
invalid LOB locator specified

at oracle.apps.fnd.framework.OAException.wrapperInvocationTargetException(Unknown Source)
at oracle.apps.fnd.framework.server.OAUtility.invokeMethod(Unknown Source)
at oracle.apps.fnd.framework.server.OAUtility.invokeMethod(Unknown Source)
at oracle.apps.fnd.framework.server.OAApplicationModuleImpl.invokeMethod(Unknown Source)
at xxtxb.oracle.apps.fnd.FileUpload.webui.FileUploadCO.processFormRequest(FileUploadCO.java:73)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanContainerHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageLayoutHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.beans.layout.OAPageLayoutBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanContainerHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.beans.form.OAFormBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanContainerHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.beans.OABodyBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(Unknown Source)
at _OA._jspService(_OA.java:75)
at com.orionserver.http.OrionHttpJspPage.service(OrionHttpJspPage.java:59)
at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:473)
at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:594)
at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:518)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:734)
at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:391)
at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:908)
at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:458)
at com.evermind.server.http.HttpRequestHandler.serveOneRequest(HttpRequestHandler.java:226)
at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:127)
at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:116)
at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
at oracle.oc4j.network.ServerSocketAcceptHandler.procClientSocket(ServerSocketAcceptHandler.java:234)
at oracle.oc4j.network.ServerSocketAcceptHandler.access$700(ServerSocketAcceptHandler.java:29)
at oracle.oc4j.network.ServerSocketAcceptHandler$AcceptHandlerHorse.run(ServerSocketAcceptHandler.java:879)
at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303)
at java.lang.Thread.run(Thread.java:662)

and the corresponding table is,
CREATE TABLE xxtxb_upload_stg
(
  file_id           NUMBER        NOT NULL,
  uploaded_by       NUMBER        NOT NULL,
  upload_file_name  VARCHAR2(300) NOT NULL,
  server_file_name  VARCHAR2(300) NOT NULL,
  file_content      BLOB,
  file_path         VARCHAR2(500),
  -- --------------------
  -- Who Columns
  -- --------------------
  last_update_date   DATE     NOT NULL,
  last_updated_by    NUMBER   NOT NULL,
  creation_date      DATE     NOT NULL,
  created_by         NUMBER   NOT NULL,
  last_update_login  NUMBER
 );

Controller Code:
import java.io.ByteArrayOutputStream;
import oracle.apps.fnd.common.VersionInfo;
import oracle.apps.fnd.framework.webui.OAControllerImpl;
import oracle.apps.fnd.framework.webui.OAPageContext;
import oracle.apps.fnd.framework.webui.beans.OAWebBean;
import oracle.cabo.ui.data.DataObject;
import java.io.InputStream;
import oracle.jbo.domain.BlobDomain;
import oracle.apps.fnd.framework.OAException;
import oracle.apps.fnd.framework.OAApplicationModule;
import java.io.Serializable;
public class FileUploadCO extends OAControllerImpl
{
 BlobDomain uploadedByteStream=new BlobDomain(); ;
 String P_FILE_NAME="";
 String s=new String();
 public static final String RCS_ID="$Header$";
 public static final boolean RCS_ID_RECORDED =
 VersionInfo.recordClassVersion(RCS_ID, "%packagename%");
 public void processRequest(OAPageContext pageContext, OAWebBean webBean)
{
 super.processRequest(pageContext, webBean);
}
public void processFormRequest(OAPageContext pageContext, OAWebBean webBean)
{
super.processFormRequest(pageContext, webBean);

if(pageContext.getParameter("Submit")!=null)
{

 upLoadFile(pageContext,webBean);

  OAApplicationModule am = pageContext.getApplicationModule(webBean);
  Serializable[] parameters1 = {P_FILE_NAME,s};
  am.invokeMethod("WriteToServer", parameters1);
}
}
public void upLoadFile(OAPageContext pageContext,OAWebBean webBean)
{
String fileUrl = null;
try
{
 System.out.println("URL---->"+fileUrl);
 DataObject fileUploadData =
 pageContext.getNamedDataObject("MessageFileUpload");
 if(fileUploadData!=null)
 {

 P_FILE_NAME = (String)fileUploadData.selectValue(null, "UPLOAD_FILE_NAME");
 String contentType = (String)
 fileUploadData.selectValue(null,"UPLOAD_FILE_MIME_TYPE");
 System.out.println("User File Name---->"+P_FILE_NAME);

 InputStream input = null;

 uploadedByteStream = (BlobDomain)fileUploadData.selectValue(null,
 P_FILE_NAME);

 input = uploadedByteStream.getInputStream();
  byte[] contents;
  ByteArrayOutputStream byteoutput = new ByteArrayOutputStream();
  byte[] buffer = new byte[1024];
  int count;
  while ((count = input.read(buffer)) != -1){byteoutput.write(buffer, 0,
  count);}//debugger says myinputstream has blksize 16384, buffcount 12742,
  and max 127394 here
  contents = byteoutput.toByteArray();
  s=contents.toString();

  input.close();
  }
 }
 catch(Exception ex)
 {
  throw new OAException(ex.getMessage(), OAException.ERROR);
 }

 }

}
Could anyone help me on this....
After completing the SQL session error as suggested now I get this conversion error.
Exception Details.
oracle.apps.fnd.framework.OAException: java.sql.SQLException: ORA-00603:
ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [kghfrmrg:nxt], [0x7F069D63B360],
[], [], [], [], [], [], [], [], [], []
ORA-24812: character set conversion to or from UCS2 failed
ORA-06512: at "APPS.XXTXB_UPLOAD_FILE_PKG", line 68
ORA-06512: at line 1

at oracle.apps.fnd.framework.OAException.wrapperInvocationTargetException(Unknown Source)
at oracle.apps.fnd.framework.server.OAUtility.invokeMethod(Unknown Source)
at oracle.apps.fnd.framework.server.OAUtility.invokeMethod(Unknown Source)
at oracle.apps.fnd.framework.server.OAApplicationModuleImpl.invokeMethod(Unknown Source)
at xxtxb.oracle.apps.fnd.FileUpload.webui.FileUploadCO.processFormRequest(FileUploadCO.java:73)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanContainerHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageLayoutHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.beans.layout.OAPageLayoutBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanContainerHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.beans.form.OAFormBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequestChildren(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAWebBeanContainerHelper.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.beans.OABodyBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.processFormRequest(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(Unknown Source)
at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(Unknown Source)
at _OA._jspService(_OA.java:75)
at com.orionserver.http.OrionHttpJspPage.service(OrionHttpJspPage.java:59)
at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:473)
at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:594)
at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:518)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:734)
at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:391)
at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:908)
at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:458)
at com.evermind.server.http.HttpRequestHandler.serveOneRequest(HttpRequestHandler.java:226)
at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:127)
at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:116)
at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
at oracle.oc4j.network.ServerSocketAcceptHandler.procClientSocket(ServerSocketAcceptHandler.java:234)
at oracle.oc4j.network.ServerSocketAcceptHandler.access$700(ServerSocketAcceptHandler.java:29)
at oracle.oc4j.network.ServerSocketAcceptHandler$AcceptHandlerHorse.run(ServerSocketAcceptHandler.java:879)
at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303)
at java.lang.Thread.run(Thread.java:662)

Comment: I don't see that error - ORA-22927: invalid LOB locator specified - in your callstack.  Is that a cut and paste error?  Does the PL/SQL work if you call your function from a SQL session?
Comment: oracle.apps.fnd.framework.OAException: java.sql.SQLException: ORA-
    22927: invalid LOB locator specified   This is the error I got when I run the Impl code from jDeveloper
Comment: ...but does it work from a SQL session?  If it only fails when called from Java I'll look at your AM code.  If it fails from SQL then of course we need to look at your PL/SQL.  Might be an idea to add some debug into both your AM and PL/SQL code so you can see where this falls over.  The last debug message I get is from your controller : FileUploadCO.processFormRequest and this does not help us to locate the problem
Comment: Hi  Christian Palmer, I have updated the CO in the Blog. I'm so new to this jDeveloper environment. just know core java. Trying to catch up the things. My need is to pass the file name and file content as Blob to PLSQL procedure. So that the PLSQL procedure catch the blob and write it into a table.    Help me on this
Answer: Disclaimer : This is not a complete answer (yet) but it is probably too much to put into a comment.
The core routine is your file_upload() PL/SQL routine - you need to get this working from a SQL session to make sure it works before you start calling it from a java session.  I am pretty sure that file_upload() does not work right now:
UTL_FILE - you are passing a raw directory path as the location parameter.  This is not correct - you should be passing the name of a directory from ALL_DIRECTORIES.  So your first step is to create a row in here for your path :
SQL> CREATE DIRECTORY XXTXB_FILE_DIR AS '/u01/Jdevloper/xxtxb';

Then when you'll call UTL file with something like :
lv_file_handle :=
 UTL_FILE.FOPEN
  ('XXTXB_FILE_DIR'
  ,LV_FILE_NAME
  ,'wb'
  , 32767
  )
;

Note this all assumes /u01/Jdevloper/xxtxb is a valid directory.
Once you have the package compiled then you can call it from a SQL session :
BEGIN

  XXTXB_UPLOAD_FILE_PKG.FILE_UPLOAD
   (P_FILE_NAME    => 'test.txt'
   ,P_FILE_CONTENT => utl_raw.cast_to_raw('blah')
   )
  ;

END ;

If this throws errors then debug here before you start calling the routine from java.  Let me know how you get on...
Comment: Hi Christian. Thanks for your guidelines. I had taken down the thinks as you guided. Now the PLSQL procedure runs fine on SQL session. Now i'm looking into the OAF part will return back to you. Thanks once again Christian....
Comment: Hi Christian after completing the SQL session error now I get some Conversion error. This comment segment is not enough to paste that so I will edit the blog and add the error log atlast.
Comment: OK I don't have much experience with character encoding.  However your PL/SQL does two operations on the BLOB : 1 - insert into a BLOB column 2 - write the BLOB contents to the filesystem.  I would comment one and then the other to see which is causing the problem (it could of course be both)
Comment: Okay Christian Will try the same
Comment: oracle.apps.fnd.framework.OAException: Message not found. Application: FND, Message Name: FND_MESSAGE_NOT_FOUND. Message not found. Application: FND, Message Name: FND_GENERIC_MESSAGE. Tokens: MESSAGE = java.sql.SQLException: ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [kghfrmrg:nxt], [0x7F6D968B7FB0], [], [], [], [], [], [], [], [], [], []
ORA-24812: character set conversion to or from UCS2 failed
ORA-06512: at "APPS.XXTXB_UPLOAD_FILE_PKG", line 68
ORA-06512: at line 1;            **Can you guide on this Christian**
Comment: have you managed to isolate what is throwing this error - the table insert or writing the data to the filesystem?
Comment: as I said, character encoding is not a specialism of mine.  It seems likely that your java environment is using UCS2 and your database is not (SELECT * FROM nls_database_parameters).  You may need to convert the encoding of your text - this link might help : https://stackoverflow.com/q/10942900/6171686


Render a webform to another node

Question: I have a new text format sample_page with the filter sample_page, I've created using hook_filter_info(). In the callback, I am rendering a webform using depending on the token in the node body, for webform I'm planning to use a token [block:webform:webform-client-form-2423].
I could render the blocks created programmatically, system blocks. But not the blocks created by other modules, I have tried [block:user:login] from http://mine.local/admin/structure/block/manage/user/login/configure to render to login blocks too. But these blocks didnt seems to work. Here is my callback for filter.
System blocks that worked for me, I used the token [block:block:25] from http://mine.local/admin/structure/block/manage/block/2/configure and for blocks created programatically by my own module, suppose my module is my_module,
I have used the token [block:my_module:block_name] from the url http://mine.local/admin/structure/block/manage/my_module/block_name/configure
<code>preg_match_all('/\\[([a-zA-z0-9_\-]+:[a-zA-z0-9_\-]+)+\\]/m', $text, $match);
$matches = $match[0];
foreach ($matches as $token) {
  $token_name = str_replace(array( '[', ']' ), '', $token);
  $type = explode(':', $token_name);
  if ($type[0] == 'block') {
    $module = $type[1];
    $bid = $type[2];
    $block = block_load($module, $bid);
    $replacement = render(_block_get_renderable_array(_block_render_blocks(array($block))));
  }
}
</code>
Can any one look into the problem. Any help would be appreciated...
Comment: Would something like 'Nodes in Block' work for you.   https://www.drupal.org/project/nodesinblock
Answer: You can add the webform using hook_page_alter().
<code>/**
 * Implements hook_page_alter().
 */
function my_module_page_alter(&$page){

  // Get the webform's nid that you want to display on the current page.
  $webform_nid = my_module_webform_to_display();

  // Load the webform node.
  $webform = node_load($webform_nid);

  // Render the webform using my custom view mode.
  $markup = drupal_render(node_view($webform, 'my_custom_view_mode'));

  // Add the markup to the page element.
  $page['content']['system_main'][] = array(
    '#markup' => $markup,
  );
}
</code>
Comment: I was wondering if could add it using https://api.drupal.org/api/drupal/modules!filter!filter.api.php/function/hook_filter_info/7. Also I wanted to handle/ render the blocks by other modules.

Render a webform to another node


why I got an output 'b(1) dc dvfunc'?

Question: My expected output here was " bc bvfunc b(1) dc dvfunc", but I got an output like "b(1) dc dvfunc" why is it so? Could somebody help me out? thanks for your precious time!
#include<iostream>

using namespace std;

class b {
 public:
  b() {
    cout<<" bc ";
    b::vfunc();
  }
  virtual void vfunc(){ cout<<" bvfunc "; }
  b(int i){ cout<<" b(1) "; }
};

class d : public b {
public:
  d(): b(1) {
    cout<<" dc ";
    d::vfunc();
  }
  void vfunc(){ cout<<" dvfunc"; }
};

main()
{
  d d;
}

Answer: The order in which things are done:
d() is called. This calls b(1), and then the rest of the constructor.
So the call order is
b(1)
d() -> which is cout fc, and then cout dvfunc

b() is never called, so it will not reach bvfunc.
both b() and b(int i) are stand-alone constructors, and only one will be called, not both.
Answer: To get the desired output you need
d(){b(1);      //move b(1) from initializer list and put it in a constructor.
    cout<<" dc ";

FYI initializer list is used to initialize the member of a class before the constructor call to default value.Constructor can overwrite these values.
Comment: What?! This creates two *different* `b`'s, one that is part of `d` and one temporary that just lives inside the constructor. Hardly useful for anything (except reproducing the output).

why I got an output 'b(1) dc dvfunc'?


Não consigo clicar no botão desse site com Python Selenium de jeito nenhum

Question: Estou fazendo uma automação para uma melhoria de processo no meu trabalho, a idéia é varrer regularmente uma pagina específica e identificar se há um novo item nessa página. Ja postei uma dúvida sobre esse projeto aqui no Stackoverflow e o meu problema era ir para o frame correto do botão, e essa etapa foi superada.
Porém agora existe um botão que é o 'Query' para fazer a pesquisa na página e aparecer se há um novo item, porém esse botão Query é IMPOSSÍVEL de clicar, sempre retorna o erro 'NoSuchElementException' ja tentei muitas coisas, esperando o elemento carregar por 1h, colocando um sleep, findelement por ID, XPATH, fullXPATH e algumas outras opções que seriam possíveis achar esse botão, ele esta no mesmo frame de outros botões que eu consigo interagir, mas esse em específico eu não consigo.
Abaixo segue todos os endereços do botão, mostrando o elemento que ele é, e de nenhuma forma ele funciona.
-A barra onde o botão esta

-O botão

-Campo dentro do botão

-Imagem da lupa do botão

-Texto do botão

-O Menu lateral esquerdo e essa barra de botões estão no mesmo frame, eu consigo clicar no menu lateral esquerdo, porém não consigo interagir com o a barra de botões, em nenhum deles por nenhum tipo de find_element possível.

Sou iniciante e tenho muito a aprender, mas ja estou estacionado nesse ítem faz algumas semanas e ainda sem nenhum avanço, toda sugestão é bem vinda.
Answer: Ricardo, já passei por um problema semelhante e o que me salvou foi o execute_script, uma função do selenium.webdriver.
Tenta utilizar ela assim se tiver com a versão mais atualizada do Selenium:
btn = webdriver.find_element(By.CLASS_NAME, 'nexacontentsbox')
webdriver.execute_script("arguments[0].click();", btn)

Outra opção caso esteja em uma versão mais antiga:
btn = webdriver.find_element_by_class_name('nexacontentsbox')
webdriver.execute_script("arguments[0].click();", btn)

Pra utilizar o execute_script é bem simples, você só precisa enviar como parâmetro o primeiro comando a ser executado (No exemplo acima estamos enviando um script com um simples click), e como segundo parâmetro o elemento em que o script vai funcionar.
Comment: Obrigado pela resposta Zé Henrique, tentei utilizar esse comando da forma que você escreveu mas não funcionou, mudei o método de busca mas ainda sim tenho a mesma falha 'AttributeError: module 'selenium.webdriver' has no attribute 'find_element'' vou dar uma estudada e aprender essa função e ver se acho a minha solução, valeu pela dica!


Identifiying performance of individual feature components in accuracy of support vector methods

Question: I'm using support vector regression (not classification) for a problem and it's working well.
However, in the older method that former lab members developed (a basic linear model, with weights determined just from OLS), there is a lot of code used to determine the individual efficacy of each different right-hand-side variable from their OLS. I'm not all that impressed with these efficacy stats (they are mostly summary stats of correlations, t-stats, etc.) but they do allow you to have some notion of which RHS variables are most informative.
In my case, for each individual datum, I use the whole row's worth of right-hand-side variables as the feature vector for support vector regression. So I am wondering if there is any commonly used analogue, maybe some measure from information theory or something, that gives an indication of how each marginal component of the overall feature vector contributes to the end performance.
One thing that does not seem to work is training N different regressions each using only a single one of the feature components as the sole predictor. This does give some matches between the univariate regression efficacy and univariate support vector efficacy, but it doesn't capture the interactions between the different predictors, and (for example) which predictor is the "second best" predictor among all the components of the feature vector.
I suppose I could work backward and test the results of removing one component at a time, to see which then causes the biggest drop in performance... but I'd prefer something less ad-hoc and perhaps backed by some statistical rigor or intuition.
Answer: There are a number of ways of performing relevance determination / feature selection in general. One common way is using an algorithm called Recursive Feature Elimination (RFE) - it's implemented in scikit-learn. This effectively computes a different objective function to what the SVM normally uses to assess the relevant of a subset of feature and keep discarding features while this objective function is increasing. It's expensive as it means training an SVM a number of times and you don't want to over-fit so these results should also be cross-validated. Here's an example implementation. Another method of feature selection is to use a ANOVA to assess the relative importance of each feature in the training set on the determination of the associated label. This should extend to the case of regression. Here's an explanation that uses a Chi squared test. It's also implemented in scikit-learn if I remember correctly.
EDIT: Found it! Chi Squared Feature selection in scikit-learn.
For an excellent overview of the different strategies of feature selection in general have a look at Guyon's paper. This breaks feature selection down into three classes (wrapper, filter and embedded methods) depending on where in the pipeline it is done. It also discusses the merits of each.
Something worth mentioning is Multiple Kernel Learning (MKL) where you learn a weight across a number of kernels of SVMs. This is more complicated but powerful framework for implicitly solving the feature selection problem. MKL is not implemented in scikit-learn, but it is implemented extensively in the Shogun toolkit. EDIT: Example. There is also an implementation called SimpleMKL in MATLAB. EDIT: Here's the code
MKL allows for a large amount of flexibility in incorporating different kernels and heterogenous datasources as well as performing feature selection at the same time. Down side is that it is slower, but if you are going to use RFE anyway then MKL might be a good choice. Two important papers showing different MKL algorithms are SimpleMKL MATLAB toolbox is also available, and Lp-Norm MKL implemented in Shogun. Ultimately SimpleMKL uses gradient descent to solve the outer problem and is fast. The other method uses cutting planes to lower-bound the objective function and is slow but can tackle large scale data. Recently they've even taken the limit of this process to Infinite Kernel Learning.
Parameter optimisation is also expensive in MKL as there are sometimes too many parameters for a grid search so I would suggest a random search, or wait for my dissertation :)
Aside: If you aren't constrained to SVM's I would also look at Gaussian processes. Specifically with an Automatic Relevant Determination (ARD) kernel. If you optimize the marginal log-likelihood of the model the ARD parameters will give you an understand of the importance of each feature in your data. Check our Rasmussen William textbook free online. The documentation for the included MATLAB toolbox includes an example doing exactly this. Scroll down to: "We specify a Gaussian process model as follows"
Hope that helps.
Comment: This is excellent. Thank you for the well formed answer.


What happens if the non-striker doesn't run but the batsman on strike does?

Question: If the batsman on strike calls "yes" and runs to the non-striker's end, but the non-striker doesn't run, who would be run out if the wicket at the striker's end was broken?
Answer: If the non-striker doesn't leave his crease, then the striker is out.
See clause 3 of Law 38 - run out


Find out if system (Windows) was booted with BIOS or UEFI with Python 3

Question: How do I find out if Windows was booted with UEFI or BIOS with Python?
Answer: According to this website, you can do something like that:

Alternative 1: Reading a command output

import subprocess

out = subprocess.check_output(['bcdedit']).decode('utf-8')

is_loader = False
for line in out.split('\n'):
    # Ignore lines until the Windows Boot Loader section
    if not is_loader and 'Windows Boot Loader' in line:
        is_loader = True

    if not is_loader:
        continue

    # Ignore lines until the path subsection
    if not line.startswith('path'):
        continue

    # Receives 'EXE' (BIOS) or 'EFI' (UIEF)
    boot_type = line[-3:].upper()

    # You can also use an if-else expression
    # Receives 'BIOS' or 'UIEF'
    # boot_type = 'BIOS' if line[-3:] == 'exe' else 'UIEF'

Alternative 2: Reading a log file

import re

with open(r'C:\Windows\Panther\setupact.log') as f:
    pattern = re.compile(r'Detected boot environment: (\w+)')

    # Iterate over every line of file until finds a match
    for line in f:
        match = pattern.search(line)
        if match:
            # Receives 'BIOS' or 'UEFI'
            boot_type = match.group(1).upper()
            break


PHP: include and file_exists scope

Question: I have a following directory structure: one folder "class" with "Test.php" and "conf.php" in it and one "index.php" file in the root:
/
index.php
class
   Test.php
   conf.php

The content of Test.php is:
<?php
class Test {

  public function __construct(){
    var_dump(file_exists("conf.php"));
    $conf = include("conf.php");
    echo $conf;
  }

}

"index.php" has following contents: (Just trying to instantiate a new Test class)
<?php
include "class/Test.php";
$Test = new Test();

And the file "conf.php" contains only this:
<?php
return "Included file";

After running this code I see the following output:
bool false
"Included file"

As you can see the "Test" object can include its local "conf.php" file and show its output,
but file_exists() doesn't see nothing. I don't understand why.
Maybe it is a php bug?
file_exists() returns "true" if I place the "conf.php" into the root (together with "index.php"). It seems that include is using the scope of the "Test.php" and "file_exists()" is using the scope of the file where the Test object were created (in the root in my case). What is the correct behavior?
(Using PHP 5.5.7 Fast-CGI)
Comment: When you call `file_exists()` without a full path, it checks from the location of the main script, not the location of the script that contains the function call. So if you include `Test.php` from `index.php`, the `file_exists()` is looking for `conf.php` in the parent directory, where it doesn't exist.  You have to think about it like an include is pulling all the code into that script, so everything is relative to that location.  You can use `__DIR__` in an included file to get the path to the actual included file.
Comment: `include` and `require` make use of `include_path`, but `file_exists()` and other filesystem functions expect real file paths.  From the root, `conf.php` is not found as a relative path.
Comment: possible duplicate of [PHP: Can include a file that file\_exists() says doesn't exist](http://stackoverflow.com/questions/2331067/php-can-include-a-file-that-file-exists-says-doesnt-exist)
Answer: Try this:
<?php
class Test {

  public function __construct(){
    var_dump(file_exists( __DIR__ . "/conf.php"));
    $conf = include( __DIR__ . "/conf.php");
    echo $conf;
  }

}

Using the magic constant __DIR__ will return the absolute path to the script it is used, instead of the script that includes it, so any file name references will be to full path of the file, rather than just relative paths, which uses the primary running script's location.

PHP: include and file_exists scope


Como pegar um input imagem e exibi-lo no background-image?

Question: Tenho um campo de input que deve receber uma imagem, ao recebe-la devo adiciona-la como background-image de uma div. Para isto, utilizei o FileReader para obter a base64 da imagem. Funciona bem, porém o CSS não consegue carregar a imagem. Como são inputs do usuário não consigo pegar uma URL.
Já tentei usando:
document.getElementById('div').style.backgoundImage = "url('${imageData}')";

.div{
        position: absolute;
        background-image: url('${imageData}');
    }

Comment: Por que colocar no CSS se o JavaScript já alterou o background? Além do que o CSS é interpretado ao abrir a página, depois disso vc não consegue alterar o código-fonte dele.
Comment: O JavaScript não alterou o background, aqueles eram exemplos de tentativas falhas.
Answer: Há dois erros no código:

Falta a letra r em .backgoundImage. Seria .backgroundImage.
A interpolação da string com placeholder ${} só vai funcionar entre os sinais de crase:

    document.getElementById('div').style.backgroundImage = `url(${imageData})`;

Veja um exemplo:

let imageData = "https://www.cleverfiles.com/howto/wp-content/uploads/2016/08/mini.jpg";
document.getElementById('div').style.backgroundImage = `url(${imageData})`;

Comment: Agradeço por ter corrigido os erros, escrevi muito rapidamente e não me atentei a isso. De qualquer forma no codigo que estou tentando usar está tudo certo.
Comment: Como disse na própria pergunta a imagem é um input, não terei a url dela.
Answer: Sim você consegue fazer isso com a api FileReader.
Vale dar uma leitura também na seção de suporte dos browsers

function onSelectImage(event) {
 // se usuário não selecionou nenhum imagem não faça nada
 if (!event.target.files.length) return;

 // aqui pegaremos a primeira imagem da lista: FileList
 var selectedFile = event.target.files[0];

 // FileReader https://developer.mozilla.org/en-US/docs/Web/API/FileReader/readAsDataURL
 var reader = new FileReader();

 // assim que terminar de ler a imagem
 // jogá-la como plano de fundo da div#image-area
 reader.onload = function(event) {
  document.querySelector('#image-area').style.background = "url('" + event.target.result + "') no-repeat 0 0";
 };

 // Informa ao reader para ler a image como base64
 reader.readAsDataURL(selectedFile);
}
#image-area {
  width: 200px;
  height: 200px;
  margin-top: 10px;
}
<input type="file" id="file_input" onchange="onSelectImage(event)"/>


Are there any examples of using cloud-init offline (or without cloud hosting)?

Question: Note: This is NOT a Raspberry Pi specific question. It is a Linux System Administration question. Please do not try to have it removed from Server Fault.
I see many "Hello, World" level demos that are specific to [insert cloud hosting platform here], but none that are geared to being used on a single local machine. I was encouraged to see that Raspbian has cloud-init in its repository. That is exactly what I want to do. However, I cannot find any information on how to do it without a AWS, OpenStack, etc.
So, I guess what I really want to know is after you apt-get install cloud-init, where do you put your config/userdata?

Update: I found some info on the NoCloud datasource, but I cannot add a volume labeled cidata. So, that is out for me. (The volume that will hold my user-data is labeled boot.)
Comment: Then you'll need to change the volume label. Or you could even put such a volume on your SD card. But, why is this not a Raspberry Pi question?
Comment: It's not an RPi question because: 1. It's not exclusive to RPi. If I were doing this on my Lenovo laptop, it wouldn't be a Lenovo question. It'd still be a cloud-init question. 2. There are exactly zero people using cloud-init on an RPi as of today.
Comment: Those are both possible directions. But I'm looking for others. Both of those options are going to have unwanted side effects.
Answer: With the current version of cloud-init you can use smbios options to provide user-data and meta-data. There is also an example for the case  you have a local HTTP Server providing these files:
-smbios type=1,serial=ds=nocloud-net;s=http://10.10.0.1:8000/

Comment: Sadly, my distro has cloud-init 0.7.9 and I can't figure out how to build/install a more recent version. Also, having an external web server as a requirement is undesirable. One of the things I'd like to configure is wifi settings.
Comment: Making progress! https://stackoverflow.com/a/48845732/117471
Answer: I have VMs in premisses configured pretty much like this article explains.


How to build a block on form submission?

Question: I know how to build a block using these functions :
<code>hook_block_info()
hook_block_configure()
hook_block_save()
hook_block_view()
</code>
But on a form submission a few information is saved into DB and based on those information one block must be created.  how do I can build a block in that way?
Answer: I will encourage you to look at bean contributed module. Using this you can create block types whose instances can be created using bean_create function.
Example snippet:
<code>$values = array(
  'delta' => 'test_bean',
  'label' => t('Test Bean'),
  'title' => t('Test Bean'),
  'type' => some_block_type_that_you_will_create_through_UI,
  'view_mode' => 'default',
  'data' => array(
    'test_boolean' => FALSE,
    'test_string' => t('New String'),
    'test_array' => array(
      'test_array_1' => 'new_value',
    ),
  ),
);

$bean = bean_create($values);
</code>
The bean content (block) can then be placed around the site just like any other block.
Comment: what is the plugin_name? it reports this error: ` Using $this when not in object context`
Comment: `plugin_name` in the snippet above has the value for **block-type** for which a block is being created. The snippet was copied from `bean.test` file included in [bean module](http://ftp.drupal.org/files/projects/bean-7.x-1.7.zip) itself. Please go through its documentation, its really simple to create one.
Comment: I'll update the snippet above to remove the confusion.
Comment: I executed the code but no block generated! it also has some inconsistency: I use `_block_rehash($theme);` statement which returns all of the blocks. the `bean` command make a warning in my website!\
Comment: @Drupalist Can you update the question with the code that you ran?
Comment: it wasn't because of the `Bean` why the warning reported. I ran your code and no block created! besides that what is `some_block_type_that_you_will_create_through_UI` ? I searched for Bean type but found nothing.
Comment: The attached snippet is not ready to be used yet. Please look into bean.test file in bean module to understand how to use bean_create function. It has been used at lines 160, 165, 200, 207 and 231. Enough examples added in that file. And `some_block_type_that_you_will_create_through_UI` is really what is says. You will have to first download and enable bean module, then create a block-type using its admin UI(bean_admin_ui module), before creating blocks of that type.
Comment: Please see [this 7:35 minutes video on BEAN module on youtube](https://www.youtube.com/watch?v=Eu1YNy-BNG8) and things will be clearer.


PHP - Get the value by key in a multi-dimension array

Question: I have a multi-dimension array like:
$fields =
    Array (
        [1] => Array
            (
                [field_special_features5_value] => Special Function 5
            )

        [2] => Array
            (
                [field_special_features6_value] => Special Function 6
            )

        [3] => Array
            (
                [field_opticalzoom_value] => Optical Zoom
            )
    )

I want to get the value by key, I tried the code below but not work
$tmp = array_search('field_special_features5_value' , $fields);
echo $tmp;

How can I get the value Special Function 5 of the key field_special_features5_value?
Thanks
Answer: print $fields[1]['field_special_features5_value'];
or if you don't know at which index your array is, something like this:
function GetKey($key, $search)
{
    foreach ($search as $array)
    {
        if (array_key_exists($key, $array))
        {
            return $array[$key];
        }
    }

    return false;
}

$tmp = GetKey('field_special_features5_value' , $fields);
echo $tmp;

Answer: If you know where it is located in the $fields array, try :
$value = $fields[1]['field_special_features5_value'];

If not, try :
function getSubkey($key,$inArray)
{
    for ($fields as $field)
    {
         $keys = array_keys($field);
         if (isset($keys[$key])) return $keys[$key];
    }

    return NULL;
}

And use it like this :
<?php
     $value = getSubkey("field_special_features5_value",$fields);
?>

Answer: You need to search recursive:
function array_search_recursive(array $array, $key) {
    foreach ($array as $k => $v) {
        if (is_array($v)) {
            if($found = array_search_recursive($v, $key)){
                return $found;
            }
        } elseif ($k == $key) {
             return $v;
        } else {
             return false;
        }
    }
}

$result = array_search_recursive($fields, 'field_special_features5_value');

Answer: Your problem is that you have a top-level index first before you can search you array. So to access that value you need to do this:
$tmp = $fields[1]['field_special_features5_value'];

Answer: You can do it with recursive function like this
<?php

function multi_array_key_exists($needle, $haystack) {
       foreach ($haystack as $key=>$value) {
         if ($needle===$key) {
           return $key;
         }
         if (is_array($value)) {
           if(multi_array_key_exists($needle, $value)) {
             return multi_array_key_exists($needle, $value);
           }
         }
       }
   return false;
 }
?>

Comment: Well, this could work... but why use recursion when in fact the original post implies that the OP knows that the array is NOT actually representing a tree-like structure with an unknown number of 'branches', but a 2-dimensional one...?


what is the hook name in order to alter theme function of specific element

Question: imagine that we have a list if items in $items array that it's theme function is list_item in specific page.
code is
theme('item_list', array('items' => $items));

but in my module, I want to alter the parameter 'item_list' and I want to use my custom render function 'custom_item_list'
I know that I must register and define custom_item_list function but I don't know in what hook I am able to alter the theme_function parameter of specific element.
Answer: You can use hook_theme() function to use custom item list.
Hook theme:
function mymodule_theme($existing, $type, $theme, $path)
{
  if ($type == 'module')
  {
    return array(
      'custom_item_list' => array(
        'variables' => array('var_name'=>NULL),
        'template' => 'custom-item-list',
      ),
    );
  }
  return array(); //will raise fatal error if void
}

Call it as usual:
theme('custom_item_list', array('var_name' => mymodule_variable()));

Comment: I agree with your idea. but as I wrote in my question, I have defined my custom theme function but I don't know in which hook I can change the theme function of specific element. for example a block that is created by another module makes list items that is rendered by item_list, now I want to alter this list and render it by my custom theme function.
Answer: You need to implement hook_theme_registry_alter() which is the hook modules can use to alter a theme function implemented by other modules. hook_theme() is for theme functions your module defines itself.
Just set $theme_registry['item_list']['function'] to 'theme_custom_item_list', and theme_custom_item_list() will be called instead of theme_item_list().
Keep in mind that, if you need to alter the content of the $variables parameter passed to a theme function, and not its output, you can implement hook_preprocess_item_list(), since every theme function has its own preprocess functions.
function mymodule_preprocess_item_list(&$variables) {
  // Change the title for the list.
  $variables['title'] = 'My own list title';
}

Comment: but if I alter $theme_registry['item_list'] function , all element that are themed by item_list will be affected. I want to define a new custom theme and set the theme function of specific item, not alter the body of item_list theme function.
Comment: If the theme function is only used by your module, then just implement `hook_theme()` to create a new theme function your module will call with (for example) `theme('custom_item_list', $variables)`. if you want to use a new theme function that is used from every module on specific pages, that is a little more harder to implement.
Comment: i want to know, If I want to set a new theme_function wrapper for specific element, in which hook I am able to do that before the element rendering.


two-point boundary value problem with piecewise function coefficients

Question: Let $u(x)$ in $(0,1)$ satisfy:$$\frac{d}{dx}(a(x)\frac{du}{dx}) = 1$$ With $u(0) = u(1) = 0$. $a(x)$ is a piece-wise function:$$a(x) = \left\{\begin{aligned}
    1,& & x\in(0,1/2) \\
    2,& & x\in(1/2,1) \\
\end{aligned}\right.$$ It also requires that $u(x)$ and $a(x)\frac{du}{dx}$ be continuous at $x=1/2$.
And the question is to solve this equation using finite-difference methods (use Matlab)
I write the code the textbook way (when $a(x)$ is continuous), but the result is quite strange:

I use the method from this article,P75

Following is function code, please tell me how to fix it if you find something wrong. Thank you!
<code>function u = Ch3_varcof_1d(a,f,N)
%--------------------------------------------------------------------------
% u = Ch3_FD1d(a,b,c,f,N,method)
%
% compute the numerical solution of the 1D example (h=1/N)
%   d/dx(a(x)du/dx) = f  in (0,1)
%   u(0) = u(1) = 0
%
%               2021-05-18
%--------------------------------------------------------------------------

h = 1/N; % Step-length
x_list = 0:1/N:1; % Discrete-Points in interval
l1 = ones(N-2,1); % Downside diagonal
l2 = ones(N-1,1); % Middle diagonal
l3 = ones(N-2,1); % Upside diagonal

for i=2:N-1
    l1(i-1) = a((x_list(i)+x_list(i+1))/2);
end

for i=1:N-1
    l2(i) = a((x_list(i+1)+x_list(i+2))/2) + a((x_list(i)+x_list(i+1))/2);
end

for i=2:N-1
    l3(i-1) = a((x_list(i+1)+x_list(i+2))/2);
end

A = (diag(l1,-1)-diag(l2)+diag(l3,1))./(h^2);

if isa(f,'function_handle') % Get the function F vector
    x = (0:h:1)';
    F = f(x(2:end-1));
else
    F = f*ones(N-1,1);
end

u = A \ F;
u = [0; u; 0];
</code>
Comment: @Ricardo Cavalcanti Thanks for the reminder. But I'm not sure I understand your word, that is, should I leave a comment here to ask for checking codes rather than in the question part?
Comment: @Ricardo Cavalcanti Thank you a lot! I have made an edit in my question. I will be more careful when putting on my codes.
Answer: You don't need any specialised method to solve this problem... To make sure that $x=\frac 12$ is one of the nodes, lets use $n = 2^k$ subintervals ($2^k+1$ points). This way the central node has index $m = \frac n2 +1$. Since $a(x)$ is piecewise constant, at every node except from $m$ the equation to be satisfied is $
u''(x) = \frac{1}{a(x)}$, yielding the linear equations
$$
u_1=0
$$
$$
u_{i+1}-2 u_i +u_{i-1} = \frac{h^2}{a(x_i)}, \quad i= 2,\cdots, n-1, \quad i \ne m
$$
$$
u_n=0
$$
Regarding the $m^{th}$ equation, the continuity of $a(x) u'(x)$ yields
$$
\frac{u_m-u_{m-1}}{h} = 2 \times \frac{u_{m+1}-u_m}{h}.
$$
so, you get the standard discretisation of the second derivative, with one modified equation. The system matrix for $n=2^3$ would be
$$
\left(
\begin{array}{ccccccccc}
 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
 1 & -2 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
 0 & 1 & -2 & 1 & 0 & 0 & 0 & 0 & 0 \\
 0 & 0 & 1 & -2 & 1 & 0 & 0 & 0 & 0 \\
 0 & 0 & 0 & -1 & 3 & -2 & 0 & 0 & 0 \\
 0 & 0 & 0 & 0 & 1 & -2 & 1 & 0 & 0 \\
 0 & 0 & 0 & 0 & 0 & 1 & -2 & 1 & 0 \\
 0 & 0 & 0 & 0 & 0 & 0 & 1 & -2 & 1 \\
 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\
\end{array}
\right)
$$
And graphically, the solution is

If you don't enforce the continuity of $a(x) u'(x)$, you get nicer solutions, which don't present the small "hiccup" at $x=\frac 12$.
Comment: Nice Answer! Thank you Carre


Sizes of tensors must match except in dimension 2. Got 16 and 32 (The offending index is 0)

Question: I'm getting following error :

Sizes of tensors must match except in dimension 2. Got 16 and 32 (The offending index is 0)

in following code:
x1=self.avg_pool(l1)
print('x1:', x1.shape)
x2=self.avg_pool(l2)
print('x2:', x2.shape)
x3=self.avg_pool(l3)
print('x3:', x3.shape)
x4=self.avg_pool(l4)
print('x4:', x4.shape)
x = self.aspp(x)
print('x:', x.shape)
x=torch.cat((x4,x3,x2,x1,x),dim=1)
cout1=x

I'm getting shapes of x1,x2,x3,x4,x as
x1: torch.Size([5, 256, 32, 32])
x2: torch.Size([5, 512, 32, 32])
x3: torch.Size([5, 1024, 32, 32])
x4: torch.Size([5, 2048, 32, 32])
x: torch.Size([5, 256, 16, 16])

Answer: You are concatenating on dim=1, well that means you need to join the tensors one after the othe ralong dim=1. The value that you get after concatenation along dim=1 is value=256+512+1024+2048+256, provided shapes of the tensors match in other dimensions too. The size of tensor x should be x=(5,256,32,32).
Answer: Inputs (values of l1, l2, l3, l4) are not provided, so I just guessed it and tried to mimic your code. Below snippet works fine.
import torch
import torch.nn as nn
import torch.nn.functional as F

class Pyramid_Pooling(nn.Module):
    def __init__(self, levels, inChans, outChans):

        super(Pyramid_Pooling, self).__init__()
        self.inChans = inChans
        self.outChans = outChans
        assert len(levels) == 4

        self.pool_4 = nn.AvgPool2d((levels[3], levels[3]))
        self.pool_3 = nn.AvgPool2d((levels[2], levels[2]))
        self.pool_2 = nn.AvgPool2d((levels[1], levels[1]))
        self.pool_1 = nn.AvgPool2d((levels[0], levels[0]))

        # lower the number of channels to desired size
        self.bottleneck_pyramid = nn.Conv2d(
            self.inChans, self.outChans, kernel_size=1
        )

    def forward(self, en1, en2, en3, en4):
        pooled_out_1 = self.pool_1(en1)
        print(pooled_out_1.shape)
        pooled_out_2 = self.pool_2(en2)
        print(pooled_out_2.shape)
        pooled_out_3 = self.pool_3(en3)
        print(pooled_out_3.shape)
        pooled_out_4 = self.pool_4(en4)
        print(pooled_out_4.shape)

        cat = torch.cat((pooled_out_1, pooled_out_2, pooled_out_3, pooled_out_4), 1)
        out = self.bottleneck_pyramid(cat)

        return out

I tried to guess your inputs and issue might be somehwere in input dimensions. You want to output 32 x 32 then input should be like below. Also added a 1x1 conv to lower the channels to desired output.
x_train_0 = torch.randn((3, 256, 32, 32), device = torch.device('cuda'))
x_train_1 = torch.randn((3, 512, 64, 64), device = torch.device('cuda'))
x_train_2 = torch.randn((3, 1024, 128, 128), device = torch.device('cuda'))
x_train_3 = torch.randn((3, 2048, 256, 256), device = torch.device('cuda'))

inChans = x_train_0.shape[1] + x_train_1.shape[1] + x_train_2.shape[1] + x_train_3.shape[1]
outChans = 512

# kernel_size = [1,2,4,8]
pyramid_pooling = Pyramid_Pooling([1, 2, 4, 8], inChans, outChans)
pyramid_pooling.to(torch.device('cuda'))
out = pyramid_pooling(x_train_0, x_train_1, x_train_2, x_train_3)
print(f"Output shape: {out.shape}")

Output will look like this:
torch.Size([3, 256, 32, 32])
torch.Size([3, 512, 32, 32])
torch.Size([3, 1024, 32, 32])
torch.Size([3, 2048, 32, 32])
Output shape: torch.Size([3, 512, 32, 32])


Significance: p-value or t-value?

Question: I'm writing an economics thesis and I've found many econometric papers in which I found regression tables without the p-value. They only show the coefficient of the independent variable and its t-value, like in the example image.

My econometrics knowledge is very elementary, but I thought that the most important thing to evaluate the significance of a variable was the p-value. Searching on the web, I've found that sometimes $t-value>2$ is considered as a good value for significance. Is that right? Can I say that when $t-value>2$ the coefficient is statistically significant?
Edit: after Rob's suggestion, here's a link to the paper
Comment: They give the number of observations, and the number of explanatory variables is (hopefully!) made clear, so you could work out the degrees of freedom, and hence the p-value. The critical value for 13 d.f. (the smallest one there if the listed variables are the only ones) is 2.16, so all but one of the coefficients that exceed 2 is actually significant.
Comment: Why not give both? At the 5% level, a $t$ is strictly only significant for $|t|>2$ if the degrees of freedom is $>60$.
Comment: That's why I don't understand these tables. Why do they give only the $t$ value and say that it's significant, even if they don't show the p-value
Answer: I think that, in general, there is far too much concentration on p-values and significance and far too little on effect sizes, so I applaud the idea of showing tables without p-values.
There are a number of problems with over-reliance on p values, but one of them is alluded to (almost in reverse) in @Rob 's answer, above: People who don't understand the model may rely on the p-value. This may give them a false sense of understanding.
The particular case that you show is a sensitivity analysis. Here, the use of p-values seems even less appealing than typically: The goal of a sensitivity analysis ought to be to see how much the parameter estimates change, not whether they change from "significant" to "non-significant"; the sensitivity analysis shows that democracies and non-democracies have quite different values on some parameters. That's important. The p-value of those parameters is much much less important.
Answer: With more complex models, it may be difficult or impossible to calculate p-values. In these cases, t-values are often given instead (if you can provide a link to the paper, we can have a look if p-values could have been calculated). An example are mixed models, where the absence of p-values is a topic of constant debate (see http://glmm.wikidot.com/faq for more information). However, if the model is simple enough to calculate reliable p-values, I personally would want to provide them to the readers - especially in applied areas where readers may have trouble judging the models without them.
I do agree with @peter-flom that ideally, the focus should be on effect sizes. However, in my experience the only way to convince an applied scientist that a result should not be considered significant is by simply saying "my calculations yielded that it is not statistically significant". When I suggested that the effect size was not e.g. biologically relevant, they usually were not very responsive ("but it's significant!"). So I would use p-values to convince an applied reader not to pursue a line of thought further without more evidence, rather than to use p-values to try to prove something meaningful is happening. And of course, the biggest effect size still is not enough if it might very well be pure chance.

Significance: p-value or t-value?


median proof given G(t)=F(t-delta)

Question:
How can I do part a? I get stuck after using the formula for difference of 2 independent variable
Comment: You need to show a proper attempt.
Answer: One has to prove $\Pr(Y-X < \Delta) =0.5$ (I remove the indexes $i$ and $j$ which are useless).
Start with
$$
\Pr(Y-X < \Delta) = \Pr(Y < \Delta +X)
$$
By independence between $X$ and $Y$,
$$
\Pr(Y < \Delta + X) = \int \Pr(Y < \Delta + x) \mathrm{d}F(x).
$$
One has
$$
\Pr(Y < \Delta + x) = G(\Delta+x) = F(x),
$$
thus
$$
\Pr(Y < \Delta + X) = \int F(x)  \mathrm{d}F(x).
$$
Now, observe that
$$
\int F(x)  \mathrm{d}F(x) = \mathbb{E}[F(X)].
$$
When $F$ is continuous, it is well known that $F(X)$ follows a uniform distribution on $(0,1)$. The result follows.
Comment: @GillianCheung More easy: you can write $Y=X'+\Delta$ where $X'$ is independent of $X$ and has the same law; then $\Pr(Y-X < \Delta) = \Pr(X'-X < 0)$, and this equals $0.5$ by symmetry ($X-X'$ has the same law as $X'-X$).

median proof given G(t)=F(t-delta)


Is the empty set closed under addition?

Question: Having trouble understanding this. The empty set has no elements, so I think that maybe it can't be false since there's nothing to make it false?
Comment: "Closed under addition", when translated to predicate logic, starts with a $\forall$ quantifier. That's an immediate sign that it is _true_ for the empty set.
Comment: That is correct.
Comment: See [Empty semigroup](https://en.m.wikipedia.org/wiki/Empty_semigroup).
Answer: This is vacuously true. For any pair of elements $x,y\in \varnothing$ it is the case that $x+y\in \varnothing$ (assuming you have defined "+"). This holds because it is a universal statement quantified over the empty set. It is true, since every pair of elements in $\varnothing$ has this property (of which there are none).
Answer: You're correct.
If it were not closed, there would exist a pair of elements $a, b \in \varnothing$ with $a+b \notin \varnothing$. Since the empty set has no elements, this cannot be the case.

Is the empty set closed under addition?


Custom Path Alias with Initial Letter as Token

Question: For various reasons, I need to change my custom path alias for a taxonomy type as follows...
FROM:
/composers/aardvark
TO:
composers/a/aardvark
So I'm looking at either creating a custom token, or a computed field to figure out the first letter of the term and feed into the pathauto module...
I'm having trouble figuring out either... Could anyone suggest the correct code to use for either of these? Most of the examples I am finding are node specific, and not taxonomy term specific...
The reason I'm looking to do this, is to provide a contextual filter to a views block... But if anyone could suggest a good way to pass 'a' from 'aardvark' into the block - so that both of these paths passes the same contextual filter, that would be equally good!
/composers/a
/composers/aardvark
Thanks in advance for your help!
Answer: There are many ways to achieve this, as you said.
The less-coding way I see is to use Custom Token and create for each node a token using the following code :
<code>return substr($node->title, 0);
</code>
Then you use this token in your URL alias pattern like [yournewtoken]/[node:title].
If you have a multi-lingual site, the first letter might not be the same in all languages, you may update this code to fit this.
Comment: Thanks Gregory.  I tried your suggestion, but as my entity is a taxonomy term and not a node I couldn't quite get it working (e.g. with $term->name and other permutations)...

So then I returned to the Computed Field route and got it working as follows...

    `$name =  field_get_items('taxonomy_term', $entity, 'name');`
    `$n = substr($name,0,1);`
    `$n = strtolower($n);`
    `$entity_field[0]['value'] = $n;`
Comment: Good one ! Glad it worked.
Answer: I found this page searching for a solution to add the first character (letter) of the node title in the breadcrumb path. The full and correct code should be
<code>$node = node_load(arg(1));
return substr($node->title, 0);
</code>
See the screen shot


I want to learn to play an electric guitar, how do I start?

Question: I'm 15, and recently I've watched a few electric guitar performances on YouTube. It got me really interested in electric guitars.
Now, I've never played a guitar before, neither electric nor acoustic, i kind of did, I'll explain in a second.
My grandfather owns a few acoustic guitars and he's pretty good at it. When i was 11 he taught me how to play Smoke on the Water, a pretty simple melody. I didn't practice that song much, I played it a few times for fun, what I'm trying to say is that I kind of know how a guitar feels on my lap and I know how it feels to play the strings. Apart from that, I never played them.
I wasn't interested in guitars before, but recently I have. I don't like acoustic guitars, I like electric ones. Anyway, I have a question:
I''m a beginner. I'd like to learn how to play an electric guitar. Should my first guitar be an electric or acoustic? Which one is easier to play and easier to learn to play?
And a bit sillier one, could I cut my fingers on a guitar's strings?
I'll be glad if someone answered my questions, I'm eager to learn how to play an electric guitar.
Comment: Buy a Yamaha Pacifica and a small amplifier with built in effects. Learn all of the basic chord shapes.Then watch YouTube... And just play.. don't think of it as practice.. just play the guitar.. as often as you can.. talk to others that play.. in a year.. you'll be as good as 70% of guitarists. Your fingers will have hard skin on the ends.. and you will have levelled up your life.
Comment: Hi Nikad, welcome. I have edited the formatting a bit to make it easier to read. And please try to just ask one question at a time next time. There are loads of posts here with information that will help you so have a good read of the questions in the guitar and electric guitar tag.
Comment: Also.. I didn't know that guitar strings snap every now and then. So don't worry about that just buy new strings. Also.. your guitar doesn't just need to be in tune with itself.. it needs to be in tune with everybody else.. so buy a guitar tuner.. or download a tuner app for your phone.
Answer:
I don't like acoustic guitars, i like electric ones.

You should start with an electric guitar.
There are arguments both ways to whether to start on electric or acoustic, but these differences are minor compared to enthusiasm. You won't get anywhere with a guitar that you don't like and don't play. You'll go far with one that gets your heart racing so you play it frequently.
The strings on electric guitars are usually lighter and closer than acoustic. This is argued in favor of electrics as easier to get started on and in favor of acoustics as forcing better habits.
If your grandfather is available I'd recommend taking him shopping for an inexpensive but well set up electric guitar and amp. There are dozens of sites with reviews of good starter guitars. The important thing when starting out is that it's set up well. A cheap but well set up guitar will play better and easier than an expensive but poorly set up one.
Once you have the guitar take some lessons (or have your grandfather give you some if he's available and able to - playing and teaching are very different skills though) and find some like minded friends to play with. Locally a lot of the teen-oriented lessons also help the students form bands. If your friends aren't already interested in playing there may be something similar near you.
Your fingers will be sore when you get started, but they shouldn't bleed. You'll need to build up strength and stamina. Take things slowly at first and stop when your fingers or hands hurt. A teacher will be able to help identify if you're hurting because of bad technique or if it's just from getting tired. Playing several 10-15 minute sessions is much safer and more valuable than one marathon session.
Answer:
Should my first guitar be an electric or acoustic?

If you want to play electric, as you said, then your first guitar should definitely be an electric.

Which one is easier to play and easier to learn to play?

Stylistically, as in how people choose to play them, they are just a little different and neither is necessarily easier. But from a technical standpoint, assuming you chose to play the same thing on both, an electric is usually a little easier to play for most people because of the string gauge, action, and neck thickness.

And a bit sillier one, could i cut my fingers on a guitar's strings?

Just from playing them? No, not really. You might get blisters at first if you practice hard enough but eventually you'll build up calluses. And you can and probably will prick yourself on the strings ends at some point when changing strings.

how do i start?

Get a guitar
Get a teacher
Have them help you learn songs that you like because you'll make more progress if you're enjoying your practice time.
Try to have a goal and keep that in mind at all times. For example if your goal is to play cover songs in a band with friends, then you should spend your practice time learning songs and playing with a band. But if your goal was something different you might spend your practice time on something else. Your practice time should always be moving you toward your goal.
Comment: I'm not sure that sentence required an exclamation point, Tim. But I'm glad that you're fired up about calluses. It's pretty common for beginners to get calluses. But then again I guess it's also common for beginners to not play properly, right?
Comment: Might build up callouses if your guitar isn't set up properly, or you're not playing it properly!
Comment: Fired up indeed. I had them when I first started playing, and that was the fault of a very cheap guitar. I'd probably call them scars at the time! After getting a half decent one, and learning how to set up, haven't had them in the last 55+ yrs of regular playing  - guitar and bass. Certainly don't believe in them, and good players find them a hindrance, as they're not playing with proper skin!
Answer: GOOD LORD! ELECTRIC GUITAR IS NOT DEAD! THANK YOU!
Another good news is: It's totally OK to start with electric guitar!
I see you already have great answers on this post, so I'll just try to bring in what I found missing from my experience, as well as some enthusiasm!

And a bit sillier one, could I cut my fingers on a guitar's strings?

Not a silly question at all: actually, you're not going to cut your fingers with the strings on regular playing, but you might cut your fingers with the frets on a poorly built instrument.
Frets should be polished (and also crouned and leveled) so the edges won't cut your fingers.
However on very cheap guitars, you'll find the frets are roughly polished and THAT might cut you seriously.
So my recommendation goes to buying a guitar: you should have a seasoned guitarist to help you on your first buy. If you don't know any body, you'll find lots of reviews on YouTube to help you. Take your time! And start with grand-pa's acoustic if you can't get your hands on an electric guitar quickly.
Basically, you should range between 100$ and 300$ for a new guitar in a retailer store. If you can get a Fender or an Epiphone for this price range, you can't go wrong. You may also want to check-out Harley Benton. It's a German low cost brand. They only sell on the internet, so check-out the reviews before you move.
Comment: My two cents worth on the buying process...go out and play every guitar on the rack in every store you can find, even if you know it's out of your price range.  You'll soon find things you like and things you don't like and will have a better grasp of what makes a good guitar.  And if the store won't let you play them...walk out and find another.
Answer: Should I start with acoustic or electric?
Well you like electric so, you'll need one of them ;-)
But .. acoustic guitars have one massive advantage: you can just pick one up an play it. No plugs, leads, amp to sort out. You can do that with an electric guitar too of course, but it's much more quiet.
I have found that playing elewctric is great for playing the songs I want to play, and have them sound how I want them to. however an acoustic s much better for clarity and hearing the detailof what you're playing.
so I'd advocate having both, if you can manage it - but spend the money & buy a nicer electric for yourself because that's what you really want.
Re your fingers:
Yes, it'll hurt at first. electric guitars use steel strings and these cut into your finger pads a bit at first for a couplke of weeks until you get used to it.
if you have a steel-string acoustic, it's even worse: the strings are usually quite thick and they really dig into your finger pads- again only until you get used to it.
a nylon-string acoustic might be a good starting point just to get the fingering right because the nylon strings are thicker generally, and do'nt hurt as much. ie you can get more practice in without any pain.
nylon strings aren't very good for bending so if you want to simulate that, you can tune it quite low so the strings are looser, and then they become bendable like an electric guitar (just in a lower key)
don't worry aboiut the finger pad wear though, it really only takes a little while before you get used to it.
Good luck ! Hope you enjoy :-)
Answer: So listen, I play Clarinet and Flute since a while and I don't know much about guitars, I never liked them :D. But I can give you a few suggestions:
1) First thing is emotion and enthusiasm. Make sure you are ready and afford the money for everything you must buy in order to keep a quite "professional" studying-time with the guitar.
2) Learn the basics. Learn the low key and the violin key as starting; keep studying them and learning them daily, don't stop. Practice and improve yourself as much as you can daily.
3) Learn reading music (solfeggio). Even if it's kind of "boring", it's one of the most important chapters of the music; it allows you to properly understand the music, so you won't have particular problems into playing a piece for the first time. Practice daily & start playing a bit after doing some solfeggio.
4) Buy everything you need (guitar, lectern SHOT, etc.), so you won't have this problem into the future.
5) Keep practicing daily.
6) Possibly study and learn with a teacher; in case you do some mistakes or errors while playing the guitar or doing solfeggio, he/she'll be surely helping you out into "fixing" that problem, and so it will be easier for you "check-'n-fix" problems with playing, etc...
7) Watch some videos on YouTube/Internet (generally), but DON'T take too much "profit" out of them; nobody's perfect, so who says that you won't do some mistakes into reading/learning from internet, or than even those who made the videos did some errors?
8) Keep studying.
9) Never leave the guitar just because you have the day too full or other null excuses. Just keep studying and remember the empathy and emotion you have got towards the guitar.
10) Love the music.
Music is the best tool God gave us to exprime our emotions out of ourselves, so don't waste it, it's a very powerful tool, ;) :).
Once again, I'm not a guitar expert, as I play Clarinet and Flute, but I hope this helped you out, even if just "a little bit". ;) :).


Easiest & fastest way to export a Python program to run on another computer?

Question: My friend does not have Python or the libraries needed installed on his machine to run a program I have written.  Is there a simple way to export my project (currently sitting in pydev on my x64 Windows 7 machine) so that he could run it on his machine? (He has OSX, but he could find Unix or Windows if needed.)  My program uses several modules, and depends on networkx, wx, and matplotlib.
Feel free to move this to programming if overflow isn't appropriate.
Thanks!
Comment: Just bundle all 3rd party dependencies locally to the project.
Comment: @Jakob care to explain what the best way to do so is?
Comment: just keep your dependancies local to your script. E.g. in the same path.
Answer: You could use py2exe (Windows), cx_Freeze (should work on any platform that Python itself works on) or py2app (Mac OS X).
Answer: OS X actually already has python installed, albeit a somewhat old version.  On OS X, a great solution to getting newer python Mac Ports package manager.
On Windows, python has an installer, and many of the most popular modules also have installers that respect the installed python.  Alternatively, you can use Cygwin to install X-Windows versions of same.
On unix like platforms, well, you're already good to go.  Python is preinstalled on just about every distro, and the package manager will have all of the popular modules ready to install, too.


Eu tenho esse json em React js! Como faço para pegar esse src de cada um JSON e passar todos eles como varias imagens no React? Tô aprendendo!

Question: Este é o JSON obtido através do Puppeteer:
[
  {
    "src": "https://instagram.ffor9-1.fna.fbcdn.net/v/t51.2885-15/sh0.08/e35/c0.280.720.720a/s640x640/243514741_829398887750929_7555907807919756011_n.jpg?_nc_ht=instagram.ffor9-1.fna.fbcdn.net&_nc_cat=101&_nc_ohc=ZjOHxgkiE0YAX8jFhig&edm=ABfd0MgBAAAA&ccb=7-4&oh=a128ed63ed08b7d792b5e3d6492d0852&oe=61572BA0&_nc_sid=7bff83"
  },
  {
    "src": "https://instagram.ffor9-1.fna.fbcdn.net/v/t51.2885-15/sh0.08/e35/c0.280.720.720a/s640x640/243161222_3063170977298797_2390805453276083538_n.jpg?_nc_ht=instagram.ffor9-1.fna.fbcdn.net&_nc_cat=101&_nc_ohc=pOUY9OkehocAX__0Wjv&edm=ABfd0MgBAAAA&ccb=7-4&oh=6a069b55664a0f3b60e7efecafddde83&oe=6156FD14&_nc_sid=7bff83"
  },]

este seria o código React JS:
export default function App(){

render(

{json.map(src_do_json=>)}

)}

Comment: O stack limitou meu codigo do React não sei o por quê
Answer: Você pode fazer um map exatamente como você estava fazendo e retornar uma imagem a cada iteração:
json.map(src_do_json => (

))


What are modes in series?

Question: I've been reading the explanation to the question about the difference between the term tonal center and guide tone and came across the statement that modes can be in parallel or series. Please explain this statement that modes can be in parallel or series.
Comment: Can you quote and/or cite the relevant passage? Where did you come across the statement?
Comment: Possible duplicate of [What is a parallel mode?](https://music.stackexchange.com/questions/20196/what-is-a-parallel-mode)
Answer: Parallel - as in Parallel Minor - C-C-C-C-C-C-C -  lined up with same root, different mode:

C Ionian - Major
C Dorian
C Phrygian
C Lydian
C Mixolydian
C Aeolian - Parallel Minor
C Locrian

Series - in serial order ->
C->
D->
E->
F->
G->
A->
B
over the scale:

C Ionian
D Dorian
E Phrygian
F Lydian
G Mixolydian
A Aeolian
B Locrian
Comment: Worth mentioning - *parallel minor in C* also incorporates those notes found in harmonic and melodic minors. Aeolian isn't exclusively the parallel minor. The answer *is* great for the question, though! +1
Comment: Succinct, well answered.
Comment: @alephzero - the question appears to be about terminology. That's what I answered. You are certainly free to elaborate on functionality if you deem it necessary.
Comment: So "series" and "parallel" is just terminology without any functional purpose?

What are modes in series?


Packet Buffer size in D-link DES-3552

Question: What is the Packet Buffer size in D-link DES-3552?
I haven't found this information on any site (including official).
Comment: For whoever it is voting to close this, this is a layer 2 managed switch, not some SOHO DLink crap.
Answer: It has a 1 MB packet buffer, according to the product specifications on the product page.
<code>10/100BASE-TX Ports
    48

10/100/1000BASE-T Stacking Ports
    2

Combo SFP Slots
    2

RS-232 Console Port
    Yes

Performance

Switch Capacity
    17.6Gbps

64-Byte Packet Forwarding Rate
    13.1 Mpps

MAC Address Table Size
    16K

Packet Buffer
    1MB
</code>
[...]

Packet Buffer size in D-link DES-3552


guess 3 of 5 repeating non-ordered numbers from 1 to 6

Question: I'm trying to calculate the probability of guessing 3 numbers on a set of 5.
The range is $[1, 6]$, the order is not relevant and numbers can repeat.
I did a small script to calculate it by brute force, and I got something unexpected.
Choosing 3 identical numbers (e.g. 1, 1, 1) has a lower probability of drawing than 2 identical numbers, and this is also lower than 3 different numbers.
I can't figure out why.
Comment: Doesn't that set have $6$ elements?
Comment: they are 5 dice that roll values from 1 to 6 each
Comment: Oh ok, I misinterpreted.
Comment: Guessing 1,1,1 can only be correct if the numbers are 1,1,1.  But guessing 1,2,3 can be correct if the numbers are 1,2,3, or 1,3,2 or 2,1,3 or 2,3,1 or 3,1,2 or 3,2,1.... If in creating the original set so that order doesn't matter so that you only have one of those six possibilities for 1,2,3 then that probabilities will not be less.
Comment: @fleablood did you mean "so that the order does matter"? In my case, the order already doesn't matter.
Comment: Well, I misunderstood the question and thought the guesser was guessing three numbers to match three numbers, not 5.  The idea is still the same that $1,2,3$ will match more sets than $1,1,1$.
Answer: OK, let's calculate. If you choose $3$ identical numbers (say $1,1,1$), then you are right when the 5 numbers are:
A. those three $1$'s plus two non-$1$'s: ${5 \choose 3}=10$ 'spots' for your three $1$'s and $5 \cdot 5 = 25$ possibilities for the other two, for a total of $250$
B. you get four $1$'s and one non-$1$: ${5 \choose 4} \cdot 5=25$ possibilities
C. All five are a $1$: $1$ possibility
Now compare this with $1,1,2$. You get this when:
A. you get $1,1,2$, and two numbers that are neither $1$ nor $2$: $5 \cdot {4 \choose 2} = 30$ 'spots for your three numbers times $4 \cdot 4=16$ other numbers, make $480$ possbilities
... OK, we already have more possibilities for this to happen!
OK, and we also see the basic reason why: with $1,1,2$ you get this extra factor of $3$, as there are three ways for your $3$ 'hits' to occur: you first get a $1$, then another $1$, and then a $2$, or first a $1$, then a $2$, and then a $1$, or first a $2$, and then a $1$, and then the last $1$.  With three $1$'s, you have no such possible permutations.
And this also explains why picking three different numbers is even better: Now you get $6$ permutations, i.e. $6$ different orders in which your 'hits' can occur.
Comment: yes, the last part made it more clear to me :)
Comment: @mcmorry glad I could help! :)
Answer: Guessing $1,1,1$ will match the sets that have $3$ or more $1$.  There is $1$ set of $5$ ones, there are $5*5=25$ sets that have $4$ ones ($5$ positions for the non-one to be in and $5$ options for the value of the non-one), there are ${5\choose 2}*5 = 50$ ways to have $3$ ones and two of another value.  And ${5\choose 2}5*4 = 200$ ways to have $3$ ones and two other values.  This is $255$ ways.
Guessing $2,2,1$ will match sets that have $4$ twos and $1$ one: $5$ sets.  $3$ twos and $2$ ones: ${5 \choose 2} = 10$ sets.  $3$ twos, $1$ one and something else: ${5\choose 3}{2\choose 1}*4 = 80$ sets.  $2$ twos, and $3$ ones: ${5\choose 2} = 10$ sets.  $2$ twos, $2$ ones, and something else: ${5\choose 2}{3\choose 2}*4 = 120$ sets.  $2$ twos, $1$ one and $2$ of something else: ${5\choose 2}*{3\choose 1}*4 = 120$ sets.  $2$ twos, $1$ one and two other things: ${5\choose 2}{3\choose 1}4*3 = 360$.  Or $360+120+120+10+80+10+5 = 705$ sets.  Many more options!
Guessing $1,2,3$ can mean.... well, three of one of the numbers and one of the other two: $3*{5\choose 3}*2! = 60$( three choices for the triple, choices for where to put the triples, ways to place the remaining two).  Or two of two of them and one of then third: $3{5\choose 2}{3\choose 2}= 90$ (Three choices for the single, where to place the first of te doubles, where to place the other doubles.)  Two of one of them, one of the other two, and forth number.  $3{5\choose 2}{3\choose 1}{2\choose 1}*3 = 540$. (3 chooses for the double, where to put them, where to put the singles, and what the fourth number is.  Or exaclty one of each.  ${5\choose 3}*3!*(3)^2 = 540$ (places to put the three, order to place the three within their posistions, 9 posible value of the remaining two numbers.) For $60 + 90 +540+540 = 1230$ possible ways.
Comment: Thanks, you actually answered also to my first point (calculate the probabilities)


Python: How to read all text file contents in multiple folders and save them into one excel file

Question: I am currently working on this task and got stuck. Please give me your advices. The code I am writing below, it returns only the .txt file name from multiple folders, while I would like to print out the content of each .txt file and save in different rows in excel. The contents of each .txt file are numbers.
import os, glob,xlwt
#create workbook and worksheet
wbk = xlwt.Workbook()
sheet = wbk.add_sheet('data')
path= 'E:\Cag\Data'
row = 0
for dir,subdir,files in os.walk(path):
    for file in files:
        if glob.fnmatch.fnmatch(file, '*.txt'):
            L = file.strip()
            sheet.write(row,5,L)
            row += 1
wbk.save('read_all_txt_in_folders.xls')
print 'success'

Comment: you should import directly fnmatch, instead of using it from glob.
Comment: What do you think `L = file.strip()` will do?
Comment: Please don't use built-in function names (`file`) as a variable name.  It's confusing to people reading your program.  Also, upper-case variable names (`L`) are confusing.  Finally, single-letter variables names are confusing.  Please use meaningful, lower-case names.
Comment: thanks for your advice. I used the L= file.strip() to list the .txt file contents into columns in excel, with out that I think it is not going to work. If you have other way of writing it. Please help me
Comment: "to list the .txt file contents".  Do you mean "read the contents"?  If so, you should check the Python documentation for how file objects work.
Answer: Well, you see the file names because you write them. L contains the file name. (And the L = file.strip() seems unnecessary to me.)
If you want to write the file contents, you should do instead L = open(os.path.join(dir, file), "r").read().
Comment: @glglgl: Thank you so much for your explanation. It works now, but the contents of those txt files(let say 100 rows of numbers/ .txt file) are saved now in excel format in rows. But I want them saved in columns. I used .strip() in my initial code and it stripped the contents of text files from top to bottom.
Comment: yeah, this is exactly what I want. I tried it and it returned this error: Traceback (most recent call last):
  File "E:\read_all_txt_in_folders.py", line 15, in
    L = open(file, 'r').read()
IOError: [Errno 2] No such file or directory: 'ttt.txt'. The ttt.txt is in the folder, why this error is printed. Thanks
Comment: IIRC you need to contatenate `dir` (or `subdir`?) and `file` to get full path.
Comment: `walk()` yields its stuff (`subdir` and `files`) relative to the subdirectory it is currently in (`dir`). So you have to `os.path.join(dir, file)` to get the correct path. Sorry, I forgot that in my answer (-> just edited). (Besides, it is bad style to use a variable name `dir` as it hides the built-in with the same name.)
Comment: -1: This leaves files open.  For a small set of files that's not a problem.  For a large collection of files, that will be a problem.
Comment: Normally, the files are gc'ed away. But you are right - this is not guaranteed, so better close them explicitly (with `close()` or with `with`).
Answer: If you want to write the file contents, you should do instead
with open(os.path.join(dir, file), "r") as source:
    L= source.read()

This assures that the file is actually closed when you're done reading them.  That releases system resources.


How to handle data imbalance in Principal Component Analysis?

Question: PCA reduces data set dimensions while trying to keep most variations in data set.
PCA can be used as a dimension reducing technique in discrimination, however it tries to keep the most discrimination power rather than keep most variations as more variation does not necessarily mean the separation of different group of data is maximum. So Chang (1983) in his paper "On using principal components before separating a mixture of two multivariate normal distributions" suggest using Mahalanobis distance in some way to measure it. His paper can be found here: http://www.jstor.org/discover/10.2307/2347949?uid=3738032&uid=2129&uid=2&uid=70&uid=4&sid=21100901419681
Does the method work well when the data size are very different (consider two groups only), Say sample sizes have ratio 1:99? If it does not work well, how to handle the problem?
I have read literature about rebalancing data set for PCA, but there are no theoretical and mathematical explanation for how rebalance data can help increase the performance of PCA. Is there any literature for the mathematical explanation part?
Comment: Let me ask you to clarify and detail your question. First, what is "imbalance"? Groups of very different _n_ or else? So, do you have your sample split into groups or not? Second, PCA (?)`tries to keep the most discrimination power rather than keep most variations` seems to contradict with PCA `trying to keep most variations in data set`. Please, elucidate.
Comment: Thanks. I have edited it. The question should be more clearly now.
Comment: Is the essence of your question this: There's a multivariate data presumably heterogeneous due to presence of clusters (groups) in it; the clusters are not known, so discriminant analysis is not applicable. Is it then possible to use PCA in place of it, to differentiate the groups? If yes, how to make PCA maximally effective for this discriminative task? Etc. What I recommend you is to rework this and the other (just next) your questions and merge them into one question.
Answer: I addressed your confusion about PCA in your other question.  The correct version of your statement is that PCA attempts to characterize most of the variation in the in a lower dimensional space. As I explain in the answer to the other post PCA is not used for discrimination.  It has been used to do clustering after reducing dimension to a subset of the principal components.
If by imbalance of the data you mean one cluster center has many more points arund it than another then that may affect the ability of any clustering method at identifying the smaller cluster but the magnitude of separation of the clusters is probably more important.


Cannot access empty property on entity.inc line 277 when trying to access entity add page

Question: I created a new entity type with hook_entity_info using this code:
/**
 * Implements hook_entity_info().
 */
function custom_payment_entity_info() {
    return array(
        'custom_payment_period' => array(
            'label' => t('Period'),
            'plural label' => t('Periods'),
            'entity class' => 'Entity',
            'controller class' => 'EntityAPIController',
            'base table' => 'custom_payment_periods',
            'revision table' => 'custom_payment_period_revisions',
            'entity keys' => array(
                'id' => 'pid',
                'revision' => 'pvid',
            ),
            'fieldable' => TRUE,
            'static cache' => FALSE,
            'label callback' => 'entity_class_label',
            'uri callback' => 'entity_class_uri',
            'access callback' => 'custom_payment_access',
            'module' => 'custom_payment',
            'views controller class' => 'EntityDefaultViewsController',
            'admin ui' => array(
                'path' => 'admin/config/custom/custom-payment',
                'file' => '/forms/custom_payment.period.form.inc',
                'controller class' => 'EntityDefaultUIController',
                'menu wildcard' => '%period',
            ),
            'bundles' => array(
                'period' => array(
                    'label' => t('Period'),
                    'admin' => array(
                        'path' => 'admin/config/custom/custom-payment',
                        'access arguments' => array('administrator custom payment periods'),
                    )
                )
            )
        )
    );
}

But when i try to create new entity through the admin add page i get this error:

Fatal error: Cannot access empty property in
  F:\xampp\htdocs\sfn\sites\all\modules\entity\includes\entity.inc on
  line 277

This is the form function:
function custom_payment_period_form($form, &$form_state, $period = NULL) {
    $form['period'] = array(
        '#type' => 'textfield',
        '#title' => t('Period'),
        '#size' => 5,
        '#required' => TRUE,
        '#element_validate' => array('element_validate_integer_positive'),
        '#default_value' => isset($period->period) ? $period->period : ''
    );
    $form['type'] = array(
        '#type' => 'select',
        '#title' => t('Period Type'),
        '#options' => array(
            1 => t('Day'),
            2 => t('Month'),
            3 => t('Year')
        ),
        '#required' => TRUE,
        '#default_value' => isset($period->type) ? $period->type : ''
    );
    $form['price'] = array(
        '#type' => 'textfield',
        '#title' => t('Price'),
        '#size' => 5,
        '#field_suffix' => t('SYP'),
        '#required' => TRUE,
        '#element_validate' => array('element_validate_integer_positive'),
        '#default_value' => isset($period->price) ? $period->price : ''
    );
    $form['weight'] = array(
        '#type' => 'weight',
        '#title' => t('Position'),
        '#default_value' => isset($period->weight) ? $period->weight : ''
    );
    field_attach_form('period', $period, $form, $form_state);
    $form['actions'] = array(
        '#type' => 'actions',
        'submit' => array(
            '#type' => 'submit',
            '#value' => t('Save')
        )
    );
    return $form;
}

Can you help me fix this?

It's my first time working with entity type creation i followed the entity API documentation and the video entity example found here github.com/Lullabot/entity-api-demos/tree/master/docroot/sites/… i don't know exactly what is messing here.
Comment: Do you have an entity create function, and/or a menu loader function for like `'menu wildcard' => '%period',`. I dont really see where you're creating an empty Entity in your code for the form to use.
Comment: @tenken it's my first time working with entity type creation i followed the entity api documenation and the video entity example found here https://github.com/Lullabot/entity-api-demos/tree/master/docroot/sites/all/modules/custom/videoentity i don't know exaactly what is messing here
Comment: Your entity keys don't seem to include bundle, label, or language.
Comment: @tenken the problem was in two things: i had to add a label entity key to fix this error and then i had to remove the `'menu wildcard' => '%period'` from the admin ui array because i got an error on edit page because of it and everything worked then as expected but is there a way to create an entity type without a `"label"` entity key?
Answer: Your entity keys don't seem to include bundle, label, or language. - @beth
Answer: The problem was in two things:

I had to add a label entity key to fix this error
I had to remove the 'menu wildcard' => '%period' from the admin UI array,

because I got an error on edit page because of it and everything worked then as expected. It seems there was a problem creating an entity type without a "label" entity key.
Source: OP comment.


How to re-order user profile form?

Question: I have this code:
function mymod_form_user_profile_form_alter(&$form, &$form_state, $form_id) {
  $form['account']['#weight'] = 100;
}

Which is supposed to push the 'account' section of stuff underneath the rest of the form elements. But the weight seems to be fixed/the order is unchanged?
I've inspected the $form array at the point my code runs, and there are all the other fields, with their respective #weight values; and 100 is larger than any of those.
Answer: The weight you set here would be overridden by the weight values you set via Mange Fields.
However...
You can get around this by changing the weight in pre_render.
E.g.
function mymod_form_user_profile_form_alter(&$form, &$form_state, $form_id) {
  $form['#pre_render'][] = 'mymod_form_alter_weight';
}

function mymod_form_alter_weight($elements) {
  $elements['account']['#weight'] = 100;

  return $elements;
}

How to re-order user profile form?


C# ImageFormat to string

Question: How can I obtain human readable string(i.e. image format itself) from System.Drawing.ImageFormat object?
I mean if I have ImageFormat.Png is that possible to convert it to "png" string?
EDIT: I see some misunderstanding here. Here is mine code:
Image objImage = Image.FromStream(file);

ImageFormat imFormat = objImage.RawFormat;

imFormat.ToString();

It returns "[ImageFormat: b96b3caf-0728-11d3-9d7b-0000f81ef32e]" but I want "Png"!
Answer: Use the ImageFormatConverter class from the System.Drawing namespace:
this.imageInfoLabel.Text =
    new ImageFormatConverter().ConvertToString(this.Image.RawFormat);

For a PNG image it returns Png, and so on.
Answer: ImageFormat.Png.ToString() returns "Png"...
EDIT: OK, it seems ToString returns the name only for ImageFormat instances returned by the static properties...
You could create a lookup dictionary to get the name from the Guid:
private static readonly Dictionary<Guid, string> _knownImageFormats =
            (from p in typeof(ImageFormat).GetProperties(BindingFlags.Static | BindingFlags.Public)
             where p.PropertyType == typeof(ImageFormat)
             let value = (ImageFormat)p.GetValue(null, null)
             select new { Guid = value.Guid, Name = value.ToString() })
            .ToDictionary(p => p.Guid, p => p.Name);

static string GetImageFormatName(ImageFormat format)
{
    string name;
    if (_knownImageFormats.TryGetValue(format.Guid, out name))
        return name;
    return null;
}

Comment: Wow! Too hard solution for C# novice. I'll better use Equals here... But thanks any way!
Comment: Great solution ! Sad though that this is not readily available in ImageFormat.
Answer: ImageFormat values are identified by Guid you need to create your own map of Guid -> Name
var dict = (
    from t in typeof(ImageFormat).GetProperties()
    where t.PropertyType == typeof(ImageFormat)
    let v = (ImageFormat)t.GetValue(null, new object[0])
    select new { v.Guid, t.Name }
    ).ToDictionary(g => g.Guid, g => g.Name);

string name;
if (dict.TryGetValue(ImageFormat.Png.Guid, out name))
{
    Console.WriteLine(name);
}

Answer: There are not so many Image formats. So you can use a switch in case you want to specify your descriptions yourself or just use the
Imageformat.Specific.ToString()

(specific is the name of the specific image format)
Comment: That will not work because I have compile time reference on ImageFormat. See my edits in my origin question

C# ImageFormat to string