<commit_before>Building VW on Windows -- only 32 bit right now; 64 bit to come

8/15/2012, <NAME> <<EMAIL>>

You need Visual Studio 2010

(1) Install boost 1.50. There are several options available.

    ==> Build boost from scratch:

      (a) Download boost_1_50_0.zip from here http://sourceforge.net/projects/boost/files/boost/1.50.0/
      (b) Unzip to someplace convenient (I use c:\src)
      (c) Open a new command window
      (d) Run "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\vcvars32.bat" to set build variables
      (e) cd to the directory where you unzipped boost, and run "bootstrap.bat"
      (f) Run "mkdir c:\boost"
      (g) Run "mkdir c:\boost\x86"
      (h) Run "mkdir c:\boost\x64"
      (i) Run "b2 --prefix=c:\boost\x86 --build-dir=x86 --toolset=msvc install" (I add " -j 16" to the end to run up to 16 procs at once.)
      (j) Run "b2 --prefix=c:\boost\x64 --build-dir=x64 --toolset=msvc address-model=64 install"

    ==> Steal binaries from someone else. I'll post pre-built binaries soon.

    ==> Get pre-built binaries from boostpro -- BUT ONLY 32 BIT BINS ARE AVAILABLE

          http://boostpro.com/download/boost_1_50_setup.exe

          NOTE -- be sure to install binaries for VS 2010, and to check
                  ALL OF THE BOXES on the right hand side! If you get a
                  boost link error, this is the most likely culprit!

          ALSO NOTE -- you'll need to install more information

(2) Pick a base directory for sources -- I'll use c:\src\vw

(3) Download zlib from here:

  http://zlib.net/zlib127.zip

(4) Unzip to %ROOT% -- on my machine, this lands in c:\src\vw\zlib-1.2.7.

  (This must have the correct relative

(5) Build zlib

    (a) Start a new CMD window
    (b) Run "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\vcvars32.bat" to set build variables
    (c) Go to the %ROOT%\zlib-1.2.7 directory (for me, c:\src\vw\zlib-1.2.5)
    (d) Run "nmake /f win32\Makefile.msc"

(5a) [optional, for 64 bit support] Build 64-bit zlib

    (a) Unzip a new copy of zlib to c:\src\vw\zlib-1.2.7-x64
    (b) Start a new CMD window
    (c) Run "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\vcvars64.bat" to set build variables
    (d) Go to the %ROOT%\zlib-1.2.7-x64 dir
    (e) Run "nmake /f win32\Makefile.msc"

(6) Get a copy of VW in %ROOT%. I ran "cd \src\vw" and "git clone http

    (a) Change to root (for me, "cd \src\vw")
    (b) "git clone https://github.com/chrisquirk/vowpal_wabbit.git"

(7) Open %ROOT%\vowpal_wabbit\vowpalwabbit\vw.sln in Visual Studio 2010 and hit Build.

(8) Build. Binaries will be in one of these two directories, based on whether you built DEBUG or RELEASE bits.

  %ROOT%\vowpal_wabbit\vowpalwabbit\Debug\vw.exe
  %ROOT%\vowpal_wabbit\vowpalwabbit\Release\vw.exe

<commit_msg>document location of saved boost binaries<commit_after>Building VW on Windows -- only 32 bit right now; 64 bit to come

8/15/2012, <NAME> <<EMAIL>>

You need Visual Studio 2010

(1) Install boost 1.50. There are several options available.

    ==> Get pre-built binaries from someone else.

      (a) Download my pre-built boost-1.50-bins.zip from SkyDrive:

            http://sdrv.ms/RXV5gt

      (b) Unzip in the root of your C: drive, so you should have c:\boost\x86 and c:\boost\x64 directories.


    ==> Build boost from scratch:

      (a) Download boost_1_50_0.zip from here http://sourceforge.net/projects/boost/files/boost/1.50.0/
      (b) Unzip to someplace convenient (I use c:\src)
      (c) Open a new command window
      (d) Run "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\vcvars32.bat" to set build variables
      (e) cd to the directory where you unzipped boost, and run "bootstrap.bat"
      (f) Run "mkdir c:\boost"
      (g) Run "mkdir c:\boost\x86"
      (h) Run "mkdir c:\boost\x64"
      (i) Run "b2 --prefix=c:\boost\x86 --build-dir=x86 --toolset=msvc install" (I add " -j 16" to the end to run up to 16 procs at once.)
      (j) Run "b2 --prefix=c:\boost\x64 --build-dir=x64 --toolset=msvc address-model=64 install"


    ==> Get pre-built binaries from boostpro -- BUT ONLY 32 BIT BINS ARE AVAILABLE

          http://boostpro.com/download/boost_1_50_setup.exe

          NOTE -- be sure to install binaries for VS 2010, and to check
                  ALL OF THE BOXES on the right hand side! If you get a
                  boost link error, this is the most likely culprit!

          ALSO NOTE -- you'll need to install more information

(2) Pick a base directory for sources -- I'll use c:\src\vw

(3) Download zlib from here:

  http://zlib.net/zlib127.zip

(4) Unzip to %ROOT% -- on my machine, this lands in c:\src\vw\zlib-1.2.7.

  (This must have the correct relative

(5) Build zlib

    (a) Start a new CMD window
    (b) Run "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\vcvars32.bat" to set build variables
    (c) Go to the %ROOT%\zlib-1.2.7 directory (for me, c:\src\vw\zlib-1.2.5)
    (d) Run "nmake /f win32\Makefile.msc"

(5a) [optional, for 64 bit support] Build 64-bit zlib

    (a) Unzip a new copy of zlib to c:\src\vw\zlib-1.2.7-x64
    (b) Start a new CMD window
    (c) Run "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\vcvars64.bat" to set build variables
    (d) Go to the %ROOT%\zlib-1.2.7-x64 dir
    (e) Run "nmake /f win32\Makefile.msc"

(6) Get a copy of VW in %ROOT%. I ran "cd \src\vw" and "git clone http

    (a) Change to root (for me, "cd \src\vw")
    (b) "git clone https://github.com/chrisquirk/vowpal_wabbit.git"

(7) Open %ROOT%\vowpal_wabbit\vowpalwabbit\vw.sln in Visual Studio 2010 and hit Build.

(8) Build. Binaries will be in one of these four directories, based on whether you built DEBUG or RELEASE bits and whether you are building x64 or Win32.

  %ROOT%\vowpal_wabbit\vowpalwabbit\Debug\vw.exe
  %ROOT%\vowpal_wabbit\vowpalwabbit\Release\vw.exe
  %ROOT%\vowpal_wabbit\vowpalwabbit\x64\Debug\vw.exe
  %ROOT%\vowpal_wabbit\vowpalwabbit\x64\Release\vw.exe


{'fraction_non_alphanumeric': 0.11980033277870217, 'fraction_numerical': 0.042928452579034944, 'mean_word_length': 2.7924290220820187, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 15, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4366263', 'n_tokens_mistral': 2356, 'n_tokens_neox': 2120, 'n_words': 756}