<commit_before><!-- Google Analytics -->
<script src="js/google-analytics.js" type="text/javascript"></script>

<!-- prettify js and CSS -->
<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js?lang=scala"></script>

<!-- scroll/follow sidebar -->
<script src="js/follow-sidebar.js" type="text/javascript"></script>

<div class="col-md-3 col-md-push-9">
    <div id="sidebar">
        <div class="sidebar-toc" style="margin-bottom: 20px;">
            <p class="toc-header">Contents</p>
            <div id="toc"></div>
        </div>
    </div>
</div>

<div class="col-md-9 col-md-pull-3">

    <h1 id="visualization" class="title">Visualization</h1>

    <p>A picture is worth a thousand words. In machine learning, we usually handle
        high-dimensional data, which is impossible to draw on display directly. But
        a variety of statistical plots are tremendously valuable for us to grasp
        the characteristics of many data points. Smile provides data visualization tools
        such as plots and maps for researchers to understand information more easily and quickly.</p>

    <h2 id="scatter" class="title">Scatter Plot</h2>

    <p>A scatter plot displays data as a collection of points. The points can be color-coded,
        which is very useful for classification tasks.
        The user can use <code>plot</code> functions to draw scatter plot easily.</p>

    <pre class="prettyprint lang-scala"><code>
    def plot(data: Array[Array[Double]], legend: Char = '*', color: Color = Color.BLACK): (JFrame, PlotCanvas)

    def plot(data: Array[Array[Double]], labels: Array[String]): (JFrame, PlotCanvas)

    def plot(data: Array[Array[Double]], label: Array[Int], legend: Char, palette: Array[Color]): (JFrame, PlotCanvas)

    def plot(data: Array[Array[Double]], label: Array[Int], legend: Array[Char], palette: Array[Color]): (JFrame, PlotCanvas)
    </code></pre>

    <p>The legends are as follows.</p>
    <ul>
        <li> <code>.</code> : dot
        <li> <code>+</code> : +
        <li> <code>-</code> : -
        <li> <code>|</code> : |
        <li> <code>*</code> : star
        <li> <code>x</code> : x
        <li> <code>o</code> : circle
        <li> <code>O</code> : large circle
        <li> <code>@</code> : solid circle
        <li> <code>#</code> : large solid circle
        <li> <code>s</code> : square
        <li> <code>S</code> : large square
        <li> <code>q</code> : solid square
        <li> <code>Q</code> : large solid square
    </ul>
    <p>For any other char, the data point will be drawn as a dot.</p>

    <p>The functions return a tuple of
        JFrame and PlotCanvas. The PlotCanvas object can be used to control the plot programmatically.
        The user can also use the popup context menu by right mouse click to print, change the title, axis labels,
        and font, etc.</p>

    <p>For both 2D and 3D plot, the user can zoom in/out by mouse wheel. For 2D plot,
        the user can shift the coordinates by moving mouse after double click. The
        user can also select an area by mouse for detailed view. For 3D plot, the user
        can rotate the view by dragging mouse.</p>

    <pre class="prettyprint lang-scala"><code>
    val iris = readArff("shell/src/universal/data/weka/iris.arff", 4)
    val (x, y) = iris.unzipInt
    val x2 = x.map(_.slice(0,2)) // take first two columns
    val (frame, canvas) = plot(x2, y, Array('*', '+', 'o'), Array(Color.RED, Color.BLUE, Color.CYAN))
    canvas.setAxisLabels(iris.attributes.map(_.getName).slice(0,2): _*)
    </code></pre>

    <p>In this example, we plot the first two columns of Iris data. We use the class
        label for legend and color coding.</p>

    <div style="width: 100%; display: inline-block; text-align: center;">
        <img src="images/iris2d.png" style="width: 480px;" />
        <div class="caption" style="min-width: 480px;">Iris 2D Scatter Plot</div>
    </div>

    <p>It is also easy to draw a 3D plot.</p>

    <pre class="prettyprint lang-scala"><code>
    val x3 = x.map(_.slice(0,3)) // take first three columns
    val (frame, canvas) = plot(x3, y, Array('*', '+', 'o'), Array(Color.RED, Color.BLUE, Color.CYAN))<commit_msg>fix data path<commit_after><!-- Google Analytics -->
<script src="js/google-analytics.js" type="text/javascript"></script>

<!-- prettify js and CSS -->
<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js?lang=scala"></script>

<!-- scroll/follow sidebar -->
<script src="js/follow-sidebar.js" type="text/javascript"></script>

<div class="col-md-3 col-md-push-9">
    <div id="sidebar">
        <div class="sidebar-toc" style="margin-bottom: 20px;">
            <p class="toc-header">Contents</p>
            <div id="toc"></div>
        </div>
    </div>
</div>

<div class="col-md-9 col-md-pull-3">

    <h1 id="visualization" class="title">Visualization</h1>

    <p>A picture is worth a thousand words. In machine learning, we usually handle
        high-dimensional data, which is impossible to draw on display directly. But
        a variety of statistical plots are tremendously valuable for us to grasp
        the characteristics of many data points. Smile provides data visualization tools
        such as plots and maps for researchers to understand information more easily and quickly.</p>

    <h2 id="scatter" class="title">Scatter Plot</h2>

    <p>A scatter plot displays data as a collection of points. The points can be color-coded,
        which is very useful for classification tasks.
        The user can use <code>plot</code> functions to draw scatter plot easily.</p>

    <pre class="prettyprint lang-scala"><code>
    def plot(data: Array[Array[Double]], legend: Char = '*', color: Color = Color.BLACK): (JFrame, PlotCanvas)

    def plot(data: Array[Array[Double]], labels: Array[String]): (JFrame, PlotCanvas)

    def plot(data: Array[Array[Double]], label: Array[Int], legend: Char, palette: Array[Color]): (JFrame, PlotCanvas)

    def plot(data: Array[Array[Double]], label: Array[Int], legend: Array[Char], palette: Array[Color]): (JFrame, PlotCanvas)
    </code></pre>

    <p>The legends are as follows.</p>
    <ul>
        <li> <code>.</code> : dot
        <li> <code>+</code> : +
        <li> <code>-</code> : -
        <li> <code>|</code> : |
        <li> <code>*</code> : star
        <li> <code>x</code> : x
        <li> <code>o</code> : circle
        <li> <code>O</code> : large circle
        <li> <code>@</code> : solid circle
        <li> <code>#</code> : large solid circle
        <li> <code>s</code> : square
        <li> <code>S</code> : large square
        <li> <code>q</code> : solid square
        <li> <code>Q</code> : large solid square
    </ul>
    <p>For any other char, the data point will be drawn as a dot.</p>

    <p>The functions return a tuple of
        JFrame and PlotCanvas. The PlotCanvas object can be used to control the plot programmatically.
        The user can also use the popup context menu by right mouse click to print, change the title, axis labels,
        and font, etc.</p>

    <p>For both 2D and 3D plot, the user can zoom in/out by mouse wheel. For 2D plot,
        the user can shift the coordinates by moving mouse after double click. The
        user can also select an area by mouse for detailed view. For 3D plot, the user
        can rotate the view by dragging mouse.</p>

    <pre class="prettyprint lang-scala"><code>
    val iris = readArff("data/weka/iris.arff", 4)
    val (x, y) = iris.unzipInt
    val x2 = x.map(_.slice(0,2)) // take first two columns
    val (frame, canvas) = plot(x2, y, Array('*', '+', 'o'), Array(Color.RED, Color.BLUE, Color.CYAN))
    canvas.setAxisLabels(iris.attributes.map(_.getName).slice(0,2): _*)
    </code></pre>

    <p>In this example, we plot the first two columns of Iris data. We use the class
        label for legend and color coding.</p>

    <div style="width: 100%; display: inline-block; text-align: center;">
        <img src="images/iris2d.png" style="width: 480px;" />
        <div class="caption" style="min-width: 480px;">Iris 2D Scatter Plot</div>
    </div>

    <p>It is also easy to draw a 3D plot.</p>

    <pre class="prettyprint lang-scala"><code>
    val x3 = x.map(_.slice(0,3)) // take first three columns
    val (frame, canvas) = plot(x3, y, Array('*', '+', 'o'), Array(Color.RED, Color.BLUE, Color.CYAN))
{'fraction_non_alphanumeric': 0.15040360873694206, 'fraction_numerical': 0.008784425451092117, 'mean_word_length': 3.0311004784688995, 'pattern_counts': {'":': 0, '<': 205, '<?xml version=': 0, '>': 205, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2390146', 'n_tokens_mistral': 2752, 'n_tokens_neox': 2614, 'n_words': 887}