<commit_before>---
layout: post
title: "TensorFlow.js, dataSync, and why you should be careful"
category: posts
---

One of the first things you stumble on when you start using TensorFlow.js is
that sometimes you need your data as a Tensor, and sometimes you need it
as a JavaScript number. Maybe it's for logging it, maybe it's for displaying
it somewhere during training, maybe it's because you don't trust the robots
to be better than you at math.

This is a quick post that tries to clarify why doing this **synchronously**
is probably bad and will leave your UI really janky. [Nikhil](https://twitter.com/nsthorat)
(who like, birthed TensorFlow.js, bless) was kind enough to explain this to me recently, so I figured
I'd return the favour, with fewer meeps and more mistakes.

## Downloading and Uploading
When you create a Tensor, it lives on the CPU. The mere fact that it's a Tensor
doesn't automatically move that data into its GPU mansion -- it needs to be used in a WebGL
program. (I'm playing fast and loose here with the words GPU and CPU btw, so
hold back the pedantics: when I say "it lives on the CPU" I mean "in main memory,
where the CPU processes stuff"; the GPU has it‘s own memory, that's where it
processes stuff, and that’s where that data has to get transferred to.
It's fine. You know I know.)

You **upload** the tensor to the GPU when you call one of the `tf.` operations on it.
Tensor operations are matrix math, and matrix math is really fast on the GPU,
so every time you call something like `sum` or `sqrt` on a Tensor, TensorFlow.js
creates a little [WebGL operation](https://js.tensorflow.org/tutorials/custom-webgl-op.html),
and sends it to the backend. Whatever data lived on the CPU is now
"uploaded" to the GPU (to a WebGL texture).
<commit_msg>steal nikhil's better title<commit_after>---
layout: post
title: "The perils of tensor.dataSync()"
category: posts
---

One of the first things you stumble on when you start using TensorFlow.js is
that sometimes you need your data as a Tensor, and sometimes you need it
as a JavaScript number. Maybe it's for logging it, maybe it's for displaying
it somewhere during training, maybe it's because you don't trust the robots
to be better than you at math.

This is a quick post that tries to clarify why doing this **synchronously**
is probably bad and will leave your UI really janky. [Nikhil](https://twitter.com/nsthorat)
(who like, birthed TensorFlow.js, bless) was kind enough to explain this to me recently, so I figured
I'd return the favour, with fewer meeps and more mistakes.

## Downloading and Uploading
When you create a Tensor, it lives on the CPU. The mere fact that it's a Tensor
doesn't automatically move that data into its GPU mansion -- it needs to be used in a WebGL
program. (I'm playing fast and loose here with the words GPU and CPU btw, so
hold back the pedantics: when I say "it lives on the CPU" I mean "in main memory,
where the CPU processes stuff"; the GPU has it‘s own memory, that's where it
processes stuff, and that’s where that data has to get transferred to.
It's fine. You know I know.)

You **upload** the tensor to the GPU when you call one of the `tf.` operations on it.
Tensor operations are matrix math, and matrix math is really fast on the GPU,
so every time you call something like `sum` or `sqrt` on a Tensor, TensorFlow.js
creates a little [WebGL operation](https://js.tensorflow.org/tutorials/custom-webgl-op.html),
and sends it to the backend. Whatever data lived on the CPU is now
"uploaded" to the GPU (to a WebGL texture).

{'fraction_non_alphanumeric': 0.06554149085794655, 'fraction_numerical': 0.0, 'mean_word_length': 4.183673469387755, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2639410', 'n_tokens_mistral': 1002, 'n_tokens_neox': 965, 'n_words': 595}