How to perform basic math with tensors

When dealing with higher-dimensional tensors, nested loops can quickly become cumbersome. This is where recursive functions or libraries like TensorFlow.js shine because they handle these operations efficiently under the hood.

Matrix multiplication, however, is a different beast. Unlike element-wise multiplication, matrix multiplication follows linear algebra rules where the inner dimensions must match. For two matrices A (m x n) and B (n x p), the resulting matrix will be of shape (m x p).

Here’s a simple implementation of matrix multiplication in JavaScript:

function matrixMultiply(A, B) {
  const rowsA = A.length;
  const colsA = A[0].length;
  const rowsB = B.length;
  const colsB = B[0].length;

  if (colsA !== rowsB) {
    throw new Error('Number of columns in A must equal number of rows in B');
  }

  const result = new Array(rowsA);
  for (let i = 0; i < rowsA; i++) {
    result[i] = new Array(colsB).fill(0);
    for (let j = 0; j < colsB; j++) {
      for (let k = 0; k < colsA; k++) {
        result[i][j] += A[i][k] * B[k][j];
      }
    }
  }
  return result;
}

const matA = [
  [1, 2, 3],
  [4, 5, 6]
];

const matB = [
  [7, 8],
  [9, 10],
  [11, 12]
];

console.log(matrixMultiply(matA, matB));
// Output: [[58, 64], [139, 154]]

Note that this operation is not element-wise but a fundamental building block for many machine learning algorithms, including neural networks. Efficient implementations rely on optimized libraries or hardware acceleration.

Broadcasting is another important concept, allowing arithmetic operations between tensors of different shapes, provided they’re compatible in specific ways. For instance, adding a vector to each row of a matrix is a common use case.

Here’s a simplified example of broadcasting a 1D tensor (vector) to add to each row of a 2D tensor (matrix):

const matrix = [
  [1, 2, 3],
  [4, 5, 6]
];

const vector = [10, 20, 30];

const broadcastAdd = (mat, vec) => {
  return mat.map(row => row.map((val, idx) => val + vec[idx]));
};

console.log(broadcastAdd(matrix, vector));
// Output: [[11, 22, 33], [14, 25, 36]]

Understanding when and how broadcasting applies can save you from writing verbose loops and can improve code readability. However, incorrect assumptions about shapes often lead to bugs or unexpected results, so it is critical to verify tensor dimensions before performing operations.

Division by zero is another pitfall when performing element-wise division. Always ensure your denominator tensor does not contain zeros or handle such cases explicitly to avoid runtime errors or NaN values.

In practice, libraries like TensorFlow.js or PyTorch manage these complexities for you, but grasping these fundamentals ensures you can debug and optimize your tensor operations when things go wrong or when performance matters.

Here’s a quick defensive implementation that avoids division by zero by substituting a small epsilon value:

const safeDivide = (a, b, epsilon = 1e-8) => {
  const result = [];
  for (let i = 0; i < a.length; i++) {
    const row = [];
    for (let j = 0; j < a[i].length; j++) {
      row.push(a[i][j] / (b[i][j] === 0 ? epsilon : b[i][j]));
    }
    result.push(row);
  }
  return result;
};

With these tools and considerations, you can confidently perform basic arithmetic operations on tensors, which form the foundation for more advanced numerical computations and machine learning tasks. The next challenge is recognizing and avoiding the common pitfalls that often trip up developers working with tensor math,

Avoiding common pitfalls when working with tensor math

When working with tensor math, there are several common pitfalls that can lead to unexpected behaviors or errors in your computations. One frequent issue arises from misunderstanding the shapes and dimensions of tensors. As mentioned earlier, tensors must have compatible shapes for many operations, including addition and multiplication. If you attempt to perform operations on tensors with incompatible shapes, you will encounter errors that can be frustrating to debug.

Another common mistake is assuming that all operations are element-wise without considering broadcasting rules. Broadcasting can make it seem like operations are valid when they’re not, especially when working with different dimensions. It’s crucial to understand how broadcasting works and ensure that your tensors align correctly when performing arithmetic operations. Misalignment can lead to subtle bugs that are difficult to trace.

Here’s an example where broadcasting might lead to confusion:

const tensor1 = [
  [1, 2, 3],
  [4, 5, 6]
];

const tensor2 = [10, 20, 30];

const result = tensor1.map(row => row.map((val, idx) => val + tensor2[idx]));
console.log(result);
// Output: [[11, 22, 33], [14, 25, 36]]

In this case, tensor2 is treated as being broadcast across each row of tensor1, which can be convenient but also misleading if you’re not careful. Always double-check the dimensions involved in your operations.

Another pitfall is neglecting to account for numerical stability. When performing calculations that involve division, especially in machine learning contexts, you may encounter issues like division by zero. It’s essential to implement checks or use small epsilon values to avoid these errors. Here’s an improved division function that addresses this concern:

const safeDivide = (a, b, epsilon = 1e-8) => {
  const result = [];
  for (let i = 0; i < a.length; i++) {
    const row = [];
    for (let j = 0; j < a[i].length; j++) {
      const denominator = b[i][j] === 0 ? epsilon : b[i][j];
      row.push(a[i][j] / denominator);
    }
    result.push(row);
  }
  return result;
};

In this implementation, we replace any zero in the denominator with a small epsilon value, which helps maintain numerical stability without introducing significant errors into the calculations.

Furthermore, it’s important to keep an eye on performance when working with large tensors. Inefficient looping and repeated calculations can lead to significant slowdowns. Consider using optimized libraries or batch processing to handle larger datasets effectively. For example, using methods from TensorFlow.js can greatly enhance performance due to its underlying optimizations.

Lastly, always validate your results. After performing operations on tensors, especially complex ones, it’s a good practice to check the outputs against expected results. Unit tests can be particularly useful in this regard, as they help catch unexpected behaviors early in the development process.

By being aware of these common pitfalls and implementing strategies to avoid them, you’ll enhance the reliability and efficiency of your tensor operations. Understanding these nuances will empower you to tackle more sophisticated machine learning tasks with confidence.