NDArray

About This Module

Prework

Prework Reading

Pre-lecture Reflections

Lecture

Learning Objectives

• An $n$-dimensional (e.g. 1-D, 2-D, 3-D, ...) container for general elements and numerics.

• Similar to Python's Numpy but with important differences.

• Rich set of methods for creating views and slices into the array

• Provides a rich set of methods and mathematical operations

Key similarities with Python Numpy (from link)

• Arrays have single element types
• Arrays can have arbitrary dimensions
• Arrays can have arbitrary strides
• Indexing starts at 0
• Default ordering is row-major (more on that below)
• Arithmetic operators (+, -, *, /) perform element-wise operations
• Arrays that are not views are contiguous in memory
• Many cheap operations that return views into the array instead of copying data

Some important differences from Numpy

• Numpy arrays and views are all mutable and can all change the contents of an array.
• NDarrays have:
  • flavors that can change contents,
  • flavors that cannot, and
  • flavors that make copies when things change.
• Numpy arrays are always dynamic in their number of dimensions. NDarrays can be static or dynamic.
• Slices with negative steps behave differently (more on that later)

To use it

To add latest ndarray crate.

% cargo add ndarray

or manually add

[dependencies]  
ndarray="0.15.6"  

or later in your Cargo.toml file

Why use NDarray over Vec or array?

It is easier to do a bunch of things like:

• Data Cleaning and Preprocessing offering functions like slicing and fill
• Statistics built in (lots of math functions come with it)
• Machine learning: Used in many of the ML libraries written in Rust
• Linear Algebra: Built in methods like matrix inversion, multiplication and decomposition.

Let's look at some example usage.

// This is required in a Jupyter notebook.
// For a cargo project, you would add it to your Cargo.toml file.
:dep ndarray = { version = "^0.15.6" }

use ndarray::prelude::*;

fn main() {
    let a = array![         // handy macro for creating arrays
                [1.,2.,3.], 
                [4.,5.,6.],
            ]; 
    assert_eq!(a.ndim(), 2);         // get the number of dimensions of array a
    assert_eq!(a.len(), 6);          // get the number of elements in array a
    assert_eq!(a.shape(), [2, 3]);   // get the shape of array a
    assert_eq!(a.is_empty(), false); // check if the array has zero elements

    println!("Print the array with debug formatting:");
    println!("{:?}", a);

    println!("\nPrint the array with display formatting:");
    println!("{}", a);
}

main();

Print the array with debug formatting:
[[1.0, 2.0, 3.0],
 [4.0, 5.0, 6.0]], shape=[2, 3], strides=[3, 1], layout=Cc (0x5), const ndim=2

Print the array with display formatting:
[[1, 2, 3],

For a side by side comparison NumPy, see https://docs.rs/ndarray/latest/ndarray/doc/ndarray_for_numpy_users/index.html

Array Creation

#![allow(unused)]
fn main() {
let a:Array<f64, Ix1> = Array::linspace(0., 4.5, 10);
println!("{:?}", a);

let b:Array<f64, _> = a.into_shape((2,5)).unwrap();
println!("\n{:?}", b);

let c:Array<f64, _> = Array::zeros((3,4).f());
println!("\n{:?}", c);
}

 [4, 5, 6]]
[0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5], shape=[10], strides=[1], layout=CFcf (0xf), const ndim=1

[[0.0, 0.5, 1.0, 1.5, 2.0],
 [2.5, 3.0, 3.5, 4.0, 4.5]], shape=[2, 5], strides=[5, 1], layout=Cc (0x5), const ndim=2

Good overview at this TDS Post, but subscription required.

#![allow(unused)]
fn main() {
:dep ndarray = { version = "^0.15.6" }

// Not working on Jupyter notebook
//:dep ndarray-rand = { version = "^0.15.0" }

use ndarray::{Array, ShapeBuilder};

// Not working on Jupyter notebook
//use ndarray_rand::RandomExt;
//use ndarray_rand::rand_distr::Uniform;


// Ones

let ones = Array::<f64, _>::ones((1, 4));
println!("{:?}", ones); 

// Output:
// [[1.0, 1.0, 1.0, 1.0]], shape=[1, 4], strides=[4, 1], layout=CFcf (0xf), const ndim=2

// Range

let range = Array::<f64, _>::range(0., 5., 1.);
println!("{:?}", range); 

// Output:
// [0.0, 1.0, 2.0, 3.0, 4.0], shape=[5], strides=[1], layout=CFcf (0xf), const ndim=1

// Linspace

let linspace = Array::<f64, _>::linspace(0., 5., 5);
println!("{:?}", linspace); 

// Output:
// [0.0, 1.25, 2.5, 3.75, 5.0], shape=[5], strides=[1], layout=CFcf (0xf), const ndim=1

// Fill

let mut ones = Array::<f64, _>::ones((1, 4));
ones.fill(0.);
println!("{:?}", ones); 

// Output:
// [[0.0, 0.0, 0.0, 0.0]], shape=[1, 4], strides=[4, 1], layout=CFcf (0xf), const ndim=2

// Eye -- Identity Matrix

let eye = Array::<f64, _>::eye(4);
println!("{:?}", eye); 

// Output:
// [[1.0, 0.0, 0.0, 0.0],
// [0.0, 1.0, 0.0, 0.0],
// [0.0, 0.0, 1.0, 0.0],
// [0.0, 0.0, 0.0, 1.0]], shape=[4, 4], strides=[4, 1], layout=Cc (0x5), const ndim=2

// Random

// Not working on Jupyter notebook

//let random = Array::random((2, 5), Uniform::new(0., 10.));
//println!("{:?}", random);

// Output:
// [[9.375493735188611, 4.088737328406999, 9.778579742815943, 0.5225866490310649, 1.518053969762827],
//  [9.860829919571666, 2.9473768443117, 7.768332993584486, 7.163926861520167, 9.814750664983297]], shape=[2, 5], strides=[5, 1], layout=Cc (0x5), const ndim=2
}

[[0.0, 0.0, 0.0, 0.0],
 [0.0, 0.0, 0.0, 0.0],
 [0.0, 0.0, 0.0, 0.0]], shape=[3, 4], strides=[1, 3], layout=Ff (0xa), const ndim=2
[[1.0, 1.0, 1.0, 1.0]], shape=[1, 4], strides=[4, 1], layout=CFcf (0xf), const ndim=2
[0.0, 1.0, 2.0, 3.0, 4.0], shape=[5], strides=[1], layout=CFcf (0xf), const ndim=1
[0.0, 1.25, 2.5, 3.75, 5.0], shape=[5], strides=[1], layout=CFcf (0xf), const ndim=1

What does the printout mean?

• The values of the array
• The shape of the array, most important dimension first
• The stride (always 1 for arrays in the last dimension, but can be different for views)
• The layout (storage order, view order)
• The number of dimensions

Indexing and Slicing

The s![] slice macro

• The s![] macro in Rust's ndarray crate is a convenient way to create slice specifications for array operations.

• It's used to create a SliceInfo object that describes how to slice or view an array.

Here's how it works:

1. The s![] macro is used to create slice specifications that are similar to Python's slice notation
2. It's commonly used with methods like slice(), slice_mut(), and other array view operations
3. Inside the macro, you can specify ranges and steps for each dimension

For example:

#![allow(unused)]
fn main() {
let mut slice = array.slice_mut(s![1.., 0, ..]);
}

This creates a slice that:

• Takes all elements from index 1 onwards in the first dimension (1..)
• Takes only index 0 in the second dimension (0)
• Takes all elements in the third dimension (..)

The syntax inside s![] supports several patterns:

• .. - take all elements in that dimension
• start..end - take elements from start (inclusive) to end (exclusive)
• start..=end - take elements from start (inclusive) to end (inclusive)
• start.. - take elements from start (inclusive) to the end
• ..end - take elements from the beginning to end (exclusive)
• index - take only that specific index

For example:

#![allow(unused)]
fn main() {
// Take every other element in the first dimension
s![..;2]

// Take elements 1 to 3 in the first dimension, all elements in the second
s![1..4, ..]

// Take elements from index 2 to the end, with step size 2
s![2..;2]

// Take specific indices
s![1, 2, 3]
}

From TDS Post.

#![allow(unused)]
fn main() {
use ndarray::{s};
{
    // Create a 3-dimensional array (2x3x4)
    let mut array = Array::from_shape_fn((2, 3, 4), |(i, j, k)| {
        (i * 100 + j * 10 + k) as f32
    });

    // Print the original 3-dimensional array
    println!("Original 3D array:\n{:?}", array);

    // Create a 2-dimensional slice (taking the first 2D layer)
    let mut slice = array.slice_mut(s![1.., 0, ..]);

    // Print the 2-dimensional slice
    println!("2D slice:\n{:?}", slice);

    // Create a 1-dimensional slice (taking the first 2D and 3D layer)
    let slice2 = array.slice(s![1, 0, ..]);

    // Print the 1-dimensional slice
    println!("1D slice:\n{:?}", slice2);

}
}

[[0.0, 0.0, 0.0, 0.0]], shape=[1, 4], strides=[4, 1], layout=CFcf (0xf), const ndim=2
[[1.0, 0.0, 0.0, 0.0],
 [0.0, 1.0, 0.0, 0.0],
 [0.0, 0.0, 1.0, 0.0],
 [0.0, 0.0, 0.0, 1.0]], shape=[4, 4], strides=[4, 1], layout=Cc (0x5), const ndim=2
Original 3D array:
[[[0.0, 1.0, 2.0, 3.0],
  [10.0, 11.0, 12.0, 13.0],
  [20.0, 21.0, 22.0, 23.0]],

 [[100.0, 101.0, 102.0, 103.0],
  [110.0, 111.0, 112.0, 113.0],
  [120.0, 121.0, 122.0, 123.0]]], shape=[2, 3, 4], strides=[12, 4, 1], layout=Cc (0x5), const ndim=3
2D slice:
[[100.0, 101.0, 102.0, 103.0]], shape=[1, 4], strides=[0, 1], layout=CFcf (0xf), const ndim=2
1D slice:
[100.0, 101.0, 102.0, 103.0], shape=[4], strides=[1], layout=CFcf (0xf), const ndim=1





()

2D and 3D datatypes, again from TDS Post.

#![allow(unused)]
fn main() {
use ndarray::{array, Array, Array2, Array3, ShapeBuilder};

// 1D array
let array_d1 = Array::from_vec(vec![1., 2., 3., 4.]);
println!("{:?}", array_d1);

// Output:
// [1.0, 2.0, 3.0, 4.0], shape=[4], strides=[1], layout=CFcf (0xf), const ndim=1

// or 

let array_d11 = Array::from_shape_vec((1, 4), vec![1., 2., 3., 4.]);
println!("{:?}", array_d11.unwrap());

// Output:
// [[1.0, 2.0, 3.0, 4.0]], shape=[1, 4], strides=[4, 1], layout=CFcf (0xf), const ndim=2

// 2D array

let array_d2 = array![
    [-1.01,  0.86, -4.60,  3.31, -4.81],
    [ 3.98,  0.53, -7.04,  5.29,  3.55],
    [ 3.30,  8.26, -3.89,  8.20, -1.51],
    [ 4.43,  4.96, -7.66, -7.33,  6.18],
    [ 7.31, -6.43, -6.16,  2.47,  5.58],
];

// or

let array_d2 = Array::from_shape_vec((2, 2), vec![1., 2., 3., 4.]);
println!("{:?}", array_d2.unwrap());

// Output:
// [[1.0, 2.0],
// [3.0, 4.0]], shape=[2, 2], strides=[2, 1], layout=Cc (0x5), const ndim=2

// or

let mut data = vec![1., 2., 3., 4.];
let array_d21 = Array2::from_shape_vec((2, 2), data);

// 3D array

let mut data = vec![1., 2., 3., 4.];
let array_d3 = Array3::from_shape_vec((2, 2, 1), data);
println!("{:?}", array_d3);

// Output:
// [[[1.0],
//  [2.0]],
//  [[3.0],
//  [4.0]]], shape=[2, 2, 1], strides=[2, 1, 1], layout=Cc (0x5), const ndim=3
}

[1.0, 2.0, 3.0, 4.0], shape=[4], strides=[1], layout=CFcf (0xf), const ndim=1
[[1.0, 2.0, 3.0, 4.0]], shape=[1, 4], strides=[4, 1], layout=CFcf (0xf), const ndim=2
[[1.0, 2.0],
 [3.0, 4.0]], shape=[2, 2], strides=[2, 1], layout=Cc (0x5), const ndim=2

Reshaping

From TDS Post.

From TDS Post.

Examining the array parameters

Simple Math

mapv vs map:

mapv iterates over the values of the array, map iterates over mutable references of the array

Array-n

Array0 to Array6 also defined in addition to Array as special cases with fixed dimensions instead dynamically defined dimensions.

#![allow(unused)]
fn main() {
{
    // Create a 2-dimensional array (3x4)
    let mut a = Array::from_shape_fn((3, 4), |(j, k)| {
        (j * 10 + k) as f32
    });

    // Print the original 2-dimensional array
    println!("Original 2D array:\n{:?}", a);

    let b = a * 2.0;
    println!("\nb:\n{:?}", b);

    let c = b.mapv(|v| v>24.8);
    println!("\nc:\n{:?}", c);
}
}

Ok([[[1.0],
  [2.0]],

 [[3.0],
  [4.0]]], shape=[2, 2, 1], strides=[2, 1, 1], layout=Cc (0x5), const ndim=3)
Original 2D array:
[[0.0, 1.0, 2.0, 3.0],
 [10.0, 11.0, 12.0, 13.0],
 [20.0, 21.0, 22.0, 23.0]], shape=[3, 4], strides=[4, 1], layout=Cc (0x5), const ndim=2

b:
[[0.0, 2.0, 4.0, 6.0],
 [20.0, 22.0, 24.0, 26.0],
 [40.0, 42.0, 44.0, 46.0]], shape=[3, 4], strides=[4, 1], layout=Cc (0x5), const ndim=2

c:
[[false, false, false, false],
 [false, false, false, true],
 [true, true, true, true]], shape=[3, 4], strides=[4, 1], layout=Cc (0x5), const ndim=2





()

Type Conversions

• std::convert::From ensures lossless, safe conversions at compile-time and is generally recommended.
• std::convert::TryFrom can be used for potentially unsafe conversions. It will return a Result which can be handled or unwrap()ed to panic if any value at runtime cannot be converted losslessly.

Basic Linear Algebra

Provided by NDArray.

Transpose

#![allow(unused)]
fn main() {
let array_d2 = Array::from_shape_vec((2, 2), vec![1., 2., 3., 4.]);
println!("{:?}", array_d2.unwrap());

// Output
// [[1.0, 2.0],
//  [3.0, 4.0]], shape=[2, 2], strides=[2, 1], layout=Cc (0x5), const ndim=2)

let binding = array_d2.expect("Expect 2d matrix");

let array_d2t = binding.t();
println!("{:?}", array_d2t);

// Output
// [[1.0, 3.0],
//  [2.0, 4.0]], shape=[2, 2], strides=[1, 2], layout=Ff (0xa), const ndim=2

}

[E0382] Error: use of moved value: `array_d2`

   ╭─[command_8:1:1]

   │

 1 │ let array_d2 = Array::from_shape_vec((2, 2), vec![1., 2., 3., 4.]);

   │     ────┬───  

   │         ╰───── move occurs because `array_d2` has type `Result<ndarray::ArrayBase<OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ShapeError>`, which does not implement the `Copy` trait

 2 │ println!("{:?}", array_d2.unwrap());

   │                  ────┬───│────┬───  

   │                      ╰────────────── help: consider calling `.as_ref()` or `.as_mut()` to borrow the type's contents

   │                          │    │     

   │                          ╰────────── help: you can `clone` the value and consume it, but this might not be your desired behavior: `.clone()`

   │                               │     

   │                               ╰───── `array_d2` moved due to this method call

   │ 

 8 │ let binding = array_d2.expect("Expect 2d matrix");

   │               ────┬───  

   │                   ╰───── value used here after move

   │ 

   │ Note: note: `Result::<T, E>::unwrap` takes ownership of the receiver `self`, which moves `array_d2`

───╯



[E0597] Error: `binding` does not live long enough

    ╭─[command_8:1:1]

    │

  8 │ let binding = array_d2.expect("Expect 2d matrix");

    │     ───┬───  

    │        ╰───── binding `binding` declared here

    │ 

 10 │ let array_d2t = binding.t();

    │                 ───┬─┬─────  

    │                    ╰───────── borrowed value does not live long enough

    │                      │       

    │                      ╰─────── argument requires that `binding` is borrowed for `'static`

────╯

Matrix Multiplication

#![allow(unused)]
fn main() {
use ndarray::{array, Array2};

let a: Array2<f64> = array![[3., 2.], [2., -2.]];
let b: Array2<f64> = array![[3., 2.], [2., -2.]];
let c = a.dot(&b);
print!("{:?}", c);

// Output
// [[13.0, 2.0],
//  [2.0, 8.0]], shape=[2, 2], strides=[2, 1], layout=Cc (0x5), const ndim=2

}

[[13.0, 2.0],

Linear Algebra with ndarray-linalg

• The crate ndarray-linalg implements more advanced lineary algebra operations that comes with NDArray.

• It relies on a native linear algebra library like OpenBLAS and can be tricky to configure.

• We show it here just for reference.

#![allow(unused)]
fn main() {
//```rust
:dep ndarray = { version = "^0.15" }
// This is the MAC version
// See ./README.md for setup instructions
:dep ndarray-linalg = { version = "^0.16", features = ["openblas-system"] }
// This is the linux verison
//:dep ndarray-linalg = { version = "^0.15", features = ["openblas"] }

use ndarray::array;
use ndarray_linalg::*;

{
    // Create a 2D square matrix (3x3)
    let matrix = array![
        [1.0, 2.0, 3.0],
        [0.0, 1.0, 4.0],
        [5.0, 6.0, 0.0]
    ];

    // Compute the eigenvalues and eigenvectors
    match matrix.eig() {
        Ok((eigenvalues, eigenvectors)) => {
            println!("Eigenvalues:\n{:?}", eigenvalues);
            println!("Eigenvectors:\n{:?}", eigenvectors);
        },
        Err(e) => println!("Failed to compute eigenvalues and eigenvectors: {:?}", e),
    }
}
//```
}

 [2.0, 8.0]], shape=[2, 2], strides=[2, 1], layout=Cc (0x5), const ndim=2EVCXR_VARIABLE_CHANGED_TYPE:b


The type of the variable b was redefined, so was lost.
The type of the variable array_d21 was redefined, so was lost.
The type of the variable array_d3 was redefined, so was lost.
The type of the variable a was redefined, so was lost.
The type of the variable range was redefined, so was lost.
The type of the variable eye was redefined, so was lost.
The type of the variable c was redefined, so was lost.
The type of the variable array_d1 was redefined, so was lost.
The type of the variable linspace was redefined, so was lost.
The type of the variable ones was redefined, so was lost.
The type of the variable b was redefined, so was lost.


Eigenvalues:
[Complex { re: 7.256022422687388, im: 0.0 }, Complex { re: -0.026352822204034426, im: 0.0 }, Complex { re: -5.229669600483354, im: 0.0 }], shape=[3], strides=[1], layout=CFcf (0xf), const ndim=1
Eigenvectors:
[[Complex { re: -0.4992701697014973, im: 0.0 }, Complex { re: -0.7576983872742932, im: 0.0 }, Complex { re: -0.22578016277159085, im: 0.0 }],
 [Complex { re: -0.4667420094775666, im: 0.0 }, Complex { re: 0.6321277120502754, im: 0.0 }, Complex { re: -0.526348454767688, im: 0.0 }],
 [Complex { re: -0.7299871192254567, im: 0.0 }, Complex { re: -0.162196515314045, im: 0.0 }, Complex { re: 0.8197442419819131, im: 0.0 }]], shape=[3, 3], strides=[1, 3], layout=Ff (0xa), const ndim=2





()

The same code is provided as a cargo project.

Python Numpy Reference

For reference you can look at mnist_fcn.ipynb which implements and trains the network with only numpy matrices, but does use PyTorch dataset loaders for conciseness.

Technical Coding Challenge

Coding Challenge

Coding Challenge Review