In March 2018 the Google Brain team announced that they had branched Apple's Swift compiler for use with TensorFlow. A lengthy article discussing that decision is here. Previously most programming against TensorFlow had been done in Python, so this effort represents a significant step both for TensorFlow and for Swift.
Swift for TensorFlow includes Swift wrapper classes for working with the library, as well as enhancements to the Swift compiler and language. The project's main goal is to provide improved usability for machine learning developers working with TensorFlow.
These notes describe introductory steps for doing something similar for Pascal, although stopping short of modifying the Free Pascal compiler and language.
The TensorFlow dynamic libraries are easy to extract anywhere on your system, including the default /usr/local/lib location. See the readme.txt file included in the previous step for links to download the library binaries.
This is the Pascal version of the inference.swift program given in the "Interpreter" section here. It demonstrates how to do a simple classification with TensorFlow. It also shows how verbose Pascal is relative to Swift, although this difference may not be as noticeable in a non-trivial program.
This is the Pascal version of the MNIST.swift program given here. This program uses the classic MNIST dataset of 60,000 images of handwritten numeric digits (0-9) collected from census workers and high school students. It uses the dataset to train a neural network to predict a scanned image's digit.
A useful, detailed discussion of the Swift code is given in part 5 here.
The parsed C headers are included as unit TensorFlow.pas, but this is a clumsy way of working with TensorFlow in Pascal. As a result, unit TF.pas is also included, which wraps TensorFlow.pas in class TTensor.
Like Swift's Tensor, TTensor wraps the newer "eager execution" API.
Any kind of multi-dimensional tensor can be created, but in practice tensors are often scalar (rank 0), vector (rank 1, or 1 dimension), or simple matrix (rank 2, or 2 dimensions). Tensors can also have almost any numeric data type. But again, in practice, tensors are often created for just a few standard numeric types. As a result, the TTensor class provides convenience constructors for easy creation of tensors of rank 0, 1 or 2 from arrays of type Single and other common Pascal types.
For example, Swift code like this that creates a tensor of type Float and rank 2 from an array of values:
Can be written in Pascal like this:
var w2 = Tensor<Float>(shape: [4, 1], scalars: [0.4, -0.5, -0.5, 0.4])
w2 := TTensor.CreateSingle([4, 1], [0.4, -0.5, -0.5, 0.4]);
Swift for TensorFlow includes functions for all supported tensor operations. These are generated automatically for each release. Something similar could be done for Pascal by parsing the ops.pbtxt file and generating the function code, but that's for a future effort.
In the meantime, the TTensor.ExecOp method can be used to execute many of the supported operations. For example, to do matrix multiplication on two tensors in Swift, you can do this:
For now, do it like this in Pascal:
These two Pascal tensor expressions are equivalent:
t1 + t2 t1.ExecOp('Add', t2)
Take this example line of Swift code:
This code does matrix multiplication of tensors x and w1, then adds tensor b1 to the result, calculates the hyperbolic tangent of that result, and finally assigns the full expression's tensor value to o1.
let o1 = tanh(matmul(x, w1) + b1)
Note that in Swift, the o1 tensor variable does not need to be declared previously and tensors are freed automatically when they go out of scope.
As a first cut at a Pascal version of this expression, you could write something like this to ensure it doesn't leak memory:
Recognizing that intermediate tensors created by an expression are temporary and that all operations, including overloaded operators, go through the ExecOp method, the TTensor class includes a handy Temp function. This Temp function sets the TTensor instance's IsTemp flag and returns the instance. ExecOp then knows whether to call Free on a tensor input. This allows the code to be simplified to this:
try temp1 := x.ExecOp('MatMul', w1); temp2 := temp1 + b1; o1 := temp2.ExecOp('Tanh'); finally temp1.Free; temp2.Free; o1.Free; end;
Note that this also eliminates the need to declare temp1 and temp2 as variables.
try o1 := (x.ExecOp('MatMul', w1).Temp + b1).Temp.ExecOp('Tanh'); finally o1.Free; end;
Furthermore, if o1 will always be used in subsequent expressions, its last occurrence could be referenced as o1.Temp, thus eliminating the o1.Free line. This might also allow for the elimination of the try/finally lines required here for memory cleanup.
macpgmr (at) icloud (dot) com
First posted June 22, 2018.
Code syntax highlighting done with highlight.js.