Ivan Yashchuk

The Summer of Code is coming to an end. 26th of August is the deadline for participants to submit their final report. The summary of the project, including all submitted PRs, is available here. As a result of this GSoC project, cholesky, qr, solve, inv, svd, pinv, eigh, eigvalsh functions were implemented for ChainerX. All of them (except eigvalsh) were made differentiable. Most of the changes are in master branch already and available both using C++ API and in chainerx. [Read More]

I have run some performance tests to compare ChainerX implementation against NumPy and CuPy. Some new chainerx.linalg routines run more slowly compared to numpy.linalg and cupy.linalg. Performance results & code to obtain them is available in this gist. The most probable reason for slowness of qr, solve, inv is allocating temporary arrays in the kernel and copying data from them to routine level arrays. Also repeated transpose-copy operations (because of ChainerX’s row-major vs column-major ordering of LAPACK/cuSOLVER) have an effect. [Read More]

It’s time for the second evaluation and so far everything goes very smoothly. New linear algebra routines are becoming to be merged and others are soon to be in this stage. Pull request that adds Solve & Inverse (#7474) functions has been merged! SVD, QR PRs are ready for review! TODOs after existing PRs: Tutorial (and docs) Support mixed/other dtypes such as fp16 Possibly cover more routines under numpy.linalg such as det Benchmark and summarize differences with other alternative implementations NumPy, CuPy, etc. [Read More]

Me and my mentor Vincent have a face-to-face meeting every Wednesday over videochat. Last week our weekly meeting was in person. I’ve visited Preferred Networks office (that’s the company behind Chainer) and met a few developers there and we had lunch together. They even have a little space with Tatami (japanese mat) in the common area! Unfortunately, I forgot to take pictures. For the past days I was mostly focused on polishing existing PRs according to the code review. [Read More]

Linear algebra routines from this issue are now working both on CPU and GPU. When it comes to the choice of linear algebra library on CPU, the variety is much broader than on GPU. However, we don’t need a full featured linear algebra matrix library like Eigen, or Armadillo, as ChainerX has its own ndarray. Therefore, it was decided to use one of the LAPACK implementations. For now it seems that working with LAPACK (which is written in Fortran) directly without C wrappers is the way to go. [Read More]

Already two weeks of coding have gone past! My progress can be tracked in this issue. Part of the project is making LAPACK routines accesible from ChainerX performing computations in CPU and GPU. Another part is making them differentiable. LAPACK (Linear Algebra Package) is a standard software library for numerical linear algebra. It provides routines for solving systems of linear equations and linear least squares, eigenvalue problems, and singular value decomposition. [Read More]

Coding period start soon!

This summer I am going to work on extending ChainerX with some Linear Algebra routines. The goal is to implement common routines that are described at numpy.linalg. And their sensitivities for reverse mode automatic differentiation has to implemented as well!