Neural networks can be used as a method for efficiently solving difficult partial differential equations. Recently this strategy has been dubbed physics-informed neural networks and has seen a resurgence because of its efficiency advantages over classical deep learning. Efficient implementations from recent papers are being explored as part of the NeuralPDE.jl package. The issue tracker contains links to papers which would be interesting new neural network based methods to implement and benchmark against classical techniques. Project work in this area includes:
Improved training strategies for PINNs.
Implementing new neural architectures that impose physical constraints like divergence-free criteria.
Demonstrating large-scale problems solved by PINN training.
Improving the speed and parallelization of PINN training routines.
This project is good for both software engineers interested in the field of scientific machine learning and those students who are interested in perusing graduate research in the field.
Recommended Skills: Background knowledge in numerical analysis and machine learning.
Expected Results: New neural network based solver methods.
Mentors: Chris Rackauckas and Kirill Zubov
Expected Project Size: 175 hour or 350 hour depending on the chosen subtasks.
Difficulty: Easy to Hard depending on the chosen subtasks.
Neural ordinary differential equations have been shown to be a way to use machine learning to learn differential equation models. Further improvements to the methodology, like universal differential equations have incorporated physical and biological knowledge into the system in order to make it a data and compute efficient learning method. However, there are many computational aspects left to explore. The purpose of this project is to enhance the universal differential equation approximation abilities of DiffEqFlux.jl, adding features like:
Improved adjoints for DAEs and SDEs
Various improvements to minibatching
Support for second order ODEs (i.e. symplectic integrators)
See the DiffEqFlux.jl issue tracker for full details.
This project is good for both software engineers interested in the field of scientific machine learning and those students who are interested in perusing graduate research in the field.
Recommended Skills: Background knowledge in numerical analysis and machine learning.
Expected Results: New and improved methods for neural and universal differential equations.
Mentors: Chris Rackauckas and Anas Abdelrehim
Expected Project Size: 175 hour or 350 hour depending on the chosen subtasks.
Difficulty: Medium to Hard depending on the chosen subtasks.
In many cases, when attempting to optimize a function f(p) each calculation of f is very expensive. For example, evaluating f may require solving a PDE or other applications of complex linear algebra. Thus, instead of always directly evaluating f, one can develop a surrogate model g which is approximately f by training on previous data collected from f evaluations. This technique of using a trained surrogate in place of the real function is called surrogate optimization and mixes techniques from machine learning to accelerate optimization.
Advanced techniques utilize radial basis functions and Gaussian processes in order to interpolate to new parameters to estimate f in areas which have not been sampled. Adaptive training techniques explore how to pick new areas to evaluate f to better hone in on global optima.
The purpose of this project is to further improve Surrogates.jl by: adding new surrogate models, adding new optimization techniques, showcasing compatibility with the SciML ecosystem and fixing unwanted behaviour with some current surrogate models.
Recommended Skills: Background knowledge of standard machine learning, statistical, or optimization techniques. Strong knowledge of numerical analysis is helpful but not required.
Expected Results: Improving Surrogates.jl with new surrogate models and new optimization techniques.
Mentors: Ludovico Bessi, Chris Rackauckas
Expected Project Size: 175 hour or 350 hour depending on the chosen subtasks.
Difficulty: Medium to Hard depending on the chosen subtasks.
Scientific machine learning requires mixing scientific computing libraries with machine learning. This blog post highlights how the tooling of Julia is fairly advanced in this field compared to alternatives such as Python, but one area that has not been completely worked out is integration of automatic differentiation with partial differential equations. FEniCS.jl is a wrapper to the FEniCS project for finite element solutions of partial differential equations. We would like to augment the Julia wrappers to allow for integration with Julia's automatic differentiation libraries like Zygote.jl by using dolfin-adjoint. This would require setting up this library for automatic installation for Julia users and writing adjoint passes which utilize this adjoint builder library. It would result in the first total integration between PDEs and neural networks.
Recommended Skills: A basic background in differential equations and Python. Having previous Julia knowledge is preferred but not strictly required.
Expected Results: Efficient and high-quality implementations of adjoints for Zygote.jl over FEniCS.jl functions.
Mentors: Chris Rackauckas
Expected Project Size: 350 hour.
Difficulty: Medium to Hard depending on the chosen subtasks.