SciML Scientific Machine Learning Software

Once again we stayed true to form and didn't solve the problems in the development list but adding a ton of new features anyways. Now that Google Summer of Code (GSoC) is in full force, a lot of these updates are due to our very awesome and productive students. Here's what we got.

Approximate Bayesian Computation (ABC)

Marc Williams (@marcjwilliams1) contributed the abc_inference function to DiffEqBayes.jl which utilizes approximate Bayesian computation (also known as ABC) from ApproxBayes.jl to perform the estimation of parameter posteriors. Compared to standard Bayesian methods, ABC is computationally cheaper and faster at the cost of making some approximations (the name is quite appropriate). For models with lots of parameters where you have a good prior guess for the posterior parameter point estimates, ABC can be a great way to get posterior distributions.

Maximum A Posteriori Estimation

GSoC student Vaibhav Dixit (@Vaibhavdixit02) added maximum a posteriori estimation to our existing parameter estimation routines in DiffEqParamEstim.jl. Now these optimization-based methods can take into account prior distributions which can help global optimizers stay in the right parameter ranges and improve fitting. All it takes is passing an optional prior distribution to any of the existing methods, so it's very easy to add it your current work!

Multiple shooting objective

In other parameter estimation news, Vaibhav Dixit (@Vaibhavdixit02) added the multiple_shooting_objective which, like the build_loss_objective method, allows for fitting ODE results to any loss function. However, multiple shooting methods are naturally more robust by solving simultaneously from many different time points.

DynamicSS Solver

For SteadyStateProblem types, a new solver DynamicSS has been added that utilizes the ODE solvers to find steady states. It builds in a callback that will halt when the derivative is sufficiently small, allowing it to be a robust automated steady-state finding machine.

Unified retcode handling

Takafumi Arakaki (@tkf) performed an awesome refactoring in the DiffEq core which allows all of the DiffEq solvers to utilize the same retcode handling code. This means that we have unified exit warning messages and the resulting retcodes. Each package should act the same for the common retcodes.

Fixed order variable time step Adams methods

Shubham Maddhashiya (@sipah00) contributed a variety of new adaptive Adams-Bashforth and Adams-Bashforth-Moulton methods. These methods are fixed order 3-5 which minimize function evaluations and are designed for large non-stiff ODE discretizations. They utilize Runge-Kutta methods to hotstart the Adams methods, making them more efficient than variable order versions when only lower orders are required or when there are a lot of events.


Yingbo Ma (@YingboMa) contributed ABDF2, an adaptive implementation of the Backwards Differentiation Formula (BDF) order 2 method. BDF schemes are the go-to choice for PDE discretizations since they only have one step (one nonlinear equation to solve) for each time step, minimizing the number of function calculations. Unlike adaptive order BDF discretizations, this adaptive BDF2 is A-B-L-stable, meaning that it should be stable for any stiff ODE that is thrown at it. This makes it a nice testing or anchor method: one that can always be relied on to do well-enough.

In development

A lot of the next developments will come from our GSoC students. Here's a list of things we are aiming for:

And here's a quick view of the rest of our "in development" list:


Are you a student who is interested in working on differential equations software and modeling? If so, please get in touch with us since we may have some funding after August for some student developers to contribute towards some related goals. It's not guaranteed yet, but getting in touch never hurts!