Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms · HackerPapers discussions

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms — mentioned by adgjlsfhk1

Fri, 30 Aug 2024 14:17:47 GMT

One of the really nice things about Julia for GPU programming is that you can write your own kernels. CUDA.jl isn't just C kernels. This is why (for example) DiffEQGPU.jl is able to be a lot faster than the other GPU based ODE solvers (see https://arxiv.org/abs/2304.06835 for details).

Paper and discussion history. A mention is not an endorsement.

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms — mentioned by ChrisRackauckas

Tue, 26 Mar 2024 12:00:05 GMT

No, there are two types of GPU usage: the ensemble form and the array form. In the array form, you simply make your state variable a GPU and can run all internal operations on the GPU. This is used for GPU acceleration of `f` functions where you have a large state and a relatively "regular" function in `f`, such as neural networks or PDE discretizations. Thus you can see this in action for example on things like the DeepEquilibriumNetworks.jl https://docs.sciml.ai/DeepEquilibriumNetworks/stable/tutoria... use cases, where it's big neural networks with nonlinear solvers in them. The other use…

Paper and discussion history. A mention is not an endorsement.

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms — mentioned by ChrisRackauckas

Fri, 29 Dec 2023 23:40:39 GMT

Indeed, and this year we created a system for compiling ODE code not just optimized CUDA kernels but also OneAPI kernels, AMD GPU kernels, and Metal. Peer reviewed version is here (https://www.sciencedirect.com/science/article/abs/pii/S00457...), open access is here (https://arxiv.org/abs/2304.06835), and the open source code is at https://github.com/SciML/DiffEqGPU.jl. The key that the paper describes is that in this case kernel generation is about 20x-100x faster than PyTorch and Jax (see the Jax compilation in multiple ways in this notebook…

Paper and discussion history. A mention is not an endorsement.

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms — mentioned by e12e

Sun, 24 Dec 2023 08:02:06 GMT

https://ar5iv.org/abs/2304.06835 Thank you! I guess that's what: > If you are familiar with ar5iv, an arXivLabs collaboration, our HTML offering is essentially bringing this impactful project fully “in-house”. Our ultimate goal is to backfill arXiv’s entire corpus so that every paper will have an HTML version, but for now this feature is reserved for new papers. Refer to.

Paper and discussion history. A mention is not an endorsement.

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms — HN submission

Sat, 23 Dec 2023 22:45:41 GMT

Translation and accelerated solving of differential equations on GPU platforms

112 points; 32 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Automated Translation and Accelerated Solving of Differential Equations on Multiple GPU Platforms — mentioned by ChrisRackauckas

Thu, 25 May 2023 15:39:38 GMT

We recently showed DiffEqGPU.jl generating customized ODE solver kernels for NVIDIA CUDA, AMD GPUs, Intel OneAPI, and Apple Metal, where for CUDA it matches state of the art (MPGOS) which is about 10x-100x something like Jax/PyTorch (where the performance difference comes from inefficiencies of using vmap vs actually writing and calling a kernel). It's all in https://arxiv.org/abs/2304.06835. So this stuff exists and people are using it. Of course the caveat here is this is the context of engineering applications so someone would need to do similar for LLMs to fully relate back to the…

Paper and discussion history. A mention is not an endorsement.