7900xtx matches 4090 in Stable Diffusion as well. And I think it’s also quite competitive in LLMs. Since LLMs are memory capacity and bandwidth bound, and 7900xtx and 4090 are close there.
RDNA3 is lacking in AI performance today, but there’s no real reason to believe it can not compete if given billions for software development. The specs are there, but the software (in comparison to NVIDIA) is in a laughable state. For now.
The law only restricted raw FLOPs, so it has to be that. But the law has a chiplet subclause so it might be there’s some interaction there that pushes the AMD gpus over the edge.
the 4070 ti at 294mm2 (full ad104) with 160 Tflops of Fp16
The 7900xtx GCD is 300 mm2 (Full Navi31 GCD only) with 122 tflops of Fp16
Doubt its that.
Where there might be reasons is that RDNA doesnt hasve AI cores. The tasks are accelerated on the shader cores.Hence the term AI Accelarators. Now assumming nvidia cards ignore the tensor cores.
The 4090 can do only 82.6 Tflop of FP16 (Non-Tensor).
The 7900xtx would still retain its 122 tflops of FP16. making it faster in Fp16 performance.
The actual rule has hard numbers, no need to speculate. And it’s no more than 300 TFLOPS of fp16 (or 150 fp32, 600 fp8, etc) so it ain’t TFLOPS that are the culprit. As for performance density, it’s equivalent to those figures at an 830mm^2 die, so again not that.
Ok I didn’t know the actual numbers that’s helpful. Maybe they’re just holding off to apply for an export license? I heard the 4090 is in a “gray area”.
AMD has had traidionally very competitive FLOPs with their shaders. The issue is that their software stack, for lack of a better word is; shit.
For specific customers, like national labs or research institutions, they can afford to pay a bunch of poor bastards to develop some of the compute kernels using the shitty tools. Because at the end of the day, most of their expenses are in terms of electricity and hardware, with salaries not being the critical cost for some of these projects. I.e. grad students are cheap!
However, when it comes to industry, things are a bit difference. First off, nobody is going to take a risk w a platform with little momentum behind it. Also they need to have access to talent pool that can develop and get the applications up and running as soon as possible. Under those scenarios, salaries (i.e. the people developing the tools) tend to be almost as important consideration as the HW. So you go with the vendor that gives you the biggest bang for your buck in terms of performance and time to market. And that is where CUDA wins hands down.
At this point AMD is just too behind, at least to get significant traction in industry.
I know that the xtx kept up with the 4090 in stable diffusion before the tensorRT update, so there might be some places where the xtx can be a replacement when you build software from the grounds up and willing to lose performance for the benefit of less eyes and hassle on Amd products
This is no longer true.
If you use NV’s TensorRT plugin with the A1111 development branch, TensorRT works very well with SDXL (it’s actually much less painful to use than SD1.5 TensorRT was initially).
The big constraint is VRAM capacity. I can use it for 1024x1024 (and similar-total-pixel-count) SDXL generations on my 4090, but can’t go much beyond that without tiling (though that is generally what you do anyway for larger resolutions).
Just like for SD1.5, TensorRT speeds up generation by almost a factor of 2 for SDXL (compared to an “optimized” baseline using SDP).
It mentions Olive. I don’t know what that is, but it’s suggesting it could cause AMD to catch back up. Is that true? Or is it more likely going to get them an extra 10% performance instead of the extra 110% they need to catch up?
Surely it cant be due to be AI performance?
The 7900xt is 103 tlops of fp16, 7900xtx is 122.
the 4070 is at 117 of fp16 (234 using sparsity) on a smaller chip and thats not banned.
The only thing 7900XTX/ W7900 beat the 4090 in is RAW video debayering in DaVinci Resolve that I’m aware of
7900xtx matches 4090 in Stable Diffusion as well. And I think it’s also quite competitive in LLMs. Since LLMs are memory capacity and bandwidth bound, and 7900xtx and 4090 are close there.
https://www.pugetsystems.com/labs/articles/stable-diffusion-performance-nvidia-geforce-vs-amd-radeon/
The 7900XTX beats a 4090 in Topaz AI, at least in benchmarks
Used to… But not anymore in the latest versions.
RDNA3 is lacking in AI performance today, but there’s no real reason to believe it can not compete if given billions for software development. The specs are there, but the software (in comparison to NVIDIA) is in a laughable state. For now.
People also have the misconception that cuda is the only software advantage.
Their AI foundries and AI Enterprise. are their biggest AI software and support.
Jensen at Microsoft Ignite told Satya that they want be the TSMC of AI.
Just like cpu/gpu makers use tsmc foundries to make chips,
Companies will use Nvidia foundries like Nemo, bionemo, picaso, etc to make AI models.
In addition there is their Omniverse and DGX Cloud.
DGX cloud even allows them to straight up bypass any restrictions and let chinese customers use Hopper chips remotely.
That’s just a pipedream. They are peacocking and it’s obviously failing since Microsoft shat directly in their face with Maia.
Nvidia’s ecosystem advantages will only diminish over the years since Microsoft, Google and Amazon etc will develop their own.
This is Glide vs Direct3D all over again. You know which one won.
You do know they have already started it right?
Adobe for example uses Nvidia Foundry for their AI Foundry.
They have been building these foundries for years now. Before even ai got popular and Microsoft jumped on Open AI
Their current balance sheets seem to indicate otherwise…
The law only restricted raw FLOPs, so it has to be that. But the law has a chiplet subclause so it might be there’s some interaction there that pushes the AMD gpus over the edge.
the 4070 ti at 294mm2 (full ad104) with 160 Tflops of Fp16
The 7900xtx GCD is 300 mm2 (Full Navi31 GCD only) with 122 tflops of Fp16
Doubt its that.
Where there might be reasons is that RDNA doesnt hasve AI cores. The tasks are accelerated on the shader cores.Hence the term AI Accelarators. Now assumming nvidia cards ignore the tensor cores.
The 4090 can do only 82.6 Tflop of FP16 (Non-Tensor).
The 7900xtx would still retain its 122 tflops of FP16. making it faster in Fp16 performance.
doesn’t RDNA3 have WAVA MMA or Wave Matrix Multiply Accumulate which is their AI cores?
It has the instruction sets in the compute units
They are called AI accelerators for that reason.
Not Ai cores.
The actual Matrix “Cores” , i.e. dedicated silicon, are on the instinct series
No. Tensor cores have seperate specialised matrix ALUs, AMD’s WMMA are instructions on existing shader ALUs.
Tensor cores can process AI tasks in parallel to CUDA cores, RDNA3 can’t do both on the same CU.
The actual rule has hard numbers, no need to speculate. And it’s no more than 300 TFLOPS of fp16 (or 150 fp32, 600 fp8, etc) so it ain’t TFLOPS that are the culprit. As for performance density, it’s equivalent to those figures at an 830mm^2 die, so again not that.
Ok I didn’t know the actual numbers that’s helpful. Maybe they’re just holding off to apply for an export license? I heard the 4090 is in a “gray area”.
No gray area, at base clocks the 4090 exceeds the limit by 10% already.
Apparently the AMD significantly outperforms Nvidia in specific calculations used for nuclear weapons simulation software.
Most people dont realize this is the original justification for the bans.
AMD has had traidionally very competitive FLOPs with their shaders. The issue is that their software stack, for lack of a better word is; shit.
For specific customers, like national labs or research institutions, they can afford to pay a bunch of poor bastards to develop some of the compute kernels using the shitty tools. Because at the end of the day, most of their expenses are in terms of electricity and hardware, with salaries not being the critical cost for some of these projects. I.e. grad students are cheap!
However, when it comes to industry, things are a bit difference. First off, nobody is going to take a risk w a platform with little momentum behind it. Also they need to have access to talent pool that can develop and get the applications up and running as soon as possible. Under those scenarios, salaries (i.e. the people developing the tools) tend to be almost as important consideration as the HW. So you go with the vendor that gives you the biggest bang for your buck in terms of performance and time to market. And that is where CUDA wins hands down.
At this point AMD is just too behind, at least to get significant traction in industry.
Amd is better at fp32 and FP64
During 2017 ish Nvidia and Amd focused on different parts with data centre cards.
Amd went in on Compute with fp32 and fp64.
Nvidia went full in on AI with Tensor cores and fp16 performance.
Amd got faster than Nvidia in some tasks. But Nvidia’s bet on AI is the clear winner.
Not FP32, MI300 has 48 TFLOPS, H100 has 60TFLOPs
https://www.topcpu.net/en/cpu/radeon-instinct-mi300
https://www.nvidia.com/en-us/data-center/h100/#:~:text=H100 triples the floating-point,of FP64 computing for HPC.
AMD FP64 still gaps Nvidia who in turn gap FP16
Nobody knows the actual flops of the mi300
The mi250x had 95.7 tflops of fp32 due the matrix cores
https://www.amd.com/en/products/server-accelerators/instinct-mi250x
That’s more than the H100 even
I know that the xtx kept up with the 4090 in stable diffusion before the tensorRT update, so there might be some places where the xtx can be a replacement when you build software from the grounds up and willing to lose performance for the benefit of less eyes and hassle on Amd products
Got a source for that keeping up?
https://www.pugetsystems.com/labs/articles/stable-diffusion-performance-nvidia-geforce-vs-amd-radeon/
That is, unfortunately, sorely outdated. Particularly with the advent of tensorRT. Best case vs best case the 4080 is about twice as fast today
https://www.tomshardware.com/pc-components/gpus/stable-diffusion-benchmarks#section-stable-diffusion-512x512-performance
The gap would be even larger if, or to be precise WHEN, Fp8 and/or sparisity will be used on the Ada Lovelace cards.
Of note, TensorRT doesn’t support SDXL yet.
This is no longer true.
If you use NV’s TensorRT plugin with the A1111 development branch, TensorRT works very well with SDXL (it’s actually much less painful to use than SD1.5 TensorRT was initially).
The big constraint is VRAM capacity. I can use it for 1024x1024 (and similar-total-pixel-count) SDXL generations on my 4090, but can’t go much beyond that without tiling (though that is generally what you do anyway for larger resolutions).
Just like for SD1.5, TensorRT speeds up generation by almost a factor of 2 for SDXL (compared to an “optimized” baseline using SDP).
Alright thanks. This stuff is moving very fast, and I was only looking at the master branch.
You cant compare using using two different impelementations. You compare only on A1111 or only on SHARK.
SHARK doesnt even seem be taking any adavantage of the 4090 being significatly slower than the 7900xtx.
The recent A1111 Olive branch made the performance of it almost equal SHARK model. A1111 also full uses the 4090.
The new results on the same A1111 implention are here -
https://www.pugetsystems.com/labs/articles/amd-microsoft-olive-optimizations-for-stable-diffusion-performance-analysis/
You can divide the 4090’s perf by half if you want no Tensor RT which is 35. Thats still significantly higher than the 7900xtx’s 23
It mentions Olive. I don’t know what that is, but it’s suggesting it could cause AMD to catch back up. Is that true? Or is it more likely going to get them an extra 10% performance instead of the extra 110% they need to catch up?
That’s seems like an arbitrary handicap. You should use whichever solution runs best on the respective hardware.