Search results

Yeah it is really crazy how much power the PM1733 takes. CD6-Rs are more efficient on the read side but have half the performance of the samsung on the write. The new CD8s pull 25 watts like the pm1733 does

I was just looking at htop but I can easily run prometheus and drill into other metrics if that is helpful. But from what I was seeing in htop the cores were pinned with 128 jobs (the same number of threads I have) and got pretty high with 64 (might be worth it to get some data on that). More...

So I replied more in the other thread we were talking in but yeah, optane's price to performance isn't what I need. DBs are in the 4tb range and I have many aiming to have about 20TB of space on each node. I'd need a lot of optane drives to get to where I need to be so normal nvme drives are a...

@i386 - Thanks for feedback. So I am not sure what queue depth most aligns with my application but that is a very good question and hardest to answer. Benchmarking the application on different drive configurations is very complicated since the data is large so I am just assuming 4k rand r/w is...

@NPS - So I didn't consider optane because my DBs are about 4TB and I have many of them hoping to store 5 of them on a node so looking for about 20TB of storage. The queries are highly async, small queries but huge volumes of them, so latency doesn't have much of an impact. Was really just...

I am trying to get the most 4k iops I can get out of a raid array for and experimenting with different configurations to try to get a sense of where the sweet spot is in terms of number of drives, performance, and raid config. I was hoping to see more linear performance gains with number of...

@Advin - I really hope I don't have to deal with that. Been running tests and restarting machines a lot and no issues thus far. I'll remember this though but unfortunately money has been spent and these guys need to go into a data center. Thanks again for your comment though.

@hmw, @DaveLTX, @rtech, @ano As a followup to this, I ended up getting RS500A-E11-RS12U with a couple different processors, mostly 32 core, one 24, and one 64 epycs. With the epyc 7532 (200w tdp) and 500gb of LR ram (8*64), and 12 pm9a3 3.84 tb drives running a fio test with all cores pinned I...

Yeah this is going to be in production - no need to take the risk. Total memory capacity is more important. Anyways, main concern was using slower memory and have done several passes in memtest now so all is good.

@zachj - I think you are totally right. That was my thought going into this but then I saw a lot of people doing it saying it was fine up to 2 steps above. Anyways, not worth it really.

@Advin - Not going to lie, your post freeked me out as I had already bought 3tb of samsung 64gb lrdimms - M386A8K40BM1 which is 4rx4 but so far after about 10 hours of memtesting have not seen an issue. Going to do another day or so of testing but I think I should be ok. I'm running on an asus...

@Advin - Why is that? Already got 2400 lrdimms. About to start testing and was hoping to overclock 1 step. Memory bandwidth / compute is not a bottleneck in my application so not super concerned.

@gb00s - Thank you for your insight.

Thanks @ano - What if I am eventually planning on running 2dpc (ie 16 dimms) at 2133? Do you think there could be some instability there?

@bayleyw - Thank you for your insight.

I am wondering about memory speed compatibility with epyc 7003/7002 server I am getting which only lists 3200 as compatible. Server is a Asus RS500A-E11-RS12U and due to budget reasons I am wanting to fill it with 2400 ram vs 3200 ($60 vs $110 per stick) as the memory performance is not...

@hmw - I'll report the power draw when I start putting these together in this thread. Will be about 1.5 months though as I am traveling for the next month. Ultimately now leaning towards 1u's now as they are cheaper and would rather have two smaller clusters vs 1 bigger one. Building on a...

@hmw - Thank you so much again for your help. So it doesn't look like Asrock supports power cap policies / monitoring like iDRAC based on this manual from 2018 - https://download.asrock.com/TSD/SMU/Manual/ASRock%20Rack%20Server%20Management_v1.0.3.pdf I think I'll be alright though if I just...

@hmw Thank you very much for the detailed response. Truly appreciated. Regarding power, indeed I am limited but really just trying to balance colocation costs with hardware costs. Also just in planning and budgeting phases so it might just turn out that I go for more expensive racks / higher...

@rtech - Sorry but is it possible to give a rough estimate of the additional power the additional 10 drives and ram would give? My math doesn't smell right. 10 drives * 12W per drive is another 120W - This seems very high 16*64gb lrdimm vs the STH article's 8*32gb - Hard to estimate that but...