[Cosmo-dev] 32Gb and no clue

[Jared Rhine]

How would you tune a 32Gb RAM box running one Cosmo instance and MySQL?

Say, hypothetically, you had a big honking server you wanted to deploy a 
large instance of Cosmo onto it; maybe call it Chandler Hub.

There's a whole art to tuning Java VMs; I am admittedly deficient in 
this art.  I can spend weeks/months googling for the finer points, but I 
was hoping to get some insight here.

Of course, the settings are also JVM-dependent; we're intending a 
JRockit deployment, so I'll use their names for things.

I assume we need to decide on the following parameters for Hub:
* Minimum heap size (initial probably set maximum heap)
* Maximum heap size (has a 1Gb max out of the box?)
* Nursery size (less than max heap of course>
* -server (decision made, I assume)
* -Xverbose:memory (show paging)
* -Xgcpause (document collection pause durations)
* Garbage thread priority (parallel, throughput, gencon?)
* Thread stack size (defaults to 320KB)
* Any other garbage collection settings
* Any other general settings?

My understanding is that setting the heap very large (20Gb) will result 
in long pauses during collection, which we want to avoid.  (Java is 
probably going to be conservative in its memory use, with most of the 
surplus going to MySQL.)

We apparently could also go with a dynamic GC setting by setting a 
"-Xpausetarget" of a specific time say 300ms.

Of course many of these options are interdependent.

On doing a little reading, I guess I remember more about this than I 
thought, but are there any books people have read with good tuning sections?

Back in cosmo-demo days, we used:

-Xns:500m -Xms:700m -Xmx:700m -server -Xgcpause -Xgcreport -Xverbose:memory

It just occurred to me that we could do another "Java Runtime Analyzer" 
run; we did that back in the cosmo-demo days to try to track down some 
performance bugs, but maybe there's some GC/heap info in it too.

There's another thread needed about MySQL tuning, but that one might be 
a little more straightforward.

-- Jared