To figure it out experimentally, for a stepper machine increase the acceleration until the motor stalls, then reduce by a safety margin. I'd say at least 30% but depending on the machine 60% might be more appropriate. The test will need to be done at or near the max velocity, because that is where a stepper is weakest.
On a servo with feed back, it is simpler, set the f-error high for testing and the acceleration to something way too high. Do a test move watching the velocity feedback in halscope. The slope of that will be the max possible acceleration. Set the max acceleration to something less, perhaps 10%-20% less.
It might be wise to load up the moving joint with the max theoretical load for testing. (put something heavy on the table, load up your biggest tools...) and make sure the machine is bolted down good, this could be quite violent if the machine is powerful. (read this as it may not be safe.) Fortunately on a servo the speed doesn't need to be high or the move large to find the limit. so make the moves small and not particularly fast, just enough to see linear acceleration.
More importantly learn how to use G64 Pn in you g-code to get the path following tolerance you need.