Currently, it is not possible to specify Rayleigh damping based on a modal analysis programmatically. You will have to define and run a modal analysis first and then use the frequency results to define the Rayleigh damping for the subsequent transient analysis. The Rayleigh damping in OSN is specified when you define the analysis case. You can assign damping to a model as regions defined by nodes or elements.