Top | ![]() |
![]() |
![]() |
![]() |
Functions
Properties
glong | burnin | Read / Write / Construct |
gchar * | filename | Read / Write / Construct |
NcmMSet * | mset | Read / Write / Construct Only |
GStrv | nadd-val-names | Read / Write / Construct Only |
GStrv | nadd-val-symbols | Read / Write / Construct Only |
guint | nadd-vals | Read / Write / Construct Only |
guint | nchains | Read / Write / Construct Only |
gboolean | read-only | Read / Write / Construct Only |
NcmRNG * | rng | Read / Write |
gchar * | run-type-string | Read / Write / Construct |
NcmMSetCatalogSync | smode | Read / Write / Construct |
gdouble | sync-interval | Read / Write / Construct |
gboolean | weighted | Read / Write / Construct Only |
Types and Values
enum | NcmMSetCatalogSync |
#define | NCM_MSET_CATALOG_EXTNAME |
#define | NCM_MSET_CATALOG_M2LNL_COLNAME |
#define | NCM_MSET_CATALOG_M2LNL_SYMBOL |
#define | NCM_MSET_CATALOG_FIRST_ID_LABEL |
#define | NCM_MSET_CATALOG_RNG_ALGO_LABEL |
#define | NCM_MSET_CATALOG_RNG_SEED_LABEL |
#define | NCM_MSET_CATALOG_RNG_STAT_LABEL |
#define | NCM_MSET_CATALOG_RNG_INIS_LABEL |
#define | NCM_MSET_CATALOG_NROWS_LABEL |
#define | NCM_MSET_CATALOG_RTYPE_LABEL |
#define | NCM_MSET_CATALOG_NCHAINS_LABEL |
#define | NCM_MSET_CATALOG_NADDVAL_LABEL |
#define | NCM_MSET_CATALOG_WEIGHTED_LABEL |
#define | NCM_MSET_CATALOG_RTYPE_BSTRAP_MEAN |
#define | NCM_MSET_CATALOG_RTYPE_UNDEFINED |
#define | NCM_MSET_CATALOG_FSYMB_LABEL |
#define | NCM_MSET_CATALOG_ASYMB_LABEL |
#define | NCM_MSET_CATALOG_DIST_EST_SD_SCALE |
Description
This class defines a catalog type object. This object can automatically synchronize with a fits file (thought cfitsio).
For Mote Carlo studies, like resampling from a fiducial model or bootstrap, it is used to save the best-fitting values of each realization. Since the order of the resampling is important, due to the fact that we use the same pseudo-random number generator for all resamplings, this object also guarantees the order of the samples added.
For Markov Chain Monte Carlo (MCMC) this object saves the value of the same likelihood in different points of the parameter space.
For both applications this object keeps an interactive mean and variance of the parameters added, this allows a sample by sample analyses of the convergence. Some MCMC convergence diagnostic functions are also implemented here.
Functions
ncm_mset_catalog_new ()
NcmMSetCatalog * ncm_mset_catalog_new (NcmMSet *mset
,guint nadd_vals
,guint nchains
,gboolean weighted
,...
);
Creates a new NcmMSetCatalog based on the NcmFit object fit
. The catalog assumes that
the fit
object will remain with the same set of free parameters during its whole lifetime.
If nchains
is larger than one, the catalog will keep track of the statistics of each chain
separately.
Parameters
mset |
a NcmMSet |
|
nadd_vals |
number of additional values |
|
nchains |
number of different chains in the catalog (>=1) |
|
weighted |
set to TRUE whenever the catalog is weighted |
|
... |
additional values name/symbol pairs |
ncm_mset_catalog_new_array ()
NcmMSetCatalog * ncm_mset_catalog_new_array (NcmMSet *mset
,guint nadd_vals
,guint nchains
,gboolean weighted
,gchar **names
,gchar **symbols
);
Creates a new NcmMSetCatalog based on the NcmFit object fit
. The catalog assumes that
the fit
object will remain with the same set of free parameters during its whole lifetime.
If nchains
is larger than one, the catalog will keep track of the statistics of each chain
separately.
Parameters
mset |
a NcmMSet |
|
nadd_vals |
number of additional values |
|
nchains |
number of different chains in the catalog (>=1) |
|
weighted |
set to TRUE whenever the catalog is weighted |
|
names |
additional values name NULL-terminated array. |
[array zero-terminated=1] |
symbols |
additional values symbol NULL-terminated array. |
[array zero-terminated=1] |
ncm_mset_catalog_new_from_file ()
NcmMSetCatalog * ncm_mset_catalog_new_from_file (const gchar *filename
,glong burnin
);
Creates a new NcmMSetCatalog from the catalog in the file file
.
It will use also the mset file (same name but with .mset extension).
ncm_mset_catalog_new_from_file_ro ()
NcmMSetCatalog * ncm_mset_catalog_new_from_file_ro (const gchar *filename
,glong burnin
);
Creates a new NcmMSetCatalog from the catalog in the file file
.
The file
is opened in a read-only fashion.
It will use also the mset file (same name but with .mset extension).
ncm_mset_catalog_ref ()
NcmMSetCatalog *
ncm_mset_catalog_ref (NcmMSetCatalog *mcat
);
Increases the reference count of mcat
atomically.
ncm_mset_catalog_free ()
void
ncm_mset_catalog_free (NcmMSetCatalog *mcat
);
Decreases the reference count of mcat
atomically.
ncm_mset_catalog_clear ()
void
ncm_mset_catalog_clear (NcmMSetCatalog **mcat
);
Decrese the reference count of *mcat
atomically and sets the pointer *mcat
to null.
ncm_mset_catalog_set_file ()
void ncm_mset_catalog_set_file (NcmMSetCatalog *mcat
,const gchar *filename
);
Sets the data filename to be used to sync/save data.
ncm_mset_catalog_set_sync_mode ()
void ncm_mset_catalog_set_sync_mode (NcmMSetCatalog *mcat
,NcmMSetCatalogSync smode
);
Sets the sync mode to smode
.
ncm_mset_catalog_set_sync_interval ()
void ncm_mset_catalog_set_sync_interval (NcmMSetCatalog *mcat
,gdouble interval
);
Sets the minimum time interval between syncs.
ncm_mset_catalog_set_first_id ()
void ncm_mset_catalog_set_first_id (NcmMSetCatalog *mcat
,gint first_id
);
Sets the first id of the catalog, mainly used to inform in which realization the catalog starts.
ncm_mset_catalog_set_run_type ()
void ncm_mset_catalog_set_run_type (NcmMSetCatalog *mcat
,const gchar *rtype_str
);
Sets the run type string.
ncm_mset_catalog_set_rng ()
void ncm_mset_catalog_set_rng (NcmMSetCatalog *mcat
,NcmRNG *rng
);
Sets the random number generator.
ncm_mset_catalog_sync ()
void ncm_mset_catalog_sync (NcmMSetCatalog *mcat
,gboolean check
);
Synchronize memory and data file. If no file was defined, it simply returns.
ncm_mset_catalog_timed_sync ()
void ncm_mset_catalog_timed_sync (NcmMSetCatalog *mcat
,gboolean check
);
Synchronize memory and data file if enough time was passed after
the last sync, see ncm_mset_catalog_set_sync_interval()
. If no
file was defined, it simply returns.
ncm_mset_catalog_reset_stats ()
void
ncm_mset_catalog_reset_stats (NcmMSetCatalog *mcat
);
Reset catalog statistical quantities.
ncm_mset_catalog_reset ()
void
ncm_mset_catalog_reset (NcmMSetCatalog *mcat
);
Clean all catalog data from memory and file. Otherwise it does not change any object's parameter.
ncm_mset_catalog_erase_data ()
void
ncm_mset_catalog_erase_data (NcmMSetCatalog *mcat
);
Erases all data from the fits file associated with the catalog.
ncm_mset_catalog_peek_filename ()
const gchar *
ncm_mset_catalog_peek_filename (NcmMSetCatalog *mcat
);
Gets the filename associated with mcat
.
ncm_mset_catalog_get_rng ()
NcmRNG *
ncm_mset_catalog_get_rng (NcmMSetCatalog *mcat
);
This function checks if any pseudo random number generator (RNG) is registred in the catalog. If so, it returns it or NULL.
ncm_mset_catalog_largest_error ()
gdouble
ncm_mset_catalog_largest_error (NcmMSetCatalog *mcat
);
This function calculates the largest proportional error of the parameters included, i.e., $\text{lre} = \sigma_{\hat{p}}/(|\hat{p}|\sqrt{n})$ where $n$ represents the number of samples in the catalog, $\hat{p}$ is the estimated mean of the parameter $p$ and $\sigma_{\hat{p}}$ its standard deviation.
It tries to guess when $p = 0$. In this case $\sigma_{\hat{p}} \approx |\hat{p}|\sqrt{n}$. Therefore, for $n > 10$, it tests if $\text{lre} \approx 1$ and, if it is the case, it returns $\text{lre} = \sigma_{\hat{p}}/\sqrt{n}$ instead.
ncm_mset_catalog_len ()
guint
ncm_mset_catalog_len (NcmMSetCatalog *mcat
);
Number of itens in the catalog.
ncm_mset_catalog_max_time ()
guint
ncm_mset_catalog_max_time (NcmMSetCatalog *mcat
);
Number of itens in the catalog divided by the number of chains.
ncm_mset_catalog_set_burnin ()
void ncm_mset_catalog_set_burnin (NcmMSetCatalog *mcat
,glong burnin
);
Sets the number of elements to ignore when reading from a catalogue, it must be set before loading data from a file.
It will not affect a catalogue in any other context, only when reading data from a file. It is recommended to be used only when analysing a catalogue.
ncm_mset_catalog_get_burnin ()
glong
ncm_mset_catalog_get_burnin (NcmMSetCatalog *mcat
);
Gets the burn-in size, see ncm_mset_catalog_set_burnin()
.
ncm_mset_catalog_add_from_mset ()
void ncm_mset_catalog_add_from_mset (NcmMSetCatalog *mcat
,NcmMSet *mset
,...
);
This function adds a new element to the catalog using the parameters from mset
.
It assumes that mset
is compatible with the catalog and expect the
right number of additional values.
ncm_mset_catalog_add_from_mset_array ()
void ncm_mset_catalog_add_from_mset_array (NcmMSetCatalog *mcat
,NcmMSet *mset
,gdouble *ax
);
This funtion adds a new element to the catalog using the parameters from mset
.
It assumes that mset
is compatible with the catalog and expect the
right number of additional values in the array ax
.
ncm_mset_catalog_add_from_vector ()
void ncm_mset_catalog_add_from_vector (NcmMSetCatalog *mcat
,NcmVector *vals
);
Adds a new element to the catalog using the values from the vector
vals
.
ncm_mset_catalog_log_current_stats ()
void
ncm_mset_catalog_log_current_stats (NcmMSetCatalog *mcat
);
Logs the current means and standard deviations of the catalog's parameters.
ncm_mset_catalog_log_current_chain_stats ()
void
ncm_mset_catalog_log_current_chain_stats
(NcmMSetCatalog *mcat
);
Logs the current means and standard deviations of the catalog's parameters for each chain.
ncm_mset_catalog_get_mset ()
NcmMSet *
ncm_mset_catalog_get_mset (NcmMSetCatalog *mcat
);
Gets the NcmMSet catalog from mcat
.
ncm_mset_catalog_get_run_type ()
const gchar *
ncm_mset_catalog_get_run_type (NcmMSetCatalog *mcat
);
Gets the run type string from mcat
.
ncm_mset_catalog_peek_row ()
NcmVector * ncm_mset_catalog_peek_row (NcmMSetCatalog *mcat
,guint i
);
Gets the i
-th row.
ncm_mset_catalog_peek_current_row ()
NcmVector *
ncm_mset_catalog_peek_current_row (NcmMSetCatalog *mcat
);
Gets the last added row.
ncm_mset_catalog_peek_current_e_mean ()
NcmVector *
ncm_mset_catalog_peek_current_e_mean (NcmMSetCatalog *mcat
);
Gets the last ensemble mean.
ncm_mset_catalog_peek_current_e_var ()
NcmVector *
ncm_mset_catalog_peek_current_e_var (NcmMSetCatalog *mcat
);
Gets the last ensemble variance.
ncm_mset_catalog_peek_e_mean_t ()
NcmVector * ncm_mset_catalog_peek_e_mean_t (NcmMSetCatalog *mcat
,guint t
);
Gets the mean of the t
-th ensemble.
ncm_mset_catalog_peek_e_var_t ()
NcmVector * ncm_mset_catalog_peek_e_var_t (NcmMSetCatalog *mcat
,guint t
);
Gets the variance of the t
-th ensemble.
ncm_mset_catalog_get_mean ()
void ncm_mset_catalog_get_mean (NcmMSetCatalog *mcat
,NcmVector **mean
);
Gets the current mean vector.
ncm_mset_catalog_get_covar ()
void ncm_mset_catalog_get_covar (NcmMSetCatalog *mcat
,NcmMatrix **cov
);
Gets the current covariance matrix.
ncm_mset_catalog_get_full_covar ()
void ncm_mset_catalog_get_full_covar (NcmMSetCatalog *mcat
,NcmMatrix **cov
);
Gets the current full (including additional values) covariance matrix.
ncm_mset_catalog_log_full_covar ()
void
ncm_mset_catalog_log_full_covar (NcmMSetCatalog *mcat
);
FIXME
ncm_mset_catalog_estimate_autocorrelation_tau ()
void
ncm_mset_catalog_estimate_autocorrelation_tau
(NcmMSetCatalog *mcat
);
Updates the internal estimates of the integrate autocorrelation time.
ncm_mset_catalog_peek_autocorrelation_tau ()
NcmVector *
ncm_mset_catalog_peek_autocorrelation_tau
(NcmMSetCatalog *mcat
);
Gets the last estimate of the autocorrelation tau calculated
in the last call of ncm_mset_catalog_estimate_autocorrelation_tau()
.
ncm_mset_catalog_get_param_shrink_factor ()
gdouble ncm_mset_catalog_get_param_shrink_factor (NcmMSetCatalog *mcat
,guint p
);
Gets the current shrink factor of parameter p
.
ncm_mset_catalog_get_shrink_factor ()
gdouble
ncm_mset_catalog_get_shrink_factor (NcmMSetCatalog *mcat
);
Gets the current shrink factor which is the multivatiate potential scale reduction factor (MPSRF), namely, $$\hat{R}^p = \sqrt{\frac{n - 1}{n} + \left( \frac{m + 1}{m} \right) \lambda_1},$$ where $n$ is the number of points of one chain, $m$ is the number of chains and $\lambda_1$ is the largest eigenvalue of the positive definite matrix $W^{-1}B/n$.
$W$ is the within-chain variance: $$W = $$ FIXME
$B$ is the between-chain variance: $$B = $$ FIXME
Refined version: $$\hat{R}^p = \sqrt{\frac{\hat{d} + 3}{\hat{d} + 1} \left(\frac{n - 1}{n} + \left( \frac{m + 1}{m} \right) \lambda_1\right)},$$ where $\hat{d} = 2 \hat{V}^2 / \widehat{Var}(\hat{V})$, $$\hat{V} = \frac{n -1}{n}W + \frac{m + 1}{m} \frac{B}{n}.$$
Some references for this MCMC convergence diagnostic: Brooks and Gelman (1998), Gelman and Rubin (1992), SAS/STAT.
ncm_mset_catalog_param_pdf ()
void ncm_mset_catalog_param_pdf (NcmMSetCatalog *mcat
,guint i
);
Bins and calculates the pdf associated with the parameter i
.
(not ready yet FIXME)
ncm_mset_catalog_param_pdf_pvalue ()
gdouble ncm_mset_catalog_param_pdf_pvalue (NcmMSetCatalog *mcat
,gdouble pval
,gboolean both
);
Calculates the p-value associated with the parameter value pval
.
ncm_mset_catalog_calc_ci_direct ()
NcmMatrix * ncm_mset_catalog_calc_ci_direct (NcmMSetCatalog *mcat
,NcmMSetFunc *func
,NcmVector *x_v
,GArray *p_val
);
Calculates the mean and the confidence interval (CI) for the value of func
for each p-value in p_val
. It stores the results in a NcmVector, where the
first element contains the mean and the following contain the lower and
upper bounds for each p-value in p_val
.
This function calculates the quantiles directly using:
gsl_stats_quantile_from_sorted_data for this reason it must allocates the
catalog size times the number of elements in x
, for a less memory intensive
version use ncm_mset_catalog_calc_ci_interp()
.
The NcmMSetFunc func
must be of dimension one.
Example:
If p_val
contains two values ($1\sigma$) 0.6827 and ($\sigma$) 0.9545,
the first element will contain the mean, the second and third, the lower
and upper bounds, respectively. Then, the fourth and fifth elements the
lower and upper bounds of $2\sigma$ CI.
Parameters
mcat |
||
func |
a NcmMSetFunc of type n-n |
|
x_v |
NcmVector of arguments of |
|
p_val |
p-values for the confidence intervals. |
[element-type double] |
Returns
a NcmVector containing the mean and lower/upper bound of the confidence interval for func
.
[transfer full]
ncm_mset_catalog_calc_ci_interp ()
NcmMatrix * ncm_mset_catalog_calc_ci_interp (NcmMSetCatalog *mcat
,NcmMSetFunc *func
,NcmVector *x_v
,GArray *p_val
,guint nodes
,NcmFitRunMsgs mtype
);
Calculates the mean and the confidence interval (CI) for the value of func
for each p-value in p_val
. It stores the results in a NcmVector, where the
first element contains the mean and the following contain the lower and
upper bounds for each p-value in p_val
.
This function creates an approximation of the distribution for each value of
the function func
and calculates the quantiles from this approximation.
The NcmMSetFunc func
must be of dimension one.
Example:
If p_val
contains two values ($1\sigma$) 0.6827 and ($\sigma$) 0.9545,
the first element will contain the mean, the second and third, the lower
and upper bounds, respectively. Then, the fourth and fifth elements the
lower and upper bounds of $2\sigma$ CI.
Parameters
mcat |
||
func |
a NcmMSetFunc of type n-n |
|
x_v |
NcmVector of arguments of |
|
p_val |
p-values for the confidence intervals. |
[element-type double] |
nodes |
number of nodes in the distribution approximations |
|
mtype |
NcmFitRunMsgs log level |
Returns
a NcmVector containing the mean and lower/upper bound of the confidence interval for func
.
[transfer full]
ncm_mset_catalog_calc_distrib ()
NcmStatsDist1d * ncm_mset_catalog_calc_distrib (NcmMSetCatalog *mcat
,NcmMSetFunc *func
,NcmFitRunMsgs mtype
);
Calculates the distribution of func
.
This function creates an approximation of the distribution for each value of
the function func
calculated in each model in mcat
.
ncm_mset_catalog_calc_param_distrib ()
NcmStatsDist1d * ncm_mset_catalog_calc_param_distrib (NcmMSetCatalog *mcat
,const NcmMSetPIndex *pi
,NcmFitRunMsgs mtype
);
Calculates the distribution of parameter pi
.
This function creates an approximation of the distribution for each value of
the parameter pi
in mcat
.
ncm_mset_catalog_calc_add_param_distrib ()
NcmStatsDist1d * ncm_mset_catalog_calc_add_param_distrib (NcmMSetCatalog *mcat
,guint add_param
,NcmFitRunMsgs mtype
);
Calculates the distribution of parameter pi
.
This function creates an approximation of the distribution for each value of
the parameter pi
in mcat
.
ncm_mset_catalog_calc_param_ensemble_evol ()
void ncm_mset_catalog_calc_param_ensemble_evol (NcmMSetCatalog *mcat
,const NcmMSetPIndex *pi
,guint nsteps
,NcmFitRunMsgs mtype
,NcmVector **pval
,NcmMatrix **t_evol
);
Calculates the time evolution of the parameter pi
distribution.
Parameters
mcat |
||
pi |
||
nsteps |
number of steps to calculate the distribution |
|
mtype |
NcmFitRunMsgs log level |
|
pval |
output NcmVector containing parameter values. |
[out callee-allocates] |
t_evol |
output NcmMatrix containing probability distribution evolution. |
[out callee-allocates] |
ncm_mset_catalog_calc_add_param_ensemble_evol ()
void ncm_mset_catalog_calc_add_param_ensemble_evol (NcmMSetCatalog *mcat
,guint add_param
,guint nsteps
,NcmFitRunMsgs mtype
,NcmVector **pval
,NcmMatrix **t_evol
);
Calculates the time evolution of the parameter pi
distribution.
Parameters
mcat |
||
add_param |
additional parameter index |
|
nsteps |
number of steps to calculate the distribution |
|
mtype |
NcmFitRunMsgs log level |
|
pval |
output NcmVector containing parameter values. |
[out callee-allocates] |
t_evol |
output NcmMatrix containing probability distribution evolution. |
[out callee-allocates] |
Types and Values
enum NcmMSetCatalogSync
Catalog sync modes.
Members
Catalog will be synchronized only when closing the file or with an explicit call of |
||
Catalog will be synchronized in every catalog addition. |
||
Catalog will be synchronized with a minimum time interval between syncs. |
Property Details
The “burnin”
property
“burnin” glong
Burn-in size.
Flags: Read / Write / Construct
Allowed values: >= 0
Default value: 0
The “filename”
property
“filename” gchar *
Catalog filename.
Flags: Read / Write / Construct
Default value: NULL
The “nadd-val-names”
property
“nadd-val-names” GStrv
Additional value names.
Flags: Read / Write / Construct Only
The “nadd-val-symbols”
property
“nadd-val-symbols” GStrv
Additional value symbols.
Flags: Read / Write / Construct Only
The “nadd-vals”
property
“nadd-vals” guint
Number of additional values.
Flags: Read / Write / Construct Only
Default value: 0
The “nchains”
property
“nchains” guint
Number of different chains in the catalog.
Flags: Read / Write / Construct Only
Allowed values: >= 1
Default value: 1
The “read-only”
property
“read-only” gboolean
If the fits catalogue must be open in the readonly mode.
Flags: Read / Write / Construct Only
Default value: FALSE
The “run-type-string”
property
“run-type-string” gchar *
Run type string.
Flags: Read / Write / Construct
Default value: "undefined-run"
The “smode”
property
“smode” NcmMSetCatalogSync
Catalog sync mode.
Flags: Read / Write / Construct
Default value: NCM_MSET_CATALOG_SYNC_AUTO
The “sync-interval”
property
“sync-interval” gdouble
Data sync interval.
Flags: Read / Write / Construct
Allowed values: [0,1000]
Default value: 10
The “weighted”
property
“weighted” gboolean
Catalog with weighted rows.
Flags: Read / Write / Construct Only
Default value: FALSE