Top | ![]() |
![]() |
![]() |
![]() |
Functions
Properties
gchar * | filename | Read / Write / Construct |
gdouble | flush-interval | Read / Write / Construct |
NcmMSetCatalogFlush | fmode | Read / Write / Construct |
NcmMSet * | mset | Read / Write / Construct Only |
guint | nadd-vals | Read / Write / Construct Only |
guint | nchains | Read / Write / Construct Only |
NcmRNG * | rng | Read / Write |
gchar * | run-type-string | Read / Write / Construct |
gboolean | weighted | Read / Write / Construct Only |
Types and Values
enum | NcmMSetCatalogFlush |
#define | NCM_MSET_CATALOG_EXTNAME |
#define | NCM_MSET_CATALOG_M2LNL_COLNAME |
#define | NCM_MSET_CATALOG_FIRST_ID_LABEL |
#define | NCM_MSET_CATALOG_RNG_ALGO_LABEL |
#define | NCM_MSET_CATALOG_RNG_SEED_LABEL |
#define | NCM_MSET_CATALOG_RNG_STAT_LABEL |
#define | NCM_MSET_CATALOG_RNG_INIS_LABEL |
#define | NCM_MSET_CATALOG_NROWS_LABEL |
#define | NCM_MSET_CATALOG_RTYPE_LABEL |
#define | NCM_MSET_CATALOG_NCHAINS_LABEL |
#define | NCM_MSET_CATALOG_NADDVAL_LABEL |
#define | NCM_MSET_CATALOG_WEIGHTED_LABEL |
#define | NCM_MSET_CATALOG_RTYPE_BSTRAP_MEAN |
#define | NCM_MSET_CATALOG_RTYPE_UNDEFINED |
#define | NCM_MSET_CATALOG_FSYMB_LABEL |
Description
This class defines a catalog type object. This object can automatically synchronize with a fits file (thought cfitsio).
For Mote Carlo studies, like resampling from a fiducial model or bootstrap, it is used to save the best-fitting values of each realization. Since the order of the resampling is important, due to the fact that we use the same pseudo-random number generator for all resamplings, this object also guarantees the order of the samples added.
For Markov Chain Monte Carlo (MCMC) this object saves the value of the same likelihood in different points of the parameter space.
For both applications this object keeps an interactive mean and variance of the parameters added, this allows a sample by sample analyses of the convergence. Some MCMC convergence diagnostic functions are also implemented here.
Functions
ncm_mset_catalog_new ()
NcmMSetCatalog * ncm_mset_catalog_new (NcmMSet *mset
,guint nadd_vals
,guint nchains
,gboolean weighted
,...
);
Creates a new NcmMSetCatalog based on the NcmFit object fit
. The catalog assumes that
the fit
object will remain with the same set of free parameters during its whole lifetime.
If nchains
is larger than one, the catalog will keep track of the statistics of each chain
separately.
Parameters
mset |
a NcmMSet |
|
nadd_vals |
number of additional values |
|
nchains |
number of different chains in the catalog (>=1) |
|
weighted |
set to TRUE whenever the catalog is weighted |
|
... |
additional values names |
ncm_mset_catalog_new_from_file ()
NcmMSetCatalog *
ncm_mset_catalog_new_from_file (const gchar *filename
);
Creates a new NcmMSetCatalog from the catalog in the file file
.
It will use also the mset file (same name but with .mset extension).
ncm_mset_catalog_ref ()
NcmMSetCatalog *
ncm_mset_catalog_ref (NcmMSetCatalog *mcat
);
Increases the reference count of mcat
atomically.
ncm_mset_catalog_free ()
void
ncm_mset_catalog_free (NcmMSetCatalog *mcat
);
Decreases the reference count of mcat
atomically.
ncm_mset_catalog_clear ()
void
ncm_mset_catalog_clear (NcmMSetCatalog **mcat
);
Decrese the reference count of *mcat
atomically and sets the pointer *mcat
to null.
ncm_mset_catalog_set_add_val_name ()
void ncm_mset_catalog_set_add_val_name (NcmMSetCatalog *mcat
,guint i
,const gchar *name
);
Sets the i
-th additional value name
ncm_mset_catalog_set_file ()
void ncm_mset_catalog_set_file (NcmMSetCatalog *mcat
,const gchar *filename
);
Sets the data filename to be used to sync/save data.
ncm_mset_catalog_set_flush_mode ()
void ncm_mset_catalog_set_flush_mode (NcmMSetCatalog *mcat
,NcmMSetCatalogFlush fmode
);
Sets the flush mode to fmode
.
ncm_mset_catalog_set_flush_interval ()
void ncm_mset_catalog_set_flush_interval (NcmMSetCatalog *mcat
,gdouble interval
);
Sets the minimum time interval between flushs.
ncm_mset_catalog_set_first_id ()
void ncm_mset_catalog_set_first_id (NcmMSetCatalog *mcat
,gint first_id
);
Sets the first id of the catalog, mainly used to inform in which realization the catalog starts.
ncm_mset_catalog_set_run_type ()
void ncm_mset_catalog_set_run_type (NcmMSetCatalog *mcat
,const gchar *rtype_str
);
Sets the run type string.
ncm_mset_catalog_set_rng ()
void ncm_mset_catalog_set_rng (NcmMSetCatalog *mcat
,NcmRNG *rng
);
Sets the random number generator.
ncm_mset_catalog_sync ()
void ncm_mset_catalog_sync (NcmMSetCatalog *mcat
,gboolean check
);
Synchronize memory and data file. If no file was defined, it simply returns.
ncm_mset_catalog_reset_stats ()
void
ncm_mset_catalog_reset_stats (NcmMSetCatalog *mcat
);
Reset catalog statistical quantities.
ncm_mset_catalog_reset ()
void
ncm_mset_catalog_reset (NcmMSetCatalog *mcat
);
Clean all catalog data from memory and file. Otherwise it does not change any object's parameter.
ncm_mset_catalog_erase_data ()
void
ncm_mset_catalog_erase_data (NcmMSetCatalog *mcat
);
Erases all data from the fits file associated with the catalog.
ncm_mset_catalog_peek_filename ()
const gchar *
ncm_mset_catalog_peek_filename (NcmMSetCatalog *mcat
);
Gets the filename associated with mcat
.
ncm_mset_catalog_get_rng ()
NcmRNG *
ncm_mset_catalog_get_rng (NcmMSetCatalog *mcat
);
This function checks if any pseudo random number generator (RNG) is registred in the catalog. If so, it returns it or NULL.
ncm_mset_catalog_largest_error ()
gdouble
ncm_mset_catalog_largest_error (NcmMSetCatalog *mcat
);
This function calculates the largest proportional error of the parameters included, i.e., $\text{lre} = \sigma_{\hat{p}}/(|\hat{p}|\sqrt{n})$ where $n$ represents the number of samples in the catalog, $\hat{p}$ is the estimated mean of the parameter $p$ and $\sigma_{\hat{p}}$ its standard deviation.
It tries to guess when $p = 0$. In this case $\sigma_{\hat{p}} \approx |\hat{p}|\sqrt{n}$. Therefore, for $n > 10$, it tests if $\text{lre} \approx 1$ and, if it is the case, it returns $\text{lre} = \sigma_{\hat{p}}/\sqrt{n}$ instead.
ncm_mset_catalog_len ()
guint
ncm_mset_catalog_len (NcmMSetCatalog *mcat
);
Number of itens in the catalog.
ncm_mset_catalog_add_from_mset ()
void ncm_mset_catalog_add_from_mset (NcmMSetCatalog *mcat
,NcmMSet *mset
,...
);
This function adds a new element to the catalog using the parameters from mset
.
It assumes that mset
is compatible with the catalog and expect the
right number of additional values.
ncm_mset_catalog_add_from_mset_array ()
void ncm_mset_catalog_add_from_mset_array (NcmMSetCatalog *mcat
,NcmMSet *mset
,gdouble *ax
);
This funtion adds a new element to the catalog using the parameters from mset
.
It assumes that mset
is compatible with the catalog and expect the
right number of additional values in the array ax
.
ncm_mset_catalog_add_from_vector ()
void ncm_mset_catalog_add_from_vector (NcmMSetCatalog *mcat
,NcmVector *vals
);
Adds a new element to the catalog using the values from the vector
vals
.
ncm_mset_catalog_log_current_stats ()
void
ncm_mset_catalog_log_current_stats (NcmMSetCatalog *mcat
);
Logs the current means and standard deviations of the catalog's parameters.
ncm_mset_catalog_log_current_chain_stats ()
void
ncm_mset_catalog_log_current_chain_stats
(NcmMSetCatalog *mcat
);
Logs the current means and standard deviations of the catalog's parameters for each chain.
ncm_mset_catalog_get_mset ()
NcmMSet *
ncm_mset_catalog_get_mset (NcmMSetCatalog *mcat
);
Gets the NcmMSet catalog from mcat
.
ncm_mset_catalog_get_run_type ()
const gchar *
ncm_mset_catalog_get_run_type (NcmMSetCatalog *mcat
);
Gets the run type string from mcat
.
ncm_mset_catalog_peek_row ()
NcmVector * ncm_mset_catalog_peek_row (NcmMSetCatalog *mcat
,guint i
);
Gets the i
-th row.
ncm_mset_catalog_peek_current_row ()
NcmVector *
ncm_mset_catalog_peek_current_row (NcmMSetCatalog *mcat
);
Gets the last added row.
ncm_mset_catalog_get_mean ()
void ncm_mset_catalog_get_mean (NcmMSetCatalog *mcat
,NcmVector **mean
);
Gets the current mean vector.
ncm_mset_catalog_get_covar ()
void ncm_mset_catalog_get_covar (NcmMSetCatalog *mcat
,NcmMatrix **cov
);
Gets the current covariance matrix.
ncm_mset_catalog_estimate_autocorrelation_tau ()
void
ncm_mset_catalog_estimate_autocorrelation_tau
(NcmMSetCatalog *mcat
);
Updates the internal estimates of the integrate autocorrelation time.
ncm_mset_catalog_peek_autocorrelation_tau ()
NcmVector *
ncm_mset_catalog_peek_autocorrelation_tau
(NcmMSetCatalog *mcat
);
Gets the last estimate of the autocorrelation tau calculated
in the last call of ncm_mset_catalog_estimate_autocorrelation_tau()
.
ncm_mset_catalog_get_param_shrink_factor ()
gdouble ncm_mset_catalog_get_param_shrink_factor (NcmMSetCatalog *mcat
,guint p
);
Gets the current shrink factor of parameter p
.
ncm_mset_catalog_get_shrink_factor ()
gdouble
ncm_mset_catalog_get_shrink_factor (NcmMSetCatalog *mcat
);
Gets the current shrink factor which is the multivatiate potential scale reduction factor (MPSRF), namely, $$\hat{R}^p = \sqrt{\frac{n - 1}{n} + \left( \frac{m + 1}{m} \right) \lambda_1},$$ where $n$ is the number of points of one chain, $m$ is the number of chains and $\lambda_1$ is the largest eigenvalue of the positive definite matrix $W^{-1}B/n$.
$W$ is the within-chain variance: $$W = $$ FIXME
$B$ is the between-chain variance: $$B = $$ FIXME
Refined version: $$\hat{R}^p = \sqrt{\frac{\hat{d} + 3}{\hat{d} + 1} \left(\frac{n - 1}{n} + \left( \frac{m + 1}{m} \right) \lambda_1\right)},$$ where $\hat{d} = 2 \hat{V}^2 / \widehat{Var}(\hat{V})$, $$\hat{V} = \frac{n -1}{n}W + \frac{m + 1}{m} \frac{B}{n}.$$
Some references for this MCMC convergence diagnostic: Brooks and Gelman (1998), Gelman and Rubin (1992), SAS/STAT.
ncm_mset_catalog_param_pdf ()
void ncm_mset_catalog_param_pdf (NcmMSetCatalog *mcat
,guint i
);
Bins and calculates the pdf associated with the parameter i
.
(not ready yet FIXME)
ncm_mset_catalog_param_pdf_pvalue ()
gdouble ncm_mset_catalog_param_pdf_pvalue (NcmMSetCatalog *mcat
,gdouble pval
,gboolean both
);
Calculates the p-value associated with the parameter value pval
.
ncm_mset_catalog_calc_ci_direct ()
NcmMatrix * ncm_mset_catalog_calc_ci_direct (NcmMSetCatalog *mcat
,guint burnin
,NcmMSetFunc *func
,gdouble *x
,GArray *p_val
);
Calculates the mean and the confidence interval (CI) for the value of func
for each p-value in p_val
. It stores the results in a NcmVector, where the
first element contains the mean and the following contain the lower and
upper bounds for each p-value in p_val
.
This function calculates the quantiles directly using:
gsl_stats_quantile_from_sorted_data for this reason it must allocates the
catalog size times the number of elements in x
, for a less memory intensive
version use ncm_mset_catalog_calc_ci_interp()
.
The NcmMSetFunc func
must be of dimension one.
Example:
If p_val
contains two values ($1\sigma$) 0.6827 and ($\sigma$) 0.9545,
the first element will contain the mean, the second and third, the lower
and upper bounds, respectively. Then, the fourth and fifth elements the
lower and upper bounds of $2\sigma$ CI.
Parameters
mcat |
||
burnin |
burn-in size |
|
func |
a NcmMSetFunc of type n-n |
|
x |
the argument of |
[array][element-type double] |
p_val |
p-values for the confidence intervals. |
[element-type double] |
Returns
a NcmVector containing the mean and lower/upper bound of the confidence interval for func
.
[transfer full]
ncm_mset_catalog_calc_ci_interp ()
NcmMatrix * ncm_mset_catalog_calc_ci_interp (NcmMSetCatalog *mcat
,guint burnin
,NcmMSetFunc *func
,gdouble *x
,GArray *p_val
,guint nodes
,NcmFitRunMsgs mtype
);
Calculates the mean and the confidence interval (CI) for the value of func
for each p-value in p_val
. It stores the results in a NcmVector, where the
first element contains the mean and the following contain the lower and
upper bounds for each p-value in p_val
.
This function creates an approximation of the distribution for each value of
the function func
and calculates the quantiles from this approximation.
The NcmMSetFunc func
must be of dimension one.
Example:
If p_val
contains two values ($1\sigma$) 0.6827 and ($\sigma$) 0.9545,
the first element will contain the mean, the second and third, the lower
and upper bounds, respectively. Then, the fourth and fifth elements the
lower and upper bounds of $2\sigma$ CI.
Parameters
mcat |
||
burnin |
burn-in size |
|
func |
a NcmMSetFunc of type n-n |
|
x |
the argument of |
[array][element-type double] |
p_val |
p-values for the confidence intervals. |
[element-type double] |
nodes |
number of nodes in the distribution approximations |
|
mtype |
NcmFitRunMsgs log level |
Returns
a NcmVector containing the mean and lower/upper bound of the confidence interval for func
.
[transfer full]
ncm_mset_catalog_calc_distrib ()
NcmStatsDist1d * ncm_mset_catalog_calc_distrib (NcmMSetCatalog *mcat
,guint burnin
,NcmMSetFunc *func
,NcmFitRunMsgs mtype
);
Calculates the distribution of func
.
This function creates an approximation of the distribution for each value of
the function func
calculated in each model in mcat
.
ncm_mset_catalog_calc_param_distrib ()
NcmStatsDist1d * ncm_mset_catalog_calc_param_distrib (NcmMSetCatalog *mcat
,guint burnin
,const NcmMSetPIndex *pi
,NcmFitRunMsgs mtype
);
Calculates the distribution of parameter pi
.
This function creates an approximation of the distribution for each value of
the parameter pi
in mcat
.
ncm_mset_catalog_calc_add_param_distrib ()
NcmStatsDist1d * ncm_mset_catalog_calc_add_param_distrib (NcmMSetCatalog *mcat
,guint burnin
,guint add_param
,NcmFitRunMsgs mtype
);
Calculates the distribution of parameter pi
.
This function creates an approximation of the distribution for each value of
the parameter pi
in mcat
.
Parameters
mcat |
||
burnin |
burn-in size |
|
add_param |
additional parameter index |
|
mtype |
NcmFitRunMsgs log level |
Property Details
The “filename”
property
“filename” gchar *
Catalog filename.
Flags: Read / Write / Construct
Default value: NULL
The “flush-interval”
property
“flush-interval” gdouble
Data flush interval.
Flags: Read / Write / Construct
Allowed values: [0,1000]
Default value: 10
The “fmode”
property
“fmode” NcmMSetCatalogFlush
Catalog flush mode.
Flags: Read / Write / Construct
Default value: NCM_MSET_CATALOG_FLUSH_AUTO
The “nadd-vals”
property
“nadd-vals” guint
Number of additional values.
Flags: Read / Write / Construct Only
Allowed values: >= 1
Default value: 1
The “nchains”
property
“nchains” guint
Number of different chains in the catalog.
Flags: Read / Write / Construct Only
Allowed values: >= 1
Default value: 1
The “run-type-string”
property
“run-type-string” gchar *
Run type string.
Flags: Read / Write / Construct
Default value: "undefined-run"
The “weighted”
property
“weighted” gboolean
Catalog with weighted rows.
Flags: Read / Write / Construct Only
Default value: FALSE