leaspy.io.outputs

Submodules

Classes

IndividualParameters

Data container for individual parameters, contains IDs, timepoints and observations values.

Result

Result object class.

Package Contents

class IndividualParameters[source]

Data container for individual parameters, contains IDs, timepoints and observations values. Output of the leaspy.algo.personalize method, contains the random effects.

There are used as output of the personalization algorithms and as input/output of the simulation algorithm, to provide an initial distribution of individual parameters.

Attributes:
_indiceslist

List of the patient indices

_individual_parametersdict

Individual indices (key) with their corresponding individual parameters {parameter name: parameter value}

_parameters_shapedict

Shape of each individual parameter

_default_saving_typestr

Default extension for saving when none is provided

VALID_IO_EXTENSIONS = ['csv', 'json']
add_individual_parameters(index, individual_parameters)[source]

Add the individual parameter of an individual to the IndividualParameters object

Parameters:
index:class:IDType

Index of the individual

individual_parametersDictParams

Individual parameters of the individual

Raises:
LeaspyIndividualParamsInputError
  • If the index is not a string or has already been added

  • Or if the individual parameters is not a dict.

  • Or if individual parameters are not self-consistent.

Parameters:

Examples

Add two individual with tau, xi and sources parameters

>>> ip = IndividualParameters()
>>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]})
>>> ip.add_individual_parameters('index-2', {"xi": 0.2, "tau": 73, "sources": [-0.4, -0.1]})
items()[source]

Get the items of the individual parameters dictionary.

Returns:
ItemsView

A view object displaying a list of tuples (individual ID, parameters dict)

Examples

>>> ip = IndividualParameters()
>>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]})
>>> list(ip.items())
['index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]}]
subset(indices, *, copy=True)[source]

Returns IndividualParameters object with a subset of the initial individuals

Parameters:
indicesIterable`[:class:`~leaspy.utils.typing.IDType]

List of strings that corresponds to the indices of the individuals to return

copybool, optional (default True)

Should we copy underlying parameters or not?

Returns:
IndividualParameters

An instance of the IndividualParameters object with the selected list of individuals

Raises:
LeaspyIndividualParamsInputError

Raise an error if one of the index is not in the IndividualParameters

Parameters:

Examples

>>> ip = IndividualParameters()
>>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]})
>>> ip.add_individual_parameters('index-2', {"xi": 0.2, "tau": 73, "sources": [-0.4, -0.1]})
>>> ip.add_individual_parameters('index-3', {"xi": 0.3, "tau": 58, "sources": [-0.6, 0.2]})
>>> ip_sub = ip.subset(['index-1', 'index-3'])
get_aggregate(parameter, function)[source]

Returns the result of aggregation by function of parameter values across all patients

Parameters:
parameterParamType

Name of the parameter

functionCallable

A function operating on iterables and supporting axis keyword, and outputing an iterable supporting the tolist method.

Returns:
list

Resulting value of the parameter

Raises:
LeaspyIndividualParamsInputError
  • If individual parameters are empty,

  • or if the parameter is not in the IndividualParameters.

Parameters:
Return type:

list

Examples

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> tau_median = ip.get_aggregate("tau", np.median)
get_mean(parameter)[source]

Returns the mean value of a parameter across all patients

Parameters:
parameterParamType

Name of the parameter

Returns:
list

Mean value of the parameter

Raises:
LeaspyIndividualParamsInputError
  • If individual parameters are empty,

  • or if the parameter is not in the IndividualParameters.

Parameters:

parameter (ParamType)

Examples

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> tau_mean = ip.get_mean("tau")
get_std(parameter)[source]

Returns the standard deviation of a parameter across all patients

Parameters:
parameterParamType

Name of the parameter

Returns:
list

Standard-deviation value of the parameter

Raises:
LeaspyIndividualParamsInputError
  • If individual parameters are empty,

  • or if the parameter is not in the IndividualParameters.

Parameters:

parameter (ParamType)

Examples

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> tau_std = ip.get_std("tau")
to_dataframe()[source]

Returns the dataframe of individual parameters

Returns:
pandas.DataFrame

Each row corresponds to one individual. The index corresponds to the individual index (‘ID’). The columns are the names of the parameters.

Return type:

DataFrame

Examples

Convert the individual parameters object into a dataframe

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> ip_df = ip.to_dataframe()
static from_dataframe(df)[source]

Static method that returns an IndividualParameters object from the dataframe

Parameters:
dfpandas.DataFrame

Dataframe of the individual parameters. Each row must correspond to one individual. The index corresponds to the individual index. The columns are the names of the parameters.

Returns:
IndividualParameters

An instance of IndividualParameters initialized from the DataFrame.

Parameters:

df (DataFrame)

Examples

>>> import pandas as pd
>>> data = {
>>>     'tau': [70, 73],
>>>     'xi': [0.1, 0.2],
>>>     'sources_0': [0.1, -0.4],
>>>     'sources_1': [-0.3, -0.1]
>>> }
>>> df = pd.DataFrame(data, index=['id1', 'id2'])
>>> ip = IndividualParameters.from_dataframe(df)
static from_pytorch(indices, dict_pytorch)[source]

Static method that returns an IndividualParameters object from the indices and pytorch dictionary

Parameters:
indiceslist`[:class:`~leaspy.utils.typing.IDType]

List of the patients indices

dict_pytorchDictParmasTorch

Dictionary of the individual parameters

Returns:
IndividualParameters

An instance of IndividualParameters initialized from the pytorch dictionary.

Raises:
LeaspyIndividualParamsInputError
Parameters:

Examples

>>> indices = ['index-1', 'index-2', 'index-3']
>>> ip_pytorch = {
>>>    "xi": torch.tensor([[0.1], [0.2], [0.3]], dtype=torch.float32),
>>>    "tau": torch.tensor([[70], [73], [58.]], dtype=torch.float32),
>>>    "sources": torch.tensor([[0.1, -0.3], [-0.4, 0.1], [-0.6, 0.2]], dtype=torch.float32)
>>> }
>>> ip_pytorch = IndividualParameters.from_pytorch(indices, ip_pytorch)
to_pytorch()[source]

Returns the indices and pytorch dictionary of individual parameters

Returns:
indices: list`[:class:`~leaspy.utils.typing.IDType]

List of patient indices

pytorch_dict: DictParamsTorch

Dictionary of the individual parameters {parameter name: pytorch tensor of values across individuals}

Return type:

tuple[list[IDType], DictParamsTorch]

Examples

Convert the individual parameters object into a dataframe

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> indices, ip_pytorch = ip.to_pytorch()
save(path, **kwargs)[source]

Saves the individual parameters (json or csv) at the path location

TODO? save leaspy version as well for retro/future-compatibility issues?

Parameters:
pathstr

Path and file name of the individual parameters. The extension can be json or csv. If no extension, default extension (csv) is used

**kwargs

Additional keyword arguments to pass to either: * pandas.DataFrame.to_csv() * json.dump() depending on saving format requested

Raises:
LeaspyIndividualParamsInputError
  • If extension not supported for saving

  • If individual parameters are empty

Parameters:

path (str)

Warning

Emits a warning if no file extension is provided and the default extension is used.

Examples

>>> ip.save("params.csv", index=False)
>>> ip.save("params.json", indent=4)
classmethod load(path)[source]

Static method that loads the individual parameters (json or csv) existing at the path location

Parameters:
pathstr

Path and file name of the individual parameters.

Returns:
IndividualParameters

Individual parameters object load from the file

Raises:
LeaspyIndividualParamsInputError

If the provided extension is not csv or not json.

Parameters:

path (str)

Examples

>>> ip = IndividualParameters.load('/path/to/individual_parameters_1.json')
>>> ip2 = IndividualParameters.load('/path/to/individual_parameters_2.csv')
class Result(data, individual_parameters, noise_std=None)[source]

Result object class. Used as logs by personalize algorithms & simulation algorithm.

Parameters:
dataData

Object containing the information of the individuals, in particular the time-points \((t_{i,j})\) and the observations \((y_{i,j})\).

individual_parametersdict [str, torch.Tensor]

Contains log-acceleration ‘xi’, time-shifts ‘tau’ & ‘sources’

noise_stdfloat or torch.FloatTensor, optional (default None)

Desired noise standard deviation level

Attributes:
dataData

Object containing the information of the individuals, in particular the time-points \((t_{i,j})\) and the observations \((y_{i,j})\).

individual_parametersdict [str, torch.Tensor]

Contains log-acceleration ‘xi’, time-shifts ‘tau’ & ‘sources’ (dictionary of torch.Tensor).

ID_to_idxdict

The keys are the individual ID & the items are their respective ordered position in the data file given by the user. This order remains the same during the computation. Example - in Result.individual_parameters[‘xi’], the first element corresponds to the first patient in ID_to_idx.

noise_stdfloat or torch.FloatTensor

Desired noise standard deviation level.

Parameters:
data
individual_parameters
ID_to_idx: dict[IDType, int]
noise_std = None
get_torch_individual_parameters(ID=None)[source]

Getter function for the individual parameters.

Parameters:
IDstr or list`[:obj:`str], optional (default None)

Contains the identifiers of the wanted subject.

Returns:
dict [str, torch.Tensor]

Contains the individual parameters.

Parameters:

ID (Union[IDType, list[IDType]])

Return type:

DictParamsTorch

get_dataframe_individual_parameters(cofactors=None)[source]

Return the dataframe of the individual parameters.

Each row corresponds to a subject. The columns correspond (in this order) to the subjects’ ID, the individual parameters (one column per individual parameter) & the cofactors (one column per cofactor).

Parameters:
cofactorsstr or list`[:obj:`str], optional (default None)

Contains the cofactor(s) to join to the logs dataframe.

Returns:
pandas.DataFrame

Contains for each patient his ID & his individual parameters (optional and his cofactors states)

Parameters:

cofactors (Union[str, list[str]])

Return type:

DataFrame

Notes

The cofactors must be present in the leaspy data object stored into the .data attribute of the result instance. See the example.

Examples

Load a longitudinal multivariate dataset & the subjects’ cofactors. Compute the individual parameters for this dataset & get the corresponding dataframe with the genetic APOE cofactor

>>> import pandas as pd
>>> from leaspy.api import Leaspy
>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.io.data import Data
>>> from leaspy.io.logs.visualization import Plotter
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')  # replace with your own path!
>>> genes_cofactors = pd.read_csv('data/genes_cofactors.csv')  # replace with your own path!
>>> print(genes_cofactors.head())
           ID      APOE4
0  sub-HS0102          1
1  sub-HS0112          0
2  sub-HS0113          0
3  sub-HS0114          1
4  sub-HS0115          0
>>> data.load_cofactors(genes_cofactors, ['GENES'])
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> individual_results_df = individual_results.get_dataframe_individual_parameters('GENES')
>>> print(individual_results_df.head())
                   tau        xi  sources_0  sources_1  APOE4
ID
sub-HS0102   70.329201  0.120465   5.969921  -0.245034      1
sub-HS0112   95.156624 -0.692099   1.520273   3.477707      0
sub-HS0113   74.900673 -1.769864  -1.222979   1.665889      0
sub-HS0114   81.792763 -1.003620   1.021321   2.371716      1
sub-HS0115   89.724648 -0.820971  -0.480975   0.741601      0
save_individual_parameters_csv(path, idx=None, cofactors=None, **args)[source]

Save the individual parameters in a csv format.

Parameters:
pathstr

The logs’ path.

idxlist [str], optional (default None)

Contain the IDs of the selected subjects. If None, all the subjects are selected.

cofactorsstr or list [str], optional (default None)

Contains the cofactor(s) to join to the logs dataframe.

**args

Parameters to pass to pandas.DataFrame.to_csv().

Parameters:

Notes

The cofactors must be present in the leaspy data object stored into the data attribute of the result instance. See the example.

Examples

Save the individual parameters of the twenty first subjects.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv') # replace with your own path!
>>> genes_cofactors = pd.read_csv('data/genes_cofactors.csv')  # replace with your own path!
>>> data.load_cofactors(genes_cofactors, ['GENES'])
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.csv'
>>> idx = list(individual_results.individual_parameters.keys())[:20]
>>> individual_results.save_individual_parameters_csv(output_path, idx, cofactors='GENES')
save_individual_parameters_json(path, idx=None, human_readable=None, **args)[source]

Save the individual parameters in a json format.

Parameters:
pathstr

The logs’ path.

idxlist [str], optional (default None)

Contain the IDs of the selected subjects. If None, all the subjects are selected.

human_readableAny, optional (default None) –> TODO change to bool

Deprecated since version 1.0:

**args

Arguments to pass to json.dump. Default to: dict(indent=2)

Raises:
NotADirectoryError

if parent directory of path does not exist.

Parameters:

Examples

Save the individual parameters of the twenty first subjects.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json'
>>> idx = list(individual_results.individual_parameters.keys())[:20]
>>> individual_results.save_individual_parameters_json(output_path, idx)
save_individual_parameters_torch(path, idx=None, **args)[source]

Save the individual parameters in a torch format.

Parameters:
pathstr

The logs’ path.

idxlist [str], optional (default None)

Contain the IDs of the selected subjects. If None, all the subjects are selected.

**args

Arguments to pass to torch.save.

Raises:
NotADirectoryError

if parent directory of path does not exist.

Parameters:

Examples

Save the individual parameters of the twenty first subjects.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.pt'
>>> idx = list(individual_results.individual_parameters.keys())[:20]
>>> individual_results.save_individual_parameters_torch(output_path, idx)
classmethod load_individual_parameters_from_csv(path, *, verbose=True, **kwargs)[source]

Load individual parameters from a csv.

Parameters:
pathstr

The file’s path. The csv file musts contain two columns named ‘tau’ and ‘xi’. If the individual parameters come from a multivariate model, it must also contain the columns ‘sources_i’ for i in [0, …, n_sources].

verbosebool (default True)

Whether to have verbose output or not

**kwargs

Parameters to pass to pandas.read_csv().

Returns:
dict [str, torch.Tensor]

A dictionary of torch.tensor which contains the individual parameters.

Parameters:

path (str)

Examples

Load an individual parameters dictionary from a saved file.

>>> from leaspy.io.outputs import Result
>>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.csv'
>>> individual_parameters = Result.load_individual_parameters_from_csv(path)
static load_individual_parameters_from_dataframe(df)[source]

Load individual parameters from a pandas.DataFrame.

Parameters:
dfpandas.DataFrame

Must contain two columns named ‘tau’ and ‘xi’. If the individual parameters come from a multivariate model, it must also contain the columns ‘sources_i’ for i in [0, …, n_sources].

Returns:
dict`[:obj:`str, torch.Tensor]

A dictionary of torch.tensor which contains the individual parameters.

Parameters:

df (DataFrame)

static load_individual_parameters_from_json(path, *, verbose=True, **kwargs)[source]

Load individual parameters from a json file.

Deprecated : also load torch files.

Parameters:
pathstr

The file’s path.

verbosebool (default True)

Whether to have verbose output or not

**kwargs

Parameters to pass to json.load().

Returns:
dict [str, torch.Tensor]

A dictionary of torch.Tensor which contains the individual parameters.

Parameters:

path (str)

Examples

Load an individual parameters dictionary from a saved file.

>>> from leaspy.io.outputs import Result
>>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json'
>>> individual_parameters = Result.load_individual_parameters_from_json(path)
static load_individual_parameters_from_torch(path, *, verbose=True, **kwargs)[source]

Load individual parameters from a torch file.

Parameters:
pathstr

The file’s path.

verbosebool (default True)

Whether to have verbose output or not

**kwargs

Parameters to pass to torch.load().

Returns:
dict [str, torch.Tensor]

A dictionary of torch.Tensor which contains the individual parameters.

Parameters:

path (str)

Examples

Load an individual parameters dictionary from a saved file.

>>> from leaspy.io.outputs import Result
>>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.pt'
>>> individual_parameters = Result.load_individual_parameters_from_torch(path)
classmethod load_individual_parameters(path_or_df, **kwargs)[source]

Load individual parameters from a pandas.DataFrame, a csv, a json file or a torch file.

Parameters:
path_or_dfstr or pandas.DataFrame

The file’s path or a DataFrame containing the individual parameters.

**kwargs

Keyword-arguments to be passed to the corresponding load function.

Returns:
dict [str, torch.Tensor]

A dictionary of torch.tensor which contains the individual parameters.

Raises:
FileNotFoundError

if path is invalid

classmethod load_result(data, individual_parameters, *, cofactors=None, **kwargs)[source]

Load a Result class object from two file - one for the individual data & one for the individual parameters.

Parameters:
datastr or pandas.DataFrame or Data

The file’s path or a DataFrame containing the features’ scores.

individual_parametersstr or pandas.DataFrame

The file’s path or a DataFrame containing the individual parameters.

cofactorsstr or pandas.DataFrame, optional (default None)

The file’s path or a DataFrame containing the individual cofactors. The ID must be in index! Thus, the shape is (n_subjects, n_cofactors).

**kwargs

Parameters to pass to Result.load_individual_parameters method.

Returns:
Result

A Result class object which contains the individual parameters and the individual data.

Examples

Launch an individual parameters estimation, save it and reload it.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.io.outputs import Result
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> path_data = 'data/my_leaspy_data.csv'
>>> path_individual_parameters = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json'
>>> individual_results.data.to_dataframe().to_csv(path_data)
>>> individual_results.save_individual_parameters_json(path_individual_parameters)
>>> individual_parameters = Result.load_result(path_data, path_individual_parameters)
get_error_distribution_dataframe(model, cofactors=None)[source]

Get signed residual distribution per patient, per sub-score & per visit. Each residual is equal to the modeled data minus the observed data.

Parameters:
modelAbstractModel
cofactorsstr, list [str], optional (default None)

Contains the cofactors’ names to be included in the DataFrame. By default, no cofactors are returned. If cofactors == “all”, all the available cofactors are returned.

Returns:
residuals_dataframepandas.DataFrame with index [‘ID’, ‘TIME’]

Examples

Get mean absolute error per feature:

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> data = Data.from_csv_file("/my/data/path")
>>> leaspy_logistic = Leaspy('logistic')
>>> settings = AlgorithmSettings("mcmc_saem", seed=0)
>>> leaspy_logistic.calibrate(data, settings)
>>> settings = AlgorithmSettings("mode_real", seed=0)
>>> results = leaspy_logistic.personalize(data, settings)
>>> residuals_dataframe = results.get_error_distribution_dataframe(model)
>>> residuals_dataframe.abs().mean()
static get_cofactor_states(cofactors)[source]

Deprecated since version 1.0.

Given a list of string return the list of unique elements.

Parameters:
cofactorslist[str]

Distribution list of the cofactors.

Returns:
list

Unique occurrences of the input vector.

Parameters:

cofactors (list)

Return type:

list

get_parameter_distribution(parameter, cofactor=None)[source]

Deprecated since version 1.0.

Return the wanted parameter distribution (one distribution per covariate state).

Parameters:
parameterstr

The wanted parameter’s name (ex: ‘xi’, ‘tau’, …). It can also be sources_i to only get the i-th dimension of multivariate sources parameter.

cofactorstr, optional (default None)

The wanted cofactor’s name.

Returns:
list[float] or dict[str, Any]
Raises:
LeaspyIndividualParamsInputError

if unsupported individual parameters

LeaspyInputError

if unknown cofactor

Parameters:

parameter (ParamType)

Notes

If cofactor is None:
  • If the parameter is univariate => return a list the parameter’s distribution:

    list[float]

  • If the parameter is multivariate => return a dictionary:

    {‘parameter1’: distribution of parameter variable 1, ‘parameter2’: …}

If cofactor is not None:
  • If the parameter is univariate => return a dictionary:

    {‘cofactor1’: parameter distribution such that patient.covariate = covariate1, ‘cofactor2’: …}

  • If the parameter is multivariate => return a dictionary:

    {‘cofactor1’: {‘parameter1’: …, ‘parameter2’: …}, ‘cofactor2’: {…}, …}

get_cofactor_distribution(cofactor)[source]

Deprecated since version 1.0.

Get the list of the cofactor’s distribution.

Parameters:
cofactorstr

Cofactor’s name

Returns:
list

Cofactor’s distribution.

Parameters:

cofactor (str)

get_patient_individual_parameters(idx)[source]

Deprecated since version 1.0.

Get the dictionary of the wanted patient’s individual parameters

Parameters:
idxstr

ID of the wanted patient

Returns:
dict[param_name:str, torch.Tensor]

Patient’s individual parameters

Parameters:

idx (IDType)