leaspy.io.outputs¶
Submodules¶
Classes¶
Data container for individual parameters, contains IDs, timepoints and observations values. |
|
Result object class. |
Package Contents¶
- class IndividualParameters[source]¶
Data container for individual parameters, contains IDs, timepoints and observations values. Output of the
leaspy.algo.personalizemethod, contains the random effects.There are used as output of the personalization algorithms and as input/output of the simulation algorithm, to provide an initial distribution of individual parameters.
- Attributes:
- _indices
list List of the patient indices
- _individual_parameters
dict Individual indices (key) with their corresponding individual parameters {parameter name: parameter value}
- _parameters_shape
dict Shape of each individual parameter
- _default_saving_type
str Default extension for saving when none is provided
- _indices
- VALID_IO_EXTENSIONS = ['csv', 'json']¶
- add_individual_parameters(index, individual_parameters)[source]¶
Add the individual parameter of an individual to the IndividualParameters object
- Parameters:
- index:class:
IDType Index of the individual
- individual_parameters
DictParams Individual parameters of the individual
- index:class:
- Raises:
LeaspyIndividualParamsInputErrorIf the index is not a string or has already been added
Or if the individual parameters is not a dict.
Or if individual parameters are not self-consistent.
- Parameters:
index (IDType)
individual_parameters (DictParams)
Examples
Add two individual with tau, xi and sources parameters
>>> ip = IndividualParameters() >>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]}) >>> ip.add_individual_parameters('index-2', {"xi": 0.2, "tau": 73, "sources": [-0.4, -0.1]})
- items()[source]¶
Get the items of the individual parameters dictionary.
- Returns:
- ItemsView
A view object displaying a list of tuples (individual ID, parameters dict)
Examples
>>> ip = IndividualParameters() >>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]}) >>> list(ip.items()) ['index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]}]
- subset(indices, *, copy=True)[source]¶
Returns IndividualParameters object with a subset of the initial individuals
- Parameters:
- indices
Iterable`[:class:`~leaspy.utils.typing.IDType] List of strings that corresponds to the indices of the individuals to return
- copy
bool, optional (default True) Should we copy underlying parameters or not?
- indices
- Returns:
IndividualParametersAn instance of the IndividualParameters object with the selected list of individuals
- Raises:
LeaspyIndividualParamsInputErrorRaise an error if one of the index is not in the IndividualParameters
- Parameters:
Examples
>>> ip = IndividualParameters() >>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]}) >>> ip.add_individual_parameters('index-2', {"xi": 0.2, "tau": 73, "sources": [-0.4, -0.1]}) >>> ip.add_individual_parameters('index-3', {"xi": 0.3, "tau": 58, "sources": [-0.6, 0.2]}) >>> ip_sub = ip.subset(['index-1', 'index-3'])
- get_aggregate(parameter, function)[source]¶
Returns the result of aggregation by function of parameter values across all patients
- Parameters:
- parameter
ParamType Name of the parameter
- function
Callable A function operating on iterables and supporting axis keyword, and outputing an iterable supporting the tolist method.
- parameter
- Returns:
listResulting value of the parameter
- Raises:
LeaspyIndividualParamsInputErrorIf individual parameters are empty,
or if the parameter is not in the IndividualParameters.
- Parameters:
parameter (ParamType)
function (Callable)
- Return type:
Examples
>>> ip = IndividualParameters.load("path/to/individual_parameters") >>> tau_median = ip.get_aggregate("tau", np.median)
- get_mean(parameter)[source]¶
Returns the mean value of a parameter across all patients
- Parameters:
- parameter
ParamType Name of the parameter
- parameter
- Returns:
listMean value of the parameter
- Raises:
LeaspyIndividualParamsInputErrorIf individual parameters are empty,
or if the parameter is not in the IndividualParameters.
- Parameters:
parameter (ParamType)
Examples
>>> ip = IndividualParameters.load("path/to/individual_parameters") >>> tau_mean = ip.get_mean("tau")
- get_std(parameter)[source]¶
Returns the standard deviation of a parameter across all patients
- Parameters:
- parameter
ParamType Name of the parameter
- parameter
- Returns:
listStandard-deviation value of the parameter
- Raises:
LeaspyIndividualParamsInputErrorIf individual parameters are empty,
or if the parameter is not in the IndividualParameters.
- Parameters:
parameter (ParamType)
Examples
>>> ip = IndividualParameters.load("path/to/individual_parameters") >>> tau_std = ip.get_std("tau")
- to_dataframe()[source]¶
Returns the dataframe of individual parameters
- Returns:
pandas.DataFrameEach row corresponds to one individual. The index corresponds to the individual index (‘ID’). The columns are the names of the parameters.
- Return type:
Examples
Convert the individual parameters object into a dataframe
>>> ip = IndividualParameters.load("path/to/individual_parameters") >>> ip_df = ip.to_dataframe()
- static from_dataframe(df)[source]¶
Static method that returns an IndividualParameters object from the dataframe
- Parameters:
- df
pandas.DataFrame Dataframe of the individual parameters. Each row must correspond to one individual. The index corresponds to the individual index. The columns are the names of the parameters.
- df
- Returns:
IndividualParametersAn instance of IndividualParameters initialized from the DataFrame.
- Parameters:
df (DataFrame)
Examples
>>> import pandas as pd >>> data = { >>> 'tau': [70, 73], >>> 'xi': [0.1, 0.2], >>> 'sources_0': [0.1, -0.4], >>> 'sources_1': [-0.3, -0.1] >>> } >>> df = pd.DataFrame(data, index=['id1', 'id2']) >>> ip = IndividualParameters.from_dataframe(df)
- static from_pytorch(indices, dict_pytorch)[source]¶
Static method that returns an IndividualParameters object from the indices and pytorch dictionary
- Parameters:
- indices
list`[:class:`~leaspy.utils.typing.IDType] List of the patients indices
- dict_pytorch
DictParmasTorch Dictionary of the individual parameters
- indices
- Returns:
IndividualParametersAn instance of IndividualParameters initialized from the pytorch dictionary.
- Raises:
- Parameters:
dict_pytorch (DictParamsTorch)
Examples
>>> indices = ['index-1', 'index-2', 'index-3'] >>> ip_pytorch = { >>> "xi": torch.tensor([[0.1], [0.2], [0.3]], dtype=torch.float32), >>> "tau": torch.tensor([[70], [73], [58.]], dtype=torch.float32), >>> "sources": torch.tensor([[0.1, -0.3], [-0.4, 0.1], [-0.6, 0.2]], dtype=torch.float32) >>> } >>> ip_pytorch = IndividualParameters.from_pytorch(indices, ip_pytorch)
- to_pytorch()[source]¶
Returns the indices and pytorch dictionary of individual parameters
- Returns:
- indices:
list`[:class:`~leaspy.utils.typing.IDType] List of patient indices
- pytorch_dict:
DictParamsTorch Dictionary of the individual parameters {parameter name: pytorch tensor of values across individuals}
- indices:
- Return type:
Examples
Convert the individual parameters object into a dataframe
>>> ip = IndividualParameters.load("path/to/individual_parameters") >>> indices, ip_pytorch = ip.to_pytorch()
- save(path, **kwargs)[source]¶
Saves the individual parameters (json or csv) at the path location
TODO? save leaspy version as well for retro/future-compatibility issues?
- Parameters:
- path
str Path and file name of the individual parameters. The extension can be json or csv. If no extension, default extension (csv) is used
- **kwargs
Additional keyword arguments to pass to either: *
pandas.DataFrame.to_csv()*json.dump()depending on saving format requested
- path
- Raises:
LeaspyIndividualParamsInputErrorIf extension not supported for saving
If individual parameters are empty
- Parameters:
path (str)
Warning
Emits a warning if no file extension is provided and the default extension is used.
Examples
>>> ip.save("params.csv", index=False) >>> ip.save("params.json", indent=4)
- classmethod load(path)[source]¶
Static method that loads the individual parameters (json or csv) existing at the path location
- Parameters:
- path
str Path and file name of the individual parameters.
- path
- Returns:
IndividualParametersIndividual parameters object load from the file
- Raises:
LeaspyIndividualParamsInputErrorIf the provided extension is not csv or not json.
- Parameters:
path (str)
Examples
>>> ip = IndividualParameters.load('/path/to/individual_parameters_1.json') >>> ip2 = IndividualParameters.load('/path/to/individual_parameters_2.csv')
- class Result(data, individual_parameters, noise_std=None)[source]¶
Result object class. Used as logs by personalize algorithms & simulation algorithm.
- Parameters:
- data
Data Object containing the information of the individuals, in particular the time-points \((t_{i,j})\) and the observations \((y_{i,j})\).
- individual_parameters
dict[str,torch.Tensor] Contains log-acceleration ‘xi’, time-shifts ‘tau’ & ‘sources’
- noise_std
floatortorch.FloatTensor, optional (default None) Desired noise standard deviation level
- data
- Attributes:
- data
Data Object containing the information of the individuals, in particular the time-points \((t_{i,j})\) and the observations \((y_{i,j})\).
- individual_parameters
dict[str,torch.Tensor] Contains log-acceleration ‘xi’, time-shifts ‘tau’ & ‘sources’ (dictionary of torch.Tensor).
- ID_to_idx
dict The keys are the individual ID & the items are their respective ordered position in the data file given by the user. This order remains the same during the computation. Example - in Result.individual_parameters[‘xi’], the first element corresponds to the first patient in ID_to_idx.
- noise_std
floatortorch.FloatTensor Desired noise standard deviation level.
- data
- Parameters:
data (Data)
individual_parameters (DictParamsTorch)
- data¶
- individual_parameters¶
- noise_std = None¶
- get_torch_individual_parameters(ID=None)[source]¶
Getter function for the individual parameters.
- Parameters:
- ID
strorlist`[:obj:`str], optional (default None) Contains the identifiers of the wanted subject.
- ID
- Returns:
dict[str,torch.Tensor]Contains the individual parameters.
- Parameters:
- Return type:
- get_dataframe_individual_parameters(cofactors=None)[source]¶
Return the dataframe of the individual parameters.
Each row corresponds to a subject. The columns correspond (in this order) to the subjects’ ID, the individual parameters (one column per individual parameter) & the cofactors (one column per cofactor).
- Parameters:
- cofactors
strorlist`[:obj:`str], optional (default None) Contains the cofactor(s) to join to the logs dataframe.
- cofactors
- Returns:
pandas.DataFrameContains for each patient his ID & his individual parameters (optional and his cofactors states)
- Parameters:
- Return type:
Notes
The cofactors must be present in the leaspy data object stored into the .data attribute of the result instance. See the example.
Examples
Load a longitudinal multivariate dataset & the subjects’ cofactors. Compute the individual parameters for this dataset & get the corresponding dataframe with the genetic APOE cofactor
>>> import pandas as pd >>> from leaspy.api import Leaspy >>> from leaspy.algo import AlgorithmSettings >>> from leaspy.io.data import Data >>> from leaspy.io.logs.visualization import Plotter >>> leaspy_logistic = Leaspy('logistic') >>> data = Data.from_csv_file('data/my_leaspy_data.csv') # replace with your own path! >>> genes_cofactors = pd.read_csv('data/genes_cofactors.csv') # replace with your own path! >>> print(genes_cofactors.head()) ID APOE4 0 sub-HS0102 1 1 sub-HS0112 0 2 sub-HS0113 0 3 sub-HS0114 1 4 sub-HS0115 0
>>> data.load_cofactors(genes_cofactors, ['GENES']) >>> model_settings = AlgorithmSettings('mcmc_saem', seed=0) >>> personalize_settings = AlgorithmSettings('mode_real', seed=0) >>> leaspy_logistic.fit(data, model_settings) >>> individual_results = leaspy_logistic.personalize(data, model_settings) >>> individual_results_df = individual_results.get_dataframe_individual_parameters('GENES') >>> print(individual_results_df.head()) tau xi sources_0 sources_1 APOE4 ID sub-HS0102 70.329201 0.120465 5.969921 -0.245034 1 sub-HS0112 95.156624 -0.692099 1.520273 3.477707 0 sub-HS0113 74.900673 -1.769864 -1.222979 1.665889 0 sub-HS0114 81.792763 -1.003620 1.021321 2.371716 1 sub-HS0115 89.724648 -0.820971 -0.480975 0.741601 0
- save_individual_parameters_csv(path, idx=None, cofactors=None, **args)[source]¶
Save the individual parameters in a csv format.
- Parameters:
- path
str The logs’ path.
- idx
list[str], optional (default None) Contain the IDs of the selected subjects. If
None, all the subjects are selected.- cofactors
strorlist[str], optional (default None) Contains the cofactor(s) to join to the logs dataframe.
- **args
Parameters to pass to
pandas.DataFrame.to_csv().
- path
- Parameters:
Notes
The cofactors must be present in the leaspy data object stored into the
dataattribute of the result instance. See the example.Examples
Save the individual parameters of the twenty first subjects.
>>> from leaspy.algo import AlgorithmSettings >>> from leaspy.api import Leaspy >>> from leaspy.io.data import Data >>> leaspy_logistic = Leaspy('logistic') >>> data = Data.from_csv_file('data/my_leaspy_data.csv') # replace with your own path! >>> genes_cofactors = pd.read_csv('data/genes_cofactors.csv') # replace with your own path! >>> data.load_cofactors(genes_cofactors, ['GENES']) >>> model_settings = AlgorithmSettings('mcmc_saem', seed=0) >>> personalize_settings = AlgorithmSettings('mode_real', seed=0) >>> leaspy_logistic.fit(data, model_settings) >>> individual_results = leaspy_logistic.personalize(data, model_settings) >>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.csv' >>> idx = list(individual_results.individual_parameters.keys())[:20] >>> individual_results.save_individual_parameters_csv(output_path, idx, cofactors='GENES')
- save_individual_parameters_json(path, idx=None, human_readable=None, **args)[source]¶
Save the individual parameters in a json format.
- Parameters:
- path
str The logs’ path.
- idx
list[str], optional (default None) Contain the IDs of the selected subjects. If
None, all the subjects are selected.- human_readableAny, optional (default None) –> TODO change to bool
Deprecated since version 1.0:
If None (default): save as json file
If not None: call
save_individual_parameters_torch().
- **args
Arguments to pass to json.dump. Default to: dict(indent=2)
- path
- Raises:
NotADirectoryErrorif parent directory of path does not exist.
- Parameters:
Examples
Save the individual parameters of the twenty first subjects.
>>> from leaspy.algo import AlgorithmSettings >>> from leaspy.api import Leaspy >>> from leaspy.io.data import Data >>> leaspy_logistic = Leaspy('logistic') >>> data = Data.from_csv_file('data/my_leaspy_data.csv') >>> model_settings = AlgorithmSettings('mcmc_saem', seed=0) >>> personalize_settings = AlgorithmSettings('mode_real', seed=0) >>> leaspy_logistic.fit(data, model_settings) >>> individual_results = leaspy_logistic.personalize(data, model_settings) >>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json' >>> idx = list(individual_results.individual_parameters.keys())[:20] >>> individual_results.save_individual_parameters_json(output_path, idx)
- save_individual_parameters_torch(path, idx=None, **args)[source]¶
Save the individual parameters in a torch format.
- Parameters:
- Raises:
NotADirectoryErrorif parent directory of path does not exist.
- Parameters:
Examples
Save the individual parameters of the twenty first subjects.
>>> from leaspy.algo import AlgorithmSettings >>> from leaspy.api import Leaspy >>> from leaspy.io.data import Data >>> leaspy_logistic = Leaspy('logistic') >>> data = Data.from_csv_file('data/my_leaspy_data.csv') >>> model_settings = AlgorithmSettings('mcmc_saem', seed=0) >>> personalize_settings = AlgorithmSettings('mode_real', seed=0) >>> leaspy_logistic.fit(data, model_settings) >>> individual_results = leaspy_logistic.personalize(data, model_settings) >>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.pt' >>> idx = list(individual_results.individual_parameters.keys())[:20] >>> individual_results.save_individual_parameters_torch(output_path, idx)
- classmethod load_individual_parameters_from_csv(path, *, verbose=True, **kwargs)[source]¶
Load individual parameters from a csv.
- Parameters:
- path
str The file’s path. The csv file musts contain two columns named ‘tau’ and ‘xi’. If the individual parameters come from a multivariate model, it must also contain the columns ‘sources_i’ for i in [0, …, n_sources].
- verbose
bool(default True) Whether to have verbose output or not
- **kwargs
Parameters to pass to
pandas.read_csv().
- path
- Returns:
dict[str,torch.Tensor]A dictionary of torch.tensor which contains the individual parameters.
- Parameters:
path (str)
Examples
Load an individual parameters dictionary from a saved file.
>>> from leaspy.io.outputs import Result >>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.csv' >>> individual_parameters = Result.load_individual_parameters_from_csv(path)
- static load_individual_parameters_from_dataframe(df)[source]¶
Load individual parameters from a
pandas.DataFrame.- Parameters:
- df
pandas.DataFrame Must contain two columns named ‘tau’ and ‘xi’. If the individual parameters come from a multivariate model, it must also contain the columns ‘sources_i’ for i in [0, …, n_sources].
- df
- Returns:
dict`[:obj:`str,torch.Tensor]A dictionary of torch.tensor which contains the individual parameters.
- Parameters:
df (DataFrame)
- static load_individual_parameters_from_json(path, *, verbose=True, **kwargs)[source]¶
Load individual parameters from a json file.
Deprecated : also load torch files.
- Parameters:
- path
str The file’s path.
- verbose
bool(default True) Whether to have verbose output or not
- **kwargs
Parameters to pass to
json.load().
- path
- Returns:
dict[str,torch.Tensor]A dictionary of torch.Tensor which contains the individual parameters.
- Parameters:
path (str)
Examples
Load an individual parameters dictionary from a saved file.
>>> from leaspy.io.outputs import Result >>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json' >>> individual_parameters = Result.load_individual_parameters_from_json(path)
- static load_individual_parameters_from_torch(path, *, verbose=True, **kwargs)[source]¶
Load individual parameters from a torch file.
- Parameters:
- path
str The file’s path.
- verbose
bool(default True) Whether to have verbose output or not
- **kwargs
Parameters to pass to
torch.load().
- path
- Returns:
dict[str,torch.Tensor]A dictionary of torch.Tensor which contains the individual parameters.
- Parameters:
path (str)
Examples
Load an individual parameters dictionary from a saved file.
>>> from leaspy.io.outputs import Result >>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.pt' >>> individual_parameters = Result.load_individual_parameters_from_torch(path)
- classmethod load_individual_parameters(path_or_df, **kwargs)[source]¶
Load individual parameters from a
pandas.DataFrame, a csv, a json file or a torch file.- Parameters:
- path_or_dfstr or
pandas.DataFrame The file’s path or a DataFrame containing the individual parameters.
- **kwargs
Keyword-arguments to be passed to the corresponding load function.
- path_or_dfstr or
- Returns:
dict[str,torch.Tensor]A dictionary of torch.tensor which contains the individual parameters.
- Raises:
FileNotFoundErrorif path is invalid
- classmethod load_result(data, individual_parameters, *, cofactors=None, **kwargs)[source]¶
Load a Result class object from two file - one for the individual data & one for the individual parameters.
- Parameters:
- data
strorpandas.DataFrameorData The file’s path or a DataFrame containing the features’ scores.
- individual_parameters
strorpandas.DataFrame The file’s path or a DataFrame containing the individual parameters.
- cofactors
strorpandas.DataFrame, optional (default None) The file’s path or a DataFrame containing the individual cofactors. The ID must be in index! Thus, the shape is (n_subjects, n_cofactors).
- **kwargs
Parameters to pass to Result.load_individual_parameters method.
- data
- Returns:
ResultA Result class object which contains the individual parameters and the individual data.
Examples
Launch an individual parameters estimation, save it and reload it.
>>> from leaspy.algo import AlgorithmSettings >>> from leaspy.io.outputs import Result >>> from leaspy.api import Leaspy >>> from leaspy.io.data import Data >>> leaspy_logistic = Leaspy('logistic') >>> data = Data.from_csv_file('data/my_leaspy_data.csv') >>> model_settings = AlgorithmSettings('mcmc_saem', seed=0) >>> personalize_settings = AlgorithmSettings('mode_real', seed=0) >>> leaspy_logistic.fit(data, model_settings) >>> individual_results = leaspy_logistic.personalize(data, model_settings) >>> path_data = 'data/my_leaspy_data.csv' >>> path_individual_parameters = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json' >>> individual_results.data.to_dataframe().to_csv(path_data) >>> individual_results.save_individual_parameters_json(path_individual_parameters) >>> individual_parameters = Result.load_result(path_data, path_individual_parameters)
- get_error_distribution_dataframe(model, cofactors=None)[source]¶
Get signed residual distribution per patient, per sub-score & per visit. Each residual is equal to the modeled data minus the observed data.
- Parameters:
- model
AbstractModel - cofactorsstr, list [str], optional (default None)
Contains the cofactors’ names to be included in the DataFrame. By default, no cofactors are returned. If cofactors == “all”, all the available cofactors are returned.
- model
- Returns:
- residuals_dataframe
pandas.DataFramewith index [‘ID’, ‘TIME’]
- residuals_dataframe
Examples
Get mean absolute error per feature:
>>> from leaspy.algo import AlgorithmSettings >>> from leaspy.api import Leaspy >>> from leaspy.io.data import Data >>> data = Data.from_csv_file("/my/data/path") >>> leaspy_logistic = Leaspy('logistic') >>> settings = AlgorithmSettings("mcmc_saem", seed=0) >>> leaspy_logistic.calibrate(data, settings) >>> settings = AlgorithmSettings("mode_real", seed=0) >>> results = leaspy_logistic.personalize(data, settings) >>> residuals_dataframe = results.get_error_distribution_dataframe(model) >>> residuals_dataframe.abs().mean()
- static get_cofactor_states(cofactors)[source]¶
Deprecated since version 1.0.
Given a list of string return the list of unique elements.
- get_parameter_distribution(parameter, cofactor=None)[source]¶
Deprecated since version 1.0.
Return the wanted parameter distribution (one distribution per covariate state).
- Parameters:
- parameterstr
The wanted parameter’s name (ex: ‘xi’, ‘tau’, …). It can also be sources_i to only get the i-th dimension of multivariate sources parameter.
- cofactorstr, optional (default None)
The wanted cofactor’s name.
- Returns:
- list[float] or dict[str, Any]
- Raises:
LeaspyIndividualParamsInputErrorif unsupported individual parameters
LeaspyInputErrorif unknown cofactor
- Parameters:
parameter (ParamType)
Notes
- If
cofactor is None: - If the parameter is univariate => return a list the parameter’s distribution:
list[float]
- If the parameter is multivariate => return a dictionary:
{‘parameter1’: distribution of parameter variable 1, ‘parameter2’: …}
- If
cofactor is not None: - If the parameter is univariate => return a dictionary:
{‘cofactor1’: parameter distribution such that patient.covariate = covariate1, ‘cofactor2’: …}
- If the parameter is multivariate => return a dictionary:
{‘cofactor1’: {‘parameter1’: …, ‘parameter2’: …}, ‘cofactor2’: {…}, …}