Skip to content

Commit 9a8ba56

Browse files
charlesfu4mfeurerfranchuteriveraXD-DENG
authored
Development for multioutput regression (#803)
* PEP8 (#718) * multioutput_regression * multioutput_regression * multioutput_regression * multioutput regression * multioutput regression * multioutput regression * multioutput regression * multioutput regression * #782 showcase pipeline components iteration * Fixed flake-8 violations * multi_output regression v1 * fix y_shape in multioutput regression * fix xy_data_manager change due to merge * automl.py missing import * Release note 070 (#842) * First version of 070 release notes * Missed a bugfix * Vim added unexpected space -- fix * prepare new release (#846) * Clip predict values to [0-1] in classification * Fix for 3.5 python! * Sensible default value of 'score_func' for SelectPercentileRegression (#843) Currently default value of 'score_func' for SelectPercentileRegression is "f_classif", which is an invalid value, and will surely be rejected and will not work * More robust tmp file naming (#854) * More robust tmp file naming * UUID approach * 771 worst possible result (#845) * Initial Commit * Make worst result a function * worst possible result in metric * Fixing the name of the scorers * Add exceptions to log file, not just stdout (#863) * Add exceptions to log file, not just stdout * Removing dummy pred as trys is not needed * Add prediction with models trained with cross-validation (#864) * add the possibility to predict with cross-validation * fix unit tests * test new feature, too * 715 ml memory (#865) * #715 Support for no ml memory limit * API update * Docs enhancement (#862) * Improved docs * Fixed example typos * Beautify examples * cleanup examples * fixed rsa equal * Move to minmax scaler (#866) * Do not read predictions in memory, only after score (#870) * Do not read predictions in memory, only after score * Precission support for string/int * Removal of competition manager (#869) * Removal of competition manager * Removed additional unused methods/files and moved metrics to estimator * Fix meta data generation * Make sure pytest is older newer than 4.6 * Unit tst fixing * flake8 fixes in examples * Fix metadata gen metrics * Fix dataprocessing get params (#877) * Fix dataprocessing get params * Add clone-test to regression pipeline * Allow 1-D threshold binary predictions (#879) * fix single output regression not working * regression need no _enusre_prediction_array_size_prediction_array_sizess * #782 showcase pipeline components iteration * Fixed flake-8 violations * Release note 070 (#842) * First version of 070 release notes * Missed a bugfix * Vim added unexpected space -- fix * prepare new release (#846) * Clip predict values to [0-1] in classification * Fix for 3.5 python! * Sensible default value of 'score_func' for SelectPercentileRegression (#843) Currently default value of 'score_func' for SelectPercentileRegression is "f_classif", which is an invalid value, and will surely be rejected and will not work * More robust tmp file naming (#854) * More robust tmp file naming * UUID approach * 771 worst possible result (#845) * Initial Commit * Make worst result a function * worst possible result in metric * Fixing the name of the scorers * Add exceptions to log file, not just stdout (#863) * Add exceptions to log file, not just stdout * Removing dummy pred as trys is not needed * Add prediction with models trained with cross-validation (#864) * add the possibility to predict with cross-validation * fix unit tests * test new feature, too * 715 ml memory (#865) * #715 Support for no ml memory limit * API update * Docs enhancement (#862) * Improved docs * Fixed example typos * Beautify examples * cleanup examples * fixed rsa equal * Move to minmax scaler (#866) * Do not read predictions in memory, only after score (#870) * Do not read predictions in memory, only after score * Precission support for string/int * Removal of competition manager (#869) * Removal of competition manager * Removed additional unused methods/files and moved metrics to estimator * Fix meta data generation * Make sure pytest is older newer than 4.6 * Unit tst fixing * flake8 fixes in examples * Fix metadata gen metrics * Fix dataprocessing get params (#877) * Fix dataprocessing get params * Add clone-test to regression pipeline * Allow 1-D threshold binary predictions (#879) * multioutput_regression * multioutput_regression * multioutput_regression * multioutput_regression * multioutput_regression * multioutput_regression * multioutput regression * multioutput regression * multioutput regression * multioutput regression * multi_output regression v1 * fix y_shape in multioutput regression * fix xy_data_manager change due to merge * fix single output regression not working * regression need no _enusre_prediction_array_size_prediction_array_sizess * Add prediction with models trained with cross-validation (#864) * add the possibility to predict with cross-validation * fix unit tests * test new feature, too * multioutput_regression * multioutput_regression * multioutput_regression * Removal of competition manager (#869) * Removal of competition manager * Removed additional unused methods/files and moved metrics to estimator * Fix meta data generation * Make sure pytest is older newer than 4.6 * Unit tst fixing * flake8 fixes in examples * Fix metadata gen metrics * multioutput after rebased to 0.7.0 Problem: Cause: Solution: * Regressor target y shape index out of range * Revision for make tester * Revision: Cancel Multiclass-MultiOuput * Resolve automl.py metrics(__init__) reg_gb reg_svm * Fix Flake8 errors * Fix automl.py flake8 * Preprocess w/ mulitout reg,automl self._n_outputs * test_estimator.py changed back * cancel multioutput multiclass for multi reg * Fix automl self._n_output update placement * fix flake8 * Kernel pca cancelled mulitout reg * Kernel PCA test skip python <3.8 * Add test unit for multioutput reg and fix. * Fix flake8 error * Kernel PCA multioutput regression * default kernel to cosine, dodge sklearn=0.22 error * Kernel PCA should be updated to 0.23 * Kernel PCA uses rbf kernel * Kernel Pca * Modify labels in reg, class, perpro in examples * Kernel PCA * Add missing supports to mincoal and truncateSVD Co-authored-by: Matthias Feurer <[email protected]> Co-authored-by: chico <[email protected]> Co-authored-by: Francisco Rivera Valverde <[email protected]> Co-authored-by: Xiaodong DENG <[email protected]>
1 parent b180a37 commit 9a8ba56

File tree

78 files changed

+206
-74
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+206
-74
lines changed

‎autosklearn/automl.py‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
from autosklearn.util.hash import hash_array_or_matrix
3737
from autosklearn.metrics import f1_macro, accuracy, r2
3838
from autosklearn.constants import MULTILABEL_CLASSIFICATION, MULTICLASS_CLASSIFICATION, \
39-
REGRESSION_TASKS, REGRESSION, BINARY_CLASSIFICATION
39+
REGRESSION_TASKS, REGRESSION, BINARY_CLASSIFICATION, MULTIOUTPUT_REGRESSION
4040

4141

4242
def _model_predict(model, X, batch_size, logger, task):
@@ -936,13 +936,16 @@ def _check_X(self, X):
936936

937937
def _check_y(self, y):
938938
y = sklearn.utils.check_array(y, ensure_2d=False)
939-
940939
y = np.atleast_1d(y)
941-
if y.ndim == 2 and y.shape[1] == 1:
940+
941+
if y.ndim == 1:
942+
return y
943+
elif y.ndim == 2 and y.shape[1] == 1:
942944
warnings.warn("A column-vector y was passed when a 1d array was"
943945
" expected. Will change shape via np.ravel().",
944946
sklearn.utils.DataConversionWarning, stacklevel=2)
945947
y = np.ravel(y)
948+
return y
946949

947950
return y
948951

@@ -1097,6 +1100,9 @@ def predict_proba(self, X, batch_size=None, n_jobs=1):
10971100
class AutoMLRegressor(BaseAutoML):
10981101
def __init__(self, *args, **kwargs):
10991102
super().__init__(*args, **kwargs)
1103+
self._task_mapping = {'continuous-multioutput': MULTIOUTPUT_REGRESSION,
1104+
'continuous': REGRESSION,
1105+
'multiclass': REGRESSION}
11001106

11011107
def fit(
11021108
self,
@@ -1110,17 +1116,20 @@ def fit(
11101116
load_models: bool = True,
11111117
):
11121118
X, y = super()._perform_input_checks(X, y)
1113-
_n_outputs = 1 if len(y.shape) == 1 else y.shape[1]
1114-
if _n_outputs > 1:
1115-
raise NotImplementedError(
1116-
'Multi-output regression is not implemented.')
1119+
y_task = type_of_target(y)
1120+
task = self._task_mapping.get(y_task)
1121+
if task is None:
1122+
raise ValueError('Cannot work on data of type %s' % y_task)
1123+
11171124
if self._metric is None:
11181125
self._metric = r2
1126+
1127+
self._n_outputs = 1 if len(y.shape) == 1 else y.shape[1]
11191128
return super().fit(
11201129
X, y,
11211130
X_test=X_test,
11221131
y_test=y_test,
1123-
task=REGRESSION,
1132+
task=task,
11241133
feat_type=feat_type,
11251134
dataset_name=dataset_name,
11261135
only_return_configuration_space=only_return_configuration_space,

‎autosklearn/constants.py‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,9 @@
44
MULTICLASS_CLASSIFICATION = 2
55
MULTILABEL_CLASSIFICATION = 3
66
REGRESSION = 4
7+
MULTIOUTPUT_REGRESSION = 5
78

8-
REGRESSION_TASKS = [REGRESSION]
9+
REGRESSION_TASKS = [REGRESSION, MULTIOUTPUT_REGRESSION]
910
CLASSIFICATION_TASKS = [BINARY_CLASSIFICATION, MULTICLASS_CLASSIFICATION,
1011
MULTILABEL_CLASSIFICATION]
1112

@@ -15,10 +16,12 @@
1516
{BINARY_CLASSIFICATION: 'binary.classification',
1617
MULTICLASS_CLASSIFICATION: 'multiclass.classification',
1718
MULTILABEL_CLASSIFICATION: 'multilabel.classification',
18-
REGRESSION: 'regression'}
19+
REGRESSION: 'regression',
20+
MULTIOUTPUT_REGRESSION: 'multioutput.regression'}
1921

2022
STRING_TO_TASK_TYPES = \
2123
{'binary.classification': BINARY_CLASSIFICATION,
2224
'multiclass.classification': MULTICLASS_CLASSIFICATION,
2325
'multilabel.classification': MULTILABEL_CLASSIFICATION,
24-
'regression': REGRESSION}
26+
'regression': REGRESSION,
27+
'multioutput.regression': MULTIOUTPUT_REGRESSION}

‎autosklearn/data/xy_data_manager.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
from scipy import sparse
55

66
from autosklearn.constants import STRING_TO_TASK_TYPES, REGRESSION, BINARY_CLASSIFICATION, \
7-
MULTICLASS_CLASSIFICATION, MULTILABEL_CLASSIFICATION
7+
MULTICLASS_CLASSIFICATION, MULTILABEL_CLASSIFICATION, MULTIOUTPUT_REGRESSION
88
from autosklearn.data.abstract_data_manager import AbstractDataManager
99

1010

@@ -27,6 +27,7 @@ def __init__(self, X, y, X_test, y_test, task, feat_type, dataset_name):
2727
label_num = {
2828
REGRESSION: 1,
2929
BINARY_CLASSIFICATION: 2,
30+
MULTIOUTPUT_REGRESSION: y.shape[-1],
3031
MULTICLASS_CLASSIFICATION: len(np.unique(y)),
3132
MULTILABEL_CLASSIFICATION: y.shape[-1]
3233
}

‎autosklearn/estimators.py‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -766,15 +766,15 @@ def fit(self, X, y,
766766
X : array-like or sparse matrix of shape = [n_samples, n_features]
767767
The training input samples.
768768
769-
y : array-like, shape = [n_samples]
769+
y : array-like, shape = [n_samples] or [n_samples, n_targets]
770770
The regression target.
771771
772772
X_test : array-like or sparse matrix of shape = [n_samples, n_features]
773773
Test data input samples. Will be used to save test predictions for
774774
all models. This allows to evaluate the performance of Auto-sklearn
775775
over time.
776776
777-
y_test : array-like, shape = [n_samples]
777+
y_test : array-like, shape = [n_samples] or [n_samples, n_targets]
778778
The regression target. Will be used to calculate the test error
779779
of all models. This allows to evaluate the performance of
780780
Auto-sklearn over time.
@@ -799,7 +799,6 @@ def fit(self, X, y,
799799
target_type = type_of_target(y)
800800
if target_type in ['multiclass-multioutput',
801801
'multilabel-indicator',
802-
'continuous-multioutput',
803802
'unknown',
804803
]:
805804
raise ValueError("regression with data of type %s is not"

‎autosklearn/evaluation/abstract_evaluator.py‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
REGRESSION_TASKS,
1616
MULTILABEL_CLASSIFICATION,
1717
MULTICLASS_CLASSIFICATION,
18+
MULTIOUTPUT_REGRESSION
1819
)
1920
from autosklearn.pipeline.implementations.util import (
2021
convert_multioutput_multiclass_to_multilabel
@@ -204,12 +205,19 @@ def _get_model(self):
204205
random_state=self.seed,
205206
init_params=self._init_params)
206207
else:
207-
dataset_properties = {
208-
'task': self.task_type,
209-
'sparse': self.datamanager.info['is_sparse'] == 1,
210-
'multilabel': self.task_type == MULTILABEL_CLASSIFICATION,
211-
'multiclass': self.task_type == MULTICLASS_CLASSIFICATION,
212-
}
208+
if self.task_type in REGRESSION_TASKS:
209+
dataset_properties = {
210+
'task': self.task_type,
211+
'sparse': self.datamanager.info['is_sparse'] == 1,
212+
'multioutput': self.task_type == MULTIOUTPUT_REGRESSION,
213+
}
214+
else:
215+
dataset_properties = {
216+
'task': self.task_type,
217+
'sparse': self.datamanager.info['is_sparse'] == 1,
218+
'multilabel': self.task_type == MULTILABEL_CLASSIFICATION,
219+
'multiclass': self.task_type == MULTICLASS_CLASSIFICATION,
220+
}
213221
model = self.model_class(config=self.configuration,
214222
dataset_properties=dataset_properties,
215223
random_state=self.seed,

‎autosklearn/metrics/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,8 @@ def __call__(self, y_true, y_pred, sample_weight=None):
6060
elif type_true == 'multilabel-indicator':
6161
y_pred[y_pred > 0.5] = 1.0
6262
y_pred[y_pred <= 0.5] = 0.0
63+
elif type_true == 'continuous-multioutput':
64+
pass
6365
else:
6466
raise ValueError(type_true)
6567

‎autosklearn/pipeline/components/base.py‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,8 @@ def add_component(self, obj):
4444
properties = set(classifier.get_properties())
4545
should_be_there = {'shortname', 'name', 'handles_regression',
4646
'handles_classification', 'handles_multiclass',
47-
'handles_multilabel', 'is_deterministic',
48-
'input', 'output'}
47+
'handles_multilabel', 'handles_multioutput',
48+
'is_deterministic', 'input', 'output'}
4949
for property in properties:
5050
if property not in should_be_there:
5151
raise ValueError('Property %s must not be specified for '
@@ -54,7 +54,8 @@ def add_component(self, obj):
5454
(property, name, str(should_be_there)))
5555
for property in should_be_there:
5656
if property not in properties:
57-
raise ValueError('Property %s not specified for algorithm %s')
57+
raise ValueError('Property %s not specified for algorithm %s' %
58+
(property, name))
5859

5960
self.components[name] = classifier
6061

@@ -292,7 +293,7 @@ def predict(self, X):
292293
293294
Returns
294295
-------
295-
array, shape = (n_samples,)
296+
array, shape = (n_samples,) or shape = (n_samples, n_targets)
296297
Returns the predicted values
297298
298299
Notes

‎autosklearn/pipeline/components/classification/adaboost.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ def get_properties(dataset_properties=None):
5757
'handles_classification': True,
5858
'handles_multiclass': True,
5959
'handles_multilabel': False,
60+
'handles_multioutput': False,
6061
'is_deterministic': True,
6162
'input': (DENSE, SPARSE, UNSIGNED_DATA),
6263
'output': (PREDICTIONS,)}

‎autosklearn/pipeline/components/classification/bernoulli_nb.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ def get_properties(dataset_properties=None):
5353
'handles_classification': True,
5454
'handles_multiclass': True,
5555
'handles_multilabel': True,
56+
'handles_multioutput': False,
5657
'is_deterministic': True,
5758
'input': (DENSE, SPARSE, UNSIGNED_DATA),
5859
'output': (PREDICTIONS,)}

‎autosklearn/pipeline/components/classification/decision_tree.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ def get_properties(dataset_properties=None):
8484
'handles_classification': True,
8585
'handles_multiclass': True,
8686
'handles_multilabel': True,
87+
'handles_multioutput': False,
8788
'is_deterministic': True,
8889
'input': (DENSE, SPARSE, UNSIGNED_DATA),
8990
'output': (PREDICTIONS,)}

0 commit comments

Comments
 (0)