is it possible/how incrementally train your code for CategoricalNB Naive Bayes

[Sandy4321]

is it possible incrementally train your code CategoricalNB Naive Bayes

do you have such kind of code example ?

lets say feature N3 has values : abc, hjki, deert, sdfr, qqqq, errttyy, etc

then model was trained initially
after new values was observed : "qwer", "abc", "abc", "abc","errttyy", "qwer","qwer","qwer","qwer"
1
when and how new value "qwer" will be used for prediction
2
how new value "qwer" willbe used for model retraining?
3
how new value "qwer" will be not used for model prediction and how prediction is happen by not using feature N3 when "qwer" observed , until we have many "qwer" observed for all classes ?

when scikit CategoricalNB can not predict / retrain for new values in features
https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.CategoricalNB.html#sklearn.naive_bayes.CategoricalNB

scikit only can deal with streaming data for MultinomialNB, BernoulliNB, and GaussianNB
they call it by "out-of-core-naive-bayes-model-fitting"

https://scikit-learn.org/stable/modules/naive_bayes.html#out-of-core-naive-bayes-model-fitting

MultinomialNB, BernoulliNB, and GaussianNB expose a partial_fit method that can be used incrementally as done with other classifiers

my guess full streaming naive-bayes should identify new values and automatically retrain model to handle this new values .
When only when we have enough observation for particular new values , we can start to use this value for predictions
Since if certain categorical value was observed let say 3 times for class "YES" and 50 times for class no; it can not be used still for prediction . Until at least we have 10 times for class "YES" .???

[raphaelsty]

Hi @Sandy4321, River provides MultinomialNB, BernoulliNB, GaussianNB and ComplementNB as sklearn does but does not provide CategoricalNB. New values and features are handled by Naive Bayes (with River). Models automatically update their states to handle these values. Naive Bayes models tend to rely on the class probability rather than the class probability given the new value when there are not enough samples for a given value, you can increase the parameter alpha to do smoothing (i.e wait for more samples to take a decision) .

[Sandy4321]

output is
befor adding new data classes
[0 1]
before adding new data feature 1
[[-0.40546511 -1.09861229]
[-0.69314718 -0.69314718]]
befor adding new data feature 2
[[-1.94591015 -0.84729786 -0.84729786]
[-0.91629073 -0.91629073 -1.60943791]]
predict z
[0 0 0 0 1 1]
predict v
[1 1 1 1 1 1]
after adding new data classes
[0 1]
after adding new data feature 1
[[-1.44691898 -2.14006616 -2.83321334 -2.83321334 -2.83321334 -2.83321334
-2.83321334 -2.83321334 -2.83321334 -2.83321334 -2.83321334 -2.14006616]
[-2.2512918 -2.2512918 -2.94443898 -2.94443898 -2.94443898 -2.94443898
-2.94443898 -2.94443898 -2.94443898 -2.94443898 -2.94443898 -1.15267951]]
after adding new data feature 2
[[-2.07944154 -0.98082925 -0.69314718]
[-1.60943791 -0.91629073 -0.91629073]]

[Sandy4321]

seems to be they not ignore new values, but add it by very strange method - add many dummy values

[MaxHalford]

Please @Sandy4321, can you put some effort into your messages? At least highlight and format the code properly. Currently it's very painful for me to understand what you're trying to communicate.

[Sandy4321]

though problem with new classes is here
befor adding new data classes
[0 1]
after adding new data classes
[0 1]
classes are the same before and after adding new class

[Sandy4321]

though it is not clear what they mean under
min_categories

https://scikit-learn.org/0.24/whats_new/v0.24.html?highlight=categoricalnb

Enhancement Adds a parameter min_categories to naive_bayes.CategoricalNB that allows a minimum number of categories per feature to be specified. This allows categories unseen during training to be accounted for. #16326 by George Armstrong.

[Sandy4321]

see also my comments to
scikit-learn/scikit-learn#16326

about new data error-
import numpy as np
rng = np.random.RandomState(1)
X = rng.randint(5, size=(4, 7))
y = np.array([1, 0, 1, 1])
from sklearn.naive_bayes import CategoricalNB
clf = CategoricalNB()
clf.min_categories = 2
clf.fit(X, y)
X=
array([[3, 4, 0, 1, 3, 0, 0],
[1, 4, 4, 1, 2, 4, 2],
[4, 3, 4, 2, 4, 2, 4],
[1, 1, 0, 1, 1, 1, 1]])
test = [[4, 3, 4, 2, 4, 2, 123]]
print(clf.predict(test))

Traceback (most recent call last):
Debug Console, prompt 30, line 11
clf.fit(X1, y1)
File "e:\Python environemnts\MYENVS\may9_sklearn0242\Lib\site-packages\sklearn\naive_bayes.py", line 75, in predict
jll = self._joint_log_likelihood(X)
File "e:\Python environemnts\MYENVS\may9_sklearn0242\Lib\site-packages\sklearn\naive_bayes.py", line 1303, in joint_log_likelihood
jll += self.feature_log_prob[i][:, indices].T
builtins.IndexError: index 123 is out of bounds for axis 1 with size 5

[Sandy4321]

also it limiting that scikit represent values only in integer form

when actual data as rule is in string form
then as they suggests to use OrdinalEncoder
that it become cumbersome to update OrdinalEncoder each time new data values coming ...
better to do native string input data with proper treating new values : retrain and and when we have enough new values predict with using new values
but until it happen - just ignore features with new values for given row prediction ?

https://github.com/gwarmstrong
@gwarmstrong
@prabhat00155
@madhuracj
https://github.com/madhuracj
https://github.com/prabhat00155
https://github.com/glemaitre

@glemaitre
https://github.com/jnothman
@jnothman
Though a more efficient solution would be to use an OrdinalEncoder on your data, which will correctly identify the actual number of categories in the data, and provide the ability to set values not seen in training data by using the unknown_value in OrdinalEncoder. Then min_categories can be set to number_of_features_in_training_data + 1

[Sandy4321]

maybe lets do it one by one

[Sandy4321]

you wrote:

In the meantime, I would suggest using River's
BernoulliNB
and one-hot encoding yourcategories. It should do a decent job.

but to do on line onehot encoder is more difficult than proper categorical NB?
in any case may you share code example for such at solution , pls
When new values treated properly by one hot encoder -
new columns are added to one hot data as new values coming ?

[Sandy4321]

seems to be you can use code ideas from https://github.com/geofizx/Streaming-Naive-Bayes

[ChandraPrakash-Bathula]

CategoricalNB in scikit-learn does not have a direct method for incremental training or partial fitting like MultinomialNB, BernoulliNB, and GaussianNB. I think, CategoricalNB does not provide a partial_fit method.

However, you can achieve incremental training for categorical data by manually updating the parameters of the model when new data becomes available.