Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Apr 09 17:15
    odmarkj opened #299
  • Apr 05 11:30
    asmaafawzy25 reopened #298
  • Apr 05 11:30
    asmaafawzy25 closed #298
  • Mar 31 14:24
    asmaafawzy25 opened #298
  • Mar 08 10:06
    Linfengscat opened #297
  • Mar 05 20:31
    ginop commented #281
  • Mar 02 01:14
    gilbertoolimpio closed #296
  • Mar 02 00:45
    gilbertoolimpio edited #296
  • Mar 01 21:27
    gilbertoolimpio opened #296
  • Mar 01 20:53
    gilbertoolimpio commented #295
  • Mar 01 20:52
    gilbertoolimpio opened #295
  • Feb 26 09:38
    michaelchiucw closed #293
  • Feb 23 16:35
    shubhamsoniXom closed #294
  • Feb 23 16:33
    shubhamsoniXom opened #294
  • Feb 04 08:21
    michaelchiucw edited #293
  • Feb 04 08:21
    michaelchiucw edited #293
  • Feb 04 08:21
    michaelchiucw edited #293
  • Feb 04 04:55
    michaelchiucw edited #293
  • Feb 04 04:54
    michaelchiucw edited #293
  • Feb 04 04:54
    michaelchiucw edited #293
Mariam Benllarch
@benllarch
Hello @jacobmontiel, is there a mechanism for hyperparameter tuning, I'm working with EFDT algorithm, and I wanna experiment its performance by testing some values for tie_threshold, grace_period, and Delta parameters.
farnaz
@farnaz2018
Hi everyone,
I got this error when I am using Hoeffding adaptive tree, any idea?
TypeError: _new_learning_node() got an unexpected keyword argument 'is_active_node'
the base is from python3.8/site-packages/skmultiflow/trees/hoeffding_tree.py", line 805, in _deactivate_learning_node
new_leaf = self._new_learning_node(
2 replies
Jacob Pfeil
@jpfeil
Hello! I have been using XGBoost, but in production, it makes more sense to use an online algorithm. I am trying several different ensemble approaches in scikit-multiflow, but I'm a little overwhelmed with all of the options. I was wondering if anyone could recommend a few recipes to try out on my data.
Claudio
@claudiobrandy
Greetings to all! I'm new to the channel, got the link on the scikit multiflow page. I'm trying to implement an online learning model to detect anomalies on security logs using multiflow algorithms, but I'm finding performance issues when comparing with similar methods in scikit learn. It gets better when I accumulate enough samples before predicting and partial fitting, but that approach isn't always feasible. Did you have any similar experience? Am I asking too much and shouldn't expect comparable performance?
asad1907
@asad1907
@benllarch hi, did you find any method for hyperparameter tuning? I am interested the same issue too but cant find any solution yet.
I have one more question , is there any feature selection method in scikit-multiflow?
brainsensors
@brainsensors
Hi, is there any example to calculate roc and auc for HalfSpaceTree in skmultiflow as i was looking at the library of 'metrics' they didn't specified anything.
Venoli
@Venoli
Hi @jacobmontiel, is there any way to use Hellinger Distance as split criterion with ExtremelyFastDecisionTreeClassifier? (I actually want to use ExtremelyFastDecisionTreeClassifier for imbalanced dataset.)
brainsensors
@brainsensors
@jacobmontiel, Hi, is there any example to calculate roc and auc for HalfSpaceTree in skmultiflow as i was looking at the library of 'metrics' they didn't specified anything.
Ecehan SAVAŞ BAŞAK
@ecehansavas

Hi everyone,

In my project, I'm using Scikit-Multiflow 0.5.3 with Python 3.6. I tried to read Kdd Cup99 dataset with FileStream method. But in the label column that is text-based (includes "Normal", "Teardrop", etc.), the scikit-multiflow gives me

"dtype: objectscikit-multiflow only supports numeric data." error. I tried to set the label column as target_idx, and also tried setting that column as categories but none of them worked. How can I make scikit-multiflow work for this dataset?

The full error looks like this:

 File "<projdir>/site-packages/skmultiflow/data/file_stream.py", line 99, in __init__
    self._prepare_for_use()
  File "<projdir>/site-packages/skmultiflow/data/file_stream.py", line 177, in _prepare_for_use
    self._load_data()
  File "<projdir>/site-packages/skmultiflow/data/file_stream.py", line 185, in _load_data
    check_data_consistency(raw_data, self.allow_nan)
  File "<projdir>/site-packages/skmultiflow/data/data_stream.py", line 443, in check_data_consistency
    .format(raw_data_frame.dtypes))
ValueError: Non-numeric data found:
 duration                         int64
src_bytes                        int64
dst_bytes                        int64
wrong_fragment                   int64
urgent                           int64
hot                              int64
num_failed_logins                int64
num_compromised                  int64
root_shell                       int64
su_attempted                     int64
num_root                         int64
num_file_creations               int64
num_shells                       int64
num_access_files                 int64
num_outbound_cmds                int64
count                            int64
srv_count                        int64
serror_rate                    float64
srv_serror_rate                float64
rerror_rate                    float64
srv_rerror_rate                float64
same_srv_rate                  float64
diff_srv_rate                  float64
srv_diff_host_rate             float64
dst_host_count                   int64
dst_host_srv_count               int64
dst_host_same_srv_rate         float64
dst_host_diff_srv_rate         float64
dst_host_same_src_port_rate    float64
dst_host_srv_diff_host_rate    float64
dst_host_serror_rate           float64
dst_host_srv_serror_rate       float64
dst_host_rerror_rate           float64
dst_host_srv_rerror_rate       float64
label                           object
dtype: objectscikit-multiflow only supports numeric data.

Thanks,

1 reply
Miso-K
@Miso-K
Hello, @ecehansavas you can use my function read_kdd_data_multilable from https://github.com/Miso-K/DDCW/blob/master/utils/data_preprocesing.py and then:
data, X, y = read_kdd_data_multilable('./data/kddcup.data_10_percent_corrected.csv'),
stream = DataStream(X, y),
stream.prepare_for_use()
rajeevraibhatia
@rajeevraibhatia
@jacobmontiel QQ: For Anomaly Detection with HalfSpace Trees, how can we load streaming data point by point (i.e. read one datapoint every minute) and feed that into the model instead of loading batch data from a csv?
3 replies
Adrien Luxey
@Adrien-Luxey
Hi people :)
I'm wondering how you make random_state work as expected in the online learning context.
The difference between you and sklearn is that you don't reset all parameters everytime you call fit, isn't it?
I'm writing my own online classifier, and I'd like to keep their BaseEstimator structure like you do (super practical for cross validation), but I'm struggling with an ensemble estimator (that should use its random_state to fix the base estimators'). Any advice? Thanks, great library :)
Arg. Your HoeffdingTreeClassifier does not have a random_state parameter, for instance. So how do you ensure experiments' reproducibility? Is HoeffdingTreeClassifier deterministic?
3 replies
Adrien Luxey
@Adrien-Luxey
AdaptiveRandomForestClassifier will be a better example. I'll dig the code. Answers still appreciated :)
Hassan Mehmood
@hassanmehmud
Hello,
is add element sensitive to 0,1 only or it can be any floating value in KSWIN
@jacobmontiel
Saulo Martiello Mastelini
@smastelini
Hi everyone, as stated in the github page, skmultiflow merged with creme to become https://riverml.xyz/latest/

So, there's no active development in skmultiflow anymore. We invite all the users to check river :D

It's more than the sum of the two projects

We are waiting for the name river be made available in pip, so that we can make an official release and statement
but since now I see a lot of messages in this community channel, I think is worth making an unofficial announcement here
for new users, the interface of river might change a little bit, but we got a lot of improvements regarding API consistency, model speedups and so on
besides that, river has a lot of extra tools and methods that are not available in the legacy skmultiflow
for instance, pre processing techniques (e.g., incremental StandardScaler)
Saulo Martiello Mastelini
@smastelini

sorry for my absence on this channel :/

@jacobmontiel and I have been focusing all our time in preparing the merge and polishing things

Welcome to river! If you have any questions, you can use Github's discussions to make questions and get feedbacks
Saulo Martiello Mastelini
@smastelini
We understand that some ongoing projects rely on skmultiflow for their functioning. For that reason, we will keep skmultiflow in its current state (stable release) and might apply eventual bug fixes
Nasrin Eshraghi Ivari
@nasrineshraghi
Hi All, Can anyone help me please? I have implemented data stream clustering. to simulate the stream, I used scikitmultiflow. But I want a sliding time window model to capture my last data. I do not know how can I implement or use sliding ?Does scikitmultiflow support windowing? I could not find anything related!
Michael Forde
@fordetek
Hey Everyone, I have been wanting to do some multivariate forecasting using the HoeffdingTreeRegressor for streamed data, Scikitmultiflow seems to suit what I want to do very well, but I noticed there isn't built-in forecasting support, I'm curious if there is any workaround I could try to do to achieve multiple step forecasting with this library?
Wannabe Maker
@AndrejLegat_twitter
Hi all, Is there any way to install scikit multiflow on new M1 Macs ? I Use Pycharm as IDE and when i tried to install scikit multiflow everytime i got a Error because it try to install multiflow for X64 architecture. Is here someone with similar problem?
Marilia-Nayara
@Marilia-Nayara
Hi @jacobmontiel , how to use concept drift detectors in regression problems (example: ADWIN, DDM,KSWIN)? obs: I don't want to use the error
samie-hash
@samie-hash
from skmultiflow.data import AnomalySineGenerator
from skmultiflow.anomaly_detection import HalfSpaceTrees
import numpy as np
import pandas as pd

stream = AnomalySineGenerator(random_state=42, n_samples=10000, n_anomalies=250)
hs_tree = HalfSpaceTrees(n_estimators=10, depth=8)
true_positive = 0
anomalies = 0
predictions = []
y_test = []
max_samples = 10000
n_samples = 0

stream.restart()
while n_samples < max_samples and stream.has_more_samples():
    X, y = stream.next_sample()
    y_pred = hs_tree.predict(X)
    if y[0] == 1.0:
        true_positive += 1
        if y_pred[0] == 1.0:
            anomalies += 1

    predictions.append(y_pred[0])
    y_test.append(y[0])  
    hs_tree.partial_fit(X, y)
    n_samples += 1

print('The data has {} anomalies'.format(true_positive))
print('Half Space Trees predicted {} anomalies'.format(anomalies))
This code on running it produces some interesting output. It predicts 0's for some time and predicts 1's later on.
Below is the classification report output which is quite poor.
from sklearn.metrics import classification_report
print(classification_report(y_test, predictions))
Hassan Mehmood
@hassanmehmud
@Marilia-Nayara please open up your problem little bit.
Venoli
@Venoli
Hi,
@jacobmontiel @all Anyone, please tell me how to use EvaluatePrequentialDelayed with extremely fast decision tree. I want to use the incremental learn part from extremely fast decision tree, while doing delayed evaluation. Please help me!!!!!
Saulo Martiello Mastelini
@smastelini
Hi everyone, just as a reminder, there is not active development in skmultiflow anymore. Skmultiflow and Creme have merged to become River. Now, users can also install River via pip
We encourage the skmultiflow users to make the leap to River. Feel free to open a discussion with your question, or asking for any assistance
Jacob Montiel and I are both maintainers of River too
Saulo Martiello Mastelini
@smastelini
I'll talk with Jacob about the possibility of creating a quick guide, maybe something like: "from skmultiflow to river"
cathrienli
@cathrienli
How do I save the results after the test as a file. Instead of saving the assessment as a file
@smastelini @jacobmontiel@all
cathrienli
@cathrienli
I am a multi-label classification. I have a total of 14 labels. Where do I set it?
medha-chippa
@medha-chippa
Hello @jacobmontiel @smastelini , i'm new to scikitmultiflow. I would like to train my classification model on one data stream (train data), and make predictions on another datastream(test stream). Could you please help me with a short piece of code that demonstrates this. Thank you.
qustea
@qustea
why my line graph doesnt show any value for the evaluation
image.png
leireib
@leireibargutxi:matrix.org
[m]
Hi, is there any way of getting a concept drift's detectors accuracy, or you have to use it with a clasification method? Thankyu
neha634
@neha634

Hi! We have a requirement to train the model using historical data as well as real time data. I am trying to use AdaptiveRandomForestRegressor model but getting error. Firstly, I am training model using data from csv and then will be training model based on real data. I am using the code below where
X and y are my features and labels
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

regressor = AdaptiveRandomForestRegressor()
regressor.fit(X_train, y_train)

I am getting below error. Can someone please help. Traceback (most recent call last):
File "C:/Users/Neha.Goyal/PycharmProjects/pythonProject/PredictionDataML_Scikit_AdaptiveRandomForest.py", line 43, in <module>
regressor.partial_fit(X_train, y_train)
File "C:\Users\Neha.Goyal\PycharmProjects\pythonProject\venv\lib\site-packages\skmultiflow\meta\adaptive_random_forest_regressor.py", line 296, in partial_fit
X[i].reshape(1, -1), [y[i]], sample_weight=[k],
File "C:\Users\Neha.Goyal\PycharmProjects\pythonProject\venv\lib\site-packages\pandas\core\series.py", line 824, in getitem
return self._get_value(key)
File "C:\Users\Neha.Goyal\PycharmProjects\pythonProject\venv\lib\site-packages\pandas\core\series.py", line 932, in _get_value
loc = self.index.get_loc(label)
File "C:\Users\Neha.Goyal\PycharmProjects\pythonProject\venv\lib\site-packages\pandas\core\indexes\base.py", line 3082, in get_loc
raise KeyError(key) from err
KeyError: 0

görkem
@gorkeem
Hello everyone. I am trying to create a dataset using SEAGenerator with 100 features but as default, it creates with 3 features. How can I create with 100 features??. What I am trying to get is like 0.6987, 0.2568, 0.570, 0.949, 0.1970, … , 0.3285, 0.4474, 0.3355, 0.585, 0.5411, 0 where the last number is the class and others are features.