the error essentially comes down to the categories being different between the categorical variables you are trying to relate. See this code example
import pandas as pd
from pandas.api.types import is_dtype_equal
s = pd.Series(["a","b","a"], dtype="category")
s2 = pd.Series(["b","b","a"], dtype="category")
s3 = pd.Series(["a","b","c"], dtype="category")
is_dtype_equal(s.dtype, s2.dtype) # this is True
is_dtype_equal(s.dtype, s3.dtype) # this is FalseYou need update your dataframe before loading it into Featuretool to make sure the Pandas Categoricals have the same values category values. Here's how you do that
# if s is missing categories from s3
new_s = s.astype(s3.dtype)
is_dtype_equal(new_s.dtype, s3.dtype) # this is True
# if both are missing categories from each other
import pandas.api.types as pdtypes
s4 = pd.Series(["b","c"], dtype="category")
categories = set(s.dtype.categories + s4.dtype.categories)
new_s = s.astype("category", categories=categories)
new_s4 = s4.astype("category", categories=categories)
is_dtype_equal(new_s.dtype, new_s4.dtype) # this is Trueplease also post on SO where I can give a more detailed answer for everyone else
Hi everyone! I love featuretools and the idea to automically engeineer features. Unfortunately I can't seem to add interesting variables and I would be happy if someone could help out :)
I suspect that it has something to do with my data because I can reproduce the example in the docs just fine..
@favstats This looks like incorrect behavior, thanks for sharing with us. I just made of fix for it on a branch. Can you try to install that branch of featuretools and run your code again? You can install that branch using pip with this command
pip install -e git://github.com/featuretools/featuretools.git@interesting-values-direct-features#egg=featuretools
Let us know if it helps!
@kmax12 one thing I just noticed.. I mistyped the value name at first and it gave me back only NaN values, which makes sense since it can't match the arguments. However, I wonder if this is intended behaviour or if it should say something like "value not found" and throw an error. Just thinking out loud :)
Anyway, thank you again for answering this so fast! :)
Hello everyone :) I have a question and hope someone can help out.
Say I would want to calculate features for specific timeframes since the cut-off value, so for example:
My cut-off value is 1 January 2005. I want to count the number of products a customer bought in the last month/ the last three months and the last year before that and have them all in the same feature matrix.
I know I can do something like this (from reading this):
feature_matrix, features = ft.dfs(
target_entity="customers",
agg_primitives=["count"],
cutoff_time=pd.Timestamp('January 1, 2005'),
training_window=ft.Timedelta("30 days"),
entityset=es,
verbose=True
)But this would of course only give me features for the 30 days before January 1, 2005, however I would like them for different time ranges as well (i.e. three months, a year or really any other time range that would interest me).
So I am not sure how to get to my goal right now. Would I need to create a new primitive for this task or can this be done with already existing functions?
Hello everyone!
I encountered a problem when I tried to create relationships between entititysets (using my own data). There is no error, but it just doesn't create features for one of my entities (the "prods" entity), although everything should be connected just fine. In some ways, this is similar to the first issue I encountered, only that it occurs just with this specific entity set up. Unfortunately, I can't share my data this time but attached you will find a minimal example with some mock data, where this problem also occurs.
Hope somebody can help and thank you for your awesome support!
Best, Fabio
Done:
It's just a lot of code so I thought a python notebook would be a bit more compact :)