ValueError: ('Unknown transform primitive is_in. ', 'Call ft.primitives.list_primitives() to get', ' a list of available primitives')
basically what i am trying to do is to get primitives definition from some outside system and feed that primitive definition to featuretools so that this new primitive can be used in creating new features suggested by featuretools
@datajanko have you seen the option to use a dataframe as the input for
cutoff_time? you can pass in a different cutoff time for each instance id. you'd have to write the code to determine what the cutoff time per customer is, but once you do you can pass it to featuretools. does that apply to your situation?
@bgoel2003 you need to pass the python definition rather than the string
custom_primitive = make_trans_primitive(...)
ft.dfs(trans_primitives=[custom_primitive],...)
@kmax12 no I didn’t realize that. Thanks for the hint. I think I can leverage cut_off_times and training window, to also construct my validation window. Then, I’d only have to do this multiple times for all the “folds” I am interested in. I hope, I’ll find some time to work on this.
@datajanko exactly. let us know how it goes. if you have any other questions, feel free to message here, or if you think it'd be helpful for other in the future post on StackOverflow with the featuretools tag: https://stackoverflow.com/questions/tagged/featuretools
Is there a way to use multiple training windows? I am trying to generate features for last 3/6/9/12... months. https://stackoverflow.com/questions/51865267/get-features-by-different-time-windows
we'll follow up and answer on stack overflow as well. thanks for posting
unfortunately, I have been working solely on time series for the last years so I did not have a dataset to try it out
now, I am the maintainer of tsfesh (https://github.com/blue-yonder/tsfresh), we also perform feature extraction, but on time series instead of relational data
but maybe we can embed tsfresh into featuretools in some way? I worked on Data Science problems where I had to process time series and relational data at the same time. A fully automated feature extraction framework would have helped a lot
Hey @MaxBenChrist ! Thanks for the kind words. tsfresh is an interesting library as well. We actually have experimented with embedding tsfresh into featuretools by using custom primitives. You can see an example of that in this notebook: https://github.com/Featuretools/predict-remaining-useful-life/blob/master/Advanced%20Featuretools%20RUL.ipynb
what are your thoughts on the best way to integrate the two libraries?
in any case, we could have a brainstorming session over skype sometime next week and discuss possible starting points for a collaboration?
Hello - under what circumstances would an entity variable from EntitySet es and defined as index show up as id when listing es.entities? I'm trying to resolve the following warning: both an index level and a column label.
Defaulting to column, but this will raise an ambiguity error in a future version
end_entity_id=child_eid)
Defaulting to column, but this will raise an ambiguity error in a future version
end_entity_id=child_eid)
hi @dugland123 - just posted an answer to you question on stack overflow: https://stackoverflow.com/questions/52013113/why-is-an-entity-index-displayed-as-id-and-not-index/52031191#52031191
you can read my full reponse there, but the warning you are seeing is unrelated to the featuretools variable type and will go away in the next release of Featuretools.
I'm Silvio.. from Uruguay
I have a question.
Is Featuretools capable of building useful attributes from semistructured data such as xml, json or rfd?
Or is necessary convert the dataset to a tabular form?
Hi all,
I've been using this tool the past few days and it has been great so far. I work heavily with financial data and noticed that when I'm creating my EntitySet if I have column names as dtype int I get a failure message.
I was wondering if this is desired behavior that the column names need to be dtype str for the entity set to work.
Thank you for the awesome product.
Waco Holve
we'd like to support that since they are valid pandas column names
@kmax12 I've attached a notebook with the csv. Thinking more about it now I should have just created some random data.
Hey all! I've been playing with the predict-appointment-noshow notebook tutorial for a bit, and I'm a bit confused by the output of the PERCENT_TRUE primitive. My understanding is that a column like
locations.PERCENT_TRUE(appointments.sms_received) gives the percent of rows for which sms_received is True, given a single location. I'd expect that column to be the same for all rows of a single location, because that's what it was conditioned on, but I'm not finding that to be the case. Any ideas why?
(sorry to interrupt current topic!)
In that notebook, if I run
fm.loc[fm.neighborhood == 'HORTO', 'locations.PERCENT_TRUE(appointments.sms_received)'].describe() I get:
count 144.00
mean 0.20
std 0.09
min 0.00
25% 0.20
50% 0.23
75% 0.26
max 0.31
Name: locations.PERCENT_TRUE(appointments.sms_received), dtype: float64
mean 0.20
std 0.09
min 0.00
25% 0.20
50% 0.23
75% 0.26
max 0.31
Name: locations.PERCENT_TRUE(appointments.sms_received), dtype: float64
@ksarmentrout that's a great question for stackoverflow. can you post it there with the featuretools tag? https://stackoverflow.com/questions/tagged/featuretools
that way we can answer it and make sure it is searchable by other users
@WacoHolve_gitlab thanks for the quick reply. i am running out of time today, but if i dont get to ill reply tomorrow