Hi @LucyMcGowan , I'm a little confused about what "tree depth" means. When we set tree_depth = 10, I think that we are allowing T0 to have a maximum depth of 10, but I'm not quite sure what that really means. Is it that T0 can have a maximum of ten splits/internal nodes?
1 reply
I'm updating the exam details (I've deleted my comment above to avoid confusion)
- The first part of the midterm will be on canvas just like midterm 02, you have until Tuesday, April 28 to complete it and you have 2 hours once you begin
- The second part will be due Tuesday, April 28, the second part is an analysis so there is not an imposed time limit, but it is due on Tuesday
Now I am getting this error message for exercise 6 @jdtrat
Thank you @jdtrat! @TaylorFowlks it looks like you need to save your result output, see this sta-363-s20/community#79
@LucyMcGowan
@TaylorFowlks - just added an issue to your repo with a suggestion: sta-363-s20/lab-06-ensemble-TaylorFowlks#1
I have a question concerning the proportion one should use when splitting their data into training and testing sets. How does one determine a 'good' proportion? To elaborate, I recall there are issues with the simple validation approach (i.e. the variability of the test error). In the class examples I've observed that we split the data into training and testing sets, perform K-fold CV on the training data to tune a parameter, and then see how the optimized model performs on our held out test data. Is this summary correct? In particular, how does the proportion affect the outcome of this process.
@Feifan-990807 I think this should answer your question
Here's some documentation for the code we're writing, but it says that
tree_depthis the maximum depth of the tree (i.e. number of splits).
1 reply
I'm getting conflicted answers
@BrianNathanWhite, when you have sufficient data, the initial splitting basically provides an additional check on your estimates via cross validation. In practice I very rarely both split and use cross validation, since cross validation is estimating the test error as well. I think as a general rule, you can feel pretty confident in your results acquired by cross validation without needing the extra split into training and testing
@traytib is your file still getting stuck?
@BrianNathanWhite that rule of thumb is mentioned here: https://www.fharrell.com/post/split-val/, Frank’s posts are a great resource for building really good prediction models for future reference
@LucyMcGowan Hi Prof. McGowan, I wonder if a variable importance check is important for improving the accuracy of the predicted model (i.e. decreasing the test error) or it is just useful in improving interpretability? Is it necessary to perform variable importance in our exam (in this case we have 30 variables)?
2 replies
@SangniHuang, see my thread above from @gregorydvor’s question - it’s fine to enter as is on Canvas but if you want to try the code yourself first, you can create a new data frame from the recipe, specify the formula in the fit, and the data as the new data frame like this:
new_dat <- your_recipe %>%
prep() %>%
juice()
fit(final_spec,
y ~ , #fill in your formula here
data = new_dat)
@noah-handwerk, This is probably easiest to see with variable importance. There are several ways to do this, one is via the vip package. Here is some code:
# install.packages("vip") #uncomment and run this ONCE in the console
library(vip)
mtcars_cv <- vfold_cv(mtcars)
linear_reg(mixture = 1, penalty = 3) %>%
set_engine("glmnet") %>%
fit(mpg ~ . , data = mtcars) -> f
f %>%
vi(lambda = lowest_rmse$penalty) %>%
mutate(
Importance = abs(Importance),
Variable = factor(Variable, levels = Variable[order(Importance)])
) %>%
ggplot(aes(x = Importance, y = Variable, fill = Sign)) +
geom_col() +
scale_x_continuous(expand = c(0, 0)) +
labs(y = NULL)
Error in system(paste(which, shQuote(names[i])), intern = TRUE, ignore.stderr = TRUE) :
cannot popen '/usr/bin/which 'pdflatex' 2>/dev/null', probable reason 'Cannot allocate memory'
cannot popen '/usr/bin/which 'pdflatex' 2>/dev/null', probable reason 'Cannot allocate memory'
I keep getting this error
I vaguely remember from the beginning of the semester that there was an issue if a lot of people were trying to knit at once hence the inability to allocate memory. I don't think we just need to knit anything, but I'm not positive. I was just going to submit my final model. @erincperry