Why tuneRF does not have a good result for ctree in R
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
1
$begingroup$
I tried to use tuneRF for selecting the minimal OOB value for best mtry value in ctree model. However, if I use the right value I get worse results than just increasing the mtry value.
Here is my code:
library(party)
dat1 <- fread('https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data',stringsAsFactors=T)
## split data to train and test
set.seed(123)
dat1 <- subset(dat1, !is.na(V1))
smp_size<-100
train_ind <- sample(seq_len(nrow(dat1)), size = smp_size)
train <- dat1[train_ind, ]
test <- dat1[-train_ind, ]
ct <- ctree(V1 ~ ., data = train)
test$CTVAL<- predict(ct, test[,V2:V9])
> mean (test$V1==test$CTVAL)
0.5020849
Now I run tuneRF:
> bestmtry <- tuneRF(train[, V2:V9], train$V1, stepFactor = 1.5, improve = 1e-5, ntree = 500)
mtry = 2 OOB error = 47%
Searching left ...
Searching right ...
mtry = 3 OOB error = 52%
-0.106383 1e-05
It recommends mtry =2.
Running same model with mtry = 2 (replace the line ct<-...):
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 2 ))
> mean (test$V1==test$CTVAL)
0.4841795
And using the same model with mtry = 10:
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 10 ))
> mean (test$V1==test$CTVAL)
0.5109149
r optimization random-forest
asked Nov 26 '18 at 14:31
AviAvi
1087
$endgroup$
migrated from stackoverflow.com Dec 2 '18 at 2:31
This question came from our site for professional and enthusiast programmers.
add a comment |
1
$begingroup$
I tried to use tuneRF for selecting the minimal OOB value for best mtry value in ctree model. However, if I use the right value I get worse results than just increasing the mtry value.
Here is my code:
library(party)
dat1 <- fread('https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data',stringsAsFactors=T)
## split data to train and test
set.seed(123)
dat1 <- subset(dat1, !is.na(V1))
smp_size<-100
train_ind <- sample(seq_len(nrow(dat1)), size = smp_size)
train <- dat1[train_ind, ]
test <- dat1[-train_ind, ]
ct <- ctree(V1 ~ ., data = train)
test$CTVAL<- predict(ct, test[,V2:V9])
> mean (test$V1==test$CTVAL)
0.5020849
Now I run tuneRF:
> bestmtry <- tuneRF(train[, V2:V9], train$V1, stepFactor = 1.5, improve = 1e-5, ntree = 500)
mtry = 2 OOB error = 47%
Searching left ...
Searching right ...
mtry = 3 OOB error = 52%
-0.106383 1e-05
It recommends mtry =2.
Running same model with mtry = 2 (replace the line ct<-...):
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 2 ))
> mean (test$V1==test$CTVAL)
0.4841795
And using the same model with mtry = 10:
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 10 ))
> mean (test$V1==test$CTVAL)
0.5109149
r optimization random-forest
asked Nov 26 '18 at 14:31
AviAvi
1087
$endgroup$
migrated from stackoverflow.com Dec 2 '18 at 2:31
This question came from our site for professional and enthusiast programmers.
$begingroup$
Auto-tuning is often times nice, but you have to keep in my mind that is is just a quick and dirty step that can be very off sometimes (for ex. if you have junk data or a lot of noise), in that case you have to adjust it manually.
$endgroup$
– user2974951
Nov 29 '18 at 9:15
add a comment |
1
1
1
$begingroup$
I tried to use tuneRF for selecting the minimal OOB value for best mtry value in ctree model. However, if I use the right value I get worse results than just increasing the mtry value.
Here is my code:
library(party)
dat1 <- fread('https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data',stringsAsFactors=T)
## split data to train and test
set.seed(123)
dat1 <- subset(dat1, !is.na(V1))
smp_size<-100
train_ind <- sample(seq_len(nrow(dat1)), size = smp_size)
train <- dat1[train_ind, ]
test <- dat1[-train_ind, ]
ct <- ctree(V1 ~ ., data = train)
test$CTVAL<- predict(ct, test[,V2:V9])
> mean (test$V1==test$CTVAL)
0.5020849
Now I run tuneRF:
> bestmtry <- tuneRF(train[, V2:V9], train$V1, stepFactor = 1.5, improve = 1e-5, ntree = 500)
mtry = 2 OOB error = 47%
Searching left ...
Searching right ...
mtry = 3 OOB error = 52%
-0.106383 1e-05
It recommends mtry =2.
Running same model with mtry = 2 (replace the line ct<-...):
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 2 ))
> mean (test$V1==test$CTVAL)
0.4841795
And using the same model with mtry = 10:
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 10 ))
> mean (test$V1==test$CTVAL)
0.5109149
r optimization random-forest
asked Nov 26 '18 at 14:31
AviAvi
1087
$endgroup$
I tried to use tuneRF for selecting the minimal OOB value for best mtry value in ctree model. However, if I use the right value I get worse results than just increasing the mtry value.
Here is my code:
library(party)
dat1 <- fread('https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data',stringsAsFactors=T)
## split data to train and test
set.seed(123)
dat1 <- subset(dat1, !is.na(V1))
smp_size<-100
train_ind <- sample(seq_len(nrow(dat1)), size = smp_size)
train <- dat1[train_ind, ]
test <- dat1[-train_ind, ]
ct <- ctree(V1 ~ ., data = train)
test$CTVAL<- predict(ct, test[,V2:V9])
> mean (test$V1==test$CTVAL)
0.5020849
Now I run tuneRF:
> bestmtry <- tuneRF(train[, V2:V9], train$V1, stepFactor = 1.5, improve = 1e-5, ntree = 500)
mtry = 2 OOB error = 47%
Searching left ...
Searching right ...
mtry = 3 OOB error = 52%
-0.106383 1e-05
It recommends mtry =2.
Running same model with mtry = 2 (replace the line ct<-...):
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 2 ))
> mean (test$V1==test$CTVAL)
0.4841795
And using the same model with mtry = 10:
ct <- ctree(V1 ~ ., data = train,controls = ctree_control(mtry = 10 ))
> mean (test$V1==test$CTVAL)
0.5109149
r optimization random-forest
r optimization random-forest
asked Nov 26 '18 at 14:31
AviAvi
1087
asked Nov 26 '18 at 14:31
AviAvi
1087
asked Nov 26 '18 at 14:31
AviAvi
1087
asked Nov 26 '18 at 14:31
AviAvi
1087
asked Nov 26 '18 at 14:31
AviAvi
1087
1087
migrated from stackoverflow.com Dec 2 '18 at 2:31
This question came from our site for professional and enthusiast programmers.
migrated from stackoverflow.com Dec 2 '18 at 2:31
This question came from our site for professional and enthusiast programmers.
$begingroup$
Auto-tuning is often times nice, but you have to keep in my mind that is is just a quick and dirty step that can be very off sometimes (for ex. if you have junk data or a lot of noise), in that case you have to adjust it manually.
$endgroup$
– user2974951
Nov 29 '18 at 9:15
add a comment |
$begingroup$
Auto-tuning is often times nice, but you have to keep in my mind that is is just a quick and dirty step that can be very off sometimes (for ex. if you have junk data or a lot of noise), in that case you have to adjust it manually.
$endgroup$
– user2974951
Nov 29 '18 at 9:15
$begingroup$
Auto-tuning is often times nice, but you have to keep in my mind that is is just a quick and dirty step that can be very off sometimes (for ex. if you have junk data or a lot of noise), in that case you have to adjust it manually.
$endgroup$
– user2974951
Nov 29 '18 at 9:15
$begingroup$
Auto-tuning is often times nice, but you have to keep in my mind that is is just a quick and dirty step that can be very off sometimes (for ex. if you have junk data or a lot of noise), in that case you have to adjust it manually.
$endgroup$
– user2974951
Nov 29 '18 at 9:15
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f379857%2fwhy-tunerf-does-not-have-a-good-result-for-ctree-in-r%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f379857%2fwhy-tunerf-does-not-have-a-good-result-for-ctree-in-r%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Auto-tuning is often times nice, but you have to keep in my mind that is is just a quick and dirty step that can be very off sometimes (for ex. if you have junk data or a lot of noise), in that case you have to adjust it manually.
$endgroup$
– user2974951
Nov 29 '18 at 9:15