Setting a sub data-frame at an index as we can select it using xs in pandas
Generally, I tend to make copies, do some work on a data-frames, in case I involuntarily alter it's structure or content, and I wish to go back to the older, I can.
This time, I need to set my thing working.
Context description (but if you need a minimal question, go to Problem please:
I have a timed data-set to train on, times can repeat,
s1
2013-01 v1
2013-01 v2
s2
2013-02 v3
2013-02 v4
-2013-02 v5
s3
2013-03 v6
2013-03 v7
-2013-03 v8
-2013-03 v9
I consider hyphen entries as deltas that need to be grouped as well as other subsets (minimum sufficient for 48 months data, here it's a sample of two months for batch size)
constructing the list with:
def sub_index_df(train_dated_n):
gp = train.groupby(pd.Grouper(freq='M'))
l = [gp.nth(i) for i in range(gp.size().max())]
subsets = pd.concat(l, keys=list(range(gp.size().max())))
return subsets
wich yields to this:
s1
2013-01 v1
2013-02 v3
2013-03 v6
s2
2013-01 v2
2013-02 v4
2013-03 v7
With residuals; Residuals, I go back to 48 months full sets and pick elements to fill each residual so it forms 48 subset, like this:
co=0;
cut=0;
for i in range(0, 10622):
if(tt.xs(i).shape[0]<12 and cut==0):
cut = i
if(tt.xs(i).shape[0]<12 and cut >0):
_ = set(tt.xs(0).index.values) - set(tt.xs(i).index.values)
borrowings = pd.DataFrame()
for time_index in _:
co = (co + 1) % cut
borrowed = tt.xs(co).loc[time_index]
borrowed['borrowed'] = 1
borrowings = borrowings.append([borrowed) #keeps track on new subsets
tt.xs(i).loc[time_index] = borrowed
tt.xs(i).sort_index(inplace=True)
tt is like (first subsets are of length 48):
tt.tail is like few entry for each subset, until one entry for each subset (at the end)
The problem is that setting tt.xs(i).loc[time_index]=borrowed doesn't work.
If I see the last borrowed subset after execution, it's well fetched from earlier full subsets
borrowings = pd.DataFrame()
_.append([borrowed]).sort_values(inplace=True)
Now the problem is: adding new rows to *tt.xs(i)* with .loc does'tn seem to work.
tt.xs(i).loc[time_index] = borrowed
python indexing
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
add a comment |
Generally, I tend to make copies, do some work on a data-frames, in case I involuntarily alter it's structure or content, and I wish to go back to the older, I can.
This time, I need to set my thing working.
Context description (but if you need a minimal question, go to Problem please:
I have a timed data-set to train on, times can repeat,
s1
2013-01 v1
2013-01 v2
s2
2013-02 v3
2013-02 v4
-2013-02 v5
s3
2013-03 v6
2013-03 v7
-2013-03 v8
-2013-03 v9
I consider hyphen entries as deltas that need to be grouped as well as other subsets (minimum sufficient for 48 months data, here it's a sample of two months for batch size)
constructing the list with:
def sub_index_df(train_dated_n):
gp = train.groupby(pd.Grouper(freq='M'))
l = [gp.nth(i) for i in range(gp.size().max())]
subsets = pd.concat(l, keys=list(range(gp.size().max())))
return subsets
wich yields to this:
s1
2013-01 v1
2013-02 v3
2013-03 v6
s2
2013-01 v2
2013-02 v4
2013-03 v7
With residuals; Residuals, I go back to 48 months full sets and pick elements to fill each residual so it forms 48 subset, like this:
co=0;
cut=0;
for i in range(0, 10622):
if(tt.xs(i).shape[0]<12 and cut==0):
cut = i
if(tt.xs(i).shape[0]<12 and cut >0):
_ = set(tt.xs(0).index.values) - set(tt.xs(i).index.values)
borrowings = pd.DataFrame()
for time_index in _:
co = (co + 1) % cut
borrowed = tt.xs(co).loc[time_index]
borrowed['borrowed'] = 1
borrowings = borrowings.append([borrowed) #keeps track on new subsets
tt.xs(i).loc[time_index] = borrowed
tt.xs(i).sort_index(inplace=True)
tt is like (first subsets are of length 48):
tt.tail is like few entry for each subset, until one entry for each subset (at the end)
The problem is that setting tt.xs(i).loc[time_index]=borrowed doesn't work.
If I see the last borrowed subset after execution, it's well fetched from earlier full subsets
borrowings = pd.DataFrame()
_.append([borrowed]).sort_values(inplace=True)
Now the problem is: adding new rows to *tt.xs(i)* with .loc does'tn seem to work.
tt.xs(i).loc[time_index] = borrowed
python indexing
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
add a comment |
Generally, I tend to make copies, do some work on a data-frames, in case I involuntarily alter it's structure or content, and I wish to go back to the older, I can.
This time, I need to set my thing working.
Context description (but if you need a minimal question, go to Problem please:
I have a timed data-set to train on, times can repeat,
s1
2013-01 v1
2013-01 v2
s2
2013-02 v3
2013-02 v4
-2013-02 v5
s3
2013-03 v6
2013-03 v7
-2013-03 v8
-2013-03 v9
I consider hyphen entries as deltas that need to be grouped as well as other subsets (minimum sufficient for 48 months data, here it's a sample of two months for batch size)
constructing the list with:
def sub_index_df(train_dated_n):
gp = train.groupby(pd.Grouper(freq='M'))
l = [gp.nth(i) for i in range(gp.size().max())]
subsets = pd.concat(l, keys=list(range(gp.size().max())))
return subsets
wich yields to this:
s1
2013-01 v1
2013-02 v3
2013-03 v6
s2
2013-01 v2
2013-02 v4
2013-03 v7
With residuals; Residuals, I go back to 48 months full sets and pick elements to fill each residual so it forms 48 subset, like this:
co=0;
cut=0;
for i in range(0, 10622):
if(tt.xs(i).shape[0]<12 and cut==0):
cut = i
if(tt.xs(i).shape[0]<12 and cut >0):
_ = set(tt.xs(0).index.values) - set(tt.xs(i).index.values)
borrowings = pd.DataFrame()
for time_index in _:
co = (co + 1) % cut
borrowed = tt.xs(co).loc[time_index]
borrowed['borrowed'] = 1
borrowings = borrowings.append([borrowed) #keeps track on new subsets
tt.xs(i).loc[time_index] = borrowed
tt.xs(i).sort_index(inplace=True)
tt is like (first subsets are of length 48):
tt.tail is like few entry for each subset, until one entry for each subset (at the end)
The problem is that setting tt.xs(i).loc[time_index]=borrowed doesn't work.
If I see the last borrowed subset after execution, it's well fetched from earlier full subsets
borrowings = pd.DataFrame()
_.append([borrowed]).sort_values(inplace=True)
Now the problem is: adding new rows to *tt.xs(i)* with .loc does'tn seem to work.
tt.xs(i).loc[time_index] = borrowed
python indexing
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
Generally, I tend to make copies, do some work on a data-frames, in case I involuntarily alter it's structure or content, and I wish to go back to the older, I can.
This time, I need to set my thing working.
Context description (but if you need a minimal question, go to Problem please:
I have a timed data-set to train on, times can repeat,
s1
2013-01 v1
2013-01 v2
s2
2013-02 v3
2013-02 v4
-2013-02 v5
s3
2013-03 v6
2013-03 v7
-2013-03 v8
-2013-03 v9
I consider hyphen entries as deltas that need to be grouped as well as other subsets (minimum sufficient for 48 months data, here it's a sample of two months for batch size)
constructing the list with:
def sub_index_df(train_dated_n):
gp = train.groupby(pd.Grouper(freq='M'))
l = [gp.nth(i) for i in range(gp.size().max())]
subsets = pd.concat(l, keys=list(range(gp.size().max())))
return subsets
wich yields to this:
s1
2013-01 v1
2013-02 v3
2013-03 v6
s2
2013-01 v2
2013-02 v4
2013-03 v7
With residuals; Residuals, I go back to 48 months full sets and pick elements to fill each residual so it forms 48 subset, like this:
co=0;
cut=0;
for i in range(0, 10622):
if(tt.xs(i).shape[0]<12 and cut==0):
cut = i
if(tt.xs(i).shape[0]<12 and cut >0):
_ = set(tt.xs(0).index.values) - set(tt.xs(i).index.values)
borrowings = pd.DataFrame()
for time_index in _:
co = (co + 1) % cut
borrowed = tt.xs(co).loc[time_index]
borrowed['borrowed'] = 1
borrowings = borrowings.append([borrowed) #keeps track on new subsets
tt.xs(i).loc[time_index] = borrowed
tt.xs(i).sort_index(inplace=True)
tt is like (first subsets are of length 48):
tt.tail is like few entry for each subset, until one entry for each subset (at the end)
The problem is that setting tt.xs(i).loc[time_index]=borrowed doesn't work.
If I see the last borrowed subset after execution, it's well fetched from earlier full subsets
borrowings = pd.DataFrame()
_.append([borrowed]).sort_values(inplace=True)
Now the problem is: adding new rows to *tt.xs(i)* with .loc does'tn seem to work.
tt.xs(i).loc[time_index] = borrowed
python indexing
python indexing
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
asked Nov 25 '18 at 9:48
Curcuma_Curcuma_
1621322
1621322
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53466330%2fsetting-a-sub-data-frame-at-an-index-as-we-can-select-it-using-xs-in-pandas%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53466330%2fsetting-a-sub-data-frame-at-an-index-as-we-can-select-it-using-xs-in-pandas%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
