Hive partition columns folders exchanged
I have a partitioned_table in hive which partitioned by "year,month"，so my hdfs list is
/user/hive/warehouse/demo.db/employee/year=2017/month=6
when I used "export" to export table and use "import" to create a new table ,the result is the year and month exchanged, the list is
/user/hive/warehouse/demo.db/new_employee/month=6/year=2017
my hive version is 1.2.2 and the query:
export table employee into /user/hadoop/data
import table new_employee from /user/hadoop/data
the partitions in hive have no different with the original table(which one I export). Even I add a new partition, the directory is not changed, it work as '/month=7/year=6'
so what's wrong happened?Thanks for help!
hive partition
asked Nov 22 '18 at 15:16
HeltHelt
63
add a comment |
I have a partitioned_table in hive which partitioned by "year,month"，so my hdfs list is
/user/hive/warehouse/demo.db/employee/year=2017/month=6
when I used "export" to export table and use "import" to create a new table ,the result is the year and month exchanged, the list is
/user/hive/warehouse/demo.db/new_employee/month=6/year=2017
my hive version is 1.2.2 and the query:
export table employee into /user/hadoop/data
import table new_employee from /user/hadoop/data
the partitions in hive have no different with the original table(which one I export). Even I add a new partition, the directory is not changed, it work as '/month=7/year=6'
so what's wrong happened?Thanks for help!
hive partition
asked Nov 22 '18 at 15:16
HeltHelt
63
tryshow partitions tablenamehad you find any difference?
– HbnKing
Nov 22 '18 at 15:32
What query did you use for import statement?
– Nishu Tayal
Nov 22 '18 at 15:49
partitions have no changed
– Helt
Nov 22 '18 at 22:56
add a comment |
I have a partitioned_table in hive which partitioned by "year,month"，so my hdfs list is
/user/hive/warehouse/demo.db/employee/year=2017/month=6
when I used "export" to export table and use "import" to create a new table ,the result is the year and month exchanged, the list is
/user/hive/warehouse/demo.db/new_employee/month=6/year=2017
my hive version is 1.2.2 and the query:
export table employee into /user/hadoop/data
import table new_employee from /user/hadoop/data
the partitions in hive have no different with the original table(which one I export). Even I add a new partition, the directory is not changed, it work as '/month=7/year=6'
so what's wrong happened?Thanks for help!
hive partition
asked Nov 22 '18 at 15:16
HeltHelt
63
I have a partitioned_table in hive which partitioned by "year,month"，so my hdfs list is
/user/hive/warehouse/demo.db/employee/year=2017/month=6
when I used "export" to export table and use "import" to create a new table ,the result is the year and month exchanged, the list is
/user/hive/warehouse/demo.db/new_employee/month=6/year=2017
my hive version is 1.2.2 and the query:
export table employee into /user/hadoop/data
import table new_employee from /user/hadoop/data
the partitions in hive have no different with the original table(which one I export). Even I add a new partition, the directory is not changed, it work as '/month=7/year=6'
so what's wrong happened?Thanks for help!
hive partition
hive partition
asked Nov 22 '18 at 15:16
HeltHelt
63
asked Nov 22 '18 at 15:16
HeltHelt
63
edited Nov 23 '18 at 14:45
Helt
asked Nov 22 '18 at 15:16
HeltHelt
63
asked Nov 22 '18 at 15:16
HeltHelt
63
asked Nov 22 '18 at 15:16
HeltHelt
63
63
tryshow partitions tablenamehad you find any difference?
– HbnKing
Nov 22 '18 at 15:32
What query did you use for import statement?
– Nishu Tayal
Nov 22 '18 at 15:49
partitions have no changed
– Helt
Nov 22 '18 at 22:56
add a comment |
tryshow partitions tablenamehad you find any difference?
– HbnKing
Nov 22 '18 at 15:32
What query did you use for import statement?
– Nishu Tayal
Nov 22 '18 at 15:49
partitions have no changed
– Helt
Nov 22 '18 at 22:56
try
show partitions tablename had you find any difference?– HbnKing
Nov 22 '18 at 15:32
try
show partitions tablename had you find any difference?– HbnKing
Nov 22 '18 at 15:32
What query did you use for import statement?
– Nishu Tayal
Nov 22 '18 at 15:49
What query did you use for import statement?
– Nishu Tayal
Nov 22 '18 at 15:49
partitions have no changed
– Helt
Nov 22 '18 at 22:56
partitions have no changed
– Helt
Nov 22 '18 at 22:56
add a comment |
1 Answer
1
active
oldest
votes
Is there issue in the way you are looking at the data as long as you dont have that issue then it is not a problem.by the way this is right export command for exporting the partition table
export table employee partition (year="2017", month="6") to 'hdfs_exports_location/employee';
import from 'hdfs_exports_location/employee';
well if you have more partition on year and month ..like each year will have 12 month of data i think you may have to do for each month separate command ..i have not tried..just try it with the above command let us know how it is coming
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433917%2fhive-partition-columns-folders-exchanged%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Is there issue in the way you are looking at the data as long as you dont have that issue then it is not a problem.by the way this is right export command for exporting the partition table
export table employee partition (year="2017", month="6") to 'hdfs_exports_location/employee';
import from 'hdfs_exports_location/employee';
well if you have more partition on year and month ..like each year will have 12 month of data i think you may have to do for each month separate command ..i have not tried..just try it with the above command let us know how it is coming
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
add a comment |
Is there issue in the way you are looking at the data as long as you dont have that issue then it is not a problem.by the way this is right export command for exporting the partition table
export table employee partition (year="2017", month="6") to 'hdfs_exports_location/employee';
import from 'hdfs_exports_location/employee';
well if you have more partition on year and month ..like each year will have 12 month of data i think you may have to do for each month separate command ..i have not tried..just try it with the above command let us know how it is coming
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
add a comment |
Is there issue in the way you are looking at the data as long as you dont have that issue then it is not a problem.by the way this is right export command for exporting the partition table
export table employee partition (year="2017", month="6") to 'hdfs_exports_location/employee';
import from 'hdfs_exports_location/employee';
well if you have more partition on year and month ..like each year will have 12 month of data i think you may have to do for each month separate command ..i have not tried..just try it with the above command let us know how it is coming
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
Is there issue in the way you are looking at the data as long as you dont have that issue then it is not a problem.by the way this is right export command for exporting the partition table
export table employee partition (year="2017", month="6") to 'hdfs_exports_location/employee';
import from 'hdfs_exports_location/employee';
well if you have more partition on year and month ..like each year will have 12 month of data i think you may have to do for each month separate command ..i have not tried..just try it with the above command let us know how it is coming
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
edited Nov 23 '18 at 9:12
Nishu Tayal
12.1k73481
12.1k73481
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
answered Nov 23 '18 at 3:45
sivarajsivaraj
144
144
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
add a comment |
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
I know your means, and all people tell me I should export or import the partitions one by one, but I only want to know this problem is whether or not a bug?Why did this happened?
– Helt
Nov 23 '18 at 10:31
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
this is not problem..you can say it is restriction from hive ..As of now this is the way to do it..May be in future they might normal way of export like how we will do it for normal tables
– sivaraj
Nov 24 '18 at 5:28
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433917%2fhive-partition-columns-folders-exchanged%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
try
show partitions tablenamehad you find any difference?– HbnKing
Nov 22 '18 at 15:32
What query did you use for import statement?
– Nishu Tayal
Nov 22 '18 at 15:49
partitions have no changed
– Helt
Nov 22 '18 at 22:56