how to read multiple .gz files in a particular directory in python without unzipping them
up vote
0
down vote
favorite
I have a folder /var/tmp in my linux directory where i have multiple .gz files in the below mentioned format (name_yyyymmddhhmmss.gz).
aakashdeep_20181120080005.gz
aakashdeep_20181120080025.gz
kalpana_20181119080005.gz
aakashdeep_20181120080025.gz
Now i want to open all the gz files with format as name_20181120*.gz without unzipping them and read the content out of them.
i have written a simple code
!/usr/bin/python
import gzip
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120080002.gz','r')
for line in output:
print (line)
and the same is giving me the output as expected, but i want to open all the files like below
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120*.gz','r')
Can anyone suggest me the way for this.??
python zip unzip
asked Nov 20 at 10:36
Aakashdeep Garg
34
add a comment |
up vote
0
down vote
favorite
I have a folder /var/tmp in my linux directory where i have multiple .gz files in the below mentioned format (name_yyyymmddhhmmss.gz).
aakashdeep_20181120080005.gz
aakashdeep_20181120080025.gz
kalpana_20181119080005.gz
aakashdeep_20181120080025.gz
Now i want to open all the gz files with format as name_20181120*.gz without unzipping them and read the content out of them.
i have written a simple code
!/usr/bin/python
import gzip
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120080002.gz','r')
for line in output:
print (line)
and the same is giving me the output as expected, but i want to open all the files like below
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120*.gz','r')
Can anyone suggest me the way for this.??
python zip unzip
asked Nov 20 at 10:36
Aakashdeep Garg
34
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I have a folder /var/tmp in my linux directory where i have multiple .gz files in the below mentioned format (name_yyyymmddhhmmss.gz).
aakashdeep_20181120080005.gz
aakashdeep_20181120080025.gz
kalpana_20181119080005.gz
aakashdeep_20181120080025.gz
Now i want to open all the gz files with format as name_20181120*.gz without unzipping them and read the content out of them.
i have written a simple code
!/usr/bin/python
import gzip
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120080002.gz','r')
for line in output:
print (line)
and the same is giving me the output as expected, but i want to open all the files like below
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120*.gz','r')
Can anyone suggest me the way for this.??
python zip unzip
asked Nov 20 at 10:36
Aakashdeep Garg
34
I have a folder /var/tmp in my linux directory where i have multiple .gz files in the below mentioned format (name_yyyymmddhhmmss.gz).
aakashdeep_20181120080005.gz
aakashdeep_20181120080025.gz
kalpana_20181119080005.gz
aakashdeep_20181120080025.gz
Now i want to open all the gz files with format as name_20181120*.gz without unzipping them and read the content out of them.
i have written a simple code
!/usr/bin/python
import gzip
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120080002.gz','r')
for line in output:
print (line)
and the same is giving me the output as expected, but i want to open all the files like below
output = gzip.open('/var/tmp/Aakashdeep/aakashdeep_20181120*.gz','r')
Can anyone suggest me the way for this.??
python zip unzip
python zip unzip
asked Nov 20 at 10:36
Aakashdeep Garg
34
asked Nov 20 at 10:36
Aakashdeep Garg
34
asked Nov 20 at 10:36
Aakashdeep Garg
34
asked Nov 20 at 10:36
Aakashdeep Garg
34
asked Nov 20 at 10:36
Aakashdeep Garg
34
34
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
accepted
Use glob.glob to obtain a list of files to process, then open each with gzip.open, do something with its contents, and move on to the next. Outline (untested):
import glob
import gzip
ZIPFILES='/var/tmp/Aakashdeep/aakashdeep_20181120*.gz'
filelist = glob.glob(ZIPFILES)
for gzfile in filelist:
# print("#Starting " + gzfile) #if you want to know which file is being processed
with gzip.open( gzfile, 'r') as f:
for line in f:
print(line)
answered Nov 20 at 12:03
nigel222
1,9901512
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
The pattern (for filename in glob.glob(...)thenwith) is one that I've used hundreds of times, but oddly never withgzip. It's the Python equivalent of most bash commands with a-rrecusive option. BTWfor gzfile in sorted(filelist)may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)
– nigel222
Nov 20 at 13:21
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53391125%2fhow-to-read-multiple-gz-files-in-a-particular-directory-in-python-without-unzip%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
Use glob.glob to obtain a list of files to process, then open each with gzip.open, do something with its contents, and move on to the next. Outline (untested):
import glob
import gzip
ZIPFILES='/var/tmp/Aakashdeep/aakashdeep_20181120*.gz'
filelist = glob.glob(ZIPFILES)
for gzfile in filelist:
# print("#Starting " + gzfile) #if you want to know which file is being processed
with gzip.open( gzfile, 'r') as f:
for line in f:
print(line)
answered Nov 20 at 12:03
nigel222
1,9901512
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
The pattern (for filename in glob.glob(...)thenwith) is one that I've used hundreds of times, but oddly never withgzip. It's the Python equivalent of most bash commands with a-rrecusive option. BTWfor gzfile in sorted(filelist)may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)
– nigel222
Nov 20 at 13:21
add a comment |
up vote
0
down vote
accepted
Use glob.glob to obtain a list of files to process, then open each with gzip.open, do something with its contents, and move on to the next. Outline (untested):
import glob
import gzip
ZIPFILES='/var/tmp/Aakashdeep/aakashdeep_20181120*.gz'
filelist = glob.glob(ZIPFILES)
for gzfile in filelist:
# print("#Starting " + gzfile) #if you want to know which file is being processed
with gzip.open( gzfile, 'r') as f:
for line in f:
print(line)
answered Nov 20 at 12:03
nigel222
1,9901512
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
The pattern (for filename in glob.glob(...)thenwith) is one that I've used hundreds of times, but oddly never withgzip. It's the Python equivalent of most bash commands with a-rrecusive option. BTWfor gzfile in sorted(filelist)may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)
– nigel222
Nov 20 at 13:21
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
Use glob.glob to obtain a list of files to process, then open each with gzip.open, do something with its contents, and move on to the next. Outline (untested):
import glob
import gzip
ZIPFILES='/var/tmp/Aakashdeep/aakashdeep_20181120*.gz'
filelist = glob.glob(ZIPFILES)
for gzfile in filelist:
# print("#Starting " + gzfile) #if you want to know which file is being processed
with gzip.open( gzfile, 'r') as f:
for line in f:
print(line)
answered Nov 20 at 12:03
nigel222
1,9901512
Use glob.glob to obtain a list of files to process, then open each with gzip.open, do something with its contents, and move on to the next. Outline (untested):
import glob
import gzip
ZIPFILES='/var/tmp/Aakashdeep/aakashdeep_20181120*.gz'
filelist = glob.glob(ZIPFILES)
for gzfile in filelist:
# print("#Starting " + gzfile) #if you want to know which file is being processed
with gzip.open( gzfile, 'r') as f:
for line in f:
print(line)
answered Nov 20 at 12:03
nigel222
1,9901512
answered Nov 20 at 12:03
nigel222
1,9901512
answered Nov 20 at 12:03
nigel222
1,9901512
answered Nov 20 at 12:03
nigel222
1,9901512
1,9901512
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
The pattern (for filename in glob.glob(...)thenwith) is one that I've used hundreds of times, but oddly never withgzip. It's the Python equivalent of most bash commands with a-rrecusive option. BTWfor gzfile in sorted(filelist)may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)
– nigel222
Nov 20 at 13:21
add a comment |
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
The pattern (for filename in glob.glob(...)thenwith) is one that I've used hundreds of times, but oddly never withgzip. It's the Python equivalent of most bash commands with a-rrecusive option. BTWfor gzfile in sorted(filelist)may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)
– nigel222
Nov 20 at 13:21
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
Thanks Nigel. The code you mentioned above, worked for me.
– Aakashdeep Garg
Nov 20 at 13:05
The pattern (
for filename in glob.glob(...) then with) is one that I've used hundreds of times, but oddly never with gzip. It's the Python equivalent of most bash commands with a -r recusive option. BTW for gzfile in sorted(filelist) may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)– nigel222
Nov 20 at 13:21
The pattern (
for filename in glob.glob(...) then with) is one that I've used hundreds of times, but oddly never with gzip. It's the Python equivalent of most bash commands with a -r recusive option. BTW for gzfile in sorted(filelist) may be an easy cosmetic improvement, though the sort order isn't always exactly what you want)– nigel222
Nov 20 at 13:21
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53391125%2fhow-to-read-multiple-gz-files-in-a-particular-directory-in-python-without-unzip%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
