How to loop to import several csv files as different DataFrames and each of the imported DataFrames has its...
up vote
0
down vote
favorite
This is my first post. I want to loop or iterate in a directory to import each of the files as a separate DataFrame with a name similar (at least with the numeration) to the name of its file.
After a lot of research I still do not know how to do it. Obviously I am a very beginner :-)
My code is:
Main_folder = os.getcwd()
Folders = os.listdir('.')
for file in Folders:
data= pd.read_csv(file, sep="t", header=0)
data.columns=data.columns.str.strip()
where for instance Folders
is a list of files names including the file extension. e.g.:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT']
What I need is just to import all files to my work space such:
Load_01=pd.read_csv('01_load.TXT', sep="t", header=0)
Load_02=pd.read_csv('02_load.TXT', sep="t", header=0)
but in a loop since I have many files.
python pandas csv dataframe import
add a comment |
up vote
0
down vote
favorite
This is my first post. I want to loop or iterate in a directory to import each of the files as a separate DataFrame with a name similar (at least with the numeration) to the name of its file.
After a lot of research I still do not know how to do it. Obviously I am a very beginner :-)
My code is:
Main_folder = os.getcwd()
Folders = os.listdir('.')
for file in Folders:
data= pd.read_csv(file, sep="t", header=0)
data.columns=data.columns.str.strip()
where for instance Folders
is a list of files names including the file extension. e.g.:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT']
What I need is just to import all files to my work space such:
Load_01=pd.read_csv('01_load.TXT', sep="t", header=0)
Load_02=pd.read_csv('02_load.TXT', sep="t", header=0)
but in a loop since I have many files.
python pandas csv dataframe import
2
Can you use a dictionary?
– Neil
Nov 19 at 22:59
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
This is my first post. I want to loop or iterate in a directory to import each of the files as a separate DataFrame with a name similar (at least with the numeration) to the name of its file.
After a lot of research I still do not know how to do it. Obviously I am a very beginner :-)
My code is:
Main_folder = os.getcwd()
Folders = os.listdir('.')
for file in Folders:
data= pd.read_csv(file, sep="t", header=0)
data.columns=data.columns.str.strip()
where for instance Folders
is a list of files names including the file extension. e.g.:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT']
What I need is just to import all files to my work space such:
Load_01=pd.read_csv('01_load.TXT', sep="t", header=0)
Load_02=pd.read_csv('02_load.TXT', sep="t", header=0)
but in a loop since I have many files.
python pandas csv dataframe import
This is my first post. I want to loop or iterate in a directory to import each of the files as a separate DataFrame with a name similar (at least with the numeration) to the name of its file.
After a lot of research I still do not know how to do it. Obviously I am a very beginner :-)
My code is:
Main_folder = os.getcwd()
Folders = os.listdir('.')
for file in Folders:
data= pd.read_csv(file, sep="t", header=0)
data.columns=data.columns.str.strip()
where for instance Folders
is a list of files names including the file extension. e.g.:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT']
What I need is just to import all files to my work space such:
Load_01=pd.read_csv('01_load.TXT', sep="t", header=0)
Load_02=pd.read_csv('02_load.TXT', sep="t", header=0)
but in a loop since I have many files.
python pandas csv dataframe import
python pandas csv dataframe import
edited Nov 21 at 1:14
Mort
2,38211325
2,38211325
asked Nov 19 at 22:53
Alejandro BM
31
31
2
Can you use a dictionary?
– Neil
Nov 19 at 22:59
add a comment |
2
Can you use a dictionary?
– Neil
Nov 19 at 22:59
2
2
Can you use a dictionary?
– Neil
Nov 19 at 22:59
Can you use a dictionary?
– Neil
Nov 19 at 22:59
add a comment |
2 Answers
2
active
oldest
votes
up vote
0
down vote
accepted
When create objects in a loop you can't give them explicit names. But, you can add them to a data structure that associates them with a relevant name. I would recommend a dictionary here. So, for example, you can do this:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT'] # These should be called filenames not folders but anyway.
data_frames = {} # Initialise a dictionary
for filename in Folders:
df = pd.read_csv(filename, sep='t', header=False)
data_frames[filename] = df
# Now you can access any of the dataframes by the filename by using the dictionary:
# Let's say you want the df associated with 02_load.TXT
df = data_frames['02_load.TXT']
print(df.head())
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
add a comment |
up vote
2
down vote
You can use a dictionary:
data = {}
for file in os.listdir('.'):
data[file] = pd.read_csv(file, sep="t", header=0)
data[file].columns = data[file].columns.str.strip()
Then you access each dataframe as a key of the dictionary, for example: data['01_load.TXT']
It is possible to set variable variable names and access them, but it's not adviced or a good practice.
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
When create objects in a loop you can't give them explicit names. But, you can add them to a data structure that associates them with a relevant name. I would recommend a dictionary here. So, for example, you can do this:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT'] # These should be called filenames not folders but anyway.
data_frames = {} # Initialise a dictionary
for filename in Folders:
df = pd.read_csv(filename, sep='t', header=False)
data_frames[filename] = df
# Now you can access any of the dataframes by the filename by using the dictionary:
# Let's say you want the df associated with 02_load.TXT
df = data_frames['02_load.TXT']
print(df.head())
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
add a comment |
up vote
0
down vote
accepted
When create objects in a loop you can't give them explicit names. But, you can add them to a data structure that associates them with a relevant name. I would recommend a dictionary here. So, for example, you can do this:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT'] # These should be called filenames not folders but anyway.
data_frames = {} # Initialise a dictionary
for filename in Folders:
df = pd.read_csv(filename, sep='t', header=False)
data_frames[filename] = df
# Now you can access any of the dataframes by the filename by using the dictionary:
# Let's say you want the df associated with 02_load.TXT
df = data_frames['02_load.TXT']
print(df.head())
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
When create objects in a loop you can't give them explicit names. But, you can add them to a data structure that associates them with a relevant name. I would recommend a dictionary here. So, for example, you can do this:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT'] # These should be called filenames not folders but anyway.
data_frames = {} # Initialise a dictionary
for filename in Folders:
df = pd.read_csv(filename, sep='t', header=False)
data_frames[filename] = df
# Now you can access any of the dataframes by the filename by using the dictionary:
# Let's say you want the df associated with 02_load.TXT
df = data_frames['02_load.TXT']
print(df.head())
When create objects in a loop you can't give them explicit names. But, you can add them to a data structure that associates them with a relevant name. I would recommend a dictionary here. So, for example, you can do this:
Folders=['01_load.TXT', '02_load.TXT', '03_load.TXT'] # These should be called filenames not folders but anyway.
data_frames = {} # Initialise a dictionary
for filename in Folders:
df = pd.read_csv(filename, sep='t', header=False)
data_frames[filename] = df
# Now you can access any of the dataframes by the filename by using the dictionary:
# Let's say you want the df associated with 02_load.TXT
df = data_frames['02_load.TXT']
print(df.head())
answered Nov 19 at 23:04
Neil
48719
48719
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
add a comment |
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Thank you very much. So my mistake is that I cannot store different variables in a loop. Thus, I need to create an empty dictionary to store each of the files, in this case with the key as the name if the file and the value the content of my file imported as a data frame, right?
– Alejandro BM
Nov 20 at 9:22
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Pretty much yea you have the idea. Technically, it is possible to create variable variable names, but you don't do that unless you want to end up throwing your computer out the window (debugging would be a total nightmare). Remember that, technically, behind the scenes, python pretty much stores everything in dictionaries anyway. So basically using a dictionary achieves the same thing as variable variable names.
– Neil
Nov 20 at 11:23
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Also, it is technically possible to achieve this without first making an empty dictionary. You can use dictionary comprehension to create and populate the dictionary in one go. But don't worry about that yet if you don't know what it is.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Top level, yes, the key is the name of the file and the value is the dataframe and this is a good way of creating names with values in a loop.
– Neil
Nov 20 at 11:24
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
Super, thank you for the clarification :-)
– Alejandro BM
Nov 20 at 14:44
add a comment |
up vote
2
down vote
You can use a dictionary:
data = {}
for file in os.listdir('.'):
data[file] = pd.read_csv(file, sep="t", header=0)
data[file].columns = data[file].columns.str.strip()
Then you access each dataframe as a key of the dictionary, for example: data['01_load.TXT']
It is possible to set variable variable names and access them, but it's not adviced or a good practice.
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
add a comment |
up vote
2
down vote
You can use a dictionary:
data = {}
for file in os.listdir('.'):
data[file] = pd.read_csv(file, sep="t", header=0)
data[file].columns = data[file].columns.str.strip()
Then you access each dataframe as a key of the dictionary, for example: data['01_load.TXT']
It is possible to set variable variable names and access them, but it's not adviced or a good practice.
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
add a comment |
up vote
2
down vote
up vote
2
down vote
You can use a dictionary:
data = {}
for file in os.listdir('.'):
data[file] = pd.read_csv(file, sep="t", header=0)
data[file].columns = data[file].columns.str.strip()
Then you access each dataframe as a key of the dictionary, for example: data['01_load.TXT']
It is possible to set variable variable names and access them, but it's not adviced or a good practice.
You can use a dictionary:
data = {}
for file in os.listdir('.'):
data[file] = pd.read_csv(file, sep="t", header=0)
data[file].columns = data[file].columns.str.strip()
Then you access each dataframe as a key of the dictionary, for example: data['01_load.TXT']
It is possible to set variable variable names and access them, but it's not adviced or a good practice.
answered Nov 19 at 23:00
Julian Peller
844511
844511
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
add a comment |
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
Dear Julian, thank you very much. It works. Very appreciated.
– Alejandro BM
Nov 20 at 9:17
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53383806%2fhow-to-loop-to-import-several-csv-files-as-different-dataframes-and-each-of-the%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Can you use a dictionary?
– Neil
Nov 19 at 22:59