Dropbox API not returning all files in list
I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:
client = dropbox.Dropbox("ACCESS_TOKEN")
metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
flist =
if metadata.has_more == True:
m1 = metadata.entries
cur = metadata.cursor
for i in m1:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
# flist now has 2000 items
m2 = client.files_list_folder_continue(cur)
while m2.has_more == True:
for i in m2.entries:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
cur = m2.cursor
m2 = client.files_list_folder_continue(cur)
Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...
I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:
dbx = dropbox.Dropbox("ACCESS_TOKEN")
result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
file_list =
def process_entries(entries):
for entry in entries:
if isinstance(entry, dropbox.files.FileMetadata):
file_list.append([entry.name])
process_entries(result.entries)
while result.has_more:
result = dbx.files_list_folder_continue(result.cursor)
process_entries(result.entries)
print(len(file_list))
This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.
Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.
python dropbox-api
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
add a comment |
I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:
client = dropbox.Dropbox("ACCESS_TOKEN")
metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
flist =
if metadata.has_more == True:
m1 = metadata.entries
cur = metadata.cursor
for i in m1:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
# flist now has 2000 items
m2 = client.files_list_folder_continue(cur)
while m2.has_more == True:
for i in m2.entries:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
cur = m2.cursor
m2 = client.files_list_folder_continue(cur)
Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...
I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:
dbx = dropbox.Dropbox("ACCESS_TOKEN")
result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
file_list =
def process_entries(entries):
for entry in entries:
if isinstance(entry, dropbox.files.FileMetadata):
file_list.append([entry.name])
process_entries(result.entries)
while result.has_more:
result = dbx.files_list_folder_continue(result.cursor)
process_entries(result.entries)
print(len(file_list))
This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.
Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.
python dropbox-api
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?
– Greg
Nov 26 '18 at 19:08
Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.
– Greg
Nov 26 '18 at 19:08
Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call toprocess_entrieshas the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.
– Greg
Nov 26 '18 at 19:08
True @Greg. I changed the indentation ofprocess_entriesbecause Python gave me an indentation error for that line and I couldn't really figure out why.
– BallerNacken
Nov 26 '18 at 20:48
I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.
– BallerNacken
Nov 26 '18 at 21:00
add a comment |
I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:
client = dropbox.Dropbox("ACCESS_TOKEN")
metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
flist =
if metadata.has_more == True:
m1 = metadata.entries
cur = metadata.cursor
for i in m1:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
# flist now has 2000 items
m2 = client.files_list_folder_continue(cur)
while m2.has_more == True:
for i in m2.entries:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
cur = m2.cursor
m2 = client.files_list_folder_continue(cur)
Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...
I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:
dbx = dropbox.Dropbox("ACCESS_TOKEN")
result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
file_list =
def process_entries(entries):
for entry in entries:
if isinstance(entry, dropbox.files.FileMetadata):
file_list.append([entry.name])
process_entries(result.entries)
while result.has_more:
result = dbx.files_list_folder_continue(result.cursor)
process_entries(result.entries)
print(len(file_list))
This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.
Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.
python dropbox-api
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
I am trying to get a full file list from my dropbox account using the python dropbox API, to later compare the local and remote folders and download only the changed files on my Raspberry PI. So for now I need the current file list on my account. I am trying to accomplish that by using this code:
client = dropbox.Dropbox("ACCESS_TOKEN")
metadata = client.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
flist =
if metadata.has_more == True:
m1 = metadata.entries
cur = metadata.cursor
for i in m1:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
# flist now has 2000 items
m2 = client.files_list_folder_continue(cur)
while m2.has_more == True:
for i in m2.entries:
if isinstance(i, dropbox.files.FileMetadata):
flist.append([i.name])
cur = m2.cursor
m2 = client.files_list_folder_continue(cur)
Which gives me a total of 4971 files. But there are 5252 files in a lot of different folders (probably). I am getting a lot of different numbers. Windows gives me 5252 files using console, using the explorer it gives me 2344 files. Unix gives me 5260 files using console and 5288 files using fhe file manager...
I noticed that a lot of files are missing I added/edited recently (few days ago). After that I tried a slightly different code:
dbx = dropbox.Dropbox("ACCESS_TOKEN")
result = dbx.files_list_folder("", recursive=True, include_deleted=False, include_media_info=False)
file_list =
def process_entries(entries):
for entry in entries:
if isinstance(entry, dropbox.files.FileMetadata):
file_list.append([entry.name])
process_entries(result.entries)
while result.has_more:
result = dbx.files_list_folder_continue(result.cursor)
process_entries(result.entries)
print(len(file_list))
This code gives me only 1997 files, but the list is containing the new files. I don't get why this is happening, as the two codes are doing the same, but neither is capable of giving me the correct list of files.
Edit: Changed the second code accordingly to Gregs comment. Getting to many files now compared to the synchronized locale Dropbox folder.
python dropbox-api
python dropbox-api
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
edited Nov 26 '18 at 20:59
BallerNacken
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
asked Nov 25 '18 at 1:37
BallerNackenBallerNacken
73110
73110
When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?
– Greg
Nov 26 '18 at 19:08
Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.
– Greg
Nov 26 '18 at 19:08
Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call toprocess_entrieshas the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.
– Greg
Nov 26 '18 at 19:08
True @Greg. I changed the indentation ofprocess_entriesbecause Python gave me an indentation error for that line and I couldn't really figure out why.
– BallerNacken
Nov 26 '18 at 20:48
I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.
– BallerNacken
Nov 26 '18 at 21:00
add a comment |
When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?
– Greg
Nov 26 '18 at 19:08
Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.
– Greg
Nov 26 '18 at 19:08
Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call toprocess_entrieshas the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.
– Greg
Nov 26 '18 at 19:08
True @Greg. I changed the indentation ofprocess_entriesbecause Python gave me an indentation error for that line and I couldn't really figure out why.
– BallerNacken
Nov 26 '18 at 20:48
I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.
– BallerNacken
Nov 26 '18 at 21:00
When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?
– Greg
Nov 26 '18 at 19:08
When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?
– Greg
Nov 26 '18 at 19:08
Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.
– Greg
Nov 26 '18 at 19:08
Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.
– Greg
Nov 26 '18 at 19:08
Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to
process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.– Greg
Nov 26 '18 at 19:08
Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to
process_entries has the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.– Greg
Nov 26 '18 at 19:08
True @Greg. I changed the indentation of
process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.– BallerNacken
Nov 26 '18 at 20:48
True @Greg. I changed the indentation of
process_entries because Python gave me an indentation error for that line and I couldn't really figure out why.– BallerNacken
Nov 26 '18 at 20:48
I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.
– BallerNacken
Nov 26 '18 at 21:00
I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.
– BallerNacken
Nov 26 '18 at 21:00
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53463959%2fdropbox-api-not-returning-all-files-in-list%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53463959%2fdropbox-api-not-returning-all-files-in-list%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
When you say "there are 5252 files in a lot of different folders (probably)", what you do you mean by the "probably"?
– Greg
Nov 26 '18 at 19:08
Also, note that not all files from the local filesystem are synced to Dropbox. E.g., see the "Ignored files" section of: dropbox.com/help/syncing-uploads/files-not-syncing That will result in different total numbers. For that reason, comparing with the local filesystem won't work well. Likewise, are you counting folders from the local filesystem? Your API code is only counting files, ignoring the folders themselves.
– Greg
Nov 26 '18 at 19:08
Finally, I see you copied the second code from dropbox.com/developers/reference/…, but your second call to
process_entrieshas the wrong indentation. The way you have it, it will only run for the last page of results, and not earlier ones.– Greg
Nov 26 '18 at 19:08
True @Greg. I changed the indentation of
process_entriesbecause Python gave me an indentation error for that line and I couldn't really figure out why.– BallerNacken
Nov 26 '18 at 20:48
I guess there is no simple way to mirror the whole dropbox using the dropbox api on the Raspberry Pi? If yes, I would not need to try it this way.
– BallerNacken
Nov 26 '18 at 21:00