Unable to use google-cloud-storage in Airflow
I am not able to use google-cloud-storage in my Airflow.
I am using cloud composer for my Airflow environment.
Dag is not accepting it when I am using the below statement in my .py code
from google.cloud import storage
Its giving me an error saying that invalid character in identifier (MyPythonCode.py, line 6(where I am using that statement)
I have installed the required pypi package (google-cloud-storage) in my cloud composer PYPI Packages.
Whenever I am commenting that line, my DAG accepts .py file.
I have seen the same issue raised by other person Here, but when he added the required package, it started working. But not in my case.
google-cloud-platform google-cloud-storage airflow google-cloud-composer
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
add a comment |
I am not able to use google-cloud-storage in my Airflow.
I am using cloud composer for my Airflow environment.
Dag is not accepting it when I am using the below statement in my .py code
from google.cloud import storage
Its giving me an error saying that invalid character in identifier (MyPythonCode.py, line 6(where I am using that statement)
I have installed the required pypi package (google-cloud-storage) in my cloud composer PYPI Packages.
Whenever I am commenting that line, my DAG accepts .py file.
I have seen the same issue raised by other person Here, but when he added the required package, it started working. But not in my case.
google-cloud-platform google-cloud-storage airflow google-cloud-composer
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
Did you install it using the instructions in cloud.google.com/composer/docs/how-to/using/… ??
– kaxil
Nov 25 '18 at 19:47
It is possible you have invalid whitespace on that line. Hard to tell without seeing more. What version of Cloud Composer, Airflow, and Python?
– Tim Swast
Nov 26 '18 at 23:59
Did you copy paste the line? Take a look at this answer and see if it helps
– Kannappan Sirchabesan
Jan 19 at 12:23
add a comment |
I am not able to use google-cloud-storage in my Airflow.
I am using cloud composer for my Airflow environment.
Dag is not accepting it when I am using the below statement in my .py code
from google.cloud import storage
Its giving me an error saying that invalid character in identifier (MyPythonCode.py, line 6(where I am using that statement)
I have installed the required pypi package (google-cloud-storage) in my cloud composer PYPI Packages.
Whenever I am commenting that line, my DAG accepts .py file.
I have seen the same issue raised by other person Here, but when he added the required package, it started working. But not in my case.
google-cloud-platform google-cloud-storage airflow google-cloud-composer
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
I am not able to use google-cloud-storage in my Airflow.
I am using cloud composer for my Airflow environment.
Dag is not accepting it when I am using the below statement in my .py code
from google.cloud import storage
Its giving me an error saying that invalid character in identifier (MyPythonCode.py, line 6(where I am using that statement)
I have installed the required pypi package (google-cloud-storage) in my cloud composer PYPI Packages.
Whenever I am commenting that line, my DAG accepts .py file.
I have seen the same issue raised by other person Here, but when he added the required package, it started working. But not in my case.
google-cloud-platform google-cloud-storage airflow google-cloud-composer
google-cloud-platform google-cloud-storage airflow google-cloud-composer
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
asked Nov 24 '18 at 17:09
Ashish KumarAshish Kumar
19213
19213
Did you install it using the instructions in cloud.google.com/composer/docs/how-to/using/… ??
– kaxil
Nov 25 '18 at 19:47
It is possible you have invalid whitespace on that line. Hard to tell without seeing more. What version of Cloud Composer, Airflow, and Python?
– Tim Swast
Nov 26 '18 at 23:59
Did you copy paste the line? Take a look at this answer and see if it helps
– Kannappan Sirchabesan
Jan 19 at 12:23
add a comment |
Did you install it using the instructions in cloud.google.com/composer/docs/how-to/using/… ??
– kaxil
Nov 25 '18 at 19:47
It is possible you have invalid whitespace on that line. Hard to tell without seeing more. What version of Cloud Composer, Airflow, and Python?
– Tim Swast
Nov 26 '18 at 23:59
Did you copy paste the line? Take a look at this answer and see if it helps
– Kannappan Sirchabesan
Jan 19 at 12:23
Did you install it using the instructions in cloud.google.com/composer/docs/how-to/using/… ??
– kaxil
Nov 25 '18 at 19:47
Did you install it using the instructions in cloud.google.com/composer/docs/how-to/using/… ??
– kaxil
Nov 25 '18 at 19:47
It is possible you have invalid whitespace on that line. Hard to tell without seeing more. What version of Cloud Composer, Airflow, and Python?
– Tim Swast
Nov 26 '18 at 23:59
It is possible you have invalid whitespace on that line. Hard to tell without seeing more. What version of Cloud Composer, Airflow, and Python?
– Tim Swast
Nov 26 '18 at 23:59
Did you copy paste the line? Take a look at this answer and see if it helps
– Kannappan Sirchabesan
Jan 19 at 12:23
Did you copy paste the line? Take a look at this answer and see if it helps
– Kannappan Sirchabesan
Jan 19 at 12:23
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53460515%2funable-to-use-google-cloud-storage-in-airflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53460515%2funable-to-use-google-cloud-storage-in-airflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Did you install it using the instructions in cloud.google.com/composer/docs/how-to/using/… ??
– kaxil
Nov 25 '18 at 19:47
It is possible you have invalid whitespace on that line. Hard to tell without seeing more. What version of Cloud Composer, Airflow, and Python?
– Tim Swast
Nov 26 '18 at 23:59
Did you copy paste the line? Take a look at this answer and see if it helps
– Kannappan Sirchabesan
Jan 19 at 12:23