“Type 'lxml.etree._ElementUnicodeResult' cannot be serialized”
1
I am using lxml to extract data from web pages, but I am unable to convert the resulting ElementUnicode object to a string. Here is my code:
from lxml import html
from lxml import etree
from lxml.etree import tostring
url = 'https://www.imdb.com/title/tt5848272/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2413b25e-e3f6-4229-9efd-599bb9ab1f97&pf_rd_r=9S5A89ZHEXE4K8SZBC40&pf_rd_s=right-2&pf_rd_t=15061&pf_rd_i=homepage&ref_=hm_otw_t0'
page = requests.get('url')
tree = html.fromstring(page.content)
a = tree.xpath('//div[@class="credit_summary_item"]/a[../h4/text() = "Directors:"]/text()')
mynewlist =
for i in a:
b = etree.tostring(i, method="text")
mynewlist.append(b)
Here is the error I get:
TypeError: Type 'lxml.etree._ElementUnicodeResult' cannot be serialized.
Any help would be greatly appreciated.
python web-scraping lxml
asked Nov 24 '18 at 15:35
FinFin
105
|
show 2 more comments
1
I am using lxml to extract data from web pages, but I am unable to convert the resulting ElementUnicode object to a string. Here is my code:
from lxml import html
from lxml import etree
from lxml.etree import tostring
url = 'https://www.imdb.com/title/tt5848272/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2413b25e-e3f6-4229-9efd-599bb9ab1f97&pf_rd_r=9S5A89ZHEXE4K8SZBC40&pf_rd_s=right-2&pf_rd_t=15061&pf_rd_i=homepage&ref_=hm_otw_t0'
page = requests.get('url')
tree = html.fromstring(page.content)
a = tree.xpath('//div[@class="credit_summary_item"]/a[../h4/text() = "Directors:"]/text()')
mynewlist =
for i in a:
b = etree.tostring(i, method="text")
mynewlist.append(b)
Here is the error I get:
TypeError: Type 'lxml.etree._ElementUnicodeResult' cannot be serialized.
Any help would be greatly appreciated.
python web-scraping lxml
asked Nov 24 '18 at 15:35
FinFin
105
1
can you give us the xml file you're parsing? In my experiencetree.xpath('//path/text()')should output a list of strings so what you're doing next is a little weird to me
– Silmathoron
Nov 24 '18 at 17:18
I'm scraping from web pages so no XML, I was under the impression that lxml could do that.
– Fin
Nov 24 '18 at 18:00
1
Then an example HTML which reproduces the problem; this is the idea of a minimal working example: we must be able to run the code
– Silmathoron
Nov 24 '18 at 18:05
Yes, please provide a Minimal, Complete, and Verifiable example.
– mzjn
Nov 25 '18 at 6:55
I edited the question with the URL link I'm working on and the XPath query.
– Fin
Nov 25 '18 at 10:02
|
show 2 more comments
1
1
1
I am using lxml to extract data from web pages, but I am unable to convert the resulting ElementUnicode object to a string. Here is my code:
from lxml import html
from lxml import etree
from lxml.etree import tostring
url = 'https://www.imdb.com/title/tt5848272/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2413b25e-e3f6-4229-9efd-599bb9ab1f97&pf_rd_r=9S5A89ZHEXE4K8SZBC40&pf_rd_s=right-2&pf_rd_t=15061&pf_rd_i=homepage&ref_=hm_otw_t0'
page = requests.get('url')
tree = html.fromstring(page.content)
a = tree.xpath('//div[@class="credit_summary_item"]/a[../h4/text() = "Directors:"]/text()')
mynewlist =
for i in a:
b = etree.tostring(i, method="text")
mynewlist.append(b)
Here is the error I get:
TypeError: Type 'lxml.etree._ElementUnicodeResult' cannot be serialized.
Any help would be greatly appreciated.
python web-scraping lxml
asked Nov 24 '18 at 15:35
FinFin
105
I am using lxml to extract data from web pages, but I am unable to convert the resulting ElementUnicode object to a string. Here is my code:
from lxml import html
from lxml import etree
from lxml.etree import tostring
url = 'https://www.imdb.com/title/tt5848272/?pf_rd_m=A2FGELUUNOQJNL&pf_rd_p=2413b25e-e3f6-4229-9efd-599bb9ab1f97&pf_rd_r=9S5A89ZHEXE4K8SZBC40&pf_rd_s=right-2&pf_rd_t=15061&pf_rd_i=homepage&ref_=hm_otw_t0'
page = requests.get('url')
tree = html.fromstring(page.content)
a = tree.xpath('//div[@class="credit_summary_item"]/a[../h4/text() = "Directors:"]/text()')
mynewlist =
for i in a:
b = etree.tostring(i, method="text")
mynewlist.append(b)
Here is the error I get:
TypeError: Type 'lxml.etree._ElementUnicodeResult' cannot be serialized.
Any help would be greatly appreciated.
python web-scraping lxml
python web-scraping lxml
asked Nov 24 '18 at 15:35
FinFin
105
asked Nov 24 '18 at 15:35
FinFin
105
edited Nov 25 '18 at 10:02
Fin
asked Nov 24 '18 at 15:35
FinFin
105
asked Nov 24 '18 at 15:35
FinFin
105
asked Nov 24 '18 at 15:35
FinFin
105
105
1
can you give us the xml file you're parsing? In my experiencetree.xpath('//path/text()')should output a list of strings so what you're doing next is a little weird to me
– Silmathoron
Nov 24 '18 at 17:18
I'm scraping from web pages so no XML, I was under the impression that lxml could do that.
– Fin
Nov 24 '18 at 18:00
1
Then an example HTML which reproduces the problem; this is the idea of a minimal working example: we must be able to run the code
– Silmathoron
Nov 24 '18 at 18:05
Yes, please provide a Minimal, Complete, and Verifiable example.
– mzjn
Nov 25 '18 at 6:55
I edited the question with the URL link I'm working on and the XPath query.
– Fin
Nov 25 '18 at 10:02
|
show 2 more comments
1
can you give us the xml file you're parsing? In my experiencetree.xpath('//path/text()')should output a list of strings so what you're doing next is a little weird to me
– Silmathoron
Nov 24 '18 at 17:18
I'm scraping from web pages so no XML, I was under the impression that lxml could do that.
– Fin
Nov 24 '18 at 18:00
1
Then an example HTML which reproduces the problem; this is the idea of a minimal working example: we must be able to run the code
– Silmathoron
Nov 24 '18 at 18:05
Yes, please provide a Minimal, Complete, and Verifiable example.
– mzjn
Nov 25 '18 at 6:55
I edited the question with the URL link I'm working on and the XPath query.
– Fin
Nov 25 '18 at 10:02
1
1
can you give us the xml file you're parsing? In my experience
tree.xpath('//path/text()') should output a list of strings so what you're doing next is a little weird to me– Silmathoron
Nov 24 '18 at 17:18
can you give us the xml file you're parsing? In my experience
tree.xpath('//path/text()') should output a list of strings so what you're doing next is a little weird to me– Silmathoron
Nov 24 '18 at 17:18
I'm scraping from web pages so no XML, I was under the impression that lxml could do that.
– Fin
Nov 24 '18 at 18:00
I'm scraping from web pages so no XML, I was under the impression that lxml could do that.
– Fin
Nov 24 '18 at 18:00
1
1
Then an example HTML which reproduces the problem; this is the idea of a minimal working example: we must be able to run the code
– Silmathoron
Nov 24 '18 at 18:05
Then an example HTML which reproduces the problem; this is the idea of a minimal working example: we must be able to run the code
– Silmathoron
Nov 24 '18 at 18:05
Yes, please provide a Minimal, Complete, and Verifiable example.
– mzjn
Nov 25 '18 at 6:55
Yes, please provide a Minimal, Complete, and Verifiable example.
– mzjn
Nov 25 '18 at 6:55
I edited the question with the URL link I'm working on and the XPath query.
– Fin
Nov 25 '18 at 10:02
I edited the question with the URL link I'm working on and the XPath query.
– Fin
Nov 25 '18 at 10:02
|
show 2 more comments
1 Answer
1
active
oldest
votes
0
The i variable is an _ElementUnicodeResult object (a special type of string). You cannot use it as an argument to tostring().
The a variable (the result of the XPath evaluation) is the list of strings that you want. If the elements of this list must be plain strings instead of _ElementUnicodeResult objects, you can use a list comprehension:
newlist = [str(s) for s in a]
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
newlist = [str(s) for s in a]should do it.
– mzjn
Nov 25 '18 at 11:36
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53459703%2ftype-lxml-etree-elementunicoderesult-cannot-be-serialized%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
The i variable is an _ElementUnicodeResult object (a special type of string). You cannot use it as an argument to tostring().
The a variable (the result of the XPath evaluation) is the list of strings that you want. If the elements of this list must be plain strings instead of _ElementUnicodeResult objects, you can use a list comprehension:
newlist = [str(s) for s in a]
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
newlist = [str(s) for s in a]should do it.
– mzjn
Nov 25 '18 at 11:36
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
add a comment |
0
The i variable is an _ElementUnicodeResult object (a special type of string). You cannot use it as an argument to tostring().
The a variable (the result of the XPath evaluation) is the list of strings that you want. If the elements of this list must be plain strings instead of _ElementUnicodeResult objects, you can use a list comprehension:
newlist = [str(s) for s in a]
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
newlist = [str(s) for s in a]should do it.
– mzjn
Nov 25 '18 at 11:36
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
add a comment |
0
0
0
The i variable is an _ElementUnicodeResult object (a special type of string). You cannot use it as an argument to tostring().
The a variable (the result of the XPath evaluation) is the list of strings that you want. If the elements of this list must be plain strings instead of _ElementUnicodeResult objects, you can use a list comprehension:
newlist = [str(s) for s in a]
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
The i variable is an _ElementUnicodeResult object (a special type of string). You cannot use it as an argument to tostring().
The a variable (the result of the XPath evaluation) is the list of strings that you want. If the elements of this list must be plain strings instead of _ElementUnicodeResult objects, you can use a list comprehension:
newlist = [str(s) for s in a]
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
edited Nov 25 '18 at 15:31
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
answered Nov 25 '18 at 11:00
mzjnmzjn
32k669156
32k669156
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
newlist = [str(s) for s in a]should do it.
– mzjn
Nov 25 '18 at 11:36
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
add a comment |
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
newlist = [str(s) for s in a]should do it.
– mzjn
Nov 25 '18 at 11:36
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
The problem is that the resulting list is composed of _ElementUnicodeResult strings; I am trying to convert these to plain strings.
– Fin
Nov 25 '18 at 11:29
newlist = [str(s) for s in a] should do it.– mzjn
Nov 25 '18 at 11:36
newlist = [str(s) for s in a] should do it.– mzjn
Nov 25 '18 at 11:36
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
Thank you, this worked for me.
– Fin
Nov 25 '18 at 14:46
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53459703%2ftype-lxml-etree-elementunicoderesult-cannot-be-serialized%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
can you give us the xml file you're parsing? In my experience
tree.xpath('//path/text()')should output a list of strings so what you're doing next is a little weird to me– Silmathoron
Nov 24 '18 at 17:18
I'm scraping from web pages so no XML, I was under the impression that lxml could do that.
– Fin
Nov 24 '18 at 18:00
1
Then an example HTML which reproduces the problem; this is the idea of a minimal working example: we must be able to run the code
– Silmathoron
Nov 24 '18 at 18:05
Yes, please provide a Minimal, Complete, and Verifiable example.
– mzjn
Nov 25 '18 at 6:55
I edited the question with the URL link I'm working on and the XPath query.
– Fin
Nov 25 '18 at 10:02