Ytukyg

I am trying to scrape prices from online commerce store. I am iterating through the products on page and included it in the regular expression. Despite escaping the curly brackets, the regular expression does not work. (findall returns an empty list)

HTML code returned by soup.findall:

[<div class="ps4-price at-min-price-1"> from 29 GBP </div>]

[<div class="ps4-price at-min-price-2"> from 35 GBP </div>]

Python coode:

for product in range(21):



        min_prices_text = str(soup.findAll("div", class_="ps4-price at- 

        min- price-{}".format(product)))



        min_price = re.findall('<div class="ps4-price at-min-price- 

        {{}}"> (.+?)<'.format(product), str(min_prices_text))

You may access the .string property of the elements you get with findAll, and only apply the regex to the plain text. E.g., since you expect only single integer numbers there, you may apply re.sub(r'D+', '', min_prices_text.string) on those strings.

See example code:

results = 

for product in range(21):

    min_prices_text = soup.find("div", class_="ps4-price at-min-price-{}".format(product))

    if min_prices_text:

        results.append(re.sub(r'D+', '', min_prices_text.string))



print(results) # => ['29', '35']

Or use list(map(int, results)) if you want to cast the list of strings to integer.