Why can't I crawl Amazon with Python?

+2 Aubree Keegan · August 23, 2014
I need to crawl Amazon with Python but it somehow doesn't seem to allow any crawling. Does anyone know how this is possible? I tried several different modules including Beautiful Soup 4. If anyone has any working crawler class, can you please share? Thx

Post a Reply


Oldest  Newest  Rating
+1 Bucky Roberts · August 24, 2014
Amazon has an API that you can use with Python. You can use that to crawl the site for items, prices, etc...
0 Yoncho Yonchev · August 25, 2014
Try scrapy, it is the easiest to configure, even from non programmers - http://doc.scrapy.org/en/latest/intro/tutorial.html

Maybe you need some help, so it will be nice to show some CODE, as I have forgotten my crystal ball. Showing what is wrong with your output might help too.
  • 1



This section is all about snakes! Just kidding.

Bucky Roberts Administrator