I need to crawl Amazon with Python but it somehow doesn't seem to allow any crawling. Does anyone know how this is possible? I tried several different modules including Beautiful Soup 4. If anyone has any working crawler class, can you please share? Thx
Why can't I crawl Amazon with Python?
Post a Reply
|Oldest Newest Rating|
· August 24, 2014
Amazon has an API that you can use with Python. You can use that to crawl the site for items, prices, etc...
· August 25, 2014
Try scrapy, it is the easiest to configure, even from non programmers - http://doc.scrapy.org/en/latest/intro/tutorial.html
Maybe you need some help, so it will be nice to show some CODE, as I have forgotten my crystal ball. Showing what is wrong with your output might help too.
This section is all about snakes! Just kidding.
|Bucky Roberts Administrator|