Python - Tutorial 27 (Webcrawler) question

0 denson devaro · October 6, 2015
Hi all,

I'm working through bucky's Python tutorials with great joy, but got stuck at the webcrawler part. Obviously this assignment has to be editted a little bit due to the fact 'buckysroom' doesnt excist no more, but thats no problem.

The thing i'm having trouble with is crawling img urls within a webpage. Which for the record is not part of the assignment, but seemed a great alternative. Everything else works btw.

Can anyone tell me what i'm doing wrong? Excuse me for not posting the entire code. Apparently theres a limit.
The img url im trying to crawl is the main profile picture (class mainProfilePic), from these user profile sites:

https://www.thenewboston.com/profile.php?user=2


for item_name in soup.findAll('img', {'class': 'mainProfilePic'}):



I tried all sorts of alternatives like below without any result:
for item_name in soup.findAll('a', {'div': 'mainProfilePic'}):
for item_name in soup.findAll('div', {'class': 'mainProfilePic'}):
for item_name in soup.findAll('a', {'img': 'mainProfilePic'}):

Post a Reply

Replies

Oldest  Newest  Rating
0 sfolje 0 · October 7, 2015

Apparently theres a limit. 0
Apparently theres a limit. 1
Apparently theres a limit. 2
Apparently theres a limit. 3
Apparently theres a limit. 4
Apparently theres a limit. 5
Apparently theres a limit. 6
Apparently theres a limit. 7
Apparently theres a limit. 8
Apparently theres a limit. 9
Apparently theres a limit. 10
Apparently theres a limit. 11
Apparently theres a limit. 12
Apparently theres a limit. 13
Apparently theres a limit. 14
Apparently theres a limit. 15
Apparently theres a limit. 16
Apparently theres a limit. 17
Apparently theres a limit. 18
Apparently theres a limit. 19
Apparently theres a limit. 20
Apparently theres a limit. 21
Apparently theres a limit. 22
Apparently theres a limit. 23
Apparently theres a limit. 24
Apparently theres a limit. 25
Apparently theres a limit. 26
Apparently theres a limit. 27
Apparently theres a limit. 28
Apparently theres a limit. 29
Apparently theres a limit. 30
Apparently theres a limit. 31
Apparently theres a limit. 32
Apparently theres a limit. 33
Apparently theres a limit. 34
Apparently theres a limit. 35
Apparently theres a limit. 36
Apparently theres a limit. 37
Apparently theres a limit. 38
Apparently theres a limit. 39
Apparently theres a limit. 40
Apparently theres a limit. 41
Apparently theres a limit. 42
Apparently theres a limit. 43
Apparently theres a limit. 44
Apparently theres a limit. 45
Apparently theres a limit. 46
Apparently theres a limit. 47
Apparently theres a limit. 48
Apparently theres a limit. 49
Apparently theres a limit. 50
Apparently theres a limit. 51
Apparently theres a limit. 52
Apparently theres a limit. 53
Apparently theres a limit. 54
Apparently theres a limit. 55
Apparently theres a limit. 56
Apparently theres a limit. 57
Apparently theres a limit. 58
Apparently theres a limit. 59
Apparently theres a limit. 60
Apparently theres a limit. 61
Apparently theres a limit. 62
Apparently theres a limit. 63
Apparently theres a limit. 64
Apparently theres a limit. 65
Apparently theres a limit. 66
Apparently theres a limit. 67
Apparently theres a limit. 68
Apparently theres a limit. 69
Apparently theres a limit. 70
Apparently theres a limit. 71
Apparently theres a limit. 72
Apparently theres a limit. 73
Apparently theres a limit. 74
Apparently theres a limit. 75
Apparently theres a limit. 76
Apparently theres a limit. 77
Apparently theres a limit. 78
Apparently theres a limit. 79
Apparently theres a limit. 80
Apparently theres a limit. 81
Apparently theres a limit. 82
Apparently theres a limit. 83
Apparently theres a limit. 84
Apparently theres a limit. 85
Apparently theres a limit. 86
Apparently theres a limit. 87
Apparently theres a limit. 88
Apparently theres a limit. 89
Apparently theres a limit. 90
Apparently theres a limit. 91
Apparently theres a limit. 92
Apparently theres a limit. 93
Apparently theres a limit. 94
Apparently theres a limit. 95
Apparently theres a limit. 96
Apparently theres a limit. 97
Apparently theres a limit. 98
Apparently theres a limit. 99
Apparently theres a limit. 100
Apparently theres a limit. 101
Apparently theres a limit. 102
Apparently theres a limit. 103
Apparently theres a limit. 104
Apparently theres a limit. 105
Apparently theres a limit. 106
Apparently theres a limit. 107
Apparently theres a limit. 108
Apparently theres a limit. 109
Apparently theres a limit. 110
Apparently theres a limit. 111
Apparently theres a limit. 112
Apparently theres a limit. 113
Apparently theres a limit. 114
Apparently theres a limit. 115
Apparently theres a limit. 116
Apparently theres a limit. 117
Apparently theres a limit. 118
Apparently theres a limit. 119
Apparently theres a limit. 120
Apparently theres a limit. 121
Apparently theres a limit. 122
Apparently theres a limit. 123
Apparently theres a limit. 124
Apparently theres a limit. 125
Apparently theres a limit. 126
Apparently theres a limit. 127
Apparently theres a limit. 128
Apparently theres a limit. 129
Apparently theres a limit. 130
Apparently theres a limit. 131
Apparently theres a limit. 132
Apparently theres a limit. 133
Apparently theres a limit. 134
Apparently theres a limit. 135
Apparently theres a limit. 136
Apparently theres a limit. 137
Apparently theres a limit. 138
Apparently theres a limit. 139
Apparently theres a limit. 140
Apparently theres a limit. 141
Apparently theres a limit. 142
Apparently theres a limit. 143
Apparently theres a limit. 144
Apparently theres a limit. 145
Apparently theres a limit. 146
Apparently theres a limit. 147
Apparently theres a limit. 148
Apparently theres a limit. 149
Apparently theres a limit. 150
Apparently theres a limit. 151
Apparently theres a limit. 152
Apparently theres a limit. 153
Apparently theres a limit. 154
Apparently theres a limit. 155
Apparently theres a limit. 156
Apparently theres a limit. 157
Apparently theres a limit. 158
Apparently theres a limit. 159
Apparently theres a limit. 160
Apparently theres a limit. 161
Apparently theres a limit. 162
Apparently theres a limit. 163
Apparently theres a limit. 164
Apparently theres a limit. 165
Apparently theres a limit. 166
Apparently theres a limit. 167
Apparently theres a limit. 168
Apparently theres a limit. 169
Apparently theres a limit. 170
Apparently theres a limit. 171
Apparently theres a limit. 172
Apparently theres a limit. 173
Apparently theres a limit. 174
Apparently theres a limit. 175
Apparently theres a limit. 176
Apparently theres a limit. 177
Apparently theres a limit. 178
Apparently theres a limit. 179
Apparently theres a limit. 180
Apparently theres a limit. 181
Apparently theres a limit. 182
Apparently theres a limit. 183
Apparently theres a limit. 184
Apparently theres a limit. 185
Apparently theres a limit. 186
Apparently theres a limit. 187
Apparently theres a limit. 188
Apparently theres a limit. 189
Apparently theres a limit. 190
Apparently theres a limit. 191
Apparently theres a limit. 192
Apparently theres a limit. 193
Apparently theres a limit. 194
Apparently theres a limit. 195
Apparently theres a limit. 196
Apparently theres a limit. 197
Apparently theres a limit. 198
Apparently theres a limit. 199
Apparently theres a limit. 200
Apparently theres a limit. 201
Apparently theres a limit. 202
Apparently theres a limit. 203
Apparently theres a limit. 204
Apparently theres a limit. 205
Apparently theres a limit. 206
Apparently theres a limit. 207
Apparently theres a limit. 208
Apparently theres a limit. 209
Apparently theres a limit. 210
Apparently theres a limit. 211
Apparently theres a limit. 212
Apparently theres a limit. 213
Apparently theres a limit. 214
Apparently theres a limit. 215
Apparently theres a limit. 216
Apparently theres a limit. 217
Apparently theres a limit. 218
Apparently theres a limit. 219
Apparently theres a limit. 220
Apparently theres a limit. 221
Apparently theres a limit. 222
Apparently theres a limit. 223
Apparently theres a limit. 224
Apparently theres a limit. 225
Apparently theres a limit. 226
Apparently theres a limit. 227
Apparently theres a limit. 228
Apparently theres a limit. 229
Apparently theres a limit. 230
Apparently theres a limit. 231
Apparently theres a limit. 232
Apparently theres a limit. 233
Apparently theres a limit. 234
Apparently theres a limit. 235
Apparently theres a limit. 236
Apparently theres a limit. 237
Apparently theres a limit. 238
Apparently theres a limit. 239
Apparently theres a limit. 240
Apparently theres a limit. 241
Apparently theres a limit. 242
Apparently theres a limit. 243
Apparently theres a limit. 244
Apparently theres a limit. 245
Apparently theres a limit. 246
Apparently theres a limit. 247
Apparently theres a limit. 248
Apparently theres a limit. 249
Apparently theres a limit. 250
Apparently theres a limit. 251
Apparently theres a limit. 252
Apparently theres a limit. 253
Apparently theres a limit. 254
Apparently theres a limit. 255
Apparently theres a limit. 256
Apparently theres a limit. 257
Apparently theres a limit. 258
Apparently theres a limit. 259
Apparently theres a limit. 260
Apparently theres a limit. 261
Apparently theres a limit. 262
Apparently theres a limit. 263
Apparently theres a limit. 264
Apparently theres a limit. 265
Apparently theres a limit. 266
Apparently theres a limit. 267
Apparently theres a limit. 268
Apparently theres a limit. 269
Apparently theres a limit. 270
Apparently theres a limit. 271
Apparently theres a limit. 272
Apparently theres a limit. 273
Apparently theres a limit. 274
Apparently theres a limit. 275
Apparently theres a limit. 276
Apparently theres a limit. 277
Apparently theres a limit. 278
Apparently theres a limit. 279
Apparently theres a limit. 280
Apparently theres a limit. 281
Apparently theres a limit. 282
Apparently theres a limit. 283
Apparently theres a limit. 284
Apparently theres a limit. 285
Apparently theres a limit. 286
Apparently theres a limit. 287
Apparently theres a limit. 288
Apparently theres a limit. 289
Apparently theres a limit. 290
Apparently theres a limit. 291
Apparently theres a limit. 292
Apparently theres a limit. 293
Apparently theres a limit. 294
Apparently theres a limit. 295
Apparently theres a limit. 296
Apparently theres a limit. 297
Apparently theres a limit. 298
Apparently theres a limit. 299
Apparently theres a limit. 300
Apparently theres a limit. 301
Apparently theres a limit. 302
Apparently theres a limit. 303
Apparently theres a limit. 304
Apparently theres a limit. 305
Apparently theres a limit. 306
Apparently theres a limit. 307
Apparently theres a limit. 308
Apparently theres a limit. 309
Apparently theres a limit. 310
Apparently theres a limit. 311
Apparently theres a limit. 312
Apparently theres a limit. 313
Apparently theres a limit. 314
Apparently theres a limit. 315
Apparently theres a limit. 316
Apparently theres a limit. 317
Apparently theres a limit. 318
Apparently theres a limit. 319
Apparently theres a limit. 320
Apparently theres a limit. 321
Apparently theres a limit. 322
Apparently theres a limit. 323
Apparently theres a limit. 324
Apparently theres a limit. 325
Apparently theres a limit. 326
Apparently theres a limit. 327
Apparently theres a limit. 328
Apparently theres a limit. 329
Apparently theres a limit. 330
Apparently theres a limit. 331
Apparently theres a limit. 332
Apparently theres a limit. 333
Apparently theres a limit. 334
Apparently theres a limit. 335
Apparently theres a limit. 336
Apparently theres a limit. 337
Apparently theres a limit. 338
Apparently theres a limit. 339
Apparently theres a limit. 340
Apparently theres a limit. 341
Apparently theres a limit. 342
Apparently theres a limit. 343
Apparently theres a limit. 344
Apparently theres a limit. 345
Apparently theres a limit. 346
Apparently theres a limit. 347
Apparently theres a limit. 348
Apparently theres a limit. 349
Apparently theres a limit. 350
Apparently theres a limit. 351
Apparently theres a limit. 352
Apparently theres a limit. 353
Apparently theres a limit. 354
Apparently theres a limit. 355
Apparently theres a limit. 356
Apparently theres a limit. 357
Apparently theres a limit. 358
Apparently theres a limit. 359
Apparently theres a limit. 360
Apparently theres a limit. 361
Apparently theres a limit. 362
Apparently theres a limit. 363
Apparently theres a limit. 364
Apparently theres a limit. 365
Apparently theres a limit. 366
Apparently theres a limit. 367
Apparently theres a limit. 368
Apparently theres a limit. 369
Apparently theres a limit. 370
Apparently theres a limit. 371
Apparently theres a limit. 372
Apparently theres a limit. 373
Apparently theres a limit. 374
Apparently theres a limit. 375
Apparently theres a limit. 376
Apparently theres a limit. 377
Apparently theres a limit. 378
Apparently theres a limit. 379
Apparently theres a limit. 380
Apparently theres a limit. 381
Apparently theres a limit. 382
Apparently theres a limit. 383
Apparently theres a limit. 384
Apparently theres a limit. 385
Apparently theres a limit. 386
Apparently theres a limit. 387
Apparently theres a limit. 388
Apparently theres a limit. 389
Apparently theres a limit. 390
Apparently theres a limit. 391
Apparently theres a limit. 392
Apparently theres a limit. 393
Apparently theres a limit. 394
Apparently theres a limit. 395
Apparently theres a limit. 396
Apparently theres a limit. 397
Apparently theres a limit. 398
Apparently theres a limit. 399
Apparently theres a limit. 400
Apparently theres a limit. 401
Apparently theres a limit. 402
Apparently theres a limit. 403
Apparently theres a limit. 404
Apparently theres a limit. 405
Apparently theres a limit. 406
Apparently theres a limit. 407
Apparently theres a limit. 408
Apparently theres a limit. 409
Apparently theres a limit. 410
Apparently theres a limit. 411
Apparently theres a limit. 412
Apparently theres a limit. 413
Apparently theres a limit. 414
Apparently theres a limit. 415
Apparently theres a limit. 416
Apparently theres a limit. 417
Apparently theres a limit. 418
Apparently theres a limit. 419
Apparently theres a limit. 420
Apparently theres a limit. 421
Apparently theres a limit. 422
Apparently theres a limit. 423
Apparently theres a limit. 424
Apparently theres a limit. 425
Apparently theres a limit. 426
Apparently theres a limit. 427
Apparently theres a limit. 428
Apparently theres a limit. 429
Apparently theres a limit. 430
Apparently theres a limit. 431
Apparently theres a limit. 432
Apparently theres a limit. 433
Apparently theres a limit. 434
Apparently theres a limit. 435
Apparently theres a limit. 436
Apparently theres a limit. 437
Apparently theres a limit. 438
Apparently theres a limit. 439
Apparently theres a limit. 440
Apparently theres a limit. 441
Apparently theres a limit. 442
Apparently theres a limit. 443
Apparently theres a limit. 444
Apparently theres a limit. 445
Apparently theres a limit. 446
Apparently theres a limit. 447
Apparently theres a limit. 448
Apparently theres a limit. 449
Apparently theres a limit. 450
Apparently theres a limit. 451
Apparently theres a limit. 452
Apparently theres a limit. 453
Apparently theres a limit. 454
Apparently theres a limit. 455
Apparently theres a limit. 456
Apparently theres a limit. 457
Apparently theres a limit. 458
Apparently theres a limit. 459
Apparently theres a limit. 460
Apparently theres a limit. 461
Apparently theres a limit. 462
Apparently theres a limit. 463
Apparently theres a limit. 464
Apparently theres a limit. 465
Apparently theres a limit. 466
Apparently theres a limit. 467
Apparently theres a limit. 468
Apparently theres a limit. 469
Apparently theres a limit. 470
Apparently theres a limit. 471
Apparently theres a limit. 472
Apparently theres a limit. 473
Apparently theres a limit. 474
Apparently theres a limit. 475
Apparently theres a limit. 476
Apparently theres a limit. 477
Apparently theres a limit. 478
Apparently theres a limit. 479
Apparently theres a limit. 480
Apparently theres a limit. 481
Apparently theres a limit. 482
Apparently theres a limit. 483
Apparently theres a limit. 484
Apparently theres a limit. 485
Apparently theres a limit. 486
Apparently theres a limit. 487
Apparently theres a limit. 488
Apparently theres a limit. 489
Apparently theres a limit. 490
Apparently theres a limit. 491
Apparently theres a limit. 492
Apparently theres a limit. 493
Apparently theres a limit. 494
Apparently theres a limit. 495
Apparently theres a limit. 496
Apparently theres a limit. 497
Apparently theres a limit. 498
Apparently theres a limit. 499
Apparently theres a limit. 500
Apparently theres a limit. 501
Apparently theres a limit. 502
Apparently theres a limit. 503
Apparently theres a limit. 504
Apparently theres a limit. 505
Apparently theres a limit. 506
Apparently theres a limit. 507
Apparently theres a limit. 508
Apparently theres a limit. 509
Apparently theres a limit. 510
Apparently theres a limit. 511
Apparently theres a limit. 512
Apparently theres a limit. 513
Apparently theres a limit. 514
Apparently theres a limit. 515
Apparently theres a limit. 516
Apparently theres a limit. 517
Apparently theres a limit. 518
Apparently theres a limit. 519
Apparently theres a limit. 520
Apparently theres a limit. 521
Apparently theres a limit. 522
Apparently theres a limit. 523
Apparently theres a limit. 524
Apparently theres a limit. 525
Apparently theres a limit. 526
Apparently theres a limit. 527
Apparently theres a limit. 528
Apparently theres a limit. 529
Apparently theres a limit. 530
Apparently theres a limit. 531
Apparently theres a limit. 532
Apparently theres a limit. 533
Apparently theres a limit. 534
Apparently theres a limit. 535
Apparently theres a limit. 536
Apparently theres a limit. 537
Apparently theres a limit. 538
Apparently theres a limit. 539
Apparently theres a limit. 540
Apparently theres a limit. 541
Apparently theres a limit. 542
Apparently theres a limit. 543
Apparently theres a limit. 544
Apparently theres a limit. 545
Apparently theres a limit. 546
Apparently theres a limit. 547
Apparently theres a limit. 548
Apparently theres a limit. 549
Apparently theres a limit. 550
Apparently theres a limit. 551
Apparently theres a limit. 552
Apparently theres a limit. 553
Apparently theres a limit. 554
Apparently theres a limit. 555
Apparently theres a limit. 556
Apparently theres a limit. 557
Apparently theres a limit. 558
Apparently theres a limit. 559
Apparently theres a limit. 560
Apparently theres a limit. 561
Apparently theres a limit. 562
Apparently theres a limit. 563
Apparently theres a limit. 564
Apparently theres a limit. 565
Apparently theres a limit. 566
Apparently theres a limit. 567
Apparently theres a limit. 568
Apparently theres a limit. 569
Apparently theres a limit. 570
Apparently theres a limit. 571
Apparently theres a limit. 572
Apparently theres a limit. 573
Apparently theres a limit. 574
Apparently theres a limit. 575
Apparently theres a limit. 576
Apparently theres a limit. 577
Apparently theres a limit. 578
Apparently theres a limit. 579
Apparently theres a limit. 580
Apparently theres a limit. 581
Apparently theres a limit. 582
Apparently theres a limit. 583
Apparently theres a limit. 584
Apparently theres a limit. 585
Apparently theres a limit. 586
Apparently theres a limit. 587
Apparently theres a limit. 588
Apparently theres a limit. 589
Apparently theres a limit. 590
Apparently theres a limit. 591
Apparently theres a limit. 592
Apparently theres a limit. 593
Apparently theres a limit. 594
Apparently theres a limit. 595
Apparently theres a limit. 596
Apparently theres a limit. 597
Apparently theres a limit. 598
Apparently theres a limit. 599
Apparently theres a limit. 600
Apparently theres a limit. 601
Apparently theres a limit. 602
Apparently theres a limit. 603
Apparently theres a limit. 604
Apparently theres a limit. 605
Apparently theres a limit. 606
Apparently theres a limit. 607
Apparently theres a limit. 608
Apparently theres a limit. 609
Apparently theres a limit. 610
Apparently theres a limit. 611
Apparently theres a limit. 612
Apparently theres a limit. 613
Apparently theres a limit. 614
Apparently theres a limit. 615
Apparently theres a limit. 616
Apparently theres a limit. 617
Apparently theres a limit. 618
Apparently theres a limit. 619
Apparently theres a limit. 620
Apparently theres a limit. 621
Apparently theres a limit. 622
Apparently theres a limit. 623
Apparently theres a limit. 624
Apparently theres a limit. 625
Apparently theres a limit. 626
Apparently theres a limit. 627
Apparently theres a limit. 628
Apparently theres a limit. 629
Apparently theres a limit. 630
Apparently theres a limit. 631
Apparently theres a limit. 632
Apparently theres a limit. 633
Apparently theres a limit. 634
Apparently theres a limit. 635
Apparently theres a limit. 636
Apparently theres a limit. 637
Apparently theres a limit. 638
Apparently theres a limit. 639
Apparently theres a limit. 640
Apparently theres a limit. 641
Apparently theres a limit. 642
Apparently theres a limit. 643
Apparently theres a limit. 644
Apparently theres a limit. 645
Apparently theres a limit. 646
Apparently theres a limit. 647
Apparently theres a limit. 648
Apparently theres a limit. 649
Apparently theres a limit. 650
Apparently theres a limit. 651
Apparently theres a limit. 652
Apparently theres a limit. 653
Apparently theres a limit. 654
Apparently theres a limit. 655
Apparently theres a limit. 656
Apparently theres a limit. 657
Apparently theres a limit. 658
Apparently theres a limit. 659
Apparently theres a limit. 660
Apparently theres a limit. 661
Apparently theres a limit. 662
Apparently theres a limit. 663
Apparently theres a limit. 664
Apparently theres a limit. 665
Apparently theres a limit. 666
Apparently theres a limit. 667
Apparently theres a limit. 668
Apparently theres a limit. 669
Apparently theres a limit. 670
Apparently theres a limit. 671
Apparently theres a limit. 672
Apparently theres a limit. 673
Apparently theres a limit. 674
Apparently theres a limit. 675
Apparently theres a limit. 676
Apparently theres a limit. 677
Apparently theres a limit. 678
Apparently theres a limit. 679
Apparently theres a limit. 680
Apparently theres a limit. 681
Apparently theres a limit. 682
Apparently theres a limit. 683
Apparently theres a limit. 684
Apparently theres a limit. 685
Apparently theres a limit. 686
Apparently theres a limit. 687
Apparently theres a limit. 688
Apparently theres a limit. 689
Apparently theres a limit. 690
Apparently theres a limit. 691
Apparently theres a limit. 692
Apparently theres a limit. 693
Apparently theres a limit. 694
Apparently theres a limit. 695
Apparently theres a limit. 696
Apparently theres a limit. 697
Apparently theres a limit. 698
Apparently theres a limit. 699
Apparently theres a limit. 700
Apparently theres a limit. 701
Apparently theres a limit. 702
Apparently theres a limit. 703
Apparently theres a limit. 704
Apparently theres a limit. 705
Apparently theres a limit. 706
Apparently theres a limit. 707
Apparently theres a limit. 708
Apparently theres a limit. 709
Apparently theres a limit. 710
Apparently theres a limit. 711
Apparently theres a limit. 712
Apparently theres a limit. 713
Apparently theres a limit. 714
Apparently theres a limit. 715
Apparently theres a limit. 716
Apparently theres a limit. 717
Apparently theres a limit. 718
Apparently theres a limit. 719
Apparently theres a limit. 720
Apparently theres a limit. 721
Apparently theres a limit. 722
Apparently theres a limit. 723
Apparently theres a limit. 724
Apparently theres a limit. 725
Apparently theres a limit. 726
Apparently theres a limit. 727
Apparently theres a limit. 728
Apparently theres a limit. 729
Apparently theres a limit. 730
Apparently theres a limit. 731
Apparently theres a limit. 732
Apparently theres a limit. 733
Apparently theres a limit. 734
Apparently theres a limit. 735
Apparently theres a limit. 736
Apparently theres a limit. 737
Apparently theres a limit. 738
Apparently theres a limit. 739
Apparently theres a limit. 740
Apparently theres a limit. 741
Apparently theres a limit. 742
Apparently theres a limit. 743
Apparently theres a limit. 744
Apparently theres a limit. 745
Apparently theres a limit. 746
Apparently theres a limit. 747
Apparently theres a limit. 748
Apparently theres a limit. 749
Apparently theres a limit. 750
Apparently theres a limit. 751
Apparently theres a limit. 752
Apparently theres a limit. 753
Apparently theres a limit. 754
Apparently theres a limit. 755
Apparently theres a limit. 756
Apparently theres a limit. 757
Apparently theres a limit. 758
Apparently theres a limit. 759
Apparently theres a limit. 760
Apparently theres a limit. 761
Apparently theres a limit. 762
Apparently theres a limit. 763
Apparently theres a limit. 764
Apparently theres a limit. 765
Apparently theres a limit. 766
Apparently theres a limit. 767
Apparently theres a limit. 768
Apparently theres a limit. 769
Apparently theres a limit. 770
Apparently theres a limit. 771
Apparently theres a limit. 772
Apparently theres a limit. 773
Apparently theres a limit. 774
Apparently theres a limit. 775
Apparently theres a limit. 776
Apparently theres a limit. 777
Apparently theres a limit. 778
Apparently theres a limit. 779
Apparently theres a limit. 780
Apparently theres a limit. 781
Apparently theres a limit. 782
Apparently theres a limit. 783
Apparently theres a limit. 784
Apparently theres a limit. 785
Apparently theres a limit. 786
Apparently theres a limit. 787
Apparently theres a limit. 788
Apparently theres a limit. 789
Apparently theres a limit. 790
Apparently theres a limit. 791
Apparently theres a limit. 792
Apparently theres a limit. 793
Apparently theres a limit. 794
Apparently theres a limit. 795
Apparently theres a limit. 796
Apparently theres a limit. 797
Apparently theres a limit. 798
Apparently theres a limit. 799
Apparently theres a limit. 800
Apparently theres a limit. 801
Apparently theres a limit. 802
Apparently theres a limit. 803
Apparently theres a limit. 804
Apparently theres a limit. 805
Apparently theres a limit. 806
Apparently theres a limit. 807
Apparently theres a limit. 808
Apparently theres a limit. 809
Apparently theres a limit. 810
Apparently theres a limit. 811
Apparently theres a limit. 812
Apparently theres a limit. 813
Apparently theres a limit. 814
Apparently theres a limit. 815
Apparently theres a limit. 816
Apparently theres a limit. 817
Apparently theres a limit. 818
Apparently theres a limit. 819
Apparently theres a limit. 820
Apparently theres a limit. 821
Apparently theres a limit. 822
Apparently theres a limit. 823
Apparently theres a limit. 824
Apparently theres a limit. 825
Apparently theres a limit. 826
Apparently theres a limit. 827
Apparently theres a limit. 828
Apparently theres a limit. 829
Apparently theres a limit. 830
Apparently theres a limit. 831
Apparently theres a limit. 832
Apparently theres a limit. 833
Apparently theres a limit. 834
Apparently theres a limit. 835
Apparently theres a limit. 836
Apparently theres a limit. 837
Apparently theres a limit. 838
Apparently theres a limit. 839
Apparently theres a limit. 840
Apparently theres a limit. 841
Apparently theres a limit. 842
Apparently theres a limit. 843
Apparently theres a limit. 844
Apparently theres a limit. 845
Apparently theres a limit. 846
Apparently theres a limit. 847
Apparently theres a limit. 848
Apparently theres a limit. 849
Apparently theres a limit. 850
Apparently theres a limit. 851
Apparently theres a limit. 852
Apparently theres a limit. 853
Apparently theres a limit. 854
Apparently theres a limit. 855
Apparently theres a limit. 856
Apparently theres a limit. 857
Apparently theres a limit. 858
Apparently theres a limit. 859
Apparently theres a limit. 860
Apparently theres a limit. 861
Apparently theres a limit. 862
Apparently theres a limit. 863
Apparently theres a limit. 864
Apparently theres a limit. 865
Apparently theres a limit. 866
Apparently theres a limit. 867
Apparently theres a limit. 868
Apparently theres a limit. 869
Apparently theres a limit. 870
Apparently theres a limit. 871
Apparently theres a limit. 872
Apparently theres a limit. 873
Apparently theres a limit. 874
Apparently theres a limit. 875
Apparently theres a limit. 876
Apparently theres a limit. 877
Apparently theres a limit. 878
Apparently theres a limit. 879
Apparently theres a limit. 880
Apparently theres a limit. 881
Apparently theres a limit. 882
Apparently theres a limit. 883
Apparently theres a limit. 884
Apparently theres a limit. 885
Apparently theres a limit. 886
Apparently theres a limit. 887
Apparently theres a limit. 888
Apparently theres a limit. 889
Apparently theres a limit. 890
Apparently theres a limit. 891
Apparently theres a limit. 892
Apparently theres a limit. 893
Apparently theres a limit. 894
Apparently theres a limit. 895
Apparently theres a limit. 896
Apparently theres a limit. 897
Apparently theres a limit. 898
Apparently theres a limit. 899
Apparently theres a limit. 900
Apparently theres a limit. 901
Apparently theres a limit. 902
Apparently theres a limit. 903
Apparently theres a limit. 904
Apparently theres a limit. 905
Apparently theres a limit. 906
Apparently theres a limit. 907
Apparently theres a limit. 908
Apparently theres a limit. 909
Apparently theres a limit. 910
Apparently theres a limit. 911
Apparently theres a limit. 912
Apparently theres a limit. 913
Apparently theres a limit. 914
Apparently theres a limit. 915
Apparently theres a limit. 916
Apparently theres a limit. 917
Apparently theres a limit. 918
Apparently theres a limit. 919
Apparently theres a limit. 920
Apparently theres a limit. 921
Apparently theres a limit. 922
Apparently theres a limit. 923
Apparently theres a limit. 924
Apparently theres a limit. 925
Apparently theres a limit. 926
Apparently theres a limit. 927
Apparently theres a limit. 928
Apparently theres a limit. 929
Apparently theres a limit. 930
Apparently theres a limit. 931
Apparently theres a limit. 932
Apparently theres a limit. 933
Apparently theres a limit. 934
Apparently theres a limit. 935
Apparently theres a limit. 936
Apparently theres a limit. 937
Apparently theres a limit. 938
Apparently theres a limit. 939
Apparently theres a limit. 940
Apparently theres a limit. 941
Apparently theres a limit. 942
Apparently theres a limit. 943
Apparently theres a limit. 944
Apparently theres a limit. 945
Apparently theres a limit. 946
Apparently theres a limit. 947
Apparently theres a limit. 948
Apparently theres a limit. 949
Apparently theres a limit. 950
Apparently theres a limit. 951
Apparently theres a limit. 952
Apparently theres a limit. 953
Apparently theres a limit. 954
Apparently theres a limit. 955
Apparently theres a limit. 956
Apparently theres a limit. 957
Apparently theres a limit. 958
Apparently theres a limit. 959
Apparently theres a limit. 960
Apparently theres a limit. 961
Apparently theres a limit. 962
Apparently theres a limit. 963
Apparently theres a limit. 964
Apparently theres a limit. 965
Apparently theres a limit. 966
Apparently theres a limit. 967
Apparently theres a limit. 968
Apparently theres a limit. 969
Apparently theres a limit. 970
Apparently theres a limit. 971
Apparently theres a limit. 972
Apparently theres a limit. 973
Apparently theres a limit. 974
Apparently theres a limit. 975
Apparently theres a limit. 976
Apparently theres a limit. 977
Apparently theres a limit. 978
Apparently theres a limit. 979
Apparently theres a limit. 980
Apparently theres a limit. 981
Apparently theres a limit. 982
Apparently theres a limit. 983
Apparently theres a limit. 984
Apparently theres a limit. 985
Apparently theres a limit. 986
Apparently theres a limit. 987
Apparently theres a limit. 988
Apparently theres a limit. 989
Apparently theres a limit. 990
Apparently theres a limit. 991
Apparently theres a limit. 992
Apparently theres a limit. 993
Apparently theres a limit. 994
Apparently theres a limit. 995
Apparently theres a limit. 996
Apparently theres a limit. 997
Apparently theres a limit. 998
Apparently theres a limit. 999

or is there?


but i noticed timeout blockage: if you write post too much time, then it will post some crap, lets say first minute of what u have written. Problem can be approached, if u write ur post first in some kind of notepad, and then copy-paste it to the post, and click 'Post'.
0 sfolje 0 · October 7, 2015
Answer to question trying not to be annoying:

Your solution works fine for me.
Tell us more about the proplem.
0 denson devaro · October 19, 2015
Excuse moi for the late reply. When i fire up the following code i get the site, username, [None] for each result. I was hoping to crawl the image url from the profile picture with the nested "get_single_item_data".

Code:

import requests
from bs4 import BeautifulSoup

def search_spider(max_pages):
page = 1
while page <= max_pages:
webpage = 'https://www.thenewboston.com/search.php?type=0&sort=reputation&page=' + str(page)
source_webpage = requests.get(webpage)
plain_text = source_webpage.text
soup = BeautifulSoup(plain_text)
for link in soup.findAll('a', {'class': 'desc-title'}):
href = "https://www.thenewboston.com" + link.get('href')
title = link.string
print(href)
print(title)
get_single_item_data(href)
page += 1

def get_single_item_data(item_url):
source_webpage = requests.get(item_url)
plain_text = source_webpage.text
soup = BeautifulSoup(plain_text)
for item_name in soup.findAll('img', {'class': 'mainProfilePic'}):
print(item_name.string)

search_spider(1)
0 sfolje 0 · October 21, 2015
Yeah, now i got it, brah.
As i wrote in the post before, your code 'works fine' for me and i 'get' the right result - your code 'crawls' actual image url from the profile picture.
Lets just forget about the last sentence and say that you dont have to use .string method on beautifulsoup object to actually show it.

---
Please let me know, if it helps or not. Also glad, for respect after me being annoying in first post (:
  • 1

Python

106,979 followers
About

This section is all about snakes! Just kidding.

Links
Moderators
Bucky Roberts Administrator