Python3 - interacting with HTML pages help

+4 James Kon · October 30, 2015
I need a plugin that allows me to do these things:

  • Read HTML codes

  • Simulate the clicking of a button

  • Be able to submit forms from a HTML page in the background of the computer

  • Can enter text into input boxes

  • Able to simulate the pressing of buttons, like pressing the enter button



Making it be in the background, and not maximized is optional, but would prefer that the user not see it.inter

Post a Reply

Replies

- page 2
Oldest  Newest  Rating
0 James Kon · October 30, 2015
Halcyon Abraham Ramirez,

Yes you did. However, selenium had a lot of trouble reading the page coding. At least for me. I tried multiple times to get it to read the page title. It also didn't seem to have many good tutorials on using it.
0 sfolje 0 · October 30, 2015
Here things get dirty.
I think (, im not sure, but) technically you dont really need to click the button (or input text etc.), to access a website behind the form, you just need to perform post method (or some kind of command, that html+javascript uses in background) to process webpage form.
Some times there are some kind of sessions, or cookies, so you cannot access the needed webpage without a session id or key  etc. so you need to first get cookie (or session) id (in my experience manually by clikcing the button) and put it in headers as @alex constantin mensioned. Things can get tricky pretty easily.

tool: requests
0 Halcyon Abraham Ramirez · October 30, 2015
@alex constantin

you can post data to forms with request
but how do you simulate the button press with it?

and didn't I mention selenium before ? :/
0 alex constantin · October 30, 2015
I have never seen a plugin that can do this but I know a way of posting data on a form that uses POST method: 
You need the requests module, which you can download using pip install requests



import requests
formData = {'username': "myusername", \
   'password': "mypass", \
   'otherForm': "whatever"
}
url="https://urltoformpage.com"

headers = {"add headers":"here"}
r = requests.post(url, data=formData, headers=headers)
  • 1
  • 2

Python

107,283 followers
About

This section is all about snakes! Just kidding.

Links
Moderators
Bucky Roberts Administrator