sockets - Efficiently retrieve IP address and status code -


just practical question. need retrieve http status code of site ip address.

given fact need parse between 10k , 150k domains, wondering efficient method.

i've seen using urllib2.urlopen(site) attempts download entire file stream connected file. @ same time urllibs2 doesn't offer method convert hostname ip.

given i'm interested in head bit collect information http status code , ip address of specific server, best way operate?

should try use socket? thanks

i think there no 1 particular magic tool retrieve http status code of site , ip address.

for getting http status code should make head request using urllib2 or httplib or requests. here's example, taken how send head http request in python 2?:

>>> import urllib2 >>> class headrequest(urllib2.request): ...     def get_method(self): ...         return "head" ...  >>> response = urllib2.urlopen(headrequest("http://google.com/index.html")) 

an example, using requests:

>>> import requests >>> requests.head('http://google.com').status_code 301 

also, might want take @ grequests in order speed things getting status codes multiple pages.

grequests allows use requests gevent make asyncronous http requests easily.

for getting ip address, should use socket:

 socket.gethostbyname_ex('google.com') 

also see these threads:

hope helps.


Comments

Popular posts from this blog

Detect support for Shoutcast ICY MP3 without navigator.userAgent in Firefox? -

web - SVG not rendering properly in Firefox -

java - JavaFX 2 slider labelFormatter not being used -