A simple Python Web Scraper (2)

Here is version 2 of the Python Web scraper to get my Wan IP address.

This one also used the BeautifulSoup library, but it does not use Selenium and the Chrome web driver.
I found that when Google updates the Chrome web driver, my scaper would fail.
I plan on rewriting it again using the Scrapy library too.

# import required modules
import bs4 as bs
import requests
import sys
from datetime import datetime

timestamp = datetime.now()
timestrng = timestamp.strftime("%m/%d/%Y %H:%M:%S") + " | "

# assign URL
URL = 'http://wanip.info/'

# http get request
url_link = requests.get(URL, headers={'User-Agent': 'Chrome'})
soup = bs.BeautifulSoup(url_link.text, "lxml")

results = []
ip= ""
i=0

# find all divs
for element in soup.findAll("div",class_="ipinfo"):
    for b in element.findAll("span"):
        results.append(b.text)
        i=i+1
        if (i==1): 
            ip = timestamp.strftime("%m/%d/%Y %H:%M:%S") + " | " + b.text
        else: 
            ip=ip + '.' + b.text    

file1 = open('C:/Users/Dan/^WanIPlog.txt', "a")  # append mode
file1.write(ip + '\n')
file1.close()

exit()
The results to my text file look like this.

WanIP check
04/10/2021 17:58:11 | 97.90.228.55 
04/11/2021 05:11:20 | 68.117.15.82 
04/18/2021 05:11:25 | 66.188.246.211 
04/25/2021 05:11:18 | 97.90.230.48	
05/02/2021 05:11:20 | 66.188.246.195 
05/09/2021 14:50:09 | 97.90.226.211 
05/16/2021 05:11:18 | 97.91.82.86  
05/23/2021 11:45:14 | 97.91.82.51  
01/17/2022 09:08:17 | 24.183.22.28

One Comment

  1. I was pretty pleased to discover this page. I wanted to thank you for your time for this particularly wonderful read!! I definitely enjoyed every little bit of it and i also have you book-marked to look at new information in your site.

Comments are closed.