A simple Python Web Scraper (1)

Learning a little Python code and web scraping principles, I wrote this little web scraper. Python is known for its plethora of code libraries and this code utilizes the BeautifulSoup library to understand the HTML code of a target page.

Besides the BeautifulSoup library, it uses the Selenium library web driver and Google’s Chromedriver.exe to retrieve the page. The Google Chromedriver was downloaded separately. I scheduled the program to run automatically each morning, the output written to a text file for later review.

#   Scraping MyWanIp off of site http://wanip.info/
#   To be run daily to see when WanIP changes
#   Results to C:\users\Dan\^WanIPlog.txt
#   see Things PYTHON, folder Scrape1
from bs4 import BeautifulSoup
from selenium import webdriver
from datetime import datetime
driver = webdriver.Chrome(executable_path='C:/Program Files (x86)/Google/Chrome/chromedriver_win32/chromedriver.exe')
driver.get('http://wanip.info/')
timestamp = datetime.now()
results = []
other_results = []
ip= ""
i=0
content = driver.page_source
soup = BeautifulSoup(content, 'html.parser')
for element in soup.findAll("div",class_="ipinfo"):
    for b in element.findAll("span"):
        results.append(b.text)
        i=i+1
        if (i==1): 
            ip = timestamp.strftime("%m/%d/%Y %H:%M:%S") + " | " + b.text
        else: 
            ip=ip + '.' + b.text    
file1 = open('C:/Users/Dan/^WanIPlog.txt', "a")  # append mode
file1.write(ip + '\n')
file1.close()
driver.quit()